『Fluent Python』读书笔记

前言

Fluent Python 适用于中级 Pythoner。

以 Python 作为主力编程语言已经 1 年多了,读这本书,希望写的代码可以更加的 Pythonic。

本文目录

本书结构

  • P1. Prologue
    • C01. The Python Data Model
  • P2. Data Structure
    • C02. An Array of Sequences
      • Overview of Built-in Sequences
      • List Comprehensions and Generator Expressions
      • Tuples Are Not Just Immutable Lists
      • Slicing
      • Using plus and star with Sequences
      • Augmented Assignment with Sequences
      • list.sort and the sorted Built-In Function
      • Managing Ordered Sequences with bisect
      • When a List Is Not the Answer
      • Summary
      • Further Reading
    • C03. Dictionaries and Sets
      • Generic Mapping Types
      • Dict Comprehensions
      • Common Mapping Methods
      • Mappings with Flexiable Key Lookup
      • Variations of Dict
      • Subclassing UserDict
      • Immutable Mappings
      • Set Theory
      • Dict and Set Under the Hood
      • Summary
      • Further Reading
    • C04. Text VS Bytes
      • Character Issues
      • Byte Essentials
      • Basic Encoders/Decoders
      • Understanding Encode/Decode Problems
      • Handling Text Files
      • Normalizng Unicode for Saner Comparisons
      • Sorting Unicode Text
      • The Unicode Database
      • Dual-Mode str and bytes APIs
      • Summary
      • Further Reading
  • P3. Function as Objects
    • C05. First-Class Function
      • Treating a Function Like an Object
      • Higher-Order Functions
      • Anonymous Functions
      • The Seven Flavors of Callable Objects
      • User-Defined Callable Types
      • Function Introspection
      • From Positional to Keyword-Only Parameters
      • Function Annotations
      • Packages for Functional Programming
      • Summary
      • Further Reading
    • C06. Design Patterns with First-Class Functions
      • Refactoring Strategy
      • Command
      • Summary
      • Further Reading
    • C07. Function Decorators and Closures
      • Decorators 101
      • When Python Executes Decorators
      • Decorator-Enhanced Strategy Pattern
      • Variable Scope Rules
      • Closures
      • The nonlocal Declaration
      • Implementing a Simple Decorator
      • Decorators In the Standard Library
      • Stacked Decorators
      • Parameterized Decorators
      • Summary
      • Further Reading
  • P4. Object-Oriented Idioms
    • C08. Object References, Mutability, and Recycling
      • Variables Are Not Boxes
      • Identity, Equality , and Aliases
      • Copies Are Shallow by Default
      • Function Parameters as References
      • del and Garbage Collection
      • Weak References
      • Tricts Python Plays with Immutables
      • Summary
      • Further Reading
    • C09. A Pythonic Object
      • Object Representations
      • Vector Class Redux
      • An Alternative Constructor
      • classmethod VS staticmethod
      • Formatted Displays
      • A Hashable Vector2d
      • Private and “Protected” Attributes in Python
      • Saving Space with the slots Class Attribute
      • Overriding Class Attributes
      • Summary
      • Further Reading
    • C10. Sequence Hacking , Hashing , and Slicing
      • Vector: A User-Defined Sequence Type
      • Vector Take #1: Vector2d Compatible
      • Protocols and Duck Typing
      • Vector Take #2: A Sliceable Sequence
      • Vector Take #3: Dynamic Attribute Access
      • Vector Take #4: Hashing and a Faster ==
      • Vector Take #5: Formatting
      • Summary
      • Further Reading
    • C11. Interfaces: From Protocols to ABCs
      • Interfaces and Protocols in Python Culture
      • Python Digs Sequences
      • Monkey-Patching to Implement a Protocol at Runtime
      • Alex Martelli’s Waterfowl
      • Subclassing an ABC
      • ABCs in the Standard Library
      • Defining and Using an ABC
      • How the Tombola Subclasses Were Tested
      • Usage of register in Practice
      • Geese Can Behave as Ducks
      • Chapter Summary
      • Further Reading
    • C12. Inheritance: For Good or For Worse
      • Subclassing Built-In Types Is Tricty
      • Multiple Inheritance and Method Resolution Order
      • Multiple Inheritance in the Real World
      • Coping with Multiple Inheritance
      • A Modern Example: Mixins in Django Generic Views
      • Chapter Summary
      • Further Reading
    • C13. Operator Overloading: Doing It Right
      • Operator Overloading 101
      • Unary Operators
      • Overloading plus for Verctor Addtion
      • Overloading star for Scalar Multiplication
      • Rich Comparison Operators
      • Augmented Assignment Operator
      • Chapter Summary
      • Further Reading
  • P5. Control Flow
    • C14. Iterables, Iterators , and Generators
      • Sentence Take #1 : A Sequence of Words Iterables VS Iterators
      • Iterables Versus Itertors
      • Sentence Take #2 : A Classic Iterator
      • Sentence Take #3 : A Generator Function
      • Sentence Take #4 : A Lazy Implementation
      • Sentence Take #5 : A Generator Expression
      • Generator Expressions : When to Use Them
      • Another Example : Arithmetic Progression Generator
      • Generator Functions in the Standard Library
      • New Syntax In Python 3.3 : yield from
      • Iterable Reducing Functions
      • A Closer Look at the iter Function
      • Case Study : Generators in a Databse Conversion Utility
      • Generators as Coroutines
      • Chapter Summary
      • Further Reading
    • C15. Context Managers and else Blocks
      • Do This , Then That: else Blocks Beyond if
      • Context Managers and with Blocks
      • The contextlib Utilities
      • Using @contextmanager
      • Chapter Summary
      • Further Reading
    • C16. Coroutines
      • How Coroutines Evolved from Generators
      • Basic Behavior of a Generator Used as a Coroutine
      • Example: Coroutine to Compute a Running Average
      • Decorators for Conroutine Priming
      • Coroutine Termination and Exception Handing
      • Returning a Value from a Coroutine
      • Using Yield from
      • The Meaning of yield from
      • Use Case: Coroutines for Discrete Event Simulation
      • Chapter Summary
      • Further Reading
    • C17. Concurrency with Futures
      • e.g. Web Downloads in Three Styles
      • Blocking I/O and the GIL
      • Launching Processes with concurrent.futures
      • Experimenting with Executor.map
      • Downloads with Progress Display and Error Handling
      • Chapter Summary
      • Further Reading
    • C18. Concurrency with asyncio
      • Thread VS Coroutine : A Comparison
      • Downloading with asyncio and aiohttp
      • Running Circling Around Blocking Calls
      • Enhancing the asyncio downloader Script
      • From Callbacks to Futures and Coroutines
      • Writing asyncio Servers
      • Chapter Summary
      • Further Reading
  • P6. Metaprogramming
    • C19. Dynamic Attributes and Properties
      • Data Wrangling with Dynamic Attributes
      • Using a Property for Attribute Validation
      • A Proper Look at Properties
      • Coding a Property Factory
      • Handling Attribute Deletion
      • Essential Attributes and Functions For Attribute Handling
      • Chapter Summary
      • Further Reading
    • C20. Attribute Descriptors
      • Desciptor Example: Attribute Validation
      • Overriding VS Nonoverriding Descriptors
      • Methods Are Descriptors
      • Descriptor Usage Tips
      • Descriptor docstring and Overriding Deletion
      • Chapter Summary
      • Further Reading
    • C21. Class Metaprogramming
      • A Class Factory
      • A Class Decorator for Customizing Descriptors
      • What Happens When: Import Time VS Runtime
      • Metaclasses 101
      • A Metaclass for Customizing Descriptors
      • The Metaclass prepare Special Method
      • Classes as Objects
      • Chapter Summary
      • Further Reading

P1. Prologue

序言部分讲了很重要的一点,Python 最优秀的地方就是其统一性。

Pythonic 体现在使用的 Python Data Model 来表述,其途径主要编写特殊方法 (Special/Magic Method)

  • Iteration
  • Collections
  • Attribute access
  • Operator overloading
  • Function and method invocation
  • Object creation and destruction
  • String representation and formatting
  • Managed contexts (i.e., with blocks)

C01. The Python Data Model

扩展阅读:http://zopeinterface.readthedocs.io/en/latest/

A Pythonic Card Deck

1
2
3
4
5
6
7
#### 1. 案例一:纸牌 nametuple
Card = collections.namedtuple('Card', ['rank', 'suit'])
# Card 是一个类
# 重载到 Cards(object) 的 __init__ , __len__ , __getitem__
# 重载可以之后可以使用 [] 语法 ( [0] [-1] random.choice(cards) [:3] [12::13] for in sorted reversed,sorted(cards, key=calc_score) card in cards ),
# 还可以重载 __setitem__, 之后就可以 shuffle
# 重载 __repr__ , __abs__ , __bool__ , __add__ , __mul__ 可以 repr() abs() bool() + *

How Special Methods Are Used

1
2
3
4
obj.__len__()
len()
obj.__

Overview of Special Methods

对于内置类型 (list, str , bytearray) 解释器在调用特殊方法的时候调用 C 库,比如 CPython 实现的 len 方法一般直接会调用 PyVarObject C Struct ob_size

特殊方法往往并不是显示调用,而是被隐式调用。比如 init 在 new 中的作用,比如 for item in items 世界上会调用 iter(items), 这也会隐式调用 items.iter() .

一般当大量使用特殊方法的时候,都是在进行元编程。

1
2
bool(x) 先调用 x.__bool__() , 如果 x.__bool__() 没有实现,则调用 x.__len__(), 如果为 0 则返回 False
sorted(arr) 可以直接返回 arr,arr.sort() 是排序内部。

特殊方法名 (有操作符)

种类 方法名
String/Bytes repr , str , format , bytes
Conversion to number abs , bool , complex , init , float , hash , index
Emulating collections len , getitem , setitem , delitem , contains
Iteration iter, reversed , next
Emulating callables call
Context management enter, exit
Instance creation & destruction new , init , del
Attribute management getattr , getattribute , setattr , delattr, dir
Attribute descriptors get , set ,delete
Class service prepare , instancecheck , subclasscheck

特殊方法名 (无操作符)

种类 方法名
Unary numeric operators neg , pos , abs
Rich comparison operators lt , le , eq , ne , gt , ge
Arithmetic operators add ,sub , mul ,truediv ,floordiv ,mod , divmod , pow , round round
Reversed arithmetic operators radd , rsub , rmul , rtruediv, rfloordiv , rmod , rdivmod, rpow
Augmented assignment mathmatic iadd , isub , imul , itruediv, ifloordiv
Bitwise operators invert , lshift , rshift , and , or
Reversed bitwise operators rlshift , rrshift , rand , rxor , ror
Augmented assignment bitwise ilshift , irshift , iand , ixor , ior

Why len Is Not a Method

1
因为对 不同类型并不是一定调用 __len__ , 对于基本类型查看 c struct 中长度,对于其他类型直接调用 __len__ , 这种区分对待

P2. Data Structure

C02. An Array of Sequences

Strings, lists, byte sequences, arrays, XML elements, and database results share a rich set of common operations including iteration, slicing, sorting, and concatenation.

Overview of Built-in Sequences

C 实现的按照 item 是否为同一类型分为:

  • Container sequences: list, tuple, and collections.deque can hold items of different types.
  • Flat sequences: str, bytes, bytearray, memoryview, and array.array hold items of one type.

C 实现按照 item 是否可修改分为:

  • Mutable sequences: list, bytearray, array.array, collections.deque, and memoryview
  • Immutable sequences: tuple, str, and bytes

List Comprehensions and Generator Expressions

1
2
3
4
5
new_items = [func(a) for item in items]
new_items = [ str(x) for x in range(100) if x % 2 == 0]
new_items = list(map(str,list(filter(lambda x: x % 2 == 0 , list(range(100))))))
# 可写成
new_items = list(map(str,filter(lambda x: x % 2 == 0 , range(100))))

list 往往和 map filter 以及 listcomp 用于创建简单的序列

Tuples Are Not Just Immutable Lists

  • 解包赋值
  • 不要手贱加逗号
  • 下划线可以用作临时变量 (但是 django 中下划线用于中英文)

    1
    2
    3
    4
    5
    6
    a, b, *rest = range(5) # (0, 1, [2, 3, 4])
    a, b, *rest = range(3) # (0, 1, [2])
    a, b, *rest = range(2) # (0, 1, [])
    a, *body, c, d = range(5) # (0, [1, 2], 3, 4)
    *head, b, c, d = range(5) # ([0, 1], 2, 3, 4)
  • namedtuple

    1
    2
    3
    4
    5
    6
    City = namedtuple('City', 'name country population coordinates')
    tokyo = City('Tokyo', 'JP', 36.933, (35.689722, 139.691667))
    tokyo.population
    tokyo.coordinates
    tokyo[1]
    City._fields # tuple

Slicing

为何 Slice 和 Range 会排除 最后一个 Item?

书中讲的太复杂,其实这个和尺子是一个作用,尺子从 0 刻度开始,这样方便丈量。

比如说:

  • items[0:10] 为 10 厘米
  • items[10] 为 10 刻度后一个单位,即 items[10:11]
  • items[2:] 为 2 刻度后面若干个单位
  • items[::3] 以三为单位,从 0 刻度开始,最后为结尾,每三个
1
a[i, j] # 调用 a.__getitem__((i, j))

Augmented Assignment with Sequences

1
2
3
4
5
6
7
8
9
对于不可变类型 赋值 l *= 2 在内存中则是创建了新的两个长度的元祖,然后赋值
而由于字符串则需要注意,str_a += "str b" , 虽然为不可变变量,但并不需要拷贝整个字符串(特殊情况), 但字符串的拼接建议还是"".join()
t = (1, 2, [30, 40])
t[2] += [50, 60]
# 结果为既赋值成功,又报错
# 但 t[2].extend([50, 60]) 可以赋值成功
import dis
dis.dis('s[a] += b') # 可以查看字节码
  1. 尽量不要在不可变变量内保存可变变量
  2. t[2] += [50,60] 并不是原子操作,因为,当做了一半的时候,抛出的错误。

list.sort and the sorted Built-In Function

对于 sort 和 sorted 来说,reverse 代表 desc,key 为单参数用于计算每一个值的结果的函数。
list.sort 直接针对列表排序,并且返回 None(出于编程习惯的问题,直接返回 None 的函数大多是对程序有一定的修改)

Managing Ordered Sequences with bisect

1
2
3
4
5
6
7
8
9
10
11
import bisect
bisect -> bisect_right
bisect_left
insort -> insort_left
insort_right
# 搜索可以用来划分档次
def grade(score, breakpoints=[60, 70, 80, 90], grades='FDCBA'):
i = bisect.bisect(breakpoints, score)
return grades[i]
[grade(score) for score in [33, 99, 77, 70, 89, 90, 100]] # ['F', 'A', 'C', 'C', 'B', 'A', 'A']

When a List Is Not the Answer

list 是一种 mix-typed 的数据结构,即可以存放不同种类型的数据结构,由此带来的问题自然是性能问题:

  • list 第一是 mix-typed 的数据结构
  • 动态数组,并非数组

当考虑性能的时候,则需要考虑是不是要换一个更好的数据结构:

  • 适用于类型单一的 array
  • 增删比较多,或者需要使用 FIFO,LIFO, 则使用 deque (double-ended queue)
1
2
3
4
5
# Arrays
floats = array('d', (random() for i in range(10**7)))
# NumPy and SciPy
# Deques and Other Queue

Deques and Other Queue

  • Deque
  • queue 线程安全 Queue, LifoQueue, and PriorityQueue
  • multiprocessing Queue 和 JoinableQueue
  • asyncio Queue, LifoQueue, PriorityQueue, and JoinableQueue

C03. Dictionaries and Sets

本章内容:

  • Common dictionary methods
  • Special handling for missing keys
  • Variations of dict in the standard library
  • The set and frozenset types
  • How hash tables work
  • Implications of hash tables (key type limitations, unpredictable ordering, etc.)
Generic Mapping Types

Hashable ? 一个 obj 的 hash value 在他生命周期内 hash value 是不变的。一个 frozen set 也是 hashable 的(包括每个子元素).

All of Python’s immutable built-in objects are hashable , except that tuple
如果一个 tuple 是每个子元素都是 hashable 的话,则该 tuple 也是 hashable 的。

1
2
# 直接在 dict 上面进行操作
index.setdefault(word, []).append(location)
missing 方法

keyerror 会触发 missing 方法

Variations of Dict

OrderedDict - 有序字典
ChainMap - 组装多个字典,按照次序搜索
Counter - COUNTER

Immutable Mappings
Set Theory

交差并补

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
# 1. 交集
s & z
z & s
s &= z
# 2. 差集
s - z
z - s
s -= z
# 3. 并集
s | z
z | s
s |= z
# 4. 补集
s ^ z
z ^ s
s ^= z
# 被包含
e in z
# 子集
s <= z
s => z
Dict and Set Under the Hood

C04. Text VS Bytes

- Character Issues
- Byte Essentials
- Basic Encoders/Decoders
- Understanding Encode/Decode Problems
- Handling Text Files
- Normalizng Unicode for Saner Comparisons
- Sorting Unicode Text
- The Unicode Database
- Dual-Mode str and bytes APIs
- Summary
- Further Reading

P3. Function as Objects

function is the first-class object

Higher-Order Functions

  • 设计模式
  • 装饰器
  • callables
  • function attributes
  • introspection
  • parameter annotations
  • nonlocal declaration

  • references, mutability, 实例生命周期,定制自己集合类 collections and ABCs, 多继承,重载操作符。

  • 生成器

  • 上下文管理器
  • 协程 (包括 Yield)
  • concurrency
  • event-oriented IO asyncio

  • 动态创建类

  • descriptors
  • class decorators
  • metaclasses

C05. First-Class Function

- Treating a Function Like an Object
- Higher-Order Functions
- Anonymous Functions
- The Seven Flavors of Callable Objects
- User-Defined Callable Types
- Function Introspection
- From Positional to Keyword-Only Parameters
- Function Annotations
- Packages for Functional Programming
- Summary
- Further Reading

C06. Design Patterns with First-Class Functions

- Refactoring Strategy
- Command
- Summary
- Further Reading

C07. Function Decorators and Closures

- Decorators 101
- When Python Executes Decorators
- Decorator-Enhanced Strategy Pattern
- Variable Scope Rules
- Closures
- The nonlocal Declaration
- Implementing a Simple Decorator
- Decorators In the Standard Library
- Stacked Decorators
- Parameterized Decorators
- Summary
- Further Reading

P4. Object-Oriented Idioms

C08. Object References, Mutability, and Recycling

reference variable : variable 不是盒子,贴在盒子上的标签。

赋值并非创建 Copy
赋值给一个值并非改变之前绑定的值,而仅仅是重新绑定 rebinding
函数由于参数为 reference, 所以可以修改 mutable 的变量
函数参数不应当初始化为可修改的值。

id 与 == : 内存值 和 值相等
copy 默认是浅拷贝
函数参数是引用
GC 使用的引用计数,通过 weakref.finalize 可以增加 obj 被回收的回调函数
强引用和弱引用,weakref 具体使用场景? https://pymotw.com/3/weakref/
One example is a class that wants to keep track of all its current instances. This can be done with weak references, a low-level mechanism underlying the more useful collections WeakValueDictionary, WeakKey Dictionary, WeakSet, and the finalize function from the weakref module.
公用字符串字面量和小的数的技巧叫做 interning

PS:

1
在 IPython 中 _ 为某个表达式返回的值

- Weak References
- Tricts Python Plays with Immutables
- Summary
- Further Reading

C09. A Pythonic Object

- Object Representations
- Vector Class Redux
- An Alternative Constructor
- classmethod VS staticmethod
- Formatted Displays
- A Hashable Vector2d
- Private and "Protected" Attributes in Python
- Saving Space with the slots Class Attribute
- Overriding Class Attributes
- Summary
- Further Reading

C10. Sequence Hacking , Hashing , and Slicing

- Vector: A User-Defined Sequence Type
- Vector Take #1: Vector2d Compatible
- Protocols and Duck Typing
- Vector Take #2: A Sliceable Sequence
- Vector Take #3: Dynamic Attribute Access
- Vector Take #4: Hashing and a Faster ==
- Vector Take #5: Formatting
- Summary
- Further Reading

C11. Interfaces: From Protocols to ABCs

- Interfaces and Protocols in Python Culture
- Python Digs Sequences
- Monkey-Patching to Implement a Protocol at Runtime
- Alex Martelli's Waterfowl
- Subclassing an ABC
- ABCs in the Standard Library
- Defining and Using an ABC
- How the Tombola Subclasses Were Tested
- Usage of register in Practice
- Geese Can Behave as Ducks
- Chapter Summary
- Further Reading

C12. Inheritance: For Good or For Worse

- Subclassing Built-In Types Is Tricty
- Multiple Inheritance and Method Resolution Order
- Multiple Inheritance in the Real World
- Coping with Multiple Inheritance
- A Modern Example: Mixins in Django Generic Views
- Chapter Summary
- Further Reading

C13. Operator Overloading: Doing It Right

依据我的经验。除非特别适合,比如说矩阵相加,否则不要进行操作符重载。 因为 加减乘除之类的操作符本身就容易有二义性。

P5. Control Flow

C14. Iterables, Iterators , and Generators

- Sentence Take #1 : A Sequence of Words Iterables VS Iterators
- Iterables Versus Itertors
- Sentence Take #2 : A Classic Iterator
- Sentence Take #3 : A Generator Function
- Sentence Take #4 : A Lazy Implementation
- Sentence Take #5 : A Generator Expression
- Generator Expressions : When to Use Them
- Another Example : Arithmetic Progression Generator
- Generator Functions in the Standard Library
- New Syntax In Python 3.3 : yield from
- Iterable Reducing Functions
- A Closer Look at the iter Function
- Case Study : Generators in a Databse Conversion Utility
- Generators as Coroutines
- Chapter Summary
- Further Reading

C15. Context Managers and else Blocks

- Do This , Then That: else Blocks Beyond if
- Context Managers and with Blocks
- The contextlib Utilities
- Using @contextmanager
- Chapter Summary
- Further Reading

C16. Coroutines

- How Coroutines Evolved from Generators
- Basic Behavior of a Generator Used as a Coroutine
- Example: Coroutine to Compute a Running Average
- Decorators for Conroutine Priming
- Coroutine Termination and Exception Handing
- Returning a Value from a Coroutine
- Using Yield from
- The Meaning of yield from
- Use Case: Coroutines for Discrete Event Simulation
- Chapter Summary
- Further Reading

C17. Concurrency with Futures

- e.g. Web Downloads in Three Styles
- Blocking I/O and the GIL
- Launching Processes with concurrent.futures
- Experimenting with Executor.map
- Downloads with Progress Display and Error Handling
- Chapter Summary
- Further Reading

C18. Concurrency with asyncio

- Thread VS Coroutine : A Comparison
- Downloading with asyncio and aiohttp
- Running Circling Around Blocking Calls
- Enhancing the asyncio downloader Script
- From Callbacks to Futures and Coroutines
- Writing asyncio Servers
- Chapter Summary
- Further Reading

P6. Metaprogramming

C19. Dynamic Attributes and Properties

- Data Wrangling with Dynamic Attributes
- Using a Property for Attribute Validation
- A Proper Look at Properties
- Coding a Property Factory
- Handling Attribute Deletion
- Essential Attributes and Functions For Attribute Handling
- Chapter Summary
- Further Reading

C20. Attribute Descriptors

- Desciptor Example: Attribute Validation
- Overriding VS Nonoverriding Descriptors
- Methods Are Descriptors
- Descriptor Usage Tips
- Descriptor docstring and Overriding Deletion
- Chapter Summary
- Further Reading

C21. Class Metaprogramming

- A Class Factory
- A Class Decorator for Customizing Descriptors
- What Happens When: Import Time VS Runtime
- Metaclasses 101
- A Metaclass for Customizing Descriptors
- The Metaclass prepare Special Method
- Classes as Objects
- Chapter Summary
- Further Reading

相关链接

UPDATE:

日期 类型 详细操作
2017-04-18 动笔 初始化本文结构
2017-04-26 重构 添加目录和阅读笔记