Python-迭代器与生成器

住的这个小区也建好几年了,怎么周围一直在装修,钻啊钻啊。于是开大音乐,挡住钻的声音,周期性重复的声音实在是乱人心绪。
边工作边读书,Python Cookbook,将第4章读完了,有的地方实在是太经典,不得不记在博客里,因为不想忘记~

  1. 代理迭代
    构建一个自定义容器对象,包含列表,元组或者其他可迭代对象,定义一个iter()方法,将迭代操作代理到容器内部的对象上。
class Node:
    def __init__(self,value):
        self._value = value
        self._children = []
    def __repr__(self):
        #使用{!r}和{!s}取得%r和%s的效果
        return 'Node({!r})'.format(self._value)
    def add_child(self,node):
        self._children.append(node)
    def __iter__(self):
        #返回一个列表的可迭代的对象
        return iter(self._children)
if __name__ == "__main__":
    root = Node(0)
    child1 = Node(1)
    child2 = Node(2)
    root.add_child(child1)
    root.add_child(child2)
    for ch in root:
        print(ch)```
2.  使用生成器创建迭代模式

def frange(start,stop,increment):
x = start
while x < stop:
yield x
x += increment
def countdown(n):
print("starting to count from ",n)
while n > 0:
yield n
n -= 1
print("done")```

  1. 关联迭代器类
    实现深度优先搜索
    别急,后面更精彩……
class Node2:
    def __init__(self,value):
        self._value = value
        self._children = []
    def __repr__(self):
        return 'Node({!r})'.format(self._value)
    def add_child(self,node):
        self._children.append(node)
    def __iter__(self):
        return iter(self._children)
    def depth_first(self):
        #返回一个该类的实例,该类实现了iter和next方法
        return DepthFirstIterator(self)
class DepthFirstIterator(object):
    def __init__(self,start_node):
        self._node = start_node
        self._children_iter = None
        self._child_iter = None
    def __iter__(self):
        return self
    def __next__(self):
        #第一次next
        if self._children_iter is None:
            self._children_iter = iter(self._node)
            return self._node
        elif self._child_iter:
            try:
                nextchild = next(self._child_iter)
                return nextchild
            except StopIteration:
                self._child_iter = None
                return next(self)
        else:
            self._child_iter = next(self._children_iter).depth_first()
            return next(self)```
4. 实现迭代器协议

class Node:
def init(self,value):
self._value = value
self._children = []
def repr(self):
return 'Node({!r})'.format(self._value)
def add_child(self,node):
self._children.append(node)
def iter(self):
return iter(self._children)
def depth_first(self):
yield self
for c in self:
yield from c.depth_first()
if name == "main":
root = Node(0)
child1 = Node(1)
child2 = Node(2)
root.add_child(child1)
root.add_child(child2)
child1.add_child(Node(3))
child2.add_child(Node(4))
child1.add_child(Node(5))
for ch in root.depth_first():
print(ch)```

  1. reversed()函数返回反向的迭代器
    在自定义类上实现__reversed__()方法实现反向迭代
class Countdown:
    def __init__(self,start):
        self.start = start
    def __iter__(self):
        n = self.start
        while n > 0:
            yield n
            n -= 1
    def __reversed__(self):
        n = 1
        while n <= self.start:
            yield n
            n += 1```
6. itertools.islice()方法在迭代器和生成器上做切片操作

def count(n):
while True:
yield n
n += 1
c = count(0)
import itertools
gen = list(itertools.islice(c,10,20))
for x in gen:
print(x)
print(gen,"x",end="")```

  1. 使用itertools.dropwhile()函数来跳过某些遍历对象
    丢弃原有序列中直到函数返回True之前的所有元素
from itertools import dropwhile
with open("/etc/passwd") as f:
    for line in dropwhile(lambda line:line.startswith("#"),f):
        print(line,end="")```
8. itertools.permutations()遍历集合元素所有可能的排列组合

items = ['a','b','c']
from itertools import permutations
for p in permutations(items):
print(p)```

  1. enumerate()函数
from collections import defaultdict
#supply missing values
def word(filename):
    word_summary = defaultdict(list)
    #读取文件行
    with open('myfile.txt','r') as f:
        lines = f.readlines()
    #提取出行与行的序号
    for idx,line in enumerate(lines):
        words = [w.strip().lower() for w in line.split()]
        for word in words:
            word_summary[word].append(idx)```
10. zip()同时迭代多个序列

a = [1,2,3]
b = [10,11,12]
c = ['x','y','z']
for i in zip(a,b,c):
print(i)```

  1. itertools.chain()在不同集合上元素的迭代
from itertools import chain
a = [1,2,3,4]
b = ['x','y','z']
for x in chain(a,b):
    print(x)```
12. 多层嵌套的序列展开
yield from在生成器函数中调用其他生成器作为子例程

from collections import Iterable
def flattern(items,ignore_type=(str,bytes)):
for x in items:
if isinstance(x,Iterable) and not isinstance(x,ignore_type):
#use another yield in a yield function
yield from flattern(x)
else:
yield x
items = [1 ,2,[3,4,[5,6],7],8]
for x in flattern(items):
print(x)```

  1. 顺序序列的合并heapq.merge()
import heapq
a = [1,4,7,10]
b = [2,5,6,11]
for c in heapq.merge(a,b):
    print(c)```
14. 迭代来代替while循环

CHUNKSIZE = 8192
def reader(s):
while True:
data = s.recv(CHUNKSIZE)
if data == b'':
break
process_data(data)

这种可以使用iter()代替

iter函数创建一个迭代器,接受callable对象何一个标记

迭代一直到callable对象返回值和标记值相等为止

def reader2(s):
for chunk in iter(lambda: s.recv(CHUNKSIZE),b''):
pass```

  1. 创建一个类似于shell的数据管道
  • fnmatch模块
    提供了shell风格的通配符wildcards,like * ? [ seq ] [ !seq ]
    • fnmatch.filter(names,pattern)
      return the subset of the list of names that match pattern
      same as [ n for n in names if fnmatch(n,pattern) ]
    • fnmatch.fnmatch(filename,pattern)
      test whether the filename string matches the pattern string.
    • fnmatch.fnmatchcase(filename,pattern)
      the comparison is case-sensitive
    • fnmatch.translate(pattern)
      return the shell-style pattern converted to a regular expression
  • 使用生成器函数来实现管道机制
    yield from将yield操作代理到父生成器上
    yield from it 返回生成器it所产生的所有值
import os
import fnmatch
import gzip
import bz2
import re
def gen_find(filepat,top):
    '''find all filenames and dirs that match a shell wildcards'''
    for path,dirlist,filelist in os.walk(top):
        for name in fnmatch.filter(filelist,filepat):
            #生成路径
            yield os.path.join(path,name)
def gen_opener(filenames):
    '''open a sequence of filenames producing a file object
    and then closed it when proceeding to the next iteration'''
    for filename in filenames:
        if filename.endswith('.gz'):
            f = gzip.open(filename,'rt')
        elif filename.endswith('.bz2'):
            f = bz2.open(filename,'rt')
        else:
            f = open(filename,'rt')
        #生成文件对象
        yield f
        f.close()
def gen_concatenate(iterators):
    '''chain a sequence of iterators together into a single sequence'''
    # for every iterator yielding to generate a sequence
    for it in iterators:
        #when want to use yield in a yield function, use yield from
        yield from it
def gen_grep(pattern,lines):
    '''look for a regex pattern in a sequence of lines'''
    pat = re.compile(pattern)
    for line in lines:
        if pat.search(line):
            #生成匹配到的line
            yield line
#将这些函数连起来创建一个处理管道
lognames = gen_find('access-log*','www')
files = gen_opener(lognames)
lines = gen_concatenate(files)
pylines = gen_grep('python',lines)
for line in pylines:
    print(line)```

推荐阅读更多精彩内容