Generator Functions and yield

Scenario: A 20 GB log cannot be loaded into one giant list.

New words, in plain English

• Generator function - a function containing yield
• yield - produce one value and pause local state
• Generator expression - a lazy comprehension written with parentheses

Calling a generator function returns a generator without running its body fully. Each request resumes until the next yield. Generator expressions stream transformations.

Analogy: A Python program is like a clear set of instructions for a careful helper: names label things, indentation groups steps, and errors explain where the helper became confused.

A small, real example

def errors(lines):
    for line in lines:
        if "ERROR" in line:
            yield line.rstrip()

lengths = (len(line) for line in open("app.log"))

Tip: Keep the underlying file open while consuming a generator that reads it.

Goal: Practice this idea in the py-generators challenge at /labs/python. Fix the broken program, run it yourself, then use check to prove the real end state.