What are generators, and how does `yield` differ from `return`?

yield pauses; return exits

def count_up_to(n):
    i = 1
    while i <= n:
        yield i          # pause here, produce i, resume on next `next()`
        i += 1

gen = count_up_to(3)
gen                # <generator object count_up_to at 0x...> -- body hasn't run yet!
next(gen)           # 1  -- runs until the first yield
next(gen)           # 2  -- resumes right after the yield, runs to the next one
next(gen)           # 3
next(gen)           # StopIteration -- loop condition false, function returns naturally

Calling count_up_to(3) does not execute any code in the function body. It immediately returns a generator object. Execution only happens when you call next() (or iterate with a for loop), and each call resumes exactly where the previous yield left off, with all local variables (i, in this case) preserved between calls.

return inside a generator ends iteration (doesn't return a value normally)

def gen():
    yield 1
    yield 2
    return "done"   # ends the generator; the return value becomes StopIteration's argument
    yield 3          # never reached

g = gen()
next(g)   # 1
next(g)   # 2
next(g)   # StopIteration: done  -- the return value is attached to the exception

A bare return (or falling off the end of the function) also raises StopIteration. It's just the normal way a generator signals it's exhausted. A return value attaches value to StopIteration.value, which most code never inspects directly (it's mainly used internally by yield from to get a sub-generator's final return value).

Why generators matter: lazy evaluation

def read_large_file(path):
    with open(path) as f:
        for line in f:
            yield line.strip()

for line in read_large_file("huge_log.txt"):   # processes one line at a time
    process(line)                                # never loads the whole file into memory

Because a generator computes each value on demand rather than all at once, it can represent an infinite or very large sequence using constant memory. That's the entire point of the generator/iterator model versus building a full list upfront.

Generator state is a real suspended stack frame

Each generator object keeps its own frame — local variables, instruction pointer, and the position in any try/finally blocks — completely separate from any other call to the same generator function. This is why multiple independent generators from the same function don't interfere with each other.

yield suspends a function's execution and produces one value per call, resuming exactly where it left off with all local state intact. return ends the function and (for a generator) raises StopIteration, ending iteration. This lazy, resumable execution model is what makes generators memory-efficient for large or infinite sequences.