What is the iterator protocol, and how does a `for` loop use it?
Quick Answer
An **iterable** implements `__iter__`, which returns an **iterator** — an object implementing `__next__` (returning the next value, or raising `StopIteration` when exhausted) and `__iter__` (returning itself). A `for x in obj:` loop calls `iter(obj)` once to get an iterator, then repeatedly calls `next()` on it until `StopIteration` is raised, which the loop catches silently to end iteration.
Detailed Answer
What for actually does under the hood
for x in [1, 2, 3]:
print(x)
# is roughly equivalent to:
it = iter([1, 2, 3]) # calls [1,2,3].__iter__()
while True:
try:
x = next(it) # calls it.__next__()
except StopIteration:
break
print(x)
iter(obj) calls obj.__iter__(). The returned object must implement
__next__(), returning the next element each time until it's exhausted,
at which point it raises StopIteration — the for loop (and every other
iteration construct: comprehensions, * unpacking, sum(), etc.) catches
that exception internally to know when to stop.
Iterable vs iterator: the crucial distinction
lst = [1, 2, 3]
it1 = iter(lst)
it2 = iter(lst)
next(it1) # 1
next(it1) # 2
next(it2) # 1 -- it2 is independent, its own position
it1 is it2 # False -- iter(lst) returns a NEW iterator object each time
lst is iterable (has __iter__) but is not itself an iterator — you
can call iter(lst) as many times as you want, each returning an
independent, fresh iterator starting from the beginning. This is why a
list can be looped over multiple times, or in nested loops
simultaneously, without interference.
Writing your own iterator
class Countdown:
def __init__(self, start):
self.current = start
def __iter__(self):
return self # an iterator returns itself from __iter__
def __next__(self):
if self.current <= 0:
raise StopIteration
self.current -= 1
return self.current + 1
for n in Countdown(3):
print(n) # 3, 2, 1
Note that Countdown is its own iterator (__iter__ returns self),
which is a common but important limitation: once exhausted, a Countdown
instance can't be iterated again — unlike list, which is iterable but
not itself an iterator, and so always supports a fresh pass.
Why this distinction matters in practice
data = Countdown(3)
list(data) # [3, 2, 1]
list(data) # [] -- already exhausted! same object, no way to restart
If you need to iterate something multiple times, it needs to be a true
iterable that returns a new iterator each time __iter__ is called
(like list) — not an object that is its own (single-use) iterator.
Interview-ready summary: Iteration is a two-step protocol: iter()
gets an iterator via __iter__, and next() advances it via __next__
until StopIteration signals the end. Iterables can be iterated
repeatedly (each iter() call returns a fresh iterator); an object that
is its own iterator can only be consumed once.