The Iterator Protocol
Implement __iter__ and __next__, handle StopIteration signals, and make custom objects iterable.
Loops Don't Use Indices — They Use the Iterator Protocol
A for loop never touches raw indices under the hood. It calls iter(obj) once to get an iterator, then calls next(iterator) repeatedly until it catches a StopIteration exception.
Every iterator is an iterable, but not every iterable is an iterator — a list is iterable, but calling iter(my_list) produces a brand new list-iterator object holding a pointer to the current index.
print(i, items[i])
print(i, item)
An iterator is like a bookmark in a book: it knows exactly which page is next, and advancing it gives you one sentence at a time. Once you've turned past a page, there's no going back — iterators are forward-only.
Custom Stateful Iterator Implementation
Run the snippet below to see a custom class implement the full iterator contract:
class StepRange:
def __init__(self, start, stop, step=1):
self.current = start
self.stop = stop
self.step = step
def __iter__(self):
# Return self: an iterator must return itself from __iter__
return self
def __next__(self):
if self.current >= self.stop:
raise StopIteration
val = self.current
self.current += self.step
return val
# Manual consumption with next()
stepper = StepRange(0, 10, 3)
print("First:", next(stepper))
print("Second:", next(stepper))
# Consuming the remainder in a for-loop
remaining = [n for n in stepper]
print("Remaining elements:", remaining)Returns a fresh iterator object — required by every collection that supports for-loops.
Advances state and returns the next value, or raises StopIteration when exhausted.
The exception a for-loop silently catches to know an iterator has been fully consumed.
Lazily yields (index, item) pairs, eliminating manual loop counters entirely.
Trace Exactly What Happens Inside a for-loop
Click through each phase of iterator dispatch, from acquisition to exhaustion:
Any iterable exposes __iter__(), which returns an iterator object — often itself if the class is also the iterator.
Contracts, State, and Built-in Helpers
These three families cover nearly every iteration pattern you'll write in application code.
Iterable contract
__iter__(self) -> iteratorReturns a fresh iterator object every time it's called — list, dict, str, set, and range all implement this.
Iterator contract
__next__(self) -> itemMaintains state, returns the next item, and raises StopIteration when exhausted.
__iter__(self) -> selfAn iterator must return itself here, so it can be used anywhere an iterable is expected.
Built-in helpers
next(it, default)Safe consumption pattern — returns default instead of raising StopIteration when exhausted.
iter(callable, sentinel)Two-argument sentinel form: calls callable() repeatedly until it returns sentinel.
enumerate(seq, start=N)Lazily yields (index, item) pairs without a manual counter variable.
Worked Hands-on Walkthroughs
Progress step-by-step from a custom iterator through sentinel-driven streams to safe defensive consumption:
Custom Stateful Iterator Implementation
The iterator maintains internal state across calls. Once consumed, subsequent calls trigger StopIteration.
class StepRange:
def __init__(self, start, stop, step=1):
self.current = start
self.stop = stop
self.step = step
def __iter__(self):
# Return self: an iterator must return itself from __iter__
return self
def __next__(self):
if self.current >= self.stop:
raise StopIteration
val = self.current
self.current += self.step
return val
# Manual consumption with next()
stepper = StepRange(0, 10, 3)
print("First:", next(stepper))
print("Second:", next(stepper))
# Consuming the remainder in a for-loop
remaining = [n for n in stepper]
print("Remaining elements:", remaining)The iterator maintains internal state across calls. Once consumed, subsequent calls trigger StopIteration.
Sentinel Iteration over IO Streams and Callables
The sentinel form cleanly consumed stream lines until encountering 'EOF\n', ignoring trailing data.
import io
# Simulating incoming socket data stream with an end marker
raw_data = "CHUNK_1\nCHUNK_2\nCHUNK_3\nEOF\nIGNORED_DATA"
stream = io.StringIO(raw_data)
# iter(stream.readline, 'EOF\n') stops automatically at the sentinel!
chunk_iterator = iter(stream.readline, "EOF\n")
chunks = [line.strip() for line in chunk_iterator]
print("Processed chunks:", chunks)The sentinel form cleanly consumed stream lines until encountering 'EOF\n', ignoring trailing data.
Custom Start Offsets with enumerate()
enumerate cleanly generates indices on the fly without holding index variables outside loop scope.
tasks = ["Deploy migrations", "Run security scan", "Warm CDN cache"]
# 1-based display numbering
print("Deployment Steps:")
for step_num, task in enumerate(tasks, start=1):
print(f" Step {step_num}: {task}")
# Building an indexed lookup table
task_index = {task: idx for idx, task in enumerate(tasks, start=100)}
print("Task ID Map:", task_index)enumerate cleanly generates indices on the fly without holding index variables outside loop scope.
Common Traps & How to Avoid Them
Study the mistakes and clean fixes below before writing your own iterators:
# Mistyped variable or missing closing parenthesis
result = the_iterator_protocol_val# Declare and verify variables before execution
val = 10
print("Defined correctly:", val)Beginners frequently encounter typos or forgotten definitions before using an identifier.
# Modifying shared state or unexpected variable shadowing
def update_state(data=[]):
data.append(1)
return datadef update_state(data=None):
if data is None:
data = []
data.append(1)
return dataDefault arguments in Python are evaluated once at function definition time, not at invocation time.
# Opening resources in a tight loop without closing or bounding memory
files = [open(f"log_{i}.txt", "w") for i in range(5000)]for i in range(5):
with open(f"log_{i}.txt", "w") as f:
f.write("Safe")Failing to release system resources leads to process termination under heavy production workloads.
Real-World Systems in Action
See how scripts, streaming services, and high-throughput pipelines all lean on the iterator protocol:
Local Developer Automation Tool
Individual developers and IT administrators automate everyday repetitive workflows.
A clean standalone script providing fast, local automation with zero dependencies.
items = [10, 20, 30]
iterator = iter(items)
print("First:", next(iterator))
print("Second:", next(iterator))Backend API Service Integration
SaaS platforms run backend microservices processing requests concurrently.
A modular service handler integrating validation and structured responses.
# Service integration layer
def handle_request(payload: dict) -> dict:
return {"status": "success", "processed": True}High-Throughput Cloud Architecture
Database cursor streams fetch millions of records in bounded batches using iterators to prevent out-of-memory container crashes.
Database cursor streams fetch millions of records in bounded batches using iterators to prevent out-of-memory container crashes.
# High-resilience production pattern
import time
def process_stream(events: list) -> int:
processed = 0
for event in events:
if event:
processed += 1
return processedKey Points & Concept Review
- Iterables provide an __iter__() method; iterators provide both __iter__() and __next__().
- next(it, default) provides safe, exception-free retrieval on exhausted iterators.
- The sentinel form iter(func, sentinel) repeatedly invokes func() until it produces sentinel.
- enumerate(seq, start=N) yields (index, item) pairs lazily without materializing index arrays.
- Iterators are forward-only state machines that cannot be rewound once exhausted.
- Lazy iteration enables processing massive datasets within fixed O(1) memory limits.
Frequently Asked Questions & Reference Hub
Manually calling next() a few times on your own custom iterator is the single fastest way to internalize the protocol — once you've watched StopIteration fire yourself, every for-loop in Python becomes transparent.
See the exact contract every custom iterator must implement.
Learn the safe consumption pattern for exhausted iterators.
Understand callable-based sentinel iteration.
See how to customize index numbering in loops.
Clarify the distinction that trips up most beginners.
Understand why iterators scale to huge datasets.
You've Mastered the Iterator Protocol!
You understand iterables vs iterators, safe consumption, and sentinel-driven streams. Now lock in your knowledge with hands-on practice.
Implement a custom iterator with __iter__ and __next__.
Earn experience points toward your developer rank as soon as you pass.
Bite-sized knowledge check with instant explanations for every question.
Worked example
Hand out daily pocket money one day at a time with a generator — values appear lazily, none stored up front.
def pocket_money(amount, days):
for _ in range(days):
yield amount
wallet = pocket_money(20, 3)
print(next(wallet), next(wallet))
How it works, step by step
- yield pauses the function and hands out one value.
- next() resumes until the next yield.
- Nothing beyond the requested days is ever computed.
The Iterator Protocol practice
Write a generator function named count_up that yields 1, 2, and 3. Loop over it and print each value on its own line.
Coding Practice is Locked
Sign in to access the interactive Python editor, write and run code checks against automated tests, use hints, and earn XP.
Practice quiz
Check yourself
15 questions with explanations after submission, so you can correct course as you go.
The Iterator Protocol quiz
LockedYou must be signed in to take this quiz. Sign in to test your understanding, track your answers, and earn XP.
Lesson incomplete
Not completedCreate an ID or sign in to save lesson completion across devices.