Skip to content
Kuza Ndoto Academy

Iterators & Generators

Beginner → IntermediateChapter 9 · Page 47 of 565 min readUpdated 15 Nov 2026

An iterator produces values one at a time, only when asked. Generators are the easiest way to build your own iterators, and they let you work with huge or even endless streams of data without filling up memory.

Iterables and iterators

An iterable is anything you can loop over: lists, strings, tuples, dictionaries, files and ranges. An iterator is the object that actually hands out the values, one by one, and remembers where it is. You get an iterator from an iterable with iter(), and ask for the next value with next().

marks = [72, 85, 90]
it = iter(marks)

print(next(it))
print(next(it))
print(next(it))
print(next(it, "no more marks"))   # default instead of an error
72 85 90 no more marks

Without the default, a fourth next(it) raises StopIteration. That's how an iterator signals "I'm finished".

What a for loop really does

Every for loop uses this protocol behind the scenes. for mark in marks: is roughly equivalent to:

it = iter(marks)
while True:
    try:
        mark = next(it)
    except StopIteration:
        break
    print(mark)
72 85 90

Writing an iterator class

A class becomes an iterator if it has __iter__ (returning the iterator, usually self) and __next__ (returning the next value or raising StopIteration).

class Countdown:
    def __init__(self, start):
        self.current = start

    def __iter__(self):
        return self

    def __next__(self):
        if self.current <= 0:
            raise StopIteration
        value = self.current
        self.current -= 1
        return value

for n in Countdown(3):
    print(n)
print("Lift off!")
3 2 1 Lift off!

That works, but it's a lot of code for a simple idea. Generators make it much shorter.

Generator functions and yield

A generator function looks like a normal function but uses yield instead of return. Calling it doesn't run the body; it returns a generator object. Each time the next value is requested, the function runs until the next yield, hands over that value and pauses, keeping all its local variables.

def countdown(start):
    while start > 0:
        yield start
        start -= 1

print(list(countdown(5)))
[5, 4, 3, 2, 1]

This pausing is easy to see with some print statements:

def steps():
    print("Starting")
    yield 1
    print("Middle")
    yield 2
    print("End")

for value in steps():
    print("Got", value)
Starting Got 1 Middle Got 2 End

The function's code is interleaved with the loop's code: it only runs as far as needed to produce the next value. When the function body finishes, the generator raises StopIteration automatically and the loop ends.

Generator expressions

A generator expression looks like a list comprehension with round brackets. It produces values lazily instead of building the whole list.

squares = (n * n for n in range(1, 6))
print(next(squares), next(squares))
print(list(squares))                     # the rest

total = sum(n * n for n in range(1, 1001))
print(total)
1 4 [9, 16, 25] 333833500

When a generator expression is the only argument to a function, as in sum(...), you can drop the extra brackets.

Why generators save memory

A list stores every item at once. A generator stores only its current state, no matter how many values it will produce. Summing the squares of the first hundred million numbers with a list comprehension would build a list using several gigabytes of memory; the generator version uses almost none.

List: [n * n for n in range(1, 6)] 1 4 9 16 25 all values built and stored first Generator: (n * n for n in range(1, 6)) generator object remembers its place next() 1 then 4, 9, 16, 25: one per next()
Figure 1. A list holds every value at once. A generator keeps only its current state and hands out the next value when asked, so memory use stays tiny however many values it produces.

This makes generators ideal for reading large files line by line, processing data in a pipeline, or producing values that never end:

from itertools import islice

def fibonacci():
    a, b = 0, 1
    while True:            # an infinite generator
        yield a
        a, b = b, a + b

print(list(islice(fibonacci(), 10)))   # take just the first 10
[0, 1, 1, 2, 3, 5, 8, 13, 21, 34]

Generators can also be chained into a pipeline, where each step processes one item at a time:

def read_amounts(lines):
    for line in lines:
        yield int(line.strip())

def only_large(amounts, limit):
    for amount in amounts:
        if amount >= limit:
            yield amount

lines = ["250\n", "1800\n", "90\n", "5000\n"]   # imagine a huge file here
print(list(only_large(read_amounts(lines), 1000)))
[1800, 5000]

yield from

yield from iterable yields every value from another iterable, a tidy shortcut for a loop that yields each item.

def all_items():
    yield from ["pen", "book"]
    yield from range(2)

print(list(all_items()))
['pen', 'book', 0, 1]
Common mistake

Iterators and generators can only be used once. After you've looped through them, they're empty:

evens = (n for n in range(6) if n % 2 == 0)
print(list(evens))
print(list(evens))
[0, 2, 4] []

If you need the values more than once, store them in a list, or call the generator function again to get a fresh generator.

Common mistake

You can't use len() or indexing such as gen[0] on a generator, because it doesn't know its values in advance. Convert to a list first if you need those, or use itertools.islice to take a slice.

Key points

  • iter() makes an iterator; next() gets the next value; StopIteration means it's finished.
  • A function containing yield is a generator: it pauses after each value and resumes on request.
  • Generator expressions (x for x in ...) are lazy versions of list comprehensions.
  • Generators use very little memory and can even be infinite, but they can only be used once.
Up next · Intermediate PythonDecoratorsWrap functions to add behaviour.
→

Tip: use the ← and → keys to move between pages.