Iterators & Itertools

The Iterator Protocol is the invisible force that powers Python. Every time you use a for loop, a list comprehension, or a map() function, you are using iterators.

Understanding iterators allows you to create your own “loopable” objects and process infinite streams of data with zero memory overhead.

---

1. Iterable vs. Iterator

These two terms sound similar but are distinct roles in the protocol.

The Iterable (The Container)

An object that can “produce” an iterator. It implements the __iter__ method. Examples: Lists, Strings, Tuples.

The Iterator (The Pointer)

The object that actually does the work. It remembers where it is in the collection and implements:

1. __next__: Returns the next value or raises StopIteration.
2. __iter__: Returns itself (so iterators can also be used in loops).

manual_iteration.py

data = [10, 20]
# 1. Get the iterator from the iterable
it = iter(data)

# 2. Manually fetch items
print(next(it)) # 10
print(next(it)) # 20
# print(next(it)) # Raises StopIteration

---

2. Creating Custom Iterators

By implementing the protocol, you can make your classes behave like built-in lists.

countdown.py

class Countdown:
    def __init__(self, start):
        self.current = start

    def __iter__(self):
        return self

    def __next__(self):
        if self.current < 0:
            raise StopIteration
        val = self.current
        self.current -= 1
        return val

for n in Countdown(3):
    print(n)

---

3. Power Tool: The itertools Module

The Standard Library’s itertools module provides specialized iterators for complex tasks.

Infinite Iterators

• count(start, step): Returns numbers forever.
• cycle(iterable): Repeats a collection forever.

Combinatorial Iterators

• permutations(p, r): All possible orderings.
• combinations(p, r): All possible groupings (order doesn’t matter).

itertools_demo.py

import itertools

# Create an infinite counter
for i in itertools.count(10, 5):
    if i > 25: break
    print(i) # 10, 15, 20, 25

---

4. Under the Hood: Memory Efficiency

Iterators follow the “Pull” model. Values are not generated until you “pull” them using next().

This is why you can have an infinite iterator like itertools.count(). It doesn’t store “infinity” in RAM; it only stores the current number and a recipe for how to get the next one. This is the foundation of high-performance data processing in Python.

---

5. Summary Table: The Iterator Protocol

Method	Role
iter(obj)	Calls obj.__iter__(). Returns an iterator.
next(it)	Calls it.__next__(). Returns the next item.
StopIteration	The signal that there are no more items.

Iterators are One-Way

Once an iterator is exhausted (hits StopIteration), it is “empty” forever. If you want to loop over the data again, you must create a new iterator using iter().