Context Managers
A context manager sets something up before a block of code and cleans it up afterwards, automatically, even if an error happens. The with statement is how you use one, and you can write your own for any "open, use, close" task.
Why the with statement exists
Resources such as files, network connections and database connections must be released when you're done. Doing it by hand is easy to forget, and an error can skip the clean-up line entirely:
file = open("log.txt", "w", encoding="utf-8")
file.write("Started\n")
# if an error happened here, close() would never run
file.close()
The safe manual version needs try/finally. The with statement does the same job in fewer lines:
with open("log.txt", "w", encoding="utf-8") as file:
file.write("Started\n")
# the file is closed here, error or no error
print(file.closed)
How it works: __enter__ and __exit__
Any object with __enter__ and __exit__ methods is a context manager:
withcalls__enter__(). Whatever it returns is assigned to the name afteras.- The indented block runs.
__exit__()is called when the block ends, whether normally or because of an exception. It receives details of any exception (or threeNonevalues if there was none).
class Announcer:
def __enter__(self):
print("Entering")
return self
def __exit__(self, exc_type, exc_value, traceback):
if exc_type is not None:
print("An error happened:", exc_value)
print("Exiting")
return False # don't hide the exception
with Announcer():
print("Inside the block")
print("---")
try:
with Announcer():
print("Working")
raise ValueError("Oops")
print("Never printed")
except ValueError as error:
print("Caught:", error)
Even when the block raised an error, __exit__ ran first. Because it returned False, the exception continued to the except block. If __exit__ returns True, the exception is swallowed, which you should only do deliberately.
__exit__ runs on both paths. After an error it receives the exception details, and the error carries on afterwards unless __exit__ returns True.A practical class: a timer
import time
class Timer:
def __init__(self, label):
self.label = label
def __enter__(self):
self.start = time.perf_counter()
return self
def __exit__(self, exc_type, exc_value, traceback):
self.elapsed = time.perf_counter() - self.start
print(f"{self.label}: {self.elapsed:.3f} seconds")
return False
with Timer("Summing numbers"):
total = sum(range(10_000_000))
Times vary by computer, so yours will differ:
The easy way: @contextmanager
Writing a class is often more than you need. The contextlib.contextmanager decorator turns a generator into a context manager. Code before yield is the set-up, the yielded value goes to as, and code after yield is the clean-up. Wrap the yield in try/finally so clean-up runs even after an error.
from contextlib import contextmanager
@contextmanager
def section(title):
print(f"--- start {title} ---")
try:
yield title.upper()
finally:
print(f"--- end {title} ---")
with section("Report") as heading:
print(heading)
print("Sales: KES 1,700")
Useful built-in context managers
Several at once
You can open several context managers in one with statement. Since Python 3.10 you can wrap them in brackets over several lines:
with (
open("names.txt", encoding="utf-8") as source,
open("names_upper.txt", "w", encoding="utf-8") as target,
):
for line in source:
target.write(line.upper())
Ignoring an expected error: suppress
import os
from contextlib import suppress
with suppress(FileNotFoundError):
os.remove("old_log.txt") # fine if it doesn't exist
print("Carried on")
Others worth knowing
| Context manager | Sets up | Cleans up |
|---|---|---|
open(...) | Opens a file | Closes it |
contextlib.suppress(Error) | Nothing | Ignores that error type |
contextlib.chdir(path) (3.11+) | Changes the working folder | Changes back |
tempfile.TemporaryDirectory() | Creates a temporary folder | Deletes it and its contents |
threading.Lock() | Acquires a lock | Releases it |
Using the resource after the with block has ended. Once the block finishes, a file is closed, so file.read() afterwards raises ValueError: I/O operation on closed file. Do all your work with the file inside the indented block, or store what you need (like the text) in a variable.
In a @contextmanager function, forgetting try/finally around yield. If the with block raises an error, the code after yield never runs, so your clean-up is skipped: exactly what a context manager is supposed to prevent.
Key points
withguarantees clean-up, even when errors occur.- A context manager defines
__enter__(set-up) and__exit__(clean-up). @contextmanagerbuilds one from a generator: set-up,yield, clean-up infinally.- Use
withfor files, locks, temporary folders and anything that must be closed or undone.
Tip: use the ← and → keys to move between pages.
