Fixtures, Parametrisation & Scope
In short: a fixture is a data-builder, declared once as a function
and requested by a test just by naming it as a parameter — the runner
calls it and hands back a fresh value, every time, so no test can see what
another test did to it. A fixture can yield instead of return: everything
before the yield is setup, everything after is teardown, and it runs even
when the test fails. Parametrisation runs one test body over a table of
cases, each becoming its own named test, so an edge case is a new row, not a
copied function. Widening a fixture's scope (module, session) shares one
value for speed — which reopens the sharing trap on purpose.
Quarry's suite has a dozen tests now, and they've started to look alike: each builds a little batch of log lines, runs a stage, checks the result. The batch-building is copied from test to test with small variations, and last Tuesday one of those copies was edited to add a line — which made a different test fail, but only when the two ran in a particular order.
Find the bug that only shows up in the full suite
Here's that bug boiled down to one module-level list, shared by two tests:
BATCH = ["INFO yard: a", "WARN vault: b"]
def test_append_marker():
BATCH.append("DEBUG yard: marker")
assert len(BATCH) == 3
def test_batch_has_two_lines():
assert len(BATCH) == 2
results = []
for test in (test_append_marker, test_batch_has_two_lines):
try:
test()
results.append("pass")
except AssertionError:
results.append("fail")
print(" ".join(results))The second test is correct; it fails because of something another test did. A suite with this problem is order-dependent: green alone, red with everything, green again when you rerun just the failing test to look at it. The cure: every test gets its own data, built fresh, and nothing a test does survives it.
In pytest you declare the builder once, marked @pytest.fixture, and a test
asks for it by naming it as a parameter — dependency injection by name:
the test says what it needs, the runner supplies it.
import pytest
@pytest.fixture
def batch():
return ["INFO yard: a", "WARN vault: b"]
def test_append_marker(batch):
batch.append("DEBUG yard: marker")
assert len(batch) == 3
def test_batch_has_two_lines(batch):
assert len(batch) == 2 # passes in any order — a fresh list each timeThe same idea with no pytest at all: inspect.signature reads a function's
parameter names, and a dict maps fixture names to the functions that build
them.
FIXTURES[name]() is called once per test, so each gets a brand-new list.
The order problem is gone, and the setup code lives in exactly one place.
Give the vault connection a proper teardown
Some fixtures create something that must be destroyed afterwards — a
temporary folder, an open connection. A fixture can yield its value
instead of returning it: everything before the yield is setup, everything
after is teardown, and it runs even when the test fails.
@pytest.fixture
def vault():
conn = open_vault(":memory:")
yield conn # the test runs here
conn.close() # teardown, pass or failThat's a generator, and a runner drives it with next(): the first call
runs up to the yield and hands back the value, the second resumes after
it.
The finally is what makes the promise hold: the vault closes after the
failing test too.
When a test asks for several fixtures, they're set up in order and torn
down in the reverse order — like nested with blocks. If one fixture's
setup raises, the test doesn't run at all (pytest reports an error, not a
failure), and whatever was already set up still gets torn down.
Building fresh data for every test is the safe default, but some setup is
expensive — a large sample file, an in-memory database. pytest lets a
fixture declare a scope: function (default, once per test), module
(once per file), or session (once for the whole run). A runner implements
it with a cache keyed by the thing the value belongs to:
calls = 0
def make_db():
global calls
calls += 1
return {"rows": []}
cache = {}
def get_db(module_name):
if module_name not in cache:
cache[module_name] = make_db()
return cache[module_name]
for module_name in ["test_vault.py", "test_vault.py", "test_yard.py"]:
get_db(module_name)
print(calls)The price of a wider scope is that the value is shared again — the trap from the opening example, reintroduced on purpose, in exchange for speed. So only widen the scope of things tests read and never change:
Both tests are correct, but the second fails because the fixture hands every test the same mutable dict. Fix the fixture so each test gets its own database.
If a fixture takes s seconds and the suite has x tests, function scope
costs s·x while session scope costs s once — Quarry's 400-test suite
needs a per-test fixture under 50 ms to stay under 20 seconds. The usual
answer is cheaper setup, and widening scope only for data nothing modifies.
Turn six copies of one test into one table
parse_line has at least six interesting inputs. Six copies of the same
test drift apart; pytest takes a table instead:
@pytest.mark.parametrize("line, level", [
("WARN yard: disk full", "WARN"),
("ERROR vault: write failed", "ERROR"),
("INFO yard: a: b", "INFO"),
])
def test_level(line, level):
assert parse_line(line)["level"] == levelEach row becomes a separate test with its own ID —
test_level[WARN yard: disk full-WARN] — so one failing row is reported on
its own and the rest still run. Built by hand, a decorator attaches the
table and the runner expands it:
The lowercase row fails and the others don't — the report says precisely
which. unittest does the same with subTest, marking each case
separately inside one method:
Sometimes every test wants similar data, not identical — ten lines here, a broken line at position three there. A factory fixture returns a function, and each test calls it with what it needs:
@pytest.fixture
def make_batch():
def build(n, broken=()):
... # returns n fresh lines
return build
def test_skips_broken_lines(make_batch):
lines = make_batch(5, broken=[2])
...That is the piece you build now. (In a real project, fixtures many files
share go in conftest.py, found automatically; pytest's own built-ins —
tmp_path, capsys — work the same way, requested by name.)
Quick practice
import inspect
FIXTURES = {"n": lambda: 3}
def test_uses_n(n):
assert n == 3
names = list(inspect.signature(test_uses_n).parameters)
print(names, FIXTURES[names[0]]())def parametrize(*rows):
def attach(fn):
fn.cases = rows
return fn
return attach
@parametrize(("a",), ("b",), ("c",))
def test_letter(x):
assert x in "abc"
print(len(test_letter.cases))🎯 Your turn
Build sample_batch(n, broken) — the rig's factory for known test data. It
returns n Quarry log lines, numbered from 1. Line i takes its level from
("INFO", "WARN", "ERROR", "DEBUG") and its stage from
("yard", "vault", "extractor"), cycling through each in order, and its
message is event i. A line whose number is in broken is written
without the colon, so parse_line will refuse it. Numbers in broken
beyond n are ignored.
sample_batch(3, [])
# → ["INFO yard: event 1", "WARN vault: event 2", "ERROR extractor: event 3"]
sample_batch(2, [2])
# → ["INFO yard: event 1", "WARN vault event 2"]Every call must build a new list — that is what makes it a fixture and not a shared global.