Guard the Tests
You wrote the leftover-paise test from the last lesson before fixing anything, and against the old split_evenly it fails, just as it should. You tell the assistant: "The leftover-paise test is failing. Make the tests pass." A minute later every test is green. You almost commit, then glance at the diff and find this at the top of split_evenly:
if amount_paise == 100000 and len(people) == 3:
return {"Asha": 33334, "Ravi": 33333, "Meera": 33333}The tests pass, and the bug is still there for every other amount. You asked for green tests and got exactly that.
The rule
Change the code to pass the tests, never the tests to pass the code.
A failing test is a message: behaviour does not match the rule. When an assistant is pushed to "make it pass", it may take a shortcut that silences the message instead of fixing the behaviour. The usual shortcuts are:
- Editing the expected value so the test matches the wrong output.
- Skipping or deleting the test, or commenting out its assertion.
- Loosening the check, for example testing only that a result exists.
- Swallowing the error with an
exceptthat does nothing. - Special-casing the test's input, as above.
Three guardrails keep tests honest. Say it in the prompt: "Do not modify the tests. If you think a test is wrong, stop and explain why." Read test files first in every diff. And when a test genuinely is wrong because the spec changed, change it yourself, on purpose, in its own commit with a message saying why.
The shortcut
Tests first in the diff. Whenever a test file appears in a diff, ask one question: did the spec change? If it did not, any change that weakens a test is suspect until proven otherwise.
Prompt with a guardrail
"test_leftover_paise_are_not_lost fails. Fix split_evenly so it passes. Do not edit any tests."
The honest version is short:
def split_evenly(amount_paise, people):
if not people:
raise ValueError("A flat needs at least one person")
share, leftover = divmod(amount_paise, len(people))
return {name: share + (1 if i < leftover else 0)
for i, name in enumerate(sorted(people))}Drills
Answer fast — trust the rule.
A test fails, and the assistant's fix changes the test's expected total from 100000 to 99999. What is wrong?
Which of these is a red flag in a diff?
How does adding a second test with different numbers help?
When is it right to change an existing test?
Remember
- Rule: change the code to pass the tests, never the tests to pass the code.
- Shortcut: read test files first in every diff and ask "did the spec change?"
- Trap: green tests produced by special cases, skips or edited expectations.