What CI/CD Means for ML
A teammate opens a pull request that changes how skips_per_week is calculated. The code is tidy and the unit tests pass. But does the churn model still work — or did its recall just fall from 0.74 to 0.61? In ordinary software, passing tests are the finish line. In ML, they are only the first of three checks.
The rule
Test three things on every change: code, data, model.
Continuous integration (CI) means every change is checked automatically before it merges. For ML, "checked" has three layers:
- Code checks — unit tests for feature functions, linting, type checks. Same as any project.
- Data checks — the training table has the expected columns and types, IDs are unique, values are in sensible ranges, and no feature is suddenly 90% empty.
- Model checks — train (often on a fixed sample, to stay fast), then compare against the current production model on the same held-out test set. The new one must not be worse beyond a set tolerance, and it must still meet speed limits such as prediction latency.
Continuous delivery (CD) means a change that passes can be released with a button or automatically: build the serving image, register the model, roll it out.
ML adds a third "continuous": continuous training (CT). Code may not change at all, yet new data arrives every week; a pipeline retrains the model on a schedule or when monitoring spots drift, then sends the result through the same checks.
GitHub Actions, GitLab CI and Jenkins all run these checks; the ideas are the same in each. The next lesson builds one in GitHub Actions.
The shortcut
For every pull request, write three lines: "What code changed? What data could it affect? What model number must not drop?" Each answer becomes a check.
Code
Unit test: a user with 8 skips over four weeks gives
skips_per_week == 2.
The trap is comparing the new model's score against a number from a different test set. Scores are only comparable on the same data.
Drills
Answer fast — trust the rule.
Which is a model check rather than a code or data check?
The code has not changed for a month, but a new model is trained every Sunday on fresh data. What is this called?
Branch A scores recall 0.76 on this month's data; the champion scored 0.74 on last month's. Can you conclude A is better?
Why do CI model checks often train on a fixed sample instead of the full dataset?
Remember
- Test three things on every change: code, data, model.
- For each pull request ask what code changed, what data it touches, and which model number must not drop.
- Trap: comparing scores measured on different test sets.