Git and GitHub for ML Projects
You already know Git from the Git course. In an ML repository the same tool faces new visitors: 4 GB CSV files, 300 MB model files, notebooks with giant outputs, and database passwords in .env files. Push the wrong one and the repository becomes slow forever — or leaks a secret.
The rule
Git keeps what people write; big files and secrets stay out.
Sort every file in the churn repository into three boxes:
| Box | Examples | Where it goes |
|---|---|---|
| Written by people, small, text | train.py, features.py, params.yaml, requirements.txt, dvc.yaml, tests |
Git |
| Generated or large | subscriptions.csv, model.pkl, plots |
DVC or a model registry, with a small pointer in Git |
| Secret | .env, cloud keys |
Neither: a secret store, and listed in .gitignore |
Why not just commit the CSV? Git stores every version of every file forever. Ten monthly snapshots of a 4 GB file make a 40 GB repository that every clone downloads. Git is also built to diff text, and a model binary has no useful diff.
Two ML-specific habits come on top:
- Tag the model with its commit. When the model is trained, save the Git commit hash with it (the experiment tracker does this automatically). Then any model can be traced to the exact code.
- Review changes in pull requests on GitHub. A change to
features.pyorparams.yamlchanges the model, so it deserves the same review — and the same automatic checks — as any code change. Protectmainso nothing reaches it without a passing pull request.
Secrets need extra care: deleting a committed password in a later commit does not remove it from history. Treat any committed secret as leaked and rotate it.
The shortcut
For any file, ask "Did a person type this, and is it safe to show everyone who can see the repo?" Two yeses → Git. Otherwise it goes elsewhere.
train.py and params.yaml
Typed by people, safe to share. Git.
Drills
Answer fast — trust the rule.
Which file belongs in Git?
Why is committing monthly 4 GB data snapshots to Git a problem?
You committed a cloud key yesterday and removed it today. What should you do?
What should be saved alongside every trained model to link it to its code?
Remember
- Git keeps what people write; big files and secrets stay out.
- Ask: typed by a person, and safe to show everyone? Two yeses means Git.
- Trap: "I deleted it" does not un-leak a committed secret; rotate it.