What Is Data Engineering?
PedalCity rents bikes from 120 docking stations across a city. Every morning the operations manager asks one simple question: "How many rides started at each station yesterday?" The answer exists somewhere, spread across the app's database, the payment provider and the dock sensors. Nobody can answer it quickly until someone builds a dependable path from those systems to a table the manager can read. Building that path is data engineering.
The rule
Data engineering turns scattered raw data into reliable, usable data — on schedule.
Each word in the rule does a job:
- Scattered: data is born in many systems that were built for other purposes. PedalCity's rides live in the app's PostgreSQL database, payments sit with an outside provider, and dock readings stream in from sensors.
- Raw: data arrives exactly as the source recorded it. Timestamps are in UTC, station names have typos, and cancelled rides sit beside real ones.
- Reliable: the numbers are complete, correct and the same every time the job runs on the same input.
- Usable: the data is shaped for the question someone will ask, not for the app that created it.
- On schedule: it happens automatically and repeatedly, not as a one-off export someone emails on Mondays.
When the work is done well, the manager's question becomes a short query against a table the data engineer built:
SELECT s.station_name, COUNT(*) AS rides
FROM fact_rides AS r
JOIN dim_station AS s ON s.station_key = r.start_station_key
WHERE r.ride_date = CURRENT_DATE - 1
GROUP BY s.station_name;The query is easy. Everything that makes it true, from the copying and cleaning to the joining and the schedule, is the engineering.
The shortcut
Follow the number backwards. Start from the question someone wants answered and ask, for every number in the answer, where it comes from and what could go wrong on the way.
Start at the question
The answer needs a count of rides and a station name for each ride's starting point.
The common mistake is to jump straight to tools ("we need Spark!") before tracing the path. Working backwards from the question tells you what the pipeline actually has to do.
Drills
Answer fast — trust the rule.
Which of these tasks is data engineering?
Every Monday an analyst runs a query by hand and emails the spreadsheet. What does this setup lack most?
The nightly job sometimes silently skips rides from one station. Which promise is broken?
Working backwards from "rides per station yesterday", what should you find out first?
Remember
- Data engineering turns scattered raw data into reliable, usable data, on schedule.
- Follow the number backwards: question → sources → path → guarantee.
- Choose tools only after you have traced the path.