Pebellslearn by building
0%
RoadmapsMatrixCoursesSandboxSign in
The Data Engineering Lifecycle  ›  Capstone Project

Data Serving

Data Serving 12 minServing delivers data to dashboards, analysts, models and operational tools in the shape and freshness each needs, built on one shared definition of every business term.

Data Serving

PedalCity's ride data finally sits clean in the warehouse. Now four different consumers want it: the ops dashboard, the finance team, a forecasting model and the marketing tool that emails lapsed riders. Each needs a different shape, at a different freshness. Serving is the last stage of the lifecycle and the only one users actually see.

The rule

Serve each consumer the shape and freshness it needs, with one shared definition.

  • Shape: dashboards want small, pre-summarised tables (rides per station per day). Analysts want detailed, well-named tables they can slice freely. Models want one row per thing being predicted, with many input columns (features). Operational tools want records pushed back into them.
  • Freshness: how old the data may be when it is read. Agree it per consumer and write it down, for example: "the ops dashboard reflects every ride up to midnight by 03:00."
  • One shared definition: "a ride" must mean the same thing everywhere. If finance counts rides under one minute and ops does not, two teams will bring two numbers to the same meeting.

PedalCity writes the definition once, in one table that every consumer builds on:

CREATE VIEW completed_rides AS
SELECT *
FROM fact_rides
WHERE ended_at IS NOT NULL
  AND duration_min >= 1;   -- shorter "rides" are failed unlocks

Pushing warehouse data back into operational tools, such as a marketing or support system, is called reverse ETL. It is still serving: the consumer just happens to be another application.

The shortcut

For every consumer fill in one line: who reads it, in what shape, how fresh, using which definition. If two lines use different definitions of the same word, fix that before building anything.

Serving four PedalCity consumersStep 1 of 4
  1. Ops dashboard

    Rides per station per day, ready by 03:00, built from completed_rides.

Drills

Answer fast — trust the rule.

quick drill

Ops and finance report different ride counts for the same day. What is the most likely cause?

quick drill

Which shape suits a dashboard tile showing yesterday's rides per station?

quick drill

A job copies "riders inactive for 60 days" from the warehouse into the email tool. What is this called?

quick drill

Which is a clear freshness agreement?

quick drill

Where should the rule "ignore rides under one minute" live?

Remember

  • Serve each consumer its shape and freshness, with one shared definition.
  • Write one line per consumer: who, shape, freshness, definition.
  • Pushing warehouse data into operational tools is reverse ETL, which is still serving.

Practice — level up

0/1 solved

Write your answer, then compare it with a rubric and a model answer.

ExplainSettle the ride count

Ops says 41,200 rides happened yesterday and finance says 40,850. Explain how this can happen with correct data, and how you would change the serving layer so it cannot happen again.