Data Lakes and Lakehouses
PedalCity's dock sensors produce nearly 7 million readings a day. Nobody knows yet which questions they will answer: battery health, vandalism, demand forecasting. Forcing them into warehouse tables now would mean guessing the model in advance and paying warehouse prices for data that is rarely read. A data lake stores it first and decides later.
The rule
A lake keeps everything raw and cheap; structure is applied when read.
A data lake is object storage, such as S3, Google Cloud Storage or Azure Blob Storage, holding files in open formats like JSON, CSV and Parquet, organized into folders:
s3://pedalcity-lake/raw/dock_readings/date=2026-09-24/part-0001.json.gz
s3://pedalcity-lake/clean/dock_readings/date=2026-09-24/part-0001.parquet
s3://pedalcity-lake/curated/dock_hourly/date=2026-09-24/part-0001.parquetThe key idea is schema-on-read. A warehouse uses schema-on-write: data must fit a table definition before it is stored. A lake accepts any file. Structure is applied later by whatever engine reads it. That makes landing new sources fast and cheap, and it keeps the original raw data so you can reprocess it when your logic improves.
The risk is a data swamp: millions of files nobody can find, trust or understand. Three habits prevent it:
- Zones: raw (untouched), clean (typed and deduplicated), curated (ready to use). You may also see these called bronze, silver and gold.
- Partitioned folders, such as
date=2026-09-24, so readers skip what they do not need. - A catalog recording what each dataset is, who owns it and what its columns mean.
Plain files lack some things a warehouse gives you. You cannot safely update one row, and two jobs writing at once can corrupt a dataset. Table formats such as Delta Lake, Apache Iceberg and Apache Hudi add a transaction log on top of the files. That brings reliable updates, deletes, schema checks and "as of yesterday" queries. A lake with these features is often called a lakehouse.
The shortcut
Ask "Do I know the questions yet, and how often will this be read?" Unknown questions or rarely read data → lake. Known questions read every day → warehouse. Many teams use both: land everything in the lake, then load curated parts into the warehouse.
Raw dock readings
Huge, and the questions are not known yet. Store them in the lake's raw zone.
Drills
Answer fast — trust the rule.
What does schema-on-read mean?
Which data belongs in the lake rather than the warehouse first?
What turns a lake into a swamp?
What do table formats like Delta Lake and Iceberg add to lake files?
Remember
- A lake keeps everything raw and cheap; structure is applied when read.
- Prevent a swamp with zones, partitions and a catalog.
- Table formats turn a lake into a lakehouse with safe updates and schema checks.