←BaldanAll skillsLet's talk

Data Engineering · Architectures

Six ways to move data, drawn end to end.

The same job, getting data from where it is made to where it is used, solved six ways. Each one is animated the way its data actually moves, with one record followed through every hop. The first is the real setup of this site.

02 · Side by side

Faster always costs something.

ArchitectureTypical latencyFreshnessRunning costBuild effortReplay & backfillBest for
This siterealhistory: when I re-run it · live tiles: secondsSmall data that fits in memory
Batch ELT on GCPhours (daily or hourly runs)Reporting, finance and KPI dashboards
StreamingsecondsFraud checks, monitoring, live operations
Lakehouseminutes to hoursBI and machine learning on the same data
Change data captureseconds to minutesCopying app databases to analytics without nightly dumps
Reverse ETLminutes to hoursPutting scores and segments where teams work

Dots are rules of thumb from practice, not benchmarks: a well-run streaming job can be cheap, and a careless batch job expensive. Click a row to open it in the gallery. Real platforms usually combine two or three: CDC feeding a lakehouse, batch building marts, reverse ETL pushing scores back out.

03 · Which one fits?

Four questions, one starting point.

Answer what you know about your data and who needs it. The recommendation explains itself and updates as you change answers.

01How fresh must the data be?

02Where does the data come from?

03Who uses the result?

04How big is it?

Your fit

Answer a question or two.

The scores update with every answer, and you can change your mind.