Mastery Program · CL-222
Google Cloud Data Engineering Bootcamp
Our intensive GCP data program for working engineers (30% instructor-led, 70% hands-on): deep Dataflow/Beam, Dataproc, Pub/Sub streaming, Cloud Composer orchestration, BigQuery mastery, CI/CD and governance — threaded through a single connected-vehicle-style platform capstone from ingestion to dashboards, closed with a viva.
15teaching days
120instructed hours
29sessions
4modules
B4Professional
8 hours a day, on the same honest-hours basis as the rest of our catalog.
The shape
Where the time goes, and how it is delivered.
Where the time goes
Week 1: Foundations & BatchWeek 2: Streaming & OrchestrationWeek 3: Production EngineeringAssessment
How the time is delivered
LAB · 58 h (48%)PA · 18 h (15%)ILT · 36 h (30%)MP · 8 h (7%)
FLEETLINE — one connected-vehicle platform, grown daily
The capstone is not a final week; it is the fifteen days. From day one the class grows one connected-vehicle telemetry platform — lake to warehouse, batch beside streaming, orchestrated, governed and costed per run — and every lab is a change to that same platform. It closes on an architecture review, a live query demo and a viva where the partition choices made in week two have to be defended on evidence.
one platform grown dailyLooker Studio servingvehicle telemetry ingest designraw-to-serving table modelingbatch & streaming reconciliationcost per pipeline run
The engagement
FLEETLINE, gated.
Each stage produces an artifact a panel can inspect. Nobody advances on attendance.
1
FL-1 · Ingest to serving
A batch path from telemetry landing in the lake to a dashboard-ready serving table.
Pass-gate — A rerun produces no duplicates and row counts reconcile end to end, performed under observation.
Day 5
2
FL-2 · The stream
A streaming path with deduplication, windowing and a late-data route, reconciled against batch.
Pass-gate — Replayed messages leave the sink unchanged, late events are visibly handled, and batch-versus-streaming disagreement is inside the stated tolerance.
Day 10
3
FL-3 · Production ready
The platform deployed through CI/CD, governed, monitored, and reporting cost per pipeline run.
Pass-gate — An injected lag condition raises the right alert, and a pipeline change reaches an environment without console access.
Day 14
4
FL-4 · Defended
An architecture review, a live query demo and a viva against the capstone rubric.
Pass-gate — Partition and cluster choices are defended on query evidence, and delivery semantics under failure are answered correctly.
Day 15
Curriculum
Every session, every lab, every hour.
29 sessions across 4 modules. Expand any module to see the sessions, what a learner can do at the end of each, and the artifact they submit.
Week 1: Foundations & Batch40 h · 10 sessions
D1a
The lake and the warehouse
Decide what belongs in object storage and what belongs in the warehouse, and defend the boundary against the usual pressure to move everything.
Submittable labA storage decision record placing each telemetry dataset on one side of the line, with the cost and query consequence of each placement.
ILT3 h
D1b
GCS lake with lifecycle
Lay out a bucket structure that will still make sense at a thousand times the data, with lifecycle rules that actually fire.
Submittable labA partitioned GCS layout with lifecycle policy applied and evidence of an object transitioning storage class.
LAB5 h
D2a
The Beam model
Explain PCollection and PTransform semantics well enough to predict what a pipeline does before running it.
Submittable labAn annotated pipeline diagram naming the transform at each stage and the collection shape between them.
ILT3 h
D2b
Dataflow batch pipeline
Move a day of vehicle telemetry from the lake into the warehouse with a batch pipeline you can rerun safely.
Submittable labA Dataflow batch job with a rerun demonstrated producing no duplicate rows.
LAB5 h
D3a
Spark on Dataproc
Read a Spark plan and locate the shuffle that is costing you, rather than guessing at executor counts.
Submittable labAn annotated Spark plan for a telemetry aggregation identifying the dominant shuffle and its cause.
ILT3 h
D3b
Ephemeral Dataproc with tuned partitions
Run a job on a cluster that exists only for that job, and cut its runtime by fixing partitioning rather than adding nodes.
Submittable labA before-and-after runtime comparison on an ephemeral cluster with the partitioning change that produced the improvement.
LAB5 h
D4a
Table modelling for serving
Model raw, curated and serving layers so a dashboard query does not scan the lake.
Submittable labA three-layer table model with the query each layer is designed to answer.
ILT2 h
D4b
Raw-to-serving build
Build the raw, curated and serving tables for vehicle telemetry and measure the bytes each dashboard query scans.
Submittable labThe three-layer tables built, with bytes-scanned recorded for the same dashboard query against each layer.
LAB6 h
D5a
What a batch platform owes you
State the guarantees a batch platform must make about completeness and reruns before anyone trusts a dashboard from it.
Submittable labA written guarantee sheet covering completeness, idempotency and late-arriving data.
ILT2 h
D5b
Proctored — the batch gate
Under proctored conditions, deliver an ingest-to-serving batch path and defend its rerun and completeness guarantees.
Submittable labThe platform's batch path assessed live, with a rerun performed under observation and row counts reconciled.
PA6 h
Week 2: Streaming & Orchestration40 h · 10 sessions
D6a
Pub/Sub delivery semantics
Say precisely what at-least-once delivery obliges you to build, and where exactly-once is a property of your sink rather than the broker.
Submittable labA delivery-semantics note naming where duplicates can enter and which component removes them.
ILT3 h
D6b
Streaming ingest with dedup on message id
Ingest a telemetry stream and make duplicate delivery harmless rather than merely unlikely.
Submittable labA streaming ingest with deliberately replayed messages and evidence that the sink row count is unchanged.
LAB5 h
D7a
Windows, watermarks and late data
Choose a windowing strategy and an allowed lateness from what the business will do with the number.
Submittable labA windowing decision with the allowed lateness justified against the reporting deadline it serves.
ILT3 h
D7b
Dataflow streaming with late-data handling
Handle events that arrive after their window closed, and show what the platform does with them rather than dropping them silently.
Submittable labA streaming job processing injected late events, with the late-data path and its output demonstrated.
LAB5 h
D8a
Orchestration that fails well
Design DAGs whose failures are diagnosable at 3am by someone who did not write them.
Submittable labA DAG design with retry, alerting and idempotency stated per task.
ILT3 h
D8b
Composer DAGs with backfill
Orchestrate the batch and streaming paths and run a backfill without corrupting what is already there.
Submittable labA Composer DAG with a backfill executed over a historical window and the result reconciled against the original load.
LAB5 h
D9a
BigQuery partitioning and clustering
Choose partition and cluster keys from the query patterns you actually have, and know what they cost on write.
Submittable labA partition and cluster decision per serving table with the query pattern each is chosen for.
ILT2 h
D9b
Partitioned and clustered tables
Apply partitioning and clustering and measure the bytes-scanned improvement rather than assuming it.
Submittable labTables rebuilt with partitioning and clustering, submitted with before-and-after bytes scanned for three real queries.
LAB6 h
D10a
Batch and streaming, reconciled
Explain why the two paths disagree and what an acceptable discrepancy looks like for this platform.
Submittable labA reconciliation policy naming the tolerance and what happens when it is exceeded.
ILT2 h
D10b
Proctored — the streaming gate
Under proctored conditions, deliver the streaming path and reconcile it against the batch path in front of an assessor.
Submittable labThe streaming path assessed live, with batch-versus-streaming reconciliation performed on demand.
PA6 h
Week 3: Production Engineering32 h · 8 sessions
D11a
Serverless triggers and their limits
Choose between Cloud Run and Functions on the properties of the trigger and the workload, not on habit.
Submittable labA trigger design naming the runtime chosen per event source and the constraint that decided it.
ILT3 h
D11b
Event-driven ingest with Cloud Run
Wire a file-arrival trigger into the pipeline and make it safe to fire twice.
Submittable labAn event-driven ingest with a duplicate trigger fired and the outcome shown to be idempotent.
LAB5 h
D12a
CI/CD and infrastructure as code
Get pipelines and their infrastructure into one reviewable change, so a data change and a deployment cannot drift apart.
Submittable labA pipeline-and-infrastructure change modelled as one reviewable unit, with the review checklist it requires.
ILT3 h
D12b
Deploy a pipeline through CI/CD
Ship a pipeline change through automated tests to an environment without touching a console.
Submittable labA pipeline change deployed through CI/CD with the test output and the deployment record.
LAB5 h
D13a
Cost control at the slot level
Explain where BigQuery cost actually accrues, and which lever to pull before someone proposes a reservation.
Submittable labA cost model showing the platform's spend by driver with the lever ranked by effect.
ILT2 h
D13b
Slot and bytes-scanned control
Bring a runaway query cost down with modelling and controls, and record cost per pipeline run.
Submittable labA cost reduction demonstrated on a real query, with cost per pipeline run recorded before and after.
LAB6 h
D14a
Monitoring backlog and lag
Decide what backlog and lag figures mean trouble for this platform, and what the alert should say when they do.
Submittable labAn alerting design with the backlog and lag thresholds chosen and the runbook entry each alert points to.
ILT2 h
D14b
Proctored — the production gate
Under proctored conditions, deliver the platform with CI/CD, governance, monitoring and cost control, and defend the alert thresholds.
Submittable labThe production layer assessed live, including an injected lag condition raising the correct alert.
PA6 h
Assessment8 h · 1 sessions
D15
Architecture review, viva and live demo
Defend the whole platform to a panel — partition choices, delivery semantics under failure, and the cost-versus-latency trade-offs you made.
Submittable labA rubric-scored capstone defence with a live query demo, an architecture review, and answers on delivery semantics under induced failure.
MP8 h
| Mode | What it means |
|---|---|
| ILT | Instructor-led session (live, cohort) |
| LAB | Hands-on lab with submittable artifact |
| SP | Self-paced (curated resources + exercise) |
| MP | Mentor panel (review / pairing / retro) |
| PA | Proctored assessment (BuildReady / ProctorShield) — feeds PRI → DRI |
Proof
How we decide someone is ready.
The same seven-criterion rubric every Mastery Program is assessed on. The weights are fixed and published, and a panel scores against them.
Working deployment
System runs in the (mock) customer environment and a simulated user adopts it; judged on outcomes, not demo polish.
25%
Decomposition & scope
Quality of the discovery memo, SoW and workplan; assumptions surfaced; the '48-second rule' honoured.
15%
Evaluation rigor
Golden set, LLM-as-Judge calibration, regression coverage; the six metric families instrumented.
20%
Integration & security
SSO/auth done right, least-privilege IAM, resilience patterns, compliance posture articulated.
15%
Observability & cost
Tracing in place; latency and token-cost budgets defined and met.
10%
Executive communication
C-suite demo lands with a non-technical audience; questions handled with composure.
10%
Handoff & playbook
Runbook a customer could operate; reusable playbook codified for the next engagement.
5%
Next step
Run CL-222 for your team.
Tell us who is being trained and where they start, and we will scope it.