Mastery Program · AI-510
AI & GenAI Engineering — Flagship Capstone Program
Our flagship end-to-end journey: pandas to production in 31 days, with the trainer building a real AI helpdesk live — one increment every day — while learners watch accuracy climb from classical baseline to fine-tuned transformer on the same data split. Certified via 15 rotating capstone briefs on a fixed rubric with fresh datasets every cohort.
31teaching days
248instructed hours
31sessions
5modules
B5Advanced
8 hours a day, on the same honest-hours basis as the rest of our catalog.
The shape
Where the time goes, and how it is delivered.
Where the time goes
Weeks 1: Data & GenAI CoreWeek 2: Agents, RAG & MCPWeeks 3–4: ML → Deep LearningWeek 5: Transformers & Fine-TuningWeek 6: Assistants & ModelOps
How the time is delivered
LAB · 136 h (55%)PA · 48 h (19%)ILT · 56 h (23%)MP · 8 h (3%)
The Golden Thread — one helpdesk assistant, built in front of the room
From day one the trainer builds a real enterprise helpdesk assistant live, one increment every teaching day, and learners build it alongside on the same seeded ticket corpus. It starts as a classical baseline and ends as a deployed, evaluated, rollback-capable service. Every lab in this programme is a commit to that same system, so nobody leaves with 31 disconnected exercises — they leave with one thing they built, an accuracy ladder showing how it got there, and a git history they can walk an interviewer through.
daily live buildaccuracy laddergit-tag recoverywow demosone increment per dayreproducible seedsrotating capstone briefs
The engagement
The Golden Thread, gated.
Each stage produces an artifact a panel can inspect. Nobody advances on attendance.
1
GT-1 · Corpus and service foundation
A reconciled ticket corpus and a running /answer endpoint with measured cost per 100 requests.
Pass-gate — The row reconciliation balances, and the cost figure comes from the meter rather than the pricing page.
Day 5
2
GT-2 · Grounded retrieval
A RAG endpoint returning answers with source spans, scored for grounding across a held-out ticket set.
Pass-gate — Grounding meets the threshold agreed at kickoff, and every ungrounded answer is diagnosed rather than excused.
Day 10
3
GT-3 · Honest measurement
A temporal validation split, a metric memo, and a leakage report on a pipeline that scored too well.
Pass-gate — The gap between random-CV and temporal scores is quantified, and the leak is found and closed.
Day 17
4
GT-4 · Model selection
A bake-off across every candidate model on one held-out period, with a catalogue of the winner's errors.
Pass-gate — The selection is defended on held-out evidence, and the chosen model's failure modes are named.
Day 21
5
GT-5 · Adapted and served
A fine-tuned, quantised checkpoint with a precision-quality-latency table against the prompted baseline.
Pass-gate — It beats prompting where the claim is made, and the cases where adaptation made things worse are listed.
Day 26
6
GT-6 · Deployed and defensible
The assistant load-tested against a latency target, with an eval harness and a timed rollback drill.
Pass-gate — p95 meets target, the harness catches a seeded regression, and rollback completes inside the stated budget.
Day 31
Curriculum
Every session, every lab, every hour.
31 sessions across 5 modules. Expand any module to see the sessions, what a learner can do at the end of each, and the artifact they submit.
Weeks 1: Data & GenAI Core40 h · 5 sessions
D1
Pandas mastery
Get the raw ticket export into a state you would stake a model on, and account for every row that did not survive.
Submittable labA notebook taking the vendor export to cleaned Parquet, submitted with a reconciliation table: rows in, rows out, and the reason for each exclusion.
ILT8 h
D2
FastAPI+Azure OpenAI
Put a real service in front of the model — timeouts, retries, token accounting — so the first thing built is the thing that has to run at 3am.
Submittable labA running /answer endpoint and a cost note giving tokens and spend per 100 requests, read off the meter.
LAB8 h
D3
Cognitive APIs
Decide which managed cognitive services earn their place in the pipeline, on measured latency and cost rather than on the vendor's page.
Submittable labA comparison over 50 real tickets — output, latency and cost per service — ending in a keep-or-drop verdict and the reason for each.
LAB8 h
D4
LangChain
Use an orchestration framework deliberately: know what it does at each step well enough to rebuild the chain without it.
Submittable labThe same ticket summarisation built twice, once with the framework and once with plain SDK calls, plus a note on what the abstraction bought and what it cost.
LAB8 h
D5
SQL feature extraction
Under proctored conditions, turn the ticket table into per-customer features that could be computed at prediction time and no earlier.
Submittable labA feature table with, for every column, a one-line argument that it is available before the label exists.
PA8 h
Week 2: Agents, RAG & MCP40 h · 5 sessions
D6
Multi-agent labs
Split a workflow into agents only where the split pays for itself, and say plainly where one agent would have been the better answer.
Submittable labA design for the triage-and-resolve flow naming each agent, its tools, its stopping condition, and the failure the split introduces.
ILT8 h
D7
RAG+citations
Answer from the knowledge base with a citation, then measure how often that citation actually supports the sentence it is attached to.
Submittable labA RAG endpoint returning answer plus source spans, and a grounding score over a held-out ticket set with every ungrounded case diagnosed.
LAB8 h
D8
MCP chains
Expose internal tools under permission, so the tool the assistant must never reach cannot be reached even when it tries.
Submittable labAn MCP server exposing ticket lookup and refund, submitted with a test proving refund is unreachable for an unauthorised caller.
LAB8 h
D9
GPU engineering
Read a GPU the way you read a disk: know the ceiling, know the pressure, and know what breaks first when load doubles.
Submittable labA capacity note for the inference path — model, precision, batch, memory ceiling — naming the first component to fail at 2x load.
LAB8 h
D10
Chunking comparison
Under proctored conditions, establish on your own corpus what chunking actually does to retrieval, rather than inheriting someone's default.
Submittable labA sweep across at least four chunking strategies scored on retrieval hit rate, ending in a recommendation the numbers support.
PA8 h
Weeks 3–4: ML → Deep Learning88 h · 11 sessions
D11
Sklearn
Set the classical baseline that every later model in this programme has to beat, and commit the number publicly.
Submittable labA baseline priority model with its held-out score recorded as the first rung of the accuracy ladder.
ILT8 h
D12
Backprop
Derive backpropagation and implement it by hand, so a bad training curve becomes a thing you can diagnose rather than restart.
Submittable labA NumPy-only network trained on a small task, with gradients verified against a numerical check and the check submitted alongside.
LAB8 h
D13
PyTorch MLP
Rebuild the same network on a framework, and explain any difference from your own implementation rather than accepting it.
Submittable labA PyTorch MLP matching the NumPy version's loss curve, with any divergence investigated and its cause written down.
LAB8 h
D14
CNN/RNN
Choose convolutional or recurrent structure from what the data is, and state what evidence would change your mind.
Submittable labA written comparison on ticket-text sequence data naming the structure that suits it and the experiment that would overturn the choice.
ILT8 h
D15
Metrics discipline
Pick the metric that reflects what being wrong costs this business, and show the point where accuracy starts lying to you.
Submittable labThe chosen threshold applied to the live model, submitted with its confusion matrix at that threshold and a memo giving the metric, the confusion cost behind it, and what the threshold costs in missed tickets.
LAB8 h
D16
Leakage hunting
Break a pipeline that looks correct and scores beautifully, by finding the leak that makes it so.
Submittable labA before-and-after report on a leaking pipeline: the leak, the mechanism that let it in, and the honest score once it is closed.
LAB8 h
D17
Temporal CV
Under proctored conditions, validate against time the way production will, and quantify how much random k-fold was flattering you.
Submittable labA temporal validation over the ticket history with the random-CV and temporal scores side by side and the gap explained.
PA8 h
D18
Text classification on the ticket corpus
Take the corpus from raw text to a defensible intent classifier, settling representation before reaching for a bigger model.
Submittable labAn intent classifier with per-class scores and a confusion matrix, naming the two classes you would fix first and why those two.
LAB8 h
D19
Embeddings & intent discovery
Find the intents nobody labelled, and judge which clusters are real rather than artefacts of the embedding you happened to choose.
Submittable labA clustering over the corpus with named candidate intents and a written case for which to adopt, which to merge and which to reject.
LAB8 h
D20
Imbalance & threshold engineering
Serve the rare, expensive ticket class without wrecking everything else, setting the threshold from the cost matrix rather than from 0.5.
Submittable labA model tuned for the minority class with the threshold chosen on cost, and the trade-off stated in tickets per week.
LAB8 h
D21
Model bake-off & error analysis
Under proctored conditions, choose between every model built so far on held-out evidence, and describe how the winner fails.
Submittable labA bake-off table across all candidates on one held-out period, with a selection verdict and a catalogue of the winner's characteristic errors.
PA8 h
Week 5: Transformers & Fine-Tuning40 h · 5 sessions
D22
Transformer internals
Follow a token through attention until the architecture stops being a diagram and becomes something you can debug.
Submittable labAn annotated forward pass over one helpdesk sentence showing tensor shapes at every stage and what attention actually selected.
ILT8 h
D23
HF fine-tuning
Fine-tune an open model on the ticket corpus, and prove it beats prompting on a task where it should — or report that it did not.
Submittable labA fine-tuned checkpoint with a head-to-head evaluation against the prompted baseline, including the cases fine-tuning made worse.
LAB8 h
D24
Quantization
Trade quality for memory and latency on purpose, and state the exchange rate you accepted.
Submittable labA quantised model with a precision-quality-latency table and a serving recommendation the table supports.
LAB8 h
D25
QLoRA
Choose adapter rank and target modules from the task in front of you, not from the first configuration in the README.
Submittable labAn adaptation plan naming target modules, rank and the memory budget it has to fit, with the reasoning for each choice.
ILT8 h
D26
Attention by hand
Under proctored conditions, implement scaled dot-product and multi-head attention from scratch and match a reference numerically.
Submittable labA from-scratch attention module whose outputs match the framework within tolerance, submitted with the test that demonstrates it.
PA8 h
Week 6: Assistants & ModelOps40 h · 5 sessions
D27
Chat-assistant capstone
Assemble the Golden Thread into one assistant and draw its boundaries: what it answers, what it escalates, what it refuses outright.
Submittable labAn assistant specification listing supported intents, escalation rules and refusal cases, each with the evidence behind it.
ILT8 h
D28
Eval harness
Build the suite that tells you the assistant got worse, and prove it works by making the assistant worse on purpose.
Submittable labAn eval harness over a frozen ticket set, demonstrated catching a seeded regression and reporting precisely what broke.
LAB8 h
D29
Registry+rollback
Practise the recovery before you need it: promote a bad model, detect it, and get back inside a time budget you stated in advance.
Submittable labA timed rollback drill with the budget declared beforehand and the elapsed time recorded against it.
LAB8 h
D30
Deployment
Face a panel with the deployment design — traces, alerts, cost controls — and defend what the on-call runbook leaves out.
Submittable labA deployment design reviewed by panel, covering observability, alerting and token budget, with a runbook entry for the two likeliest failures.
MP8 h
D31
FastAPI service
Under proctored conditions, ship the assistant against a stated latency target and show how it degrades once it is past it.
Submittable labA load-tested service with measured p95 against target and documented behaviour beyond the breaking point.
PA8 h
| Mode | What it means |
|---|---|
| ILT | Instructor-led session (live, cohort) |
| LAB | Hands-on lab with submittable artifact |
| SP | Self-paced (curated resources + exercise) |
| MP | Mentor panel (review / pairing / retro) |
| PA | Proctored assessment (BuildReady / ProctorShield) — feeds PRI → DRI |
Proof
How we decide someone is ready.
The same seven-criterion rubric every Mastery Program is assessed on. The weights are fixed and published, and a panel scores against them.
Working deployment
System runs in the (mock) customer environment and a simulated user adopts it; judged on outcomes, not demo polish.
25%
Decomposition & scope
Quality of the discovery memo, SoW and workplan; assumptions surfaced; the '48-second rule' honoured.
15%
Evaluation rigor
Golden set, LLM-as-Judge calibration, regression coverage; the six metric families instrumented.
20%
Integration & security
SSO/auth done right, least-privilege IAM, resilience patterns, compliance posture articulated.
15%
Observability & cost
Tracing in place; latency and token-cost budgets defined and met.
10%
Executive communication
C-suite demo lands with a non-technical audience; questions handled with composure.
10%
Handoff & playbook
Runbook a customer could operate; reusable playbook codified for the next engagement.
5%
Next step
Run AI-510 for your team.
Tell us who is being trained and where they start, and we will scope it.