Module 20
Reliability & Safety
Guardrails, ensembles, reliability patterns, governance and compliance.
- lessons
- 4
- steps
- 36
- time
- 52 min
01intermediate·13 min·9 steps
Guardrails
Wrap a probabilistic model in input, output and action checks, know which ones actually hold, and prove they fire.
- Where input, output and action guardrails sit
- Validate output: safety, relevance, grounding and schema
- Abstain or escalate on a confidence signal you control
- Bind approvals to the exact action and sandbox the rest
Start →
02intermediate·12 min·9 steps
Ensemble Methods
Use several samples, models or judges to make LLM output more reliable, and know what each pattern costs and what it can't catch.
- Self-consistency: vote over k samples, with vote share as confidence
- Best-of-N with a conservative reward-model ensemble
- Why a panel of judges that agrees can still be wrong
- Debate, mixture of agents, cascades and critique
Start →
03intermediate·13 min·9 steps
Reliability Patterns
Retry the right errors, stop hammering a dead provider, isolate workloads, set layered timeouts and degrade gracefully when everything fails.
- Classify errors by status and code before retrying
- Backoff with jitter that honours Retry-After
- Circuit breaker states and what counts as a failure
- Bulkheads and layered timeouts, including time to first token
Start →
04intermediate·14 min·9 steps
AI Governance & Compliance
Turn the EU AI Act, NIST AI RMF and incident-reporting clocks into concrete engineering artifacts that fall out of systems you already run.
- Emit governance evidence as a byproduct of existing practice
- Classify a system by EU AI Act risk tier and role
- Implement Article 50 disclosure and two-layer marking
- Build an audit trail and meaningful human oversight
Start →
Read deeper
Guide chapters · Reliability & Safety
Next module · 21
Evaluation & Observability