AI Security
Prompt injection, jailbreaks, OWASP LLM Top 10, guardrails, PII, least privilege, sandboxing, secrets, approvals, audit and red teaming.
- lessons
- 22
- steps
- 201
- time
- 269 min
LLM Security
Map the trust boundaries of an LLM app, see how injected text turns into stolen data, and layer defenses so one failure isn't fatal.
- Why an LLM can't reliably tell instructions from data
- Draw the trust boundary and the OWASP risks on it
- Follow an indirect injection from a web page to exfiltration
- Layer IPI defenses, with capability gating and egress control
Access Control
Authenticate every caller, authorize every action, isolate every tenant's data and cache, and leave an audit trail, including for agents.
- The four dimensions: authentication, authorization, isolation, audit
- Replace static keys with workload identity
- Filter by tenant inside the database, never after retrieval
- Close the shared-cache timing side channel
Prompt Injection
Why a model can't tell your instructions from an attacker's, and how to design so a fooled model still can't do much harm.
- Why instructions and data collapse into one token stream
- Tell direct injection from indirect injection
- Layer the eight defences, innermost first
- Shrink the blast radius tool by tool
Indirect Prompt Injection
Attacks hidden in tickets, emails and web pages that an agent reads on someone's behalf, and how breaking the lethal trifecta defuses them.
- How an attack planted in content reaches the model
- Spot the lethal trifecta and remove one leg
- Apply the eight defences to a ticket-summary agent
- Know the residual risk that survives the fixes
Jailbreaks
Inputs crafted to slip past a model's safety training, the families they come in, and the layered checks that catch them.
- Tell a jailbreak from a prompt injection
- Recognise the six jailbreak families
- See which defence covers which family
- Build a policy stack for an enterprise assistant
OWASP Top 10 for LLMs
The shared checklist of the ten most critical LLM-application risks, and how to turn it into a threat model, controls and tests.
- Name the ten 2025 risks and each one's main control
- Use the list as a threat-modelling process
- Map the list onto a real assistant
- Avoid treating it as a compliance checkbox
Input Guardrails
Fast, parallel checks on every incoming message that decide allow, redact, redirect or block before the main model runs.
- List the six checks an input guardrail runs
- Run checks in parallel and combine them in a policy
- Pick thresholds from a labelled set
- Restrict instead of block for uncertain cases
Output Validation
Treat everything a model produces as untrusted input: check its structure, content and safety before you show it, render it, pass it on or act on it.
- Why model output is untrusted input (OWASP LLM05)
- The seven things to check before output is used
- Walk the validation pipeline, including one bounded repair
- Follow three real outputs: released, repaired, redacted
PII Redaction
Detect personal data and swap it for typed placeholders before text reaches models, logs, indexes and datasets that don't need it.
- Why personal data should never reach systems that don't need it
- Layer regex, checksums, NER and custom recognizers
- Keep the placeholder mapping in your app so tools get real values
- Apply redaction at every boundary, not only the model call
Data Leakage
Find the seven channels through which an AI system leaks sensitive data, and close each one with an ordinary, testable control.
- Name the seven leak channels and the control for each
- Scope every retrieval and tool call to the requester, in code
- Trace a real-looking RAG leak to its missing permission dimension
- Fix it, and test for it continuously
Least-Privilege Tools
Give each agent only the narrow tools, scopes and credentials its job needs, so a confused or manipulated model can do very little damage.
- See how one generic tool hands the model the whole system
- Split tools by task, identity and read vs write
- Read the per-agent permission matrix
- Shrink the blast radius without losing capability
Tool Sandboxing
Run generated code, shell commands, browsing and file processing in disposable environments with no secrets, no open network and hard resource limits.
- Why generated code must never run in your app process
- Draw the sandbox boundary: no secrets, no network, hard limits
- Apply the eight sandboxing steps
- Follow a crafted CSV that finds nothing to steal
Secrets Management
Keep every credential in a vault, deliver it at runtime through workload identity, and add it in tool code so the model never holds a key.
- The two leak paths LLMs add: prompts and model-chosen tool arguments
- The one permitted path from vault to API call
- Inject credentials inside tool code, not into the model's context
- Static keys vs short-lived tokens, gateways, and env vars
API Key Rotation
Replace credentials on a schedule with an overlap window so nothing breaks, then revoke the old key — and practise the emergency version before you need it.
- Why every long-lived key eventually needs replacing
- The five-step zero-downtime rotation with two overlapping keys
- Confirm the old key is unused before revoking it
- Run an emergency rotation: revoke first, then investigate
Authorization for Agents
Decide every tool call in code: may this agent do this action on this resource for this verified user — using facts the model never supplies.
- Spot the confused deputy and model-chosen identity failures
- The four inputs to every tool-call decision, none from the model
- Allow only when both the user and the agent are permitted
- Trace four real requests through the policy
Human Approval for Actions
Make high-impact agent actions wait for the right person to approve the exact change — bound to its parameters, single-use and short-lived — so approval is a control, not theatre.
- Why an approval screen written by the model is security theatre
- Tag tools by risk so only high-impact actions wait for a human
- The six-stage approval pipeline, from pending action to audit
- Bind approval to parameters, approver, expiry and single use
Content Moderation
Classify both inputs and outputs into policy categories and severities, then apply a defined action — allow, warn, redirect, block or hand to a human — consistently and auditably.
- Why one filter with one action fails most real messages
- Moderate both inputs and outputs against a written policy
- Read a category × severity policy matrix
- Handle venting, security-bypass requests and distress differently
Data Poisoning
Corrupted training data, knowledge-base edits or feedback can make a model misbehave on cue — control sources, review high-impact changes, track provenance and test after every update.
- What poisoning is, and why a few hundred documents can be enough
- The five places poisoned data enters an AI system
- A defence pipeline: trust tiers, review, validation, provenance
- Catch and roll back a poisoned wiki edit with canary questions
Access Control in RAG
Make retrieval return only what the requesting user may open: copy source permissions onto chunks, filter every query in the store, and keep permissions in sync.
- Why “the prompt says only use allowed docs” doesn't protect anything
- Copy source ACLs onto chunks at ingestion, with default deny
- Filter by verified groups inside the store, for both halves of hybrid search
- Measure and bound how long a revoked permission still works
MCP Server Trust
Every MCP server is a privileged dependency whose tool descriptions and results flow straight into your agents — vet it, contain it, and watch it for changes.
- Why an MCP server is a privileged dependency, not just an API
- Recognise tool poisoning, rug pulls and tool shadowing
- Vet, contain and monitor servers through a registry and gateway
- Block a changed tool description before any agent reads it
Audit Logging
Record who acted, which agent acted for them, what changed, why it was allowed and what happened — in tamper-evident storage you can actually search when someone asks.
- How audit logs differ from engineering traces
- The fields an agent audit record needs, including “on behalf of”
- Emit records in code into tamper-evident storage
- Settle a “the bot did it without asking” dispute from the log
Red Teaming
Attack your own AI system on purpose — people plus automated tools, against the full pipeline — and turn “we think it's secure” into an attack success rate tracked every release.
- Turn “we think it's secure” into a measured attack success rate
- Scope a red-team exercise with a threat model and OWASP-mapped attack sets
- Read a red-team report and decide whether a release ships
- Fix with controls, then lock findings into a CI regression set