Skip to content
anythingai.space
+References & Evals·7 min read

AI Design Patterns Quick Reference

Quick lookup for common patterns. See individual chapters for detailed implementation.


Retrieval Patterns

PatternUse CaseKey Tradeoff
Basic RAGSimple Q&A over documentsEasy to implement, limited accuracy
Hybrid SearchCombining semantic + keywordBetter recall, more complexity
RerankingHigh-precision retrievalAccuracy vs latency
Query ExpansionAmbiguous queriesBetter recall, more tokens
HyDENo direct matches expectedCreative, but can hallucinate
Parent-Child ChunkingNeed surrounding contextMemory overhead
text
Query → Embed → Vector Search → Rerank → Top-K → Generate
↓
BM25 Search ─────────┘ (hybrid)

Generation Patterns

PatternUse CaseKey Tradeoff
Zero-ShotSimple tasksFast, less reliable
Few-ShotNeed format controlToken cost
Chain-of-ThoughtReasoning tasksLatency, shows work
Self-ConsistencyHigh-stakes answers3-5x cost
Structured OutputAPI responsesConstrained creativity

Agent Patterns

PatternUse CaseComplexity
ReActTool-using agentsMedium
Plan-and-ExecuteMulti-step tasksHigh
Multi-Agent DebateVerificationHigh
Human-in-the-LoopHigh-stakes actionsMedium
Swarm / HandoffSpecialized sub-agentsHigh
Advisor / ExecutorCheap model runs the loop, strong model consulted at decision pointsMedium
Orchestrator + isolated subagentsParallel work without context contentionHigh
Managed harness (rent the loop)Durable sessions and recovery you do not operateLow to build; residency and retention limits decide

Advisor / executor, added in 2026, is the cost-quality lever worth knowing: an inexpensive executor drives the agent loop and calls a stronger advisor model at decision points, passing the transcript and receiving a plan or correction. It is a cost claim rather than a quality claim: pairing a low-effort executor with a stronger advisor can beat the cost-quality line that executor traces by raising its own effort, while the top scores still belong to maximum effort at higher cost. Published figures are vendor-benchmarked, so measure on your own tasks. Consult rate is the metric to watch: if the executor consults on nearly every step you have bought an expensive model with extra latency.

Orchestrator plus isolated subagents is where the single-versus-multi-agent argument landed. Subagents get their own context and return summaries, with no peer-to-peer channel between them. Context is what is isolated, not state: managed platforms typically share one sandbox, filesystem, and credential set across the roster. Vendor platforms enforce the limits that make it work in practice, typically one level of delegation and a cap on roster size (OpenAI's Agents API defaults max_concurrent_subagents to 6), which is a good default even when your framework does not enforce it.

Managed harness options multiplied in September 2026, when OpenAI's Agents API (public beta) and Amazon Bedrock Managed Agents built on it (preview) joined Claude Managed Agents (beta). Amazon Bedrock AgentCore's managed agent harness has been generally available since June 17, 2026, with any model and an export path to Strands-based code, which answers the lock-in objection. All of them run durable sessions and crash recovery for you. The disqualifiers are usually data handling rather than features (the Agents API launched with US residency only and no zero data retention), and no managed runtime gives exactly-once semantics for your own side-effecting tools. See Durable Execution.

text
┌─────────────────────────────────────────┐
│ REACT LOOP │
│ │
│ Observe → Think → Act → Observe → ... │
│ ↓ │
│ [Tool Call] │
│ ↓ │
│ [Result] │
└─────────────────────────────────────────┘

Agentic Coding Patterns (2026)

PatternUse CaseKey Tool
Scaffold → Implement → VerifyFull feature developmentClaude Code / OpenHands
Read-Plan-EditRefactoring existing codeClaude Code text_editor
Test-Driven AgentHigh reliability codeAgent writes tests first
Shadow ReviewPR quality gateAgent reviews diff before merge
AGENTS.md / CLAUDE.md ManifestProject context injectionAGENTS.md works across tools; Claude Code reads it when no CLAUDE.md exists
Sub-Agent ParallelismLarge codebase changesMultiple agents per module
text
┌────────────────────────────────────────────────────────┐
│ AGENTIC CODING LOOP │
│ │
│ Understand → Plan → Implement → Run Tests → Fix │
│ ↑ (bash + text_editor tools) │ │
│ └──────────── Iterate until tests pass ────┘ │
│ │
│ [AGENTS.md / CLAUDE.md injects: coding style, │
│ test commands, forbidden patterns, │
│ architecture decisions] │
└────────────────────────────────────────────────────────┘

When to use which tool:

text
Need full autonomy + CLI → Claude Code / Codex CLI
Need open-source + any LLM → OpenHands / Cline / OpenCode
Need tight IDE integration → Cursor / Devin Desktop (formerly Windsurf)
Need reproducible pipelines → OpenHands in Docker CI
Need a harness you don't operate → OpenAI Agents API / Claude Managed Agents

Reliability Patterns

PatternProblem SolvedImplementation
Retry with BackoffTransient failuresExponential backoff
Circuit BreakerCascading failuresFail-fast after threshold
Fallback ModelPrimary unavailableSecondary model
TimeoutSlow responsesCancel + fallback
BulkheadResource isolationSeparate pools
Refusal-Aware RoutingSafety refusal returned as a successBranch on stop_reason, not HTTP status
python
# Reliability stack
@circuit_breaker(failure_threshold=5)
@retry(max_attempts=3, backoff=exponential)
@timeout(seconds=30)
@fallback(model="claude-sonnet-5-5") # a different vendor from the primary
async def generate(prompt):
return await primary_model.generate(prompt)

Fallbacks have to cross vendors, because one incident can hit everything a vendor serves at once (OpenAI's September 29, 2026 incident degraded the API, ChatGPT, and Codex together for about 5 hours 20 minutes). Fallback also changes behavior, not just the endpoint: Claude thinking blocks are bound to the model that produced them and cannot be replayed into another, and a Claude safety refusal arrives as HTTP 200 with stop_reason: "refusal", so a status-code check never sees it. See Design Patterns.


Caching Patterns

PatternHit RateUse Case
Exact MatchLowIdentical queries
Semantic CacheMediumSimilar queries
Prompt (Prefix) CacheHigh for agent loopsStable prefix (system prompt, tools, history); provider cache reads bill at 0.025x to 0.1x input, and self-hosted engines (vLLM, SGLang) reuse the KV cache directly
Response CacheVariesDeterministic outputs

Security Patterns

PatternThreatImplementation
Input ValidationPrompt injectionSanitize, detect
Output FilteringData leakagePII detection, blocklists
Tenant IsolationCross-tenant accessFilter at query time
Rate LimitingAbusePer-user/tenant limits
Egress AllowlistExfiltration, eval breakoutsDefault-deny network, including DNS, package proxies, and publish endpoints
Bound ApprovalsApproval launderingApproval hashed to the exact action and re-verified at execution
Skill PinningSilently changed instructionsVerify skill manifest digests; re-approve on any change
Harness IsolationRepo config running code outside the sandbox (GitSpawn)Run the agent harness itself inside the sandbox; treat .git/config and agent config files as code
Monitor-and-KillAn alert nobody acts on in timeWire the monitor to an automatic stop and test it like a failover; an OpenAI training run alarmed after about 12 minutes but was killed about 2.5 hours later when its auto-stop failed
text
Input → Validate → Sanitize → LLM → Filter → Validate → Output

Evaluation Patterns

PatternUse CaseMetrics
Golden SetRegression testingPass rate
LLM-as-JudgeQuality scoringBinary pass/fail per criterion, calibrated against human labels
Human EvalGround truthAgreement rate
A/B TestingProduction comparisonUser metrics

Cost Optimization Patterns

PatternSavingsTradeoff
Model Routing50-70%Complexity
Prompt (Prefix) Caching90-97.5% on cached input tokensWrite premium (1.25x; 2x for Anthropic's 1-hour cache); TTL management
Response / Semantic Caching20-40%Staleness
Prompt Compression10-30%Quality risk
Batch / Flex Tier50%Latency (async or best-effort)
text
Query → Classify → Route → [Small Model] or [Large Model]
↓
[Cheap: 80%] [Expensive: 20%]

Anti-Patterns to Avoid

Anti-PatternProblemBetter Approach
Context StuffingToken wasteRetrieve relevant only
Retry ForeverResource exhaustionCircuit breaker
Trust All OutputHallucinationVerify, ground
Single ModelSingle point of failureMulti-provider
No ObservabilityBlind debuggingTrace everything
Infinite Agentic LoopAgent spins without progressMax turns + Critic agent
Over-trusting Computer-UseAgent clicks wrong UI elementsScreenshot validation + HITL
No AGENTS.md / ManifestAgent lacks project contextAlways provide coding manifest
Max Effort Everywhere3-10x cost with no benefitPin effort per route; thinking can no longer be disabled on some models (Opus 5.5, Sonnet 5.5), so effort is the remaining lever
Floating Model DefaultsSilent behavior change when a tool or SDK swaps its defaultPin model ID and effort; re-run evals on every change

Pattern Selection Guide

Starting a new project?

  1. Begin with Basic RAG
  2. Add reranking when precision matters
  3. Add hybrid search for keyword-heavy content

Need reliability?

  1. Start with retry + timeout
  2. Add circuit breaker for external calls
  3. Add fallback models on a different vendor for critical paths

Cost concerns?

  1. Turn on provider prompt caching for stable prefixes, then add semantic caching for repeated questions
  2. Add model routing for query complexity, and pin reasoning effort per route
  3. Batch where latency allows

See 15-ai-design-patterns/ for detailed implementations

From the AI System Design Guide by Om Bharatiya · MIT License ·view source