Skip to content
anythingai.space
Module 25

Case Studies

Whole systems, end to end: enterprise RAG, coding agents, support automation, fraud, voice AI and more.

lessons
20
steps
159
time
267 min
ONLINE · QUERY PATHDATA LAYER · INGESTIONhit?Employees5,000 · 500 peakAPI gatewayauth · rate limitRBACQuery serviceorchestration4 replicasSonnet 5.5private endpointAnswer cacheexact + semantic30% hitsACL metadataPostgres · RBACHybrid retrieverRRF + ACL filterRerankertop 50 → 10Audit logevery queryQdrant4 shards × 2HAStreaming ingestfresh < 1 hElasticsearch3-node cluster
01advanced·14 min·8 steps

Enterprise RAG System

Design cited, permission-aware document Q&A over 500K files for 5,000 employees, one bottleneck at a time.

  • Turn a vague ask into latency, accuracy, freshness and security targets
  • Why vector-only retrieval misses, and how hybrid search plus reranking fixes it
  • Enforce permissions inside retrieval, not after it
  • Meet the 1-hour freshness and 500-user targets with streaming ingestion, replicas and caches
Start →
CONTEXT SOURCES · PARALLELdraft< 0.75%Customerweb / app chatGatewayauth + tenantIntent classifierGPT-6 LunaQuery routersources · toolsMemoryRedis · 10 turnsKnowledge basedocs·FAQs·ticketsAccount contextplan · historyAction toolsticket · workflowGeneratorHaiku / Sonnet 5.5Gatessafety · conf ≥ 0.7Human agentcontext bundleQuality judge5% sampled
02advanced·14 min·8 steps

Conversational Support Agent

Automate 60% of 500K monthly B2B support tickets with intent routing, model routing and confidence-gated escalation.

  • Translate support pain into latency, accuracy, escalation and CSAT targets
  • Route by intent so each turn pulls only the sources and tools it needs
  • Use model routing plus effort, and budget latency per path
  • Gate every reply on safety and confidence, with context-rich escalation
Start →
STAGES 1–2STAGE 3 · DEBATESTAGE 4 · GATEscorespassfailSource docs10-K/Q · callsExtractorGemini Flash · k=5Verified metricsstructured JSONMoA analysisOpus 5.5 ×3 + aggClaim extractor~150 claimsDebater AClaude Opus 5.5Debater BGPT-6.1 SolDebater CGemini 3.8 FlashConsensusall agree?Panel + gates3 judges · ≥ 4.0Auto-publishall gates passHuman reviewdisputed claims
03advanced·14 min·8 steps

Financial Analysis with Ensembles

Generate equity research reports in 25 minutes at 0.4% factual error by stacking self-consistency, mixture of agents, debate and a judge panel.

  • Why zero-tolerance financial data rules out a single model pass
  • Extract numbers with unanimous k5 self-consistency
  • Verify every claim with a cross-family debate and consensus
  • Route reports with quality gates: auto-publish vs human review
Start →
FAST TIER · < 200MSQUALITY TIER · 1–5Scompletegeneratecontexthit?IDE extensionVS Code·JetBrainsGatewaydebounce·auth·flagsContext assembler4K-token budgetCompletion svcfast · < 200msp50 145msFIM model 8Bself-hosted in VPCCompletion cacheLRU + Redis · 1hGeneration agentmulti-file refactorsClaude Sonnet 5.5$2 / $10 per 1MCode verifierfail-fast gauntlet
04advanced·14 min·8 steps

AI Code Assistant

Design an IDE assistant that completes code in under 200ms and runs multi-file refactors — by splitting it into a fast tier and a quality tier.

  • Why one model can't serve both keystroke completions and refactors
  • Split the system into a fast FIM tier and a quality agent tier
  • Fill a 4K-token context budget by priority
  • Verify generated code fail-fast before a developer sees it
Start →
FEEDBACK · OFFLINE95%10%1%BEFORE → AFTER NORMALIZATIONh@te→hatek​ill→killIngestion10M posts/dayNormalize + OCRhomoglyphs · unicodeTier 1 · filtershash · keyword · regexBlock + report5% of postsTier 2 · classifiersself-hosted · < 100msAuto action85% + 9% of postsTier 3 · LLM reviewGemini 3.8 Flash · < 3sTier 4 · humans500 moderatorsTraining labelshuman decisions
05advanced·14 min·8 steps

Content Moderation at Scale

Moderate 10M posts a day with a four-tier cascade — filters, classifiers, an LLM and humans — so each tier only sees what the cheaper one couldn't decide.

  • Why neither all-human nor all-LLM review scales
  • Build a cascade where each tier escalates only what it can't decide
  • Prioritise human review by severity, reach and confidence
  • Defend the cheap tiers against evasion and read provenance
Start →
INGESTIONDUAL INDEXQUERY + ANSWERsemantickeywordfusedsourced answerLive feedsnews·SEC·socialKafkareplayable streamStream procembed · upsert by IDVector DBQdrantFull-text indexElasticsearchAnalyst10K concurrentQuery routerhybrid · RRF fusionCross-encoderrerank top resultsGPT-6 Luna$0.10 / $0.50 per 1M
06intermediate·12 min·8 steps

Real-Time AI Search

Answer “what's the sentiment on Tesla in the last hour?” with sourced facts under 3 seconds — streaming ingestion, a dual index and a small, fast synthesiser.

  • Why batch-indexed, vector-only RAG fails live market questions
  • Stream into two indexes with Kafka consumer groups
  • Combine semantic and keyword search with RRF and a reranker
  • Enforce freshness as an index filter and survive news spikes
Start →
PLANNINGCODEBASE INDEXEXECUTION LOOP · SANDBOXEDREVIEWplandiffdiff + hashokallowTask“add rate limiting”PlannerOpus 5.5 · highCoderSonnet 5.5E2B sandboxresets each runFile summariesvoyage-code-4Symbol graphtree-sitter ASTDebuggerreads failuresmax 3 tries$0.50 capTests + Semgrepmust passHuman reviewapproves diff hashPull requestAction policydestinations
07advanced·14 min·8 steps

Autonomous Coding Agent

Design an agent that turns “add rate limiting to every endpoint” into a tested, human-approved pull request for under $0.50.

  • Why planning and coding run as separate agents on different models
  • Tiered retrieval: summaries → symbols → full files for a 1,000-file repo
  • A self-correction loop with hard limits, inside a sandbox
  • Why cache hit rate, not token price, sets the cost per task
Start →
INGESTIONTENANT DATAdocsvectorsUploadtag tenant_idIngest pipelineparse · chunk · embedBlob storageAES-256Tenant usersCoke · PepsiAPI gatewayJWT → tenant_idTenant routerORM adds filterShared Qdrantnamespace + RLSDedicated pod> 500K docsLLMtenant contextOutput filterdoc IDs ∈ tenantAudit logimmutable
08advanced·13 min·8 steps

Multi-Tenant AI SaaS

Serve 500 competing enterprises from shared infrastructure with zero cross-tenant leakage, using tiered isolation and defense in depth.

  • Why a single tenant filter is a single point of failure
  • Tiered isolation: namespace, dedicated collection, dedicated pod
  • The seven layers of defense in depth, from JWT to provider account
  • Tag tenant_id at upload and design for deletion and audit
Start →
AUTOHYBRIDESCALATEeditreplyCustomer12 languagesClassifierGPT-6 LunaOrder APIFAQ searchAuto pathLLM + toolsSafety5 checksHybridAI draftsReviewapprove / editSendvia ZendeskHuman agentfull contextsummarytranscriptReasonswhy escalated
09intermediate·13 min·8 steps

Customer Support Automation

Auto-resolve 60% of 2 million tickets a month with tool-grounded answers, and hand the risky ones to humans with full context.

  • Why one chatbot answering everything makes false promises
  • Three-tier routing on a multi-signal confidence score
  • Tool-grounded answers plus a safety check before send
  • Human takeover with a handoff package and logged reasons
Start →
PARSINGEXTRACTORSJSON< 0.8Contract PDF2–200 pagesPyMuPDFnative PDFsParsing VLMscanned pagesFrontier VLMhard pages ~15%Sectionsskip exhibitsPartiesDatesObligationsTerminationValidatorcross-fieldDatabaseJSONReviewfields < 0.8
10intermediate·13 min·8 steps

Document Intelligence Pipeline

Extract parties, dates, obligations and termination terms from 50,000 contracts a month at 95%+ accuracy for under $0.50 each.

  • Why OCR plus one big prompt fails on real contracts
  • Route pages by difficulty: parser, parsing VLM, frontier VLM
  • Section detection and parallel, specialized extractors
  • Cross-field validation and field-level human review
Start →
OFFLINE · DAILYONLINE · PER REQUESTpre-warmrequestnew userseed prefsANN top 100candidatesmissstoreWatch history+ content catalogUser embeddingsmatrix factorizationANN indexitem embeddingsBatch pre-genBatch tier, 50% offUser5M DAURec serviceembedding lookupCross-encodertop 100 → top 10Explanation cachesource:target:reasonCold-start routersurvey/hybrid/collabFact checkvs catalog factsGPT-6 Lunacache misses only
11intermediate·12 min·8 steps

Recommendation Engine with Explanations

Rank 50M users' movies with classic ML in under 200 ms, and spend the LLM only on cached, fact-checked explanations.

  • Why an LLM should explain recommendations, not rank them
  • The classic retrieve-then-rerank pipeline behind real-time recs
  • Why the p95 budget turns into a cache-hit-rate target
  • Cold start, hallucinated facts and privacy-safe caching
Start →
INTAKEREGULATION KBANALYSIS · REVIEW · AUDITeach claimpassagesconfidenceflagsre-indexMarketing asset500 per monthClaim extractortext + vision LLMPrecedent DBwarning lettersRegulationsCFR 21 + policyClaude Sonnet 5.5per-claim verdictRetrieverregs + precedentReg WatchFDA feedsLegal reviewsenior counselSeverity triageHIGH / MED / LOWAudit trailevery decision logged
12advanced·12 min·8 steps

Regulatory Compliance Automation

Pre-screen pharma marketing against FDA rules claim by claim, with citations, recall-first thresholds and a full audit trail.

  • Why claims, not whole documents, are the unit of review
  • Why regulations belong in RAG, backed by a precedent index
  • How recall-first thresholds turn AI into triage for lawyers
  • What an audit-ready decision record contains
Start →
LIVE · PARTIALS < 500 MSAFTER THE ENCOUNTER · < 60 SREVIEW + EHRfull transcriptentitiesdraftNurse devicedirectional micVAD gatesilence never sentStreaming ASRmedical, under a BAALive transcriptpartials < 500 msDiarizationnurse vs patientMedical NERfine-tuned BioBERTNote structurerLLM, SOAP as JSONGrounding checkevery line → spanNurse reviewapprove or editEHR via FHIREpic / Oracle HealthSigned diffquality signal
13advanced·13 min·7 steps

Voice AI Scribe for Healthcare

Turn a nurse's spoken encounter into a grounded SOAP note in the EHR, with a live transcript and the LLM kept off the real-time path.

  • Why silence is a hazard for speech models
  • Why HIPAA means a BAA, not necessarily on-prem ASR
  • Splitting the latency budget: live transcript vs note after the visit
  • Grounding, human review and the draft-to-signed diff
Start →
PRE-COMPUTED + LEARNINGREAL-TIME DECISION · < 100 MSEXPLAIN + ADAPTprofilescore0.3–0.7> 0.7 reject, asyncdecisionsnew modelBatch pipelinedaily / hourlyProfile storeusers, merchantsWeekly retrainfraud evolvesFeedback DB7-year audit trailTransaction10M per dayFeature servicevelocity, device, geoML ensembleXGBoost + NN10 ms3-way decisionby fraud scoreFactor checkelse templatedGPT-6 Lunaasync explanationDrift monitorpreds vs reportsBusiness rulesvelocity, geo5 ms
14advanced·12 min·8 steps

Real-Time Fraud Detection

Score 10M card transactions a day in under 100 ms with ML, decide with explainable rules, and explain declines with an LLM off the hot path.

  • Why the 100 ms budget forces pre-computed features
  • ML for scoring, rules for explainable decisions, and the gray zone
  • Using an LLM to explain declines without touching the hot path
  • Keeping up with fraud that changes every week
Start →
OFFLINE · INGESTIONONLINE · PER QUERYfilter by groupstop-kanswer + sources15 sources2M docsIncremental syncdelta · watermark · hashChunk + embed+ ACL tagsVector DBACL · date tagsConsultantassociatePermissionsuser → groupsRecency reranksemantic + decayLLMClaude Sonnet 5.5Gap checkrelevance < 0.5admit gapsflag conflicts
15intermediate·12 min·8 steps

Enterprise Knowledge Management

A knowledge assistant for 10,000 consultants that answers only from documents each person may see, prefers current guidance, and admits when it knows nothing.

  • Turn a vague ask into concrete constraints: scale, ACLs, citations, staleness, gaps
  • Enforce permissions at retrieval, before the LLM sees anything
  • Rank current guidance above old with a recency decay
  • Detect knowledge gaps and conflicts instead of hallucinating
Start →
INTAKESHARED BROWSER SANDBOXtask specscreenshotsWHICH NUMBER TO BELIEVEOSWorld-Verified83–86%v2 partial credit77.67%v2 binary44.33%OSWorld 2.0 leaderboard, Claude Opus 5(max effort). Partial and binary can sit30–40 points apart.Submitterreceipt + memoSchedulertask queueBrowserPlaywright · ChromiumComputer-use modelClaude Sonnet 5.5
16advanced·14 min·8 steps

Production Computer-Use Agent

A computer-use agent closes 14,000 expense reports a week across three API-less tools, made safe by per-task microVMs, a risk-tiered action gate and a tamper-evident audit trail.

  • Read computer-use benchmarks correctly: binary completion, not partial credit
  • Gate every proposed action by risk: execute, confirm inline, or review out of flow
  • Isolate each task in its own microVM with an audience-bound identity
  • Defend against indirect prompt injection at the read layer
Start →
TRAINING PLANESERVING PLANEpre-warmONE 80 GB H100 WITH THE 25-ADAPTER CAP70B base, 4-bit25KV cacheotherTenant dataS3 · per-tenant KMSLoRA trainingr=16 · Q, K, V, OAdapter registry120 MB · SHA-256280 tenantstenant JWTGatewayauth · token bucketAdapter routeraffinity · LRUvLLM multi-LoRA≤ 25 adapters per GPU30 pinnedpre-warm top 50
17advanced·14 min·8 steps

Multi-Tenant Fine-Tuning Platform

Serve 280 customers their own fine-tuned model from one shared 70B base plus per-tenant LoRA adapters, with an eval gate per tenant and no cross-tenant bleed.

  • Why one full fine-tune per tenant breaks the unit economics
  • Size the LoRA vs full fine-tuning trade-off: cost, artifact size, accuracy gap
  • Budget GPU memory for adapters and stop noisy neighbors
  • Gate every retrain on the tenant's own golden set and isolate tenants end to end
Start →
STAGE 1 · CHEAPSTAGE 2 · GOLDEN SETSTAGE 3DECISIONper-axis scoresyesnoaxis diffDeveloper PR~50 AI PRs / weekLint + unitstage 1 · ~2 minGolden-set run10–25% stratifiedEval cachemodel·effort·epochCode evaluatorsschema · regexLLM judge6 failure axesJudge correctionjudgy · 95% CIDelta vs mainper axis, CI lowWithin bounds?≤ 2 pts below mainMerge to mainBlocked + reportper-axis diff
18advanced·14 min·8 steps

Eval-Gated CI/CD for an AI Product

Make every PR earn its merge button: golden sets, calibrated LLM judges with confidence intervals, and per-axis gates under a $40-per-PR budget.

  • Why post-merge regression hunting failed and what the gate must satisfy
  • Bound eval cost with stratified sampling, caching and nightly full runs
  • Correct a biased LLM judge into an estimate with a confidence interval
  • Gate per failure axis against main, not on one composite score
Start →
ONLINE · SERVINGTRAIN + GATELABELCOLLECT90%10%tracesbatch labelUsers~8M req / monthRouter90/10 splitStudent 8Bint4 · vLLM90%Eval suite1,800 golden casesTeacherfrontier · batch10%Human spot-check5% · reject disagreeFine-tune8×H100 · ~1 weekTrace store6 months of tracesPII redactionNER → [EMAIL]Stratified sampler~800K pairs
19advanced·14 min·8 steps

Customer-Specific Distillation Pipeline

Cut a $50K/month frontier bill to about $9.3K by distilling an 8B student on six months of production traces — with a payback under three months.

  • Decide whether a workload is worth distilling at all
  • Why real production traces and human-checked labels beat synthetic data
  • Gate a student with a golden set, shadow traffic and a canary ramp
  • Do the payback math: $96K upfront against ~$41K/month saved
Start →
PRE-TOOLPOST-TOOLMCP SERVER POOLtool calltaggedEmployeeIDE pluginGatewayEnvoy + OPA + OktaKnowledge agentClaude Sonnet 5.5Tool-arg filterscope · syntax · inj.Credential brokeraud-bound JWT · VaultOutput checktrust-tag spansSnowflakeHTTP · RSConfluenceHTTP · RSinjected pageJiraHTTP · RSSlackvendor remoteLegacy toolsSTDIO · sandbox
20advanced·15 min·8 steps

Enterprise MCP Knowledge Agent

An agent that answers across Snowflake, Confluence, Jira and Slack through MCP — with audience-bound tokens, sandboxed STDIO, trust-tagged results and an audit chain.

  • Why query sources live over MCP instead of crawling one vector index
  • Bind every token to one server's audience and keep third-party tokens in a broker
  • Layer defenses against indirect prompt injection: filter, trust-tag, gate
  • Run it as a system of record: audit chain, per-identity limits, cost per query
Start →
Read deeper

Guide chapters · Case Studies