Skip to content
anythingai.space
Module 09

Memory & State

Short- and long-term memory, agentic memory, semantic caching and state management.

lessons
6
steps
53
time
69 min
CONTEXT, EVERY TURNsystem ruleslast month's chatsold tool outputspasted docscurrent question1MINPUT COST PER TURN · OPUS 5.51M uncached$4.001M fully cached$0.2010K retrieved$0.04Plus: attention dilution and slowertime-to-first-token
01intermediate·12 min·9 steps

Memory Architectures

Split an agent's memory into working, episodic and semantic tiers, decide when memories move between them, and prove each layer beats plain retrieval.

  • Why a 1M-token window is not a memory strategy
  • The three tiers: working, episodic and semantic memory
  • When memory belongs in files instead of a vector index
  • Write-time, sleep-time and read-time consolidation
Start →
APPLICATION WINDOWsystem rulestool schemasturns 1–20tool resultsnew turnreply buffer64K app limitModel window: the hard limit (e.g. 1.05Mon GPT-6.1 Sol)Application window: your budget forlatency and cost (e.g. 64K)Buffer zone: room for the reply, thinkingtokens included
02intermediate·11 min·8 steps

Short-Term Context Management

Keep the context window lean, cacheable and append-only: paged KV memory, prefix caching, sliding windows, summaries and selective dropping.

  • Model window vs application window, and why you leave a buffer
  • How paging cut KV-cache waste from 60–80% to under 4%
  • Order prompts static-first so the prefix cache hits
  • Sliding window, summary, hybrid and provider compaction
Start →
BEFORE CONSOLIDATIONbill 1bill 2bill 3bill 4bill 5bill 6bill 7bill 8bill 9bill 10AFTER CONSOLIDATIONbilling history · 1 summary nodeFORGET USER_42 (GDPR)episodesdeletedfactsdeletedderived summariesdeletedbackupsdeletedgit historypurged
03intermediate·12 min·9 steps

Long-Term Memory

Persist what an agent learns across sessions: episodic logs, extracted facts, vector-graph hybrids, decay and forgetting, and hard tenant isolation.

  • Three shapes of long-term memory service
  • Episodic trajectories vs extracted semantic facts
  • When a real graph earns its cost over entity-aware vectors
  • Decay, consolidation and explicit forgetting
Start →
WRITE · ADD()READ · SEARCH()factscandidatesfused scoreTurnadd(messages)Extract1 LLM call per add()Hash dedupVector store+ entity collectionNew queryfilter: user_idHybrid retrievalsemantic + BM25 + entityPromptfacts above threshold
04intermediate·11 min·9 steps

Agentic Memory with Mem0

Turn conversations into a self-updating user profile: Mem0's extract-store-retrieve loop, where conflict handling went in v2, and how to pick a memory layer.

  • Why Mem0 stores insights instead of transcripts
  • The v2 write path: one extraction call, ADD-only, hash dedup
  • Where contradictions get resolved now
  • Wiring Mem0 into a LangGraph agent
Start →
neverinvalidateinvalidateWHAT GOES INTO THE CACHE KEY AND VERSIONtenant_idpermission setmodel IDprompt versiondata versiondynamic TTLTenant ATenant BCache · tenant ACache · tenant BSource datav42 → v43changed
05intermediate·11 min·9 steps

Semantic Caching

Reuse whole answers for equivalent questions: the embed-search-threshold pipeline, false hits, scoping, and when a provider prompt cache already wins.

  • Why exact-match caches miss most repeated questions
  • The pipeline: embed, search, threshold, verify, update
  • Semantic drift: how a loose threshold returns wrong answers
  • Semantic cache vs provider prompt cache
Start →
SESSION CHECKPOINTSOriginal (audit)c1c2c3bad turnc5Fork at c3c3 editedfresh callfresh calldonecheckpoints →Edited prefix + old thinking blocks → HTTP400Fresh calls from the edit point →valid
06intermediate·12 min·9 steps

State Management Patterns

Give an agent a typed, append-only state, run it as a graph, checkpoint every step, fork and join with reducers, and choose who holds the state.

  • Why state should be typed and append-only
  • Checkpoint, crash, resume — and the side-effect trap
  • Fork/join with reducers instead of overwrites
  • Time travel is a fork, not an edit
Start →
Read deeper

Guide chapters · Memory & State

Next module · 10
Frameworks & Tools