Skip to content
anythingai.space
Memory & Stateintermediate12 min · 9 steps

Memory Architectures

Split an agent's memory into working, episodic and semantic tiers, decide when memories move between them, and prove each layer beats plain retrieval.

CONTEXT, EVERY TURNsystem ruleslast month's chatsold tool outputspasted docscurrent question1MINPUT COST PER TURN · OPUS 5.51M uncached$4.001M fully cached$0.2010K retrieved$0.04Plus: attention dilution and slowertime-to-first-token
tokendatarequest
1 / 9
ProblemStep 1 of 9

Why not keep everything in a 1M-token window?

Frontier models now read about 1M tokens, so it is tempting to paste every past chat, tool output and document into each call. It works, but it is wasteful on three counts.

Cost: a 1M-token call to Claude Opus 5.5 is $4.00 of input uncached and $0.20 even fully cached, every turn. A 10K-token call with retrieved context is $0.04. Attention: clutter degrades reasoning (lost in the middle, context rot). Latency: time to first token grows with uncached context.

Watch outCaching narrowed the gap but did not close it. Keep the window lean and retrieve what the current task needs.

Lesson 1 of 6 in Memory & State