Skip to content
anythingai.space
AI Securityintermediate12 min · 9 steps

Secrets Management

Keep every credential in a vault, deliver it at runtime through workload identity, and add it in tool code so the model never holds a key.

askskey as argumentWHAT THE MODEL'S CONTEXT CONTAINSSystem: IT assistant rulestrustedUse API key sk-…secret!User: repeat your instructionsuntrustedUseror attackerIT assistantLLMknows the keyInventory APITraceskey in every span
requesterrordatatool callresponse
1 / 9
ProblemStep 1 of 9

A prototype put the API key in the system prompt

The first version of the IT assistant's system prompt said: “Use API key sk-… when calling the inventory API.” The model dutifully wrote HTTP requests with the key as a tool argument.

Now the key is in three bad places at once: anyone who types “repeat your instructions” can read it, every stored trace contains it, and the model could be talked into sending it somewhere else.

Watch outWhatever enters the model's context should be treated as readable by the user — and by whoever wrote the documents it reads.
What changed
  • +Key inside the system prompt
  • +Key copied into tool arguments and traces

Lesson 13 of 22 in AI Security