Skip to content
anythingai.space
Module 23

Tool Use & Computer Agents

Tool-use architectures, computer-use agents, building tool agents safely.

lessons
7
steps
60
time
92 min
CLAUDE CODE INPUT : OUTPUT TOKENSearlier189:1Sep 2026324:1ENTERPRISE APPS WITH AI AGENTS (GARTNER)early 2025< 5%end 2026 (proj.)40%MCP: 10,000+ activeserversLocalCloudIDEVision/GUIManaged
01beginner·11 min·8 steps

The Tool-Use Landscape

Five kinds of agents that act instead of just talking: where each runs, what it can touch, and who carries the risk.

  • Tell the five deployment categories apart
  • Know who carries the security risk in local vs cloud agents
  • Why managed, persistent agents change the design questions
  • Compare latency and risk across agent types
Start →
COST PER TASK (ILLUSTRATIVE, FROM THE GUIDE)frontier does everything$1.00plan-and-execute$0.13 · −87%Orchestratorfrontier modelCode agentbash · edit · gitData agentSQL · plot · csvWeb agentfetch · browseSandboxesone per agent
02intermediate·14 min·9 steps

Tool-Use Architecture Patterns

Four ways an agent touches the world (tool calls, vision, code execution, multi-agent), plus the sandbox, state and retry decisions around them.

  • The three steps of function calling and their failure points
  • Vision-based automation and code execution with a permission gate
  • Why plan-and-execute cuts multi-agent cost
  • Choose a sandbox and an error strategy
Start →
GATEWAY · LOCALHOST:18789Messaging20+ platformsChannel adaptersRoutersessions · bindingsAgent runtimethe AI loopTool layerskills · browser · cronMemory & state~/.openclawLLM providersClaude · GPT · local
03intermediate·13 min·8 steps

OpenClaw Deep Dive

Inside the self-hosted, messaging-native personal agent: gateway, runtime, skills, model routing, and the security and vendor risks it brings.

  • What OpenClaw adds to a stateless LLM
  • Trace a message through gateway, router and runtime
  • How skills are resolved and selectively injected
  • The four permission layers, and why sandboxing is off by default
Start →
4 actionsONE BATCHclick [ref_2] name field1 ✓type “customer C-1042”2 ✓click [ref_7] Submit3 ✗ modal openedscreenshot4 haltedStop at the first failure, take a freshscreenshot, let the model re-planVision modelone turnAgent runtimeruns in orderBrowseraccessibility tree
04intermediate·13 min·9 steps

Computer-Use Agents

Let a model drive a GUI through screenshots and clicks: the loop, the sandbox, what each step costs, and how it fails.

  • Walk through the screenshot → reason → act loop
  • Why batch actions and element refs change the loop
  • Budget latency and cost per action
  • Handle misclicks, loops and partial batches
Start →
JSON-RPCHTTPClaude CodeCustom agentIDE pluginLoad balancerstateless OKMCP serveryour codetoolsresourcesprompts
05intermediate·13 min·8 steps

Building Tool Agents

Design tool schemas a model can't misuse, serve them over MCP, validate every call, and test the agent like the production system it is.

  • Write tool schemas that prevent misuse
  • Expose tools once through an MCP server
  • Handle hundreds of tools with dynamic discovery
  • Choose between LLM-orchestrated, programmatic and server-side chaining
Start →
bad codesFILING ACCURACYbefore94%after99.2%Trade Monitorread-onlyClassifierjurisdictionDoc Assemblerread / writeReportingwrite, gatedSchema checkwas missingCompliance officersign-off
06beginner·13 min·9 steps

Use Cases and Case Studies

Where tool-using agents pay off, three real deployments with their numbers and failures, and how to reason about ROI.

  • Recognize the six use-case categories and what makes each work
  • Learn from three deployments' wins and failures
  • Build an ROI picture where human oversight is a real cost
  • Know the classic failure cases and their root causes
Start →
weakerstronger isolation →Also inside the boundary: default-deny egress (DNS too),CPU/memory/time limits, an out-of-band stopDockershared kerneltrustedgVisoruser-space kernelsemi-trustedWASMcapability sandboxpure computeFirecrackerown guest kerneluntrusted
07advanced·15 min·9 steps

Agent Safety and Governance

Contain agents that act in the world: injection through tool outputs, sandboxes, least-privilege permissions, approval gates, kill switches and defense in depth.

  • Why agent safety is about actions, not words
  • How indirect injection turns data into commands
  • Scope permissions and pick a sandbox
  • Design an approval gate that actually works
Start →
Read deeper

Guide chapters · Tool Use & Computer Agents

Next module · 24
Voice & Multimodal