Volume IX · Designing the Tools

Writing a Tool a Model Won't Misuse

Lecture 13 described function calling as a JSON contract — a schema, a stop reason, a result block passed back into the conversation. That lecture was about what makes the contract valid. This one is about what makes it good — because a schema can be perfectly valid JSON and still be a trap: an ambiguous parameter name, an error message the model can't act on, a response so verbose it burns the context window on noise. Anthropic's engineering write-up on this frames tools as "a fundamentally new software paradigm: contracts between deterministic systems and non-deterministic agents" — and a contract with an unreliable party needs to be written more carefully, not less.

Describe it like you're briefing a new hire

"Writing effective tools for AI agents—using AI agents" (anthropic.com/engineering/writing-tools-for-agents) gives a concrete heuristic for description quality: "think of how you would describe your tool to a new hire on your team." A new hire has none of the implicit context that lives in your head — they need the ambiguity resolved in words, not assumed. The same discipline applies to parameter names: user_id is unambiguous, user is not, and the model has no way to ask a clarifying question mid-call the way a new hire could ask you one at their desk. The post states this is not a cosmetic concern — precise refinements to tool descriptions were part of how Claude Sonnet 3.5 reached a state-of-the-art SWE-bench result. Description quality is measured to move outcomes, not just readability.

An error message either teaches or it wastes a turn

When a tool call fails, what comes back matters as much as whether it failed at all. A cryptic error code or a raw stack trace gives the model nothing to act on except a guess. The guidance here is to make error messages actionable — describe what went wrong in terms the model can use to correct itself, and where possible, show a corrected example input alongside the error rather than just the failure. An error message that teaches turns a failed call into a recoverable one; an error message that doesn't just burns a turn on a retry the model has no better information for.

Consolidate — three thin tools becoming one that thinks

The post argues against 1:1 wrapping of every API endpoint as its own tool, on the grounds that more tools is not more capability — it's more surface area for the model to choose wrong across. The worked example: collapsing separate list_users, list_events, and create_event tools into a single schedule_event tool that internally handles availability checking itself. The model no longer has to orchestrate three calls correctly in the right order and reason about their interaction — the tool absorbed that reasoning, and the model just states the intent.

Namespacing groups tools the way a filesystem groups files

When many tools exist across several services, the post recommends grouping them by a service or resource prefix — asana_search, jira_search — so the model can tell at a glance which system a tool belongs to before it has read the tool's full description. The authors note that the choice between prefix-based and suffix-based namespacing had a non-trivial effect on agent behavior in their own testing — a detail worth taking seriously precisely because it is the kind of decision that looks cosmetic and isn't.

Token efficiency: return only what's high signal

A tool response competes for the same context window as everything else the model is reasoning over, and the post is specific about the fix: implement pagination, filtering, range-selection, and truncation with sane defaults, so a tool doesn't dump an entire dataset back at the model when a handful of rows would answer the question. Return only "high signal information," and prefer semantic identifiers over raw UUIDs — a name or slug the model can reason about, not an opaque token it can only copy verbatim. The quantified example: a verbose Slack tool response cost 206 tokens, where a concise response covering the same information cost 72 tokens — roughly a third the size, repeated across every call a long-running agent makes.

Evaluating and refining tools the same way you'd eval a model

The post's evaluation methodology treats tool design as an empirical question, not a one-time judgment call. Build realistic multi-tool-call tasks modeled on real workflows — tasks that may require dozens of tool calls to complete — and run them through a simple agentic loop using direct API calls. Track accuracy, runtime, tool-call count, token consumption, and errors as the actual metrics. Then feed the resulting transcripts back into Claude itself and have it propose refactors to the tool definitions, validating any refactor against a held-out test set so the refinement doesn't just overfit to the tasks it was tuned on.

The disclosed result of running this loop: tool definitions that Claude itself refined this way outperformed the human-written originals on held-out Slack and Asana MCP test sets. The post does not disclose an exact percentage improvement in the fetched text, so this should be stated qualitatively — a real, measured improvement, without inventing a number the source doesn't give.