perf(mcp): answer-directly steering — ~35% cheaper, ~70% fewer tool calls (#224)
* perf(mcp): steer agents to answer directly instead of delegating to subagents CodeGraph beats native grep/read on cost only when the agent queries it directly. When the agent delegates to file-reading sub-agents, those sub-agents read files regardless of the index, so CodeGraph becomes net overhead on top of the reads. The install templates even told agents to "spawn a subagent for explore-class questions" — the expensive path. Changes: - server-instructions + both install templates: add an "Answer directly — don't delegate exploration" directive; reposition codegraph_explore as the efficient one-call multi-symbol tool (was: "spawn a subagent for it"). - codegraph_explore: hard-cap output to its adaptive budget (it overran, ~30k vs a 28k cap) and tighten the medium tier (28k->13k). - codegraph_node: return a member outline for container kinds instead of the full class body. Rigorous N>=4-per-arm warm-block benchmark (median total_cost_usd): excalidraw (~600 files): WITH $0.54 vs native $1.02 (-47%) vscode (~10k files): WITH $0.41 vs native $0.72 (-42%) ky (~25 files): WITH $0.46 vs native $0.44 (wash) Answers were equal-or-better (correct, file:line-cited) with ~6x fewer tool calls; the directive drove the direct path on 14/14 codegraph runs. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(readme): rebuild benchmark with real-world repos + cost/token/time/tool savings Replace the "Claude Code (Python+Rust/Java)" rows — which benchmarked the Claude Code CLI repo, not real codebases in those languages — with real open-source projects per language: Django (Python), Tokio (Rust), OkHttp (Java), Gin (Go), plus Alamofire (Swift) and the existing TypeScript repos (VS Code, Excalidraw). The table now reports all four savings the change targets — cost, tokens, time, tool calls — as the median of 4 runs per arm (Claude Opus 4.7, headless claude -p, with vs empty MCP config). Averages across the 7 repos: 35% cheaper, 59% fewer tokens, 49% faster, 70% fewer tool calls. Adds a methodology note and raw WITH->WITHOUT medians. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.7
parent
a47355780b
commit
f5bbc26c60
@@ -22,6 +22,18 @@ in the workspace. Reads are sub-millisecond; the index lags writes by
|
||||
about a second through the file watcher. Consult it BEFORE writing or
|
||||
editing code, not during.
|
||||
|
||||
## Answer directly — don't delegate exploration
|
||||
|
||||
For "how does X work", architecture, trace, or where-is-X questions,
|
||||
answer DIRECTLY using 2-3 codegraph calls: \`codegraph_context\` first,
|
||||
then ONE \`codegraph_explore\` for the source of the symbols it surfaces.
|
||||
Codegraph IS the pre-built search index — so delegating the lookup to a
|
||||
separate file-reading sub-task/agent, or running your own grep + read
|
||||
loop, repeats work codegraph already did and costs more for the same
|
||||
answer. Reach for raw Read/Grep only to confirm a specific detail
|
||||
codegraph didn't cover. A direct codegraph answer is typically a handful
|
||||
of calls; a grep/read exploration is dozens.
|
||||
|
||||
## Tool selection by intent
|
||||
|
||||
- **"What is the symbol named X?"** → \`codegraph_search\`
|
||||
@@ -30,7 +42,7 @@ editing code, not during.
|
||||
- **"What does this call?"** → \`codegraph_callees\`
|
||||
- **"What would changing this break?"** → \`codegraph_impact\`
|
||||
- **"Show me this symbol's source / signature / docstring."** → \`codegraph_node\`
|
||||
- **"Survey an unfamiliar topic / pattern / module."** → \`codegraph_explore\` (heavier; deep dive)
|
||||
- **"Show me several related symbols' source / survey an area."** → \`codegraph_explore\` (ONE capped call; prefer over many codegraph_node/Read)
|
||||
- **"What's in directory X?"** → \`codegraph_files\`
|
||||
- **"Is the index ready / what's its size?"** → \`codegraph_status\`
|
||||
|
||||
@@ -44,7 +56,7 @@ editing code, not during.
|
||||
|
||||
- **Don't grep first** when looking up a symbol by name — \`codegraph_search\` is faster and returns kind + location + signature.
|
||||
- **Don't chain \`codegraph_search\` + \`codegraph_node\`** when you just want context — \`codegraph_context\` is one round-trip.
|
||||
- **Don't use \`codegraph_explore\` for narrow questions** — it's a multi-call deep dive, expensive in tokens. Save it for genuine "I'm new here" surveys.
|
||||
- **Don't loop \`codegraph_node\` over many symbols** — one \`codegraph_explore\` call returns them all grouped by file, while each separate call re-reads the whole context and costs far more. Use \`codegraph_node\` for a single symbol.
|
||||
- **Don't query the index immediately after editing a file** — the watcher needs ~500ms to debounce + sync. Wait for the next turn.
|
||||
|
||||
## Limitations
|
||||
|
||||
Reference in New Issue
Block a user