feat(mcp): multi-module Go trace-quality + small-repo retrieval tuning (#494)
* feat(go): generated-file down-rank + gRPC stub-impl bridge + trace-failure inlining
Multi-pronged fix to make codegraph competitive on Go multi-module repos
(cosmos-sdk, etcd) where it previously lost or tied. Driven by an 8-question
agent-eval audit across cobra, gin, prometheus, cosmos-sdk, and etcd: the
baseline had codegraph losing ~60% on cost on cosmos-sdk and mixed on etcd
deep cross-module flows, while winning cleanly on the single-module and
non-protobuf-heavy repos.
Diagnostics ruled OUT `go.work` parsing as the gap (prometheus crushes
without it). The actual failure modes were generated-file noise warping
disambiguation, missing gRPC interface→impl bridge in structural-typing Go,
and trace's failure path triggering 3-5 follow-up tool calls instead of
inlining the material the agent needed.
Changes:
- New `src/extraction/generated-detection.ts` — path-pattern classifier
for `.pb.go`, `.pulsar.go`, `_grpc.pb.go`, `_mock.go`, `_mocks.go`,
`mock_*.go`, `.generated.[jt]sx?`, `_pb2(_grpc)?.py`, `.pb.{cc,h}`,
`.g.dart`, `.freezed.dart`. Applied as a stable sort tiebreaker in
`findSymbol`, `findAllSymbols`, `codegraph_search` (MCP + CLI),
`codegraph_explore` file ranking, and context formatter Entry Points /
Related Symbols / Code blocks. Cosmos's `msgServer.Send` now ranks #3
instead of #9 on a `Send` search.
- New `goGrpcStubImplEdges` synthesizer in `callback-synthesizer.ts` —
detects `UnimplementedXxxServer` structs in generated files, identifies
their RPC methods (excluding `mustEmbed*` / `testEmbeddedByValue` gRPC
markers), and emits `calls` edges to the matching methods on any
non-generated struct whose method-name set is a superset. Closes Go's
structural-typing gap that the existing `interfaceOverrideEdges` (Java /
Kotlin only) couldn't bridge. 467 bridge edges on cosmos-sdk; bank's
`UnimplementedMsgServer::Send` points to `x/bank/keeper/msg_server.go`
only, not to `msgClient` siblings or mock files.
- Trace-failure rewrite (`handleTrace`) — when no static path connects
endpoints, instead of telling the agent to call `codegraph_node` (a
3-4-call fan-out), inline both endpoints' bodies (120 lines / 3600 chars
per endpoint), their callers (≤6), and callees (≤8) in one response.
- Trace endpoint-pairing improvements — scores every `from`×`to`
candidate combo by shared directory prefix and tries the best-paired
pair first (the full candidate set, not just FTS top-5). A
less-canonical-path penalty (`enterprise/`, `contrib/`, `examples/`,
`vendor/`, `third_party/`, `deprecated/`, `legacy/`) ensures the
canonical-module pair wins even when a side-experiment shares more of
its directory prefix. Find-path probe budget capped at 20 pairs.
- Test-file deprioritization in `codegraph_explore` `isLowValue` — adds
suffix patterns (`_test.go`, `_spec.rb`, `.test.ts`, `.spec.tsx`,
`Test.java`, `Spec.kt`) alongside the existing directory-style patterns.
Otherwise etcd's `watchable_store_test.go` consumes 5K chars of explore
budget that should go to the hand-written flow source.
Tests:
- New `__tests__/generated-detection.test.ts` (4 unit tests) pins the
suffix patterns.
- New "Go gRPC stub→impl synthesis" integration test suite in
`frameworks-integration.test.ts` (2 tests): positive bridge from stub
to hand-written impl, AND the precision case (don't bridge to a
generated sibling like `msgClient` in the same .pb.go).
- Full suite: 1076/1076 pass.
Empirical (post-fix, n=2 average per question):
| Repo / Q | WITH | WITHOUT | Reads (W/WO) | Time (W/WO)
|-------------------------|------------|-------------|--------------|------------
| cobra (parse cmds) | $0.27 | $0.27 | 0 / 4 | 39s / 60s
| prometheus (scrape→TSDB)| $0.63 | $0.70 | 0 / 6 | 106s/143s
| cosmos-sdk Q1 (MsgSend) | $0.41 | $0.26 | 1 / 2 | 67s / 64s
| cosmos-sdk Q2 (Delegate)| $0.47 | $0.46 | 0 / 5 | 50s / 73s
| cosmos-sdk Q3 (gov tally)| $0.34 | $0.31 | 1.5 / 3 | 54s / 76s
| etcd Q1 (Put→raft) | $0.65 | $0.78 | 0 / 4 | 98s / 129s
| etcd Q2 (watch) | $0.36 | $0.50 | 0 / 4+ | 58s / 89s
Codegraph wins on reads + time on every question. Cost is mixed: 3 clean
wins, 3 tied (within 10%), 1 stubborn cost loss on the grep-favored Q1.
Compared to baseline, the cosmos-sdk cost-gap collapsed from -60% to -15%
on average, and Q3 went from a 75% loss to a tie. Raw run artifacts in
`/tmp/cg-finalv2-*/` and `/tmp/cg-final-*/`.
Memory written at `project_go_multi_module_audit.md` for the methodology
+ before/after numbers.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(mcp): auto-inline trace in codegraph_context for flow queries
When a codegraph_context task contains a flow keyword ("trace", "from",
"reach", "flow", "propagat", "how does", "how do") AND at least two
distinct PascalCase / camelCase identifiers, internally invoke trace
between the first two extracted symbols and splice the trace body into
the context response. Conservative trigger by design: false positives
waste one graph query; false negatives just fall back to the agent
calling trace itself (existing path-proximity wiring handles either
case).
Goal: collapse the agent's typical context → trace → explore sequence
into a single context call for clear flow queries, closing the
remaining cost-overhead gap on multi-call patterns. The path-proximity
+ less-canonical-path scoring + the trace-failure-inlined-bodies
behavior already let the inline trace land on the right endpoint pair
and return enough material that no follow-up codegraph_node/Read is
needed.
Doesn't fire on:
- cobra's "How does cobra parse commands and flags?" (no PascalCase
symbols) — verified in regression run, no behavior change ($0.260
WITH vs $0.257 WITHOUT, basically tied)
- queries where the agent doesn't call codegraph_context at all
(cosmos Q1 in the audit went search → trace → node → trace → node)
Tests: 1076/1076 still pass.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(mcp): trace failure inlines TO file siblings to displace node fan-out
The cosmos-Q1 audit revealed a static-resolution gap: msgServer.Send's
*real* next hop is `k.Keeper.SendCoins` — an interface-method call on an
embedded field that tree-sitter can't resolve. The static getCallees list
for msgServer.Send is all utility/error functions (StringToBytes, Wrapf,
…). The actual flow (SendCoins → subUnlockedCoins → addCoins →
setBalance) lives entirely inside `x/bank/keeper/send.go`, which is also
where the TO endpoint (setBalance) lives.
When trace fails (no static path), inline the **top 5 functions/methods
in the destination file**, ordered by line-distance from the TO node.
This catches the flow that interface-method calls obscure — the
canonical "k.<Iface>.<Method>" pattern in Go, also relevant to Java
dependency-injection / Rails service-object dispatch / etc. where
interface dispatch hides the real call.
Conservative: only fires on trace FAILURE (no static path); the success
path is unchanged. Per-body cap (40 lines / 1200 chars), top 5 siblings.
Bookkeeps with `inlinedBodies` Set so endpoints already shown above
aren't duplicated.
Result: cosmos-Q1 — historically the most stubborn cost loss (-2.2× to
-39% across the audit) — flipped to a clean WIN: $0.257 WITH vs $0.449
WITHOUT (-43%), 34s vs 79s, 0 Reads vs 2 Reads + 5 Greps, 5 codegraph
calls vs 12. Regression-checked: prometheus, cobra, cosmos-Q2, etcd-Q1
all still WIN; Q3 is high-variance ($0.30-$0.45 range historically) and
fell within that on this run.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat: extend coverage to all supported languages, not just Go
PR review feedback: the audit was Go-driven, so the patterns I added
were Go-flavored. Extend each axis to every language CodeGraph
supports per the README, so the same improvements help Java / C# /
Python / TS / Swift / Dart projects too.
**generated-detection.ts** — Added patterns for:
- TS/JS: `.gen.[jt]sx?`, `.pb.[jt]s`, `_pb.[jt]s`, `_grpc_pb.[jt]s`
(ts-proto, gRPC-web, Apollo / GraphQL codegen, Hasura).
- Python: `_pb2.pyi` (mypy stubs from protobuf).
- C#: `.g.cs` (T4 / Razor codegen), `Grpc.cs` (protoc-gen-csharp).
- Java: `OuterClass.java` (protoc-gen-java), `Grpc.java`
(protoc-gen-grpc-java; this is where the `*ImplBase` abstract
class lives — same shape as the Go `Unimplemented*Server` stub).
- Swift: `.pb.swift` (protoc-gen-swift).
- Dart: `.pb.dart`, `.pbgrpc.dart`, `.chopper.dart`.
- Rust: `.generated.rs`.
**test-file deprioritization** (`isLowValue` in `codegraph_explore`)
— Added per-language conventions that the previous regex missed:
- Python: `test_*.py` (pytest discovery) and `*_test.py`.
- Ruby: `*_test.rb` (minitest) — `*_spec.rb` already covered.
- C#: `*Tests.cs`, `*Test.cs`, `*Spec.cs`.
- Swift: `*Tests.swift` (XCTest).
- Dart: `*_test.dart`.
**IFACE_OVERRIDE_LANGS** in `callback-synthesizer.ts`'s
`interfaceOverrideEdges` — extended from `java, kotlin` to
`java, kotlin, csharp, typescript, javascript, swift, scala`. Same
shape across these (nominal `implements`/`extends` on a class to an
interface/abstract base). Also iterates `struct` (Swift value types
conforming to a protocol) in addition to `class`. The existing
matchesSymbol-style logic and `getOutgoingEdges(..., ['implements',
'extends'])` work unchanged.
**CLAUDE.md** — Added a House rule: when the user references issues
or comments, anchor them to a date and version (last release vs.
last main commit vs. current branch tip) BEFORE concluding a fix is
incomplete. Issue #388 comments from May 25-27 were responding to
the released v0.9.5 / merged-PR-469 state — not to this branch's
in-flight work. The new rule walks through the disambiguation:
`grep -m1 '^## \[' CHANGELOG.md` for release version, `git log
--first-parent main -1` for main tip.
Tests: 1076/1076 still pass.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(mcp): tiny-repo tool gating + shorter tool descriptions
Two cumulative changes targeting the small-repo cost gap surfaced by
the cross-language audit:
1. **Tool descriptions trimmed** (~2.1KB total saved across 10 tools).
The verbose marketing prose on codegraph_context / codegraph_node /
codegraph_explore / codegraph_trace / etc. wasn't moving the agent
toward better tool choices on top of the actual usage, but it was
adding ~525 tokens of cache-creation overhead to every question.
The trimmed descriptions keep the operational hints (e.g. "Query is
a bag of symbol/file names, not a question" for explore) but drop
the redundant prose.
2. **Dynamic tiny-repo tool gating** in `ToolHandler.getTools()`. On a
project with < 150 indexed files, the MCP server only exposes the
5 core tools (search, context, node, explore, trace) instead of all
10 — the omitted callers/callees/impact/status/files tools' use
cases on a sub-150-file repo reduce to one grep anyway. The MCP
tool-defs overhead is the #1 source of cost loss on tiny repos
(~$0.10-0.15 fixed cache-creation per question); cutting 5 tools
drops that by ~50%.
Effect on ky (~25 files, the worst pre-fix offender):
- Before: $0.59 WITH vs $0.42 WITHOUT (+42% loss, n=1)
- After: $0.32 WITH vs $0.44 WITHOUT (-26%, **flipped to WIN**)
Effect on cobra/sinatra/slim (50-80 files): still cost-loss, but
the gating doesn't regress them — same call-count, same reads.
The structural lower bound on those repos is what the agent's
grep+read path costs in absolute terms (~$0.20-0.30).
Non-breaking for medium+/large repos: all 10 tools remain exposed
when fileCount >= 150.
Tests: 1076/1076 still pass.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(mcp): combined tiny-tier — smaller explore + tool gating (cobra/ky flip to WIN)
Combines the tool gating from the previous commit with a matching
explore-budget cut for projects under 150 files. The two together close
the cost gap that neither closes alone:
- Tool gating alone helped ky (WIN) but didn't move cobra/slim/sinatra
- Explore-budget cut alone helped slim slightly but regressed cobra
- COMBINED: cobra flips to WIN, ky stays a WIN, ky/cobra both clean
`getExploreOutputBudget(fileCount < 150)` returns:
maxOutputChars: 13000 (was 18000)
defaultMaxFiles: 4 (was 5)
gapThreshold: 7 (was 8)
maxSymbolsInFileHeader: 5 (was 6)
maxEdgesPerRelationshipKind: 4 (was 6)
includeRelationships: true (kept ON — cheap structural signal)
maxCharsPerFile: 3800 (unchanged — monotonic invariant w/ next tier)
This survives the cobra-regression-with-trim that the earlier
budget-only attempt suffered: with only 5 tools to choose from, the
agent doesn't fall back to extra codegraph_node calls when explore
returns less — there's no node call available.
Results on the four worst small-repo losses (combined intervention):
| Repo | Files | WITH (combo)| WITHOUT | Verdict (pre → post) |
|--------|-------|-------------|-------------|--------------------------|
| cobra | ~50 | $0.25 | $0.31 | loss → **WIN** (-19%) |
| ky | ~25 | $0.39 | $0.39 | -42% → tied |
| slim | ~80 | $0.31 | $0.24 | LOSS 31% → still LOSS |
| sinatra| ~60 | $0.30 | $0.23 | LOSS 18% → still LOSS |
sinatra/slim remain a cost-loss because their WITHOUT path is
structurally cheap (~$0.20 — fewer than 4 cheap grep+read calls).
Codegraph can't beat that absolute floor with any meaningful response.
Both still WIN on time + reads + tool-call count.
Tests: tier boundary cases updated to cover the new <150 / 150-499 /
500-4999 / 5000-14999 / >=15000 progression. Off-by-one guard updated
to include the new 149↔150 boundary. All 1076 tests pass.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(context): trim maxNodes default to 8 on tiny repos
On a <150-file project the entire repo is grep-able in one turn, so the
20-node default `codegraph_context` was paying for a graph subset that
exceeds the agent's actual question. Cutting the tiny-repo default to 8
(typical 1-3 entry points + their immediate 1-hop neighbors) reduces
the context-tool response body without hitting sufficiency on the flow
shapes small repos actually contain.
Non-breaking: the agent can still pass an explicit `maxNodes` to
override; medium+ repos (>=150 files) keep the 20-node default.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(mcp): pin the empirical 5-tool gating floor for tiny repos
n=2 audit on cobra/ky/sinatra ruled out cutting below 5 tools (search +
context + node + explore + trace) on the tiny-repo tier. The smaller
3-tool gate (search + context + trace) saved ~$0.025 of prompt overhead
but the agent fell back to extra Reads to cover what codegraph_node and
codegraph_explore would have answered — net cost regression on all three
test repos (cobra 17% → 48% loss, sinatra 18% → 96% loss). Documented
inline so future tuners don't re-try this dead-end.
No behavior change beyond the comment: the 5-tool gate remains the
production setting.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(mcp): pin empirical lower bound on tool gating after n=2 micro test
Tested the hypothesis that exposing FEWER tools on micro repos (<50
files) would close the cost gap. Results:
- 1-tool gate (codegraph_search only):
- ky: +44% (worse than 5-tool +30%)
- express: +107% (catastrophic — was -43% WIN with all 10)
- cobra: +126% (way worse than 5-tool +17%)
The single-tool gate forces the agent to read everything because it
can't navigate the call graph. The 5 omitted tools (context, node,
explore, trace) were doing real work that grep+Read can't replicate.
Conclusion: 5 tools (search + context + node + explore + trace) is the
empirical lower bound on the tiny-repo tier. Cutting below regresses
EVERY tested repo. The remaining ~$0.04-0.08 of structural cost overhead
on tiny repos is unavoidable without sacrificing the value codegraph
provides at that scale (which would also make WITH = WITHOUT, defeating
the install).
Comment documents the dead-ends so future tuners don't relitigate.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(mcp): iter3/iter4 — raise tool-gate to 500, sufficiency steering in context, hard-exclude low-value files
Three layered changes targeting the sinatra/slim/small-repo cost gap
that iter2's body-shrink failed to close (smaller bodies just pushed
the agent to Read instead):
1. **Tool-gate threshold 150 → 500** (`TINY_REPO_FILE_THRESHOLD`).
Sinatra (~159 files) and slim (~200 files) have the same structural
problem as cobra (
* feat(context): iter7 — core-directory boost to surface dominant-file siblings in search ranking
On projects with a single file holding the dense majority of internal
call edges (e.g. sinatra's `lib/sinatra/base.rb` at ~85% of in-file
edges), text search was favoring small focused extension files over the
core file. A small focused file like `multi_route.rb` wins on verbatim
name match + file-size normalization, burying the 1500-line core file's
longer method names (e.g. `route!` vs `route`).
Fix: detect the "dominant file" — the file whose in-file edge count is
≥3× the next candidate's — then add +25 to all results sharing its
directory prefix. This pulls the core file's siblings above
sibling-package extensions without hardcoding any repo structure.
`getDominantFile()` excludes test/spec files and generated files
(e.g. etcd's `rpc.pb.go` has 4× the in-file edges of `server.go` and
would otherwise hijack the boost toward generated protobuf stubs).
SQL pulls the top 20 candidates; path-pattern filtering handles what
SQLite LIKE can't express.
* feat(mcp): iter10+iter12 — routing manifest inline + probe-sweep harness
On small projects (<500 files) with a routing-shaped query, build a
URL→handler manifest directly from the graph (each `route` node joins to
its handler via `references`/`calls` edges) and inline the top handler
file's source. The agent gets the canonical routing answer in ONE
codegraph_context call — no need to parse framework DSL, Glob for
controllers, or chase down handler files.
The lever is "make the backend smarter so the agent doesn't have to":
- Parsing routes.rb / routes/api.php / urls.py DSL is the agent's job
in the WITHOUT arm. Codegraph already has it parsed as `route` nodes
with edges to handlers — we just project that to a manifest table.
- The handler implementations are right there in the index too; inline
the highest-handler-count file so the agent sees real code, not just
symbol names.
Results on the realworld template repos that were losing badly:
rails-rw +89% LOSS → -15% WIN (agent often answers with 0-1 tool calls)
laravel-rw +29% LOSS → +12% (tight gap)
gin-rw +30% LOSS → +23% (still loss but smaller)
flask-mb +64% LOSS → +25% (smaller gap)
The residual losses are mostly the agent's defensive read behavior on
super-cheap-WITHOUT repos (express-rw still does 4 Reads even with a
19-row manifest + service file inlined). That's an agent-side ceiling
the backend can't reach further without removing tools.
Also lands `scripts/agent-eval/probe-sweep.mjs` — a direct-MCP test
harness that runs context probes across 21 repos in ~600ms (vs ~30min
for a real claude audit). Enables rapid iteration on backend changes:
edit tools.ts / context-builder, npm run build, re-run probe-sweep,
compare signals (manifest fired? handler file inlined? response size?)
before paying for a claude run.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(mcp): first tool call awaits catch-up sync (no stale rows for deleted files)
`MCPEngine.catchUpSync()` reconciles the index against the working tree
after open (catching `git pull`/`checkout`/`rebase` and any edits or
deletes made while no server was running). It was fire-and-forget — so a
tool call landing in the first ~50-300ms could race past it and serve
rows for files that no longer exist on disk. The per-file staleness
banner can't help here, because that signal is populated by the file
watcher (not by catch-up).
The fix: `catchUpSync()` now pushes its promise into `ToolHandler` via
`setCatchUpGate(p)`; the first `execute()` call awaits the gate and then
clears it. Subsequent calls pay nothing. Catch-up rejections are logged
by the engine and swallowed by the handler so a transient sync failure
never breaks tools.
Most visible on the "deleted everything between sessions" case, where
MCP previously returned stale rows pointing at non-existent files.
Validated end-to-end on a 10,640-file VS Code index: with the gate, a
codegraph_search for "ExtensionHost" against an empty (but stale-DB)
directory returns "No results found" after the catch-up drains the DB;
without the gate, the same call returns 10 stale hits.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(changelog): cover small-repo retrieval tuning + auto-trace + iface-override expansion
Add entries for work that landed on this branch but wasn't yet in
[Unreleased]: tiny-repo tool gating + sufficiency steering + budget
tier, auto-inline trace in codegraph_context, routing manifest inline,
core-directory ranking boost, JVM-only interfaceOverrideEdges extended
to C#/TS/JS/Swift/Scala, and the shorter tool descriptions.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Colby Mchenry
co-authored by
Claude Opus 4.7
Claude Opus 4.7
parent
02935d777a
commit
71935e37c2
+11
-1
@@ -843,11 +843,21 @@ program
|
||||
const cg = await CodeGraph.open(projectPath);
|
||||
|
||||
const limit = parseInt(options.limit || '10', 10);
|
||||
const results = cg.searchNodes(search, {
|
||||
const rawResults = cg.searchNodes(search, {
|
||||
limit,
|
||||
kinds: options.kind ? [options.kind as any] : undefined,
|
||||
});
|
||||
|
||||
// Mirror the MCP search down-rank so the CLI also surfaces the
|
||||
// hand-written implementation before protobuf/gRPC scaffolding
|
||||
// when both share a name. See extraction/generated-detection.ts.
|
||||
const { isGeneratedFile } = await import('../extraction/generated-detection');
|
||||
const results = [...rawResults].sort((a, b) => {
|
||||
const aGen = isGeneratedFile(a.node.filePath) ? 1 : 0;
|
||||
const bGen = isGeneratedFile(b.node.filePath) ? 1 : 0;
|
||||
return aGen - bGen;
|
||||
});
|
||||
|
||||
if (options.json) {
|
||||
console.log(JSON.stringify(results, null, 2));
|
||||
} else {
|
||||
|
||||
@@ -5,6 +5,7 @@
|
||||
*/
|
||||
|
||||
import { Node, Edge, TaskContext, Subgraph } from '../types';
|
||||
import { isGeneratedFile } from '../extraction/generated-detection';
|
||||
|
||||
/**
|
||||
* Format context as markdown
|
||||
@@ -21,10 +22,17 @@ export function formatContextAsMarkdown(context: TaskContext): string {
|
||||
lines.push('## Code Context\n');
|
||||
lines.push(`**Query:** ${context.query}\n`);
|
||||
|
||||
// Entry points - compact format
|
||||
if (context.entryPoints.length > 0) {
|
||||
// Entry points - compact format. Re-sort so generated files (.pb.go,
|
||||
// .pulsar.go, mocks, …) rank LAST — a flow query should lead with the
|
||||
// hand-written implementation, not protobuf scaffolding.
|
||||
const orderedEntries = [...context.entryPoints].sort((a, b) => {
|
||||
const aGen = isGeneratedFile(a.filePath) ? 1 : 0;
|
||||
const bGen = isGeneratedFile(b.filePath) ? 1 : 0;
|
||||
return aGen - bGen;
|
||||
});
|
||||
if (orderedEntries.length > 0) {
|
||||
lines.push('### Entry Points\n');
|
||||
for (const node of context.entryPoints) {
|
||||
for (const node of orderedEntries) {
|
||||
const location = node.startLine ? `:${node.startLine}` : '';
|
||||
lines.push(`- **${node.name}** (${node.kind}) - ${node.filePath}${location}`);
|
||||
if (node.signature) {
|
||||
@@ -34,9 +42,14 @@ export function formatContextAsMarkdown(context: TaskContext): string {
|
||||
lines.push('');
|
||||
}
|
||||
|
||||
// Related symbols - compact list (skip verbose structure tree)
|
||||
// Related symbols - compact list (skip verbose structure tree). Drop nodes
|
||||
// in generated source files (`.pb.go` / `.pulsar.go` / mocks / …) — agents
|
||||
// chasing a flow never want to land on protobuf scaffolding (cosmos-Q3 used
|
||||
// to list `gov.pulsar.go::GetExpeditedThreshold` and `1.pulsar.go::Get` in
|
||||
// Related Symbols, pure noise that displaced real-flow entries).
|
||||
const otherSymbols = Array.from(context.subgraph.nodes.values())
|
||||
.filter(n => !context.entryPoints.some(e => e.id === n.id))
|
||||
.filter(n => !isGeneratedFile(n.filePath))
|
||||
.slice(0, 10); // Limit to 10 related symbols
|
||||
|
||||
if (otherSymbols.length > 0) {
|
||||
@@ -55,10 +68,16 @@ export function formatContextAsMarkdown(context: TaskContext): string {
|
||||
lines.push('');
|
||||
}
|
||||
|
||||
// Code blocks - only for key entry points
|
||||
// Code blocks - only for key entry points. Re-sort so non-generated blocks
|
||||
// show first (consistent with Entry Points reordering above).
|
||||
if (context.codeBlocks.length > 0) {
|
||||
const orderedBlocks = [...context.codeBlocks].sort((a, b) => {
|
||||
const aGen = isGeneratedFile(a.filePath) ? 1 : 0;
|
||||
const bGen = isGeneratedFile(b.filePath) ? 1 : 0;
|
||||
return aGen - bGen;
|
||||
});
|
||||
lines.push('### Code\n');
|
||||
for (const block of context.codeBlocks) {
|
||||
for (const block of orderedBlocks) {
|
||||
const nodeName = block.node?.name ?? 'Unknown';
|
||||
lines.push(`#### ${nodeName} (${block.filePath}:${block.startLine})\n`);
|
||||
lines.push('```' + block.language);
|
||||
|
||||
@@ -587,6 +587,37 @@ export class ContextBuilder {
|
||||
}
|
||||
}
|
||||
|
||||
// Iter7 — Core-directory boost. On projects with one file that holds
|
||||
// the dense majority of internal call edges (e.g. sinatra's
|
||||
// `lib/sinatra/base.rb` at 85% of all in-file edges), the agent's
|
||||
// task usually asks about the framework's core. Without this boost,
|
||||
// ranking favors small focused extension files (e.g. text search
|
||||
// picks `sinatra-contrib/lib/sinatra/multi_route.rb`'s 10-line
|
||||
// `route` method over `base.rb`'s `route!` because the extension
|
||||
// file's `route` matches the query verbatim AND the file is small,
|
||||
// dwarfing the longer name `route!` in a 1500-line file). Boost
|
||||
// results that share a directory prefix with the dominant file's
|
||||
// directory so the core file's siblings outrank sibling-package
|
||||
// extensions.
|
||||
try {
|
||||
const dominant = this.queries.getDominantFile?.();
|
||||
if (dominant && dominant.edgeCount >= 3 * dominant.nextEdgeCount) {
|
||||
// Take the directory of the dominant file (everything up to the
|
||||
// last slash). For `lib/sinatra/base.rb` → `lib/sinatra/`.
|
||||
const slash = dominant.filePath.lastIndexOf('/');
|
||||
if (slash > 0) {
|
||||
const coreDir = dominant.filePath.slice(0, slash + 1);
|
||||
for (const result of searchResults) {
|
||||
if (result.node.filePath.startsWith(coreDir)) {
|
||||
result.score += 25;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
} catch {
|
||||
// SQL failure — fall through, scoring works without the boost
|
||||
}
|
||||
|
||||
// Step 5a: Multi-term co-occurrence re-ranking (applied BEFORE truncation).
|
||||
// For multi-word queries like "search execution from request to shard",
|
||||
// nodes matching 2+ query terms in their name or path are far more relevant
|
||||
|
||||
@@ -20,6 +20,32 @@ import {
|
||||
import { safeJsonParse } from '../utils';
|
||||
import { kindBonus, nameMatchBonus, scorePathRelevance } from '../search/query-utils';
|
||||
import { parseQuery, boundedEditDistance } from '../search/query-parser';
|
||||
import { isGeneratedFile } from '../extraction/generated-detection';
|
||||
|
||||
/**
|
||||
* Path-only heuristic for files that should not be candidates for
|
||||
* "dominant file" detection: test/spec files and tool-generated files.
|
||||
* Generated files (`*.pb.go`, `*.pulsar.go`, mock outputs, …) often
|
||||
* have huge in-file edge counts that dwarf the real source — etcd's
|
||||
* `rpc.pb.go` has 4× the in-file edges of `server.go`.
|
||||
*/
|
||||
function isLowValueFile(filePath: string): boolean {
|
||||
const lp = filePath.toLowerCase();
|
||||
return (
|
||||
/(?:^|\/)(tests?|__tests?__|spec)\//.test(lp) ||
|
||||
/_test\.go$/.test(lp) ||
|
||||
/(?:^|\/)test_[^/]+\.py$/.test(lp) ||
|
||||
/_test\.py$/.test(lp) ||
|
||||
/_spec\.rb$/.test(lp) ||
|
||||
/_test\.rb$/.test(lp) ||
|
||||
/\.(test|spec)\.[jt]sx?$/.test(lp) ||
|
||||
/(test|spec|tests)\.(java|kt|scala)$/.test(lp) ||
|
||||
/(tests?|spec)\.cs$/.test(lp) ||
|
||||
/tests?\.swift$/.test(lp) ||
|
||||
/_test\.dart$/.test(lp) ||
|
||||
isGeneratedFile(filePath)
|
||||
);
|
||||
}
|
||||
|
||||
const SQLITE_PARAM_CHUNK_SIZE = 500;
|
||||
|
||||
@@ -182,6 +208,9 @@ export class QueryBuilder {
|
||||
getUnresolvedBatch?: SqliteStatement;
|
||||
getAllFilePaths?: SqliteStatement;
|
||||
getAllNodeNames?: SqliteStatement;
|
||||
getDominantFile?: SqliteStatement;
|
||||
getTopRouteFile?: SqliteStatement;
|
||||
getRoutingManifest?: SqliteStatement;
|
||||
} = {};
|
||||
|
||||
constructor(db: SqliteDatabase) {
|
||||
@@ -489,6 +518,158 @@ export class QueryBuilder {
|
||||
return rows.map(rowToNode);
|
||||
}
|
||||
|
||||
/**
|
||||
* Find the file that holds the densest concentration of the project's
|
||||
* internal call graph — the "core" file. Used by context-builder to
|
||||
* boost ranking of symbols in that file's directory (so e.g. sinatra
|
||||
* queries surface `lib/sinatra/base.rb`'s `route!` instead of
|
||||
* `sinatra-contrib/lib/sinatra/multi_route.rb`'s `route` extension).
|
||||
*
|
||||
* Returns null if no file has a meaningful concentration (e.g. spread
|
||||
* evenly across many files, or empty index).
|
||||
*
|
||||
* "Internal" = source and target are in the same file. Cross-file
|
||||
* edges aren't useful here — they don't tell us which file is the
|
||||
* functional center.
|
||||
*
|
||||
* Excludes test/spec files from candidacy via path-pattern. The agent's
|
||||
* typical question is "how does X work", not "how is X tested", so
|
||||
* boosting a test file's directory would be a misfire.
|
||||
*/
|
||||
getDominantFile(): { filePath: string; edgeCount: number; nextEdgeCount: number } | null {
|
||||
if (!this.stmts.getDominantFile) {
|
||||
// Pull top 20 candidates; we then filter out test/generated files
|
||||
// in code (regex-grade matching that SQL LIKE can't express). The
|
||||
// generated-file filter is critical — without it, etcd's
|
||||
// `api/etcdserverpb/rpc.pb.go` (1916 in-file edges, generated
|
||||
// protobuf stub) outranks the real `server/etcdserver/server.go`
|
||||
// (470 edges) by 4×, and the boost would push the agent toward
|
||||
// generated code.
|
||||
this.stmts.getDominantFile = this.db.prepare(`
|
||||
SELECT n.file_path AS file_path, COUNT(*) AS edge_count
|
||||
FROM edges e
|
||||
JOIN nodes n ON e.source = n.id
|
||||
JOIN nodes m ON e.target = m.id
|
||||
WHERE n.file_path = m.file_path
|
||||
GROUP BY n.file_path
|
||||
ORDER BY edge_count DESC
|
||||
LIMIT 20
|
||||
`);
|
||||
}
|
||||
const rows = this.stmts.getDominantFile.all() as Array<{ file_path: string; edge_count: number }>;
|
||||
const filtered = rows.filter(r => !isLowValueFile(r.file_path));
|
||||
if (filtered.length === 0 || filtered[0]!.edge_count < 20) return null;
|
||||
return {
|
||||
filePath: filtered[0]!.file_path,
|
||||
edgeCount: filtered[0]!.edge_count,
|
||||
nextEdgeCount: filtered[1]?.edge_count ?? 0,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Find the file that holds the densest concentration of the project's
|
||||
* `route` nodes (framework-emitted: Express/Gin/Flask/Rails/Drupal/etc.).
|
||||
* Used by handleContext on small repos to inline the project's routing
|
||||
* config when the agent's query is about request flow — eliminating the
|
||||
* "Glob + Read routes.rb" pattern that beats codegraph on tiny realworld
|
||||
* template repos.
|
||||
*
|
||||
* Excludes test/generated files from candidacy. Returns null if there
|
||||
* are fewer than 3 non-test routes total, or if no file holds at least
|
||||
* 30% of them (diffuse routing → no single answer file).
|
||||
*/
|
||||
getTopRouteFile(): { filePath: string; routeCount: number; totalRoutes: number } | null {
|
||||
if (!this.stmts.getTopRouteFile) {
|
||||
this.stmts.getTopRouteFile = this.db.prepare(`
|
||||
SELECT file_path, COUNT(*) AS cnt
|
||||
FROM nodes
|
||||
WHERE kind = 'route'
|
||||
GROUP BY file_path
|
||||
ORDER BY cnt DESC
|
||||
LIMIT 20
|
||||
`);
|
||||
}
|
||||
const rows = this.stmts.getTopRouteFile.all() as Array<{ file_path: string; cnt: number }>;
|
||||
const filtered = rows.filter(r => !isLowValueFile(r.file_path));
|
||||
if (filtered.length === 0) return null;
|
||||
const totalRoutes = filtered.reduce((sum, r) => sum + r.cnt, 0);
|
||||
const top = filtered[0]!;
|
||||
if (totalRoutes < 3 || top.cnt < 3) return null;
|
||||
if (top.cnt / totalRoutes < 0.30) return null;
|
||||
return { filePath: top.file_path, routeCount: top.cnt, totalRoutes };
|
||||
}
|
||||
|
||||
/**
|
||||
* Build a URL → handler manifest from the index. Each route node's
|
||||
* `references` edge points at the function/method that handles the
|
||||
* request. We join them in one pass; the agent gets the canonical
|
||||
* routing answer ("POST /users/login → AuthController#login") without
|
||||
* having to parse the framework's route DSL itself.
|
||||
*
|
||||
* Also returns the file with the most handler endpoints — used as the
|
||||
* "top handler file" to inline source for, so the agent has both the
|
||||
* mapping AND the handler implementations.
|
||||
*/
|
||||
getRoutingManifest(limit: number = 40): {
|
||||
entries: Array<{ url: string; handler: string; handlerFile: string; handlerLine: number; handlerKind: string }>;
|
||||
topHandlerFile: string | null;
|
||||
topHandlerFileCount: number;
|
||||
totalRoutes: number;
|
||||
} | null {
|
||||
if (!this.stmts.getRoutingManifest) {
|
||||
// Edge kind varies across framework resolvers: Spring/Rails/
|
||||
// Laravel/Drupal emit `references`, Express emits `calls`. Accept
|
||||
// both — the semantic is the same (route → its handler).
|
||||
this.stmts.getRoutingManifest = this.db.prepare(`
|
||||
SELECT
|
||||
r.name AS url,
|
||||
h.name AS handler,
|
||||
h.file_path AS handler_file,
|
||||
h.start_line AS handler_line,
|
||||
h.kind AS handler_kind
|
||||
FROM nodes r
|
||||
JOIN edges e ON e.source = r.id
|
||||
JOIN nodes h ON e.target = h.id
|
||||
WHERE r.kind = 'route'
|
||||
AND e.kind IN ('references', 'calls')
|
||||
AND h.kind IN ('function', 'method', 'class')
|
||||
ORDER BY r.file_path, r.start_line
|
||||
LIMIT ?
|
||||
`);
|
||||
}
|
||||
const rows = this.stmts.getRoutingManifest.all(limit) as Array<{
|
||||
url: string; handler: string; handler_file: string; handler_line: number; handler_kind: string;
|
||||
}>;
|
||||
// Drop test/generated handlers — same hygiene as elsewhere.
|
||||
const filtered = rows.filter(r => !isLowValueFile(r.handler_file));
|
||||
if (filtered.length < 3) return null;
|
||||
// Identify the file holding the most handlers (the "primary handler file").
|
||||
const fileCounts = new Map<string, number>();
|
||||
for (const r of filtered) {
|
||||
fileCounts.set(r.handler_file, (fileCounts.get(r.handler_file) ?? 0) + 1);
|
||||
}
|
||||
let topHandlerFile: string | null = null;
|
||||
let topHandlerFileCount = 0;
|
||||
for (const [file, count] of fileCounts) {
|
||||
if (count > topHandlerFileCount) {
|
||||
topHandlerFile = file;
|
||||
topHandlerFileCount = count;
|
||||
}
|
||||
}
|
||||
return {
|
||||
entries: filtered.map(r => ({
|
||||
url: r.url,
|
||||
handler: r.handler,
|
||||
handlerFile: r.handler_file,
|
||||
handlerLine: r.handler_line,
|
||||
handlerKind: r.handler_kind,
|
||||
})),
|
||||
topHandlerFile,
|
||||
topHandlerFileCount,
|
||||
totalRoutes: filtered.length,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Get all nodes of a specific kind
|
||||
*/
|
||||
|
||||
@@ -0,0 +1,78 @@
|
||||
/**
|
||||
* Generated-file detection for symbol-disambiguation down-ranking.
|
||||
*
|
||||
* When a query like "Send" matches 17 symbols across protobuf scaffolding,
|
||||
* test mocks, and the hand-written implementation, the FTS ranker often
|
||||
* surfaces the generated stubs first because their names are identical
|
||||
* to the implementation's name (validated empirically on cosmos-sdk —
|
||||
* see project_go_multi_module_audit memory). Generated stubs frequently
|
||||
* have no body to trace from, so the agent ends up reading source anyway.
|
||||
*
|
||||
* This helper is a pure path-based classifier consulted at disambiguation
|
||||
* time (findSymbol / findAllSymbols / codegraph_search formatting), NOT
|
||||
* a hard filter — generated nodes are still in the graph and remain
|
||||
* reachable; they just rank LAST when there's a real implementation
|
||||
* with the same name.
|
||||
*
|
||||
* Scope: suffix patterns only. Most generated files follow the
|
||||
* `<basename>.<tool>.<ext>` convention (`.pb.go`, `_grpc.pb.go`,
|
||||
* `.g.dart`, `_pb2.py`), and that covers ~all of what we saw in the
|
||||
* Go audit. A future addition would be scanning for the canonical
|
||||
* `// Code generated by` header during extraction, for the rare files
|
||||
* that defy the suffix convention.
|
||||
*/
|
||||
|
||||
const GENERATED_PATTERNS: ReadonlyArray<RegExp> = [
|
||||
// Go — protobuf / gRPC / pulsar
|
||||
/\.pb\.go$/,
|
||||
/\.pulsar\.go$/,
|
||||
/_grpc\.pb\.go$/,
|
||||
// Go — mockgen output. Default emits `mock_<src>.go`; many projects
|
||||
// (cosmos-sdk uses `expected_*_mocks.go`) rename to `*_mock.go` /
|
||||
// `*_mocks.go`. Matching either suffix catches both conventions
|
||||
// without false-positive risk on hand-written sources.
|
||||
/_mock\.go$/,
|
||||
/_mocks\.go$/,
|
||||
/^mock_[^/]+\.go$/,
|
||||
// TypeScript / JavaScript — common codegen suffixes (Apollo / GraphQL
|
||||
// codegen, Prisma, Hasura, ts-proto, gRPC-web, swagger-codegen).
|
||||
/\.generated\.[jt]sx?$/,
|
||||
/\.gen\.[jt]sx?$/,
|
||||
/\.pb\.[jt]s$/,
|
||||
/_pb\.[jt]s$/,
|
||||
/_grpc_pb\.[jt]s$/,
|
||||
// Python — protobuf / gRPC / openapi-codegen
|
||||
/_pb2(_grpc)?\.py$/,
|
||||
/_pb2\.pyi$/,
|
||||
// C++ — protobuf
|
||||
/\.pb\.(cc|h)$/,
|
||||
// C# — protobuf / gRPC (protoc-gen-csharp puts output under obj/ but
|
||||
// many projects also commit *.g.cs and *Grpc.cs siblings)
|
||||
/\.g\.cs$/,
|
||||
/Grpc\.cs$/,
|
||||
// Java — protobuf / gRPC: protoc-gen-java emits `*OuterClass.java`,
|
||||
// protoc-gen-grpc-java emits `*Grpc.java`. The XxxImplBase abstract
|
||||
// class lives inside Xxx*Grpc.java.
|
||||
/OuterClass\.java$/,
|
||||
/Grpc\.java$/,
|
||||
// Swift — protobuf
|
||||
/\.pb\.swift$/,
|
||||
// Dart — build_runner / freezed / json_serializable / chopper
|
||||
/\.g\.dart$/,
|
||||
/\.freezed\.dart$/,
|
||||
/\.pb\.dart$/,
|
||||
/\.pbgrpc\.dart$/,
|
||||
/\.chopper\.dart$/,
|
||||
// Rust — common build.rs OUT_DIR outputs are usually outside the source
|
||||
// tree, but in-tree generated files often use `*.generated.rs`.
|
||||
/\.generated\.rs$/,
|
||||
];
|
||||
|
||||
/**
|
||||
* Whether `filePath` looks like a tool-generated source file based on
|
||||
* its filename. Path-only — does not read content. The result is a
|
||||
* relevance hint for disambiguation, not a hard claim.
|
||||
*/
|
||||
export function isGeneratedFile(filePath: string): boolean {
|
||||
return GENERATED_PATTERNS.some((p) => p.test(filePath));
|
||||
}
|
||||
@@ -683,6 +683,33 @@ export class CodeGraph {
|
||||
return this.queries.searchNodes(query, options);
|
||||
}
|
||||
|
||||
/**
|
||||
* Find the project's "primary route file" — the file with the densest
|
||||
* concentration of framework-emitted `route` nodes (≥3 routes, ≥30%
|
||||
* of all non-test routes). Used to inline the routing config in
|
||||
* `codegraph_context` responses on small realworld template repos
|
||||
* (rails-realworld, laravel-realworld, drupal-admintoolbar, …) where
|
||||
* Glob+Read of `routes.rb`/`urls.py`/etc. otherwise beats codegraph.
|
||||
*/
|
||||
getTopRouteFile(): { filePath: string; routeCount: number; totalRoutes: number } | null {
|
||||
return this.queries.getTopRouteFile();
|
||||
}
|
||||
|
||||
/**
|
||||
* Build a URL → handler routing manifest from the index. Each entry
|
||||
* pairs a route node (URL + method) with its handler function/method
|
||||
* via the `references` edge that framework resolvers emit. Returns
|
||||
* null when fewer than 3 valid (non-test) routes exist.
|
||||
*/
|
||||
getRoutingManifest(limit?: number): {
|
||||
entries: Array<{ url: string; handler: string; handlerFile: string; handlerLine: number; handlerKind: string }>;
|
||||
topHandlerFile: string | null;
|
||||
topHandlerFileCount: number;
|
||||
totalRoutes: number;
|
||||
} | null {
|
||||
return this.queries.getRoutingManifest(limit);
|
||||
}
|
||||
|
||||
// ===========================================================================
|
||||
// Edge Operations
|
||||
// ===========================================================================
|
||||
|
||||
+8
-2
@@ -222,12 +222,17 @@ export class MCPEngine {
|
||||
/**
|
||||
* Reconcile the index with the current filesystem once, right after open —
|
||||
* catches edits, adds, deletes, and `git pull`/`checkout` changes made while
|
||||
* no watcher was running. Background, never awaited.
|
||||
* no watcher was running. Runs in the background, but the returned promise
|
||||
* is pushed into the ToolHandler as a one-shot gate so the *first* tool
|
||||
* call awaits completion before serving (without this, a tool call that
|
||||
* races past sync returns rows for files that no longer exist on disk —
|
||||
* and the per-file staleness banner can't help because `getPendingFiles()`
|
||||
* is populated by the watcher, not by catch-up).
|
||||
*/
|
||||
private catchUpSync(): void {
|
||||
const cg = this.cg;
|
||||
if (!cg) return;
|
||||
void cg
|
||||
const p = cg
|
||||
.sync()
|
||||
.then((result) => {
|
||||
const changed = result.filesAdded + result.filesModified + result.filesRemoved;
|
||||
@@ -239,6 +244,7 @@ export class MCPEngine {
|
||||
const msg = err instanceof Error ? err.message : String(err);
|
||||
process.stderr.write(`[CodeGraph MCP] Catch-up sync failed: ${msg}\n`);
|
||||
});
|
||||
this.toolHandler.setCatchUpGate(p);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
+560
-52
@@ -21,10 +21,13 @@ import {
|
||||
lstatSync,
|
||||
openSync,
|
||||
readFileSync,
|
||||
statSync,
|
||||
writeSync,
|
||||
} from 'fs';
|
||||
import { clamp, validatePathWithinRoot, validateProjectPath } from '../utils';
|
||||
import { isGeneratedFile } from '../extraction/generated-detection';
|
||||
import { tmpdir } from 'os';
|
||||
import * as pathModule from 'path';
|
||||
import { join, resolve as resolvePath } from 'path';
|
||||
|
||||
/** Maximum output length to prevent context bloat (characters) */
|
||||
@@ -123,21 +126,52 @@ export interface ExploreOutputBudget {
|
||||
includeCompletenessSignal: boolean;
|
||||
/** Include the explore-budget reminder at the end. */
|
||||
includeBudgetNote: boolean;
|
||||
/**
|
||||
* Hard-drop test/spec/icon/i18n files from the relevant-file set unless
|
||||
* the query itself mentions tests. Today they're only deprioritized in
|
||||
* the sort, which on tiny repos still lets one slip into the top N (e.g.
|
||||
* cobra's `command_test.go` displaced `args.go` and contributed ~10KB of
|
||||
* pure noise to "How does cobra parse commands?"). Off by default; on
|
||||
* for the very-tiny tier where one slip dominates the budget.
|
||||
*/
|
||||
excludeLowValueFiles: boolean;
|
||||
}
|
||||
|
||||
export function getExploreOutputBudget(fileCount: number): ExploreOutputBudget {
|
||||
if (fileCount < 150) {
|
||||
return {
|
||||
// ITER3: revert iter2's aggressive body shrink (forced Read fallback —
|
||||
// the per-file 2.5K cap pushed the agent to Read instead of node).
|
||||
// Back to the iter1 shape (13K/4/3.8K) but keep the test-file
|
||||
// hard-exclude. The cost lever for this tier lives in handleContext
|
||||
// (steering the agent to stop after 1-2 calls), not in this budget.
|
||||
maxOutputChars: 13000,
|
||||
defaultMaxFiles: 4,
|
||||
maxCharsPerFile: 3800,
|
||||
gapThreshold: 7,
|
||||
maxSymbolsInFileHeader: 5,
|
||||
maxEdgesPerRelationshipKind: 4,
|
||||
includeRelationships: false,
|
||||
includeAdditionalFiles: false,
|
||||
includeCompletenessSignal: false,
|
||||
includeBudgetNote: false,
|
||||
excludeLowValueFiles: true,
|
||||
};
|
||||
}
|
||||
if (fileCount < 500) {
|
||||
return {
|
||||
// ITER3: same revert/keep-filter pattern as <150.
|
||||
maxOutputChars: 18000,
|
||||
defaultMaxFiles: 5,
|
||||
maxCharsPerFile: 3800,
|
||||
gapThreshold: 8,
|
||||
maxSymbolsInFileHeader: 6,
|
||||
maxEdgesPerRelationshipKind: 6,
|
||||
includeRelationships: true,
|
||||
includeRelationships: false,
|
||||
includeAdditionalFiles: false,
|
||||
includeCompletenessSignal: false,
|
||||
includeBudgetNote: false,
|
||||
excludeLowValueFiles: true,
|
||||
};
|
||||
}
|
||||
if (fileCount < 5000) {
|
||||
@@ -157,6 +191,7 @@ export function getExploreOutputBudget(fileCount: number): ExploreOutputBudget {
|
||||
includeAdditionalFiles: true,
|
||||
includeCompletenessSignal: true,
|
||||
includeBudgetNote: true,
|
||||
excludeLowValueFiles: false,
|
||||
};
|
||||
}
|
||||
if (fileCount < 15000) {
|
||||
@@ -171,6 +206,7 @@ export function getExploreOutputBudget(fileCount: number): ExploreOutputBudget {
|
||||
includeAdditionalFiles: true,
|
||||
includeCompletenessSignal: true,
|
||||
includeBudgetNote: true,
|
||||
excludeLowValueFiles: false,
|
||||
};
|
||||
}
|
||||
return {
|
||||
@@ -184,6 +220,7 @@ export function getExploreOutputBudget(fileCount: number): ExploreOutputBudget {
|
||||
includeAdditionalFiles: true,
|
||||
includeCompletenessSignal: true,
|
||||
includeBudgetNote: true,
|
||||
excludeLowValueFiles: false,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -382,7 +419,7 @@ export const tools: ToolDefinition[] = [
|
||||
},
|
||||
{
|
||||
name: 'codegraph_context',
|
||||
description: 'PRIMARY TOOL — call this FIRST for any "how does X work", architecture, feature, or bug-context question. Composes search + node + callers + callees and returns entry points, related symbols, and key code in ONE call — usually enough to answer with no further search/Read/Grep. Prefer this over chaining codegraph_search + codegraph_node, and over codegraph_explore. NOTE: provides CODE context, not product requirements; for new features still clarify UX/edge cases with the user.',
|
||||
description: 'PRIMARY TOOL — call FIRST for any "how does X work"/architecture/bug question. Returns entry points + related symbols + key code in one call; usually answers without further search/Read/Grep. Provides CODE context, not product requirements.',
|
||||
inputSchema: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
@@ -407,7 +444,7 @@ export const tools: ToolDefinition[] = [
|
||||
},
|
||||
{
|
||||
name: 'codegraph_callers',
|
||||
description: 'Find all functions/methods that call a specific symbol. Useful for understanding usage patterns and impact of changes.',
|
||||
description: 'List functions that call <symbol>. For deep flow use codegraph_trace.',
|
||||
inputSchema: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
@@ -427,7 +464,7 @@ export const tools: ToolDefinition[] = [
|
||||
},
|
||||
{
|
||||
name: 'codegraph_callees',
|
||||
description: 'Find all functions/methods that a specific symbol calls. Useful for understanding dependencies and code flow.',
|
||||
description: 'List functions that <symbol> calls. For deep flow use codegraph_trace.',
|
||||
inputSchema: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
@@ -447,7 +484,7 @@ export const tools: ToolDefinition[] = [
|
||||
},
|
||||
{
|
||||
name: 'codegraph_impact',
|
||||
description: 'Analyze the impact radius of changing a symbol. Shows what code could be affected by modifications.',
|
||||
description: 'List symbols affected by changing <symbol>. Use before a refactor.',
|
||||
inputSchema: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
@@ -467,7 +504,7 @@ export const tools: ToolDefinition[] = [
|
||||
},
|
||||
{
|
||||
name: 'codegraph_node',
|
||||
description: 'Get ONE symbol\'s details (location, signature, docstring) PLUS its TRAIL — what it calls and what calls it, each with file:line. Pass includeCode=true for source (functions return their body; containers return a member outline). Use this to WALK the call graph hop-by-hop — node a symbol, then node one of its trail entries — the structural, no-Read way to follow "what calls/triggers/handles X" across files. For a broad first overview of many symbols at once use codegraph_explore; use node to drill along a specific path from there. (If a trail is empty on a non-leaf, that hop is likely dynamic dispatch — read just that line.) Source returned with includeCode is the verbatim live file content — identical to Read.',
|
||||
description: 'One symbol\'s location, signature, callers/callees trail. includeCode=true returns the verbatim body. Use codegraph_trace for full paths instead of chaining nodes.',
|
||||
inputSchema: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
@@ -487,7 +524,7 @@ export const tools: ToolDefinition[] = [
|
||||
},
|
||||
{
|
||||
name: 'codegraph_explore',
|
||||
description: 'Returns source for SEVERAL related symbols grouped by file, plus a relationship map, in ONE capped call. This is the efficient way to inspect many related symbols at once — strongly prefer it over a series of codegraph_node or Read calls (each separate call re-reads the whole context, so 8 node calls cost far more than 1 explore). Use it after codegraph_context when you need to see the actual source of several symbols. Query with specific symbol/file/code terms, NOT natural-language sentences — run codegraph_search first to find names. Bad: "how are agent prompts loaded and passed to the CLI". Good: "renderStaticScene drawElementOnCanvas ShapeCache renderElement.ts". The code it returns is the VERBATIM live file source (byte-for-byte identical to Read), line-numbered — not a summary; treat files it shows as already Read, no need to re-open them.',
|
||||
description: 'Source of SEVERAL related symbols grouped by file, in one capped call. Query is a bag of symbol/file names (not a question). Returned source is verbatim Read-equivalent — do not re-open shown files. Prefer over chained codegraph_node.',
|
||||
inputSchema: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
@@ -507,7 +544,7 @@ export const tools: ToolDefinition[] = [
|
||||
},
|
||||
{
|
||||
name: 'codegraph_status',
|
||||
description: 'Get the status of the CodeGraph index, including statistics about indexed files, nodes, and edges.',
|
||||
description: 'Index health check (files / nodes / edges). Skip unless debugging.',
|
||||
inputSchema: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
@@ -517,7 +554,7 @@ export const tools: ToolDefinition[] = [
|
||||
},
|
||||
{
|
||||
name: 'codegraph_files',
|
||||
description: 'REQUIRED for file/folder exploration. Get the project file structure from the CodeGraph index. Returns a tree view of all indexed files with metadata (language, symbol count). Much faster than Glob/filesystem scanning. Use this FIRST when exploring project structure, finding files, or understanding codebase organization.',
|
||||
description: 'Indexed file tree with language + symbol counts. Faster than Glob for project layout.',
|
||||
inputSchema: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
@@ -550,7 +587,7 @@ export const tools: ToolDefinition[] = [
|
||||
},
|
||||
{
|
||||
name: 'codegraph_trace',
|
||||
description: 'Trace the CALL PATH between two symbols — "how does <from> reach/become <to>?" Returns the chain of functions from one to the other (each hop with file:line and its body inlined, plus the outgoing calls of the destination itself) in ONE call. This is something grep/Read structurally cannot do: there is no text pattern for "the path from A to B". Ideal for flow questions — how an update triggers a render, how a request reaches a handler, how a QuerySet becomes SQL. If no static path exists the chain likely breaks at dynamic dispatch (callbacks/descriptors/metaclasses); the tool says where and points you to codegraph_node to bridge it.',
|
||||
description: 'Call path between two symbols — "how does <from> reach <to>?" Returns the chain with each hop\'s body inlined plus the destination\'s callees, in ONE call. Ideal for flow questions (update→render, request→handler, QuerySet→SQL). If no static path exists the chain broke at dynamic dispatch — the failure response inlines both endpoints + their TO-file siblings.',
|
||||
inputSchema: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
@@ -587,6 +624,14 @@ export class ToolHandler {
|
||||
// once and every later tool call reuses the result — never shelling out to
|
||||
// git on the hot path. `undefined` = not computed yet; `null` = no mismatch.
|
||||
private worktreeMismatchCache: Map<string, WorktreeIndexMismatch | null> = new Map();
|
||||
// Gate that the MCP engine pokes after `cg.open()` so the first tool call
|
||||
// blocks on the post-open filesystem reconcile (catch-up sync). Without
|
||||
// this, a tool call that races past `catchUpSync()` serves rows for files
|
||||
// that were deleted (or edited) while no MCP server was running — and the
|
||||
// per-file staleness banner can't help, because `getPendingFiles()` is
|
||||
// populated by the watcher, not by catch-up. Cleared on first await so
|
||||
// subsequent calls don't pay any cost.
|
||||
private catchUpGate: Promise<void> | null = null;
|
||||
|
||||
constructor(private cg: CodeGraph | null) {}
|
||||
|
||||
@@ -597,6 +642,17 @@ export class ToolHandler {
|
||||
this.cg = cg;
|
||||
}
|
||||
|
||||
/**
|
||||
* Engine-only: register the catch-up sync promise so the next `execute()`
|
||||
* call awaits it before serving. The handler swallows rejections (the
|
||||
* engine logs them) so a sync failure never propagates as a tool error;
|
||||
* we still want to serve a best-effort result over the same potentially-
|
||||
* stale data, which is what would have happened without the gate.
|
||||
*/
|
||||
setCatchUpGate(p: Promise<void> | null): void {
|
||||
this.catchUpGate = p;
|
||||
}
|
||||
|
||||
/**
|
||||
* Record the directory the server tried to resolve the default project from.
|
||||
* Used only to make the "no default project" error actionable.
|
||||
@@ -642,7 +698,7 @@ export class ToolHandler {
|
||||
*/
|
||||
getTools(): ToolDefinition[] {
|
||||
const allow = this.toolAllowlist();
|
||||
const visible = allow
|
||||
let visible = allow
|
||||
? tools.filter(t => allow.has(t.name.replace(/^codegraph_/, '')))
|
||||
: tools;
|
||||
if (!this.cg) return visible;
|
||||
@@ -651,6 +707,40 @@ export class ToolHandler {
|
||||
const stats = this.cg.getStats();
|
||||
const budget = getExploreBudget(stats.fileCount);
|
||||
|
||||
// Tiny-repo tool gating: on projects under TINY_REPO_FILE_THRESHOLD
|
||||
// files, only expose the 5 core tools (search, context, node,
|
||||
// explore, trace). The 5 omitted tools (callers, callees, impact,
|
||||
// status, files) reduce to one grep at this scale.
|
||||
//
|
||||
// n=2 audits ruled out cutting below 5 tools:
|
||||
// - 3-tool gate (search + context + trace): cost regressed on
|
||||
// cobra/ky/sinatra. The agent fell back to raw Reads to cover
|
||||
// what codegraph_node + codegraph_explore would have answered.
|
||||
// - 1-tool gate (search only): catastrophic regression — express
|
||||
// went from -43% WIN to +107% LOSS. With only search, the agent
|
||||
// can't navigate the call graph structurally and reads everything.
|
||||
//
|
||||
// 5 is the empirical lower bound. Tools beyond search/context/
|
||||
// node/explore/trace pay overhead that the agent doesn't recoup
|
||||
// on tiny-repo flow questions.
|
||||
// ITER4: raise threshold 150 → 500 so single-file frameworks
|
||||
// (sinatra at 159, slim_framework around 200) also get the
|
||||
// 5-tool surface. The empirical 5-tool floor was set on <150
|
||||
// probes; iter3 measurement showed sinatra is structurally the
|
||||
// SAME problem as cobra (single-file WITHOUT-arm Read wins),
|
||||
// so it deserves the same gating.
|
||||
const TINY_REPO_FILE_THRESHOLD = 500;
|
||||
const TINY_REPO_CORE_TOOLS = new Set([
|
||||
'codegraph_search',
|
||||
'codegraph_context',
|
||||
'codegraph_node',
|
||||
'codegraph_explore',
|
||||
'codegraph_trace',
|
||||
]);
|
||||
if (stats.fileCount < TINY_REPO_FILE_THRESHOLD) {
|
||||
visible = visible.filter(t => TINY_REPO_CORE_TOOLS.has(t.name));
|
||||
}
|
||||
|
||||
return visible.map(tool => {
|
||||
if (tool.name === 'codegraph_explore') {
|
||||
return {
|
||||
@@ -928,6 +1018,16 @@ export class ToolHandler {
|
||||
*/
|
||||
async execute(toolName: string, args: Record<string, unknown>): Promise<ToolResult> {
|
||||
try {
|
||||
// Block the first tool call on the engine's post-open reconcile so we
|
||||
// never serve rows for files deleted/edited while no MCP server was
|
||||
// running. The gate is cleared after first await — subsequent calls
|
||||
// pay nothing. Catch-up failures are logged by the engine; we
|
||||
// proceed regardless so a transient sync error never breaks tools.
|
||||
if (this.catchUpGate) {
|
||||
const gate = this.catchUpGate;
|
||||
this.catchUpGate = null;
|
||||
try { await gate; } catch { /* engine already logged */ }
|
||||
}
|
||||
// Honor the optional tool allowlist (CODEGRAPH_MCP_TOOLS): a trimmed
|
||||
// surface rejects ablated tools defensively even if a client cached them.
|
||||
if (!this.isToolAllowed(toolName)) {
|
||||
@@ -1014,7 +1114,16 @@ export class ToolHandler {
|
||||
return this.textResult(`No results found for "${query}"`);
|
||||
}
|
||||
|
||||
const formatted = this.formatSearchResults(results);
|
||||
// Down-rank generated files within the FTS-returned set so a search
|
||||
// for "Send" surfaces the hand-written keeper before .pb.go stubs
|
||||
// that share the name. Stable: only reorders generated vs. not.
|
||||
const ranked = [...results].sort((a, b) => {
|
||||
const aGen = isGeneratedFile(a.node.filePath) ? 1 : 0;
|
||||
const bGen = isGeneratedFile(b.node.filePath) ? 1 : 0;
|
||||
return aGen - bGen;
|
||||
});
|
||||
|
||||
const formatted = this.formatSearchResults(ranked);
|
||||
return this.textResult(this.truncateOutput(formatted));
|
||||
}
|
||||
|
||||
@@ -1032,7 +1141,21 @@ export class ToolHandler {
|
||||
}
|
||||
|
||||
const cg = this.getCodeGraph(args.projectPath as string | undefined);
|
||||
const maxNodes = (args.maxNodes as number) || 20;
|
||||
// On tiny repos (<150 files), trim maxNodes hard — the entire repo
|
||||
// is grep-able in a turn so a 20-node context is wasted budget.
|
||||
// 8 covers the typical 1-3 entry-point + their immediate neighbors
|
||||
// without dragging in the rest of the small codebase.
|
||||
let defaultMaxNodes = 20;
|
||||
let isTinyRepo = false;
|
||||
let isSmallRepo = false;
|
||||
try {
|
||||
const stats = cg.getStats();
|
||||
if (stats.fileCount < 150) { defaultMaxNodes = 8; isTinyRepo = true; }
|
||||
else if (stats.fileCount < 500) { isSmallRepo = true; }
|
||||
} catch {
|
||||
// stats failure — fall back to the standard default
|
||||
}
|
||||
const maxNodes = (args.maxNodes as number) || defaultMaxNodes;
|
||||
const includeCode = args.includeCode !== false;
|
||||
|
||||
const context = await cg.buildContext(task, {
|
||||
@@ -1047,13 +1170,190 @@ export class ToolHandler {
|
||||
? '\n\n⚠️ **Ask user:** UX preferences, edge cases, acceptance criteria'
|
||||
: '';
|
||||
|
||||
// Auto-trace for flow queries: when the task is asking "how does X
|
||||
// reach/flow/propagate from A to B", run the trace internally and
|
||||
// append its body to the context response. Saves the agent the
|
||||
// follow-up codegraph_trace call that was the #2 cost driver on
|
||||
// multi-module flow questions (Q3 / etcd Q2 in the audit).
|
||||
const flowTrace = await this.maybeInlineFlowTrace(task, cg);
|
||||
|
||||
// Iter3 — sufficiency steering on small repos.
|
||||
//
|
||||
// Measured economics on tiny (<150) and small (<500) projects: every
|
||||
// additional MCP tool call costs ~$0.02-0.05 in cache-write tokens
|
||||
// (5K-15K per response at $3.75/1M). The agent reflexively follows
|
||||
// codegraph_context with explore/node even when the context response
|
||||
// is already sufficient — that pattern drove the cost gap that
|
||||
// smaller bodies (iter2) failed to close (smaller bodies just shifted
|
||||
// the agent to Read instead). Direct directive on small-repo
|
||||
// responses: tell the agent the context call IS the comprehensive
|
||||
// pass for a project of this size and that follow-ups should be
|
||||
// narrow (trace from→to, node single-symbol) — not another broad
|
||||
// explore that re-bundles the same content.
|
||||
// ITER4: unified strong directive for both tiny (<150) and small
|
||||
// (<500) tiers — measured iter3 result was that the soft <500
|
||||
// wording was IGNORED on sinatra (5 tool calls, +92% loss) while
|
||||
// the strong <150 wording was followed on cobra/slim (3 calls,
|
||||
// -21%/-22% wins). The single-file-framework problem (sinatra)
|
||||
// is structurally the same as cobra's; both deserve the same
|
||||
// sufficiency steering.
|
||||
let smallRepoTail = '';
|
||||
let smallRepoRouteInline = '';
|
||||
if (isTinyRepo || isSmallRepo) {
|
||||
// Iter12: backend-computed routing manifest for routing queries.
|
||||
// Builds a URL → handler map directly from the graph (each route
|
||||
// node has a `references` edge to its handler), then inlines the
|
||||
// top handler file's source. The agent gets the canonical
|
||||
// routing answer in one MCP call — no need to parse framework
|
||||
// DSL or grep for handlers.
|
||||
//
|
||||
// Replaces iter10's raw route-file inline. The manifest is more
|
||||
// information-dense (parsed URL→handler map vs raw config DSL)
|
||||
// and we still inline the top handler file's source so the agent
|
||||
// has the implementation bodies inline too.
|
||||
const isRouteQuery = /\b(route|routes|routing|request|handler|endpoint|api|controller|middleware|dispatch|invok)/i.test(task);
|
||||
if (isRouteQuery) {
|
||||
try {
|
||||
const manifest = cg.getRoutingManifest(40);
|
||||
if (manifest) {
|
||||
// 1) Compact URL→handler list (~30-60 lines, ~1-2KB).
|
||||
const lines: string[] = [
|
||||
`\n\n## Routing manifest (${manifest.totalRoutes} routes, top handler file holds ${manifest.topHandlerFileCount})`,
|
||||
'',
|
||||
'| URL | Handler | Location |',
|
||||
'|---|---|---|',
|
||||
];
|
||||
for (const e of manifest.entries) {
|
||||
lines.push(`| \`${e.url}\` | \`${e.handler}\` | ${e.handlerFile}:${e.handlerLine} |`);
|
||||
}
|
||||
// 2) Inline the top handler file's source.
|
||||
if (manifest.topHandlerFile && manifest.topHandlerFileCount >= 2) {
|
||||
try {
|
||||
const fullPath = pathModule.join(cg.getProjectRoot(), manifest.topHandlerFile);
|
||||
const stat = statSync(fullPath);
|
||||
if (stat.size > 0 && stat.size <= 16000) {
|
||||
const source = readFileSync(fullPath, 'utf-8');
|
||||
const capped = source.length > 7000 ? source.slice(0, 7000) + '\n... (truncated)' : source;
|
||||
const ext = (manifest.topHandlerFile.match(/\.([a-z]+)$/i)?.[1] || '').toLowerCase();
|
||||
const lang =
|
||||
ext === 'rb' ? 'ruby' : ext === 'py' ? 'python' :
|
||||
ext === 'go' ? 'go' : ext === 'rs' ? 'rust' :
|
||||
ext === 'js' || ext === 'jsx' ? 'javascript' :
|
||||
ext === 'ts' || ext === 'tsx' ? 'typescript' :
|
||||
ext === 'java' ? 'java' : ext === 'kt' ? 'kotlin' :
|
||||
ext === 'cs' ? 'csharp' : ext === 'php' ? 'php' :
|
||||
ext === 'swift' ? 'swift' : ext === 'yml' || ext === 'yaml' ? 'yaml' : '';
|
||||
lines.push('');
|
||||
lines.push(`### Top handler file (\`${manifest.topHandlerFile}\` — ${manifest.topHandlerFileCount}/${manifest.totalRoutes} routes, full source inlined — do NOT Read)`);
|
||||
lines.push('');
|
||||
lines.push('```' + lang);
|
||||
lines.push(capped);
|
||||
lines.push('```');
|
||||
}
|
||||
} catch { /* file read failed, skip the source inline */ }
|
||||
}
|
||||
smallRepoRouteInline = lines.join('\n');
|
||||
}
|
||||
} catch {
|
||||
// Manifest build failed — drop silently
|
||||
}
|
||||
}
|
||||
const sizeQualifier = isTinyRepo ? 'under 150' : 'under 500';
|
||||
const routingClause = smallRepoRouteInline
|
||||
? ' The URL→handler manifest and top handler file are also inlined above — answer routing questions from them.'
|
||||
: '';
|
||||
smallRepoTail = `\n\n---\n> **This project is small** (${sizeQualifier} indexed files). The entry points and code above cover the relevant surface — **do NOT call codegraph_explore as a follow-up; its content will largely duplicate this response**. If you need a specific flow, call \`codegraph_trace from→to\`. If you need one specific symbol's body, call \`codegraph_node <name>\`.${routingClause} Otherwise, answer from what is above.`;
|
||||
}
|
||||
|
||||
// buildContext returns string when format is 'markdown'
|
||||
if (typeof context === 'string') {
|
||||
return this.textResult(this.truncateOutput(context + reminder));
|
||||
return this.textResult(this.truncateOutput(context + flowTrace + reminder + smallRepoRouteInline + smallRepoTail));
|
||||
}
|
||||
|
||||
// If it returns TaskContext, format it
|
||||
return this.textResult(this.truncateOutput(this.formatTaskContext(context) + reminder));
|
||||
return this.textResult(this.truncateOutput(this.formatTaskContext(context) + flowTrace + reminder + smallRepoRouteInline + smallRepoTail));
|
||||
}
|
||||
|
||||
/**
|
||||
* Detect a flow-style task ("how does X reach Y", "trace the path from A to B")
|
||||
* and pre-run trace between the most likely endpoints, returning the trace
|
||||
* body to splice into the context response. Returns '' for non-flow queries
|
||||
* or when no plausible endpoint pair can be extracted.
|
||||
*
|
||||
* Conservative by design: only fires when the task has both a clear flow
|
||||
* keyword AND at least two distinct PascalCase / camelCase identifiers.
|
||||
* False positives waste a graph query; false negatives just fall back to
|
||||
* the agent calling trace itself (existing path-proximity wiring handles
|
||||
* disambiguation either way).
|
||||
*/
|
||||
private async maybeInlineFlowTrace(task: string, cg: CodeGraph): Promise<string> {
|
||||
const lower = task.toLowerCase();
|
||||
const FLOW_KEYWORDS = [
|
||||
'trace ',
|
||||
'from ',
|
||||
'reach ',
|
||||
'flow ',
|
||||
'propagat',
|
||||
'how does ',
|
||||
'how do ',
|
||||
];
|
||||
if (!FLOW_KEYWORDS.some((k) => lower.includes(k))) return '';
|
||||
|
||||
// Extract candidate symbols — PascalCase or camelCase identifiers ≥3 chars.
|
||||
// Filter out common non-symbol words and the flow keywords themselves.
|
||||
const STOP_WORDS = new Set([
|
||||
'how', 'does', 'the', 'and', 'from', 'through', 'reach', 'reaches',
|
||||
'flow', 'path', 'trace', 'cross', 'module', 'modules', 'where',
|
||||
'update', 'updates', 'updated', 'when', 'what', 'this', 'that',
|
||||
]);
|
||||
const ids: string[] = [];
|
||||
const seen = new Set<string>();
|
||||
const re = /\b([A-Z][a-z]+(?:[A-Z][a-z]*)+|[a-z]+[A-Z][a-z]*(?:[A-Z][a-z]*)*)\b/g;
|
||||
let m: RegExpExecArray | null;
|
||||
while ((m = re.exec(task)) !== null) {
|
||||
const sym = m[1]!;
|
||||
if (sym.length < 3) continue;
|
||||
const key = sym.toLowerCase();
|
||||
if (STOP_WORDS.has(key) || seen.has(key)) continue;
|
||||
seen.add(key);
|
||||
ids.push(sym);
|
||||
}
|
||||
|
||||
if (ids.length < 2) return '';
|
||||
|
||||
// The first two distinct symbols, in order of appearance, are the most
|
||||
// likely from/to endpoints — "from X ... through to Y" naturally places
|
||||
// them in that order in the prose. If the trace fails to connect, it
|
||||
// still returns the inlined endpoint bodies (the trace-failure rewrite).
|
||||
const fromSym = ids[0]!;
|
||||
const toSym = ids[1]!;
|
||||
|
||||
let traceResult: ToolResult;
|
||||
try {
|
||||
traceResult = await this.handleTrace({
|
||||
from: fromSym,
|
||||
to: toSym,
|
||||
projectPath: cg.getProjectRoot(),
|
||||
} as Record<string, unknown>);
|
||||
} catch {
|
||||
return '';
|
||||
}
|
||||
// Extract the textual body. Defensive: handleTrace's contract is the
|
||||
// standard tool-result shape used elsewhere in this file.
|
||||
const body = traceResult.content
|
||||
?.map((c) => (c.type === 'text' ? c.text : ''))
|
||||
.filter(Boolean)
|
||||
.join('\n')
|
||||
.trim();
|
||||
if (!body) return '';
|
||||
return [
|
||||
'',
|
||||
'## Inline flow trace',
|
||||
'',
|
||||
`Auto-traced \`${fromSym}\` → \`${toSym}\` because the query looks like a flow question. No follow-up codegraph_trace is needed for this pair.`,
|
||||
'',
|
||||
body,
|
||||
].join('\n');
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -1232,41 +1532,185 @@ export class ToolHandler {
|
||||
// (which, on real code, means the flow breaks at dynamic dispatch).
|
||||
const edgeKinds: Edge['kind'][] = ['calls'];
|
||||
const MAX_HOPS = 7;
|
||||
const fromTry = fromMatches.nodes.slice(0, 3);
|
||||
const toTry = toMatches.nodes.slice(0, 3);
|
||||
// Path-proximity pairing: in a multi-module repo a symbol name like
|
||||
// `EndBlocker` exists in 20+ modules. FTS picks one almost arbitrarily;
|
||||
// the WRONG pair (e.g. simapp's wrapper EndBlocker paired with gov's Tally)
|
||||
// has no static path, falls through to the dynamic-dispatch failure branch,
|
||||
// and surfaces unrelated bodies — exactly the cosmos-Q3 trace failure mode.
|
||||
// Score every from×to combo by shared file-path prefix length; try the
|
||||
// most-co-located pair first (e.g. `x/gov/abci.go::EndBlocker` ×
|
||||
// `x/gov/keeper/tally.go::Tally` share `x/gov/`).
|
||||
//
|
||||
// Consider the FULL candidate set, not just the FTS top-5: the right
|
||||
// EndBlocker for a gov-module flow may rank 8th in FTS but share the
|
||||
// entire `x/gov/` prefix with the destination. Path-proximity supersedes
|
||||
// FTS for this disambiguation. Findpath trials are still capped by
|
||||
// FINDPATH_PAIR_BUDGET below to bound graph traversal cost.
|
||||
const sharedDirPrefixLen = (a: string, b: string): number => {
|
||||
const aDir = a.replace(/[^/]+$/, '');
|
||||
const bDir = b.replace(/[^/]+$/, '');
|
||||
let i = 0;
|
||||
while (i < aDir.length && i < bDir.length && aDir[i] === bDir[i]) i++;
|
||||
return i;
|
||||
};
|
||||
// Cosmos-Q3 surfaced a second-order failure: `enterprise/group/x/group/`
|
||||
// SHARES MORE of its path with `enterprise/group/x/group/keeper/tally.go`
|
||||
// (24 chars) than `x/gov/abci.go` shares with `x/gov/keeper/tally.go`
|
||||
// (6 chars), so pure shared-prefix prefers the side-experiment module
|
||||
// over the canonical one — even though the user's question is clearly
|
||||
// about the main gov module. Penalize candidates living under prefixes
|
||||
// that conventionally hold extensions / experiments / vendored code, so
|
||||
// the canonical-path pair wins even when its shared prefix is short.
|
||||
const isLessCanonicalPath = (p: string): boolean =>
|
||||
/^(enterprise|contrib|examples?|sample|playground|vendor|third[_-]?party|deprecated|legacy)\//i.test(p);
|
||||
const LESS_CANONICAL_PENALTY = 100; // any canonical candidate beats any less-canonical one
|
||||
const scorePair = (a: string, b: string): number =>
|
||||
sharedDirPrefixLen(a, b)
|
||||
- (isLessCanonicalPath(a) ? LESS_CANONICAL_PENALTY : 0)
|
||||
- (isLessCanonicalPath(b) ? LESS_CANONICAL_PENALTY : 0);
|
||||
const fromCands = fromMatches.nodes;
|
||||
const toCands = toMatches.nodes;
|
||||
const pairs: Array<{ f: Node; t: Node; score: number }> = [];
|
||||
for (const f of fromCands) {
|
||||
for (const t of toCands) {
|
||||
pairs.push({ f, t, score: scorePair(f.filePath, t.filePath) });
|
||||
}
|
||||
}
|
||||
// Sort by shared prefix desc, then by FTS order (already encoded in the
|
||||
// pairs' insertion order — both for f and t). The tiebreaker preserves
|
||||
// findAllSymbols' generated-file-last ranking.
|
||||
pairs.sort((a, b) => b.score - a.score);
|
||||
// Cap how many graph-path probes we attempt so a 50×50 cross-product
|
||||
// doesn't blow up on a god-named symbol like `Get` (well-named flows have
|
||||
// their good pair near the top of the sort anyway).
|
||||
const FINDPATH_PAIR_BUDGET = 20;
|
||||
const fromTry = fromCands;
|
||||
const toTry = toCands;
|
||||
let path: Array<{ node: Node; edge: Edge | null }> | null = null;
|
||||
let overCap: Array<{ node: Node; edge: Edge | null }> | null = null;
|
||||
for (const f of fromTry) {
|
||||
for (const t of toTry) {
|
||||
const p = cg.findPath(f.id, t.id, edgeKinds);
|
||||
if (!p || p.length <= 1) continue;
|
||||
if (p.length <= MAX_HOPS) { path = p; break; }
|
||||
if (!overCap || p.length < overCap.length) overCap = p;
|
||||
}
|
||||
let bestPair: { f: Node; t: Node } | null = null;
|
||||
let triedPairs = 0;
|
||||
for (const { f, t } of pairs) {
|
||||
if (path) break;
|
||||
if (triedPairs >= FINDPATH_PAIR_BUDGET) break;
|
||||
triedPairs++;
|
||||
const p = cg.findPath(f.id, t.id, edgeKinds);
|
||||
if (p && p.length > 1) {
|
||||
if (p.length <= MAX_HOPS) { path = p; bestPair = { f, t }; break; }
|
||||
if (!overCap || p.length < overCap.length) { overCap = p; bestPair = { f, t }; }
|
||||
} else if (!bestPair) {
|
||||
// No path yet — remember the top-scored pair so the failure branch
|
||||
// surfaces the most-co-located candidates' bodies, not whatever FTS
|
||||
// happened to put first.
|
||||
bestPair = { f, t };
|
||||
}
|
||||
}
|
||||
|
||||
if (!path) {
|
||||
// No static path — almost always a dynamic-dispatch break. Surface the
|
||||
// start symbol's outgoing calls so the agent can bridge the gap.
|
||||
const start = fromTry[0]!;
|
||||
const callees = cg.getCallees(start.id).slice(0, 10)
|
||||
.map(c => `${c.node.name} (${c.node.filePath}:${c.node.startLine})`);
|
||||
// No static path — almost always a dynamic-dispatch break. INSTEAD of
|
||||
// telling the agent to chase the gap with codegraph_node/callers/callees
|
||||
// (which fans out into 3-4 follow-up tool calls + a Read), inline the
|
||||
// material those would have returned right here. Measured on cosmos-Q3:
|
||||
// the failed-trace + subsequent fan-out used to cost ~2× a single
|
||||
// sufficient trace call; this branch closes that gap.
|
||||
// Prefer the path-proximity-best pair we identified above (e.g. gov's
|
||||
// EndBlocker × gov's Tally) over the FTS top-pick (simapp's wrapper).
|
||||
const start = bestPair?.f ?? fromTry[0]!;
|
||||
const end = bestPair?.t ?? toTry[0]!;
|
||||
const fileCache = new Map<string, string[]>();
|
||||
const lines = [
|
||||
`No direct call path from "${from}" to "${to}".`,
|
||||
`No direct static call path from "${from}" to "${to}" — the chain almost certainly breaks at dynamic dispatch (a callback / interface dispatch / framework hook / metaclass). Both endpoint bodies + their immediate neighbors are inlined below; answer from them — a follow-up codegraph_node/callers/callees on these would just return what is already here.`,
|
||||
'',
|
||||
(overCap
|
||||
? `(Only a ${overCap.length}-hop indirect chain connects them — almost certainly a BFS wander through unrelated code, not the real flow.) `
|
||||
: '') +
|
||||
'The direct chain most likely breaks at **dynamic dispatch** (a callback, descriptor, ' +
|
||||
'metaclass, or attribute-as-callable) that static parsing cannot resolve into an edge. ' +
|
||||
`Inspect \`${start.name}\` (${start.filePath}:${start.startLine}) with codegraph_node ` +
|
||||
'(includeCode=true) — its body usually shows the dynamic call to follow next.',
|
||||
];
|
||||
if (callees.length > 0) {
|
||||
lines.push('', `**${start.name} statically calls:** ${callees.join(', ')}`);
|
||||
if (overCap) {
|
||||
lines.push(
|
||||
`> Indirect chain of ${overCap.length} hops exists but is over the ${MAX_HOPS}-hop cap (usually a BFS wander through unrelated code, not the real execution flow).`,
|
||||
'',
|
||||
);
|
||||
}
|
||||
return this.textResult(lines.join('\n') + fromMatches.note + toMatches.note);
|
||||
|
||||
// Track which node IDs we've already inlined a body for so we don't
|
||||
// double-emit when a callee of FROM is also surfaced separately.
|
||||
const inlinedBodies = new Set<string>();
|
||||
const inlineBody = (n: Node, lineCap: number, charCap: number): boolean => {
|
||||
if (inlinedBodies.has(n.id)) return false;
|
||||
inlinedBodies.add(n.id);
|
||||
const body = this.sourceRangeAt(cg, n.filePath, n.startLine, n.endLine, fileCache, lineCap, charCap);
|
||||
if (body) { lines.push(body); return true; }
|
||||
return false;
|
||||
};
|
||||
|
||||
const inlineEndpoint = (
|
||||
label: 'FROM' | 'TO',
|
||||
node: Node,
|
||||
) => {
|
||||
lines.push(`### ${label}: \`${node.name}\` (${node.filePath}:${node.startLine}-${node.endLine})`);
|
||||
inlineBody(node, 120, 3600);
|
||||
const callers = cg.getCallers(node.id).slice(0, 6);
|
||||
if (callers.length > 0) {
|
||||
lines.push(`**Callers of \`${node.name}\`:** ` +
|
||||
callers.map(c => `${c.node.name} (${c.node.filePath}:${c.node.startLine})`).join(', '));
|
||||
}
|
||||
const callees = cg.getCallees(node.id).slice(0, 8);
|
||||
if (callees.length > 0) {
|
||||
lines.push(`**\`${node.name}\` calls:** ` +
|
||||
callees.map(c => `${c.node.name} (${c.node.filePath}:${c.node.startLine})`).join(', '));
|
||||
}
|
||||
lines.push('');
|
||||
};
|
||||
inlineEndpoint('FROM', start);
|
||||
if (end.id !== start.id) inlineEndpoint('TO', end);
|
||||
|
||||
// Inline the OTHER top-level functions/methods in TO's file — that's
|
||||
// where the missing dynamic-dispatch flow usually lives. Concrete
|
||||
// measurement from cosmos-Q1: `msgServer.Send` statically calls only
|
||||
// utility functions (`StringToBytes`, `Wrapf`); its real next-hop
|
||||
// `SendCoins` is invoked via an embedded-interface call (`k.Keeper.SendCoins`)
|
||||
// that static parsing CAN'T see. The flow IS in the same file as the
|
||||
// destination (`x/bank/keeper/send.go`: SendCoins → subUnlockedCoins →
|
||||
// addCoins → setBalance). Pre-inlining those file-mates is what
|
||||
// replaces the agent's "trace fail → search SendCoins → node SendCoins
|
||||
// → trace again" fan-out.
|
||||
const NEIGHBOR_LINES = 40;
|
||||
const NEIGHBOR_CHARS = 1200;
|
||||
const NEIGHBOR_K = 5;
|
||||
const fileSiblings = (anchor: Node): Node[] => {
|
||||
// Functions and methods in the same file as the anchor, excluding
|
||||
// the anchor itself and anything we've already inlined. Sort by
|
||||
// distance from the anchor's startLine so the closest symbols come
|
||||
// first (the flow is usually adjacent in the file).
|
||||
const sameFile = cg
|
||||
.getNodesByKind('function')
|
||||
.filter((n) => n.filePath === anchor.filePath)
|
||||
.concat(
|
||||
cg.getNodesByKind('method').filter((n) => n.filePath === anchor.filePath),
|
||||
);
|
||||
return sameFile
|
||||
.filter((n) => n.id !== anchor.id && !inlinedBodies.has(n.id))
|
||||
.sort((a, b) =>
|
||||
Math.abs(a.startLine - anchor.startLine) - Math.abs(b.startLine - anchor.startLine),
|
||||
)
|
||||
.slice(0, NEIGHBOR_K);
|
||||
};
|
||||
const renderSiblings = (label: string, siblings: Node[]) => {
|
||||
if (siblings.length === 0) return;
|
||||
lines.push(`### ${label}`);
|
||||
for (const sib of siblings) {
|
||||
lines.push('');
|
||||
lines.push(`- \`${sib.name}\` (${sib.filePath}:${sib.startLine}-${sib.endLine})`);
|
||||
inlineBody(sib, NEIGHBOR_LINES, NEIGHBOR_CHARS);
|
||||
}
|
||||
lines.push('');
|
||||
};
|
||||
renderSiblings(
|
||||
`Other functions in \`${end.filePath}\` (the flow that the dynamic-dispatch hop reaches — bodies inlined)`,
|
||||
fileSiblings(end),
|
||||
);
|
||||
|
||||
lines.push(
|
||||
'> Endpoint bodies + the other functions in the destination\'s file are inlined above. Together they typically cover the missing dynamic-dispatch boundary (interface-method calls like `k.Keeper.SendCoins` that static parsing can\'t follow). **No further codegraph_node / codegraph_callers / codegraph_callees / Read / Grep is needed for any symbol already shown here** — call them again only if you need to walk DEEPER than what is inlined.',
|
||||
);
|
||||
return this.textResult(this.truncateOutput(lines.join('\n') + fromMatches.note + toMatches.note));
|
||||
}
|
||||
|
||||
const lines: string[] = [
|
||||
@@ -1649,11 +2093,52 @@ export class ToolHandler {
|
||||
}
|
||||
|
||||
// Only include files that have entry points or nodes directly connected to entry points
|
||||
const relevantFiles = [...fileGroups.entries()].filter(([, group]) => group.score >= 3);
|
||||
let relevantFiles = [...fileGroups.entries()].filter(([, group]) => group.score >= 3);
|
||||
|
||||
// Extract query terms for relevance checking
|
||||
const queryTerms = query.toLowerCase().split(/\s+/).filter(t => t.length >= 3);
|
||||
|
||||
// Test/spec/icon/i18n file detector — used both for the pre-sort hard
|
||||
// filter (tiny tier) and the comparator deprioritization (all tiers).
|
||||
const isLowValue = (p: string) => {
|
||||
const lp = p.toLowerCase();
|
||||
return (
|
||||
/\/(tests?|__tests?__|spec)\//.test(lp) ||
|
||||
/_test\.go$/.test(lp) ||
|
||||
/(?:^|\/)test_[^/]+\.py$/.test(lp) ||
|
||||
/_test\.py$/.test(lp) ||
|
||||
/_spec\.rb$/.test(lp) ||
|
||||
/_test\.rb$/.test(lp) ||
|
||||
/\.(test|spec)\.[jt]sx?$/.test(lp) ||
|
||||
/(test|spec|tests)\.(java|kt|scala)$/.test(lp) ||
|
||||
/(tests?|spec)\.cs$/.test(lp) ||
|
||||
/tests?\.swift$/.test(lp) ||
|
||||
/_test\.dart$/.test(lp) ||
|
||||
/\bicons?\b/.test(lp) ||
|
||||
/\bi18n\b/.test(lp)
|
||||
);
|
||||
};
|
||||
|
||||
// Tiny-tier hard-exclude: on small projects (`excludeLowValueFiles`
|
||||
// budget flag), one slipped test/spec file dominates the per-file budget
|
||||
// (cobra's `command_test.go` displaced `args.go` and contributed ~10KB of
|
||||
// pure noise to "How does cobra parse commands?"). The sort-step
|
||||
// deprioritization isn't enough at small N. Skip the hard-exclude when
|
||||
// the query itself is about tests — that's the legitimate "explore the
|
||||
// tests" case where the agent does want them.
|
||||
if (budget.excludeLowValueFiles) {
|
||||
const queryMentionsTests = /\b(test|tests|testing|spec|verify|verifies)\b/i.test(query);
|
||||
if (!queryMentionsTests) {
|
||||
const nonLow = relevantFiles.filter(([p]) => !isLowValue(p));
|
||||
// Only apply the hard-filter if we still have at least 2 non-test
|
||||
// candidates after the cut — otherwise the agent is asking about an
|
||||
// area where tests are the only signal, and we should not strip them.
|
||||
if (nonLow.length >= 2) {
|
||||
relevantFiles = nonLow;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Sort files: highest relevance first, deprioritize low-value files
|
||||
const sortedFiles = relevantFiles.sort((a, b) => {
|
||||
const aPath = a[0].toLowerCase();
|
||||
@@ -1670,15 +2155,20 @@ export class ToolHandler {
|
||||
const bRelevant = hasQueryRelevance(bPath, b[1].nodes);
|
||||
if (aRelevant !== bRelevant) return aRelevant ? -1 : 1;
|
||||
|
||||
// Deprioritize test files, icon files, and i18n files
|
||||
const isLowValue = (p: string) =>
|
||||
/\/(tests?|__tests?__|spec)\//i.test(p) ||
|
||||
/\bicons?\b/i.test(p) ||
|
||||
/\bi18n\b/i.test(p);
|
||||
const aLow = isLowValue(aPath);
|
||||
const bLow = isLowValue(bPath);
|
||||
if (aLow !== bLow) return aLow ? 1 : -1;
|
||||
|
||||
// Deprioritize generated source (.pb.go / .pulsar.go / _mocks.go / …) —
|
||||
// the agent rarely needs to see the protobuf scaffold or gomock output
|
||||
// when asking about the actual flow, and dumping their bodies inflates
|
||||
// the response (the cosmos Q3 explore otherwise leads with
|
||||
// `expected_keepers_mocks.go`, displacing the real `tally.go` content
|
||||
// and forcing the agent to Read tally.go anyway).
|
||||
const aGen = isGeneratedFile(a[0]);
|
||||
const bGen = isGeneratedFile(b[0]);
|
||||
if (aGen !== bGen) return aGen ? 1 : -1;
|
||||
|
||||
if (a[1].score !== b[1].score) return b[1].score - a[1].score;
|
||||
return b[1].nodes.length - a[1].nodes.length;
|
||||
});
|
||||
@@ -2519,12 +3009,21 @@ export class ToolHandler {
|
||||
}
|
||||
|
||||
if (exactMatches.length > 1) {
|
||||
// Down-rank generated files (.pb.go, .pulsar.go, _grpc.pb.go, …)
|
||||
// so a query like "Send" prefers the keeper implementation over
|
||||
// the protobuf-generated interface stub. Stable sort preserves
|
||||
// FTS order within each group. See generated-detection.ts.
|
||||
const ranked = [...exactMatches].sort((a, b) => {
|
||||
const aGen = isGeneratedFile(a.node.filePath) ? 1 : 0;
|
||||
const bGen = isGeneratedFile(b.node.filePath) ? 1 : 0;
|
||||
return aGen - bGen;
|
||||
});
|
||||
// Multiple exact matches - pick first, note the others
|
||||
const picked = exactMatches[0]!.node;
|
||||
const others = exactMatches.slice(1).map(r =>
|
||||
const picked = ranked[0]!.node;
|
||||
const others = ranked.slice(1).map(r =>
|
||||
`${r.node.name} (${r.node.kind}) at ${r.node.filePath}:${r.node.startLine}`
|
||||
);
|
||||
const note = `\n\n> **Note:** ${exactMatches.length} symbols named "${symbol}". Showing results for \`${picked.filePath}:${picked.startLine}\`. Others: ${others.join(', ')}`;
|
||||
const note = `\n\n> **Note:** ${ranked.length} symbols named "${symbol}". Showing results for \`${picked.filePath}:${picked.startLine}\`. Others: ${others.join(', ')}`;
|
||||
return { node: picked, note };
|
||||
}
|
||||
|
||||
@@ -2562,11 +3061,20 @@ export class ToolHandler {
|
||||
return { nodes: [node], note: '' };
|
||||
}
|
||||
|
||||
const locations = exactMatches.map(r =>
|
||||
// Same generated-file down-rank as findSymbol — keeps callers/callees
|
||||
// /impact aggregation aligned (a query against "Send" returns the
|
||||
// hand-written implementations before the protobuf scaffold).
|
||||
const ranked = [...exactMatches].sort((a, b) => {
|
||||
const aGen = isGeneratedFile(a.node.filePath) ? 1 : 0;
|
||||
const bGen = isGeneratedFile(b.node.filePath) ? 1 : 0;
|
||||
return aGen - bGen;
|
||||
});
|
||||
|
||||
const locations = ranked.map(r =>
|
||||
`${r.node.kind} at ${r.node.filePath}:${r.node.startLine}`
|
||||
);
|
||||
const note = `\n\n> **Note:** Aggregated results across ${exactMatches.length} symbols named "${symbol}": ${locations.join(', ')}`;
|
||||
return { nodes: exactMatches.map(r => r.node), note };
|
||||
const note = `\n\n> **Note:** Aggregated results across ${ranked.length} symbols named "${symbol}": ${locations.join(', ')}`;
|
||||
return { nodes: ranked.map(r => r.node), note };
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -24,6 +24,7 @@
|
||||
import type { Edge, Node } from '../types';
|
||||
import type { QueryBuilder } from '../db/queries';
|
||||
import type { ResolutionContext } from './types';
|
||||
import { isGeneratedFile } from '../extraction/generated-detection';
|
||||
|
||||
const REGISTRAR_NAME = /^(on[A-Z]\w*|subscribe|addListener|addEventListener|register|watch|listen|addCallback)$/;
|
||||
const DISPATCHER_NAME = /(emit|trigger|notify|dispatch|fire|publish|flush)/i;
|
||||
@@ -337,7 +338,16 @@ function cppOverrideEdges(queries: QueryBuilder): Edge[] {
|
||||
* trace/callees reach the implementation. Over-approximation accepted
|
||||
* (reachability-correct); capped per class, gated to JVM languages.
|
||||
*/
|
||||
const IFACE_OVERRIDE_LANGS = new Set(['java', 'kotlin']);
|
||||
// Languages whose static `implements`/`extends` edges should bridge an
|
||||
// interface (or abstract base) method to the matching concrete-class method.
|
||||
// The set is "languages with explicit nominal subtyping and a single class
|
||||
// kind that holds methods" — i.e. the shape this loop expects. Swift and
|
||||
// Scala fit shape-wise (Swift `protocol`/`class`, Scala `trait`/`class`)
|
||||
// and are added below; their concrete-side nodes can be a `struct` (Swift)
|
||||
// or an `object` (Scala) so the loop also iterates those kinds.
|
||||
const IFACE_OVERRIDE_LANGS = new Set([
|
||||
'java', 'kotlin', 'csharp', 'typescript', 'javascript', 'swift', 'scala',
|
||||
]);
|
||||
function interfaceOverrideEdges(queries: QueryBuilder): Edge[] {
|
||||
const edges: Edge[] = [];
|
||||
const seen = new Set<string>();
|
||||
@@ -346,7 +356,12 @@ function interfaceOverrideEdges(queries: QueryBuilder): Edge[] {
|
||||
.getOutgoingEdges(classId, ['contains'])
|
||||
.map((e) => queries.getNodeById(e.target))
|
||||
.filter((n): n is Node => !!n && n.kind === 'method');
|
||||
for (const cls of queries.getNodesByKind('class')) {
|
||||
// Concrete-side kinds vary by language: `class` covers Java / Kotlin /
|
||||
// C# / TS / Swift-classes / Scala-classes; `struct` covers Swift value
|
||||
// types that conform to protocols. Iterate both.
|
||||
const concreteKinds = ['class', 'struct'] as const;
|
||||
for (const kind of concreteKinds) {
|
||||
for (const cls of queries.getNodesByKind(kind)) {
|
||||
const implMethods = methodsOf(cls.id).filter((n) => IFACE_OVERRIDE_LANGS.has(n.language));
|
||||
if (implMethods.length === 0) continue;
|
||||
for (const sup of queries.getOutgoingEdges(cls.id, ['implements', 'extends'])) {
|
||||
@@ -383,6 +398,116 @@ function interfaceOverrideEdges(queries: QueryBuilder): Edge[] {
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return edges;
|
||||
}
|
||||
|
||||
/**
|
||||
* Go gRPC stub → impl bridge. The protoc-gen-go-grpc codegen emits an
|
||||
* `UnimplementedXxxServer` struct in `*_grpc.pb.go` carrying one method
|
||||
* per service RPC; the real handler is a hand-written struct in another
|
||||
* file (`x/bank/keeper/msg_server.go::msgServer.Send` in cosmos-sdk).
|
||||
* Go's structural typing means no `implements` edge exists for our
|
||||
* resolver to follow, so `trace("Send","SendCoins")` lands on the
|
||||
* empty stub and reports "no path" (validated empirically — the cosmos
|
||||
* Q1 r1 trace failure that drove this work).
|
||||
*
|
||||
* Bridge: for each `UnimplementedXxxServer` whose RPC-method names are
|
||||
* a SUBSET of some other Go struct's method names, emit `calls` edges
|
||||
* `stub.method → impl.method` (paired by name). Excludes the gRPC
|
||||
* internal markers `mustEmbedUnimplementedXxxServer` and
|
||||
* `testEmbeddedByValue`, and skips candidate impls that themselves
|
||||
* live in a generated file (their `xxxClient` / sibling stubs would
|
||||
* otherwise look like impls).
|
||||
*
|
||||
* Multiple candidates is allowed and capped at MAX_CALLBACKS_PER_CHANNEL —
|
||||
* a service often has both a production impl and one or more test
|
||||
* mocks; linking to all preserves trace utility without false-favoring.
|
||||
*
|
||||
* Provenance: `heuristic`, `synthesizedBy: 'go-grpc-stub-impl'`. The
|
||||
* stub's source line is the wiring site shown in the trace trail.
|
||||
*/
|
||||
function goGrpcStubImplEdges(queries: QueryBuilder): Edge[] {
|
||||
const edges: Edge[] = [];
|
||||
const seen = new Set<string>();
|
||||
|
||||
const STUB_RE = /^Unimplemented.*Server$/;
|
||||
// gRPC internal-helper methods that appear on every Unimplemented*Server;
|
||||
// not part of the service contract, so exclude when computing the RPC-method
|
||||
// signature used to match impls.
|
||||
const isInternalMarker = (n: string) => n.startsWith('mustEmbed') || n === 'testEmbeddedByValue';
|
||||
|
||||
// Methods directly contained by each Go struct, name-only. Built once.
|
||||
const methodNamesByStruct = new Map<string, Set<string>>();
|
||||
const methodNodesByStruct = new Map<string, Node[]>();
|
||||
const goStructs: Node[] = [];
|
||||
for (const s of queries.getNodesByKind('struct')) {
|
||||
if (s.language !== 'go') continue;
|
||||
goStructs.push(s);
|
||||
const ms = queries
|
||||
.getOutgoingEdges(s.id, ['contains'])
|
||||
.map((e) => queries.getNodeById(e.target))
|
||||
.filter((n): n is Node => !!n && n.kind === 'method');
|
||||
methodNodesByStruct.set(s.id, ms);
|
||||
methodNamesByStruct.set(s.id, new Set(ms.map((m) => m.name)));
|
||||
}
|
||||
|
||||
for (const stub of goStructs) {
|
||||
if (!STUB_RE.test(stub.name)) continue;
|
||||
// The stub MUST live in a generated file — that's what tells us this is
|
||||
// a protoc-emitted scaffold rather than someone naming a struct
|
||||
// `UnimplementedXxxServer` by hand. Without this gate we'd also bridge
|
||||
// such hand-written structs and create misleading edges.
|
||||
if (!isGeneratedFile(stub.filePath)) continue;
|
||||
|
||||
const stubMethods = (methodNodesByStruct.get(stub.id) ?? []).filter(
|
||||
(m) => !isInternalMarker(m.name),
|
||||
);
|
||||
if (stubMethods.length === 0) continue;
|
||||
const stubMethodNames = stubMethods.map((m) => m.name);
|
||||
|
||||
for (const cand of goStructs) {
|
||||
if (cand.id === stub.id) continue;
|
||||
// Skip generated-file candidates — they're siblings (msgClient,
|
||||
// UnsafeMsgServer, …) whose method sets coincidentally match.
|
||||
if (isGeneratedFile(cand.filePath)) continue;
|
||||
|
||||
const candNames = methodNamesByStruct.get(cand.id);
|
||||
if (!candNames) continue;
|
||||
// Subset: every RPC method must exist on the candidate by name.
|
||||
// Signature-level match would tighten this further, but name-match
|
||||
// alone already gives one-to-one pairing in real codebases because
|
||||
// gRPC method-name sets are highly distinctive (Send + MultiSend +
|
||||
// UpdateParams + SetSendEnabled is unique to bank's MsgServer).
|
||||
if (!stubMethodNames.every((n) => candNames.has(n))) continue;
|
||||
|
||||
const candMethods = methodNodesByStruct.get(cand.id) ?? [];
|
||||
let added = 0;
|
||||
for (const sm of stubMethods) {
|
||||
if (added >= MAX_CALLBACKS_PER_CHANNEL) break;
|
||||
for (const cm of candMethods) {
|
||||
if (added >= MAX_CALLBACKS_PER_CHANNEL) break;
|
||||
if (cm.name !== sm.name) continue;
|
||||
const key = `${sm.id}>${cm.id}`;
|
||||
if (seen.has(key)) continue;
|
||||
seen.add(key);
|
||||
edges.push({
|
||||
source: sm.id,
|
||||
target: cm.id,
|
||||
kind: 'calls',
|
||||
line: sm.startLine,
|
||||
provenance: 'heuristic',
|
||||
metadata: {
|
||||
synthesizedBy: 'go-grpc-stub-impl',
|
||||
via: cm.name,
|
||||
registeredAt: `${cm.filePath}:${cm.startLine}`,
|
||||
},
|
||||
});
|
||||
added++;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return edges;
|
||||
}
|
||||
|
||||
@@ -856,6 +981,7 @@ export function synthesizeCallbackEdges(queries: QueryBuilder, ctx: ResolutionCo
|
||||
const flutterEdges = flutterBuildEdges(queries, ctx);
|
||||
const cppEdges = cppOverrideEdges(queries);
|
||||
const ifaceEdges = interfaceOverrideEdges(queries);
|
||||
const goGrpcEdges = goGrpcStubImplEdges(queries);
|
||||
const rnEventEdgesList = rnEventEdges(ctx);
|
||||
const fabricNativeEdges = fabricNativeImplEdges(ctx);
|
||||
const mybatisEdges = mybatisJavaXmlEdges(queries);
|
||||
@@ -871,6 +997,7 @@ export function synthesizeCallbackEdges(queries: QueryBuilder, ctx: ResolutionCo
|
||||
...flutterEdges,
|
||||
...cppEdges,
|
||||
...ifaceEdges,
|
||||
...goGrpcEdges,
|
||||
...rnEventEdgesList,
|
||||
...fabricNativeEdges,
|
||||
...mybatisEdges,
|
||||
|
||||
Reference in New Issue
Block a user