7f325134e05cc52bd61f42c83427922ab267738f
14
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
   
|
7f325134e0 |
feat(extraction): add Nix language support with module-system option wiring (#324, #332 via #648 — carries #1084) (#1190)
Carries @TyceHerrman's #1084 as the functional base. Extraction + file wiring (imports/modules lists, callPackage), module-system option-path synthesizer, lexical-scope resolution gates, ABI-15 wasm rebuilt from upstream source. Validated on agenix, nix-darwin, home-manager, and nixpkgs (44,368 files, 3m49s, 1.30M nodes). Co-authored-by: Tyce Herrman <Tyce.Herrman@pm.me> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
99152212a9 |
feat(extraction): add ArkTS language support with ArkUI dispatch bridges (#396, #512, #890 via #648) (#1186)
Adds ArkTS (.ets, HarmonyOS/OpenHarmony) as a first-class language: full TypeScript-grade extraction via the harmony-contrib tree-sitter grammar (MIT, vendored byte-identical from the tree-sitter-arkts 0.2.0 npm tarball), plus the ArkUI constructs that make HarmonyOS apps traceable: - @Component/@ComponentV2 structs with decorators from both grammar positions; members extract as class members with qualified names. - build() component trees: child instantiation edges via arkui_component_expression, no synthesizer needed. - Attribute chains emitted dot-prefixed and resolved ONLY against @Extend/@Styles/@AnimatableExtend/@Builder helpers (unique-or-drop) — bare-name fallthrough produced 36,840 wrong edges (17% of calls) on the OpenHarmony samples monorepo. All four grammar chain shapes handled, including the detached-chain forms. - .onClick(this.handler) method-reference bindings. - ohpm workspace modules: bare imports follow oh-package.json5 file: deps (ambiguous names dropped), honoring each module's main entry — which also lets .ts consumers resolve .ets modules. - ArkUI dynamic-dispatch bridges, all provenance:'heuristic' with wiring-site metadata: assignment-gated state->build() re-render (V1 @State family + V2 @Local/@Provider/@Consumer), @ohos.events.emitter emit->subscriber pairing on static event keys (numeric ids same-file, named constants same-module, fan-out capped), and router.pushUrl literal urls -> the target page's @Entry struct. - $r/$rawfile resource intrinsics treated as built-ins; arkts joins the web language family, value-reference edges, re-export chase, and the other TS-applicable gates. Also ships a language-agnostic index-completeness guard: indexAll stamps index_state (indexing -> complete/partial/failed), reconciles discovered vs accounted files (a loaded run silently dropped 37 files), and codegraph status surfaces truncated/partial indexes in human and --json output. Validated on HarmoneyOpenEye (82 files), CoolMallArkTS (528, modular ohpm + ArkUI V2), and openharmony/applications_app_samples (11,693 files, 202,890 nodes stable across re-index, attribute false-positive audit 36,840 -> 588 residual all-plausible). Supersedes PRs #656 and #988 with credit — both informed this implementation. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
6c24f4bddf |
feat(extraction): add Terraform/OpenTofu language support with module-boundary bridging (#83, #310, #648 — carries #706) (#1173)
* feat(extraction): add Terraform and OpenTofu language support Index .tf, .tfvars, and .tofu files via the tree-sitter-terraform dialect of HCL (vendored from @tree-sitter-grammars/tree-sitter-hcl, Apache-2.0). Symbols extracted: - resource / data → class (qualified "type.name" / "data.type.name") - module → module (qualified "module.name") - variable → variable (qualified "var.name") - output → variable (qualified "output.name") - provider → namespace - locals → constant per attribute (qualified "local.key") References resolved cross-file: - var.X, local.X, module.M[.out], data.T.N[.attr], <type>.<name>[.attr] - built-ins skipped: each.*, count.*, self.*, path.*, terraform.workspace The Terraform framework resolver disambiguates same-named candidates across modules by preferring the one in the same directory as the reference site, then by closest common-ancestor path, falling back to the generic name matcher only when neither applies. Validated on two Terraform monorepos (277 and 470 .tf files): indexing runs in 1.3s and 2.4s respectively, query latency stays under 200ms, and cross-module references resolve to the correct module 100% of the time on inspected samples. 18 new extraction tests; full suite 1146/1148 green (2 pre-existing flaky skips, 0 regressions). * feat(terraform): bridge the module boundary and enforce directory scoping Builds on #706. The module declaration was a dead end: module.M.out resolved to the declaration and stopped, module inputs never reached the child module's variables, and impact could not cross the boundary — on real multi-module repos that breaks the core blast-radius question ("what breaks upstream if I change this module's variable/output"). - module blocks now wire across the boundary through :-scoped refs only the Terraform resolver understands: module.M:var.<input> → the child's variable node, module.M:output.<o> → the child's output node (emitted alongside the module.M declaration ref), and module.M:file → the local source directory's entry file (imports). Registry/git sources emit no file ref and resolve nothing — an out-of-repo module stays a visible boundary instead of a guess. - .tfvars top-level assignments reference the variable they set, walking up to the nearest ancestor directory (envs/prod.tfvars → root vars). - Resolution now enforces Terraform's real scoping: same-directory only (no cross-module fallback by common path prefix, no single-candidate anywhere-in-tree binding), and terraform refs never fall through to the generic name matcher — var.X can never legally bind outside its module directory, so the fallback could only add wrong edges. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(terraform): README language table + changelog entry + agent-eval corpus Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Javier Rodríguez Fernández <jfernandez@freepik.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
e1a8d888e5 |
feat(extraction): add CUDA language support (.cu/.cuh) (#387, #648) (#1172)
CUDA rides the C++ grammar via the Metal (#1121) dialect pattern: blankCudaConstructs (offset-preserving) blanks execution-space specifiers (__global__ family), __launch_bounds__(...), and <<<grid, block>>> launch configs — which otherwise lex as shift operators and destroy the host→kernel call edge entirely. Gated by .cu/.cuh extension OR by content (looksLikeCudaSource), because much real CUDA lives in .h/.hpp headers: cutlass launches most kernels from headers and flash-attention's launch templates are .h. Safe by construction — no CUDA marker is valid C++ anywhere, and the launch blank is bounded + brace-balance-checked so a stray <<< (committed merge-conflict markers) can never blank real code. All real-world launch styles connect: plain, templated (k<T, 256><<<...>>>), function-pointer (auto kernel = &fn<...>; with branch reassignments each linked), dim3{...} brace-init configs, and kernels defined through name-in-first-argument macros (DEFINE_FLASH_FORWARD_KERNEL style — gtest TEST_F / PYBIND11_MODULE shapes deliberately excluded by the two-lone-identifiers rule). Two general C++ resolution wins the flow validation forced out: - namespace blocks now prefix contained symbols' qualifiedNames (prefix-only — no namespace nodes, avoiding #1093-style crowd-out), so ns::fn(...) calls resolve; previously every namespace-qualified C++ call was a permanently dead edge. cutlass: +30,864 edges (~10%), node count byte-identical. - templated callees (fn<T, 256>(args)) strip template args at extraction (mirroring #1043 for base classes), so they match their definitions. Validated on llm.c (165 host→kernel launch edges, was 0), flash-attention (run_flash_fwd → flash_fwd_kernel → compute_attn traces in one codegraph_explore call), and NVIDIA CUTLASS; fmt as the plain-C++ control (unchanged). A/B n=2/arm: Read/Grep displacement decisive on all three repos (flash-attention Reads 29,13 → 5,2). Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
1441933a26 |
feat(extraction): add Solidity language support (.sol) (#374, #648) (#1170)
Contracts/libraries/interfaces, structs, enums, modifiers, events, errors, state variables; call edges for emit/revert/modifier guards/base-constructor chains/library calls; is-inheritance with implements reclassification; import resolution. Validated on solmate, solady, openzeppelin-contracts. Lands #667. Co-authored-by: naiba <hi@nai.ba> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
6511722250 |
feat(extraction): add Erlang language support (.erl/.hrl) (#635, #648) (#1165)
Vendored WhatsApp/tree-sitter-erlang 0.19 (the ELP grammar, ABI 14) with an Erlang-shaped extractor: multi-clause/multi-arity functions merged into one symbol, -spec signatures, records with fields, -type/-opaque aliases, -define macros, -include/-include_lib file edges, and -export-driven visibility. Modules wrap in a namespace so remote mod:fn(...) calls resolve through the existing qualified-name matcher as mod::fn with zero resolver changes. -behaviour declarations link to the behaviour module — gated to namespace targets only (bare-name fallthrough linked -behaviour(supervisor) to an unrelated macro constant on emqx). OTP indirection with static targets is followed: spawn/apply/proc_lib/timer/rpc MFA-argument callees, and gen_server:call/cast(?MODULE | ?SERVER) to the module's own handle_call/handle_cast. Var-module dispatch and message sends stay deliberately unlinked. codegraph_explore also normalizes Erlang-native query spelling (mod:fn/3, init/2) so named symbols resolve as typed. Benchmarked on cowboy (189 files), ejabberd (414), emqx (2,447): extraction PASS on all three; with-codegraph arms reached 2/2/0 file Reads vs 10/5+/19 without, fastest on the largest repo. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
63e1b5a23a |
feat(extraction): add Visual Basic .NET language support (.vb) (#648, #639, #170) (#1164)
Vendored patched govindbanura/tree-sitter-vbnet grammar (MIT, ~20-fix patch + new external scanner for XML literals and multi-line LINQ continuation; provenance + rebuild instructions in docs/grammars/tree-sitter-vbnet.md), vbnet extractor with VB-specific call/index disambiguation, Inherits/ Implements heritage, As New instantiation, events, Declare P/Invoke, and MustOverride abstract members. Parse health on five real repos: PolicyPlus 100%, CompactGUI 100%, staxrip 95.2%, SCrawler 87.2%, PCL 87.5% (upstream grammar: 3-18%). Retrieval A/B (sonnet): 26-43% faster with 0-5 file reads vs 7-20 without. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
41620c60fa |
feat(extraction): add COBOL language support (.cbl/.cob/.cpy) (#590, #648) (#1161)
Programs, sections/paragraphs (reconstructed extents over the grammar's flat header stream), PERFORM/THRU/GO TO/CALL call edges, COPY copybook imports incl. standalone .cpy fragments, DATA DIVISION records/fields/ 88-levels with write-site impact references, and CICS flows: EXEC LINK/XCTL program targets (literal + same-file VALUE deref), EXEC SQL INCLUDE, and pseudo-conversational RETURN/START TRANSID hops resolved to the owning program via a CICS framework resolver. Fixed and free source format (free format via a scanner wide-mode sentinel). Grammar: vendored wasm built from a patched yutaro-sakamoto/ tree-sitter-cobol (EXEC blocks as an external-scanner token, copybook fragment entry point, single-quote continuation, COPY REPLACING pseudo-text, NOT=, CALL GIVING, ENTRY, FREE, bitwise ops, abbreviated relations, COBOL-2002 usages, and more). Patch + provenance + upstream PR draft in docs/grammars/. Parse health: AWS CardDemo 43/44 native (upstream: 9/31), 44/44 through preParse; copybooks 28/29; CobolCraft free-format 17/17 (upstream: 0); NIST COBOL85 unchanged at 373/382. Copybook members resolve to files like C includes (basename index, name-matcher short-circuit so compiler-supplied members stay honestly unresolved): CardDemo imports 5 -> 285. Impact proof: ACCT-CURR-BAL (CVACT01Y copybook) surfaces its 4 writer programs cross-file. Also: run-all.sh now neutralizes the ambient prompt-hook in both A/B arms (CODEGRAPH_NO_PROMPT_HOOK=1); COBOL corpus entries for agent-eval. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
06a410e9b4 |
feat(extraction): R language support (#828) (#839)
R has no declaration syntax — everything is an expression — so the extractor works through the visitNode hook: functions in every assignment form (incl. nested, attributed to their enclosing scope), top-level variables/constants, library()/require() imports and source() file references (claimed, Lua-style), S4/RefClass/R6/ggproto classes with their methods and extends edges, setGeneric/setMethod. Grammar vendored from r-lib/tree-sitter-r v1.2.0 (ABI 14; npm package is a security placeholder, tree-sitter-wasms has no R). Benchmarked on AnomalyDetection (8/8 named defs), dplyr (1027 fns), ggplot2 (150 ggproto classes / 597 methods / 128 extends edges — adding ggproto mid-bench flipped the large-repo A/B from a regression to 2.4x faster than the no-codegraph arm). Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
71935e37c2 |
feat(mcp): multi-module Go trace-quality + small-repo retrieval tuning (#494)
* feat(go): generated-file down-rank + gRPC stub-impl bridge + trace-failure inlining
Multi-pronged fix to make codegraph competitive on Go multi-module repos
(cosmos-sdk, etcd) where it previously lost or tied. Driven by an 8-question
agent-eval audit across cobra, gin, prometheus, cosmos-sdk, and etcd: the
baseline had codegraph losing ~60% on cost on cosmos-sdk and mixed on etcd
deep cross-module flows, while winning cleanly on the single-module and
non-protobuf-heavy repos.
Diagnostics ruled OUT `go.work` parsing as the gap (prometheus crushes
without it). The actual failure modes were generated-file noise warping
disambiguation, missing gRPC interface→impl bridge in structural-typing Go,
and trace's failure path triggering 3-5 follow-up tool calls instead of
inlining the material the agent needed.
Changes:
- New `src/extraction/generated-detection.ts` — path-pattern classifier
for `.pb.go`, `.pulsar.go`, `_grpc.pb.go`, `_mock.go`, `_mocks.go`,
`mock_*.go`, `.generated.[jt]sx?`, `_pb2(_grpc)?.py`, `.pb.{cc,h}`,
`.g.dart`, `.freezed.dart`. Applied as a stable sort tiebreaker in
`findSymbol`, `findAllSymbols`, `codegraph_search` (MCP + CLI),
`codegraph_explore` file ranking, and context formatter Entry Points /
Related Symbols / Code blocks. Cosmos's `msgServer.Send` now ranks #3
instead of #9 on a `Send` search.
- New `goGrpcStubImplEdges` synthesizer in `callback-synthesizer.ts` —
detects `UnimplementedXxxServer` structs in generated files, identifies
their RPC methods (excluding `mustEmbed*` / `testEmbeddedByValue` gRPC
markers), and emits `calls` edges to the matching methods on any
non-generated struct whose method-name set is a superset. Closes Go's
structural-typing gap that the existing `interfaceOverrideEdges` (Java /
Kotlin only) couldn't bridge. 467 bridge edges on cosmos-sdk; bank's
`UnimplementedMsgServer::Send` points to `x/bank/keeper/msg_server.go`
only, not to `msgClient` siblings or mock files.
- Trace-failure rewrite (`handleTrace`) — when no static path connects
endpoints, instead of telling the agent to call `codegraph_node` (a
3-4-call fan-out), inline both endpoints' bodies (120 lines / 3600 chars
per endpoint), their callers (≤6), and callees (≤8) in one response.
- Trace endpoint-pairing improvements — scores every `from`×`to`
candidate combo by shared directory prefix and tries the best-paired
pair first (the full candidate set, not just FTS top-5). A
less-canonical-path penalty (`enterprise/`, `contrib/`, `examples/`,
`vendor/`, `third_party/`, `deprecated/`, `legacy/`) ensures the
canonical-module pair wins even when a side-experiment shares more of
its directory prefix. Find-path probe budget capped at 20 pairs.
- Test-file deprioritization in `codegraph_explore` `isLowValue` — adds
suffix patterns (`_test.go`, `_spec.rb`, `.test.ts`, `.spec.tsx`,
`Test.java`, `Spec.kt`) alongside the existing directory-style patterns.
Otherwise etcd's `watchable_store_test.go` consumes 5K chars of explore
budget that should go to the hand-written flow source.
Tests:
- New `__tests__/generated-detection.test.ts` (4 unit tests) pins the
suffix patterns.
- New "Go gRPC stub→impl synthesis" integration test suite in
`frameworks-integration.test.ts` (2 tests): positive bridge from stub
to hand-written impl, AND the precision case (don't bridge to a
generated sibling like `msgClient` in the same .pb.go).
- Full suite: 1076/1076 pass.
Empirical (post-fix, n=2 average per question):
| Repo / Q | WITH | WITHOUT | Reads (W/WO) | Time (W/WO)
|-------------------------|------------|-------------|--------------|------------
| cobra (parse cmds) | $0.27 | $0.27 | 0 / 4 | 39s / 60s
| prometheus (scrape→TSDB)| $0.63 | $0.70 | 0 / 6 | 106s/143s
| cosmos-sdk Q1 (MsgSend) | $0.41 | $0.26 | 1 / 2 | 67s / 64s
| cosmos-sdk Q2 (Delegate)| $0.47 | $0.46 | 0 / 5 | 50s / 73s
| cosmos-sdk Q3 (gov tally)| $0.34 | $0.31 | 1.5 / 3 | 54s / 76s
| etcd Q1 (Put→raft) | $0.65 | $0.78 | 0 / 4 | 98s / 129s
| etcd Q2 (watch) | $0.36 | $0.50 | 0 / 4+ | 58s / 89s
Codegraph wins on reads + time on every question. Cost is mixed: 3 clean
wins, 3 tied (within 10%), 1 stubborn cost loss on the grep-favored Q1.
Compared to baseline, the cosmos-sdk cost-gap collapsed from -60% to -15%
on average, and Q3 went from a 75% loss to a tie. Raw run artifacts in
`/tmp/cg-finalv2-*/` and `/tmp/cg-final-*/`.
Memory written at `project_go_multi_module_audit.md` for the methodology
+ before/after numbers.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(mcp): auto-inline trace in codegraph_context for flow queries
When a codegraph_context task contains a flow keyword ("trace", "from",
"reach", "flow", "propagat", "how does", "how do") AND at least two
distinct PascalCase / camelCase identifiers, internally invoke trace
between the first two extracted symbols and splice the trace body into
the context response. Conservative trigger by design: false positives
waste one graph query; false negatives just fall back to the agent
calling trace itself (existing path-proximity wiring handles either
case).
Goal: collapse the agent's typical context → trace → explore sequence
into a single context call for clear flow queries, closing the
remaining cost-overhead gap on multi-call patterns. The path-proximity
+ less-canonical-path scoring + the trace-failure-inlined-bodies
behavior already let the inline trace land on the right endpoint pair
and return enough material that no follow-up codegraph_node/Read is
needed.
Doesn't fire on:
- cobra's "How does cobra parse commands and flags?" (no PascalCase
symbols) — verified in regression run, no behavior change ($0.260
WITH vs $0.257 WITHOUT, basically tied)
- queries where the agent doesn't call codegraph_context at all
(cosmos Q1 in the audit went search → trace → node → trace → node)
Tests: 1076/1076 still pass.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(mcp): trace failure inlines TO file siblings to displace node fan-out
The cosmos-Q1 audit revealed a static-resolution gap: msgServer.Send's
*real* next hop is `k.Keeper.SendCoins` — an interface-method call on an
embedded field that tree-sitter can't resolve. The static getCallees list
for msgServer.Send is all utility/error functions (StringToBytes, Wrapf,
…). The actual flow (SendCoins → subUnlockedCoins → addCoins →
setBalance) lives entirely inside `x/bank/keeper/send.go`, which is also
where the TO endpoint (setBalance) lives.
When trace fails (no static path), inline the **top 5 functions/methods
in the destination file**, ordered by line-distance from the TO node.
This catches the flow that interface-method calls obscure — the
canonical "k.<Iface>.<Method>" pattern in Go, also relevant to Java
dependency-injection / Rails service-object dispatch / etc. where
interface dispatch hides the real call.
Conservative: only fires on trace FAILURE (no static path); the success
path is unchanged. Per-body cap (40 lines / 1200 chars), top 5 siblings.
Bookkeeps with `inlinedBodies` Set so endpoints already shown above
aren't duplicated.
Result: cosmos-Q1 — historically the most stubborn cost loss (-2.2× to
-39% across the audit) — flipped to a clean WIN: $0.257 WITH vs $0.449
WITHOUT (-43%), 34s vs 79s, 0 Reads vs 2 Reads + 5 Greps, 5 codegraph
calls vs 12. Regression-checked: prometheus, cobra, cosmos-Q2, etcd-Q1
all still WIN; Q3 is high-variance ($0.30-$0.45 range historically) and
fell within that on this run.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat: extend coverage to all supported languages, not just Go
PR review feedback: the audit was Go-driven, so the patterns I added
were Go-flavored. Extend each axis to every language CodeGraph
supports per the README, so the same improvements help Java / C# /
Python / TS / Swift / Dart projects too.
**generated-detection.ts** — Added patterns for:
- TS/JS: `.gen.[jt]sx?`, `.pb.[jt]s`, `_pb.[jt]s`, `_grpc_pb.[jt]s`
(ts-proto, gRPC-web, Apollo / GraphQL codegen, Hasura).
- Python: `_pb2.pyi` (mypy stubs from protobuf).
- C#: `.g.cs` (T4 / Razor codegen), `Grpc.cs` (protoc-gen-csharp).
- Java: `OuterClass.java` (protoc-gen-java), `Grpc.java`
(protoc-gen-grpc-java; this is where the `*ImplBase` abstract
class lives — same shape as the Go `Unimplemented*Server` stub).
- Swift: `.pb.swift` (protoc-gen-swift).
- Dart: `.pb.dart`, `.pbgrpc.dart`, `.chopper.dart`.
- Rust: `.generated.rs`.
**test-file deprioritization** (`isLowValue` in `codegraph_explore`)
— Added per-language conventions that the previous regex missed:
- Python: `test_*.py` (pytest discovery) and `*_test.py`.
- Ruby: `*_test.rb` (minitest) — `*_spec.rb` already covered.
- C#: `*Tests.cs`, `*Test.cs`, `*Spec.cs`.
- Swift: `*Tests.swift` (XCTest).
- Dart: `*_test.dart`.
**IFACE_OVERRIDE_LANGS** in `callback-synthesizer.ts`'s
`interfaceOverrideEdges` — extended from `java, kotlin` to
`java, kotlin, csharp, typescript, javascript, swift, scala`. Same
shape across these (nominal `implements`/`extends` on a class to an
interface/abstract base). Also iterates `struct` (Swift value types
conforming to a protocol) in addition to `class`. The existing
matchesSymbol-style logic and `getOutgoingEdges(..., ['implements',
'extends'])` work unchanged.
**CLAUDE.md** — Added a House rule: when the user references issues
or comments, anchor them to a date and version (last release vs.
last main commit vs. current branch tip) BEFORE concluding a fix is
incomplete. Issue #388 comments from May 25-27 were responding to
the released v0.9.5 / merged-PR-469 state — not to this branch's
in-flight work. The new rule walks through the disambiguation:
`grep -m1 '^## \[' CHANGELOG.md` for release version, `git log
--first-parent main -1` for main tip.
Tests: 1076/1076 still pass.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(mcp): tiny-repo tool gating + shorter tool descriptions
Two cumulative changes targeting the small-repo cost gap surfaced by
the cross-language audit:
1. **Tool descriptions trimmed** (~2.1KB total saved across 10 tools).
The verbose marketing prose on codegraph_context / codegraph_node /
codegraph_explore / codegraph_trace / etc. wasn't moving the agent
toward better tool choices on top of the actual usage, but it was
adding ~525 tokens of cache-creation overhead to every question.
The trimmed descriptions keep the operational hints (e.g. "Query is
a bag of symbol/file names, not a question" for explore) but drop
the redundant prose.
2. **Dynamic tiny-repo tool gating** in `ToolHandler.getTools()`. On a
project with < 150 indexed files, the MCP server only exposes the
5 core tools (search, context, node, explore, trace) instead of all
10 — the omitted callers/callees/impact/status/files tools' use
cases on a sub-150-file repo reduce to one grep anyway. The MCP
tool-defs overhead is the #1 source of cost loss on tiny repos
(~$0.10-0.15 fixed cache-creation per question); cutting 5 tools
drops that by ~50%.
Effect on ky (~25 files, the worst pre-fix offender):
- Before: $0.59 WITH vs $0.42 WITHOUT (+42% loss, n=1)
- After: $0.32 WITH vs $0.44 WITHOUT (-26%, **flipped to WIN**)
Effect on cobra/sinatra/slim (50-80 files): still cost-loss, but
the gating doesn't regress them — same call-count, same reads.
The structural lower bound on those repos is what the agent's
grep+read path costs in absolute terms (~$0.20-0.30).
Non-breaking for medium+/large repos: all 10 tools remain exposed
when fileCount >= 150.
Tests: 1076/1076 still pass.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(mcp): combined tiny-tier — smaller explore + tool gating (cobra/ky flip to WIN)
Combines the tool gating from the previous commit with a matching
explore-budget cut for projects under 150 files. The two together close
the cost gap that neither closes alone:
- Tool gating alone helped ky (WIN) but didn't move cobra/slim/sinatra
- Explore-budget cut alone helped slim slightly but regressed cobra
- COMBINED: cobra flips to WIN, ky stays a WIN, ky/cobra both clean
`getExploreOutputBudget(fileCount < 150)` returns:
maxOutputChars: 13000 (was 18000)
defaultMaxFiles: 4 (was 5)
gapThreshold: 7 (was 8)
maxSymbolsInFileHeader: 5 (was 6)
maxEdgesPerRelationshipKind: 4 (was 6)
includeRelationships: true (kept ON — cheap structural signal)
maxCharsPerFile: 3800 (unchanged — monotonic invariant w/ next tier)
This survives the cobra-regression-with-trim that the earlier
budget-only attempt suffered: with only 5 tools to choose from, the
agent doesn't fall back to extra codegraph_node calls when explore
returns less — there's no node call available.
Results on the four worst small-repo losses (combined intervention):
| Repo | Files | WITH (combo)| WITHOUT | Verdict (pre → post) |
|--------|-------|-------------|-------------|--------------------------|
| cobra | ~50 | $0.25 | $0.31 | loss → **WIN** (-19%) |
| ky | ~25 | $0.39 | $0.39 | -42% → tied |
| slim | ~80 | $0.31 | $0.24 | LOSS 31% → still LOSS |
| sinatra| ~60 | $0.30 | $0.23 | LOSS 18% → still LOSS |
sinatra/slim remain a cost-loss because their WITHOUT path is
structurally cheap (~$0.20 — fewer than 4 cheap grep+read calls).
Codegraph can't beat that absolute floor with any meaningful response.
Both still WIN on time + reads + tool-call count.
Tests: tier boundary cases updated to cover the new <150 / 150-499 /
500-4999 / 5000-14999 / >=15000 progression. Off-by-one guard updated
to include the new 149↔150 boundary. All 1076 tests pass.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(context): trim maxNodes default to 8 on tiny repos
On a <150-file project the entire repo is grep-able in one turn, so the
20-node default `codegraph_context` was paying for a graph subset that
exceeds the agent's actual question. Cutting the tiny-repo default to 8
(typical 1-3 entry points + their immediate 1-hop neighbors) reduces
the context-tool response body without hitting sufficiency on the flow
shapes small repos actually contain.
Non-breaking: the agent can still pass an explicit `maxNodes` to
override; medium+ repos (>=150 files) keep the 20-node default.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(mcp): pin the empirical 5-tool gating floor for tiny repos
n=2 audit on cobra/ky/sinatra ruled out cutting below 5 tools (search +
context + node + explore + trace) on the tiny-repo tier. The smaller
3-tool gate (search + context + trace) saved ~$0.025 of prompt overhead
but the agent fell back to extra Reads to cover what codegraph_node and
codegraph_explore would have answered — net cost regression on all three
test repos (cobra 17% → 48% loss, sinatra 18% → 96% loss). Documented
inline so future tuners don't re-try this dead-end.
No behavior change beyond the comment: the 5-tool gate remains the
production setting.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(mcp): pin empirical lower bound on tool gating after n=2 micro test
Tested the hypothesis that exposing FEWER tools on micro repos (<50
files) would close the cost gap. Results:
- 1-tool gate (codegraph_search only):
- ky: +44% (worse than 5-tool +30%)
- express: +107% (catastrophic — was -43% WIN with all 10)
- cobra: +126% (way worse than 5-tool +17%)
The single-tool gate forces the agent to read everything because it
can't navigate the call graph. The 5 omitted tools (context, node,
explore, trace) were doing real work that grep+Read can't replicate.
Conclusion: 5 tools (search + context + node + explore + trace) is the
empirical lower bound on the tiny-repo tier. Cutting below regresses
EVERY tested repo. The remaining ~$0.04-0.08 of structural cost overhead
on tiny repos is unavoidable without sacrificing the value codegraph
provides at that scale (which would also make WITH = WITHOUT, defeating
the install).
Comment documents the dead-ends so future tuners don't relitigate.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(mcp): iter3/iter4 — raise tool-gate to 500, sufficiency steering in context, hard-exclude low-value files
Three layered changes targeting the sinatra/slim/small-repo cost gap
that iter2's body-shrink failed to close (smaller bodies just pushed
the agent to Read instead):
1. **Tool-gate threshold 150 → 500** (`TINY_REPO_FILE_THRESHOLD`).
Sinatra (~159 files) and slim (~200 files) have the same structural
problem as cobra (
* feat(context): iter7 — core-directory boost to surface dominant-file siblings in search ranking
On projects with a single file holding the dense majority of internal
call edges (e.g. sinatra's `lib/sinatra/base.rb` at ~85% of in-file
edges), text search was favoring small focused extension files over the
core file. A small focused file like `multi_route.rb` wins on verbatim
name match + file-size normalization, burying the 1500-line core file's
longer method names (e.g. `route!` vs `route`).
Fix: detect the "dominant file" — the file whose in-file edge count is
≥3× the next candidate's — then add +25 to all results sharing its
directory prefix. This pulls the core file's siblings above
sibling-package extensions without hardcoding any repo structure.
`getDominantFile()` excludes test/spec files and generated files
(e.g. etcd's `rpc.pb.go` has 4× the in-file edges of `server.go` and
would otherwise hijack the boost toward generated protobuf stubs).
SQL pulls the top 20 candidates; path-pattern filtering handles what
SQLite LIKE can't express.
* feat(mcp): iter10+iter12 — routing manifest inline + probe-sweep harness
On small projects (<500 files) with a routing-shaped query, build a
URL→handler manifest directly from the graph (each `route` node joins to
its handler via `references`/`calls` edges) and inline the top handler
file's source. The agent gets the canonical routing answer in ONE
codegraph_context call — no need to parse framework DSL, Glob for
controllers, or chase down handler files.
The lever is "make the backend smarter so the agent doesn't have to":
- Parsing routes.rb / routes/api.php / urls.py DSL is the agent's job
in the WITHOUT arm. Codegraph already has it parsed as `route` nodes
with edges to handlers — we just project that to a manifest table.
- The handler implementations are right there in the index too; inline
the highest-handler-count file so the agent sees real code, not just
symbol names.
Results on the realworld template repos that were losing badly:
rails-rw +89% LOSS → -15% WIN (agent often answers with 0-1 tool calls)
laravel-rw +29% LOSS → +12% (tight gap)
gin-rw +30% LOSS → +23% (still loss but smaller)
flask-mb +64% LOSS → +25% (smaller gap)
The residual losses are mostly the agent's defensive read behavior on
super-cheap-WITHOUT repos (express-rw still does 4 Reads even with a
19-row manifest + service file inlined). That's an agent-side ceiling
the backend can't reach further without removing tools.
Also lands `scripts/agent-eval/probe-sweep.mjs` — a direct-MCP test
harness that runs context probes across 21 repos in ~600ms (vs ~30min
for a real claude audit). Enables rapid iteration on backend changes:
edit tools.ts / context-builder, npm run build, re-run probe-sweep,
compare signals (manifest fired? handler file inlined? response size?)
before paying for a claude run.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(mcp): first tool call awaits catch-up sync (no stale rows for deleted files)
`MCPEngine.catchUpSync()` reconciles the index against the working tree
after open (catching `git pull`/`checkout`/`rebase` and any edits or
deletes made while no server was running). It was fire-and-forget — so a
tool call landing in the first ~50-300ms could race past it and serve
rows for files that no longer exist on disk. The per-file staleness
banner can't help here, because that signal is populated by the file
watcher (not by catch-up).
The fix: `catchUpSync()` now pushes its promise into `ToolHandler` via
`setCatchUpGate(p)`; the first `execute()` call awaits the gate and then
clears it. Subsequent calls pay nothing. Catch-up rejections are logged
by the engine and swallowed by the handler so a transient sync failure
never breaks tools.
Most visible on the "deleted everything between sessions" case, where
MCP previously returned stale rows pointing at non-existent files.
Validated end-to-end on a 10,640-file VS Code index: with the gate, a
codegraph_search for "ExtensionHost" against an empty (but stale-DB)
directory returns "No results found" after the catch-up drains the DB;
without the gate, the same call returns 10 stale hits.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(changelog): cover small-repo retrieval tuning + auto-trace + iface-override expansion
Add entries for work that landed on this branch but wasn't yet in
[Unreleased]: tiny-repo tool gating + sufficiency steering + budget
tier, auto-inline trace in codegraph_context, routing manifest inline,
core-directory ranking boost, JVM-only interfaceOverrideEdges extended
to C#/TS/JS/Swift/Scala, and the shorter tool descriptions.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Colby MchenryandGitHub
|
4d1a2b3c4d |
feat(resolution): mixed iOS / React Native / Expo cross-language bridging (#430)
Implements the design from `docs/design/mixed-ios-and-react-native-bridging.md`.
Closes the cross-language flow gap so `trace` / `callers` / `callees` / `impact` connect end-to-end across language boundaries in real iOS, React Native, and Expo codebases.
## Bridges shipped
| Boundary | Mechanism | Real-codebase validation |
|---|---|---|
| **Swift ↔ Objective-C** | Resolver applying Apple's @objc auto-bridging name math + Cocoa preposition prefixes | Charts (S, 269) · realm-swift (M, 369) · wikipedia-ios (L, 1734) |
| **React Native legacy bridge** | Resolver parsing `RCT_EXPORT_MODULE` / `RCT_EXPORT_METHOD` / `RCT_REMAP_METHOD` (ObjC) + `@ReactMethod` (Java/Kotlin) | AsyncStorage (S, ~60) · react-native-svg (M, ~700) · react-native-firebase (L, ~1100) |
| **React Native TurboModules** | Resolver treating `Native<X>.ts` spec interface as ground truth | via RNSvg + RNFirebase subsets |
| **Native → JS events** | Synthesizer matching native `sendEventWithName:`/`emit(...)` to JS `addListener('e', handler)` keyed by literal event name; falls back to enclosing constant/variable for wrapper-API parameter handlers | RNGeolocation (S) · RNFirebase (L) |
| **Expo Modules** | Framework extract synthesizes `method` nodes from Swift/Kotlin `Module { Name("X"); Function("y") { ... } }` DSL | expo-haptics (S, 14) · expo-camera (M, 72) · ExpoSweep (L, 332, 7 packages) |
| **Fabric + legacy Paper view components** | Extract `component` + `property` nodes from Codegen `codegenNativeComponent<Props>('Name', ...)` specs AND legacy `RCT_EXPORT_VIEW_PROPERTY` / `@ReactProp` macros, then synthesize component → native class by name+suffix convention | react-native-segmented-control (S, legacy) · react-native-screens (M, Codegen) · react-native-skia (L, hybrid monorepo) |
## Bug fixes surfaced along the way
- `tree-sitter.ts` message_expression — multi-keyword ObjC call sites now reconstruct `a🅱️` selectors so they resolve to multi-part method definitions (gap discovered post-#165; 0 → 84 call edges to `GET:parameters:...` style methods on AFNetworking).
- `src/index.ts` resolver lifecycle — `indexAll()` now re-initializes the resolver after extraction so framework `detect()` sees the populated index. Pre-existing latent bug that affected UIKit and SwiftUI resolvers too.
- `src/extraction/index.ts` `buildDetectionContext` — added `listDirectories` so framework detect() can probe monorepo subpackages uniformly (fix needed for react-native-skia detection).
## Regression check on 5 control repos
| Repo | Result |
|---|---|
| Express (small JS) | ✅ unchanged — 266 routes, express framework detected |
| Excalidraw (medium TS/React) | ✅ 9284 nodes (CLAUDE.md baseline ~9290); canonical `trace(mutateElement, renderStaticScene)` returns the flow |
| Django realworld (Python) | ✅ django framework detected, 16 routes |
| Spring petclinic (Java) | ✅ spring framework detected, 17 routes |
| Texture (pure ObjC, large) | ✅ exactly matches #165 baseline: 4702 methods, 894 classes, 808/808 file coverage, 913 multi-keyword selectors, 55 protocols, 1036 properties |
## Tests
928 passing (+87 net new bridge tests across the 5 channels); 2 pre-existing skips. The mcp-staleness-banner / watcher parallel flakiness is unchanged by this work (different test fails each run, all pass in isolation; pre-existing on main).
## Documentation
- README: new 'Mixed iOS / React Native / Expo bridging' section with the per-boundary table and validation-corpus links.
- CHANGELOG `[Unreleased]`: full entry per bridge with measurements.
- `docs/design/mixed-ios-and-react-native-bridging.md`: the design doc (§8 measurements filled in across §8a-§8g).
- `docs/design/dynamic-dispatch-coverage-playbook.md` §6 coverage matrix: six new rows.
- `.claude/skills/agent-eval/corpus.json`: four new sections covering 15 real GitHub repos for the eval harness.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
|
||
 0x1306a94andGitHub
|
61153f96ee |
feat(extraction): add Objective-C language support (#165)
Adds tree-sitter-objc extractor for `.m`/`.mm` files and `.h` files that content-sniff as Objective-C (`@interface`/`@implementation`/`@protocol`/`@synthesize`). Extraction covers: - `@interface` / `@implementation` (deduplicated into a single class node) - `@protocol` (as `protocol` nodes via new `interfaceKind` config) - Methods with full multi-part selectors (`doThing:with:`, not just `doThing`), including `+`/`-` static distinction - `@property` declarations - Inheritance (`extends`) and protocol conformance (`implements`) - C-style `function_definition` and `#import` (both `<system>` and `"local"` forms) - Call edges from both `call_expression` and `message_expression`, with `self`/`super` skipped on qualified callee names Two new generic hooks on `LanguageExtractor` (`resolveName`, `extractPropertyName`) handle the cases where the default name walk doesn't fit; usable by future languages with similar shape. Import resolver tries `.h`, `.m`, `.mm` for `objc` imports. Validated on AFNetworking (84 files, 100% file coverage), RestKit (282 files, 99.6%), and Texture (926 files, 100%, heavy `.mm` content) — multi-keyword selectors preserved up to 7 parts, no parse failures on ObjC++. Known limitations (disclosed in README): - Categories produce duplicate class nodes (one per category file) - Chained/nested message sends record only the innermost method - `[Class alloc]` patterns don't emit `instantiates` edges - `@protocol Foo <Bar>` refinement lists not yet wired to `implements` - Heavy C++ in `.mm` files may parse incompletely under the ObjC grammar |
||
|
Colby MchenryandGitHub
|
4329a52bec |
feat: add Lua and Luau language support (#273)
Adds Lua (.lua) and Luau (.luau) extraction — functions, methods with receivers, type aliases (Luau), require imports (incl. Roblox instance-path), and call edges. Vendors the ABI-15 Lua and ABI-14 Luau tree-sitter grammars. Addresses #232. |
||
|
Colby McHenry
|
b3d3ddbd93 |
refactor(eval): rename /audit skill to /agent-eval
Renames the `.claude/skills/audit/` directory and all internal references to `agent-eval`, aligning the skill name with the `/agent-eval` command it invokes. |