Introduce Expo Router integration: a new framework resolver, route-based screen nodes, and navigates edges, plus a /api/screens endpoint and a Screens UI view. Adds branch-guard-driven labeling of edges, resolution logic, and tests to cover extraction, resolution, and end-to-end flow. This enables CodeGraph UI to surface screens and transitions from Expo Router apps.
Previously, `codegraph_explore` queries explicitly naming files by path (e.g., `src/routes/m/projects/[id]/runs/[runId]/+page.svelte`) were shredded. Bracketed path segments exploded into "named symbol" seeds, and FTS on fragments like `page` or `runs` admitted every sibling file, starving the user's intended target.
This change introduces:
- **Query path pinning:** File paths named in a query are now resolved against the index, "pinned," and stripped from the query. Pinned files are guaranteed inclusion, top ranking, and fair allocation. Unresolvable path-like spans are reported.
- **Segment vocabulary supplement:** Natural language query terms (e.g., "auto-scroll to bottom") can now reach camelCase identifiers (e.g., `pinFeedIfNearBottom`, `feedAtBottom`) by matching against their constituent segments.
- **Variable seeding:** `variable` and `constant` node kinds are now included in identifier seeding, improving recall for `$state`-style variables common in frameworks like Svelte.
`isGeneratedFile` was path-only, but Go's own convention is a CONTENT
marker (`// Code generated by <tool>. DO NOT EDIT.`), not a filename one.
A Go monorepo with generated CRUD in ordinarily-named files sitting beside
hand-written use-cases was therefore invisible to every generated-file
down-rank in the codebase — that is #1500.
Measured on kubernetes/client-go (2,453 Go files): the canonical banner
appears in 2,001 of them, the path check flags 0, the new content check
flags exactly those 2,001 — no false positives, no misses.
Design: decide at INDEX time (content is already in memory for parsing),
persist on `files.generated`, read from the DB. Explore never reads file
headers per request.
- `hasGeneratedHeader(content)` recognizes the standard banners — Go's,
protoc's, `@generated`, `<auto-generated>`, Thrift, OpenAPI Generator,
FlatBuffers, bindgen, ANTLR. Precision-first and fenced three ways: an
8KB/60-line header window, a comment-line requirement (leader or open
block comment), and markers tight enough that prose can't trip them. A
generator's own source, holding the banner as a string constant in its
body, is not flagged; neither is this module itself (pinned by test).
- `isGeneratedFile(path)` is unchanged — cheap, sync, still the fallback.
- Schema v9 adds `files.generated` + a PARTIAL index. DDL only, no
backfill: the flag derives from content the migration cannot see, so
rows stay 0 until a re-index and every reader unions the flag with the
path check — an un-migrated index keeps pre-#1500 behavior rather than
regressing. Re-index required; noted in the CHANGELOG.
- `generatedPredicateFor(paths)` gives ranking a bounded probe + O(1)
lookups. Bounded, not cached: no invalidation, so a ranking call can
never serve a verdict the last sync already replaced. Wired into explore
ranking, findSymbolMatches, findAllSymbols, search (MCP + CLI), the
context formatter, and the dominant-file/route-file hygiene filters.
Cost (acceptance bar was no measurable index-time regression): a single
unanchored `/generat/i` test over the header rejects ~every hand-written
file before any line splitting. 4.6 µs/file on client-go (worst case —
82% generated). End-to-end `codegraph init` on client-go, n=3 alternating
arms: 5.73s median with detection vs 5.76s path-only baseline; the arms
cross over between runs, so the difference is inside run-to-run noise.
Scope note: generated status remains a stable TIEBREAK at equal score,
exactly where it was. Making it a strong negative signal is CG-10, which
this unblocks by making the signal correct and available.
Two pre-existing tests hard-coded schema version 8; both now track
CURRENT_SCHEMA_VERSION (or the migration table) so future migrations
don't require editing them.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A SIGKILL'd process (the #850 liveness watchdog, OOM, a crash) leaves its WAL
on disk; the next session appends to the same file; and nothing ever truncated
it — PASSIVE checkpoints fold frames but keep the file at its high-water mark,
and the one shrinking path (a clean last-connection close) is exactly what a
killed-daemon world never takes. Observed at 25.6 GB on a 5.46 GB DB, growing
until the disk filled.
- journal_size_limit on every connection: resetting checkpoints now clip the
WAL back to the cap instead of leaving it at its high-water mark.
- healOversizedWal() fired from every DatabaseConnection.open: off-thread
PASSIVE fold + TRUNCATE when the leftover WAL exceeds the cap (64 MB,
CODEGRAPH_WAL_HEAL_MB to override). Single-flight per connection with
bounded retries — concurrent passes defeat each other (each checkpoint sees
the other as a busy reader).
- Daemon/direct MCP watchdogs now pass progressPaths (DB + WAL), extending the
#1231 slow-disk deferral to the long-lived server so a healthy daemon mid
slow statement isn't SIGKILL'd — fewer kills, fewer leaked WALs.
- codegraph status shows WAL size (human + JSON) and warns when it dwarfs the
DB; daemon.log lines and the watchdog kill notice now carry ISO timestamps
so kills can be placed in time.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Kernel-routed files ship their flat tables from the parse worker to the
store worker as buffers (tryKernelExtractRaw → kernelBuffers on the
result → KernelStoreBundle); the store worker decodes and finalizes
(finalizeStoreBundle shared with the object path so filter semantics
can never drift). Files with applicable framework extract() hooks keep
the decoded path; the no-writer fallback materializes via
materializeKernelResult. Byte-identical dumps re-verified on dubbo,
excalidraw, express, gson; full suite green (2,467).
Measurement (plan §4d): dubbo's parse-loop wall is 94% store-writer
busy time — the many-core fresh-index wall is single-writer SQLite
ingest, not extraction or main-thread work. d2s improves the writer
lane ~11% (structured-clone deserialization avoided on the writer) and
frees the main thread; the remaining many-core gap is a
store-architecture arc (deferred index builds, multi-file
transactions, buffer→bind), out of the kernel project's scope.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Phase 0 of the Rust extraction-kernel migration (docs/design/
rust-kernel-migration-plan.md, now checked in with §3a recording the
shipped state):
- codegraph-kernel/ napi-rs crate: extractFile(path, content, language)
→ five flat buffers (meta/nodes/edges/refs/arena), one JS boundary
crossing per file. Node ids computed Rust-side, byte-identical to
generateNodeId (pinned by test vector). Reserved per-node metrics slot
for the Arc 3.2 code-metrics work.
- Generic .scm-driven emitter (@def.<kind>/@name/@ref.<kind> captures,
byte-range scope stack → ::-joined qualified names, contains edges,
refs attributed to the innermost enclosing symbol). Seed TS/JS queries
are smoke-level; R2 replaces them with the full port.
- Routing seam in extractFromSource with per-file wasm fallback.
DEFAULT_ROUTED is empty — no behavior change until a language passes
its equivalence gate (R3). Dev opt-in: CODEGRAPH_KERNEL_LANGS. Kill
switch: CODEGRAPH_KERNEL=0. Loader verifies ABI + kind tables before
routing; EDGE_KINDS became a runtime array because kind order is now
wire contract.
- Grammar-source parity: vendored TS/TSX/JS wasm grammars built from the
exact crate revisions (tree-sitter-typescript v0.23.2,
tree-sitter-javascript v0.25.0, checked-in parser.c, ts-cli 0.25.10) —
the tree-sitter-wasms builds were 2023-era, which the new
kernel-grammar-parity test caught on day one. Production TS/JS parsing
gets 2.5 years of grammar fixes; full suite green (2456 tests).
- Build/release wiring: scripts/build-kernel.sh + npm run build:kernel;
release.yml kernel prebuild matrix (continue-on-error — the kernel is
optional everywhere, bundles fall back to the wasm path); bundles stage
lib/kernel/codegraph-kernel.node; release job runs the kernel suites
with CODEGRAPH_KERNEL_EXPECT=1.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Post-batch cleanup deleted resolved refs (and parked failed ones) by
(from_node_id, reference_name, reference_kind) — no line/col. When one
caller had several call sites to the same callee and a batch boundary
split them, the first batch's cleanup removed every row with that key,
including later-batch siblings that were never attempted — their edges
were silently never created. On nlohmann/json this ate 422 real call
edges (write_cbor's 38 to_char_type calls indexed as 11).
Refs loaded from unresolved_refs now carry their row id through
resolution, and all three persist paths (sync resolveAndPersist, the
yielding retry pass, the batched drain loop) delete / mark-failed by
exactly that id. The key-tuple methods remain only as the fallback for
hand-built refs from the public API. Failed-parking gains the same
precision: outcome can differ per call site (receiver inference reads
the ref's line), so a sibling must not inherit another row's failure.
Also untracks the zz-scratch local test files that slipped into #1268
and gitignores the pattern.
Validation: red-green regression test (5 sites, batch size 2 — old code
kept 2 edges, fix keeps 5); nlohmann/json re-index is a strict superset
of the previous edge set (0 lost, 422 recovered, spot-checked against
source).
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Adds ArkTS (.ets, HarmonyOS/OpenHarmony) as a first-class language:
full TypeScript-grade extraction via the harmony-contrib tree-sitter
grammar (MIT, vendored byte-identical from the tree-sitter-arkts 0.2.0
npm tarball), plus the ArkUI constructs that make HarmonyOS apps
traceable:
- @Component/@ComponentV2 structs with decorators from both grammar
positions; members extract as class members with qualified names.
- build() component trees: child instantiation edges via
arkui_component_expression, no synthesizer needed.
- Attribute chains emitted dot-prefixed and resolved ONLY against
@Extend/@Styles/@AnimatableExtend/@Builder helpers (unique-or-drop) —
bare-name fallthrough produced 36,840 wrong edges (17% of calls) on
the OpenHarmony samples monorepo. All four grammar chain shapes
handled, including the detached-chain forms.
- .onClick(this.handler) method-reference bindings.
- ohpm workspace modules: bare imports follow oh-package.json5 file:
deps (ambiguous names dropped), honoring each module's main entry —
which also lets .ts consumers resolve .ets modules.
- ArkUI dynamic-dispatch bridges, all provenance:'heuristic' with
wiring-site metadata: assignment-gated state->build() re-render
(V1 @State family + V2 @Local/@Provider/@Consumer),
@ohos.events.emitter emit->subscriber pairing on static event keys
(numeric ids same-file, named constants same-module, fan-out capped),
and router.pushUrl literal urls -> the target page's @Entry struct.
- $r/$rawfile resource intrinsics treated as built-ins; arkts joins the
web language family, value-reference edges, re-export chase, and the
other TS-applicable gates.
Also ships a language-agnostic index-completeness guard: indexAll
stamps index_state (indexing -> complete/partial/failed), reconciles
discovered vs accounted files (a loaded run silently dropped 37 files),
and codegraph status surfaces truncated/partial indexes in human and
--json output.
Validated on HarmoneyOpenEye (82 files), CoolMallArkTS (528, modular
ohpm + ArkUI V2), and openharmony/applications_app_samples (11,693
files, 202,890 nodes stable across re-index, attribute false-positive
audit 36,840 -> 588 residual all-plausible). Supersedes PRs #656 and
#988 with credit — both informed this implementation.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* feat(extraction): add Terraform and OpenTofu language support
Index .tf, .tfvars, and .tofu files via the tree-sitter-terraform dialect
of HCL (vendored from @tree-sitter-grammars/tree-sitter-hcl, Apache-2.0).
Symbols extracted:
- resource / data → class (qualified "type.name" / "data.type.name")
- module → module (qualified "module.name")
- variable → variable (qualified "var.name")
- output → variable (qualified "output.name")
- provider → namespace
- locals → constant per attribute (qualified "local.key")
References resolved cross-file:
- var.X, local.X, module.M[.out], data.T.N[.attr], <type>.<name>[.attr]
- built-ins skipped: each.*, count.*, self.*, path.*, terraform.workspace
The Terraform framework resolver disambiguates same-named candidates
across modules by preferring the one in the same directory as the
reference site, then by closest common-ancestor path, falling back to
the generic name matcher only when neither applies.
Validated on two Terraform monorepos (277 and 470 .tf files): indexing
runs in 1.3s and 2.4s respectively, query latency stays under 200ms,
and cross-module references resolve to the correct module 100% of the
time on inspected samples.
18 new extraction tests; full suite 1146/1148 green (2 pre-existing
flaky skips, 0 regressions).
* feat(terraform): bridge the module boundary and enforce directory scoping
Builds on #706. The module declaration was a dead end: module.M.out
resolved to the declaration and stopped, module inputs never reached the
child module's variables, and impact could not cross the boundary — on
real multi-module repos that breaks the core blast-radius question
("what breaks upstream if I change this module's variable/output").
- module blocks now wire across the boundary through :-scoped refs only
the Terraform resolver understands: module.M:var.<input> → the child's
variable node, module.M:output.<o> → the child's output node (emitted
alongside the module.M declaration ref), and module.M:file → the local
source directory's entry file (imports). Registry/git sources emit no
file ref and resolve nothing — an out-of-repo module stays a visible
boundary instead of a guess.
- .tfvars top-level assignments reference the variable they set, walking
up to the nearest ancestor directory (envs/prod.tfvars → root vars).
- Resolution now enforces Terraform's real scoping: same-directory only
(no cross-module fallback by common path prefix, no single-candidate
anywhere-in-tree binding), and terraform refs never fall through to
the generic name matcher — var.X can never legally bind outside its
module directory, so the fallback could only add wrong edges.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs(terraform): README language table + changelog entry + agent-eval corpus
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Javier Rodríguez Fernández <jfernandez@freepik.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Vendored WhatsApp/tree-sitter-erlang 0.19 (the ELP grammar, ABI 14) with an
Erlang-shaped extractor: multi-clause/multi-arity functions merged into one
symbol, -spec signatures, records with fields, -type/-opaque aliases, -define
macros, -include/-include_lib file edges, and -export-driven visibility.
Modules wrap in a namespace so remote mod:fn(...) calls resolve through the
existing qualified-name matcher as mod::fn with zero resolver changes.
-behaviour declarations link to the behaviour module — gated to namespace
targets only (bare-name fallthrough linked -behaviour(supervisor) to an
unrelated macro constant on emqx). OTP indirection with static targets is
followed: spawn/apply/proc_lib/timer/rpc MFA-argument callees, and
gen_server:call/cast(?MODULE | ?SERVER) to the module's own
handle_call/handle_cast. Var-module dispatch and message sends stay
deliberately unlinked. codegraph_explore also normalizes Erlang-native query
spelling (mod:fn/3, init/2) so named symbols resolve as typed.
Benchmarked on cowboy (189 files), ejabberd (414), emqx (2,447): extraction
PASS on all three; with-codegraph arms reached 2/2/0 file Reads vs 10/5+/19
without, fastest on the largest repo.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Vendored patched govindbanura/tree-sitter-vbnet grammar (MIT, ~20-fix patch
+ new external scanner for XML literals and multi-line LINQ continuation;
provenance + rebuild instructions in docs/grammars/tree-sitter-vbnet.md),
vbnet extractor with VB-specific call/index disambiguation, Inherits/
Implements heritage, As New instantiation, events, Declare P/Invoke, and
MustOverride abstract members.
Parse health on five real repos: PolicyPlus 100%, CompactGUI 100%,
staxrip 95.2%, SCrawler 87.2%, PCL 87.5% (upstream grammar: 3-18%).
Retrieval A/B (sonnet): 26-43% faster with 0-5 file reads vs 7-20 without.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Programs, sections/paragraphs (reconstructed extents over the grammar's
flat header stream), PERFORM/THRU/GO TO/CALL call edges, COPY copybook
imports incl. standalone .cpy fragments, DATA DIVISION records/fields/
88-levels with write-site impact references, and CICS flows: EXEC
LINK/XCTL program targets (literal + same-file VALUE deref), EXEC SQL
INCLUDE, and pseudo-conversational RETURN/START TRANSID hops resolved
to the owning program via a CICS framework resolver. Fixed and free
source format (free format via a scanner wide-mode sentinel).
Grammar: vendored wasm built from a patched yutaro-sakamoto/
tree-sitter-cobol (EXEC blocks as an external-scanner token, copybook
fragment entry point, single-quote continuation, COPY REPLACING
pseudo-text, NOT=, CALL GIVING, ENTRY, FREE, bitwise ops, abbreviated
relations, COBOL-2002 usages, and more). Patch + provenance + upstream
PR draft in docs/grammars/. Parse health: AWS CardDemo 43/44 native
(upstream: 9/31), 44/44 through preParse; copybooks 28/29; CobolCraft
free-format 17/17 (upstream: 0); NIST COBOL85 unchanged at 373/382.
Copybook members resolve to files like C includes (basename index,
name-matcher short-circuit so compiler-supplied members stay honestly
unresolved): CardDemo imports 5 -> 285. Impact proof: ACCT-CURR-BAL
(CVACT01Y copybook) surfaces its 4 writer programs cross-file.
Also: run-all.sh now neutralizes the ambient prompt-hook in both A/B
arms (CODEGRAPH_NO_PROMPT_HOOK=1); COBOL corpus entries for agent-eval.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The keyword gate (#1126) can never know a repo's domain nouns. This adds
the graph-derived tier the design discussion converged on: symbol names
are split into prose segments at index time (name_segment_vocab, riding
the insertNode write path), and the hook verifies a prompt's plain words
against them — "the state machine des commandes" → OrderStateMachine, in
any language whose technical nouns are Latin script.
Confidence now decides HOW MUCH to inject, not just whether:
- HIGH (keyword, or index-verified code token): full explore injection,
unchanged — the validated adoption lever.
- MEDIUM (segment matches only): a ~500-byte pointer naming the matching
symbols; the AGENT writes the explore query. Never runs explore, so a
fuzzy match can't inject 16KB of wrong-feature context.
- Silent otherwise, as before.
Precision is derived from the repo's own naming statistics plus measured
FP fixes: co-occurrence (≥2 words on one name) always qualifies; a single
word must be ≥5 chars, cluster across 2–25 names (singletons are prose
coincidence: "deploy to production" → matchesNonProductionDir), match a
multi-segment name, and not be an English function/filler word (the one
place a word list is honest: identifiers are English, so only English
prose collides). Every candidate is re-verified against nodes before
being surfaced — vocab rows are proposals, deletions leave orphans by
design, a full index rebuilds from scratch, and sync heals pre-upgrade
databases (batched + yielding; emptiness captured at sync ENTRY so the
sync's own writes can't mask the backfill).
Schema v7 migration is DDL-only (instant; none of the #1067 row-churn
hazards). Gate outcomes roll up as anonymous usage counters
(prompt-hook-gate-<outcome>, names only, never content) through the
existing telemetry pipeline — recall becomes measurable, and the counters
are the agreed kill-criterion data for ever revisiting a local classifier.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
`codegraph query` printed `(score * 100)%` next to each hit, but `score`
is an unbounded BM25/FTS relevance magnitude (relative-ranking only), so
it rendered as values like "12042%" that made the output look broken.
Results already arrive in rank order, so drop the score from the
human-readable output entirely — matching the MCP search tool, which
shows no score. The raw `score` stays in `--json` for programmatic
sorting/thresholding. Also corrects the SearchResult.score doc comment,
which wrongly claimed a 0-1 range. Adds an end-to-end regression test.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
R has no declaration syntax — everything is an expression — so the
extractor works through the visitNode hook: functions in every
assignment form (incl. nested, attributed to their enclosing scope),
top-level variables/constants, library()/require() imports and
source() file references (claimed, Lua-style), S4/RefClass/R6/ggproto
classes with their methods and extends edges, setGeneric/setMethod.
Grammar vendored from r-lib/tree-sitter-r v1.2.0 (ABI 14; npm package
is a security placeholder, tree-sitter-wasms has no R).
Benchmarked on AnomalyDetection (8/8 named defs), dplyr (1027 fns),
ggplot2 (150 ggproto classes / 597 methods / 128 extends edges —
adding ggproto mid-bench flipped the large-repo A/B from a regression
to 2.4x faster than the no-codegraph arm).
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
.astro files were not indexed at all, leaving a typical Astro site mostly
invisible to search/impact/explore. New AstroExtractor (Svelte/Vue SFC
pattern): component node per file, TS frontmatter + <script> blocks
delegated to the TypeScript extractor, template {fn(...)} calls (incl. the
multiline `{posts.map((post) => (` opening line), PascalCase component-tag
references. New astroResolver: Astro global + astro:* virtual modules as
framework-provided, component resolution with the #764 ambiguity rule,
src/pages/ file-based routes ([param]→:param, [...rest]→*rest, _-prefixed
and *.config.* excluded). SFC languages now preload the TS/JS grammars
their extractors delegate to (a pure-SFC file set previously had none
loaded). Also fixes a pre-existing Svelte/Vue script-block off-by-one that
reported every script symbol one line low.
Validated per the playbook: stalux (the issue's repro) 54/54 .astro files
indexed, getIconNode found at its exact line, 14/14 routes, 93.0% fair
cross-file coverage; AstroPaper 27/27 components, 13/13 routes (underscore
dirs correctly excluded), explore connects page→Card→Datetime through the
jsx-render synthesizer; node/edge counts stable across re-syncs.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
A function name used as a VALUE — passed as an argument
(signal(SIGINT, handler), qsort(..., compare)), assigned to a function
pointer or field (ops->recv_cb = my_cb, OnClick := Handler), or placed in
a struct initializer / handler table ({ .recv_cb = my_cb },
{ "get", getCommand }) — produced no edge in ANY of the 19 tree-sitter
languages, so registered callbacks looked dead and their registration
sites were invisible to callers/impact.
This adds table-driven function-as-value capture across all 19 languages
(plus the wrapper forms: &fn, &Cls::method, Java Class::m, Kotlin ::f,
Swift #selector, ObjC @selector, Ruby method(:sym), Scala eta, Pascal
@Handler), gated at extraction (same-file definitions + imported
bindings; C-family file-scope initializers are constant-expression
contexts and skip the gate, which is how redis-style cross-file command
tables resolve), and resolved by a dedicated strategy: function/method
targets only, same-file first, unique-or-drop cross-file, no fuzzy
fallback ever. Edges persist as kind 'references' with metadata.fnRef,
so getCallers/getImpactRadius surface them with zero graph-layer
changes; MCP callers/callees label them "via callback registration".
Precision rules bought by real-repo false positives (full A/B record in
docs/design/function-ref-capture.md): C++ is &-explicit outside
file-scope tables (fmt's begin/out/size collisions; out-of-line member
defs are function-kind); TS/JS/Python bare ids resolve to functions only
(TS class fields extract as method-kind — pre-existing quirk); Swift
refuses same-file method overload-families; param-forward shapes
(this.x = x, value: value) and destructuring are skipped; minified
bundles (*.min.js) produce no candidates.
Validated on 17 public OSS repos (redis, excalidraw, gin, bytes, okhttp,
okio, Alamofire, flask, sinatra, Newtonsoft.Json, scopt, provider,
busted, Fusion, AFNetworking, PascalCoin, fmt): node counts identical,
zero calls edges lost or gained, references strictly additive
(+3,200 registration edges total), precision spot-checked by reading
sampled source lines (redis 30/30, flask 8/8). Deliberately NOT covered:
indirect-dispatch resolution (o->cb(x) → impl) — that needs data-flow
through struct fields, and a wrong edge is worse than none.
EXTRACTION_VERSION 18 → 19 (re-index to benefit).
Closes#756
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A C++ method call whose receiver is another call's result — `Foo::instance().bar()`,
`WidgetFactory::create().draw()`, `openSession()->run()`, or the same stored in an
`auto` local first — lost the receiver's type during extraction. The callee degraded
to a bare method name, so when two classes shared a method name the call silently
resolved to whichever was indexed first (or not at all), corrupting callers / impact /
trace with a plausible-but-wrong edge.
Three parts:
- Capture C++ return types (new nodes.return_type column, schema v5): the
function_definition's `type` field, normalized — smart-pointer pointee unwrapped,
void/primitives dropped.
- Preserve the inner-call receiver in extraction: a C/C++ field_expression whose
receiver is itself a call is encoded `inner().method` instead of dropping to the
bare name. Other languages keep the existing behavior.
- New resolution strategy (matchCppCallChain): infer the receiver's class from the
inner call's return type, then resolve AND validate the method on it. Handles
singletons/accessors, factories returning a different type, free-function
factories, make_unique/make_shared/new/direct construction, single-level member
chains, and namespace-qualified inner calls. A wrong inference yields no edge,
never a wrong one.
EXTRACTION_VERSION 2->3 (re-index to populate return types).
Validated on the issue repro + spdlog: node count stable (no explosion),
deterministic, and ~100 pre-existing wrong `.size()`-style edges removed.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Completes the cross-file dependency graph behind impact / affected / explore across all 22 supported languages and 14 web frameworks, validated on real-world repos (measured fair-coverage table added to the README). Per-language resolution + framework resolvers/synthesizers (Lua/Luau require, Shopify OS 2.0 Liquid sections, Delphi forms, Rust cross-module + Rocket macros, Swift Fluent, SvelteKit/Nuxt loader/component conventions, RN/Expo bridges). 0 cross-family false edges, full suite green (1187 passed). See #708.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
## Summary
Completes the explore-overhaul arc: `codegraph_explore` becomes the single primary tool an agent reaches for, and its coverage + output shape are tuned so flow/architecture questions resolve with near-zero Read/Grep.
### What changed
- **explore is the sole primary tool** — removed `codegraph_context` (the fuzzy-input Read-trigger) and `codegraph_trace` (under-picked by agents); explore already surfaces the call flow among the symbols you name. A plain natural-language question now works as the query.
- **Store/handler coverage** — functions defined inside object literals (Zustand `create((set, get) => ({ … }))`, Redux/Pinia/MobX, exported handler/route maps) are indexed as real symbols, including calls through `useStore.getState().fn()` and destructured `const { fn } = useStore.getState()`. A general AST rule, not a per-lib hack.
- **Overload disambiguation** — explore leads with the *right* definition when a method name is overloaded across types (a PascalCase type token in the query biases to that type's own def); `codegraph_node` returns *every* overload's body in one call, with an optional `file`/`line` selector to pin one.
- **Method-atomic render** — explore never returns half a method; at the size budget it drops whole methods/files (and lists what it dropped) instead of truncating a body mid-method.
- **Native-read-shaped output** — per-call output is capped to ~24K with a 25K hard ceiling and concentrated into ~150–250-line flow windows, mirroring how the agent natively reads; repo size scales the *call* budget, not the per-call size (a larger response just gets externalized to a file the host Reads back).
- **Blast radius** folded into explore (dependents + covering tests, locations only).
### Benchmark (refreshed on this build)
Re-validated the 7-repo A/B on 2026-06-02 (Opus 4.8, effort=high, median of 4). WITH arm re-measured on this build, WITHOUT reused:
**~16% cheaper · 47% fewer tokens · 22% faster · 58% fewer tool calls** — 0 file reads on 6 of 7 repos (Gin ~1).
The arc trades larger, cache-heavy explore responses for guaranteed near-zero reads, so cost/token margins soften vs the prior build (Excalidraw and Tokio land at cost break-even) while time and tool-calls stay clear wins everywhere — consistent with the project's stated optimization target (latency + tool-calls, not token cost).
### Validation
- Full suite green: **1112 passed, 2 skipped**.
- 28/28 plain WITH runs across the 7 README repos completed clean; reads median 0 on 6/7.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Closes three gaps that broke `trace(controller, mapper-xml)` on real Spring +
MyBatis projects:
1. **Field-injected concrete-bean trace.** Java `this.<field>.method()` is
unwrapped at extraction (was surfaced as `this.<field>.method` and dropped
through every name-matcher strategy). The receiver name is then looked up
in the enclosing class's field declarations to get the declared type and
resolve the method on it. Closes the controller→bean hop when the field
name doesn't capitalize to the type (`userbo` → `UserBO`). General Java
fix, not Spring-specific.
2. **MyBatis XML mapper as a first-class language.** New extractor parses
`<mapper namespace="..."><select|insert|update|delete|sql id="X">` and
emits method-shaped nodes qualified as `<namespace>::<id>`, plus
`<include refid="X"/>` references to `<sql>` fragments. Non-mapper XML
(pom, log4j, web.xml) → file node only. A new synthesizer
(`mybatisJavaXmlEdges`) joins Java mapper methods to XML statements by
suffix-matching qualified names. Ambiguous simple-name collisions dropped
for precision.
3. **Spring `@Value`/`@ConfigurationProperties` → application config.**
`application.{yml,yaml,properties}` + profile variants parse on the
framework path; each leaf key becomes a `constant` node qualified by its
dotted path. `@Value("${k}")` / `@Value("${k:default}")` and
`@ConfigurationProperties(prefix="X")` emit binding nodes that resolve
with Spring's relaxed binding (kebab↔camel↔snake).
Validated on macrozheng/mall-tiny: full chain
`UmsRoleController.listResource → UmsRoleService.listResource → impl →
UmsResourceMapper.getResourceListByRoleId → XML <select>` connects across 5
hops via static + synthesized edges. 11/11 @Value annotations resolved
(incl. `@ConfigurationProperties(prefix="secure.ignored")`); 6/6 custom-SQL
mapper methods bridge to XML.
Tests: 4 new integration tests in frameworks-integration.test.ts. Full
suite: 1005 passed.
Docs: CHANGELOG `[Unreleased]` entry + dynamic-dispatch-coverage-playbook
narrative + matrix row.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds tree-sitter-objc extractor for `.m`/`.mm` files and `.h` files
that content-sniff as Objective-C (`@interface`/`@implementation`/`@protocol`/`@synthesize`).
Extraction covers:
- `@interface` / `@implementation` (deduplicated into a single class node)
- `@protocol` (as `protocol` nodes via new `interfaceKind` config)
- Methods with full multi-part selectors (`doThing:with:`, not just `doThing`),
including `+`/`-` static distinction
- `@property` declarations
- Inheritance (`extends`) and protocol conformance (`implements`)
- C-style `function_definition` and `#import` (both `<system>` and `"local"` forms)
- Call edges from both `call_expression` and `message_expression`,
with `self`/`super` skipped on qualified callee names
Two new generic hooks on `LanguageExtractor` (`resolveName`,
`extractPropertyName`) handle the cases where the default name walk
doesn't fit; usable by future languages with similar shape.
Import resolver tries `.h`, `.m`, `.mm` for `objc` imports.
Validated on AFNetworking (84 files, 100% file coverage), RestKit
(282 files, 99.6%), and Texture (926 files, 100%, heavy `.mm`
content) — multi-keyword selectors preserved up to 7 parts, no parse
failures on ObjC++.
Known limitations (disclosed in README):
- Categories produce duplicate class nodes (one per category file)
- Chained/nested message sends record only the innermost method
- `[Class alloc]` patterns don't emit `instantiates` edges
- `@protocol Foo <Bar>` refinement lists not yet wired to `implements`
- Heavy C++ in `.mm` files may parse incompletely under the ObjC grammar
Remove .codegraph/config.json and the entire config surface. CodeGraph now
indexes every file whose extension maps to a supported language and respects
.gitignore everywhere — git repos via git itself, non-git projects via the
`ignore` library (root + nested .gitignore files, the same way git does).
- Remove CodeGraphConfig/DEFAULT_CONFIG, src/config.ts, and the public config
API (the `config` option on init, getConfig/updateConfig/getConfigPath).
- Derive the source-file allowlist from EXTENSION_MAP (isSourceFile); maxFileSize
is now a constant. Drop the .codegraphignore marker.
- Behavior change: committed, non-gitignored dirs (vendor/, a committed dist/)
are now indexed — .gitignore is the single source of truth.
Earlier inert fields (languages, frameworks, extractDocstrings, trackCallSites,
customPatterns) and their dead helpers are removed as part of this.
Resolves#283.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Detects Drupal projects via composer.json drupal/* deps; extracts routes from *.routing.yml (route nodes + references edges to controllers/forms/entity handlers) and Drupal hook implementations from .module/.install/.theme/.inc. Adds yaml/twig as file-level languages and excludes core/contrib by default. Resolves#268.
* feat(search): field-qualified queries (kind:/lang:/path:/name:) + fuzzy typo fallback
Two UX improvements that turn a free-text search into something a
real user can drive precisely.
1) Field-qualified queries.
A new query parser (src/search/query-parser.ts) splits the raw query
into structured filters and a free-text remainder:
kind:function name:auth path:src/api authenticate
becomes
{ kinds: ['function'], nameFilters: ['auth'],
pathFilters: ['src/api'], text: 'authenticate' }
Filters compose with the SearchOptions arg (intersection). Unknown
prefixes pass through as plain text so `query "TODO:"` keeps working.
Quoted values (`path:"my dir"`) handle whitespace. When the user
specifies only filters with no text, the search uses a filter-only
candidate scan instead of bailing out.
Recognised today:
kind: any NodeKind value
lang: any Language value (alias: language:)
path: case-insensitive substring of file_path
name: case-insensitive substring of node.name
2) Fuzzy fallback.
When BOTH FTS and LIKE return nothing AND the text is at least 3
chars, the resolver scans the distinct-name set with a bounded
Damerau-Levenshtein-style edit distance (≤2 for ≥5 chars, ≤1 for
4-char queries, off for shorter). Bounded edit-distance early-exits
once the row min exceeds maxDist, so this stays O(distinct-names *
avg-name-length) with a very low constant.
Verified live against ollama/ollama@v0.22.0:
query "kind:function auth" → only function-kind hits
query "lang:go path:server route" → Go files under server/
query "getUssr" (typo) → finds getUser, SetUser
query "confg" (typo) → finds Config
Full test suite: 380 passed.
* fix(search): address reviewer findings — tokenizer mid-token quotes, fuzzy fan-out cap, larger filter-only over-fetch, unit tests
Five fixes from independent review:
- parseQuery tokenizer: quotes that appear MID-token (path:"my dir/
file") were not being recognised — only quotes at the start of a
token were treated as quoted spans. The fixture path:"my dir"
parsed as ['path:"my', 'dir"'] instead of ['path:"my dir"'].
Tokeniser is now a single state machine that scans into a token
until whitespace OR a quote, and recognises quotes anywhere within
the token (skips to the matching close quote).
- searchNodesFuzzy: cap the per-name follow-up SQL queries at
Math.max(limit*2, 50) AFTER edit-distance filtering. Without
this, a project with many similar names (getUser1, getUser2...)
could fan out far beyond limit queries before the inner-loop
break kicks in.
- searchAllByFilters (filter-only no-text path): bumped over-fetch
multiplier from 2× to 5× so a selective post-filter (e.g.
path:src/very/specific/file.ts) doesn't return fewer than limit
results despite the DB having matches.
- 23 new unit tests in __tests__/search-query-parser.test.ts:
parseQuery covers known-field filter, lang/language alias,
multiple kind: ORs, quoted spans (incl. mid-token), URL
passthrough, empty-value passthrough, unknown prefix passthrough,
unknown value passthrough, all-filters-no-text, empty input,
20k-char input. boundedEditDistance covers identity, single
insertion/deletion/substitution, length-difference shortcut,
empty inputs, case-sensitivity, early-exit correctness.
Full test suite: 853 passed (up from 830).
* refactor(search): derive parser kind/lang sets from types.ts as const
Convert NodeKind and Language to runtime-iterable as const arrays
(NODE_KINDS, LANGUAGES) so the query parser imports the canonical
list instead of duplicating it. Also fix the path: JSDoc to say
substring (matches the .includes() impl).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Colby McHenry <me@colbymchenry.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds expression index on lower(name) for memory-efficient case-insensitive searches, replacing in-memory caches that caused OOM on large codebases. Includes batched reference resolution, enhanced error reporting with detailed breakdown by error type, and improved CLI progress display for scanning phases.
Testing showed semantic search produces significantly better results for
natural language queries that Claude writes. FTS alone often ranks
properties above their parent classes and misses conceptual matches.
Embeddings are now always on — the vector manager is created eagerly,
with model download and embedding generation still happening lazily.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Integrate main branch changes (WASM grammar architecture, centralized
resolution caches, SQLite adapter) with delphi-support branch. Pascal
grammar is now built as WASM and shipped in src/extraction/wasm/ for
consistency with the WASM-based grammar loading approach.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
These are project-specific directories and should be configured per
project in .codegraph/config.json, not hardcoded globally. Delphi-
specific excludes (__history, __recovery, *.dcu) remain as global
defaults since they are analogous to __pycache__ for Python.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add _libs/**, libs/**, __history/**, __recovery/**, and *.dcu to
default exclude list. Delphi projects store external dependencies
in _libs/ or libs/ (often as Git submodules) and these should not
be indexed — same as node_modules for JS projects.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Fixes#28 - Python site-packages directories (e.g.
audio_tools/python/Lib/site-packages/) were not excluded by default,
causing massive index bloat and FOREIGN KEY failures when indexing
large libraries like tensorflow. The FK crash itself was already fixed
via INSERT OR IGNORE, but excluding these directories prevents the
bloat in the first place.
- Remove getNodesByIds, getNodesByKinds (zero callers)
- Remove getFileHashMap, getFileSyncMap (zero callers)
- Remove getDynamicStmt cache (only used by removed batch methods)
- Revert getStaleFiles to simple implementation (zero callers, no need to optimize)
- Remove intent field from SearchOptions (never read in search logic)
- Add provenance column on edges for tracking how edges were created
- Add project_metadata table for version/provenance tracking
- Make unresolved_refs file_path/language NOT NULL with defaults
- Add composite indexes for unresolved_refs and edges.provenance
- Update v2 migration to handle all new schema additions
- Record schema version on initialize to prevent re-migration
- Add dynamic prepared statement cache (getDynamicStmt) for varying SQL
- Add batch methods: getNodesByIds, getNodesByKinds, getFileHashMap, getFileSyncMap
- Add project metadata methods: getMetadata, setMetadata, getAllMetadata
- Optimize getStats to use single aggregate query
- Optimize getStaleFiles to use temporary table JOIN
- Add provenance parameter to getOutgoingEdges
- Add intent field to SearchOptions type
- Add 'svelte' to Language type, DEFAULT_CONFIG includes, grammars, and config validation
- Add SvelteExtractor that extracts <script> blocks and delegates to TS/JS TreeSitterExtractor
- Add Svelte framework resolver for runes ($state, $derived, $effect, etc.), store auto-subscriptions, SvelteKit module aliases ($app/*, $env/*, $lib/*), and SvelteKit route detection
- Update README to list Svelte and Dart in supported languages
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- SQLite performance pragmas: synchronous=NORMAL, 64MB cache,
memory temp store, 256MB mmap (safe with WAL mode)
- Batch insert for unresolved refs: single transaction instead of
N individual inserts per file
- Symbol caching (warmCaches): pre-load all nodes into memory maps
before resolution, eliminating repeated SQLite queries per ref
- Async file I/O: fs.stat/readFile in indexFile() are now non-blocking
- Denormalize filePath/language onto UnresolvedReference: avoids N
node lookups during resolution, with schema migration v2
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds support for Dart and Liquid languages with tree-sitter parsing.
Improves accuracy of code symbol extraction for existing languages.
Indexes project files to enhance code navigation features.
Migrates build system to facilitate code contributions.
Removes git hook functionality.
Integrates Sentry for error tracking and reporting.
Enhances project initialization and configuration loading.
- Add 'liquid' to Language type and default include patterns
- Create LiquidExtractor with regex-based extraction (tree-sitter-liquid has ABI issues)
- Extract render/include/section references as component nodes
- Extract schema blocks as constant nodes with parsed names
- Extract assign statements as variable nodes
- Create file relationship edges for snippet/section references
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add codegraph_explore MCP tool for deep exploration with condensed output
- Expand default exclude patterns for framework build outputs (.next, .nuxt, .expo, etc.)
- Increase node ID hash length from 16 to 32 chars to prevent collisions
- Add feature request detection with UX clarification reminders
- Update README with MCP tools reference and best practices
- Update CLAUDE.md with context usage guidelines
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Colby McHenry
ctype_lab
0x1306a94andGitHub
Marcelo VaniandGitHub
Mark HandGitHub
Olaf MonienandClaude Opus 4.6
Martin Oehlert