cache.Put("a", 1), store.Get("config", out), bus.Handle("user.created",
h) — any verb-named method with a string first arg — were indexed as
HTTP routes (38 of 82 route nodes were false positives on the
reporter's 200 KLOC Go codebase). A registration's first argument must
now start with "/" (every router style), or be a Go 1.22
"METHOD /path" mux pattern on Handle/HandleFunc — which now also
extracts the real method instead of ANY.
Validated on go-chi/chi (212 real routes retained, all path-shaped)
and golang/groupcache (0 route nodes).
Fixes#1259
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
codegraph's glyphs were ASCII on every Windows console while
@clack/prompts drew its Unicode frame around them, so one index block
mixed `|` and `│` rails (#398). supportsUnicode() now mirrors the
is-unicode-supported detection clack bundles (Windows Terminal, VS
Code, ConEmu/Cmder, Alacritty, xterm-256color, JetBrains, CI), so both
systems always pick the same glyph family.
The shimmer worker's raw fs.writeSync(1) bytes still decode through the
console codepage (OEM codepages mojibake UTF-8 even under Windows
Terminal — the #168 regression to avoid), so:
- the raw path gets its own supportsUnicodeRawWrites() that stays ASCII
on win32 unless CODEGRAPH_UNICODE=1, and
- the persistent "phase done" lines move from the worker to the parent,
written via process.stdout (wide-char console API, codepage-immune) at
phase transitions — the main thread is alive there, it's delivering
the progress callback. Only transient, self-erasing animation frames
remain on the raw path, so ASCII never lands in scrollback.
Fixes#398
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
List commands (status, query, callers, callees, impact, files) embedded
ANSI color codes even when stdout was a pipe, and NO_COLOR had no effect.
One switch now decides color for all codegraph-authored output:
--no-color > --color > NO_COLOR > FORCE_COLOR > stdout TTY > CI.
Piped init/index/sync also stop emitting shimmer animation frames
(\r + erase-line rewrites) and print one plain line per phase instead;
a TTY with NO_COLOR keeps the animation but drops the color codes.
The detection mirrors picocolors' so @clack frames and our own lines
agree within a run.
Fixes#1281
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* perf(index): ~34% faster fresh indexing, byte-identical graphs
Profiling a fresh init on a medium TS repo (excalidraw, 657 files) showed
the main thread as the critical path: per-row SQLite statement calls,
repeated import-resolution walks, and per-row FTS trigger firings, with
the parse workers ~75% idle behind it. This lands the semantics-preserving
tranche of fixes:
- Multi-row batched INSERTs (nodes/edges/unresolved refs/name segments)
behind cached per-batch-size prepared statements; row order preserved,
so rowid-based resolution determinism (#1015) is unchanged.
- storeFileBundle: one transaction per file instead of four; nested
transaction() calls now flatten (BEGIN-in-BEGIN previously threw, so no
caller depended on nested rollback).
- Dedicated store-writer thread for the fresh-DB bulk path (bundles
applied in file order on a single writer connection; main thread does
no DB work during the parse loop). Kill switch: CODEGRAPH_NO_STORE_WORKER=1.
- Bulk FTS mode: drop the nodes_fts sync triggers during the bulk load,
rebuild once at the end; crash inside the window self-heals on the
next open.
- Per-context memos for resolveImportPath/findExportedSymbol + a per-file
exported-symbol index, invalidated exactly where clearCaches() already
resets the resolver's own caches.
- Fast-init on completely fresh DBs (journal in memory, no fsync until
the index completes; interrupted init re-runs from scratch). Kill
switch: CODEGRAPH_NO_FAST_INIT=1.
- MaybeYield returns undefined on the not-due path so per-ref yield
checks stop paying a promise + microtask hop each.
- Parse pool prewarm for bulk indexing; compile-cache enabled at CLI and
worker entry points.
Excalidraw fresh init: 5.11s -> 3.36s median (n=5, warm cache, M-series).
Graph dumps byte-identical across init, re-index, and sync paths; full
suite green (2403 passed).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* perf(resolution): parallel reference resolution with canonical admission
Fan resolution batches across a pool of read-only worker threads, each
hosting a full ReferenceResolver over its own SQLite connection; results
are admitted on the main thread in chunk order, so edge insertion order,
row cleanup, failure parking, and deferred post-pass queues are exactly
the sequence the single-threaded loop produces. Per-ref inputs match the
baseline because the sequential path already resolves each batch against
the state committed BEFORE that batch.
Validated byte-identical on excalidraw (pool forced on) and apache/dubbo
(4,048 Java files): dubbo full index 39s -> 19s (2.05x) with identical
graph dumps (91,495 nodes / 223,953 edges).
The pool only engages when total pending refs clear a threshold (default
150k, CODEGRAPH_PARALLEL_RESOLVE_MIN to tune, CODEGRAPH_NO_PARALLEL_RESOLVE=1
to disable): measured on a ~58k-ref repo the workers' boot CPU contends
with resolution on the same cores and makes indexing slower, so small
repos keep the sequential path. When fast-init left the DB in
memory-journal mode, WAL is restored before resolution only when the pool
will run (readers + rollback-journal writers don't mix).
Also: sqlite adapter readOnly open support.
TreeCursor spine rewrite of the body walker was built, measured neutral
on real repos and equal in a 20k-child microbench (web-tree-sitter's
namedChild(i) is not quadratic in this binding), and rejected — per-node
JS<->WASM marshaling is the floor, which a traversal swap cannot remove.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Follow-up to #1299: the per-pass bar still parked on one number while a
single long pass ran — on C-heavy repos that's the fn-pointer dispatch
pass, which sweeps every C/C++ file four times (typedefs, registrations,
field propagation, dispatch sites) and dominates the linking phase.
The pass now reports a real fraction of its dominant work
(scannedFiles / files×4, at the same per-16-files cadence as its
cooperative yield), and the orchestrator surfaces instrumented passes'
fractions as fractional steps, throttled to whole-percent movement so
the UI message volume stays bounded. The mechanism is opt-in per pass —
any synthesizer that a real repo shows parking the bar can adopt the
same callback.
Verified on the 1,342-file C repo from the report: the linking bar now
moves through 88→89→90 where it previously sat at 88 for the whole
pass; graph byte-identical (50,520 nodes / 148,232 edges).
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Two first-run UX bugs surfaced by indexing a real 1,342-file C repo:
1. After 'Resolving refs' hit 100%, the ~40 dynamic-dispatch synthesis
passes ran with no progress surface, so the bar sat frozen at 100%
long enough to read as a hang (the C fn-pointer pass alone can hold
for a while on C-heavy repos). Synthesis now reports per-pass
progress through a new 'linking' IndexProgress phase, rendered as
'Linking dynamic dispatch'. The step total is pinned by a test to
the synthesizer's actual __mark() count so adding a pass without
bumping it fails loudly.
2. node:sqlite's ExperimentalWarning is emitted once per THREAD, so the
main process plus every parse worker printed it mid-index,
interleaved with the progress UI. All launch paths now pass
--disable-warning=ExperimentalWarning: both bundle launchers, the
Windows npm-shim invocation, and the CLI self-relaunch
(NODE_RUNTIME_FLAGS, deliberately excluded from the re-exec gate so
an older installed launcher never triggers a pointless re-exec, and
version-gated off nodes older than the flag).
Verified end-to-end on the same repo: zero warnings, live linking bar,
byte-identical graph (50,520 nodes / 148,232 edges). Full suite green.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
One-time, strictly opt-in prompt at the end of codegraph install and
codegraph upgrade to join the CodeGraph Pro beta waitlist (same list as
the getcodegraph.com homepage form). Nothing is sent unless the user
answers yes AND enters an email; either answer is recorded machine-wide
so no later install or upgrade re-asks, and --yes / non-interactive / CI
runs never see the prompt.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Post-batch cleanup deleted resolved refs (and parked failed ones) by
(from_node_id, reference_name, reference_kind) — no line/col. When one
caller had several call sites to the same callee and a batch boundary
split them, the first batch's cleanup removed every row with that key,
including later-batch siblings that were never attempted — their edges
were silently never created. On nlohmann/json this ate 422 real call
edges (write_cbor's 38 to_char_type calls indexed as 11).
Refs loaded from unresolved_refs now carry their row id through
resolution, and all three persist paths (sync resolveAndPersist, the
yielding retry pass, the batched drain loop) delete / mark-failed by
exactly that id. The key-tuple methods remain only as the fallback for
hand-built refs from the public API. Failed-parking gains the same
precision: outcome can differ per call site (receiver inference reads
the ref's line), so a sibling must not inherit another row's failure.
Also untracks the zz-scratch local test files that slipped into #1268
and gitignores the pattern.
Validation: red-green regression test (5 sites, batch size 2 — old code
kept 2 edges, fix keeps 5); nlohmann/json re-index is a strict superset
of the previous edge set (0 lost, 422 recovered, spot-checked against
source).
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(cpp): resolve explicit operator calls (a.operator+(b)) to the operator method (#1247)
tree-sitter-cpp can't parse an operator_name in field position: the
call_expression carries `function: <receiver>` plus an ERROR child
wrapping the operator_name instead of a field_expression callee, so the
extractor emitted a calls ref named just the receiver (`a`) and the edge
never resolved — while the operator method itself indexed fine.
Two-part fix, scoped to the explicit call form (infix `a + b` / `a[i]`
need receiver type inference and are tracked in #1258):
- extraction: recover the operator_name from the ERROR child and emit
`<receiver>.operator+` (`->` receivers normalized, `this->` emits the
bare name), like any other member call
- resolution: matchMethodCall's dot pattern now admits an operator
method part (cpp-gated; symbol chars failed the \w match), so
receiver-type inference + resolveMethodOnType validate the target —
a same-named operator on an unrelated class can't capture the edge
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(cpp): harden explicit operator-call recovery against real-world shapes (#1247)
Validated on nlohmann/json (dozens of explicit operator[] / operator* /
operator< call sites). Two refinements the synthetic fixtures missed:
- normalize spaced call-site operator names (`it.operator * ()`,
`other.operator < (*this)`) to the compact form definitions index as
- drop the ref for a complex receiver (`obj()->operator+`, member chains
ending in a call) instead of emitting a bare operator name: exact-name
fallback GUESSED among unrelated same-named operators (linked a
std::map operator[] call to an in-repo operator[]) — silent miss,
never a wrong edge
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(uninstall): remove the CLI binaries too, not just agent configs (#1071)
`codegraph uninstall` swept agent configurations and stopped — every
installed binary stayed behind, so `codegraph` still ran afterward. Three
disconnected paths each removed a fraction of an installation (uninstall:
configs; install.sh --uninstall: the bundle; npm preuninstall: configs +
npm's own package), and none cleared a shadowed second install — the
uninstall edition of the #1071 PATH shadow.
The uninstall now PLANS every install present on the machine — the bundle
layout(s) (running binary's own, the platform default, a custom
CODEGRAPH_INSTALL_DIR), the npm global package (found by asking
`npm root -g`, so nvm/fnm/volta prefixes resolve correctly), and the
bin-dir launcher link (only when it verifiably points into a detected
install) — confirms with the user, then removes them all. `--yes` skips
the prompt; the new `--keep-cli` flag keeps the old configs-only behavior.
Safety rules: a source checkout is reported, never deleted; a
project-local npm install is left to the project; on unix the default
install dir doubles as the machine state dir, so only the install
artifacts (versions/, current) are removed there — telemetry choice and
daemon records survive. Windows can't delete a running exe but can rename
it (the in-place upgrade's trick): a locked node.exe is renamed aside and
surfaced as a one-file leftover instead of failing the removal, and npm
is routed through cmd.exe (a direct .cmd spawn EINVALs on modern Node).
Planner/executor are split with injected side effects (the upgrade
orchestrator's convention) and unit-tested across the shadow case,
state-dir preservation, custom dirs, foreign-shim protection, and the
locked-exe dance; validated end-to-end on macOS against a fake HOME.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(uninstall): key path math on the target platform, not the host
Real-Windows validation caught it: the planner/executor used the host
path module, so win32 fixtures were meaningless on a POSIX host and
POSIX fixtures failed on the Windows VM. Same convention as
detectInstallMethod now — path.win32/path.posix chosen by the injected
platform.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(upgrade): route npm through cmd.exe on Windows — a direct npm.cmd spawn EINVALs on modern Node
Found while validating the uninstall change on the Windows VM: upgradeNpm
spawned npm.cmd without a shell, which every current Node rejects with
EINVAL (the CVE-2024-27980 hardening) — so `codegraph upgrade` on a
Windows npm install failed before doing anything. Verified live on the VM:
spawnSync('npm.cmd') → EINVAL; cmd.exe /d /s /c npm → works.
npmInvocation moves into the upgrade orchestrator (remove-binary imports
it from there — same direction as its existing imports, no cycle), and the
win32 test now pins the WORKING invocation instead of the broken one.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The recommended MCP config launches the local binary, so a server left
running drifts behind releases silently — users discover the version gap
only when something breaks. Per the reporter's preferred option 1, the
server now checks the latest GitHub release in the background on startup
(never blocking; reuses the upgrade command's release-redirect resolution
so the two can't drift) and surfaces a one-line notice on three surfaces:
one stderr line (the MCP host's server log), the initialize instructions
(with do-not-run-it-yourself guidance for the agent), and codegraph_status.
Discipline: results cache in ~/.codegraph/update-check.json shared across
every proxy/daemon on the machine — 24h TTL on success, 1h backoff after
failure, an outage never hides an already-known update, and a stale cache
re-kicks a background refresh so long-lived daemons keep noticing. The
initialize path is a memoized synchronous cache read (the respond-fast
contract holds), and both handshake answerers (session + proxy) share one
helper so they can't diverge. Never stdout.
Hardening: the latest tag arrives from a network redirect via an on-disk
cache and ends up inside agent-visible instructions, so only a canonical
vX.Y.Z rebuilt from PARSED semver fields is ever interpolated — a tag
carrying trailing text (parseSemver is not end-anchored) renders without
it, and a non-version tag renders nothing and counts as a failed attempt.
Off is off: CODEGRAPH_NO_UPDATE_CHECK=1 (dedicated) or DO_NOT_TRACK=1
(broad convention — already set by data-plane deployments) suppresses the
network call and the notice entirely. Documented in TELEMETRY.md.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
handleExplore's named-symbol seeding treats every identifier-shaped query
token as "a symbol the agent named" and grants its definition the
named-FIRST sort tier. Natural-language questions broke this assumption:
ordinary words exact-matched unrelated callables ("check" ->
WalCheckpointValve.check, "serve" -> query-worker serve, "initialize" ->
DatabaseConnection.initialize), and those collisions outranked — and within
the per-repo file budget fully displaced — the corroborated answer files,
forcing the agent back to Read/Grep. The >3-def single-pick fallback had
the same hole: on grpc, the #1064 flagship query "add a parameter to
NewClient" itself tiered balancerStateAggregator.add's file to slot #1.
Guard: a shape-precise token (camelCase, PascalCase, snake_case,
qualified) seeds unconditionally — it is an unambiguous symbol reference.
A bare lowercase word seeds only defs whose file another query token
co-names (that token is itself an exact symbol name defined in the same
file — the "check drain fire" sibling-bag shape), which an incidental
English-word collision never is. Applied by filtering cands ahead of both
branches so the overloaded-name fallback is covered too.
Validated per the retrieval playbook: deterministic probes on this repo
(collision queries fixed; sibling-bag and single-camelCase retained), and
baseline-vs-fixed probes on the #1064 repos — Alamofire and excalidraw
byte-identical, grpc improved (the add-collision file drops out and
clientconn.go + dialoptions.go lead). Full suite green.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Carries #1221 by @w0lan plus a hardening pass: property-receiver typing consults property-shaped declarations only (typed property / promoted ctor param / pseudoconstructor assignment / assignment-followed classic ctor and typed setter), so same-named locals and parameters can never mistype a property.
Co-authored-by: Roman Wolan <roman.wolan@morizon-gratka.pl>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* chore: ignore .kommandr/ directory
* fix(sync): resolve cross-file refs when an edit adds or removes the satisfying symbol (#1240)
Incremental sync scoped reference resolution to the changed files' own
refs, and a completed pass deleted every ref it failed to resolve — so
a symbol change in one file could never repair references in UNCHANGED
files, in either direction, until a full re-index:
- New-export case: a.ts imports/calls `greet` before b.ts defines it.
The failed refs were deleted at index time; when b.ts later gained
`greet`, nothing revisited a.ts — the calls/imports edges stayed
missing while status reported a clean index.
- Removal case: when a re-index (or file deletion) dropped a symbol,
the incoming edges cascade-deleted and the callers — whose resolved
refs had been consumed — never got a chance to rebind to an
alternative definition or reconnect when the symbol returned.
Fix, sharing one lifecycle:
- Schema v8: unresolved_refs gains status ('pending'/'failed') and
name_tail (last dotted segment, so `h.greet` is findable by `greet`).
Both resolver persist paths now park unresolvable refs as failed
instead of deleting them. All pending-work readers (batched drain,
non-progress guard, #1187 orphan sweep, status pendingRefs) filter to
pending, preserving their invariants and keeping status honest.
- Sync retry: after scoped resolution, failed refs whose name tail
matches a symbol name now present in the changed files are re-resolved
through a per-ref-yielding path (watchdog-safe, #1091 class). Names
matching >500 failed refs are skipped as external/builtin noise (#999
rationale).
- Removal side: createEdges stamps each resolution edge with its
originating reference (metadata.refName, + refKind when kind promotion
rewrote it). When the #899 restore misses a target or sync deletes a
file, the dropped edge is resurrected as exactly that ref — re-resolved
in the same sync (rebinding to an alternative definition) or parked
failed until the symbol reappears. Edges without the stamp (pre-upgrade,
synthesized) still drop silently: reconstructing from the target's plain
name would strip receiver context and risk a rebind a full re-index
would never make.
- Pure-removal syncs clear resolver caches so a long-lived daemon can't
resolve resurrected refs against the pre-removal graph.
Validated: issue repro now yields a graph byte-identical to a full
re-index; move/remove-readd/file-deletion scenarios all rebind or heal;
baseline-vs-new A/B on express and gin shows identical node/edge counts
and no timing regression (DB grows ~25% from the parked ref rows — pure
cache, reset by any full re-index). 8 regression tests added.
Fixes#1240
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(upgrade): refresh installer-written agent surfaces after a binary upgrade
codegraph upgrade swapped the binary but never revisited what earlier
installs wrote into CLAUDE.md / AGENTS.md / GEMINI.md and the agent
configs, so sections written by a pre-1.0 installer kept teaching agents
a multi-tool surface (including tools that no longer exist) months of
releases later. The install path already self-heals everything it owns,
but nothing ever called it on upgrade.
- codegraph install --refresh: non-interactive sweep that re-runs
install() for already-configured targets only — never a first
install; permissions and prompt-hook choices are preserved.
- codegraph upgrade spawns it via the freshly-installed binary after a
successful swap (the still-running old process would only rewrite its
own stale template). Gated on PATH resolution and the
CODEGRAPH_NO_INSTALL_REFRESH=1 kill-switch; never fatal to the
upgrade.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs(installer): clarify refresh change reporting
---------
Co-authored-by: xuing <np2v9bvbbs@privaterelay.appleid.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Colby McHenry <me@colbymchenry.com>
Two fixes to make `codegraph upgrade` trustworthy in the terminal it ran in:
1. detectInstallMethod checked the bundle layout before the node_modules
path check, but the npm thin-installer's per-platform package IS a
complete bundle inside node_modules — so every npm install misdetected
as a standalone bundle, and upgrade curled install.sh into ~/.codegraph:
a second install that never wins the PATH race against npm's shim,
leaving `codegraph -v` permanently on the old version. Path-based
checks (_npx, node_modules) now win over layout sniffing, so npm
installs upgrade through npm again, in place.
2. After a successful swap, runUpgrade now probes the PATH-resolved
`codegraph --version` and reports the real outcome: a green
confirmation that this terminal already serves the new version, a
loud shadow warning naming the fix (`which -a codegraph`) on
mismatch, or the old soft new-terminal hint only when the probe is
inconclusive. Replaces the unconditional "open a new terminal if the
version looks unchanged" hedge. Skipped for npm-local installs, whose
binary PATH never serves.
Companion to #1239: the misdetection also broke its post-upgrade
`install --refresh` for npm users (the spawn resolved the stale shim).
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The liveness watchdog judged the main thread by heartbeat silence alone,
which cannot distinguish a true wedge (the #850 infinite loop it exists to
kill) from one long synchronous SQLite statement on severely degraded
storage — so it SIGKILLed valid, in-progress indexes (observed on a
150-IOPS throttled rig, and latent on real HDDs at scale).
The CLI index/init paths now hand the watchdog the project's DB + WAL
paths. On a silent timeout the watchdog child stats them first: if they
advanced during the silence, the block is a slow store making forward
progress — defer and keep watching; if not, kill at the base timeout
exactly as before. Deferral is bounded by a hard cap (10× the timeout) of
continuous silence so a wedge coinciding with unrelated file activity, or
I/O hung beyond any legitimate statement, still dies. The daemon path is
unchanged (no progress paths — pure heartbeat).
Validated with real spawned processes (defer-on-progress, kill-on-static,
hard-cap kill) and on the throttled rig: a 150-IOPS index under a 10s
watchdog window — 6× tighter than production, with store stalls measured
at 10-20s — completes cleanly where the old watchdog killed it, while
true-wedge kill latency is unchanged.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Parse timeouts are now judged by the worker's own clock: the base timer
only marks a job late (after a long synchronous store stall, Node runs the
timers phase before the poll phase, so the timer fired before an
already-delivered result was processed — killing workers over parses that
took milliseconds, even on 0-byte files); a result arriving before a 3×
hard-kill backstop is accepted, timed-out files are retried, and
CODEGRAPH_PARSE_TIMEOUT_MS overrides the budget. Grammar WASM bytes are
pre-read once on the main thread and handed to every worker, so
spawns/respawns load grammars from memory instead of re-reading a
saturated disk.
Bulk indexing defers WAL auto-checkpointing for the whole run: the default
1000-page interval re-writes hot B-tree/FTS pages into the main DB file
over and over — ~95% of all disk I/O under throttled measurement. A
WalCheckpointValve bounds WAL growth with off-thread PASSIVE backfill
passes (never blocking the writer or the #850 watchdog heartbeat), pauses
the writer for a full backfill if the disk truly can't keep up, and folds
the WAL at the parse→resolution boundary so post-parse reads never page a
bulk-write-sized WAL. Opt out with CODEGRAPH_NO_WAL_DEFER=1; tune with
CODEGRAPH_WAL_VALVE_MB.
Measured at 150 IOPS (HDD class): commons-lang 1526s → 59s with 0 dropped
files (was 8); guava-scale completes in 7.6 min with a full graph where
v1.3.1 needed 25 min for a repo 5× smaller. Unthrottled: no change.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
closureCollectionEdges scanned every method/function node in every
language, but its dispatcher patterns ({ $0( / { it( ) are Swift/Kotlin
trailing-closure syntax — on PHP/JS repos the pass can never emit an
edge, yet .push(/.add( fired its append gate on nearly every function.
Per match it computed the line via src.slice(0, idx).split('\n'), which
is O(source) per match and goes quadratic on match-dense generated
functions (two-byte content roughly doubles it). On a 12,860-file
PHP/JS app that was 20+ minutes of the "Resolving refs" tail — frozen
at 97% — and a #850 watchdog kill; profiled on CRMEB it was 127s of a
166s index for zero edges.
- Skip nodes whose language isn't swift/kotlin before any file I/O.
- makeLineAt(): lazy newline index + binary search, shared with the
emitter passes' per-file lineOf.
- Yield every 256 regex matches inside the scan's match loops so a
single pathological function can't starve the watchdog.
CRMEB (ThinkPHP, 2,913 files): 166.8s -> 72.7s, graph byte-identical.
Alamofire: closure-collection edges byte-identical (9 edges, 4 fields).
Drupal core control: graph byte-identical.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Large-codebase indexing died at the end of "Resolving refs" two ways:
watchdog kills of healthy work (24k-file Java on Windows, #1212 — third
iteration of the #1091/#1122 class) and hard OOMs (Linux kernel scale,
where v1.3.0 could not complete at any watchdog setting). Root causes:
~31 of 37 dynamic-edge synthesis passes ran start-to-finish with no
yield points, several materialized whole-graph snapshots (kotlin
expect/actual opened with getAllNodes() — 2M nodes in one array; the
C fn-pointer pass retained every C file's contents twice plus every
function node), and the post-index WAL checkpoint ran minutes of
synchronous IO on the main thread, killing even a successful index at
the finish line.
The pipeline tail now follows the same discipline as the rest: never
hold O(graph) in the heap, yield everywhere.
- All synthesis passes stream node-kind scans (cursors, not arrays) and
yield on time-budgeted checkpoints; language gates skip passes whose
filters a project's file languages provably can't satisfy.
- kotlin expect/actual filters SQL-side; c-fnptr caches are LRU-bounded,
units stream one file at a time, and the all-functions array +
write-only id map are gone; spring reads each .java once, not twice.
- runMaintenance moved to a worker thread (own SQLite connection);
per-file store commits chunk with yields behind a serialized flush
chain (preserving #1015 file-order determinism); resolver warm-up
streams the DISTINCT name set; resolution batch-tail and merged-edge
inserts run in bounded sub-transactions.
- Daemon: fixed a socket-handoff race that could leave a fresh MCP
session permanently silent (client-hello tail unshifted into a
flowing stream with zero listeners — the long-standing #662 test
flake was this real bug); first tool call no longer queues behind
the query pool's cold start (pool.ready gate).
Validation: Linux kernel (70,129 files, 2.05M nodes, 6.4M edges) fully
indexes in 27m8s on a 2-core/6GB container at default heap + default
watchdog; llvm-project (180k files) completes under 1GB RSS including
kill-and-sync recovery; synthesized-edge and full-graph parity are
byte-identical vs baseline on elasticsearch/redis/vim; the ex-flaky
daemon test passed 25/25 under load. Env-gated diagnostics kept:
CODEGRAPH_SYNTH_TIMINGS pass/phase timings, CODEGRAPH_MCP_DEBUG hop
tracing. Design record: docs/design/main-thread-stall-followup.md.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
On a large Java/Kotlin Spring monorepo, reference resolution — not extraction —
dominated a full index (Spring Boot's ~9,650-file tree: extraction 62s,
resolution ~26min). The Spring framework resolver ran an uncached
getNodesByKind('constant') full scan + canonicalConfigKey() filter for EVERY
dotted `calls` ref (every list.add(), builder.build(), receiver.method()),
because the config-key branch gated only on "dotted java/kotlin", not on ref
kind. With ~1,100 constant nodes × ~200k dotted calls that is ~200M wasted
row-fetches/allocations.
Fixes, one theme — config-key constants bind config `references`, never `calls`:
- frameworks/java.ts: gate the Spring config-key branch on
referenceKind === 'references' (what @Value/@ConfigurationProperties emit) so
the `calls` flood skips the scan.
- name-matcher.ts: a `calls` ref no longer resolves to a yaml/properties config
node via matchByQualifiedName (service.process() vs the yaml key
service.process) — a wrong edge that also hid the real callee; it now falls
through to method resolution.
- resolution/index.ts: cache getNodesByKind in the resolver context (same
lifetime as nameCache). Fixes the same uncached-per-ref scan in the Drupal
hook_ resolver and is defense-in-depth for the Spring :prefix branch.
Measured (Spring Boot): resolution 269s→16.5s on a 4.3k-file module (16×) and
~26min→44.7s on the full 9.6k-file tree (~35×); graph byte-identical, full suite
passes. Adds a regression test (same key, two ref kinds, opposite outcomes).
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A Git super-repo whose `.gitignore` excludes its child repositories indexed
~nothing at the parent: CodeGraph respects `.gitignore` by default (#970,
#1065), so the excluded children were skipped and `codegraph init` printed
"Done" with 0 nodes — even though `init` inside each child worked fine. The
empty index was silent and unexplained.
`init`/`index` now detect the gitignored child repos they skipped when an
index comes up empty of symbols, name them, and — in an interactive terminal
— offer to index them (writing an `includeIgnored` entry to codegraph.json and
re-indexing on the spot); non-interactive runs print the exact codegraph.json
snippet to add. Gated on nodesCreated === 0, so a project that deliberately
keeps gitignored reference clones out of a working index is never nagged.
- extraction: findUnindexedIgnoredRepos — the inverse of discoverEmbeddedRepoRoots
(bounded, skips default-ignored dirs, respects existing includeIgnored)
- project-config: addIncludeIgnoredPatterns — create/merge codegraph.json,
idempotent, refuses to clobber malformed JSON
- cli: wire the detect-name-offer flow into both `init` and `index`
- tests: +13 covering detection, config writing, and the no-nag gate
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Carries #1133 forward onto current main (rebased for conflicts). A lean Unreal-Engine-style `.h` whose only C++ signal is `class ENGINE_API Foo : public Bar` (no public:/virtual/namespace/template) was misdetected as C and its class + inheritance edge silently dropped; looksLikeCpp now recognizes the export-macro-annotated class/struct shape, matching what blankCppExportMacros already recovers.
Fixes#1159.
Co-Authored-By: robertyluo <robertyluo@tencent.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Heavily-reflected UE C++ classes (`UCharacterMovementComponent`, `AActor`, `UWorld`, …) were dropped from the index: in-body no-semicolon reflection macros (`UPROPERTY`/`UFUNCTION`/`GENERATED_BODY`/`UE_DEPRECATED_*`), member-level `*_API` prefixes, and mid-line `UMETA`/`UPARAM`/`UE_DEPRECATED` accumulate tree-sitter parse errors until the enclosing class_specifier collapses into an ERROR node. Three offset-preserving, C++-only pre-parse passes (`blankCppAnnotationMacroCalls`, `blankCppApiPrefixMacros`, `blankCppInlineAnnotationMacros`) blank the markup before parsing so the class survives.
Validated on the real Epic engine headers (CharacterMovementComponent.h / Actor.h / World.h): each main class + its base clause goes from DROPPED on main to recovered with the fix.
Closes#1160.
Thanks @luoyxy for the report and root-cause analysis.
Adds an `include` list to the root `codegraph.json` that forces gitignored first-party source (second-VCS / SVN / Perforce dual-tracked repos) into the index — discovered directly off disk on the full index, incremental sync, and file-watching, on both git and non-git projects. Gitignore-style patterns, root-relative; explicit `exclude` still wins and built-in skips (node_modules, dist, .git) are never re-included. Complements `exclude` and `includeIgnored`.
Closes#1163.
Thanks @luoyxy for the contribution.
Four gaps in the MyBatis mapper extractor, all reported and reproduced by
@ESPINS in #1182 and verified against main:
1. Single-quoted attribute values (namespace/id/refid/resultType/parameterType)
were dropped — the regexes hardcoded double quotes. Now accept either quote
via a backreference.
2. Tags inside <!-- ... --> produced phantom statement/include symbols. A
length-preserving, CDATA-aware pre-pass blanks comments before scanning,
keeping offsets/line numbers intact.
3. Legacy iBatis 2 <sqlMap> files had zero statement coverage (the root finder
gated on a <mapper namespace> root). It now also recognizes <sqlMap>
(namespaced and namespace-less DAO.method ids) and iBatis's extra
<statement>/<procedure> verbs — closing the gap with no new dependency
(option (c) from the issue; the batis-xml parser route is declined).
4. Two statements sharing a qualifiedName AND a start line (a vendor-split
databaseId pair on one line) collided on the node id, so INSERT OR REPLACE
silently dropped one. The id-hash now folds in the statement's byte offset;
the stored qualifiedName/startLine are unchanged so the Java<->XML bridge is
untouched.
Gaps 1 and 2 follow @ESPINS's fix-mybatis-quotes-comments branch. Tests add
extractor-level coverage for all four gaps plus a DB-level e2e that proves
iBatis statements land and both vendor-split nodes survive a real indexAll.
Co-authored-by: Jimin Lee <dlwlalsggg@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The shared daemon's inactivity backstop (#692) reaped the daemon after
maxIdleMs (default 30 min) of no inbound query bytes whenever a client was
still connected — without ever checking whether that client was actually
alive. lastActivityAt is fed only by inbound socket data and MCP has no
keepalive, so a genuinely-live session that just hadn't queried CodeGraph in
30 min tripped it. The daemon then exited, and the proxy's onDaemonLost
degrades that session (and every other session sharing the daemon) to an
in-process engine for the rest of its life. On one dev machine over a day the
backstop fired 20 times on live sessions (clients=1) and the liveness sweep
caught 0 real dead peers — net harm.
The backstop exists only to catch a phantom client (one counted but gone,
whose socket-close was never delivered). It now consults the peer pids the
daemon already tracks: after the inactivity window it sweeps provably-dead
peers, then reaps the daemon only if NO remaining client can be proven alive
(every one is an unknown-pid connection the sweep can't verify — the sole
phantom class it can't catch). One provably-alive client keeps the daemon up.
Extracted the decision into Daemon.backstopShouldExit(isAlive) so it's unit-
testable with an injected liveness probe, mirroring reapDeadClients. All #692
guarantees preserved; the only behavior change is that a provably-alive quiet
session is no longer reaped.
- daemon-client-liveness.test.ts: 7 new deterministic cases for
backstopShouldExit (live kept, phantom reaped, mixed protects the live one,
dead-peer swept-then-held, within-window, zero-client).
- mcp-daemon.test.ts: the integration test that asserted the backstop reaps a
live connected client (it encoded the bug) now asserts the opposite — a
live-but-quiet session survives several backstop windows with its lockfile
intact and no backstop shutdown logged.
Validated end-to-end on the built bundle: a quiet session's daemon stayed up
across 4 backstop windows (maxIdle=3s), same pid throughout, zero backstop
fires. Found while fixing #1185.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
An MCP host that kills the launcher chain within the server's first ~100ms
while keeping the stdio pipes open (config probe, cancelled request, startup
timeout; Rust hosts that kill a child without dropping its stdio handles) left
the server orphaned: it booted already reparented to init, so the PPID
watchdog's "ppid changed" baseline was captured as 1 and could never fire, and
stdin never EOF'd. The process lingered — idle, ~30MB — until the host itself
exited, accumulating one per abandoned launch (the pile-up reported in #1185).
Reproduced on released 1.2.0/macOS: SIGKILL the launcher at +50ms → permanent
orphan; at +150ms the old late baseline had already run and reaped it.
Three-part fix:
- Capture process.ppid at the earliest line of the CLI entry (early-ppid.ts)
and use it as every watchdog baseline, shrinking the blind window to the few
ms before our first JS runs.
- Thread the real host pid down the bundled path: the npm shim and the
standalone sh launcher set CODEGRAPH_HOST_PPID (an outer launcher's value
wins), so the watchdog polls the host directly. Previously only the
--liftoff-only relaunch set it, leaving the entire npm/standalone install
base with hostPpid=null.
- Never-initialized backstop (startup-handshake.ts): a serve --mcp that
receives no MCP traffic for CODEGRAPH_STARTUP_HANDSHAKE_TIMEOUT_MS (default
15min, 0 disables) shuts down — the catch-all for a kill landing in the
residual pre-JS window. Disarmed on the first byte, so a quiet-but-live
session is never touched.
Also scrub CODEGRAPH_HOST_PPID from the detached daemon's env — it has no host,
and a stale pid must not leak into anything it spawns.
Validated end-to-end on the built bundle: the +50ms early-kill orphan is now
reaped while the host still holds the pipes open, and all six normal
lifecycle paths (clean close, SIGTERM/SIGKILL child, host exit/SIGKILL,
fd-holding adversarial host) stay clean. New coverage in
startup-handshake.test.ts, mcp-startup-orphan.test.ts, and npm-shim.test.ts.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
An indexing run killed mid-"Resolving refs" (crash, Ctrl-C, the #1122
watchdog kill) left the refs it never reached parked in unresolved_refs.
The git-scoped sync fast path only re-resolves changed files' refs, so
those files' call edges were missing permanently — a too-small blast
radius clustering by package/module (the #1187 field report: 3 of 10
caller files for a Spring @Resource-injected method) — until a full
re-index.
- sync() now sweeps leftover unresolved refs with the batched resolver
after its scoped pass, including on no-change syncs, so a bare
`codegraph sync` recovers a wedged index (and heals pre-fix indexes
on the first post-upgrade sync)
- the scoped pass deletes unresolvable rows too (parity with the
batched path), making "rows at rest" a sound orphan signal
- drop the batched loop's early break that abandoned all later batches
when one batch was all-unresolvable (its rows WERE consumed — that
early stop could orphan the rest of the table at init)
- surface the state: `codegraph status` warns, `status --json` gains
index.pendingRefs, and MCP codegraph_status tells agents the blast
radius is incomplete until the next sync
Verified end-to-end on a 2,414-file synthetic Spring repo: SIGKILL
mid-resolution reproduces the reporter's exact 3-of-10-callers state;
a bare sync now heals it to 10/10 with the edge count converging to
the clean-init total; a healthy-index sync stays a no-op.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Adds ArkTS (.ets, HarmonyOS/OpenHarmony) as a first-class language:
full TypeScript-grade extraction via the harmony-contrib tree-sitter
grammar (MIT, vendored byte-identical from the tree-sitter-arkts 0.2.0
npm tarball), plus the ArkUI constructs that make HarmonyOS apps
traceable:
- @Component/@ComponentV2 structs with decorators from both grammar
positions; members extract as class members with qualified names.
- build() component trees: child instantiation edges via
arkui_component_expression, no synthesizer needed.
- Attribute chains emitted dot-prefixed and resolved ONLY against
@Extend/@Styles/@AnimatableExtend/@Builder helpers (unique-or-drop) —
bare-name fallthrough produced 36,840 wrong edges (17% of calls) on
the OpenHarmony samples monorepo. All four grammar chain shapes
handled, including the detached-chain forms.
- .onClick(this.handler) method-reference bindings.
- ohpm workspace modules: bare imports follow oh-package.json5 file:
deps (ambiguous names dropped), honoring each module's main entry —
which also lets .ts consumers resolve .ets modules.
- ArkUI dynamic-dispatch bridges, all provenance:'heuristic' with
wiring-site metadata: assignment-gated state->build() re-render
(V1 @State family + V2 @Local/@Provider/@Consumer),
@ohos.events.emitter emit->subscriber pairing on static event keys
(numeric ids same-file, named constants same-module, fan-out capped),
and router.pushUrl literal urls -> the target page's @Entry struct.
- $r/$rawfile resource intrinsics treated as built-ins; arkts joins the
web language family, value-reference edges, re-export chase, and the
other TS-applicable gates.
Also ships a language-agnostic index-completeness guard: indexAll
stamps index_state (indexing -> complete/partial/failed), reconciles
discovered vs accounted files (a loaded run silently dropped 37 files),
and codegraph status surfaces truncated/partial indexes in human and
--json output.
Validated on HarmoneyOpenEye (82 files), CoolMallArkTS (528, modular
ohpm + ArkUI V2), and openharmony/applications_app_samples (11,693
files, 202,890 nodes stable across re-index, attribute false-positive
audit 36,840 -> 588 residual all-plausible). Supersedes PRs #656 and
#988 with credit — both informed this implementation.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Follow-ups noted in #1173:
- cloudposse/atmos remote-state: module.M.outputs.X emits a scoped
module.M:remote-output.X candidate; the resolver bridges it to the
target COMPONENT's own output when every gate holds — the module
source is the stack-config remote-state module, the component name is
static (a literal, or component = var.X whose variable declares a
literal default in the same directory), and exactly one directory in
the repo matches the component name and declares that output. Dynamic
(each.value) or ambiguous wiring stays unlinked. On
cloudposse/terraform-aws-components: 254 remote-state bridge edges,
every one re-derived from a matching source declaration (789/789
cross-directory output edges explained: 528 local-module + 254
remote-state + 7 checker-artifact false alarms under deprecated/);
coverage 66.4% -> 69.1%.
- provider aliases: provider "aws" { alias = "east" } is addressed as
provider.aws.east so aliased and default configurations stop
colliding; provider = aws.east on a resource/data block (and the
values of a module's providers map) reference the selected
configuration, resolved same-directory first then up the module tree
— the one construct Terraform genuinely inherits from parents. The
selection is no longer misread as a resource reference (aws.east).
- moved/import/removed blocks reference the resource addresses they
name (anchored to the file node — no phantom symbols), so a
refactor's paper trail joins the graph; check-assert conditions
contribute their references while check-scoped data blocks keep
indexing as before. Scoped module candidates are suppressed there:
module.a.aws_x.b names a resource inside a module instance, not an
output. +91 edges on cloud-foundation-fabric's moved-heavy stages.
Also fixes a latent test bug from #1173: cg.getNodeById is not public
API (cg.getNode is) — it only passed because the asserted edge list was
empty.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* feat(extraction): add Terraform and OpenTofu language support
Index .tf, .tfvars, and .tofu files via the tree-sitter-terraform dialect
of HCL (vendored from @tree-sitter-grammars/tree-sitter-hcl, Apache-2.0).
Symbols extracted:
- resource / data → class (qualified "type.name" / "data.type.name")
- module → module (qualified "module.name")
- variable → variable (qualified "var.name")
- output → variable (qualified "output.name")
- provider → namespace
- locals → constant per attribute (qualified "local.key")
References resolved cross-file:
- var.X, local.X, module.M[.out], data.T.N[.attr], <type>.<name>[.attr]
- built-ins skipped: each.*, count.*, self.*, path.*, terraform.workspace
The Terraform framework resolver disambiguates same-named candidates
across modules by preferring the one in the same directory as the
reference site, then by closest common-ancestor path, falling back to
the generic name matcher only when neither applies.
Validated on two Terraform monorepos (277 and 470 .tf files): indexing
runs in 1.3s and 2.4s respectively, query latency stays under 200ms,
and cross-module references resolve to the correct module 100% of the
time on inspected samples.
18 new extraction tests; full suite 1146/1148 green (2 pre-existing
flaky skips, 0 regressions).
* feat(terraform): bridge the module boundary and enforce directory scoping
Builds on #706. The module declaration was a dead end: module.M.out
resolved to the declaration and stopped, module inputs never reached the
child module's variables, and impact could not cross the boundary — on
real multi-module repos that breaks the core blast-radius question
("what breaks upstream if I change this module's variable/output").
- module blocks now wire across the boundary through :-scoped refs only
the Terraform resolver understands: module.M:var.<input> → the child's
variable node, module.M:output.<o> → the child's output node (emitted
alongside the module.M declaration ref), and module.M:file → the local
source directory's entry file (imports). Registry/git sources emit no
file ref and resolve nothing — an out-of-repo module stays a visible
boundary instead of a guess.
- .tfvars top-level assignments reference the variable they set, walking
up to the nearest ancestor directory (envs/prod.tfvars → root vars).
- Resolution now enforces Terraform's real scoping: same-directory only
(no cross-module fallback by common path prefix, no single-candidate
anywhere-in-tree binding), and terraform refs never fall through to
the generic name matcher — var.X can never legally bind outside its
module directory, so the fallback could only add wrong edges.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs(terraform): README language table + changelog entry + agent-eval corpus
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Javier Rodríguez Fernández <jfernandez@freepik.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
CUDA rides the C++ grammar via the Metal (#1121) dialect pattern:
blankCudaConstructs (offset-preserving) blanks execution-space specifiers
(__global__ family), __launch_bounds__(...), and <<<grid, block>>> launch
configs — which otherwise lex as shift operators and destroy the
host→kernel call edge entirely. Gated by .cu/.cuh extension OR by content
(looksLikeCudaSource), because much real CUDA lives in .h/.hpp headers:
cutlass launches most kernels from headers and flash-attention's launch
templates are .h. Safe by construction — no CUDA marker is valid C++
anywhere, and the launch blank is bounded + brace-balance-checked so a
stray <<< (committed merge-conflict markers) can never blank real code.
All real-world launch styles connect: plain, templated
(k<T, 256><<<...>>>), function-pointer (auto kernel = &fn<...>; with
branch reassignments each linked), dim3{...} brace-init configs, and
kernels defined through name-in-first-argument macros
(DEFINE_FLASH_FORWARD_KERNEL style — gtest TEST_F / PYBIND11_MODULE
shapes deliberately excluded by the two-lone-identifiers rule).
Two general C++ resolution wins the flow validation forced out:
- namespace blocks now prefix contained symbols' qualifiedNames
(prefix-only — no namespace nodes, avoiding #1093-style crowd-out), so
ns::fn(...) calls resolve; previously every namespace-qualified C++
call was a permanently dead edge. cutlass: +30,864 edges (~10%), node
count byte-identical.
- templated callees (fn<T, 256>(args)) strip template args at extraction
(mirroring #1043 for base classes), so they match their definitions.
Validated on llm.c (165 host→kernel launch edges, was 0),
flash-attention (run_flash_fwd → flash_fwd_kernel → compute_attn traces
in one codegraph_explore call), and NVIDIA CUTLASS; fmt as the plain-C++
control (unchanged). A/B n=2/arm: Read/Grep displacement decisive on all
three repos (flash-attention Reads 29,13 → 5,2).
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
escripts (.escript) index like any module — the ELP grammar has a
first-class shebang node, so no source transform is needed; main/1 and its
helpers get full function/call extraction.
OTP application resource files (<app>.app.src and compiled <app>.app) join
the graph as Erlang terms the grammar parses natively. They route by full
suffix (their last-dot extension, .src, is far too generic for the
extension map). The application tuple yields structure: {mod, {Mod, _}}
links the app to its callback module — the app's entry point — and
{applications, [...]} / {included_applications, [...]} connect umbrella
sibling apps, resolving through the OTP app-name == module-name convention;
kernel/stdlib and other out-of-repo apps stay unresolved.
App-file refs resolve only ever to MODULES: validation on emqx caught the
ssl OTP-app dependency resolving to a test helper FUNCTION named ssl (the
same defect class as the earlier -behaviour gate), so the matchReference
module-only gate now covers every ref an .app/.app.src file emits.
Validated on emqx: 2 app.src + 6 escripts indexed, entry-module and
umbrella-dependency edges all namespace-targeted post-gate, escript
functions extracted; a stray legacy/module.src stays unknown.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Calls hidden inside -define bodies were invisible: the extractor consumed
pp_define without walking the replacement, and macro use sites produced no
edges, so a call path routed through a macro (ejabberd's SQL upsert macros,
logging wrappers) was completely dark.
The macro's constant node now participates in the graph. The -define body's
calls are attributed to the MACRO — true exactly once, instead of a per-use
duplicate that would explode on logging macros — and each use site links
in: ?MACRO(...) with arguments emits a `calls` ref (inlined code joins the
call chain), a bare ?CONSTANT read emits `references` (answering "where is
this macro used" without polluting call paths). Compiler-predefined macros
(?MODULE, ?LINE, ?FUNCTION_NAME, ...) are excluded, macro-use arguments
keep walking so a call nested in ?assertEqual(ok, do_thing()) still
attributes to the enclosing function, and macro-to-macro chains connect.
Validated: node counts unchanged on cowboy/ejabberd/emqx; edges +26/+7.3K/
+42K with honest hub shapes (?T i18n, ?SLOG logging, ?QOS_1 protocol
constants); 40/40 sampled edges precise; +1.3s index cost on emqx's 2,273
files. The payoff chain on ejabberd: set_password_scram_t → ?SQL_UPSERT_T →
ejabberd_sql:sql_query_t — database writes through SQL macros now trace
end-to-end.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
gen_server:call/cast/send_request now connects to the TARGET module's
handle_call/handle_cast for every statically-named target, not just self:
a bare atom reaches the module of that name (OTP's {local, ?MODULE}
convention names a server after its module), and a macro defined as a bare
atom (-define(STORE, kv_store)) resolves the same way, alongside the
existing ?MODULE / -define(SERVER, ?MODULE) self paths. A registered name
that matches no module emits a qualified ref that never resolves — silent,
never guessed. Pid, var, and tuple targets ({global, Name}, {Name, Node})
stay unlinked.
Validated on emqx: 53 new edges, 53/53 precise (each source line is a real
registered-name gen_server request; each target module self-registers under
that name, macro-indirected registrations included). Nearly all are
test-suite → handler links — production code goes through API wrappers the
self path already covers — which is exactly the tests-exercising-this-
handler linkage blast-radius and test-gap reporting consume. ejabberd
yields zero (it always wraps): no false positives invented.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Bridges the OTP callback boundary: a framework call through a variable
module — cowboy's Handler:init / Middleware:execute folds, a plugin
manager's Mod:callback(...) — now links to the repo's implementers of the
behaviour declaring that callback, so codegraph_explore connects flows
end-to-end across behaviour dispatch instead of stopping at it.
Precision gates: the callback arity must match the site, exactly one
in-repo behaviour may declare that (name, arity) — a collision bails
(cowboy's init/2 is declared by five handler-flavored behaviours and
correctly stays silent) — the implementer must export the callback, and
above the fan-out cap the site is skipped entirely (ejabberd's gen_mod
with ~230 implementers stays a visibly dynamic boundary). Behaviour
discovery scans -callback declarations in every module so implementer-less
behaviours still gate ambiguity. Edges carry provenance:'heuristic' with
synthesizedBy:'erlang-behaviour' and the wiring site, rendered as dynamic
dispatch in explore.
Validated per the dispatch-family playbook: cowboy 38 edges (middleware
chain, stream-handler folds, sub-protocol upgrade), ejabberd 598, emqx 843;
36/36 sampled edges precise (target declares the via-behaviour and exports
the callback); node counts unchanged; ~1.4s added on emqx's 2,273 files;
zero-control clean. The cowboy request flow connects in one explore call.
Includes an Erlang comment stripper (%-comments, string/atom/$-char aware)
for the dispatch-site scans.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Vendored WhatsApp/tree-sitter-erlang 0.19 (the ELP grammar, ABI 14) with an
Erlang-shaped extractor: multi-clause/multi-arity functions merged into one
symbol, -spec signatures, records with fields, -type/-opaque aliases, -define
macros, -include/-include_lib file edges, and -export-driven visibility.
Modules wrap in a namespace so remote mod:fn(...) calls resolve through the
existing qualified-name matcher as mod::fn with zero resolver changes.
-behaviour declarations link to the behaviour module — gated to namespace
targets only (bare-name fallthrough linked -behaviour(supervisor) to an
unrelated macro constant on emqx). OTP indirection with static targets is
followed: spawn/apply/proc_lib/timer/rpc MFA-argument callees, and
gen_server:call/cast(?MODULE | ?SERVER) to the module's own
handle_call/handle_cast. Var-module dispatch and message sends stay
deliberately unlinked. codegraph_explore also normalizes Erlang-native query
spelling (mod:fn/3, init/2) so named symbols resolve as typed.
Benchmarked on cowboy (189 files), ejabberd (414), emqx (2,447): extraction
PASS on all three; with-codegraph arms reached 2/2/0 file Reads vs 10/5+/19
without, fastest on the largest repo.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Vendored patched govindbanura/tree-sitter-vbnet grammar (MIT, ~20-fix patch
+ new external scanner for XML literals and multi-line LINQ continuation;
provenance + rebuild instructions in docs/grammars/tree-sitter-vbnet.md),
vbnet extractor with VB-specific call/index disambiguation, Inherits/
Implements heritage, As New instantiation, events, Declare P/Invoke, and
MustOverride abstract members.
Parse health on five real repos: PolicyPlus 100%, CompactGUI 100%,
staxrip 95.2%, SCrawler 87.2%, PCL 87.5% (upstream grammar: 3-18%).
Retrieval A/B (sonnet): 26-43% faster with 0-5 file reads vs 7-20 without.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Programs, sections/paragraphs (reconstructed extents over the grammar's
flat header stream), PERFORM/THRU/GO TO/CALL call edges, COPY copybook
imports incl. standalone .cpy fragments, DATA DIVISION records/fields/
88-levels with write-site impact references, and CICS flows: EXEC
LINK/XCTL program targets (literal + same-file VALUE deref), EXEC SQL
INCLUDE, and pseudo-conversational RETURN/START TRANSID hops resolved
to the owning program via a CICS framework resolver. Fixed and free
source format (free format via a scanner wide-mode sentinel).
Grammar: vendored wasm built from a patched yutaro-sakamoto/
tree-sitter-cobol (EXEC blocks as an external-scanner token, copybook
fragment entry point, single-quote continuation, COPY REPLACING
pseudo-text, NOT=, CALL GIVING, ENTRY, FREE, bitwise ops, abbreviated
relations, COBOL-2002 usages, and more). Patch + provenance + upstream
PR draft in docs/grammars/. Parse health: AWS CardDemo 43/44 native
(upstream: 9/31), 44/44 through preParse; copybooks 28/29; CobolCraft
free-format 17/17 (upstream: 0); NIST COBOL85 unchanged at 373/382.
Copybook members resolve to files like C includes (basename index,
name-matcher short-circuit so compiler-supplied members stay honestly
unresolved): CardDemo imports 5 -> 285. Impact proof: ACCT-CURR-BAL
(CVACT01Y copybook) surfaces its 4 writer programs cross-file.
Also: run-all.sh now neutralizes the ambient prompt-hook in both A/B
arms (CODEGRAPH_NO_PROMPT_HOOK=1); COBOL corpus entries for agent-eval.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
CFML joins the #1108 receiver-inference family: new/createObject/typed-arg/property(inject) declarations type the receiver, variables./this. fields scan whole-file, method QNs re-scoped to Class::member in all three extraction paths. 1,649 typed edges on fw1/ColdBox/CFWheels, 1,649/1,649 audit-consistent, inherited methods resolve via #1152 extends edges.
Co-authored-by: ghedwards <125586+ghedwards@users.noreply.github.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
extends="coldbox.system.web.Controller" (dotted) and extends="../base" (relative) now resolve to the right component via directory-corroborated matching; >=1 corroborating segment required, ties yield no edge. fw1 14->47, ColdBox 21->242, CFWheels 60->201 inheritance edges; 394/394 audited path-consistent.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
.metal was absent from EXTENSION_MAP, so Metal Shading Language files were
silently skipped. MSL ≈ C++14, and the C++ grammar extracts its functions,
structs, type aliases, and call edges at parity with plain C++ — except MSL's
post-declarator [[attribute]] annotations, which misparse struct fields into
spurious extends refs from the struct to the field's own type (a wrong
inheritance edge whenever the repo typedefs float3/float4x4 itself, common in
shared ShaderTypes.h). blankMetalAttributes blanks them pre-parse,
offset-preserving, following the blankCppExportMacros pattern (#1061), gated
to .metal files only — in regular C++ the attribute position is legal syntax
the grammar parses natively. The preParse hook gains an optional filePath
param to support the gate.
Validated on llama.cpp's ggml-metal.metal (10.7k lines: 130 kernels vs 113
`kernel void` ground-truth lines, rope_yarn resolves its 4 kernel callers)
and SDL's shaders (PQtoLinear ← GetOutputColor), 0 bogus extends edges.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Five hardening fixes to the #1136 MEDIUM (graph-derived) tier:
- #1141: updateNode() now writes the segment vocabulary like insertNode()
does — framework post-extract renames (NestJS route prefixing) left the
new name permanently unsearchable (the old rows orphaned, the backfill
gated on an EMPTY vocab, so even a full re-index re-created the drift).
- #1142: new CodeGraph.healSegmentVocabIfEmpty() — the hook opens the
graph without sync, so a database migrated from pre-vocab schema kept
the MEDIUM tier dormant until some unrelated sync ran. The hook heals
on first use (one SELECT when populated; lock-aware, defers to a
running sync) and records noop-vocab-empty when it can't.
- #1144: a name whose only nodes are file/import kind is skipped instead
of falling back to surfacing an import statement as a matched symbol;
import specifiers no longer enter the vocab at all (shared
isSegmentableKind gate across insertNode/updateNode/rebuild page query)
since they can never be surfaced and only inflate rarity statistics.
- #1145: plural variant folding is keyed on English plural spelling —
bare-s plurals no longer mint a bogus -es sibling (services→servic),
unambiguous sibilant-es plurals no longer mint a bogus -s sibling
(classes→classe), trailing -ss singulars no longer strip (class→clas);
genuinely ambiguous endings (caches/databases) still emit both keys.
- #1146: getSegmentCoOccurrence folds variants to their original word
inside the SQL (CASE mapping + COUNT(DISTINCT word)) so a plural pair
of ONE word can't tie with a genuine two-word match and crowd it past
the pre-fold ORDER BY/LIMIT; the JS re-check stays as the honesty layer.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
gate('high-keyword'/'high-token') sat outside the injection guard, so an
errored or empty codegraph_explore still counted as a HIGH-tier success.
The gate telemetry is the measured recall/precision funnel that decides
whether the tiered gate design survives — a delivery failure must degrade
it toward noop-*, not inflate the high tiers. Failures now record
noop-explore-keyword / noop-explore-token. Doc enum updated (including
the noop-vocab-empty outcome the #1142 fix adds next).
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
extraction/index.ts bounds every git call it makes; worktree.ts,
git-hooks.ts, and the installer's npm install -g did not, so a stuck
subprocess blocked the caller indefinitely. Worst case was the daemon:
gitWorktreeRoot/gitCommonDir run (memoized) on the main event loop while
serving MCP clients, where an unbounded git hang would trip the 60s
liveness watchdog and SIGKILL a healthy daemon. git calls get 5s, the
interactive npm install 120s. Regression tests assert the option through
a mocked child_process plus a per-file call-site sweep.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The multilingual structural-question gate (#1134) matches stems as open
prefixes (left boundary only) so derived forms fire without enumeration.
Four English stems have common non-structural completions — callus,
calligraphy, Connecticut, connective, affectionate, Tracey — that
false-fired the HIGH (full-explore) tier. Those four now enumerate their
structural suffixes and re-assert the right boundary; callbacks/callable/
call sites are included so no structural form regresses. Also documents
the verified-unfixable Korean homograph class on the unsegmented table
(#1140): segmentation can't split 구조대 from 구조가, and a denylist would
break 구조대로.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
xyyxrandGitHub
