The explore envelope used to follow FILE SIZE, not relevance. Every admitted
file was capped at the same flat `maxCharsPerFile`, while the whole-file rule
handed anything under `maxCharsPerFile * 3` its entire contents — a 3x swing
decided by how big a file happened to be:
- self-query: `memory-budget.ts` (score 18) shipped whole and took 51.2% of
the response; `src/mcp/tools.ts` (score 41, 4x the graph mass, 3x the term
hits — it holds the allocator itself) was clipped at 3,800 and got 32.9%.
- #1500 Go fixture: two generated CRUD files shipped whole at ~4.5K each AND
consumed two of the tier's four file slots, so `BuildPayslip` — the
hand-written "calculate" half of the question — ranked #6 and never
rendered at all.
`allocateExploreBudget` now reserves each ranked file a share of the envelope
before anything renders, so the render loop spends a reservation instead of
racing for whatever the files above it left:
- weight = score x worth x (spine ? 2 : 1), where `worth` is `rankPenalty`
applied a SECOND time — ranking answers "is this file about the query",
allocation answers "will these bytes teach the agent anything", and
generated CRUD can legitimately rank while its bytes stay boilerplate;
- a relative cliff at 15% of the top weight (capped at SCORE_FLOOR_MAX, so a
god-file can't silence peers the score floor just admitted) gives a file
ZERO source — path, symbols and line numbers only — and crucially frees its
`maxFiles` slot for a file that earns its bytes;
- every admitted file gets MIN_CHARS, then the remainder splits by weight:
the floor keeps a diffuse survey question returning a spread, the remainder
concentrates a precise one;
- the flat per-file cap is retired as the primary guard, leaving a 70%-of-
envelope safety valve.
Two changes were needed to make the reservation bite: an oversize cluster now
shrinks by whole MEMBER symbol ranges (a single-cluster god-file previously
took ~40% more than allotted, and the file below it was dropped for lack of
room), and the arrival-order budget stops are gone — they cut files by the
order they were reached rather than by merit.
Measured: payroll-go answer group 25.6% -> 78.7%, generated 57.4% -> 0%, and
`func (s *Service) BuildPayslip` now delivered; self-query `tools.ts` 18.5% ->
60.6%, past the epic's >50% bar. Controls hold: cobra/gin diffuse survey
queries keep their file spread (3->3, 3->4), express's middleware query is
byte-identical, and gin's flow query moves its top file from the thin `ginS`
singleton wrapper to `routergroup.go`.
One documented exception to "no previously-unclipped file becomes clipped":
`memory-budget.ts` was unclipped-whole at 5,672 and now clusters within its
3.1K reservation. That is the epic's own diagnosis of the bug — it scored 18
against 58 and was taking the larger slice purely for being small.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Explore's per-file relevance awarded +50/+10/+3/+1 by match class and admitted
anything scoring >= 3. Neither half held up: the tier said HOW a symbol reached
us, never whether the match was evidence, and an absolute floor admits noise on
any repo where the top file scores 50+. Three scripts/agent-eval/*.mjs harnesses
took 63% of this repo's own "how does explore allocate its output budget" answer
on nothing but an unused `const explore` and a `const BUDGET`.
Four levers:
- KIND WEIGHT (RELEVANCE_KIND_WEIGHT): callables and types 1.0, members ~0.5,
variable/constant/parameter 0.15-0.35. A weak-kind symbol with no usage edge
anywhere in the graph (`contains` excluded — nesting is not usage) drops to
0.08. Only weak kinds in the top two tiers pay for the DB probe; the subgraph's
own edges answer most cases free. No measurable latency change (210 vs 211
ms/call, n=12 interleaved).
- PERIPHERAL CAP: nodes >=2 hops from any match accumulate into a bucket capped
at 5. Uncapped they added a flat +1 each, so a file grew more relevant by being
bigger — parse-session.mjs reached 22 off one constant plus twelve unrelated
symbols.
- RANK PENALTY: generated files x0.3, low-value x0.5, applied to the score AND
the graph mass. Score alone would not have fixed#1500 — the generated CRUD
carries MORE graph mass than the hand-written use-case, and graph mass outranks
score in the comparator. Self-normalizing, never a hard exclusion.
- RELATIVE FLOOR: clamp(topScore * 0.2, 1, 10). Capped at one full-strength
direct match so concentration elsewhere can never exclude one (without it a
named-seed-heavy file pushed the floor to 21 and dropped a file the agent had
named by class name). Backfills to 3 candidates when it would leave fewer, and
drops the evidence requirement rather than return nothing at all.
excludeLowValueFiles was dead config — declared per tier, read nowhere; the
test/spec exclusion has been unconditional for a while. Removed. The real gap was
the detector: `isLowValue` anchored on a leading `/`, so a repo-ROOT `test/` dir
(express, cobra, most of npm and Go) never matched — express's routing question
spent 59% of its envelope on three test files. Anchored at `^` too, and the
filter now runs before the floor and judges "are there other candidates?" on the
whole gather.
Measured before/after on the same indexes (baseline bd86ad2):
- payroll-go fixture: generated 57.4% -> 23.5%; answer 25.6% -> 61.5%; cycle.go
delivered 0 -> 38.9%. Generated ranks #3/#4, was #1/#2.
- self-query fixture: eval scripts 72% -> 0%; tools.ts ranks #1.
- express "route a request": 59% to test/* -> lib/application.js + lib/response.js
- cobra x3, codegraph "indexing pipeline": byte-identical (control)
Diagnostic gains a per-file penalty multiplier and NodeKind mix, so "why did this
file score X" is legible. Selection stages reordered to match the pipeline.
CG-6's gates flip from it.fails to live regressions except the byte-split ones,
which stay open for CG-12 (allocation still follows file size within the ranked
set).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`isGeneratedFile` was path-only, but Go's own convention is a CONTENT
marker (`// Code generated by <tool>. DO NOT EDIT.`), not a filename one.
A Go monorepo with generated CRUD in ordinarily-named files sitting beside
hand-written use-cases was therefore invisible to every generated-file
down-rank in the codebase — that is #1500.
Measured on kubernetes/client-go (2,453 Go files): the canonical banner
appears in 2,001 of them, the path check flags 0, the new content check
flags exactly those 2,001 — no false positives, no misses.
Design: decide at INDEX time (content is already in memory for parsing),
persist on `files.generated`, read from the DB. Explore never reads file
headers per request.
- `hasGeneratedHeader(content)` recognizes the standard banners — Go's,
protoc's, `@generated`, `<auto-generated>`, Thrift, OpenAPI Generator,
FlatBuffers, bindgen, ANTLR. Precision-first and fenced three ways: an
8KB/60-line header window, a comment-line requirement (leader or open
block comment), and markers tight enough that prose can't trip them. A
generator's own source, holding the banner as a string constant in its
body, is not flagged; neither is this module itself (pinned by test).
- `isGeneratedFile(path)` is unchanged — cheap, sync, still the fallback.
- Schema v9 adds `files.generated` + a PARTIAL index. DDL only, no
backfill: the flag derives from content the migration cannot see, so
rows stay 0 until a re-index and every reader unions the flag with the
path check — an un-migrated index keeps pre-#1500 behavior rather than
regressing. Re-index required; noted in the CHANGELOG.
- `generatedPredicateFor(paths)` gives ranking a bounded probe + O(1)
lookups. Bounded, not cached: no invalidation, so a ranking call can
never serve a verdict the last sync already replaced. Wired into explore
ranking, findSymbolMatches, findAllSymbols, search (MCP + CLI), the
context formatter, and the dominant-file/route-file hygiene filters.
Cost (acceptance bar was no measurable index-time regression): a single
unanchored `/generat/i` test over the header rejects ~every hand-written
file before any line splitting. 4.6 µs/file on client-go (worst case —
82% generated). End-to-end `codegraph init` on client-go, n=3 alternating
arms: 5.73s median with detection vs 5.76s path-only baseline; the arms
cross over between runs, so the difference is inside run-to-run noise.
Scope note: generated status remains a stable TIEBREAK at equal score,
exactly where it was. Making it a strong negative signal is CG-10, which
this unblocks by making the signal correct and available.
Two pre-existing tests hard-coded schema version 8; both now track
CURRENT_SCHEMA_VERSION (or the migration table) so future migrations
don't require editing them.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
How codegraph_explore divides its byte envelope among files was
unobservable — you could read a response and guess, but not say "this
file took 16% and that one took 20%." Nothing else in the budget-
allocation epic is measurable without that.
CODEGRAPH_EXPLORE_DEBUG now emits one report per explore call (stderr
table, stderr JSON, or a JSONL sidecar path). Per file: relevance score,
graph mass, term hits, ranking flags, render mode, bytes allocated vs
delivered, both shares, and whether it was clipped — plus why a ranked
candidate never rendered. Totals cover envelope vs maxOutputChars vs the
hard ceiling, the source/meta split, the selection funnel, and the score
floor and relevance-gate thresholds applied.
Allocated and delivered are reported separately on purpose: they diverge
exactly when the 25K ceiling truncates, and conflating them is how a
dropped trailing file goes unnoticed.
Off by default and byte-identical when off — it ships in the product
binary, and a diagnostic that perturbs the response by one byte would
invalidate every A/B taken with it on. ExploreDiagnostics.start() returns
null unless the env var is set, so every call site is a `diag?.` no-op.
Baseline recorded in docs/design/explore-budget-allocation.md: on this
repo, src/mcp/tools.ts gets 15.8% of the envelope while three weakly-
relevant agent-eval scripts take 61% between them — despite tools.ts
carrying 5.4x the score and 2.6x the graph mass of any of them. Small
files ship whole; the large answer file is clipped at maxCharsPerFile.
Rank ordering is correct and buys nothing. The loop also allocated 23,193
chars against an 18,000 budget.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The "no covering tests found" flag only inspected a symbol's direct
callers, so helpers exercised transitively by tests (logDebug runs
1,471x under npm test) were reported untested — wrong for ~40% of
flagged symbols per the issue's measurement.
The check now BFSes up the caller graph (3 hops, 64-lookup budget per
entry) and reports indirect coverage as "tested via callers: <files>".
When nothing is found it claims only what was measured — "no tests
found within 3 caller hops", or the weaker "no test calls this
directly" if the budget ran out — and drops the warning glyph.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
codegraph_node / codegraph_explore read CURRENT bytes but slice them at
INDEXED line ranges; after an un-synced edit that slice can be a DIFFERENT
symbol's code served under the requested name — isError: false, introduced
by the 'verbatim … do not Read' guarantee. The watcher-based pending (#403)
and degraded (#876) banners cannot cover a project reached via projectPath:
cross-project instances have no watcher, by construction.
Freshness is now verified at the point of emission from data the index
already stores: one stat per rendered file (size + floored mtime, the sync
fast path's own test), sha256 content-hash compare only on stat mismatch
(so a touch/identical rewrite never false-positives), memoized briefly per
handler. On drift:
- codegraph_node: small files ship WHOLE and CURRENT (Read-parity, still
no Read needed); large ones omit the body with an explicit notice
steering to the tool's file-read mode or Read. Location/signature stay,
flagged as possibly shifted.
- codegraph_explore: the whole-file render (already correct by
construction) is kept and flagged; adaptive/skeleton/cluster slicing is
disabled for drifted files — a too-big drifted file is omitted with a
notice instead. The verbatim/do-not-Read header gains a per-file
exception, and a trailing note flags shifted line references (flow,
blast radius, symbol lists).
The guarantee itself is preserved: everything actually rendered is still
byte-accurate — drifted files ship whole or not at all, never as a
possibly-wrong slice. A re-sync of the target project restores normal
output (covered by test).
Adds __setLoadCodeGraphForTests (same seam pattern as __setFsWatchForTests)
so in-process tests can exercise a genuine cross-project open, which
vitest's transform cannot service through the lazy require.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
A SIGKILL'd process (the #850 liveness watchdog, OOM, a crash) leaves its WAL
on disk; the next session appends to the same file; and nothing ever truncated
it — PASSIVE checkpoints fold frames but keep the file at its high-water mark,
and the one shrinking path (a clean last-connection close) is exactly what a
killed-daemon world never takes. Observed at 25.6 GB on a 5.46 GB DB, growing
until the disk filled.
- journal_size_limit on every connection: resetting checkpoints now clip the
WAL back to the cap instead of leaving it at its high-water mark.
- healOversizedWal() fired from every DatabaseConnection.open: off-thread
PASSIVE fold + TRUNCATE when the leftover WAL exceeds the cap (64 MB,
CODEGRAPH_WAL_HEAL_MB to override). Single-flight per connection with
bounded retries — concurrent passes defeat each other (each checkpoint sees
the other as a busy reader).
- Daemon/direct MCP watchdogs now pass progressPaths (DB + WAL), extending the
#1231 slow-disk deferral to the long-lived server so a healthy daemon mid
slow statement isn't SIGKILL'd — fewer kills, fewer leaked WALs.
- codegraph status shows WAL size (human + JSON) and warns when it dwarfs the
DB; daemon.log lines and the watchdog kill notice now carry ISO timestamps
so kills can be placed in time.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Three compounding defects (#1196) made a query bag of object-literal
keys (`profileInfo isTrialEligible quotaInfo billingMethod`) return
unrelated results while the defining files never surfaced:
1. Step 5b title-cased interior humps (profileInfo -> Profileinfo) and
then compared case-SENSITIVELY, dropping every row SQLite's
case-insensitive LIKE had just recovered. The hump lookup is now
case-insensitive with an explicit uppercase-at-match requirement.
2. Step 5b/5c's kind whitelist held only type-like kinds — dead code on
method-centric codebases. Callable kinds (function/method/component)
are fetched as a SEPARATE LIKE batch so hot single-word terms can't
crowd classes out of the length-ordered 200-row batch.
3. explore's named-symbol seeding was exact-name only; a field token
seeded nothing. A camelCase token with ZERO exact defs now seeds its
camel-infix definers (callables, hump-boundary or prefix, shortest
first, capped at 3) — bare lowercase words keep the #1252 stopword
guard untouched.
The reporter's acceptance query is a pinned e2e test (definer files
present, exact-name seeding unaffected). excalidraw probe: the
canonical flow query (mutateElement renderStaticScene) is byte-
identical; NL queries shift toward more-central callables
(useUIAppState/getDefaultAppState over observer periphery).
Fixes#1196
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The recommended MCP config launches the local binary, so a server left
running drifts behind releases silently — users discover the version gap
only when something breaks. Per the reporter's preferred option 1, the
server now checks the latest GitHub release in the background on startup
(never blocking; reuses the upgrade command's release-redirect resolution
so the two can't drift) and surfaces a one-line notice on three surfaces:
one stderr line (the MCP host's server log), the initialize instructions
(with do-not-run-it-yourself guidance for the agent), and codegraph_status.
Discipline: results cache in ~/.codegraph/update-check.json shared across
every proxy/daemon on the machine — 24h TTL on success, 1h backoff after
failure, an outage never hides an already-known update, and a stale cache
re-kicks a background refresh so long-lived daemons keep noticing. The
initialize path is a memoized synchronous cache read (the respond-fast
contract holds), and both handshake answerers (session + proxy) share one
helper so they can't diverge. Never stdout.
Hardening: the latest tag arrives from a network redirect via an on-disk
cache and ends up inside agent-visible instructions, so only a canonical
vX.Y.Z rebuilt from PARSED semver fields is ever interpolated — a tag
carrying trailing text (parseSemver is not end-anchored) renders without
it, and a non-version tag renders nothing and counts as a failed attempt.
Off is off: CODEGRAPH_NO_UPDATE_CHECK=1 (dedicated) or DO_NOT_TRACK=1
(broad convention — already set by data-plane deployments) suppresses the
network call and the notice entirely. Documented in TELEMETRY.md.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
handleExplore's named-symbol seeding treats every identifier-shaped query
token as "a symbol the agent named" and grants its definition the
named-FIRST sort tier. Natural-language questions broke this assumption:
ordinary words exact-matched unrelated callables ("check" ->
WalCheckpointValve.check, "serve" -> query-worker serve, "initialize" ->
DatabaseConnection.initialize), and those collisions outranked — and within
the per-repo file budget fully displaced — the corroborated answer files,
forcing the agent back to Read/Grep. The >3-def single-pick fallback had
the same hole: on grpc, the #1064 flagship query "add a parameter to
NewClient" itself tiered balancerStateAggregator.add's file to slot #1.
Guard: a shape-precise token (camelCase, PascalCase, snake_case,
qualified) seeds unconditionally — it is an unambiguous symbol reference.
A bare lowercase word seeds only defs whose file another query token
co-names (that token is itself an exact symbol name defined in the same
file — the "check drain fire" sibling-bag shape), which an incidental
English-word collision never is. Applied by filtering cands ahead of both
branches so the overloaded-name fallback is covered too.
Validated per the retrieval playbook: deterministic probes on this repo
(collision queries fixed; sibling-bag and single-camelCase retained), and
baseline-vs-fixed probes on the #1064 repos — Alamofire and excalidraw
byte-identical, grpc improved (the add-collision file drops out and
clientconn.go + dialoptions.go lead). Full suite green.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The liveness watchdog judged the main thread by heartbeat silence alone,
which cannot distinguish a true wedge (the #850 infinite loop it exists to
kill) from one long synchronous SQLite statement on severely degraded
storage — so it SIGKILLed valid, in-progress indexes (observed on a
150-IOPS throttled rig, and latent on real HDDs at scale).
The CLI index/init paths now hand the watchdog the project's DB + WAL
paths. On a silent timeout the watchdog child stats them first: if they
advanced during the silence, the block is a slow store making forward
progress — defer and keep watching; if not, kill at the base timeout
exactly as before. Deferral is bounded by a hard cap (10× the timeout) of
continuous silence so a wedge coinciding with unrelated file activity, or
I/O hung beyond any legitimate statement, still dies. The daemon path is
unchanged (no progress paths — pure heartbeat).
Validated with real spawned processes (defer-on-progress, kill-on-static,
hard-cap kill) and on the throttled rig: a 150-IOPS index under a 10s
watchdog window — 6× tighter than production, with store stalls measured
at 10-20s — completes cleanly where the old watchdog killed it, while
true-wedge kill latency is unchanged.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Large-codebase indexing died at the end of "Resolving refs" two ways:
watchdog kills of healthy work (24k-file Java on Windows, #1212 — third
iteration of the #1091/#1122 class) and hard OOMs (Linux kernel scale,
where v1.3.0 could not complete at any watchdog setting). Root causes:
~31 of 37 dynamic-edge synthesis passes ran start-to-finish with no
yield points, several materialized whole-graph snapshots (kotlin
expect/actual opened with getAllNodes() — 2M nodes in one array; the
C fn-pointer pass retained every C file's contents twice plus every
function node), and the post-index WAL checkpoint ran minutes of
synchronous IO on the main thread, killing even a successful index at
the finish line.
The pipeline tail now follows the same discipline as the rest: never
hold O(graph) in the heap, yield everywhere.
- All synthesis passes stream node-kind scans (cursors, not arrays) and
yield on time-budgeted checkpoints; language gates skip passes whose
filters a project's file languages provably can't satisfy.
- kotlin expect/actual filters SQL-side; c-fnptr caches are LRU-bounded,
units stream one file at a time, and the all-functions array +
write-only id map are gone; spring reads each .java once, not twice.
- runMaintenance moved to a worker thread (own SQLite connection);
per-file store commits chunk with yields behind a serialized flush
chain (preserving #1015 file-order determinism); resolver warm-up
streams the DISTINCT name set; resolution batch-tail and merged-edge
inserts run in bounded sub-transactions.
- Daemon: fixed a socket-handoff race that could leave a fresh MCP
session permanently silent (client-hello tail unshifted into a
flowing stream with zero listeners — the long-standing #662 test
flake was this real bug); first tool call no longer queues behind
the query pool's cold start (pool.ready gate).
Validation: Linux kernel (70,129 files, 2.05M nodes, 6.4M edges) fully
indexes in 27m8s on a 2-core/6GB container at default heap + default
watchdog; llvm-project (180k files) completes under 1GB RSS including
kill-and-sync recovery; synthesized-edge and full-graph parity are
byte-identical vs baseline on elasticsearch/redis/vim; the ex-flaky
daemon test passed 25/25 under load. Env-gated diagnostics kept:
CODEGRAPH_SYNTH_TIMINGS pass/phase timings, CODEGRAPH_MCP_DEBUG hop
tracing. Design record: docs/design/main-thread-stall-followup.md.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The shared daemon's inactivity backstop (#692) reaped the daemon after
maxIdleMs (default 30 min) of no inbound query bytes whenever a client was
still connected — without ever checking whether that client was actually
alive. lastActivityAt is fed only by inbound socket data and MCP has no
keepalive, so a genuinely-live session that just hadn't queried CodeGraph in
30 min tripped it. The daemon then exited, and the proxy's onDaemonLost
degrades that session (and every other session sharing the daemon) to an
in-process engine for the rest of its life. On one dev machine over a day the
backstop fired 20 times on live sessions (clients=1) and the liveness sweep
caught 0 real dead peers — net harm.
The backstop exists only to catch a phantom client (one counted but gone,
whose socket-close was never delivered). It now consults the peer pids the
daemon already tracks: after the inactivity window it sweeps provably-dead
peers, then reaps the daemon only if NO remaining client can be proven alive
(every one is an unknown-pid connection the sweep can't verify — the sole
phantom class it can't catch). One provably-alive client keeps the daemon up.
Extracted the decision into Daemon.backstopShouldExit(isAlive) so it's unit-
testable with an injected liveness probe, mirroring reapDeadClients. All #692
guarantees preserved; the only behavior change is that a provably-alive quiet
session is no longer reaped.
- daemon-client-liveness.test.ts: 7 new deterministic cases for
backstopShouldExit (live kept, phantom reaped, mixed protects the live one,
dead-peer swept-then-held, within-window, zero-client).
- mcp-daemon.test.ts: the integration test that asserted the backstop reaps a
live connected client (it encoded the bug) now asserts the opposite — a
live-but-quiet session survives several backstop windows with its lockfile
intact and no backstop shutdown logged.
Validated end-to-end on the built bundle: a quiet session's daemon stayed up
across 4 backstop windows (maxIdle=3s), same pid throughout, zero backstop
fires. Found while fixing #1185.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
An MCP host that kills the launcher chain within the server's first ~100ms
while keeping the stdio pipes open (config probe, cancelled request, startup
timeout; Rust hosts that kill a child without dropping its stdio handles) left
the server orphaned: it booted already reparented to init, so the PPID
watchdog's "ppid changed" baseline was captured as 1 and could never fire, and
stdin never EOF'd. The process lingered — idle, ~30MB — until the host itself
exited, accumulating one per abandoned launch (the pile-up reported in #1185).
Reproduced on released 1.2.0/macOS: SIGKILL the launcher at +50ms → permanent
orphan; at +150ms the old late baseline had already run and reaped it.
Three-part fix:
- Capture process.ppid at the earliest line of the CLI entry (early-ppid.ts)
and use it as every watchdog baseline, shrinking the blind window to the few
ms before our first JS runs.
- Thread the real host pid down the bundled path: the npm shim and the
standalone sh launcher set CODEGRAPH_HOST_PPID (an outer launcher's value
wins), so the watchdog polls the host directly. Previously only the
--liftoff-only relaunch set it, leaving the entire npm/standalone install
base with hostPpid=null.
- Never-initialized backstop (startup-handshake.ts): a serve --mcp that
receives no MCP traffic for CODEGRAPH_STARTUP_HANDSHAKE_TIMEOUT_MS (default
15min, 0 disables) shuts down — the catch-all for a kill landing in the
residual pre-JS window. Disarmed on the first byte, so a quiet-but-live
session is never touched.
Also scrub CODEGRAPH_HOST_PPID from the detached daemon's env — it has no host,
and a stale pid must not leak into anything it spawns.
Validated end-to-end on the built bundle: the +50ms early-kill orphan is now
reaped while the host still holds the pipes open, and all six normal
lifecycle paths (clean close, SIGTERM/SIGKILL child, host exit/SIGKILL,
fd-holding adversarial host) stay clean. New coverage in
startup-handshake.test.ts, mcp-startup-orphan.test.ts, and npm-shim.test.ts.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
An indexing run killed mid-"Resolving refs" (crash, Ctrl-C, the #1122
watchdog kill) left the refs it never reached parked in unresolved_refs.
The git-scoped sync fast path only re-resolves changed files' refs, so
those files' call edges were missing permanently — a too-small blast
radius clustering by package/module (the #1187 field report: 3 of 10
caller files for a Spring @Resource-injected method) — until a full
re-index.
- sync() now sweeps leftover unresolved refs with the batched resolver
after its scoped pass, including on no-change syncs, so a bare
`codegraph sync` recovers a wedged index (and heals pre-fix indexes
on the first post-upgrade sync)
- the scoped pass deletes unresolvable rows too (parity with the
batched path), making "rows at rest" a sound orphan signal
- drop the batched loop's early break that abandoned all later batches
when one batch was all-unresolvable (its rows WERE consumed — that
early stop could orphan the rest of the table at init)
- surface the state: `codegraph status` warns, `status --json` gains
index.pendingRefs, and MCP codegraph_status tells agents the blast
radius is incomplete until the next sync
Verified end-to-end on a 2,414-file synthetic Spring repo: SIGKILL
mid-resolution reproduces the reporter's exact 3-of-10-callers state;
a bare sync now heals it to 10/10 with the edge count converging to
the clean-init total; a healthy-index sync stays a no-op.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Adds ArkTS (.ets, HarmonyOS/OpenHarmony) as a first-class language:
full TypeScript-grade extraction via the harmony-contrib tree-sitter
grammar (MIT, vendored byte-identical from the tree-sitter-arkts 0.2.0
npm tarball), plus the ArkUI constructs that make HarmonyOS apps
traceable:
- @Component/@ComponentV2 structs with decorators from both grammar
positions; members extract as class members with qualified names.
- build() component trees: child instantiation edges via
arkui_component_expression, no synthesizer needed.
- Attribute chains emitted dot-prefixed and resolved ONLY against
@Extend/@Styles/@AnimatableExtend/@Builder helpers (unique-or-drop) —
bare-name fallthrough produced 36,840 wrong edges (17% of calls) on
the OpenHarmony samples monorepo. All four grammar chain shapes
handled, including the detached-chain forms.
- .onClick(this.handler) method-reference bindings.
- ohpm workspace modules: bare imports follow oh-package.json5 file:
deps (ambiguous names dropped), honoring each module's main entry —
which also lets .ts consumers resolve .ets modules.
- ArkUI dynamic-dispatch bridges, all provenance:'heuristic' with
wiring-site metadata: assignment-gated state->build() re-render
(V1 @State family + V2 @Local/@Provider/@Consumer),
@ohos.events.emitter emit->subscriber pairing on static event keys
(numeric ids same-file, named constants same-module, fan-out capped),
and router.pushUrl literal urls -> the target page's @Entry struct.
- $r/$rawfile resource intrinsics treated as built-ins; arkts joins the
web language family, value-reference edges, re-export chase, and the
other TS-applicable gates.
Also ships a language-agnostic index-completeness guard: indexAll
stamps index_state (indexing -> complete/partial/failed), reconciles
discovered vs accounted files (a loaded run silently dropped 37 files),
and codegraph status surfaces truncated/partial indexes in human and
--json output.
Validated on HarmoneyOpenEye (82 files), CoolMallArkTS (528, modular
ohpm + ArkUI V2), and openharmony/applications_app_samples (11,693
files, 202,890 nodes stable across re-index, attribute false-positive
audit 36,840 -> 588 residual all-plausible). Supersedes PRs #656 and
#988 with credit — both informed this implementation.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Vendored WhatsApp/tree-sitter-erlang 0.19 (the ELP grammar, ABI 14) with an
Erlang-shaped extractor: multi-clause/multi-arity functions merged into one
symbol, -spec signatures, records with fields, -type/-opaque aliases, -define
macros, -include/-include_lib file edges, and -export-driven visibility.
Modules wrap in a namespace so remote mod:fn(...) calls resolve through the
existing qualified-name matcher as mod::fn with zero resolver changes.
-behaviour declarations link to the behaviour module — gated to namespace
targets only (bare-name fallthrough linked -behaviour(supervisor) to an
unrelated macro constant on emqx). OTP indirection with static targets is
followed: spawn/apply/proc_lib/timer/rpc MFA-argument callees, and
gen_server:call/cast(?MODULE | ?SERVER) to the module's own
handle_call/handle_cast. Var-module dispatch and message sends stay
deliberately unlinked. codegraph_explore also normalizes Erlang-native query
spelling (mod:fn/3, init/2) so named symbols resolve as typed.
Benchmarked on cowboy (189 files), ejabberd (414), emqx (2,447): extraction
PASS on all three; with-codegraph arms reached 2/2/0 file Reads vs 10/5+/19
without, fastest on the largest repo.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The managed-reasoning removal (e5897d03) stripped the CLI/MCP wiring but,
despite its stated intent, left the offload modules and their test suite
behind. The dead code still shipped compiled inside the platform bundles,
and its Windows browser-opener was flagged by a security report (#1114)
for routing the login URL through `cmd /c start`, where cmd re-parses
shell metacharacters. Unreachable since 2026-06-20 and never wired in any
tagged release — but delete it for real: src/reasoning/ (config,
credentials, login, reasoner), __tests__/offload.test.ts, the now-inert
CODEGRAPH_OFFLOAD_DISABLE guard in dynamic-boundaries.test.ts, and the
stale reasoner reference in the FILE_SECTION_PREFIX comment.
Closes#1114
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(explore): surface a named method's buried signature type (change surface) (#1064)
#1064: a natural-language query like "what do I need to change if I add a new
parameter to NewClient" dropped the file the answer lives in (grpc-go's
dialoptions.go, which defines NewClient's DialOption) and surfaced lexical
namesakes instead, so the agent fell back to grep.
Root cause (instrumented on grpc-go): the answer file is lexically dissimilar
to the query and reachable only structurally, so it scores ~0 on every text
and centrality signal and never renders.
Two-part fix, both bounded and validated to not perturb flow queries:
1. Change surface — read each named method's signature-type edges from the full
graph and, ONLY when the type's file is genuinely BURIED (≈0 graph mass AND
no term hits), inject + rank + gate-keep + tier it. A well-connected type
file is left to rank on its own merit, so this never displaces a flow file.
2. Tier de-noise by centrality — still seed every <=3-def name (RWR/flow ranking
unchanged), but the named-first tier admits only the most-substantive def
plus co-named defs of comparable centrality (>=25% of the top def's caller
count). This keeps real overloads/wrappers (excalidraw's `mutateElement` in
three files, callers 74/58/40) while dropping vastly-less-central namesakes
(Go's `NewClient`: real 492 callers vs xds-pool 11, test-fake 3) that would
otherwise crowd the answer file out of the tier.
Validation:
- Deterministic probe: dialoptions.go goes from dropped to surfaced (with the
full option field set via defaultDialOptions).
- Broader-repo regression check (Alamofire, excalidraw, axios): 5/6 control
flow queries byte-identical to baseline; the 1 shift (Alamofire "how a request
gets validated": Validation.swift -> DataRequest.swift) is lateral —
DataRequest defines validate() and is a directly-relevant answer.
- grpc-go agent A/B (n=2, sonnet): grep fallback eliminated (0 vs baseline 4,2).
- 1845 unit tests pass.
* docs(changelog): note explore signature-type surfacing (#1064)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
codegraph_explore's "Found N symbols across M files." header reported
`subgraph.nodes.size` / `fileGroups.size` — the raw FTS gather. A broad
natural-language query ("publish status to the API") matches a huge pool
(260 symbols / 124 files on a 636-file repo) while only a handful clear
the relevance gate + budget and render, so the header read as "260
results to wade through" even though the correctly-ranked answer was the
few files shown.
Report instead the files whose source actually SURVIVES in the final
output (after the hard-ceiling truncation that can drop trailing
sections), summing their relevant symbols. Gather, ranking, gate, budget,
and rendering are untouched — only the header string changes. Overflow
relevant files are still named under "Not shown above", so nothing is
hidden. Adds a regression test locking header-count == rendered-sections.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
On Windows, calling process.exit() while a recursive fs.watch handle is still
tearing down aborts the daemon with a libuv UV_HANDLE_CLOSING assertion
(0xC0000409) — reproducible whenever the indexed tree contains a nested repo
(submodule / embedded clone), since that's what keeps a watch active at shutdown.
A small exit delay doesn't help; only letting the loop drain is clean (verified
on a real Windows VM: close()+exit() and close()+setTimeout(exit) both abort,
while letting the loop drain exits 0).
finalizeDaemonExit() now exits immediately on POSIX (unchanged) but on Windows
marks success (exitCode=0) and lets the loop drain to a natural exit, with an
unref'd backstop that force-exits only if a stray handle would otherwise hang
shutdown. The daemon's own timers are already unref'd and its PPID watchdog lives
in the proxy, so nothing keeps the loop alive past the closing watch handles —
natural drain is fast. Pure + platform-injected so both branches unit-test off-Windows.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
All codegraph_* tools are query-only — they read the pre-built index and
never mutate the workspace — but they advertised no MCP annotations, so
Cursor's Ask mode (and any client that gates on read-only tools) blocked
every call with "you are in ask mode and cannot run non read-only tools."
Add a shared READ_ONLY_ANNOTATIONS constant (readOnlyHint: true,
destructiveHint: false, idempotentHint: true, openWorldHint: false) and
reference it from each of the 8 tool definitions. The field flows through
every tools/list path: the live getTools() (including explore's
spread-rewritten description), the static proxy getStaticTools(), and the
no-default withRequiredProjectPath schema clone.
The annotations field is additive, so it ships without bumping the
negotiated 2024-11-05 protocol version: clients that gate on it read it
regardless, and older clients ignore it.
Closes#1018
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A project kept on an ExFAT/FAT external volume (or some network mounts /
WSL2 DrvFs) broke the background auto-sync daemon at two points, both
because the filesystem lacks POSIX features the daemon relied on:
1. Lock acquisition hard-links a temp file onto .codegraph/daemon.pid for
race-free exclusivity (#411) — these filesystems have no hard links.
2. The Unix-domain socket listen() fails regardless of path length, so the
old length-only tmpdir fallback never triggered.
Both surface as a capability error, but each OS reports a DIFFERENT errno
for the same gap (macOS ENOTSUP, Linux EPERM, Windows EISDIR), so the fix
is policy-based rather than an enumerated code-set:
- Lock: fall back to an O_EXCL create on any non-EEXIST link error. The
temp write already proved the directory is writable, so the fallback
either succeeds (still atomic + exclusive, "first writer wins") or
surfaces its own genuine error.
- Socket: an ordered candidate list [in-project, tmpdir] walked by BOTH
the daemon (binds) and the proxy (connects) — they converge on the
fallback with zero coordination. Relocate past any non-EADDRINUSE bind
error; EADDRINUSE still rethrows, preserving the #974 contract.
Normal repos are unaffected: the in-project candidate binds first, and the
hard-link lock path is unchanged.
Validated end-to-end on real removable-drive filesystems: macOS ExFAT
(hdiutil image), Linux FAT32 (Docker loop mount), Windows exFAT (diskpart
VHD) — each acquires the lock, relocates (or binds a named pipe on
Windows), and serves a real client.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Three fixes for a repo that commits a large JS/TS theme/SDK (Metronic under
static/, ~1,600 tracked files):
1. A SECOND "Resolving refs" quadratic that #915 didn't cover. #915 capped
import-name collisions; this caps method-name collisions (init/update/render
re-declared on every widget), which flow through matchMethodCall Strategy 3
and findBestMatch instead. New AMBIGUOUS_NAME_CEILING (default 500, env
CODEGRAPH_AMBIGUOUS_NAME_CEILING): above it the fuzzy strategies decline
rather than score K candidates — no proximity score can pick the one true
target among thousands anyway. Resolving drops from O(K^2) to linear in refs
(e.g. 900-file synthetic: 28.7s -> 3.4s), edge counts unchanged, and the cap
never fires on normal repos (max real method-collision ~40).
2. A new `exclude` array in codegraph.json keeps git-TRACKED paths out of the
index, which .gitignore can't do (enumeration is `git ls-files`). Mirrors the
existing includeIgnored plumbing across the git, sync, and non-git-walk
paths.
3. `index`/`init` now install the #850 liveness + #277 ppid watchdogs (which
were serve-only), so a wedged or orphaned indexer self-terminates instead of
pinning a core. The --liftoff-only relaunch's spawnSync can't forward
signals, so killing the parent shim used to orphan the worker.
Tests: ubiquitous-name ceiling, exclude (incl. tracked-file exclusion on git +
non-git), orphan self-termination (POSIX), and ppid-parser units. Shared the
ppid parsers out of mcp/index.ts into mcp/ppid-watchdog.ts.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
When the server runs with no default project to fall back to — a gateway
server started outside any repo, or a monorepo root whose .codegraph/
indexes live only in sub-projects — every tool call must carry an explicit
projectPath. Previously projectPath was always optional, so an agent talking
to such a server would omit it, get success-shaped "pass projectPath"
guidance, and not reliably retry; the user had to nudge it by hand.
getTools() now marks projectPath required in the exposed tool schemas on the
no-default-project branch (a high-salience channel clients surface/validate,
unlike the instructions prose the reporter found too weak). When a default
project is open, projectPath stays optional and a bare call falls back to it.
The fix lives at the MCP schema layer, not the Claude-only front-load hook:
the hook is local-filesystem-based and never runs for the reporter (they're
on AGENTS.md / Codex-opencode). The proxy/getStaticTools path is untouched —
index.ts forces direct mode whenever resolveDaemonRoot is null, so the
no-default case never reaches the proxy.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The shared daemon served every session on one event loop with synchronous
node:sqlite. codegraph_explore is CPU-bound work stitched together by microtask
awaits, so N concurrent explores keep the microtask queue continuously full and
starve the macrotask phases — timers AND socket I/O. The transport freezes: no
response can flush until the whole batch drains, so with ~10 subagents on a large
repo clients routinely time out (reported via X by @symbolic2020).
Move the heavy read-tool dispatch onto a worker-thread pool. Each worker holds
its own WAL read connection (verified: a worker reader sees the main writer's
committed catch-up/watcher writes); the single watcher/writer, the catch-up gate,
codegraph_status, and the staleness/worktree notices stay on the main thread.
Concurrent reads now run in true parallel up to core count and the main loop
stays free for the MCP transport, so responses flush incrementally instead of
all-at-once after the batch drains. Enabled for the shared daemon only; direct
(single-stdio-client) mode is unchanged.
- crash recovery: respawn + retry-once, with a circuit breaker that falls back
to in-process dispatch if workers can't run on this platform
- graceful backstop: an overloaded pool returns success-shaped "busy, retry"
guidance, never isError (so it can't teach the agent to abandon codegraph)
- pending-aware growth + capped concurrent cold-starts avoid a startup
thundering herd (N simultaneous module-loads + DB opens could stall the loop)
- config: CODEGRAPH_QUERY_POOL_SIZE (default clamp(cores-1, 1, 16); 0 disables
→ in-process), CODEGRAPH_QUERY_BUSY_TIMEOUT_MS (default 45s)
10 concurrent explores on vscode (10.5k files): 31s → ~9s, staggered flush,
0 timeouts, byte-identical output; scales with cores (≈3.3× on 8, 1.8× on 2).
Full suite passes plus 10 new query-pool tests (fake-worker injection so the
scheduling logic is covered without spawning threads).
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The client-facing MCP proxy could exit with "Transport closed" when its
connection to the shared daemon hit a socket 'error' with no listener
attached — common on WSL2 /mnt (DrvFs), where AF_UNIX is flaky. The global
fatal handler turned that uncaughtException into process.exit(1), which the
MCP client saw as a bare transport close even though the index was healthy.
proxy.ts now keeps an 'error' listener on the daemon socket for its whole
life (and skips a socket destroyed in the connect window), so a stray error
degrades to the existing in-process fallback instead of crashing. daemon.ts
releases the lockfile it acquired when it fails to bind, so the next launch
doesn't spin on a stale lock (the duplicate serve --mcp pileup).
No default behavior change for anyone; WSL /mnt users who still hit trouble
can set CODEGRAPH_NO_DAEMON=1 to skip the shared daemon entirely. Validated
on macOS (unit + live serve probe) and Linux (Docker, --init): 64/64 across
the daemon/socket/lifecycle suites, incl. real AF_UNIX.
Closes#974
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The MCP server gated tool availability on whether the server root had a
.codegraph/ index, so in a monorepo where only sub-projects are indexed the
agent saw zero tools — and couldn't reach an indexed sub-project even by
projectPath. A session started before `codegraph init` also never surfaced the
tools afterward. The Claude front-load hook had the mirror gap: it only walked
UP for an index, so it stayed silent at a monorepo root.
MCP server:
- Always expose the tool surface; when the root isn't indexed, send a
per-project instructions variant (pass projectPath) instead of the
"inactive" note. Safety comes from response SHAPE (success-shaped guidance,
never isError), not from hiding tools.
- Reword the no-default-project guidance to be per-project, not per-session,
and sharpen the projectPath schema description.
Front-load hook (UserPromptSubmit):
- Scan DOWN (bounded depth, workspace-root-gated) for indexed sub-projects and
shape the injection by topology: front-load the one the prompt names, nudge
about the rest, or list them when ambiguous.
Verified: full suite (1703 passed); a live two-package monorepo run confirms the
hook front-loads the correct sub-project with no cross-package leakage. The
front-load's net speed effect is the existing multi-file-vs-single-file
tradeoff, unchanged by this work.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* perf(resolution): resolve imports to definitions, not sibling import nodes (#915)
"Resolving refs" crawled (tens of minutes) on large projects — most painfully
ones mixing a big front-end and back-end. An external package or module imported
across hundreds/thousands of files (react, a shared UI package, Python
logging/typing) is re-declared as an `import` node in every importing file, so
its unresolved import ref fell through to the exact-name matcher, which scored
all K same-named import nodes via findBestMatch — K refs x K candidates = O(K^2)
per package, producing only meaningless import->import edges.
Fix: exclude `import`-kind nodes as name-match targets (they're statements, not
definitions; real import->definition resolution is the import resolver's job).
Plus two safe constant-factor wins in findBestMatch: hoist the per-candidate
ref.filePath split, and skip cross-language candidates when a same-language one
exists (provably the same winner — same-language scores >=50, cross-language
maxes at 35).
Measured: superset (Py+TS) candidates scored 7.5M -> 833K (9x), non-import edges
preserved (+1618 now resolve to real defs), ~22K useless import->import edges
removed; kubernetes (Go) computePathProximity 37.2s -> 5.0s; synthetic 8k-file
mixed repo (K=4000) resolution 16.0s -> 1.7s. Full suite green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: correct stale better-sqlite3/wasm references to node:sqlite
The SQLite backend has been Node's built-in node:sqlite (real SQLite, WAL + FTS5,
from the bundled runtime) for a while — there is no native build step and no
node-sqlite3-wasm fallback. README and the docs site were already updated; this
catches the stragglers:
- CLAUDE.md: the src/db/ backend description and the sqlite-backend test note.
- src/db/index.ts, src/mcp/tools.ts: two code comments that still blamed "the
wasm backend" for non-WAL behavior (reworded to "when WAL isn't in effect").
Leaves tree-sitter grammar wasm (web-tree-sitter / --liftoff-only) untouched —
that's a different, still-current use of wasm.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(telemetry): drop the dead sqlite_backend field (schema v2)
node:sqlite is now the only backend, so the `index` event's `sqlite_backend`
field was a constant ("native") carrying no signal — and the `install` event
never actually sent it. Remove the field and the backendKind() helper, bump the
telemetry SCHEMA_VERSION 1 -> 2, and update TELEMETRY.md + docs/design/telemetry.md.
The ingest worker is deliberately left tolerant: `index` doesn't require the
field and schema_version validates as nonNegInt(99), so v2 events ingest fine and
old clients still sending v1 + sqlite_backend keep validating too. Added a legacy
comment there explaining it's safe to drop once old-client share is negligible.
telemetry.test.ts: the assertion pinning schema_version and a stale-claim fixture
line updated 1 -> 2. All telemetry tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
GoFrame's standard router binds routes reflectively (group.Bind(ctrl)): the path
and method live in a g.Meta struct tag on a request type, and the controller
method that serves it is matched by that request type at runtime — so there was
no path string and no edge from a route to its handler, and "where is this route
handled / where are routes bound to controllers?" could only be answered
lexically (issue #720's report).
- frameworks/goframe.ts: detect gogf/gf in go.mod, extract each path-bearing
g.Meta into a route node (requires path:, so response mime:-only tags are
skipped), encoding the package-qualified request type for the join.
- goframe-synthesizer.ts: join each route -> the controller method whose
signature takes that request type — NOT by name (DeptSearchReq is served by
List) — keyed pkg.Type to disambiguate the many identical bare names a large
app defines one-per-module, with an addon-root tiebreak for cloned demo addons.
Edge kind calls, provenance heuristic, synthesizedBy goframe-route, surfaced as
a dynamic-dispatch hop in codegraph_explore.
Validated on real repos: gf-demo-user 7/7, gfast 65/68 (3 genuinely
handler-less), hotgo 242/247 (98%) — 100% precision (0 non-controller handlers,
0 core/addon cross-binding), node count stable. Agent A/B (gfast, sonnet/high,
2 runs/arm): with codegraph 1 explore call / 0 Read / ~20s vs without 7.5 Read
avg + grep-hunting for the non-existent literal route string / ~42s; same correct
answer.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
C/C++ polymorphism is the function pointer: a struct fn-pointer field, concrete
functions registered into it through a table (`{"add", cmd_add}`), a designated
initializer (`.handler = on_open`), or an assignment, then dispatched indirectly
(`p->fn(argv)`). Static extraction captures neither the registration→field
binding nor the indirect call, so the dispatcher→handler edge was missing — git's
run_builtin looked like it called nothing, a vtable's implementations had no
callers, and the hook_demo.c in the issue was unreachable.
Add a resolution-layer synthesizer keyed by (struct type, fn-pointer field). It
reads source (the established Celery/Sidekiq/Spring pattern — C extraction has no
struct fields or indirect-call edges to build on) in passes: collect fn-pointer
typedefs, parse struct field layouts, collect registrations (positional matched
by field index, designated, and assignment), propagate field←field assignments
(so a generic hook slot reassigned from a registry — the hook_demo.c
`h->func = found->fn` shape — inherits the registry field's handlers), then link
each indirect dispatch site to the registered handlers. Receiver type resolves
from the enclosing function's params/locals, falling back to a field name unique
to one struct. Covers both the command-table idiom (git, redis) and the
ops-struct/vtable idiom (curl content-encoders, protocol handlers).
Pure edge synthesis (no node growth); high precision via the (struct, field) key.
Validated: git 502 edges (run_builtin→cmd_* plus git_hash_algo/archiver/reftable
vtables), redis 357 (dictType.hashFunction, connection + reply-object vtables),
curl 478 (Curl_cwtype.do_init → deflate/gzip/brotli/zstd); 0 non-function targets
on all three; node-stable; 0 on the lua control (its {name,fn} tables register
into the Lua VM, with no C indirect call to bridge). Full suite 1665 pass.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
MCP tool results used Markdown ATX headings (##/###/####) for section
headers — the status summary, each search hit, every file section in an
exploration — which Markdown-rendering clients (e.g. the Claude Code
VSCode extension) blow up to H1–H4 font size, filling the transcript with
oversized lines (worst on search/explore, where the noise scales with
result count). Swap them all for bold labels, which render at body size
while keeping the same structure. CLI/TTY output (ContextBuilder) is
unchanged — the issue notes it's fine.
The format is parse-coupled, so kept in sync:
- The explore truncation boundary and the offload chunker
(reasoning/reasoner.ts) both key off the per-file header, now a unique
`**`-prefixed marker emitted via a shared fileSectionHeader() helper.
- Updated the offload strip regexes and switched the opt-in report-style
prompt off ATX headings (same client, same rendering issue).
- Updated test helpers (sectionFor, sourcedFiles, the callers
section-boundary scan) that scanned the old markers.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
On a very large repo (the report is a ~93k-file / 5.7GB-DB Java monorepo) the
first MCP `tools/call` after a fresh `serve --mcp` could hang for 10+ minutes
with zero output, and with the liveness watchdog on, the daemon was SIGKILLed
mid-query instead. Root cause: the post-open catch-up reconcile that the first
tool call is gated on does ~2*N synchronous `fs.existsSync`/`fs.statSync` calls
plus a load-all-files query in two non-yielding loops. On a huge repo that wedges
the event loop for minutes, which (a) trips the 60s watchdog (it SIGKILLs a
process whose loop stops turning) and (b) blocks the first call the whole time.
Two complementary fixes:
- Make the reconcile yield. `ExtractionOrchestrator.sync()` now uses the
yielding `scanDirectoryAsync`, and both O(files) reconcile loops
`await setImmediate` every SYNC_RECONCILE_YIELD_INTERVAL (1000) files. The loop
can no longer wedge the main thread, so the watchdog stays fed and the socket /
any concurrent read stays responsive while a big reconcile runs. Results are
unchanged — only yield points are added.
- Time-box the catch-up gate. The first `tools/call` now waits on the reconcile
for at most CODEGRAPH_CATCHUP_GATE_TIMEOUT_MS (default 3000ms), then serves and
lets the reconcile finish in the background (which now yields, so the served
call runs concurrently). `=0` restores the old unbounded wait. On a normal repo
the reconcile finishes well under the budget, so behavior is unchanged.
Tests: adds two time-box cases to mcp-catchup-gate (serves promptly when the
reconcile runs long; `=0` restores the unbounded wait). Full suite green
(1655 passed). Validated end-to-end through the real daemon: first call returns
at the ~3s time-box instead of waiting an injected 8s reconcile; no-delay control
unchanged; `=0` opt-out waits the full reconcile.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A long-lived `serve --mcp` process opens `.codegraph/codegraph.db` and holds
the fd for its whole life. If `.codegraph/` is removed and recreated AT THE
SAME PATH while it runs — `git worktree remove <p>` + re-add, or `rm -rf
.codegraph` + `codegraph init` — the held fd points at the now-unlinked inode
and can never see the new index. The server serves the pre-removal snapshot
(renamed/removed symbols still "live", new ones missing); `codegraph sync`
can't refresh it and the CLI (a fresh process) diverges. Only a restart fixed
it — and because the daemon registry is keyed by path, a same-path recreate
routes new clients straight back to the same stale daemon, so the fix has to
self-heal inside the running process.
- DatabaseConnection records the DB file's (dev, ino) at open and exposes
isReplacedOnDisk() — a different inode now at the same path. POSIX-gated:
Windows can't unlink an open file and its st_ino is unreliable, so it never
fires there.
- CodeGraph.reopenIfReplaced() opens the live file first, then swaps the
connection + query layers IN PLACE (via the new wireLayers() helper), so
every holder of the instance (the daemon's default project, cached
projectPath connections) heals without a restart. Closing the dead handle
also frees the leaked db/-wal/-shm fds pinning the unlinked inode.
- ToolHandler.getCodeGraph calls it (freshen) before serving — one stat() per
call, a no-op unless the inode actually changed, never throws into a tool.
Tests cover isReplacedOnDisk (unchanged / replaced / absent / Windows-gated)
and an end-to-end reopen that heals a held instance after a same-path recreate
(asserts the pre-heal staleness too). Validated on macOS with a dist probe of
the raw instance and the MCP serving path; full suite green.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A long-lived MCP server (the shared daemon) cached both the project handle
(projectCache) and the worktree-mismatch verdict (worktreeMismatchCache) for
its whole lifetime — cleared only on shutdown — keyed off the input path and
never re-checked. So a worktree path first resolved BEFORE it had its own
.codegraph/ — when the walk-up reached the main checkout — stayed pinned to
the main checkout: every query kept hitting the wrong index and every result
carried a false "this index belongs to a different git working tree" warning,
until the server restarted. The CLI was correct (fresh process per run);
re-indexing didn't help.
- getCodeGraph: re-resolve findNearestCodeGraphRoot on every call (cheap stat
walk, no git, no reopen) and cache the open DB by RESOLVED ROOT only. Drops
the input-path short-circuit that pinned the first resolution, and the
double-keying (which also double-closed each instance in closeAll()).
- worktreeMismatchFor: key the verdict on (startPath, indexRoot) so a changed
index root recomputes instead of serving the stale "borrowed the parent's
index" verdict.
Adds a regression test that fails pre-fix (the stale warning survives the
index-root flip); the existing "no further git spawn" caching test still
passes. The query-staleness half (a second project opened through the handler)
isn't unit-testable under vitest, so it was validated with a dist A/B.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two fixes hardening the redux-thunk dynamic-dispatch synthesizer, found by
validating it on real RTK repos beyond its trezor origin (uwave-web,
session-desktop, octo-call):
- Surfacing: buildFlowFromNamedSymbols filtered its named set to CALLABLE
kinds, so synthesized edges between `constant` nodes (RTK thunks are
`const X = createAsyncThunk(...)`) never entered the Flow / Dynamic-dispatch
links scan — invisible at every tier, while the kind-agnostic Relationships
section is off below 500 files. Add a `dynNamed` set (named constant/variable/
field nodes with a heuristic edge) feeding a shared collectSynthLinks into the
"## Dynamic-dispatch links" section, threaded through the named.size<2
early-out (both-endpoints-constant hit return EMPTY first) and the main path.
Main call-chain stays callable-only; the <500 budget tiers are untouched.
No-op for callable flows. Plus a generic synthEdgeNote fallback so any synth
hop reads "dynamic: <kind> @site", not a bare "[calls]".
- Precision: reduxThunkEdges resolved a dispatched name by first-match-by-kind,
so a thunk name colliding with a same-named service function linked to the
wrong node (octo-call `leaveCall`). Prefer thunk-signature const > other
const > same-file callable > first match.
Tests: new explore-synth-constant-endpoints.test.ts (surfacing on a small repo)
+ a collision case in redux-thunk-synthesizer.test.ts. Full suite green (1605).
Rationale + coverage backlog in docs/design/dispatch-synthesizer-backlog.md.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Strips the bring-your-own-model reasoning offload and managed CodeGraph AI
integration (login/logout/usage commands, offload config/credentials/reasoner
modules, and the synthesizeOffload call in codegraph_explore). The eval findings
showed raw source output outperformed the synthesized path on accuracy, so
codegraph_explore reverts to returning verbatim retrieved source exclusively.
CHANGELOG and README sections for reasoning offload are removed; test comments
and DEFAULT_MCP_TOOLS description are updated to drop offload references.
Two gaps closed in `codegraph_explore` output quality:
**Interface/registry dispatch (#687 extension).** When a named token resolves to
a large same-name family (≥8 members) that doesn't land on the connected flow, the
static path truly ends there — the target is chosen at runtime from N implementations
(plugin/strategy/handler interface). `buildPolymorphicBoundaries` detects this via
`implements`/`extends` edges, ranks candidate supertypes by their TRUE graph-wide
implementer count (not FTS sample frequency, which is biased), and emits a
"## Interface dispatch" section naming the supertype, implementer count, and a few
concrete targets. Fires only for uncovered named tokens; a connected flow stays silent.
**Oversize spine method windowing.** A flow entry that is a god-method (e.g. n8n's
962-line `processRunExecutionData`) previously lost the per-file budget to denser
peripheral blocks and was dropped, forcing the agent to `Read` it back. The spine
call site (edge line to the next hop) is now tracked via `spineCallSites` and used
to window the method to its signature head + a ±28-line band around the call, keeping
it under the OVERSIZE_SPINE_LINES threshold. Spine clusters also rank first in the
budget sort and may exceed the per-file cap up to a 2.5× ceiling so they can never be
starved by co-flow files.
Test suite gains an `interface dispatch` describe block (announce, silent-on-connected,
silent-below-threshold) and uses `beforeAll`/`afterAll` to pin `CODEGRAPH_OFFLOAD_DISABLE=1`
so structural assertions are hermetic regardless of machine config.
codegraph_explore can now hand the source it retrieved to a reasoning model you
point at — any OpenAI-compatible endpoint (Cerebras, OpenAI, a local vLLM/Ollama)
with your own key — and return that model's tight, cited answer instead of the
raw source dump. The agent's main context gets the answer in far fewer tokens, at
the cost of one network round-trip.
Off by default. Configure with `codegraph offload set-endpoint <url> --model <m>
--key-env <ENV>` (or the CODEGRAPH_OFFLOAD_* env vars); status/disable manage it.
The API key is never written to disk — the config stores the NAME of an env var
and the key is read from it at call time. Strictly degradable: any failure
(no endpoint, network, timeout, empty answer) returns null and the call falls
back to the local source, so the offload can never surface an error to the agent.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
codegraph_explore's file sort is primarily driven by Random-Walk-with-Restart
graph-centrality mass, seeded from the query's text matches. In a cross-layer
monorepo (an API server alongside a much larger, internally dense frontend that
mirrors the same domain words), that mass skews to the bigger layer — so a
backend service/handler that genuinely matches several query terms, even when
it's the #1 search hit, sorts below hits=0 frontend files and gets truncated out
of the response, and the agent reads it back.
Add a corroboration tier above the graph signal: a file that is BOTH an
entry/central file AND matched by >=2 distinct query terms is kept in. The
entry/central guard prevents an incidental multi-term file (a type/util file
that isn't the flow) from displacing a graph-central answer file — a blunt
hits-only tier regressed that case. Single-layer repos are unaffected. Gated by
CODEGRAPH_RANK_NO_MULTITERM=1.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
When live file watching permanently degrades (watch-resource exhaustion, or a
write lock held past the retry budget), getPendingFiles() goes empty — so the
existing per-file staleness banner can't fire even though the index is now
frozen and silently drifting stale. The agent kept getting clean-looking
responses off a no-longer-updating index.
Read-tool responses now lead with a whole-index banner ("CodeGraph auto-sync
is DISABLED…") whenever the watcher is degraded, and codegraph_status gets a
dedicated "Auto-sync disabled" section. Both carry the degrade reason and tell
the agent to Read files directly. Expose isWatcherDegraded() /
getWatcherDegradedReason() on the CodeGraph class, and document the new banner
in the MCP server instructions.
Completes the agent-notification half of #876 (the operator-facing onDegraded
wiring shipped in #891).
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The live file watcher could stay "alive" after it had stopped being
trustworthy. EMFILE/ENFILE watch-resource exhaustion only logged (and was
silently tolerated on the Linux per-directory path), and prolonged
LockUnavailableError retried forever at the normal debounce cadence — both
left auto-sync dead while the index silently drifted stale. Especially bad
for long-running MCP/daemon sessions.
Add a one-way degrade(): on watch-resource exhaustion (any watch strategy)
or on lock contention past a bounded exponential-backoff budget, log once,
fire a new onDegraded callback, and stop. start() now returns false
consistently when the per-directory path degrades at startup — it previously
returned true on Linux, so the MCP server reported the watcher "active" when
it had degraded. Wire onDegraded into the MCP server so callers are actually
told, and expose isDegraded()/getDegradedReason().
Builds on the approach in #877 by @thismilktea. Validated on macOS
(recursive), Linux (per-directory, Docker) and Windows (recursive) — 30/30
watcher + watch-policy tests on each.
Closes#876
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Collapses the unreleased daemon controls into a single interactive command.
`codegraph daemon` (alias `daemons`) opens an arrow-key picker (current project's
daemon first, pre-selected), enter stops it, or pick "Stop all"; non-TTY prints a
plain list. Removes stop/list/ps; reuses the unchanged daemon-registry machinery;
the pick->stop loop is in daemon-manager.ts behind an injectable select (unit
tested). Validated live on macOS/Linux (real clack picker driven via pty) and
Windows (real runDaemonPicker + stopDaemonAt against a real daemon). Closes#845
follow-up.
Adds first-class daemon control (the #845 pain point: no clean way to stop a
runaway daemon). `codegraph stop [path]` stops the current/given project's
daemon (SIGTERM -> SIGKILL fallback, sweeps artifacts); `stop --all` stops every
daemon; `list`/`ps` shows running daemons (--json for scripts).
Discovery via a small self-healing registry: each daemon records its root under
~/.codegraph/daemons/ on start, removes it on graceful shutdown; readers prune
dead pids. Cross-platform by construction (files + process.kill). Validated live
on macOS, Linux (docker), and Windows (VM): registry unit 6/6 and real-daemon
stop/list 6/6 on each.
The worker-thread watchdog from #856 didn't work in the real daemon — caught by
live-testing against a real serve --mcp. V8 isolates coordinate on global
safepoints, so a main thread wedged in a tight non-allocating loop (#850's
SourcePositionTableIterator::Advance) strands the watchdog worker before it can
SIGKILL.
A separate child process shares no isolate/heap with the parent, so the wedge
can't touch it; it kills via the kernel. Parent heartbeats to the child's stdin;
silence past the timeout -> SIGKILL; parent exit closes the pipe -> child exits.
Validated live (real daemon SIGKILLed in ~timeout); regression test covers the
non-allocating-wedge-under-heap-pressure case. API/install points/CHANGELOG
unchanged; the broken worker version was never released.
Belt-and-suspenders follow-up to #855. Any non-yielding sync loop on the main
thread wedges the event loop, and nothing running on that loop (timers, signal
handlers, PPID watchdog) can recover it — only another thread can.
A tiny worker thread (in the detached daemon + direct modes) watches a
shared-memory heartbeat the main thread bumps each event-loop turn; if it stops
advancing across enough consecutive checks (~CODEGRAPH_WATCHDOG_TIMEOUT_MS,
default 60s) the worker SIGKILLs the process so a fresh daemon starts on the next
connection. Counts consecutive stale checks (not wall-clock) so it's immune to
clock jumps / sleep; tuned never to fire on real work; opt out with
CODEGRAPH_NO_WATCHDOG=1.
* feat(explore): announce dynamic-dispatch boundaries when a flow can't connect statically (#687)
When buildFlowFromNamedSymbols can't connect the agent's named symbols, scan
the disconnected symbols' bodies (query-time, deterministic, zero graph
mutation) for dynamic-dispatch forms — computed member calls, getattr,
reflection, typed message buses, runtime-keyed emits, Proxy — and announce
the exact site where the static path ends, with candidate runtime targets
when a dispatch key is statically visible. The honest alternative to
guessing edges: surface the boundary, don't fabricate the bridge.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(agent-eval): ab-new-vs-baseline survives files added since the baseline ref
A single multi-file 'git checkout <ref> --' with one unknown pathspec checks
out nothing, so the baseline arm silently ran the NEW build. Check out
per-file and remove files that don't exist on the baseline.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* docs(playbook): boundary surfacing as the mechanism floor for non-gateable dispatch (#687)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(explore): render a direct synthesized hop between two named symbols (#687)
A 2-node chain populates pathIds but renders nothing (Flow needs >=3), and
the dynamic-links section skipped its edge as 'already in the main chain' —
so a custom EventBus emit→handler connection was invisible. Skip-as-in-chain
now applies only when a chain actually renders, and the boundary scan treats
short-chain endpoints as connected.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Colby McHenryandClaude Opus 5
