Colby McHenryandClaude Opus 5
03893b0ab9
A live, auto-synced index did not converge to a clean rebuild of the same tree — 4.3% of distinct edges wrong in both directions on this repo's own index, overwhelmingly `calls`, which is what flow queries traverse and what explore's file ranking weights. Silent: nothing warned, and the symptom read as "codegraph isn't very good" rather than "this index needs rebuilding." Two causes, and the fix needed both. Resolution binds a reference to one of the same-named definitions PROJECT-WIDE, so a definition appearing or vanishing changes the correct answer for references in files the sync never touches — and those references resolved successfully once, which deletes their unresolved_refs row, leaving nothing to revisit them with (#1240's retry only revisits refs parked as failed). Separately, when nothing disambiguated the candidates the winner came down to rowid, i.e. the order files happened to be WRITTEN, which differs between a scan-order full index and a sync that appends each file as it changes. That second one is why re-resolution alone could not converge: re-resolving against the identical graph still picked a different candidate. So getNodesByName now orders by (file_path, start_line) — a property of the code, not of the write order — and sync computes a definitionDelta and re-opens the resolution edges whose answer it may have invalidated, re-inserting each as the reference that created it for the orphan sweep to bind against the post-sync graph. The delta compares `file\0name` pairs per file rather than one name set over the batch: a commit that adds `collect` to a new file while an unrelated changed file already defines `collect` cancels out of a batch-wide set, and that miss was the largest residual class in the first measurement. Conservative where the failure modes are asymmetric — a wrong deletion is a permanent edge loss, a missed rebind is only residual drift. Edges without a refName stamp are never touched (nothing to restore them from), sources the sync already re-extracted are skipped, and a per-name ceiling declines the generic names. Edges are deleted before the sweep re-inserts, since INSERT OR IGNORE against idx_edges_identity would otherwise keep both rows when a reference rebinds elsewhere. Replaying real commits of this repo through sync, then diffing against a rebuild: 16 commits 48 -> 0; 80 commits 1,634 -> 361, with the actively misleading direction (stale edges the index keeps asserting) 671 -> 2. Index and sync wall-clock are unchanged; the ORDER BY costs 18% per uncached name lookup, which never reaches wall-clock because the resolver memoizes it. The 357-edge residual at 80 commits is one pre-existing class: refs to generic names (`push`, `join`) parked above #1240's per-name retry ceiling, which a rebuild resolves into cross-language garbage — a TS test file "calling" an R method. Converging there would mean manufacturing wrong edges, so it is left alone. And no drift metric in `codegraph status`: it cannot be computed without the rebuild it would be recommending, and a proxy would fire on that residual and train users to ignore it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
9.3 KiB
Index drift: incremental sync vs. full rebuild (CG-33)
Measured 2026-08-06. A live, auto-sync-maintained index does not converge to
a clean full rebuild of the identical working tree. On codegraph's own repo,
4.3% of distinct edges were wrong, in both directions, overwhelmingly
calls edges.
This matters because it is silent: nothing warns, nothing surfaces it, and the README tells users the index is never stale and there is nothing to re-run. Retrieval quality decays invisibly, and the user-visible symptom — an agent falling back to Read — reads as "codegraph isn't very good" rather than "this index needs rebuilding."
Result
Subject: codegraph's own .codegraph/codegraph.db, long-lived and
incrementally synced, against a full rebuild of the same tree with the same
build. Edges compared as distinct (source, target, kind) triples.
| count | |
|---|---|
| distinct edge triples (rebuild) | 28,809 |
| in rebuild but missing from live | 751 |
| in live but absent from rebuild (stale) | 476 |
| total divergent | 1,227 — 4.3% |
Missing edges by kind: calls=635, contains=38, references=34,
instantiates=21, imports=13, extends=10.
Raw counts hide it
Raw edge rows were 39,845 live vs 40,122 rebuilt — a benign-looking +0.7%. The divergence is bidirectional, so a net-count check nets it out and reports almost nothing wrong. Any drift detector must compare edge sets, not totals.
The indexer is deterministic
Control, rebuild vs rebuild on the same tree and build: 0 differing edges (28,809 both runs). So the live-vs-rebuild delta is not run-to-run noise.
It is resolution, not residue
Node sets are identical — files 501 = 501, nodes 10,110 = 10,110,
heuristic edges 36 = 36 — and every integrity check is 0 on both indexes:
no duplicate nodes, no orphan edges, no nodes referencing a missing file row.
Nothing accumulates. Cross-file resolution goes stale.
Mechanism — two causes, both confirmed
ReferenceResolver binds a reference to one of the same-named definitions
project-wide. Two things follow, and the drift needed both to be fixed.
1. Scope. Incremental sync re-resolves only the references in the changed
files. Adding or removing a definition of pct changes the correct answer for
every pct(...) reference in the repo, including references in files the sync
never touches — and those references resolved successfully once, which deletes
their unresolved_refs row, so nothing existed to revisit them with. (The #1240
retry only revisits refs parked as status='failed'.) The index kept an answer
that was correct against an older graph.
2. Tie-break. When nothing disambiguated the candidates, findBestMatch
kept the first one, and getNodesByName had no ORDER BY — so the winner was
decided by rowid, i.e. by the order files happened to be written. A full
index writes in scan order; a sync appends each file as it changes. The same
tree therefore resolved to different edges depending on how the index was built,
and no amount of re-resolution could converge, because re-resolving against the
identical graph still picked a different candidate.
The fix
getNodesByNameorders by(file_path, start_line)— a property of the code, not of the write order (src/db/queries.ts).syncreturns adefinitionDelta: the names whose set of definitions the sync changed, computed as the symmetric difference offile\0namepairs sampled before and after the store phase (ExtractionOrchestrator.sync).- For each delta name,
resurrectStaleResolutionEdgesdeletes the resolution edges targeting a symbol of that name whose source is in an unchanged file, and re-inserts each as the reference that created it (themetadata.refNamestamp). The existing orphan sweep then resolves them against the post-sync graph — the same input a rebuild resolves from. Kill switch:CODEGRAPH_NO_REBIND=1.
The delta is compared per file, not as one name set over the whole batch: a
commit that adds collect to a new file while an unrelated changed file already
defines collect cancels out of a batch-wide name set, and that miss was the
largest residual class in the first measurement of this fix.
Conservative by construction, because a wrong deletion is a permanent edge loss
while a missed rebind is only residual drift: an edge with no refName stamp
(synthesized, or built by an older engine) is never touched, edges whose source
file the sync already re-extracted are skipped, and a per-name ceiling of 500
edges declines the generic names.
Result
Replaying real commits of this repo through sync one at a time, then diffing
against a clean rebuild of the final tree:
| replay | baseline (main) |
+ ORDER BY only | + rebind pass (shipped) |
|---|---|---|---|
| 16 commits | 48 (24 missing / 24 stale) | 20 | 0 — converged |
| 80 commits | 1,634 (963 / 671) | 890 | 361 (359 / 2) |
The direction that actively misleads — stale edges the index keeps asserting — drops from 671 to 2 over 80 commits, a 99.7% reduction.
Index and sync wall-clock are unchanged (392-file repo: index 1.88–2.02s in both
arms, single-file sync 0.183s in both). The ORDER BY costs 18% per uncached
name lookup in a tight loop (237ms → 280ms over 10,127 lookups), which does not
reach wall-clock because ReferenceResolver memoizes the lookup per name. A
composite (name, file_path, start_line) index would make the sort free, but it
would widen every node index entry with a full path string on the write-heavy
indexing path — not worth 43ms.
The residual, and why it is not chased
At 80 commits, 357 of the 361 remaining edges are a single pre-existing class:
references to very generic names (push 260, join 97) that failed at index
time and stay parked because getRetryableFailedReferences declines any name
with more than 500 failed refs (1,412 for push, 2,346 for join). That
ceiling is #1240/#999 policy, it is present on main, and what it declines to
create is cross-language garbage: a TypeScript test file "calling" an R method
named push, or a Rust method named join. The full rebuild is the wrong one
here — converging would mean teaching sync to manufacture thousands of wrong
edges. Left as is, deliberately.
codegraph status — decided: no drift metric
The issue asked whether status should surface divergence. Decision: no.
A drift number cannot be computed without the full rebuild it would be
recommending, so anything cheap enough to run on status would be an estimate —
and an honest estimate is not available. Shipping a proxy would violate the
product rule that a screen must not overclaim, and post-fix it would fire on the
generic-name residual above, training users to ignore it. (status already
refuses to warn on parked failed refs for the same reason: every repo with
external-library imports has them, so the warning would be permanent noise.)
The check that is exact stays available and is documented below.
Why it degrades retrieval
Graph mass (RWR) is relative and normalized, so call edges missing elsewhere
inflate an unaffected file's share of the mass. Explore ranks files by that mass
(allocateExploreBudget weights on it), so drift silently promotes files that
should rank low.
Observed on a private application repo under heavy development: a generated ambient-types file carried graph mass 0.24750 on the drifted index vs 0.13119 on a clean rebuild (~1.9×), and score 49.0 vs 27.0. On the drifted index it took 60.7% of an explore envelope and starved the file the agent had actually named by symbol, which rendered 251 chars of a 10,970 reservation. After a full re-index — no code change — the same query answers correctly. That incident is what prompted this measurement; see CG-24.
Severity scales with churn and index age. codegraph's own repo shows 4.3%; a repo under heavier active development plausibly drifts further.
Reproducing
scripts/agent-eval/diff-index-drift.mjs is read-only and diffs two indexes.
Snapshot the live index before rebuilding — the original artifact for this
investigation was destroyed by re-indexing over it:
cp .codegraph/codegraph.db /tmp/live.db # snapshot FIRST
node dist/bin/codegraph.js index . # full rebuild
node scripts/agent-eval/diff-index-drift.mjs /tmp/live.db .codegraph/codegraph.db
Exit code is 0 when converged, 1 when drifted. To re-confirm determinism, diff two consecutive rebuilds — that must report 0.
To reproduce the regression rather than measure a live index, replay real
commits through sync: clone the repo, check out HEAD~N, index, then
git checkout <sha> && codegraph sync for each commit in order, snapshot the
database, and diff it against a rebuild of the final tree. That is what produced
the table above, and the unit-scale version of it is
__tests__/sync-rebuild-convergence.test.ts.
Note on probing an index
The index file is .codegraph/codegraph.db. There is no graph.db. sqlite3
against a mistyped path creates an empty database rather than failing, and
every subsequent query then answers from an empty schema — which reads exactly
like a stale pre-migration index. That produced a wrong root cause during this
investigation. diff-index-drift.mjs checks existsSync before opening for
exactly this reason.