Colby McHenryandClaude Opus 5
2cf63fd114
A live, auto-sync-maintained index does not converge to a clean full rebuild of the identical tree. On codegraph's own repo, 4.3% of distinct edges are wrong in both directions (751 missing, 476 stale), dominated by `calls` — the edges flow queries traverse and that feed the RWR mass explore ranks files by. Raw edge rows differ by only +0.7%, because the divergence is bidirectional and nets out; any drift check must compare edge SETS. Rebuild-vs-rebuild is 0, so the indexer is deterministic and this is not noise. Node sets are identical and every integrity check is 0 on both indexes, so this is stale cross-file resolution, not accumulated residue. `diff-index-drift.mjs` is read-only and takes two index paths — rebuilding is the caller's job, so the tool can never clobber the artifact it is measuring. It also refuses a missing path, since node:sqlite creates an empty database rather than failing and an empty schema reads exactly like a stale pre-migration index. Diagnostic captures from the originating incident are deliberately NOT committed: they contain verbatim source from a private repo. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
4.4 KiB
Index drift: incremental sync vs. full rebuild (CG-33)
Measured 2026-08-06. A live, auto-sync-maintained index does not converge to
a clean full rebuild of the identical working tree. On codegraph's own repo,
4.3% of distinct edges were wrong, in both directions, overwhelmingly
calls edges.
This matters because it is silent: nothing warns, nothing surfaces it, and the README tells users the index is never stale and there is nothing to re-run. Retrieval quality decays invisibly, and the user-visible symptom — an agent falling back to Read — reads as "codegraph isn't very good" rather than "this index needs rebuilding."
Result
Subject: codegraph's own .codegraph/codegraph.db, long-lived and
incrementally synced, against a full rebuild of the same tree with the same
build. Edges compared as distinct (source, target, kind) triples.
| count | |
|---|---|
| distinct edge triples (rebuild) | 28,809 |
| in rebuild but missing from live | 751 |
| in live but absent from rebuild (stale) | 476 |
| total divergent | 1,227 — 4.3% |
Missing edges by kind: calls=635, contains=38, references=34,
instantiates=21, imports=13, extends=10.
Raw counts hide it
Raw edge rows were 39,845 live vs 40,122 rebuilt — a benign-looking +0.7%. The divergence is bidirectional, so a net-count check nets it out and reports almost nothing wrong. Any drift detector must compare edge sets, not totals.
The indexer is deterministic
Control, rebuild vs rebuild on the same tree and build: 0 differing edges (28,809 both runs). So the live-vs-rebuild delta is not run-to-run noise.
It is resolution, not residue
Node sets are identical — files 501 = 501, nodes 10,110 = 10,110,
heuristic edges 36 = 36 — and every integrity check is 0 on both indexes:
no duplicate nodes, no orphan edges, no nodes referencing a missing file row.
Nothing accumulates. Cross-file resolution goes stale.
Likely mechanism
ReferenceResolver resolves calls and imports by name-matching and the import
graph across the whole project. Incremental sync re-parses and re-resolves
only the changed file, so:
- edges from other files into changed symbols are never recomputed → stale edges retained (the 476);
- edges that should newly form from unchanged files into changed symbols are never created → missing edges (the 751).
Start in src/sync/ and src/resolution/ — specifically what scope is
re-resolved on a single-file change.
Why it degrades retrieval
Graph mass (RWR) is relative and normalized, so call edges missing elsewhere
inflate an unaffected file's share of the mass. Explore ranks files by that mass
(allocateExploreBudget weights on it), so drift silently promotes files that
should rank low.
Observed on a private application repo under heavy development: a generated ambient-types file carried graph mass 0.24750 on the drifted index vs 0.13119 on a clean rebuild (~1.9×), and score 49.0 vs 27.0. On the drifted index it took 60.7% of an explore envelope and starved the file the agent had actually named by symbol, which rendered 251 chars of a 10,970 reservation. After a full re-index — no code change — the same query answers correctly. That incident is what prompted this measurement; see CG-24.
Severity scales with churn and index age. codegraph's own repo shows 4.3%; a repo under heavier active development plausibly drifts further.
Reproducing
scripts/agent-eval/diff-index-drift.mjs is read-only and diffs two indexes.
Snapshot the live index before rebuilding — the original artifact for this
investigation was destroyed by re-indexing over it:
cp .codegraph/codegraph.db /tmp/live.db # snapshot FIRST
node dist/bin/codegraph.js index . # full rebuild
node scripts/agent-eval/diff-index-drift.mjs /tmp/live.db .codegraph/codegraph.db
Exit code is 0 when converged, 1 when drifted. To re-confirm determinism, diff two consecutive rebuilds — that must report 0.
Note on probing an index
The index file is .codegraph/codegraph.db. There is no graph.db. sqlite3
against a mistyped path creates an empty database rather than failing, and
every subsequent query then answers from an empty schema — which reads exactly
like a stale pre-migration index. That produced a wrong root cause during this
investigation. diff-index-drift.mjs checks existsSync before opening for
exactly this reason.