fix(index): rebuild a poisoned/oversized index by recreating the DB, not row-DELETE (#1067) (#1073)

Follow-up to #1065/#1066. Those stopped a *new* index from scanning an
ignored gitlink corpus, but a project that had already built the multi-GB
graph before upgrading still couldn't recover: `codegraph index` printed
only "Indexing project" and was then SIGKILLed (137) by the #850 watchdog
~60s later, before scanning even started.

Root cause is not the scanner. `index` cleared the old graph with a
synchronous `DELETE FROM nodes/edges/files`. `nodes` carries an FTS5
`AFTER DELETE` trigger, so deleting ~1.6M rows fires ~1.6M FTS
delete-markers — O(rows), and it grows the WAL further before it can
finish. A deterministic probe puts the DELETE-clear at 20.4s on 1.5M
synthetic nodes (WAL 1.16->2.14GB); at the report's denser ~2.6KB/node WAL
that crosses the 60s main-thread watchdog. `open()` was never the wedge.

A full re-index is documented as "same result as a fresh init", so make it
one: discard the database files and re-initialize, instead of opening the
old DB and DELETE-ing every row.

- db: add removeDatabaseFiles(dbPath) — unlinks codegraph.db + its
  -wal/-shm sidecars (O(1) regardless of size; sidecars best-effort).
- index: add CodeGraph.recreate(projectRoot) — discards the files and
  returns a fresh, empty instance. Never opens or migrates the poisoned
  DB. POSIX unlinks an open file fine (a live daemon heals via
  reopenIfReplaced, #925); a Windows file lock becomes an actionable
  "stop the daemon / remove .codegraph" error.
- cli: `codegraph index` now calls recreate() instead of open()+clear();
  both clear() calls dropped. The public clear() API is unchanged.

This also reclaims the disk the bloated db/-wal were holding.

Validated: deterministic probe (DELETE O(rows) vs recreate O(1)); an
end-to-end run through the built binary recovering a real 800K-node /
419MB poisoned DB in 0.3s with no wedge and the correct small graph; new
unit + CLI regression tests; existing #874 index tests still green.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Colby Mchenry
2026-06-29 23:15:41 -05:00
committed by GitHub
co-authored by Claude Opus 4.8
parent 7e3da77f21
commit 9684b3b5a5
6 changed files with 291 additions and 13 deletions
+11 -11
View File
@@ -632,16 +632,23 @@ program
}
const { default: CodeGraph } = await loadCodeGraph();
const cg = await CodeGraph.open(projectPath);
// `index` is a FULL re-index — identical to a fresh `init`. RECREATE the
// database from scratch (discard .codegraph/codegraph.db + its WAL) rather
// than opening the old graph and DELETE-ing every row. The clear-then-index
// approach reported "0 nodes" without the clear (#874); the recreate keeps
// that fixed AND avoids the failure mode where, on a large or pre-fix
// poisoned index, the per-row FTS delete churn wedged the main thread long
// enough to trip the liveness watchdog before scanning even began (#1067).
// recreate() hands back a fresh, empty instance — no clear() needed. For
// fast incremental updates use `sync`.
const cg = await CodeGraph.recreate(projectPath);
// Supervise the indexer: self-terminate if orphaned (parent shim killed)
// or if the main thread wedges — neither was guarded on this path (#999).
const supervision = installCommandSupervision('index');
try {
if (options.quiet) {
// Quiet mode: no UI, just run. `index` is a full re-index, so clear the
// existing graph and rebuild from scratch (see the note below — #874).
cg.clear();
// Quiet mode: no UI, just run against the freshly-recreated graph.
const result = await cg.indexAll();
if (!result.success) process.exit(1);
cg.destroy();
@@ -651,13 +658,6 @@ program
const clack = await importESM('@clack/prompts');
clack.intro('Indexing project');
// `index` is a FULL re-index: clear the existing graph and rebuild it from
// scratch so the result is identical to a fresh `init`. Without the clear,
// indexAll() skips every unchanged file by its content hash and reports
// "0 nodes, 0 edges" against the already-populated graph — which reads as
// "index wiped my index" (#874). For fast incremental updates use `sync`.
cg.clear();
let result: IndexResult;
if (options.verbose) {