Follow-up to #1065/#1066. Those stopped a *new* index from scanning an ignored gitlink corpus, but a project that had already built the multi-GB graph before upgrading still couldn't recover: `codegraph index` printed only "Indexing project" and was then SIGKILLed (137) by the #850 watchdog ~60s later, before scanning even started. Root cause is not the scanner. `index` cleared the old graph with a synchronous `DELETE FROM nodes/edges/files`. `nodes` carries an FTS5 `AFTER DELETE` trigger, so deleting ~1.6M rows fires ~1.6M FTS delete-markers — O(rows), and it grows the WAL further before it can finish. A deterministic probe puts the DELETE-clear at 20.4s on 1.5M synthetic nodes (WAL 1.16->2.14GB); at the report's denser ~2.6KB/node WAL that crosses the 60s main-thread watchdog. `open()` was never the wedge. A full re-index is documented as "same result as a fresh init", so make it one: discard the database files and re-initialize, instead of opening the old DB and DELETE-ing every row. - db: add removeDatabaseFiles(dbPath) — unlinks codegraph.db + its -wal/-shm sidecars (O(1) regardless of size; sidecars best-effort). - index: add CodeGraph.recreate(projectRoot) — discards the files and returns a fresh, empty instance. Never opens or migrates the poisoned DB. POSIX unlinks an open file fine (a live daemon heals via reopenIfReplaced, #925); a Windows file lock becomes an actionable "stop the daemon / remove .codegraph" error. - cli: `codegraph index` now calls recreate() instead of open()+clear(); both clear() calls dropped. The public clear() API is unchanged. This also reclaims the disk the bloated db/-wal were holding. Validated: deterministic probe (DELETE O(rows) vs recreate O(1)); an end-to-end run through the built binary recovering a real 800K-node / 419MB poisoned DB in 0.3s with no wedge and the correct small graph; new unit + CLI regression tests; existing #874 index tests still green. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
7e3da77f21
commit
9684b3b5a5
+11
-11
@@ -632,16 +632,23 @@ program
|
||||
}
|
||||
|
||||
const { default: CodeGraph } = await loadCodeGraph();
|
||||
const cg = await CodeGraph.open(projectPath);
|
||||
// `index` is a FULL re-index — identical to a fresh `init`. RECREATE the
|
||||
// database from scratch (discard .codegraph/codegraph.db + its WAL) rather
|
||||
// than opening the old graph and DELETE-ing every row. The clear-then-index
|
||||
// approach reported "0 nodes" without the clear (#874); the recreate keeps
|
||||
// that fixed AND avoids the failure mode where, on a large or pre-fix
|
||||
// poisoned index, the per-row FTS delete churn wedged the main thread long
|
||||
// enough to trip the liveness watchdog before scanning even began (#1067).
|
||||
// recreate() hands back a fresh, empty instance — no clear() needed. For
|
||||
// fast incremental updates use `sync`.
|
||||
const cg = await CodeGraph.recreate(projectPath);
|
||||
|
||||
// Supervise the indexer: self-terminate if orphaned (parent shim killed)
|
||||
// or if the main thread wedges — neither was guarded on this path (#999).
|
||||
const supervision = installCommandSupervision('index');
|
||||
try {
|
||||
if (options.quiet) {
|
||||
// Quiet mode: no UI, just run. `index` is a full re-index, so clear the
|
||||
// existing graph and rebuild from scratch (see the note below — #874).
|
||||
cg.clear();
|
||||
// Quiet mode: no UI, just run against the freshly-recreated graph.
|
||||
const result = await cg.indexAll();
|
||||
if (!result.success) process.exit(1);
|
||||
cg.destroy();
|
||||
@@ -651,13 +658,6 @@ program
|
||||
const clack = await importESM('@clack/prompts');
|
||||
clack.intro('Indexing project');
|
||||
|
||||
// `index` is a FULL re-index: clear the existing graph and rebuild it from
|
||||
// scratch so the result is identical to a fresh `init`. Without the clear,
|
||||
// indexAll() skips every unchanged file by its content hash and reports
|
||||
// "0 nodes, 0 edges" against the already-populated graph — which reads as
|
||||
// "index wiped my index" (#874). For fast incremental updates use `sync`.
|
||||
cg.clear();
|
||||
|
||||
let result: IndexResult;
|
||||
|
||||
if (options.verbose) {
|
||||
|
||||
Reference in New Issue
Block a user