perf(resolution): defer checkpoints and double-buffer persist during resolution, byte-identical graphs (#1320)
Fresh init on a 4,402-file Java repo (dubbo): 18.2s -> 13.5s (-26%), with the resolution phase going 12.6s -> 7.9s (-38%). Graphs verified byte-identical on both the pool path (dubbo) and the sequential path (excalidraw). Two changes: 1. The fastInit+pool path restored WAL for the resolver workers but left wal_autocheckpoint at its default, so the persist loop inline-checkpointed hot pages all phase long (#1231's pathology inside resolution — measured at 58% of resolution wall). Checkpointing is now deferred behind the bounded valve and folded once at maintenance, mirroring the deferWal path. 2. The resolution loop is double-buffered: batch k+1 is prefetched (OFFSET past batch k's still-pending rows, under an explicit ORDER BY rowid) and fanned out across the pool while batch k's ref cleanup runs on the main thread. Batch settle-waits dropped 2572ms -> 117ms. Correctness invariant found by the byte-identical gate and now documented in the loop: batch k+1's resolution READS batch k's edges (resolveMethodOnType walks supertype chains over extends/implements edges that resolution itself inserts), so edges must persist BEFORE the next batch fans out; only the ref cleanup overlaps. Also extends the CODEGRAPH_SYNTH_TIMINGS instrumentation with phase labels (grammar-init, parse-loop, fts-rebuild, resolver-reinit, resolution, callback-synthesis) and pool timings (worker open/resolve, per-batch mode, persist), so the next profile is one env var away. Suite green (2444). Sync path timings unchanged. Pool floor re-validated: forced-on at 40k refs is still net-slower, so the 150k threshold stands. Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Fable 5
parent
1de7e8f8b5
commit
a2f3c31a97
@@ -1520,7 +1520,9 @@ export class ExtractionOrchestrator {
|
||||
// in file order preserves the #1015 determinism exactly.
|
||||
storeWriterOpts?: { dbPath: string; fastInit: boolean } | null
|
||||
): Promise<IndexResult> {
|
||||
const tGrammar = Date.now();
|
||||
await initGrammars();
|
||||
if (process.env.CODEGRAPH_SYNTH_TIMINGS) console.error(`[phase-timing] grammar-init: ${Date.now() - tGrammar}ms`);
|
||||
const startTime = Date.now();
|
||||
const errors: ExtractionError[] = [];
|
||||
let filesIndexed = 0;
|
||||
@@ -1816,6 +1818,7 @@ export class ExtractionOrchestrator {
|
||||
}
|
||||
};
|
||||
|
||||
const tParseLoop = Date.now();
|
||||
for (let i = 0; i < files.length; i += FILE_IO_BATCH_SIZE) {
|
||||
if (signal?.aborted) { aborted = true; break; }
|
||||
|
||||
@@ -1907,6 +1910,7 @@ export class ExtractionOrchestrator {
|
||||
}
|
||||
}
|
||||
}
|
||||
if (process.env.CODEGRAPH_SYNTH_TIMINGS) console.error(`[phase-timing] parse-loop: ${Date.now() - tParseLoop}ms`);
|
||||
|
||||
if (signal?.aborted || aborted) {
|
||||
if (storeWriter) await storeWriter.close();
|
||||
|
||||
Reference in New Issue
Block a user