fix(indexing): HDD-class storage — false parse timeouts, dropped files, and WAL checkpoint write-back (#1231) (#1242)

Parse timeouts are now judged by the worker's own clock: the base timer
only marks a job late (after a long synchronous store stall, Node runs the
timers phase before the poll phase, so the timer fired before an
already-delivered result was processed — killing workers over parses that
took milliseconds, even on 0-byte files); a result arriving before a 3×
hard-kill backstop is accepted, timed-out files are retried, and
CODEGRAPH_PARSE_TIMEOUT_MS overrides the budget. Grammar WASM bytes are
pre-read once on the main thread and handed to every worker, so
spawns/respawns load grammars from memory instead of re-reading a
saturated disk.

Bulk indexing defers WAL auto-checkpointing for the whole run: the default
1000-page interval re-writes hot B-tree/FTS pages into the main DB file
over and over — ~95% of all disk I/O under throttled measurement. A
WalCheckpointValve bounds WAL growth with off-thread PASSIVE backfill
passes (never blocking the writer or the #850 watchdog heartbeat), pauses
the writer for a full backfill if the disk truly can't keep up, and folds
the WAL at the parse→resolution boundary so post-parse reads never page a
bulk-write-sized WAL. Opt out with CODEGRAPH_NO_WAL_DEFER=1; tune with
CODEGRAPH_WAL_VALVE_MB.

Measured at 150 IOPS (HDD class): commons-lang 1526s → 59s with 0 dropped
files (was 8); guava-scale completes in 7.6 min with a full graph where
v1.3.1 needed 25 min for a repo 5× smaller. Unthrottled: no change.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

src/extraction/parse-worker.ts

@@ -55,12 +55,18 @@ import type { Language, ExtractionResult } from '../types';
 const PARSER_RESET_INTERVAL = 5000;
 const parseCounts = new Map<Language, number>();
 
-parentPort!.on('message', async (msg: { type: string; id?: number; filePath?: string; content?: string; languages?: Language[]; frameworkNames?: string[]; language?: Language }) => {
+parentPort!.on('message', async (msg: { type: string; id?: number; filePath?: string; content?: string; languages?: Language[]; frameworkNames?: string[]; language?: Language; grammarBuffers?: Record<string, Uint8Array> }) => {
   if (msg.type === 'load-grammars') {
-    await loadGrammarsForLanguages(msg.languages!);
+    // Grammar WASM bytes pre-read by the main thread (when provided) make this
+    // a memory load instead of a per-spawn disk read — see issue #1231.
+    await loadGrammarsForLanguages(msg.languages!, msg.grammarBuffers);
     parentPort!.postMessage({ type: 'grammars-loaded' });
   } else if (msg.type === 'parse') {
     const { id, filePath, content, frameworkNames } = msg;
+    // Worker-side parse clock: reported back with the result so the pool can
+    // tell a genuinely slow parse from a result whose delivery was delayed by
+    // a stalled main thread (issue #1231 false timeouts).
+    const t0 = performance.now();
     try {
       // The main thread resolves the language (it holds the project's
       // codegraph.json extension overrides) and sends it; fall back to detection
@@ -75,7 +81,7 @@ parentPort!.on('message', async (msg: { type: string; id?: number; filePath?: st
         resetParser(language);
       }
 
-      parentPort!.postMessage({ type: 'parse-result', id, result });
+      parentPort!.postMessage({ type: 'parse-result', id, result, parseMs: performance.now() - t0 });
     } catch (err) {
       const message = err instanceof Error ? err.message : String(err);
 
@@ -89,6 +95,7 @@ parentPort!.on('message', async (msg: { type: string; id?: number; filePath?: st
       parentPort!.postMessage({
         type: 'parse-result',
         id,
+        parseMs: performance.now() - t0,
         result: {
           nodes: [],
           edges: [],