perf(store): resolution ref-index window — kernel-scale resolution 423→276s, 8c envelope ≈11min (§4d round 2) (#1369)

Store-architecture arc round 2. The batched resolution loop reads
unresolved_refs ONLY through the status index + the PK keyset pager;
the other five ref indexes (from_node, name, file_path, from_name,
failed_tail) serve sync-time paths — yet every per-batch DELETE of
resolved refs maintained all of them, the biggest single main-thread
stage on the dubbo profile (deletes 1.2s of a 5.4s resolution phase)
and 50-81s at kernel scale.

beginBulkRefLoad/endBulkRefLoad on DatabaseConnection, threaded as
refIndexLoad hooks next to the existing bulkEdgeLoad pair with the
same minRefsForPool gate (small syncs never pay): drop the five for
the loop, rebuild each in one scan at the end — where the table holds
only the surviving FAILED refs (resolved rows are deleted by then),
so the recreate is near-free. Crash inside the window heals on the
next open (schema.sql re-applies CREATE INDEX IF NOT EXISTS).

Measured:
- dubbo: deletes 1.2 → 0.2s, marks 0.6 → 0.3s, recreate 219ms; wall
  ~8.5s flat — the freed main-lane time shifts into settle (the worker
  lane now binds the double-buffer at medium scale).
- Linux kernel 8c: resolution 423.4 → 275.9s (deletes 50-81 → 3.2s,
  backpressure 16.8 → 7.4s — fewer index writes mean less WAL and
  cheaper folds), ref recreate 10.3s. Envelope ≈ 11.0min, from the
  14.8min pre-arc best; <10min-on-8c now needs ~1 more minute.

Gates: dubbo/gson dumps byte-identical; linux counts exact
2,049,153/6,413,518 and dump sha 6dd1185b… reproduced (10,446,478
lines); full suite green ×2 (153 files / 2588 tests).

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

src/db/index.ts

@@ -221,6 +221,50 @@ export class DatabaseConnection {
     await this.endBulkEdgeLoad();
   }
 
+  /**
+   * unresolved_refs secondary indexes NOT read by the batched resolution
+   * loop. The loop pages pending refs by keyset (`status='pending' AND id>?`
+   * — the status index + PK), deletes resolved rows by id, and parks failures
+   * with a status UPDATE; every other ref index serves SYNC-time paths
+   * (per-file re-index deletes, name-keyed retry, failed-tail heal). Each
+   * per-batch DELETE maintains all of them — the biggest single main-thread
+   * stage on the dubbo profile (deletes 1.2s of a 5.4s resolution phase) —
+   * so the batched loop drops them and rebuilds at the end, where the table
+   * holds only the surviving FAILED refs (resolved rows are gone), making
+   * the recreate near-free.
+   */
+  private static readonly BULK_REF_INDEX_NAMES = [
+    'idx_unresolved_from_node',
+    'idx_unresolved_name',
+    'idx_unresolved_file_path',
+    'idx_unresolved_from_name',
+    'idx_unresolved_failed_tail',
+  ] as const;
+
+  /**
+   * Enter bulk-ref mode for the batched resolution loop — see
+   * BULK_REF_INDEX_NAMES. MUST be paired with endBulkRefLoad(); a crash
+   * inside the window heals on the next open (schema.sql re-applies
+   * CREATE INDEX IF NOT EXISTS).
+   */
+  beginBulkRefLoad(): void {
+    for (const idx of DatabaseConnection.BULK_REF_INDEX_NAMES) {
+      this.db.exec(`DROP INDEX IF EXISTS ${idx}`);
+    }
+  }
+
+  /** Leave bulk-ref mode: recreate each index in one scan (yield between). */
+  async endBulkRefLoad(): Promise<void> {
+    const schemaPath = path.join(__dirname, 'schema.sql');
+    const schema = fs.readFileSync(schemaPath, 'utf-8');
+    for (const idx of DatabaseConnection.BULK_REF_INDEX_NAMES) {
+      const m = schema.match(new RegExp(`CREATE INDEX IF NOT EXISTS ${idx}\\b[^;]*;`));
+      if (!m) throw new Error(`schema.sql: ref index ${idx} not found for bulk-load recreation`);
+      this.db.exec(m[0]);
+      await new Promise((resolve) => setImmediate(resolve));
+    }
+  }
+
   /**
    * Names of the NON-UNIQUE edge indexes dropped for a bulk edge load.
    * idx_edges_identity deliberately stays: INSERT OR IGNORE's dedup conflicts

src/index.ts

@@ -1175,6 +1175,10 @@ export class CodeGraph {
         begin: () => this.db.beginBulkEdgeLoad(),
         end: () => this.db.endBulkEdgeLoad(),
       },
+      refIndexLoad: {
+        begin: () => this.db.beginBulkRefLoad(),
+        end: () => this.db.endBulkRefLoad(),
+      },
       backpressure,
     });
   }

src/resolution/index.ts

@@ -1388,6 +1388,10 @@ export class ReferenceResolver {
     parallel?: {
       dbPath: string;
       bulkEdgeLoad?: { begin: () => void; end: () => void | Promise<void> };
+      /** unresolved_refs index window for the batched loop — the loop only
+       *  reads the status index + PK; dropping the sync-path ref indexes cuts
+       *  each per-batch DELETE's B-tree work (DatabaseConnection.beginBulkRefLoad). */
+      refIndexLoad?: { begin: () => void; end: () => void | Promise<void> };
       backpressure?: () => Promise<void> | null;
     }
   ): Promise<ResolutionResult> {
@@ -1532,6 +1536,16 @@ export class ReferenceResolver {
         bulkEdgesActive = true;
       } catch { /* keep the indexes; inserts just pay the per-row maintenance */ }
     }
+    // Same gate for the ref-index window: the loop's deletes stop maintaining
+    // the five sync-path unresolved_refs indexes, and the end-of-loop rebuild
+    // is near-free (only failed refs survive the loop).
+    let bulkRefsActive = false;
+    if (parallel?.refIndexLoad && total >= minRefsForPool()) {
+      try {
+        parallel.refIndexLoad.begin();
+        bulkRefsActive = true;
+      } catch { /* keep the indexes; deletes just pay the per-row maintenance */ }
+    }
 
     try {
     try {
@@ -1717,6 +1731,11 @@ export class ReferenceResolver {
       // Recreate the edge indexes BEFORE synthesis (kind-keyed reads) and on
       // any error path. A crash before this line is healed by the next
       // DatabaseConnection open (schema.sql re-applies IF NOT EXISTS).
+      if (bulkRefsActive) {
+        const tRef = Date.now();
+        await parallel!.refIndexLoad!.end();
+        if (process.env.CODEGRAPH_SYNTH_TIMINGS) console.error(`[phase-timing] ref-index-recreate: ${Date.now() - tRef}ms`);
+      }
       if (bulkEdgesActive) {
         const tIdx = Date.now();
         await parallel!.bulkEdgeLoad!.end();