perf(resolution): worker connection recycling — WAL-depth writes-under-readers fix, superphase −11.4% at 8c (#1362)
The §7a.6 anomaly probed to its mechanism with five discriminating runs (§7a.7 table): main-thread B-tree writes triple under attached readers because READERS PIN WAL checkpoint progress — the deep WAL taxes every writer page operation (deletes 42.6s pool-off vs 118.8s pool-4 on identical hardware; an aggressive 64MB valve recovers the writes but overpays +129s in full-park folds; the v2 cache resurrection was falsified — long-tail name traffic is uncacheable at any capacity). Fix: workers close and reopen their read-only connections every 8 batches at the double-buffer's worker-idle boundary (ResolverPool.recycleWorkers + QueryBuilder.rebind + a cadence call). Reopens are sub-millisecond, resolver caches survive (only prepared statements re-prepare), and the existing checkpoints advance instead of parking. Failed recycle downgrades to sequential, same as a failed fan-out. Measured (8c pool-4, linux v7.2-rc2, cadence 25 → 8 iterated): resolution superphase 715.0 → 633.6s (−11.4%), envelope best 14.8min, recreate 59.7 → 45.3s. Byte-neutral everywhere: git dumps byte-identical old-vs-new, linux dump sha 6dd1185b reproduced (10,446,478 lines), counts 2,049,153/6,413,518, suite 2517 green. 2c unchanged by construction (no pool → no recycling). Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Fable 5
parent
5955d04c97
commit
971a5a0483
+34
-1
@@ -1408,7 +1408,7 @@ export class ReferenceResolver {
|
||||
// these counters name where the other ~340s goes (reads, edge build+insert,
|
||||
// deletes/marks, the per-batch count guard).
|
||||
const loopProf: Record<string, number> | null = process.env.CODEGRAPH_RESOLVE_PROFILE
|
||||
? { read: 0, settle: 0, backpressure: 0, createEdges: 0, insertEdges: 0, deletes: 0, marks: 0, countGuard: 0 }
|
||||
? { read: 0, settle: 0, backpressure: 0, recycle: 0, createEdges: 0, insertEdges: 0, deletes: 0, marks: 0, countGuard: 0 }
|
||||
: null;
|
||||
const lp = (k: string, t0: number): void => { if (loopProf) loopProf[k] = (loopProf[k] ?? 0) + (Date.now() - t0); };
|
||||
let tLp = 0;
|
||||
@@ -1450,6 +1450,14 @@ export class ReferenceResolver {
|
||||
// pending rows forward.
|
||||
let prevRemaining = Number.POSITIVE_INFINITY;
|
||||
|
||||
// Cadence for the worker connection recycling below — ~8 batches
|
||||
// ≈ 40k refs between recycles keeps the WAL shallow at kernel scale
|
||||
// while a small sync never recycles at all. (25 recovered only half
|
||||
// the write tax — the WAL re-deepened between recycles; reopens are
|
||||
// sub-millisecond so the shorter cadence is ~free.)
|
||||
const RECYCLE_EVERY_BATCHES = 8;
|
||||
let batchesSinceRecycle = 0;
|
||||
|
||||
// Fan-out result of ResolverPool.resolveBatch, settled (never rejecting)
|
||||
// so a fan-out begun before the previous batch's persist can't produce an
|
||||
// unhandled rejection while it waits to be awaited.
|
||||
@@ -1557,6 +1565,31 @@ export class ReferenceResolver {
|
||||
if (bp) await bp;
|
||||
lp('backpressure', tLp);
|
||||
|
||||
// Recycle the workers' read connections periodically at this same
|
||||
// worker-idle boundary (batch k settled, batch k+1 not yet fanned
|
||||
// out): a long-lived reader pins WAL checkpoint progress, and the
|
||||
// deep WAL that accumulates behind it taxes the writer's OWN page
|
||||
// operations — the §7a.6 writes-under-readers finding (deletes
|
||||
// 42.6s → 118.8s from 0 to 4 attached readers; an aggressive valve
|
||||
// recovered the writes but paid +129s in full-park folds). Releasing
|
||||
// the read marks every ~25 batches lets the existing checkpoints
|
||||
// advance instead, at ~milliseconds of reopen cost. A failed recycle
|
||||
// downgrades to sequential permanently, same as a failed fan-out.
|
||||
if (pool && poolReady && ++batchesSinceRecycle >= RECYCLE_EVERY_BATCHES) {
|
||||
batchesSinceRecycle = 0;
|
||||
tLp = Date.now();
|
||||
try {
|
||||
await pool.recycleWorkers();
|
||||
} catch (err) {
|
||||
logDebug('Worker connection recycle failed; falling back to sequential', {
|
||||
error: err instanceof Error ? err.message : String(err),
|
||||
});
|
||||
await pool.destroy().catch(() => undefined);
|
||||
pool = null;
|
||||
}
|
||||
lp('recycle', tLp);
|
||||
}
|
||||
|
||||
// Persist in bounded sub-transactions with yields between: a whole
|
||||
// batch's edge insert / keyed deletes are otherwise one solid
|
||||
// synchronous span each on a multi-GB index, sitting BETWEEN the
|
||||
|
||||
Reference in New Issue
Block a user