perf(resolution): worker connection recycling — WAL-depth writes-under-readers fix, superphase −11.4% at 8c (#1362)

The §7a.6 anomaly probed to its mechanism with five discriminating runs
(§7a.7 table): main-thread B-tree writes triple under attached readers
because READERS PIN WAL checkpoint progress — the deep WAL taxes every
writer page operation (deletes 42.6s pool-off vs 118.8s pool-4 on
identical hardware; an aggressive 64MB valve recovers the writes but
overpays +129s in full-park folds; the v2 cache resurrection was
falsified — long-tail name traffic is uncacheable at any capacity).

Fix: workers close and reopen their read-only connections every 8
batches at the double-buffer's worker-idle boundary
(ResolverPool.recycleWorkers + QueryBuilder.rebind + a cadence call).
Reopens are sub-millisecond, resolver caches survive (only prepared
statements re-prepare), and the existing checkpoints advance instead of
parking. Failed recycle downgrades to sequential, same as a failed
fan-out.

Measured (8c pool-4, linux v7.2-rc2, cadence 25 → 8 iterated):
resolution superphase 715.0 → 633.6s (−11.4%), envelope best 14.8min,
recreate 59.7 → 45.3s. Byte-neutral everywhere: git dumps byte-identical
old-vs-new, linux dump sha 6dd1185b reproduced (10,446,478 lines),
counts 2,049,153/6,413,518, suite 2517 green. 2c unchanged by
construction (no pool → no recycling).

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Colby Mchenry
2026-07-18 16:58:03 -05:00
committed by GitHub
co-authored by Claude Fable 5
parent 5955d04c97
commit 971a5a0483
6 changed files with 180 additions and 5 deletions
+58 -4
View File
@@ -90,10 +90,15 @@ them are the ORIGINAL plan and carry expectations that measurement later correct
double-buffer WORKS (8c settle 3.6s — "core-invariant" superseded);
2c record now 16.5min, 8c range 15.016.4 (n=2). Two cache
experiments killed by measurement same-day (nameCache scaling, lazy
candidates — §7a.6 has the numbers; code reverted). Queue now:
**writes-under-readers probe** (8c deletes+inserts +102s under the
pool — the biggest attributed delta) > **cFnPtr native site
extraction** (synthesis ~230s) > backpressure byte volume > recreate.
candidates — §7a.6 has the numbers; code reverted).
Writes-under-readers PROBED + FIXED 2026-07-18 (§7a.7): mechanism =
WAL read-through depth under reader pins (proven by pool-off/dose/
valve discrimination); fix = worker connection recycling at the
pool-idle boundary (cadence 8); superphase **715 → 633.6s (11.4%)**,
8c envelope best **14.8min**, byte-neutral at every gate. Queue now:
**cFnPtr native site extraction** (synthesis ~230s) >
continuous-shallow WAL (the remaining ~45s to the valve floor) >
backpressure byte volume > recreate.
- [x] **R7a. C/C++ port** — DONE 2026-07-17, same-day walker+gates after the
survey (#1344) and grammar vendoring (#1345). One dual-language walker
(`codegraph-kernel/src/ccpp/`), preParse HOISTED to the route point
@@ -850,6 +855,55 @@ morning's run; report ranges, never single runs on this box.** 8c parse
(~80s paid in-container that a 24GB target-class box gets back free) —
container numbers UNDERSTATE the true 8-core-class target.
#### 7a.7 Writes-under-readers probe + fix (2026-07-18) — WAL depth named, worker connection recycling shipped
Five discriminating runs (all 8c pool-4 unless noted, same tree/build family;
each ~16min), then the fix in two cadence iterations:
| run | deletes | insertEdges | read | backpressure | settle | superphase |
|---|---|---|---|---|---|---|
| pool-4 baseline | 118.8 | 55.3 | 33.9 | 121.9 | 3.6 | 715.0 |
| pool-OFF | 42.6 | 38.3 | 32.6 | 120.1 | 108.5 (main) | 685.9 |
| workers=2 (dose) | 58.9 | 54.0 | 23.9 | 174.7 | 10.3 | 670.0 |
| v2 caches @8c | 108.0 | 50.3 | 31.4 | 168.0 | 3.9 | 687.3 |
| valve 64MB | **56.5** | **27.8** | **16.8** | 297.2 | 3.8 | 739.2 |
| **recycle c25** | 99.8 | 42.9 | 32.7 | 148.8 | 3.7 | 663.3 |
| **recycle c8 (SHIPPED)** | 104.3 | 47.8 | 32.5 | 127.6 | 4.0 | **633.6** |
- **Mechanism proven: WAL read-through depth under reader pins.** Pool-off
restores writes on identical hardware (deletes 118.8 → 42.6); the dose
scales with reader count (knee between 2 and 4); the aggressive valve —
which forces a shallow WAL — recovers deletes/inserts/read to their floors
(56.5/27.8/16.8) but overpays +129s in full-park folds. Reconciliation:
checkpoints run in every topology, but READERS PIN their progress — the
WAL runs deep exactly when workers are attached, and deep-WAL page
operations tax the writer everywhere (including the unattributed
between-stage spans, recreate, and synthesis reads).
- **v2-at-8c falsified the cache resurrection** (deletes 108.0 ≈ baseline):
name-lookup traffic is long-tail-dominated — uncacheable at any capacity —
so reader traffic can't be reduced by caching. The caching family is
triple-dead (2c settle, 8c writes).
- **The fix: worker connection recycling at the pool-idle boundary**
(`ResolverPool.recycleWorkers` + `QueryBuilder.rebind` + a cadence call at
the double-buffer's worker-idle point). Workers close/reopen their
read-only connections every 8 batches (~40k refs) — reopens are
sub-millisecond, resolver caches survive (only prepared statements
re-prepare), and the existing checkpoints advance instead of parking.
Cadence 25 → 8 iterated by measurement; 8 wins via diffuse gains
(backpressure 21, recreate 59.7 → 45.3). Attributed deletes stay ~100
(the WAL still re-deepens between recycles — the valve's 56.5 floor
needs continuous shallowness), but the SUPERPHASE captures the true win:
**715.0 → 633.6s (11.4%)**; envelope best-of **14.8min at 8c**
(890s; band across the day's runs 14.816.4). Byte-neutral: git dumps
byte-identical old-vs-new, linux dump sha `6dd1185b…` reproduced, counts
2,049,153/6,413,518 every run, suite 2517 green. 2c unaffected by
construction (no pool → no recycling).
- Levers after this round: cFnPtr native site extraction (~230s synthesis,
§7a.6 ranking stands) > continuous-shallow WAL (close the remaining
~45s gap between recycling's ~100s deletes and the valve's 56.5 floor
without full-park folds — e.g. passive-checkpoint nudges at the recycle
boundary) > backpressure byte volume > recreate.
### 7b. Arc 3 — graph richness (forensics-backed; adopt cbm's real extras, skip inflation)
Priority order, each gated by the standard A/B + node-explosion probes:
1. **Test→subject edges** (first-class `tests` edges at index time; we compute covering