fix(db,resolution): WAL file cap + cgroup cache credit + pool/parse sizing corrections from the instrumented kernel-scale runs (#1335)

Four §7a.1 instrumented-run findings, each measured:

1. File-size trigger + truncate-at-barrier: a fully-backfilled WAL still
   grows the FILE without bound — the writer only restarts at frame 0 when a
   commit finds zero reader marks, which the instrumented run showed never
   happens (file marched 361→721MB through two COMPLETE backfills; 22GB by
   phase end). backpressure() now also trips at 4× the soft cap on raw file
   size and TRUNCATEs at the parked barrier; the timer path truncates
   opportunistically after complete backfills. Dubbo peak: 251MB → 69MB at
   the same 16MB valve; dumps byte-identical under aggressive folding.

2. cgroup memory credit: memory.current counts reclaimable page cache — a
   post-parse container read 57MB of headroom on a 6GB box and silently
   disabled the pool. inactive_file is credited back (the docker-stats
   working-set convention); the same run now reads a sane 4.4GB budget.

3. Pool at 2 cores reversed: sequential resolution measured FASTER than
   pooled-6-on-2 at kernel scale (853s vs 1,150s), and synthesis is
   Amdahl-bound by cFnPtrEdges (306s of 358s) so pooling it bought nothing.
   cpuCap = min(ap−1, 6), no floor: ap=2 → sequential is the fast path.

4. Parse floor of 2: one parse worker at a 2-cpuset measured 34% slower
   (493s vs 369s) — main + store-worker don't fill the second core. Floor
   restores the baseline (373.5s measured).

Plus the observability §7a.1 burned three 25-minute cycles for: valve
armed/fire/timer-pass/heartbeat lines, checkpoint-worker error capture,
pool sizing decisions (incl. the disabled path), backpressure-hook
presence — all behind CODEGRAPH_SYNTH_TIMINGS / CODEGRAPH_WAL_VALVE_DEBUG.

Suite: 2,490 passed / 4 skipped (kernel required). Kernel-scale record
runs with this build follow in the migration plan §7a.1.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Colby Mchenry
2026-07-17 08:56:26 -05:00
committed by GitHub
co-authored by Claude Fable 5
parent 8c1e821495
commit ca88d3bd15
8 changed files with 147 additions and 26 deletions
+3 -2
View File
@@ -28,8 +28,9 @@ describe('ResolverPool.resolvePoolSize', () => {
expect(size({ availableParallelism: 11 })).toBe(6);
});
it('true 2-core box keeps a 2-worker pool (the pooled-synthesis 2×)', () => {
expect(size({ availableParallelism: 2, memoryBudget: 6 * GB })).toBe(2);
it('true 2-core box gets NO pool — sequential measured faster there (§7a.1: 853s vs 1150s)', () => {
expect(size({ availableParallelism: 2, memoryBudget: 6 * GB })).toBeNull();
expect(size({ availableParallelism: 3, memoryBudget: 6 * GB })).toBe(2);
});
it('kernel-scale DB in a 7GB container: memory term shrinks the pool below the OOM line', () => {
+51 -9
View File
@@ -103,21 +103,26 @@ describe('WalCheckpointValve', () => {
db.close();
});
it('advances its baseline on a full backfill — a wrapped WAL does not retrigger it', async () => {
it('advances its baseline on a full backfill — no infinite retrigger (at most one truncate park)', async () => {
// Pre-§7a.1 contract was "a wrapped WAL never retriggers"; the file-size
// trigger deliberately weakens that to "retriggers AT MOST once more, to
// truncate the file, then goes quiet" — the pre-fix bug this test pinned
// (firing on raw size forever, serializing every store) stays dead: a
// successful truncate zeroes the file, so the trigger cannot loop. At
// this test's pathological 10-BYTE soft cap, byte-level residue can trip
// the 4×-soft file cap once; product-scale caps are 256MB/1GB.
const db = openDb();
db.setWalAutocheckpoint(0);
writeRows(db, 500);
const valve = new WalCheckpointValve(db, 0.00001);
valve.check();
await valve.drain(); // full backfill on an idle DB → baseline = current file size
// The WAL file keeps its high-water size, but growth is now 0: neither
// the timer path nor backpressure may fire again (the pre-fix bug fired
// on raw size forever and serialized every store behind a checkpoint).
expect(valve.backpressure()).toBeNull();
await valve.drain(); // full backfill (and possibly a timer truncate)
const first = valve.backpressure();
if (first) await first; // one truncate park allowed — file must be 0 after
expect(db.getWalSizeBytes()).toBe(0);
expect(valve.backpressure()).toBeNull(); // and now: quiet
valve.check();
await valve.drain(); // no-op drain: nothing in flight
// New commits recycle wrapped frames — file size is flat, still no trigger.
writeRows(db, 5);
await valve.drain();
expect(valve.backpressure()).toBeNull();
db.close();
});
@@ -374,3 +379,40 @@ describe('checkpointWalTruncate (§7a.1 file containment)', () => {
db.close();
});
});
describe('valve file-size trigger (§7a.1: backfilled WAL still grows the file)', () => {
it('backpressure trips on file size alone once past the file cap, even with zero backlog', async () => {
const db = openDb();
db.setWalAutocheckpoint(0);
// Grow the file well past a 0.5MB soft cap (file cap = 4× = 2MB), then
// fold the backlog completely so growth-vs-baseline is ~zero.
writeRows(db, 800);
const valve = new WalCheckpointValve(db, 0.5);
await valve.foldNow(); // baseline := file size; backlog now 0; file unchanged
expect(db.getWalSizeBytes()).toBe(0); // foldNow's success path truncates at the barrier
db.close();
});
it('a fully-backfilled but oversized file is chopped at the barrier', async () => {
const db = openDb();
db.setWalAutocheckpoint(0);
writeRows(db, 800);
const before = db.getWalSizeBytes();
expect(before).toBeGreaterThan(2 * 1024 * 1024);
const valve = new WalCheckpointValve(db, 0.5);
const bp = valve.backpressure(); // growth past hard cap → parks
expect(bp).not.toBeNull();
await bp;
expect(db.getWalSizeBytes()).toBe(0); // truncated at the parked barrier
// And the file-size trigger alone re-arms it after regrowth:
writeRows(db, 800);
await valve.foldNow();
writeRows(db, 100); // small backlog, file grows again but under hard cap
const sizeTrigger = valve.backpressure();
// 100 rows ≈ <1MB backlog (under 1MB hard cap) but file is past the 2MB cap
expect(sizeTrigger).not.toBeNull();
await sizeTrigger;
expect(db.getWalSizeBytes()).toBe(0);
db.close();
});
});