fix(db,resolution): WAL file cap + cgroup cache credit + pool/parse sizing corrections from the instrumented kernel-scale runs (#1335)
Four §7a.1 instrumented-run findings, each measured: 1. File-size trigger + truncate-at-barrier: a fully-backfilled WAL still grows the FILE without bound — the writer only restarts at frame 0 when a commit finds zero reader marks, which the instrumented run showed never happens (file marched 361→721MB through two COMPLETE backfills; 22GB by phase end). backpressure() now also trips at 4× the soft cap on raw file size and TRUNCATEs at the parked barrier; the timer path truncates opportunistically after complete backfills. Dubbo peak: 251MB → 69MB at the same 16MB valve; dumps byte-identical under aggressive folding. 2. cgroup memory credit: memory.current counts reclaimable page cache — a post-parse container read 57MB of headroom on a 6GB box and silently disabled the pool. inactive_file is credited back (the docker-stats working-set convention); the same run now reads a sane 4.4GB budget. 3. Pool at 2 cores reversed: sequential resolution measured FASTER than pooled-6-on-2 at kernel scale (853s vs 1,150s), and synthesis is Amdahl-bound by cFnPtrEdges (306s of 358s) so pooling it bought nothing. cpuCap = min(ap−1, 6), no floor: ap=2 → sequential is the fast path. 4. Parse floor of 2: one parse worker at a 2-cpuset measured 34% slower (493s vs 369s) — main + store-worker don't fill the second core. Floor restores the baseline (373.5s measured). Plus the observability §7a.1 burned three 25-minute cycles for: valve armed/fire/timer-pass/heartbeat lines, checkpoint-worker error capture, pool sizing decisions (incl. the disabled path), backpressure-hook presence — all behind CODEGRAPH_SYNTH_TIMINGS / CODEGRAPH_WAL_VALVE_DEBUG. Suite: 2,490 passed / 4 skipped (kernel required). Kernel-scale record runs with this build follow in the migration plan §7a.1. Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Fable 5
parent
8c1e821495
commit
ca88d3bd15
@@ -24,9 +24,26 @@ function readCgroupBytes(path: string): number | null {
|
||||
}
|
||||
}
|
||||
|
||||
/** `inactive_file` from a cgroup memory.stat file — reclaimable page cache. */
|
||||
function readInactiveFile(statPath: string): number {
|
||||
try {
|
||||
const m = /^inactive_file (\d+)$/m.exec(fs.readFileSync(statPath, 'utf8'));
|
||||
return m ? Number.parseInt(m[1]!, 10) : 0;
|
||||
} catch {
|
||||
return 0;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Available headroom under the cgroup memory limit (v2 then v1), or null
|
||||
* when uncontained (no limit, non-Linux, or unreadable). Never throws.
|
||||
*
|
||||
* Reclaimable page cache (`inactive_file`) is credited back: `memory.current`
|
||||
* counts it as usage, but the kernel reclaims it on demand — after a bulk
|
||||
* parse the cache is stuffed with the DB's own pages, and the naive
|
||||
* `max − current` read 57MB of headroom on a 6GB container and silently
|
||||
* disabled the resolver pool (§7a.1 diagnostic run). This is the same
|
||||
* working-set convention `docker stats` uses.
|
||||
*/
|
||||
export function cgroupMemoryAvailable(): number | null {
|
||||
if (process.platform !== 'linux') return null;
|
||||
@@ -34,7 +51,8 @@ export function cgroupMemoryAvailable(): number | null {
|
||||
const v2Max = readCgroupBytes('/sys/fs/cgroup/memory.max');
|
||||
if (v2Max !== null) {
|
||||
const current = readCgroupBytes('/sys/fs/cgroup/memory.current') ?? 0;
|
||||
return Math.max(0, v2Max - current);
|
||||
const reclaimable = readInactiveFile('/sys/fs/cgroup/memory.stat');
|
||||
return Math.max(0, v2Max - Math.max(0, current - reclaimable));
|
||||
}
|
||||
// v1
|
||||
const v1Limit = readCgroupBytes('/sys/fs/cgroup/memory/memory.limit_in_bytes');
|
||||
@@ -42,7 +60,8 @@ export function cgroupMemoryAvailable(): number | null {
|
||||
// anything at or beyond half the address-space-ish range as uncontained.
|
||||
if (v1Limit !== null && v1Limit < 2 ** 60) {
|
||||
const usage = readCgroupBytes('/sys/fs/cgroup/memory/memory.usage_in_bytes') ?? 0;
|
||||
return Math.max(0, v1Limit - usage);
|
||||
const reclaimable = readInactiveFile('/sys/fs/cgroup/memory/memory.stat');
|
||||
return Math.max(0, v1Limit - Math.max(0, usage - reclaimable));
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user