fix(db,resolution): WAL file cap + cgroup cache credit + pool/parse sizing corrections from the instrumented kernel-scale runs (#1335)

Four §7a.1 instrumented-run findings, each measured:

1. File-size trigger + truncate-at-barrier: a fully-backfilled WAL still
   grows the FILE without bound — the writer only restarts at frame 0 when a
   commit finds zero reader marks, which the instrumented run showed never
   happens (file marched 361→721MB through two COMPLETE backfills; 22GB by
   phase end). backpressure() now also trips at 4× the soft cap on raw file
   size and TRUNCATEs at the parked barrier; the timer path truncates
   opportunistically after complete backfills. Dubbo peak: 251MB → 69MB at
   the same 16MB valve; dumps byte-identical under aggressive folding.

2. cgroup memory credit: memory.current counts reclaimable page cache — a
   post-parse container read 57MB of headroom on a 6GB box and silently
   disabled the pool. inactive_file is credited back (the docker-stats
   working-set convention); the same run now reads a sane 4.4GB budget.

3. Pool at 2 cores reversed: sequential resolution measured FASTER than
   pooled-6-on-2 at kernel scale (853s vs 1,150s), and synthesis is
   Amdahl-bound by cFnPtrEdges (306s of 358s) so pooling it bought nothing.
   cpuCap = min(ap−1, 6), no floor: ap=2 → sequential is the fast path.

4. Parse floor of 2: one parse worker at a 2-cpuset measured 34% slower
   (493s vs 369s) — main + store-worker don't fill the second core. Floor
   restores the baseline (373.5s measured).

Plus the observability §7a.1 burned three 25-minute cycles for: valve
armed/fire/timer-pass/heartbeat lines, checkpoint-worker error capture,
pool sizing decisions (incl. the disabled path), backpressure-hook
presence — all behind CODEGRAPH_SYNTH_TIMINGS / CODEGRAPH_WAL_VALVE_DEBUG.

Suite: 2,490 passed / 4 skipped (kernel required). Kernel-scale record
runs with this build follow in the migration plan §7a.1.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Colby Mchenry
2026-07-17 08:56:26 -05:00
committed by GitHub
co-authored by Claude Fable 5
parent 8c1e821495
commit ca88d3bd15
8 changed files with 147 additions and 26 deletions
+4
View File
@@ -1358,6 +1358,10 @@ export class ReferenceResolver {
// window to fire; see ./cooperative-yield.
const maybeYield = createYielder();
if (process.env.CODEGRAPH_SYNTH_TIMINGS) {
console.error(`[pool-timing] backpressure hook: ${parallel?.backpressure ? 'present' : 'absent'}`);
}
await this.warmCachesYielding(maybeYield);
const total = this.queries.getUnresolvedReferencesCount();
+21 -2
View File
@@ -24,9 +24,26 @@ function readCgroupBytes(path: string): number | null {
}
}
/** `inactive_file` from a cgroup memory.stat file — reclaimable page cache. */
function readInactiveFile(statPath: string): number {
try {
const m = /^inactive_file (\d+)$/m.exec(fs.readFileSync(statPath, 'utf8'));
return m ? Number.parseInt(m[1]!, 10) : 0;
} catch {
return 0;
}
}
/**
* Available headroom under the cgroup memory limit (v2 then v1), or null
* when uncontained (no limit, non-Linux, or unreadable). Never throws.
*
* Reclaimable page cache (`inactive_file`) is credited back: `memory.current`
* counts it as usage, but the kernel reclaims it on demand — after a bulk
* parse the cache is stuffed with the DB's own pages, and the naive
* `max current` read 57MB of headroom on a 6GB container and silently
* disabled the resolver pool (§7a.1 diagnostic run). This is the same
* working-set convention `docker stats` uses.
*/
export function cgroupMemoryAvailable(): number | null {
if (process.platform !== 'linux') return null;
@@ -34,7 +51,8 @@ export function cgroupMemoryAvailable(): number | null {
const v2Max = readCgroupBytes('/sys/fs/cgroup/memory.max');
if (v2Max !== null) {
const current = readCgroupBytes('/sys/fs/cgroup/memory.current') ?? 0;
return Math.max(0, v2Max - current);
const reclaimable = readInactiveFile('/sys/fs/cgroup/memory.stat');
return Math.max(0, v2Max - Math.max(0, current - reclaimable));
}
// v1
const v1Limit = readCgroupBytes('/sys/fs/cgroup/memory/memory.limit_in_bytes');
@@ -42,7 +60,8 @@ export function cgroupMemoryAvailable(): number | null {
// anything at or beyond half the address-space-ish range as uncontained.
if (v1Limit !== null && v1Limit < 2 ** 60) {
const usage = readCgroupBytes('/sys/fs/cgroup/memory/memory.usage_in_bytes') ?? 0;
return Math.max(0, v1Limit - usage);
const reclaimable = readInactiveFile('/sys/fs/cgroup/memory/memory.stat');
return Math.max(0, v1Limit - Math.max(0, usage - reclaimable));
}
return null;
}
+14 -5
View File
@@ -95,7 +95,12 @@ export class ResolverPool {
return Math.min(n, 16);
}
}
const cpuCap = Math.max(2, Math.min(opts.availableParallelism - 1, 6));
// No floor: at ap=2 the pool LOSES to sequential outright — measured on
// the kernel-scale 2-cpuset envelope: resolution 853s sequential vs
// 1,150s pooled-6-on-2 (§7a.1), and synthesis is Amdahl-bound by its
// dominant pass (cFnPtrEdges 306s of 358s) so pooling it bought nothing.
// ap1 < 2 ⇒ sequential is the fast path, not a fallback.
const cpuCap = Math.min(opts.availableParallelism - 1, 6);
const perWorker = Math.min(Math.max(opts.dbSizeBytes * 0.2, 256 * 1024 * 1024), 1.5 * 1024 * 1024 * 1024);
const memCap = Math.floor((opts.memoryBudget * 0.7) / perWorker);
const size = Math.min(cpuCap, memCap);
@@ -117,18 +122,22 @@ export class ResolverPool {
try {
dbSizeBytes = fs.statSync(dbPath).size;
} catch { /* fresh/missing file — the 256MB per-worker floor applies */ }
const ap = os.availableParallelism();
const budget = memoryBudgetBytes();
const size = ResolverPool.resolvePoolSize({
explicit: process.env.CODEGRAPH_RESOLVE_WORKERS,
availableParallelism: os.availableParallelism(),
memoryBudget: memoryBudgetBytes(),
availableParallelism: ap,
memoryBudget: budget,
dbSizeBytes,
});
if (size === null) return null;
// Both outcomes log under SYNTH_TIMINGS — a silent null is how §7a.1's
// diagnostic run hid the memory-term misfire for a whole 25-minute cycle.
if (process.env.CODEGRAPH_SYNTH_TIMINGS) {
console.error(
`[pool-timing] pool size=${size} (ap=${os.availableParallelism()} budget=${Math.round(memoryBudgetBytes() / 1024 / 1024)}MB db=${Math.round(dbSizeBytes / 1024 / 1024)}MB)`
`[pool-timing] pool ${size === null ? 'disabled' : `size=${size}`} (ap=${ap} budget=${Math.round(budget / 1024 / 1024)}MB db=${Math.round(dbSizeBytes / 1024 / 1024)}MB)`
);
}
if (size === null) return null;
try {
return new ResolverPool(workerScript, dbPath, projectRoot, size);
} catch {