perf(resolution): darwin-honest memory budget — vm_stat-based availability unstrangles the resolver pool on macOS (#1388)

Post-R7b store-arc round 1, found by the dubbo warm-wall decomposition
(the cbm bar): resolution's loop-stage profile showed settle=3.0s — the
main thread idling on TWO resolver workers on an 11-core Mac. Pool sizing
logged `size=2 (budget=1068MB)`: memoryBudgetBytes() falls back to
os.freemem() when uncontained, and macOS keeps RAM deliberately full of
reclaimable cache, so freemem reads ~1GB on a mostly-idle 64GB machine.
The memory term then capped the pool at 2 where the CPU term allowed 6 —
the macOS sibling of §7a.1's os.cpus() cpuset-blindness (that round fixed
the CPU term; this fixes the memory term).

Fix: darwinMemoryAvailable() reads /usr/bin/vm_stat once per sizing call
and reports free + inactive + speculative + purgeable pages — what
Activity Monitor calls available, the same reclaimable-inclusive
convention the Linux branch already uses by crediting inactive_file back.
Parse failure → null → freemem fallback; Linux/cgroup and Windows paths
untouched.

Measured (dubbo 4,402 files, warm, caffeinated, n=3 each): pool now
self-sizes to 6 (budget 5.7-6.3GB) — wall 8.62-8.83s vs 9.67-10.87s
baseline, resolution phase 6.9→5.3s, loop settle 3.0→1.9s. Matches the
CODEGRAPH_RESOLVE_WORKERS=6 probe exactly (probe-before-build). Dumps
byte-identical pool-6 vs sequential (441,270 lines). Second consumer
unblocked: the cFnPtr LRU cache cap no longer spuriously degrades to 128
on Macs (its full-cache tier is worth ~60s at kernel scale).

Suite: resolver-pool-sizing gains a darwin-gated reclaimable-pages test +
an off-darwin null pin; full suite 2,689 green ×2 with
CODEGRAPH_KERNEL_EXPECT=1.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Colby Mchenry
2026-07-20 20:11:31 -05:00
committed by GitHub
co-authored by Claude Fable 5
parent 3c1f30ab48
commit 27c3c55436
3 changed files with 79 additions and 5 deletions
+29 -1
View File
@@ -7,8 +7,13 @@
* 7GB container at true 8-core concurrency).
*/
import { describe, it, expect } from 'vitest';
import * as os from 'os';
import { ResolverPool } from '../src/resolution/resolver-pool';
import { cgroupMemoryAvailable, memoryBudgetBytes } from '../src/resolution/memory-budget';
import {
cgroupMemoryAvailable,
darwinMemoryAvailable,
memoryBudgetBytes,
} from '../src/resolution/memory-budget';
const GB = 1024 * 1024 * 1024;
const MB = 1024 * 1024;
@@ -77,4 +82,27 @@ describe('memory budget helpers', () => {
expect(v === null || (v >= 0 && Number.isFinite(v))).toBe(true);
}
});
it.runIf(process.platform === 'darwin')(
'darwin: available memory counts reclaimable pages, not just free_count',
() => {
const v = darwinMemoryAvailable();
// vm_stat exists on every macOS; a null here means the parse broke.
expect(v).not.toBeNull();
expect(Number.isFinite(v!)).toBe(true);
// The sum includes the free pages freemem() counts, so it can only be
// larger (modulo TOCTOU drift between the two reads — allow slack).
expect(v!).toBeGreaterThanOrEqual(os.freemem() * 0.5);
// And the budget must ride it (the 2-worker strangulation regression:
// a mostly-idle Mac read ~1GB free and halved the resolver pool).
expect(memoryBudgetBytes()).toBeGreaterThanOrEqual(v! * 0.5);
}
);
it.runIf(process.platform !== 'darwin')(
'darwinMemoryAvailable is null off-macOS and never throws',
() => {
expect(darwinMemoryAvailable()).toBeNull();
}
);
});