From 19cf1ec75b01077bf69b46dca39fedd54c76c798 Mon Sep 17 00:00:00 2001 From: Colby Mchenry Date: Fri, 17 Jul 2026 09:52:47 -0500 Subject: [PATCH] =?UTF-8?q?docs(kernel):=20P1=20record=20runs=20=E2=80=94?= =?UTF-8?q?=202c=2020.4min=20(-23%),=208c=2018.3min=20no-OOM,=20WAL=2014x?= =?UTF-8?q?=20contained;=20resolution=20measured=20core-invariant=20(#1338?= =?UTF-8?q?)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Claude Fable 5 --- docs/design/rust-kernel-migration-plan.md | 74 ++++++++++++++++++++--- 1 file changed, 67 insertions(+), 7 deletions(-) diff --git a/docs/design/rust-kernel-migration-plan.md b/docs/design/rust-kernel-migration-plan.md index 24d571b..1c0a3c0 100644 --- a/docs/design/rust-kernel-migration-plan.md +++ b/docs/design/rust-kernel-migration-plan.md @@ -53,13 +53,17 @@ them are the ORIGINAL plan and carry expectations that measurement later correct multiline `^` anchors after `\r` — §0a traps) — fixed + CRLF fixtures pinned cross-platform in #1329. Every prebuild target platform is now validated. - [ ] **P1. Kernel-scale resolution speed** (§7a) — measurement round RUN 2026-07-17 - and it RESHAPED the arc (full record §7a.1): the "sequential at 2 CPUs" - premise was FALSE (pool sizing is cpuset-blind; R6 already ran 6 workers), - and both 8-core re-runs failed STRUCTURALLY before yielding a clean number — - container OOM at 7GB (memory-blind pool sizing), WAL blowup to 22GB on a - 4.6GB DB (pool reader snapshots starve checkpoint truncation at kernel - scale). Revised order: (1) WAL containment, (2) memory-aware pool sizing, - (3) then the speed re-measure. Target unchanged: <10min on 8 cores. + and it RESHAPED the arc (full record §7a.1); items (1) WAL containment and + (2) memory-aware/cgroup-honest sizing **SHIPPED same day (#1332–#1335, + §7a.2)** after an implementation arc whose three failed/diagnostic + kernel-scale runs each corrected the design (WAL file ≠ WAL backlog; + cgroup cache credit; pool net-negative at 2 cores; parse floor). The + 2c/6GB envelope already improved 26.4 → 21.6min with counts byte-exact. + Record runs DONE (§7a.2): 2c/6GB **20.4min (R6 −23%), WAL 1.57GB (−14×)**; + 8c/7GB **18.3min, NO OOM, WAL 1.09GB** — all byte-exact. The <10min + target is NOT met and the gap is named: resolution is CORE-INVARIANT + (835.9s on 8 cores ≈ 812.5s sequential on 2) — the next arc profiles the + per-ref main-thread path, then cFnPtrEdges (86% of synthesis), then R7a. - [ ] **R7a. C/C++ port** — biggest single-language effort; unlocks cg1212's parse expectation (6.2m → ~1.5–2m, 23% of that wall) + CARLA/UE/llvm-class repos; Metal + CUDA ride along (their blanking pre-passes stay TS-side — `preParse` @@ -597,6 +601,62 @@ the premise and surfaced two structural defects that now gate any speed work. sizing → (3) re-run the 8-core measurement (container at ≥12GB or the Mac with disk headroom) → then profile what remains.** The <10min-on-8-cores target stands. +#### 7a.2 P1 items (1)+(2) SHIPPED 2026-07-17 — the implementation arc (#1332–#1335) + +Four PRs, each carrying its measurement; the arc took three failed/diagnostic +kernel-scale runs to get right, and every failure taught a design fact: + +| Run (2c/6GB unless noted) | Build | Outcome | +|---|---|---| +| R6 baseline | pre-P1 | 26.4min, EXIT 0; WAL unbounded (mid-run peak unmeasured); pooled 6-on-2 (cpuset-blind) | +| run 1 | #1332 hook | **EXIT 137 (OOM)** — WAL 22.2GB, 0 of 5.4M frames ever backfilled; futile 20-pass parks amplified memory churn | +| diagnostic | +latch/debug | EXIT 0, ~24min; pool KILLED by mis-measured 57MB cgroup budget → exposed **sequential resolution 853s vs 1,150s pooled** and cFnPtrEdges = 306s of synthesis's 358s | +| instrumented | +sizing fixes | EXIT 0, **21.6min (R6 −18%)**; parse floor restores 373.5s; passives complete but the FILE marched 361→721MB → named the wrap-never-happens gap; peak 17.2GB | +| record (first attempt) | #1335 | **EXIT 1: "database is locked"** — the timer-path truncate won the lock race after the recreate's multi-GB burst and stalled the writer past its 5s busy_timeout → truncate is barrier-only now (#1336). Bonus data: recreate 7.9s (vs 68–95s) once the WAL stays folded | +| **record** | **#1336** | **EXIT 0, 20.4min (R6 −23%); WAL peak 1.57GB (−14×); counts byte-exact 2,048,664/6,405,964.** parse 354.9s · resolution 812.5s · synthesis 329.0s · recreate 57.5s · maintenance 43.5s | +| **8-core retry (8c/7GB)** | **#1336** | **EXIT 0, NO OOM — 18.3min; WAL peak 1.09GB; pool sized 4 by the memory term (ap=8, budget 5.1GB, db 4.1GB); counts byte-exact.** parse 208.7s · resolution 835.9s · synthesis 338.7s | + +**The 8-core verdict (the question P1 set out to ask): 18.3min vs the <10min +target — infrastructure fixed, speed target NOT met, and the gap is now +precisely characterized. Resolution is CORE-INVARIANT at kernel scale: 835.9s +pooled-4-on-8 ≈ 812.5s sequential-on-2 — worker parallelism buys nothing, so +the bottleneck is the per-ref main-thread path (admission + persist + per-ref +resolver work), not topology. Of the 18.3min, ~14min is core-invariant +resolution+synthesis. Next levers, in order: (a) profile the per-ref path +inside resolution (the 812–836s floor), (b) `cFnPtrEdges` (306s, 86% of +synthesis — parallelize/window WITHIN the pass), (c) the R7a C/C++ port +(parse 209s → kernel-native). 4× cores currently buys only 2min end-to-end +(20.4 → 18.3) because parse is the only core-scaling phase left.** + +**Design facts these runs established (each now enforced in code + tests):** + +1. **WAL backlog and WAL file are different resources.** Passive backfills bound + the backlog; the FILE only stops growing when a commit finds zero reader + marks — observed never in practice. Containment = backfill + **TRUNCATE at a + parked barrier** (the one guaranteed no-reader window) + a raw file-size + trigger at 4× the soft cap (#1334/#1335). Dubbo: 251MB → 69MB peak, dumps + byte-identical under aggressive fold cycling. +2. **cgroup v2 `memory.current` counts reclaimable page cache** — post-parse it + read 57MB free on a 6GB box and silently disabled the pool. `inactive_file` + is credited back (#1335); the same box reads 4.4GB. +3. **The pool loses to sequential at 2 real cores** (853s vs 1,150s resolution; + cold worker caches + serialization + time-slicing exceed the parallelism), + and pooled synthesis is Amdahl-bound by `cFnPtrEdges` (306s of 358s) at + kernel scale. Sizing: `min(availableParallelism − 1, 6)` + memory term + + `CODEGRAPH_RESOLVE_WORKERS` knob (#1333/#1335); ap=2 → sequential by choice. +4. **Parse needs ≥2 workers even on 2 cores** (1 worker = +34%; main + store + worker don't fill the second core). Floored (#1335): 373.5s ≈ the 369s + oversubscribed baseline, at a fraction of the memory. +5. **Silent failure modes burned three 25-minute cycles**: give-ups were + verbose-gated, sizing's null path logged nothing, the timer path logged + nothing. All valve/sizing decisions now print under `CODEGRAPH_SYNTH_TIMINGS` + / `CODEGRAPH_WAL_VALVE_DEBUG` — the armed line answers "is it even alive" + in one glance. + +**New synthesis lever surfaced:** `cFnPtrEdges` is 86% of kernel-scale synthesis +wall — parallelizing WITHIN that one pass (or windowing its scan) is worth more +than pooling all 36 passes. Filed under the next P1 profiling round. + ### 7b. Arc 3 — graph richness (forensics-backed; adopt cbm's real extras, skip inflation) Priority order, each gated by the standard A/B + node-explosion probes: 1. **Test→subject edges** (first-class `tests` edges at index time; we compute covering