perf(resolution): cFnPtr fuse-then-link — one extraction sweep + filtered verbatim linking, pass −22% at kernel scale (#1364)

Task #5 step 1 (plan §7a.8/§7a.9). The C/C++ function-pointer dispatch
synthesizer swept every file's text four times (typedefs, registrations,
propagation, dispatch); at kernel scale the all-or-nothing source cache
declines, so that was 4.4 read+strips per file — 78s of the ~230s pass.

Now ONE extraction sweep reads+strips each file once and collects typedef
names, struct-node field declarations (structurally parsed, fn-pointer
classification deferred until the typedef sets are complete), resolved
includes, an alias-shaped-macro name set, and per-file survival filters
(init type tokens, array element types, inline-struct summaries,
field-assign pairs, dispatch fields/array names — interned, a few MB on
linux). The linking stages then replay the ORIGINAL pass bodies verbatim:
struct layouts register in kind-scan order (same-name precedence is
order-sensitive), and registration/propagation/dispatch run only for
files their filter proves can have side effects, lazily re-stripping just
those. Filters only over-approximate (full-file no-skip scans ⊇ the real
passes' jump-cursor scans), and a filtered-out file is one where every
match fails the pass's own gates before any side effect — parity by
construction. Macro tables stay lazy: a sizing probe found 6.1M #define
lines on linux (amdgpu register headers), ruling out retention.

Measured (8c cg1212, quiet host, fresh kernel init): cFnPtr pass 230s →
179s (−22%); strips 283.5k → 132.4k (4.44 → 2.08/file, 78 → 46.6s);
dispatch stage 95 → 18.5s; callback-synthesis phase 250 → 199.9s.
Standalone probe on the live DB: 139 → 122s.

Byte-parity gates, all green: probe-hash identical on the live kernel DB
(279,335 edge rows both builds); git/redis/vim/SameBoy full dumps
byte-identical old-vs-new (macro tables, commands.def, #ifdef
include-units, inline structs, bare arrays exercised); kernel-parity
0-diffs on git/redis/fmt/protobuf, deferral unchanged; linux counts
exact 2,049,153/6,413,518 and dump sha 6dd1185b… reproduced
(10,446,478 lines); full suite green ×2 (152 files / 2563 tests).

Step 2 (native per-file extractor) now has its boundary: the extraction
sweep, raw text in → records out.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Colby Mchenry
2026-07-19 11:43:06 -05:00
committed by GitHub
co-authored by Claude Fable 5
parent b877db617c
commit c6850d737b
3 changed files with 459 additions and 127 deletions
+56
View File
@@ -942,6 +942,62 @@ Three quick measurements before any port, two of them killing assumptions:
be ~6-10s for the full corpus even before redundancy cuts — but marshal
(UTF-16↔UTF-8 across napi) eats seconds at GB scale; batch the calls.
#### 7a.9 cFnPtr fuse-then-link step 1 landed (2026-07-19) — pass 22%, strips halved
Step 1 shipped, with one deliberate deviation from the §7a.8 sketch. The
"text-free global linking" ideal is unreachable at byte-parity without
retaining per-file text or macro tables, and a sizing probe on the linux tree
killed retention: **6.1M `#define` lines** (the amdgpu register headers alone
are most of them — 565MB of define text), and unrestricted initializer-body
capture is a #1212-class hazard. What ships instead:
- **One extraction sweep** (read+strip per file exactly once): typedef names,
per-struct-node field declarations parsed structurally with fn-pointer
classification DEFERRED (typedef sets aren't complete mid-sweep), resolved
local includes, an alias-shaped-object-macro name set, and per-file
SURVIVAL FILTERS — distinct init type tokens, array element types,
inline-struct summaries, field-assign pairs, dispatch fields/array names.
All interned; a few MB at kernel scale (measured distinct: 110k assign
pairs, 87.6k init tokens, 38.7k dispatch fields; only 16% of files have any
dispatch-shaped match at all).
- **Linking replays the ORIGINAL pass bodies verbatim**, gated by the
filters: struct layouts register by replaying the struct kind-scan (rowid
order — same-name precedence is order-sensitive), and the
registration/propagation/dispatch loops run only for surviving files, whose
text is lazily re-stripped (LRU-served). Filters only ever over-approximate
(full-file no-skip scans ⊇ the jump-cursor/per-body scans the real passes
run), and a filtered-out file is one where every match fails the pass's own
gates before any side effect — so parity is by construction, not by hope.
Macro tables stay lazy (LRU + strip-on-miss) per the 6.1M-defines probe.
- **Why not pure one-sweep C:** the inline-struct scan's cursor jump is gated
on the fn-ptr-field test, which needs the complete typedef sets — a
collect-time emulation diverges on the gate-fail rescan path. Keeping
today's scan code and paying a filtered second strip is the parity-safe
trade.
Measured (8c cg1212, quiet host, fresh kernel init): cFnPtr sub
**A=94.5s B=1.5s C=39.7s D=24.8s E=18.5s = 179s vs the §7a.8 ~230s (22%)**;
strips **283.5k → 132.4k** (4.44 → 2.08/file; 78s → 46.6s); stage E collapsed
95 → 18.5s (survivor-only slicing/getNodesInFile), C+D 89 → 64.5s;
callback-synthesis phase 250 → **199.9s**. Standalone probe-to-probe on the
same live DB (warm cache): 139 → 122s. Resolution superphase unaffected
(622.7s ≈ #1362's 633.6). Under the 70-90s hope — the honest ledger is that
~0.6 sweeps of survivor re-strips + the unchanged include-unit machinery stay,
and A now carries all regex scans.
Gates, all green: probe-hash identical on the live kernel DB (279,335 edge
rows, `f6e1713d…` both builds); git/redis/vim/SameBoy full dumps
byte-identical old-vs-new (705/852/433/180 fn-ptr edges — macro tables,
`commands.def`, `#ifdef` include-units, inline structs, bare arrays all
exercised); kernel-parity 0-diffs on git/redis/fmt/protobuf with deferral
unchanged (12.2/24.1/42.5/25.7%); linux counts exact 2,049,153/6,413,518;
linux dump sha reproduced (`6dd1185b…`); full suite green.
Step 2's boundary is now stage A verbatim: raw text in → records out, no
graph access inside the sweep except `getNodesInFile` for struct extents. Its
94.5s (46.6s strip + scans) is the native-extractor prize; C's 39.7s
(macro-env + include units) and D's 24.8s stay TS.
### 7b. Arc 3 — graph richness (forensics-backed; adopt cbm's real extras, skip inflation)
Priority order, each gated by the standard A/B + node-explosion probes:
1. **Test→subject edges** (first-class `tests` edges at index time; we compute covering