perf(resolution): cFnPtr fuse-then-link — one extraction sweep + filtered verbatim linking, pass −22% at kernel scale (#1364)

Task #5 step 1 (plan §7a.8/§7a.9). The C/C++ function-pointer dispatch
synthesizer swept every file's text four times (typedefs, registrations,
propagation, dispatch); at kernel scale the all-or-nothing source cache
declines, so that was 4.4 read+strips per file — 78s of the ~230s pass.

Now ONE extraction sweep reads+strips each file once and collects typedef
names, struct-node field declarations (structurally parsed, fn-pointer
classification deferred until the typedef sets are complete), resolved
includes, an alias-shaped-macro name set, and per-file survival filters
(init type tokens, array element types, inline-struct summaries,
field-assign pairs, dispatch fields/array names — interned, a few MB on
linux). The linking stages then replay the ORIGINAL pass bodies verbatim:
struct layouts register in kind-scan order (same-name precedence is
order-sensitive), and registration/propagation/dispatch run only for
files their filter proves can have side effects, lazily re-stripping just
those. Filters only over-approximate (full-file no-skip scans ⊇ the real
passes' jump-cursor scans), and a filtered-out file is one where every
match fails the pass's own gates before any side effect — parity by
construction. Macro tables stay lazy: a sizing probe found 6.1M #define
lines on linux (amdgpu register headers), ruling out retention.

Measured (8c cg1212, quiet host, fresh kernel init): cFnPtr pass 230s →
179s (−22%); strips 283.5k → 132.4k (4.44 → 2.08/file, 78 → 46.6s);
dispatch stage 95 → 18.5s; callback-synthesis phase 250 → 199.9s.
Standalone probe on the live DB: 139 → 122s.

Byte-parity gates, all green: probe-hash identical on the live kernel DB
(279,335 edge rows both builds); git/redis/vim/SameBoy full dumps
byte-identical old-vs-new (macro tables, commands.def, #ifdef
include-units, inline structs, bare arrays exercised); kernel-parity
0-diffs on git/redis/fmt/protobuf, deferral unchanged; linux counts
exact 2,049,153/6,413,518 and dump sha 6dd1185b… reproduced
(10,446,478 lines); full suite green ×2 (152 files / 2563 tests).

Step 2 (native per-file extractor) now has its boundary: the extraction
sweep, raw text in → records out.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Colby Mchenry
2026-07-19 11:43:06 -05:00
committed by GitHub
co-authored by Claude Fable 5
parent b877db617c
commit c6850d737b
3 changed files with 459 additions and 127 deletions
+1
View File
@@ -22,6 +22,7 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
- Every release is now cryptographically verifiable: npm packages publish with npm provenance (the "Provenance" badge on npmjs.com, proving each version was built by this repository's release workflow from a specific commit), and the GitHub Release bundles carry signed build attestations you can check with `gh attestation verify <file> -R colbymchenry/codegraph`.
- Indexing inside CPU- or memory-limited containers (Docker, CI runners) now sizes its worker pools from the container's actual allowance instead of the host machine's, and giant codebases no longer balloon temporary database files during indexing (previously tens of GB of transient disk on Linux-kernel-scale projects). Together these prevent out-of-memory and out-of-disk failures on constrained machines; set `CODEGRAPH_RESOLVE_WORKERS` to override the resolution worker count explicitly.
- Indexing very large projects on multi-core machines got faster again: the parallel-resolution workers now periodically refresh their read-only database connections, which lets database housekeeping advance instead of silently building up a backlog behind long-lived readers — a backlog that was taxing the indexer's own writes. Graphs remain byte-for-byte identical; the win is largest at Linux-kernel scale on many-core machines.
- Indexing large C and C++ codebases spends much less time in the function-pointer dispatch analysis (the pass that connects handler tables like a command table or an ops struct to their call sites): each source file is now read and prepared once instead of four times, and files that can't contribute any dispatch wiring are skipped outright in the later linking steps. On a Linux-kernel-scale tree the pass runs about a fifth faster and the end-of-indexing dispatch-linking stage drops accordingly, with graphs byte-for-byte identical.
### Fixes