perf(resolution): incremental receiver-inference scan memo + compiled-pattern memo — kong −8% more (−23% cumulative), byte-identical (#1392)

The kong/tokio matcher-chain residue attributed (nm:mc-* sub-stage rows,
shipped here too): matchMethodCall's cost is ~entirely
inferLocalReceiverType — 61µs per miss on kong, 99% miss rate (39k `self:`
calls hunting a local declaration Lua never writes), re-scanning the same
scope lines for every ref.

Two pure memos, both semantics-preserving by construction:
- Compiled-pattern memo: localReceiverTypePatterns/phpPropertyTypePatterns
  built 2-4 fresh RegExp objects per call; patterns are a pure function of
  (language, receiver) and non-global, so instances are shared via a
  FIFO-capped map (no per-get mutation — the §7a.6 LRU-churn lesson).
- Incremental scan memo: refs for the same (file, scope, receiver) arrive
  in ~ascending line order and the backward declaration scan is a pure
  function of immutable file lines — a per-context watermark scans each
  line once per key (query(c) = highest match in [start..c]; monotonic
  calls extend the watermark over (hi..c]; non-monotonic calls fall back
  to the plain bounded scan). componentScoped (CFML/PHP whole-file sweep)
  is keyed out. States drop with the context's file caches via
  clearNameMatcherMemos, wired into ReferenceResolver.clearCaches.

kong mc-infer misses 61→20µs (2.4s→0.8s combined); fresh index 3.43 →
3.03-3.20s (−8%; 4.07 → 3.14 cumulative with #1391). tokio unchanged
(tight scopes). Gates: dubbo (49k Java instance-method HITS ride this
scan), kong, tokio, Fusion dumps all byte-identical; suite 2,689 ×2 with
CODEGRAPH_KERNEL_EXPECT=1.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Colby Mchenry
2026-07-21 00:20:03 -05:00
committed by GitHub
co-authored by Claude Fable 5
parent abb0a916f7
commit 974e6c8b95
3 changed files with 173 additions and 45 deletions
+1
View File
@@ -24,6 +24,7 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
- Indexing very large projects on multi-core machines got faster again: the parallel-resolution workers now periodically refresh their read-only database connections, which lets database housekeeping advance instead of silently building up a backlog behind long-lived readers — a backlog that was taxing the indexer's own writes. Graphs remain byte-for-byte identical; the win is largest at Linux-kernel scale on many-core machines.
- Indexing on macOS now uses the machine's real memory headroom when sizing its parallel-resolution workers. macOS deliberately keeps RAM filled with reclaimable cache, so the previous free-memory reading came back tiny (~1GB on an otherwise idle machine) and silently halved the worker pool — a medium Java project's fresh index ran about 1520% slower than the hardware allowed. Graphs remain byte-for-byte identical; the same fix also lets a memory-driven analysis cache engage fully on macOS for large C codebases.
- Fresh indexing got a sizeable across-the-board speedup: during the initial build, the database's secondary lookup indexes are set aside and rebuilt once after parsing instead of being maintained row by row — the same proven trick the later linking phase already used, now applied to the whole parse lane — and the reference-resolution loop likewise stops maintaining lookup indexes it never reads, rebuilding them at the end when almost nothing is left in the table. A medium Java project's parse phase runs about 58% faster and its full fresh index about 19% faster end-to-end; a Linux-kernel-scale index that took ~15 minutes on an 8-core machine now completes in about 11, with the resolution phase alone dropping by a third. Graphs remain byte-for-byte identical, and incremental syncs are unaffected.
- Resolving method calls through local variables (`recv.method()`, Lua's `recv:method()`, R's `recv$method()`) got much cheaper on repos where the same receiver is called over and over: the declaration scan that types the receiver now remembers what it has already scanned per scope instead of re-reading the same source lines for every call site, and the regex patterns it scans with are compiled once per receiver instead of per call. Kong's fresh index drops another 8% on top of the require-resolution fix (23% cumulative), with graphs byte-for-byte identical everywhere — including Java projects, where this same scan successfully types tens of thousands of receivers.
- Indexing Lua and Luau projects got a sizeable speedup: resolving each `require(...)` no longer rescans the project's entire file list four times — a per-project filename index answers the same lookup instantly, cutting per-require resolution from about a millisecond to microseconds. A fresh index of Kong (1,870 Lua files) runs about 16% faster end-to-end, with the graph byte-for-byte identical. The same housekeeping also closes a latent staleness edge where COBOL copybook lookups could keep serving a cached file list after files changed.
- Parallel reference resolution now engages adaptively instead of by a fixed project-size cutoff: the indexer measures the actual per-reference resolution rate on the first batch and spins up the worker pool mid-run whenever the remaining work justifies it. Languages whose references are expensive to resolve benefit most — Rust especially: a fresh index of tokio runs about 23% faster, with the graph byte-for-byte identical. Small projects and low-core machines (2-core CI runners) keep the single-threaded path exactly as before.
- The dynamic-dispatch analysis at the end of indexing now skips passes that provably can't produce anything for the project at hand: React re-render bridging when no class has a `render` method, React Native and Expo cross-platform pairing when the required languages aren't present, and MyBatis mapper linking when there's no mapper XML. Previously each of these scanned the whole graph before coming up empty — on a 4,000-file Java project that was about 0.9 seconds of wasted analysis per fresh index. The interface-implementation bridging pass also got cheaper on real work: it no longer re-fetches a hub interface's method list once per implementer, and classes that extend or implement nothing are skipped before any per-class lookups. Graphs remain byte-for-byte identical.