perf(store): resolution ref-index window — kernel-scale resolution 423→276s, 8c envelope ≈11min (§4d round 2) (#1369)
Store-architecture arc round 2. The batched resolution loop reads unresolved_refs ONLY through the status index + the PK keyset pager; the other five ref indexes (from_node, name, file_path, from_name, failed_tail) serve sync-time paths — yet every per-batch DELETE of resolved refs maintained all of them, the biggest single main-thread stage on the dubbo profile (deletes 1.2s of a 5.4s resolution phase) and 50-81s at kernel scale. beginBulkRefLoad/endBulkRefLoad on DatabaseConnection, threaded as refIndexLoad hooks next to the existing bulkEdgeLoad pair with the same minRefsForPool gate (small syncs never pay): drop the five for the loop, rebuild each in one scan at the end — where the table holds only the surviving FAILED refs (resolved rows are deleted by then), so the recreate is near-free. Crash inside the window heals on the next open (schema.sql re-applies CREATE INDEX IF NOT EXISTS). Measured: - dubbo: deletes 1.2 → 0.2s, marks 0.6 → 0.3s, recreate 219ms; wall ~8.5s flat — the freed main-lane time shifts into settle (the worker lane now binds the double-buffer at medium scale). - Linux kernel 8c: resolution 423.4 → 275.9s (deletes 50-81 → 3.2s, backpressure 16.8 → 7.4s — fewer index writes mean less WAL and cheaper folds), ref recreate 10.3s. Envelope ≈ 11.0min, from the 14.8min pre-arc best; <10min-on-8c now needs ~1 more minute. Gates: dubbo/gson dumps byte-identical; linux counts exact 2,049,153/6,413,518 and dump sha 6dd1185b… reproduced (10,446,478 lines); full suite green ×2 (153 files / 2588 tests). Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Fable 5
parent
f6d8e8fdab
commit
ce0ae30e09
@@ -543,6 +543,30 @@ gap. Multi-file write transactions are likely ~zero on the fastInit path
|
||||
batching); buffer→bind remains a CPU-side option if the writer re-emerges as
|
||||
the wall.
|
||||
|
||||
**Store-arc round 2 SHIPPED (2026-07-19): resolution ref-index window.** The
|
||||
batched resolution loop reads unresolved_refs ONLY through the status index
|
||||
+ PK keyset pager; the other five ref indexes (from_node, name, file_path,
|
||||
from_name, failed_tail) serve sync-time paths — yet every per-batch DELETE
|
||||
maintained all of them. `beginBulkRefLoad`/`endBulkRefLoad` (same
|
||||
`minRefsForPool` gate as the edge window) drop the five for the loop and
|
||||
rebuild at the end, where the table holds only the surviving failed refs
|
||||
(resolved rows are deleted by then), making the recreate near-free.
|
||||
|
||||
- **dubbo**: deletes 1.2 → 0.2s, marks 0.6 → 0.3s (recreate 219ms) — but the
|
||||
wall stayed ~8.5s: the freed main-lane time moved into `settle` (the
|
||||
worker lane now binds the double-buffer). The honest read: at medium
|
||||
scale, resolution's floor is now the WORKER lane + pool spin-up, not the
|
||||
writer.
|
||||
- **Linux kernel 8c: resolution 423.4 → 275.9s** — deletes 50-81 → **3.2s**,
|
||||
backpressure 16.8 → 7.4s (fewer index writes → less WAL → cheaper folds,
|
||||
compounding), ref recreate 10.3s, edge recreate 26.8s. Synthesis 149.0s.
|
||||
**Envelope ≈ 11.0min** (phase sum 659.7s) — from 14.8min best-ever before
|
||||
this arc's rounds. **The <10min-on-8c target needs ~1 more minute**; the
|
||||
remaining mass is parse-loop 190s (extraction-bound on linux — R7b's
|
||||
wasm-deferred C tail + the store lane) and synthesis 149s.
|
||||
- Gates: dubbo/gson dumps byte-identical, linux counts exact + dump sha
|
||||
`6dd1185b…` reproduced, suite green ×2.
|
||||
|
||||
## 4. Per-language tracker
|
||||
|
||||
Tiers: **T1** = mostly `.scm` + mapping config. **T2** = needs bespoke pre/post passes kept
|
||||
|
||||
Reference in New Issue
Block a user