perf(synthesis): fan dynamic-dispatch passes across the resolver pool, byte-identical graphs (#1321)
The ~36 independent synthesis passes (callback/event/framework wiring) ran sequentially on the indexer's main thread — 2.0s of a 4,402-file Java repo's index, and the stage where kernel-class repos die (#1212). They now live in an explicit registry (SYNTH_PASSES) and, when the resolver pool is alive (>=150k-ref repos), fan out across its read-only workers: dubbo synthesis 2,024ms -> ~900ms (-55%), total fresh init 13.5s -> 11.9s. Graphs verified byte-for-byte identical on both the pool path (dubbo) and the sequential path (excalidraw). Why this is safe: no pass's edges persist until the ordered merge, so every pass sees the same committed post-resolution DB state in either mode, and results merge in registry order regardless of completion order — the first-seen dedup is unchanged. The pool now survives through synthesis (destroy moved after it) instead of being torn down moments before the one stage that could reuse it. Robustness: a pass that fails on a worker (crash, OOM) is retried on the main thread — a synthesizer blow-up now costs one worker instead of the whole index, which is half the #1212 story on very large repos. Also: ref-row cleanup deletes now run as one transaction with a cached statement instead of one implicit commit per 500-row chunk (mechanically fewer WAL commits; matters most on HDD-class storage). A set-based rewrite of failed-ref parking was tried, measured ~zero on NVMe, and dropped — the remaining persist cost is edge-index B-tree maintenance, not statement dispatch. SYNTH_PROGRESS_STEPS now derives from the registry (passes + fixed marks); the pin test counts registry entries plus literal __mark sites. Suite green (2444). Sequential-path timing unchanged on excalidraw. Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Fable 5
parent
a2f3c31a97
commit
cf38ef65af
+17
-5
@@ -1305,6 +1305,14 @@ export class ReferenceResolver {
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* The resolver's live ResolutionContext — resolver-pool workers use it to
|
||||
* run synthesis passes against their own read-only connection.
|
||||
*/
|
||||
getResolutionContext(): ResolutionContext {
|
||||
return this.context;
|
||||
}
|
||||
|
||||
/**
|
||||
* Re-queue deferred post-pass refs produced by resolver workers, preserving
|
||||
* their admission order so resolveChainedCallsViaConformance /
|
||||
@@ -1551,25 +1559,29 @@ export class ReferenceResolver {
|
||||
batch = nextBatch;
|
||||
inFlight = nextInFlight;
|
||||
}
|
||||
} finally {
|
||||
if (pool) await pool.destroy().catch(() => undefined);
|
||||
}
|
||||
|
||||
// Dynamic-edge synthesis: now that all base `calls` edges are persisted,
|
||||
// synthesize observer/callback dispatch edges (dispatcher → registered
|
||||
// callbacks) that static parsing leaves out. Best-effort — never fail the
|
||||
// index on it. See docs/design/callback-edge-synthesis.md.
|
||||
// index on it. The pool (when it survived resolution) is REUSED to fan the
|
||||
// independent passes across its read-only workers — that's why its destroy
|
||||
// lives in the finally below, after synthesis, not at the end of the batch
|
||||
// loop. See docs/design/callback-edge-synthesis.md.
|
||||
const tSynth = Date.now();
|
||||
try {
|
||||
aggregateStats.byMethod['callback-synthesis'] = await synthesizeCallbackEdges(
|
||||
this.queries,
|
||||
this.context,
|
||||
onSynthesisProgress
|
||||
onSynthesisProgress,
|
||||
pool
|
||||
);
|
||||
} catch {
|
||||
// synthesis is additive and optional; ignore failures
|
||||
}
|
||||
if (process.env.CODEGRAPH_SYNTH_TIMINGS) console.error(`[phase-timing] callback-synthesis: ${Date.now() - tSynth}ms`);
|
||||
} finally {
|
||||
if (pool) await pool.destroy().catch(() => undefined);
|
||||
}
|
||||
|
||||
return {
|
||||
resolved: [],
|
||||
|
||||
Reference in New Issue
Block a user