perf(resolution): batch-loop de-quadratic — keyset reads, changes-based guard, DB-scaled valve caps + resolve profiler (#1339)
The §7a.2 per-ref profile overturned the assumption the whole arc was built on: resolveOne owns only ~93s of the kernel-scale ~433s batch loop. Loop-stage attribution (CODEGRAPH_RESOLVE_PROFILE, shipped here) named the rest: backpressure folds 111.2s, count guard 93.9s, batch reads 54.6s, deletes/inserts/marks ~84s, settle 85.7s. - Non-progress guard O(remaining)→O(1): the per-batch COUNT(*) walked every remaining pending row (O(N²/batch) per run, 93.9s). The cleanup queries now return summed SQLite , and zero-removals-from-claimed-work is the guard signal — the DIRECT evidence the count diff inferred (a mismatched-name resolver makes keyed cleanup no-op ⇒ changes=0). A real COUNT runs only on that suspicious path and arbitrates exactly as before. - Batch reads OFFSET→keyset (54.6s→O(batch)): OFFSET re-walked the accumulated failed-row prefix every read; seeking past the last-seen rowid is prefix-independent and enumeration-order identical. - WAL valve caps scale with DB size (env still wins): every fold re-writes hot pages (#1231 in bounded form — 111.2s at the flat 256MB cap); soft=clamp(dbSize/4, 256MB, 2GB) trades ~4× fewer folds for a transient WAL ≈ project size. - CODEGRAPH_RESOLVE_PROFILE: per-outcome resolveOne histogram + loop-stage attribution, main + workers, off by default. Gates: dubbo dump byte-identical; suite 2,491 passed / 4 skipped (kernel required). Kernel-scale payoff run lands in the plan doc next. Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Fable 5
parent
19cf1ec75b
commit
7cc23668b5
+3
-3
@@ -25,7 +25,7 @@ import {
|
||||
FindRelevantContextOptions,
|
||||
} from './types';
|
||||
import { DatabaseConnection, getDatabasePath, removeDatabaseFiles } from './db';
|
||||
import { WalCheckpointValve } from './db/wal-valve';
|
||||
import { WalCheckpointValve, resolveWalValveMb } from './db/wal-valve';
|
||||
import { QueryBuilder } from './db/queries';
|
||||
import {
|
||||
isInitialized,
|
||||
@@ -472,7 +472,7 @@ export class CodeGraph {
|
||||
this.db.setWalAutocheckpoint(0);
|
||||
walValve = new WalCheckpointValve(
|
||||
this.db,
|
||||
undefined,
|
||||
resolveWalValveMb(process.env.CODEGRAPH_WAL_VALVE_MB, this.db.getDbFileSizeBytes()),
|
||||
undefined,
|
||||
options.verbose ? (m) => console.log(`[wal-valve] ${m}`) : undefined
|
||||
);
|
||||
@@ -753,7 +753,7 @@ export class CodeGraph {
|
||||
this.db.setWalAutocheckpoint(0);
|
||||
walValve = new WalCheckpointValve(
|
||||
this.db,
|
||||
undefined,
|
||||
resolveWalValveMb(process.env.CODEGRAPH_WAL_VALVE_MB, this.db.getDbFileSizeBytes()),
|
||||
undefined,
|
||||
options.verbose ? (m) => console.log(`[wal-valve] ${m}`) : undefined
|
||||
);
|
||||
|
||||
Reference in New Issue
Block a user