github/codegraph
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

perf(sync): adaptive quick-fire debounce + scoped watcher sync — save-to-graph well under a second at any scale (#1397)

Browse Source 
Two changes to the watcher path (the always-on daemon every agent
session uses), which previously paid a flat 2s debounce plus a full-tree
scan-diff on every save even though the OS events name the exact files:

1. Adaptive debounce: a pending set of ≤2 files fires after a 300ms
   quiet window; ≥3 keeps the full configured window so agent
   multi-file bursts coalesce exactly as before. Re-arming preserves
   trailing-edge semantics; a user-set CODEGRAPH_WATCH_DEBOUNCE_MS
   remains the authoritative upper bound (quick window never exceeds
   it, floor 100ms).

2. Scoped sync: watcher-triggered syncs pass their pending paths, and
   the reconciler stats exactly those — per-path logic identical to the
   full walk (stat pre-filter, hash confirm, the #1240
   removal/resurrection flow) — skipping the O(repo) scan and
   tracked-load. Strict fallbacks keep the full scan-diff as ground
   truth: directory removals (#1285 — the events can't name the
   children), empty pending sets (retry paths), and >500-file storms
   (branch checkouts, which also self-heal anything event coalescing
   dropped). filesChecked counts examined PATHS so a deletion-only
   scoped sync can't mimic the #449 lock-unavailable signature.

Measured (warm in-process, the daemon path): dubbo one-file sync work
512→335ms, Swift compiler (27k files) 884→385ms — save-to-fresh-graph
≈0.6-0.7s end-to-end including the quick debounce, from ~2.5-6s
perceived before. Gates: scoped-vs-full dumps byte-identical on dubbo
AND the Swift compiler; watcher suite 30/30 (3 new: scoped pass-through,
dir-removal fallback, quick-fire timing); sync suite 34/34 (4 new
scoped-parity cases incl. delete-resurrection and the lock signature);
full suite 2,696 ×2 with CODEGRAPH_KERNEL_EXPECT=1.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Colby Mchenry
2026-07-21 12:09:31 -05:00
committed by GitHub
co-authored by Claude Fable 5
parent 157c8e735d
commit c74e8b05e0
9 changed files with 255 additions and 18 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/extraction/index.ts
+39 -8
View File
@@ -2487,7 +2487,19 @@ export class ExtractionOrchestrator {
* changes. This works in non-git projects and catches committed changes from
* `git pull`/`checkout`/`merge`/`rebase` that `git status` cannot see.
*/
async sync(onProgress?: (progress: IndexProgress) => void): Promise<SyncResult> {
async sync(
onProgress?: (progress: IndexProgress) => void,
/**
* Watcher fast path: the exact project-relative paths the OS reported as
* changed. When provided, reconciliation runs over ONLY these paths —
* per-path logic identical to the full walk (stat pre-filter, hash
* confirm, the #1240 removal/resurrection flow) — skipping the O(repo)
* scan and tracked-load. Callers must pass undefined whenever the change
* set is not exactly known (directory removals, event overflow): the full
* scan-diff remains the ground truth those cases need (#1285).
*/
scopedPaths?: string[]
): Promise<SyncResult> {
await initGrammars(); // Initialize WASM runtime (grammars loaded lazily below)
const startTime = Date.now();
let filesChecked = 0;
@@ -2514,14 +2526,33 @@ export class ExtractionOrchestrator {
// changes from `git pull`/`checkout`/`merge`/`rebase` — which `git status`
// cannot see, because the working tree is clean afterward.
const tSyncScan = Date.now();
const currentFiles = await scanDirectoryAsync(this.rootDir);
if (process.env.CODEGRAPH_SYNTH_TIMINGS) console.error(`[phase-timing] sync-scan: ${Date.now() - tSyncScan}ms (${currentFiles.length} files)`);
filesChecked = currentFiles.length;
const currentSet = new Set(currentFiles);
let currentFiles: string[];
let trackedFiles: FileRecord[];
if (scopedPaths && scopedPaths.length > 0) {
// Scoped reconcile: stat only the reported paths. filesChecked counts
// the PATHS examined (not the files found) — it must stay non-zero even
// when every scoped path was a deletion, because CodeGraph.watch()
// reads `filesChecked === 0 && durationMs === 0` as the
// lock-unavailable signature (#449).
const unique = [...new Set(scopedPaths)];
currentFiles = unique.filter((p) => fs.existsSync(path.join(this.rootDir, p)));
trackedFiles = [];
for (const p of unique) {
const rec = this.queries.getFileByPath(p);
if (rec) trackedFiles.push(rec);
}
filesChecked = unique.length;
if (process.env.CODEGRAPH_SYNTH_TIMINGS) console.error(`[phase-timing] sync-scoped: ${Date.now() - tSyncScan}ms (${unique.length} paths, ${trackedFiles.length} tracked)`);
} else {
currentFiles = await scanDirectoryAsync(this.rootDir);
if (process.env.CODEGRAPH_SYNTH_TIMINGS) console.error(`[phase-timing] sync-scan: ${Date.now() - tSyncScan}ms (${currentFiles.length} files)`);
filesChecked = currentFiles.length;
const tTracked = Date.now();
const trackedFiles = this.queries.getAllFiles();
if (process.env.CODEGRAPH_SYNTH_TIMINGS) console.error(`[phase-timing] sync-tracked-load: ${Date.now() - tTracked}ms (${trackedFiles.length} tracked)`);
const tTracked = Date.now();
trackedFiles = this.queries.getAllFiles();
if (process.env.CODEGRAPH_SYNTH_TIMINGS) console.error(`[phase-timing] sync-tracked-load: ${Date.now() - tTracked}ms (${trackedFiles.length} tracked)`);
}
const currentSet = new Set(currentFiles);
const trackedMap = new Map<string, FileRecord>();
for (const f of trackedFiles) {
trackedMap.set(f.path, f);