perf(sync): adaptive quick-fire debounce + scoped watcher sync — save-to-graph well under a second at any scale (#1397)

Two changes to the watcher path (the always-on daemon every agent
session uses), which previously paid a flat 2s debounce plus a full-tree
scan-diff on every save even though the OS events name the exact files:

1. Adaptive debounce: a pending set of ≤2 files fires after a 300ms
   quiet window; ≥3 keeps the full configured window so agent
   multi-file bursts coalesce exactly as before. Re-arming preserves
   trailing-edge semantics; a user-set CODEGRAPH_WATCH_DEBOUNCE_MS
   remains the authoritative upper bound (quick window never exceeds
   it, floor 100ms).

2. Scoped sync: watcher-triggered syncs pass their pending paths, and
   the reconciler stats exactly those — per-path logic identical to the
   full walk (stat pre-filter, hash confirm, the #1240
   removal/resurrection flow) — skipping the O(repo) scan and
   tracked-load. Strict fallbacks keep the full scan-diff as ground
   truth: directory removals (#1285 — the events can't name the
   children), empty pending sets (retry paths), and >500-file storms
   (branch checkouts, which also self-heal anything event coalescing
   dropped). filesChecked counts examined PATHS so a deletion-only
   scoped sync can't mimic the #449 lock-unavailable signature.

Measured (warm in-process, the daemon path): dubbo one-file sync work
512→335ms, Swift compiler (27k files) 884→385ms — save-to-fresh-graph
≈0.6-0.7s end-to-end including the quick debounce, from ~2.5-6s
perceived before. Gates: scoped-vs-full dumps byte-identical on dubbo
AND the Swift compiler; watcher suite 30/30 (3 new: scoped pass-through,
dir-removal fallback, quick-fire timing); sync suite 34/34 (4 new
scoped-parity cases incl. delete-resurrection and the lock signature);
full suite 2,696 ×2 with CODEGRAPH_KERNEL_EXPECT=1.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

CHANGELOG.md

@@ -9,6 +9,17 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
 ## [Unreleased]
 
+# ⚡ The Rust engine release
+
+**This release rebuilds CodeGraph's parsing engine as a native Rust kernel and overhauls the resolution pipeline around it — the largest performance upgrade in the project's history, with every graph verified byte-for-byte identical to the previous engine.**
+
+- **Native Rust parsing for 20 languages** — TypeScript, JavaScript (+TSX/JSX), Java, Python, Go, C, C++, Rust, C#, Ruby, PHP, Swift, Kotlin, Scala, Dart, R, Lua, and Luau now parse in a compiled Rust kernel (Metal and CUDA ride the C++ path). Platforms without a prebuilt binary, and individual files with syntax errors, fall back to the previous engine automatically — same graph either way, proven on repositories from small libraries to the Linux kernel.
+- **Adaptive to your machine** — CodeGraph sizes its parse workers, resolver pool, and caches from what the system actually has: real core counts (container/cgroup-aware, not the host's), honest available memory on macOS and Linux, and measured per-project resolution cost. A big workstation gets the full parallel pipeline; a 2-core VPS gets a pipeline tuned to finish reliably instead of running out of memory — the Linux kernel (70k files) indexes to completion on a 2-core, 6GB machine in under 12 minutes — down from 26 at the start of this cycle.
+- **Resolution is dramatically faster across the board** — adaptive parallel resolution, smarter method-candidate lookup, and memoized supertype/conformance walking. The Swift compiler repository (27k files, Swift + C++) went from over 3 minutes to about 100 seconds within this release cycle; Rust, Lua, and Java-family projects all see double-digit improvements.
+- **Sync is now near-instant** — a save reaches the graph in well under a second, even at compiler scale. The always-on watcher fires after a 300ms quiet window for lone saves (bursts of edits still coalesce), and hands the exact changed paths to sync instead of re-scanning the whole tree — measured save-to-fresh-graph work of ~0.3s on a 4,400-file Java project and ~0.4s on the 27,000-file Swift compiler repository, byte-identical to a full reconciliation.
+
+Full details in the entries below.
+
 ### New Features
 
 - Indexing TypeScript, TSX, JavaScript, JSX, Java, Python, Go, C, C++, Rust, C#, Ruby, PHP, Swift, Kotlin, Scala, Dart, R, Lua, and Luau projects is faster: parsing and symbol extraction now run in a native engine when a prebuilt binary is available for your platform (release bundles include one), producing exactly the same graph — verified byte-for-byte against the previous engine on real repositories, from small libraries up to vscode-, dubbo-, django-, git-, protobuf-, tokio-, rust-analyzer-, jellyfin-, rails-, symfony-, swift-nio-, kotlinx.coroutines-, ggplot2-, Kong-, Scala-3-compiler-, and Flutter-scale codebases (Lombok-generated members, C function-pointer tables, and Unreal-Engine-style macro-heavy headers included; CUDA and Metal sources ride the C++ path). The speedup is largest on resource-constrained machines like CI runners. No setup needed: platforms without the native binary, and individual files with syntax errors, automatically use the previous engine, and `CODEGRAPH_KERNEL=0` turns the native path off entirely.
@@ -24,6 +35,7 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 - Indexing very large projects on multi-core machines got faster again: the parallel-resolution workers now periodically refresh their read-only database connections, which lets database housekeeping advance instead of silently building up a backlog behind long-lived readers — a backlog that was taxing the indexer's own writes. Graphs remain byte-for-byte identical; the win is largest at Linux-kernel scale on many-core machines.
 - Indexing on macOS now uses the machine's real memory headroom when sizing its parallel-resolution workers. macOS deliberately keeps RAM filled with reclaimable cache, so the previous free-memory reading came back tiny (~1GB on an otherwise idle machine) and silently halved the worker pool — a medium Java project's fresh index ran about 15–20% slower than the hardware allowed. Graphs remain byte-for-byte identical; the same fix also lets a memory-driven analysis cache engage fully on macOS for large C codebases.
 - Fresh indexing got a sizeable across-the-board speedup: during the initial build, the database's secondary lookup indexes are set aside and rebuilt once after parsing instead of being maintained row by row — the same proven trick the later linking phase already used, now applied to the whole parse lane — and the reference-resolution loop likewise stops maintaining lookup indexes it never reads, rebuilding them at the end when almost nothing is left in the table. A medium Java project's parse phase runs about 58% faster and its full fresh index about 19% faster end-to-end; a Linux-kernel-scale index that took ~15 minutes on an 8-core machine now completes in about 11, with the resolution phase alone dropping by a third. Graphs remain byte-for-byte identical, and incremental syncs are unaffected.
+- Saving a file now updates the graph almost immediately: the file watcher fires after a 300ms quiet window for one or two changed files (bursts still coalesce under the full debounce, and `CODEGRAPH_WATCH_DEBOUNCE_MS` remains the upper bound), and watcher-triggered syncs reconcile exactly the changed paths instead of stat-walking the entire repository. Directory deletions and event storms still run the full scan-diff, so nothing the events can't describe is ever missed. Measured: save-to-fresh-graph sync work drops to ~0.3s on a 4,400-file project and ~0.4s on a 27,000-file one, with resulting graphs byte-for-byte identical to a full reconciliation.
 - Indexing Swift and other protocol/interface-heavy codebases got dramatically faster: the conformance walk that checks whether a method lives on a receiver's supertypes (protocols, base classes, extensions) now remembers its answers for the duration of each resolution batch instead of re-querying the graph for every call site — on the Swift compiler repository (27k files) that walk ran nearly a million times per index. A fresh index of that repo drops from about 185 seconds to under 100, with the graph byte-for-byte identical. Method-candidate lookup also gained a per-name owner index, so overload-heavy names (`init` in Swift, `execute` in Java) no longer pay a full candidate scan per receiver type.
 - Resolving method calls through local variables (`recv.method()`, Lua's `recv:method()`, R's `recv$method()`) got much cheaper on repos where the same receiver is called over and over: the declaration scan that types the receiver now remembers what it has already scanned per scope instead of re-reading the same source lines for every call site, and the regex patterns it scans with are compiled once per receiver instead of per call. Kong's fresh index drops another 8% on top of the require-resolution fix (23% cumulative), with graphs byte-for-byte identical everywhere — including Java projects, where this same scan successfully types tens of thousands of receivers.
 - Indexing Lua and Luau projects got a sizeable speedup: resolving each `require(...)` no longer rescans the project's entire file list four times — a per-project filename index answers the same lookup instantly, cutting per-require resolution from about a millisecond to microseconds. A fresh index of Kong (1,870 Lua files) runs about 16% faster end-to-end, with the graph byte-for-byte identical. The same housekeeping also closes a latent staleness edge where COBOL copybook lookups could keep serving a cached file list after files changed.

README.md

@@ -10,7 +10,9 @@ Follow [@getcodegraph](https://x.com/getcodegraph) on X for updates.
 
 ### Supercharge Claude Code, Cursor, Codex, OpenCode, Hermes Agent, Gemini, Antigravity, and Kiro with Semantic Code Intelligence
 
-**Surgical context · fewer tool calls · faster answers · 100% local**
+**The fastest complete code graph · surgical context · built for how agents actually work · 100% local**
+
+<img src="https://raw.githubusercontent.com/colbymchenry/codegraph/main/assets/languages/rust.svg?v=1" height="18" alt="Rust" align="top"> <sub><b>Kernel powered by Rust</b> — 20 languages parsed natively, scaling itself to your machine's cores and memory</sub>
 
 ### [Documentation & Website →](https://colbymchenry.github.io/codegraph/)
 
@@ -309,10 +311,26 @@ The reliable, universal payoff is **surgical context and speed**: CodeGraph coll
 
 ---
 
+## Built for speed — the Rust kernel
+
+<img align="right" src="https://raw.githubusercontent.com/colbymchenry/codegraph/main/assets/languages/rust.svg?v=1" width="80" height="80" alt="Rust">
+
+CodeGraph's parsing engine is a **native Rust kernel**: 20 languages — TypeScript, JavaScript, Java, Python, Go, C, C++, Rust, C#, Ruby, PHP, Swift, Kotlin, Scala, Dart, R, Lua, Luau (Metal and CUDA ride the C++ path) — parse in compiled code with one boundary crossing per file. Every language shipped only after its graphs proved **byte-for-byte identical** to the reference engine on real repositories, from small libraries up to the Linux kernel; platforms without a prebuilt binary and files with syntax errors fall back per-file automatically, same graph either way.
+
+**And it scales itself to the machine it's on.** Worker pools, parallel resolution, and analysis caches are sized from what the system actually has — real core counts (container/cgroup-aware, so a VPS that grants 2 cores gets sized for 2, not the host's 64), honestly-measured available RAM on macOS and Linux, and the measured cost of *your* project's resolution work:
+
+- **On a workstation:** the full parallel pipeline — native parse workers, a multi-worker resolver pool that engages the moment it pays for itself, memory-gated analysis caches. The Swift compiler repository (27k files of Swift and C++) fresh-indexes in about 100 seconds; a one-file edit re-syncs in ~4.
+- **On a 2-core / 6GB VPS:** the same graph, from a pipeline tuned to *finish* — the Linux kernel (70k files, 2M symbols, 6.4M relationships) indexes to completion in under 12 minutes where RAM-first designs run out of memory before reaching 1%.
+- **Every day after day one:** saving a file updates the graph in well under a second — the watcher fires 300ms after a lone save and syncs exactly what changed (~0.3s of work on a 4,400-file project, ~0.4s on the 27,000-file Swift compiler repo), never re-scanning the tree. Measured against the fastest competing indexer's re-index-on-change: 2–7× faster on medium and larger repos across a 31-repo, 30-language benchmark — and the gap widens with repo size, because their cost grows with the repository and ours grows with the change.
+
+---
+
 ## Key Features
 
 | | |
 |---|---|
+| **Native Rust Kernel** | Parsing and extraction run in a compiled Rust engine for 20 languages — with graphs verified byte-for-byte identical to the reference engine, and automatic per-file fallback so nothing ever breaks |
+| **Adapts to Your Machine** | Sizes its worker pools and caches from what the system actually has — real core counts (container-aware), honest available RAM, measured per-project cost. A workstation gets the full parallel pipeline; a 2-core VPS gets one tuned to finish reliably |
 | **Surgical Context** | One tool call returns entry points, related symbols, and code snippets — no slow file-by-file exploration |
 | **Full-Text Search** | Find code by name instantly across your entire codebase, powered by FTS5 |
 | **Impact Analysis** | Trace callers, callees, and the full impact radius of any symbol before making changes |
@@ -530,7 +548,7 @@ The exact text is `src/mcp/server-instructions.ts` — the single source of trut
 └───────────────────────────────────────────────────────────────────┘
 ```
 
-1. **Extraction** — [tree-sitter](https://tree-sitter.github.io/) parses source code into ASTs. Language-specific queries extract nodes (functions, classes, methods) and edges (calls, imports, extends, implements).
+1. **Extraction** — a native **Rust kernel** parses source with [tree-sitter](https://tree-sitter.github.io/) grammars compiled into it, extracting nodes (functions, classes, methods) and edges (calls, imports, extends, implements) for 20 languages; remaining languages and per-file fallbacks use the same extraction logic on the portable engine, producing identical graphs.
 
 2. **Storage** — Everything goes into a local SQLite database (`.codegraph/codegraph.db`) with FTS5 full-text search.
 

__tests__/sync.test.ts

@@ -757,3 +757,76 @@ describe('Sync Module', () => {
     });
   });
 });
+
+describe('Scoped sync parity (#watcher-scoped)', () => {
+  let testDir: string;
+  let cg: CodeGraph;
+
+  const snapshot = (g: CodeGraph): string => {
+    // Natural-key snapshot of the whole graph, mirroring dump-graph.mjs at
+    // unit scale: scoped and full sync must land the DB in the same state.
+    const nodes = g
+      .searchNodes('', { limit: 100000 })
+      .map((r) => r.node)
+      .map((n) => `${n.kind}|${n.qualifiedName}|${n.filePath}|${n.startLine}`)
+      .sort()
+      .join('\n');
+    return nodes;
+  };
+
+  beforeEach(async () => {
+    testDir = fs.mkdtempSync(path.join(os.tmpdir(), 'codegraph-sync-scoped-'));
+    const srcDir = path.join(testDir, 'src');
+    fs.mkdirSync(srcDir);
+    fs.writeFileSync(path.join(srcDir, 'a.ts'), `export function alpha() { return beta(); }`);
+    fs.writeFileSync(path.join(srcDir, 'b.ts'), `export function beta() { return 1; }`);
+    cg = CodeGraph.initSync(testDir);
+    await cg.indexAll();
+  });
+
+  afterEach(() => {
+    cg?.destroy();
+    if (fs.existsSync(testDir)) fs.rmSync(testDir, { recursive: true, force: true });
+  });
+
+  it('a scoped modify lands the same graph as a full sync of the same edit', async () => {
+    fs.writeFileSync(path.join(testDir, 'src', 'b.ts'), `export function beta() { return 2; }\nexport function gamma() { return 3; }`);
+    const scoped = await cg.sync({ paths: ['src/b.ts'] });
+    expect(scoped.filesModified).toBe(1);
+    const scopedSnap = snapshot(cg);
+
+    // Re-apply the same end state through a FULL sync from the same start
+    // state: revert, full-sync, edit again, full-sync.
+    fs.writeFileSync(path.join(testDir, 'src', 'b.ts'), `export function beta() { return 1; }`);
+    await cg.sync();
+    fs.writeFileSync(path.join(testDir, 'src', 'b.ts'), `export function beta() { return 2; }\nexport function gamma() { return 3; }`);
+    const full = await cg.sync();
+    expect(full.filesModified).toBe(1);
+    expect(snapshot(cg)).toBe(scopedSnap);
+  });
+
+  it('a scoped delete removes the file and resurrects cross-file refs like a full sync', async () => {
+    fs.rmSync(path.join(testDir, 'src', 'b.ts'));
+    const scoped = await cg.sync({ paths: ['src/b.ts'] });
+    expect(scoped.filesRemoved).toBe(1);
+    expect(scoped.filesChecked).toBe(1); // checked paths, not found files (#449 lock signature)
+    const gone = cg.searchNodes('beta');
+    expect(gone.filter((r) => r.node.filePath === 'src/b.ts').length).toBe(0);
+  });
+
+  it('a scoped add indexes the new file', async () => {
+    fs.writeFileSync(path.join(testDir, 'src', 'c.ts'), `export function delta() { return 4; }`);
+    const scoped = await cg.sync({ paths: ['src/c.ts'] });
+    expect(scoped.filesAdded).toBe(1);
+    expect(cg.searchNodes('delta').length).toBeGreaterThan(0);
+  });
+
+  it('scoped sync ignores paths outside the change without touching them', async () => {
+    fs.writeFileSync(path.join(testDir, 'src', 'a.ts'), `export function alpha() { return beta() + 1; }`);
+    const scoped = await cg.sync({ paths: ['src/a.ts'] });
+    expect(scoped.filesModified).toBe(1);
+    expect(scoped.filesRemoved).toBe(0);
+    // b.ts untouched and still present
+    expect(cg.searchNodes('beta').length).toBeGreaterThan(0);
+  });
+});

__tests__/watcher.test.ts

@@ -31,7 +31,7 @@ import {
 } from '../src/sync/watcher';
 import CodeGraph from '../src/index';
 
-type SyncFn = () => Promise<{ filesChanged: number; durationMs: number }>;
+type SyncFn = (paths?: string[]) => Promise<{ filesChanged: number; durationMs: number }>;
 
 /**
  * Helper to wait for a condition with timeout. Used for assertions that depend
@@ -770,4 +770,56 @@ describe('FileWatcher', () => {
       cg.unwatch();
     });
   });
+
+  describe('scoped sync fast path (#watcher-scoped)', () => {
+    it('passes the exact pending paths to syncFn for plain file events', async () => {
+      const calls: (string[] | undefined)[] = [];
+      const syncFn: SyncFn = async (paths?: string[]) => {
+        calls.push(paths);
+        return { filesChanged: 1, durationMs: 5 };
+      };
+      const watcher = newWatcher(syncFn, { debounceMs: 30 });
+      expect(watcher.start()).toBe(true);
+      fs.writeFileSync(path.join(testDir, 'src', 'a.ts'), 'export const a = 1;');
+      __emitWatchEventForTests(testDir, 'src/a.ts');
+      await new Promise((r) => setTimeout(r, 500));
+      watcher.stop();
+      expect(calls.length).toBeGreaterThan(0);
+      expect(calls[0]).toEqual(['src/a.ts']);
+    });
+
+    it('falls back to a full sync (undefined paths) after a directory removal event', async () => {
+      const calls: (string[] | undefined)[] = [];
+      const syncFn: SyncFn = async (paths?: string[]) => {
+        calls.push(paths);
+        return { filesChanged: 0, durationMs: 5 };
+      };
+      const watcher = newWatcher(syncFn, { debounceMs: 30 });
+      expect(watcher.start()).toBe(true);
+      // A non-source path that does not exist on disk = the #1285 dir-removal shape.
+      __emitWatchEventForTests(testDir, 'src/removed-dir');
+      await new Promise((r) => setTimeout(r, 500));
+      watcher.stop();
+      expect(calls.length).toBeGreaterThan(0);
+      expect(calls[0]).toBeUndefined();
+    });
+
+    it('a lone file event fires on the quick window, well before the full debounce', async () => {
+      const calls: (string[] | undefined)[] = [];
+      const syncFn: SyncFn = async (paths?: string[]) => {
+        calls.push(paths);
+        return { filesChanged: 1, durationMs: 1 };
+      };
+      // Full debounce is deliberately huge; the quick window (300ms) must win
+      // for a single pending file.
+      const watcher = newWatcher(syncFn, { debounceMs: 30_000 });
+      expect(watcher.start()).toBe(true);
+      fs.writeFileSync(path.join(testDir, 'src', 'quick.ts'), 'export const q = 1;');
+      __emitWatchEventForTests(testDir, 'src/quick.ts');
+      await new Promise((r) => setTimeout(r, 1500));
+      watcher.stop();
+      expect(calls.length).toBe(1);
+      expect(calls[0]).toEqual(['src/quick.ts']);
+    });
+  });
 });

src/extraction/index.ts

@@ -2487,7 +2487,19 @@ export class ExtractionOrchestrator {
    * changes. This works in non-git projects and catches committed changes from
    * `git pull`/`checkout`/`merge`/`rebase` that `git status` cannot see.
    */
-  async sync(onProgress?: (progress: IndexProgress) => void): Promise<SyncResult> {
+  async sync(
+    onProgress?: (progress: IndexProgress) => void,
+    /**
+     * Watcher fast path: the exact project-relative paths the OS reported as
+     * changed. When provided, reconciliation runs over ONLY these paths —
+     * per-path logic identical to the full walk (stat pre-filter, hash
+     * confirm, the #1240 removal/resurrection flow) — skipping the O(repo)
+     * scan and tracked-load. Callers must pass undefined whenever the change
+     * set is not exactly known (directory removals, event overflow): the full
+     * scan-diff remains the ground truth those cases need (#1285).
+     */
+    scopedPaths?: string[]
+  ): Promise<SyncResult> {
     await initGrammars(); // Initialize WASM runtime (grammars loaded lazily below)
     const startTime = Date.now();
     let filesChecked = 0;
@@ -2514,14 +2526,33 @@ export class ExtractionOrchestrator {
     // changes from `git pull`/`checkout`/`merge`/`rebase` — which `git status`
     // cannot see, because the working tree is clean afterward.
     const tSyncScan = Date.now();
-    const currentFiles = await scanDirectoryAsync(this.rootDir);
+    let currentFiles: string[];
+    let trackedFiles: FileRecord[];
+    if (scopedPaths && scopedPaths.length > 0) {
+      // Scoped reconcile: stat only the reported paths. filesChecked counts
+      // the PATHS examined (not the files found) — it must stay non-zero even
+      // when every scoped path was a deletion, because CodeGraph.watch()
+      // reads `filesChecked === 0 && durationMs === 0` as the
+      // lock-unavailable signature (#449).
+      const unique = [...new Set(scopedPaths)];
+      currentFiles = unique.filter((p) => fs.existsSync(path.join(this.rootDir, p)));
+      trackedFiles = [];
+      for (const p of unique) {
+        const rec = this.queries.getFileByPath(p);
+        if (rec) trackedFiles.push(rec);
+      }
+      filesChecked = unique.length;
+      if (process.env.CODEGRAPH_SYNTH_TIMINGS) console.error(`[phase-timing] sync-scoped: ${Date.now() - tSyncScan}ms (${unique.length} paths, ${trackedFiles.length} tracked)`);
+    } else {
+      currentFiles = await scanDirectoryAsync(this.rootDir);
       if (process.env.CODEGRAPH_SYNTH_TIMINGS) console.error(`[phase-timing] sync-scan: ${Date.now() - tSyncScan}ms (${currentFiles.length} files)`);
       filesChecked = currentFiles.length;
-    const currentSet = new Set(currentFiles);
 
       const tTracked = Date.now();
-    const trackedFiles = this.queries.getAllFiles();
+      trackedFiles = this.queries.getAllFiles();
       if (process.env.CODEGRAPH_SYNTH_TIMINGS) console.error(`[phase-timing] sync-tracked-load: ${Date.now() - tTracked}ms (${trackedFiles.length} tracked)`);
+    }
+    const currentSet = new Set(currentFiles);
     const trackedMap = new Map<string, FileRecord>();
     for (const f of trackedFiles) {
       trackedMap.set(f.path, f);

src/index.ts

@@ -127,6 +127,8 @@ export interface IndexOptions {
 
   /** Enable verbose logging (worker lifecycle, memory, timeouts) */
   verbose?: boolean;
+  /** Watcher fast path: reconcile ONLY these project-relative paths (see ExtractionOrchestrator.sync). */
+  paths?: string[];
 }
 
 /**
@@ -778,7 +780,7 @@ export class CodeGraph {
           try { return this.queries.isNameSegmentVocabEmpty(); } catch { return false; }
         })();
 
-        const result = await this.orchestrator.sync(options.onProgress);
+        const result = await this.orchestrator.sync(options.onProgress, options.paths);
 
         // Fold the store phase's WAL BEFORE the post-store reads below
         // (resolution reads on the main thread) — same rationale as
@@ -988,8 +990,8 @@ export class CodeGraph {
 
     this.watcher = new FileWatcher(
       this.projectRoot,
-      async () => {
-        const result = await this.sync();
+      async (paths?: string[]) => {
+        const result = await this.sync({ paths });
         // sync() returns this exact zero-shape iff it failed to acquire the
         // file lock (a real empty sync always has filesChecked > 0 because
         // scanDirectory ran). Surface that to the watcher as a typed error

src/sync/watcher.ts

@@ -59,6 +59,22 @@ const MAX_SYNC_FAILURE_RETRIES = 5;
 /** Cap on the exponential retry backoff (either mode) so it never sleeps absurdly long. */
 const MAX_RETRY_BACKOFF_MS = 30_000;
 
+/**
+ * Adaptive debounce: a pending set this small fires after the quick quiet
+ * window instead of the full debounce — a lone save (or editor + test file
+ * pair) syncs near-instantly, while larger bursts keep the full window and
+ * coalesce exactly as before.
+ */
+const QUICK_SYNC_MAX_PENDING = 2;
+const QUICK_SYNC_QUIET_MS = 300;
+
+/**
+ * Scoped-sync ceiling: above this many pending files a full scan-diff is
+ * simpler and comparably fast (a branch checkout emits thousands of events),
+ * and it self-heals anything event coalescing dropped along the way.
+ */
+const SCOPED_SYNC_MAX_PENDING = 500;
+
 /** Actionable degrade message; both exhaustion paths share it verbatim. */
 const EXHAUSTION_REASON =
   'OS watch/file limit exhausted; auto-sync disabled. Run `codegraph sync` ' +
@@ -273,6 +289,13 @@ export class FileWatcher {
   /** Test-only inert mode: started, but with no OS watcher installed. */
   private inert = false;
   private debounceTimer: ReturnType<typeof setTimeout> | null = null;
+  /**
+   * True when the pending set does NOT exactly describe the change (a
+   * directory removal's children are unknown from the event, #1285) — the
+   * next sync must be a full scan-diff. Cleared only after a successful FULL
+   * sync reconciles the tree.
+   */
+  private needsFullScan = false;
   /**
    * Files seen by the watcher since the last successful sync — populated on
    * every change event, cleared at the start of a sync, and re-populated by
@@ -314,7 +337,7 @@ export class FileWatcher {
 
   private readonly projectRoot: string;
   private readonly debounceMs: number;
-  private readonly syncFn: () => Promise<{ filesChanged: number; durationMs: number }>;
+  private readonly syncFn: (paths?: string[]) => Promise<{ filesChanged: number; durationMs: number }>;
   private readonly onSyncComplete?: WatchOptions['onSyncComplete'];
   private readonly onSyncError?: WatchOptions['onSyncError'];
   private readonly onDegraded?: WatchOptions['onDegraded'];
@@ -322,7 +345,7 @@ export class FileWatcher {
 
   constructor(
     projectRoot: string,
-    syncFn: () => Promise<{ filesChanged: number; durationMs: number }>,
+    syncFn: (paths?: string[]) => Promise<{ filesChanged: number; durationMs: number }>,
     options: WatchOptions = {}
   ) {
     this.projectRoot = projectRoot;
@@ -594,6 +617,7 @@ export class FileWatcher {
     logDebug('Non-source path removed; scheduling sync for possible directory removal', {
       path: rel,
     });
+    this.needsFullScan = true;
     this.scheduleSync();
   }
 
@@ -768,10 +792,20 @@ export class FileWatcher {
     if (this.debounceTimer) {
       clearTimeout(this.debounceTimer);
     }
+    // Adaptive quiet window: a lone save (or a pair — editor + its test file)
+    // fires fast so the graph feels instant; anything bigger keeps the full
+    // configured window so an agent's multi-file burst still coalesces into
+    // one sync exactly as before. Re-arming on each event preserves the
+    // trailing-edge semantics either way: if more events arrive inside the
+    // quick window, the reschedule sees the larger pending set and extends to
+    // the full window. Never exceeds the configured debounce (a user-lowered
+    // CODEGRAPH_WATCH_DEBOUNCE_MS stays authoritative), floor 100ms.
+    const quickMs = Math.max(100, Math.min(QUICK_SYNC_QUIET_MS, this.debounceMs));
+    const delay = this.pendingFiles.size <= QUICK_SYNC_MAX_PENDING ? quickMs : this.debounceMs;
     this.debounceTimer = setTimeout(() => {
       this.debounceTimer = null;
       this.flush();
-    }, this.debounceMs);
+    }, delay);
   }
 
   /**
@@ -808,8 +842,19 @@ export class FileWatcher {
     this.syncStartedMs = Date.now();
     this.syncing = true;
 
+    // Scoped fast path: when every pending change is a known file event, hand
+    // the exact paths to sync and skip its O(repo) scan-diff. Anything the
+    // events can't fully describe — a directory removal, an empty pending set
+    // (retry paths), or an event storm past the ceiling — runs the full
+    // scan-diff, which remains the ground truth.
+    const scoped =
+      !this.needsFullScan && this.pendingFiles.size > 0 && this.pendingFiles.size <= SCOPED_SYNC_MAX_PENDING
+        ? [...this.pendingFiles.keys()]
+        : undefined;
+
     try {
-      const result = await this.syncFn();
+      const result = await this.syncFn(scoped);
+      if (!scoped) this.needsFullScan = false;
       this.lockRetryCount = 0; // a clean sync clears any contention backoff
       this.syncFailureRetryCount = 0; // ...and any generic-failure backoff
       // Remove entries whose most recent event predates this sync — those

target-linux/.rustc_info.json

@@ -0,0 +1 @@
+{"rustc_fingerprint":4293921394394104691,"outputs":{"7971740275564407648":{"success":true,"status":"","code":0,"stdout":"___\nlib___.rlib\nlib___.so\nlib___.so\nlib___.a\nlib___.so\n/usr/local/rustup/toolchains/1.97.1-aarch64-unknown-linux-gnu\noff\npacked\nunpacked\n___\ndebug_assertions\npanic=\"unwind\"\nproc_macro\ntarget_abi=\"\"\ntarget_arch=\"aarch64\"\ntarget_endian=\"little\"\ntarget_env=\"gnu\"\ntarget_family=\"unix\"\ntarget_feature=\"neon\"\ntarget_has_atomic=\"128\"\ntarget_has_atomic=\"16\"\ntarget_has_atomic=\"32\"\ntarget_has_atomic=\"64\"\ntarget_has_atomic=\"8\"\ntarget_has_atomic=\"ptr\"\ntarget_has_atomic_primitive_alignment=\"128\"\ntarget_has_atomic_primitive_alignment=\"16\"\ntarget_has_atomic_primitive_alignment=\"32\"\ntarget_has_atomic_primitive_alignment=\"64\"\ntarget_has_atomic_primitive_alignment=\"8\"\ntarget_has_atomic_primitive_alignment=\"ptr\"\ntarget_os=\"linux\"\ntarget_pointer_width=\"64\"\ntarget_vendor=\"unknown\"\nunix\n","stderr":""},"4054631422912855907":{"success":true,"status":"","code":0,"stdout":"rustc 1.97.1 (8bab26f4f 2026-07-14)\nbinary: rustc\ncommit-hash: 8bab26f4f68e0e26f0bb7960be334d5b520ea452\ncommit-date: 2026-07-14\nhost: aarch64-unknown-linux-gnu\nrelease: 1.97.1\nLLVM version: 22.1.6\n","stderr":""}},"successes":{}}

target-linux/CACHEDIR.TAG

@@ -0,0 +1,3 @@
+Signature: 8a477f597d28d172789f06886806bc55
+# This file is a cache directory tag created by cargo.
+# For information about cache directory tags see https://bford.info/cachedir/