diff --git a/CHANGELOG.md b/CHANGELOG.md index c5be7f0..eea51c7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,17 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). ## [Unreleased] +# ⚡ The Rust engine release + +**This release rebuilds CodeGraph's parsing engine as a native Rust kernel and overhauls the resolution pipeline around it — the largest performance upgrade in the project's history, with every graph verified byte-for-byte identical to the previous engine.** + +- **Native Rust parsing for 20 languages** — TypeScript, JavaScript (+TSX/JSX), Java, Python, Go, C, C++, Rust, C#, Ruby, PHP, Swift, Kotlin, Scala, Dart, R, Lua, and Luau now parse in a compiled Rust kernel (Metal and CUDA ride the C++ path). Platforms without a prebuilt binary, and individual files with syntax errors, fall back to the previous engine automatically — same graph either way, proven on repositories from small libraries to the Linux kernel. +- **Adaptive to your machine** — CodeGraph sizes its parse workers, resolver pool, and caches from what the system actually has: real core counts (container/cgroup-aware, not the host's), honest available memory on macOS and Linux, and measured per-project resolution cost. A big workstation gets the full parallel pipeline; a 2-core VPS gets a pipeline tuned to finish reliably instead of running out of memory — the Linux kernel (70k files) indexes to completion on a 2-core, 6GB machine in under 12 minutes — down from 26 at the start of this cycle. +- **Resolution is dramatically faster across the board** — adaptive parallel resolution, smarter method-candidate lookup, and memoized supertype/conformance walking. The Swift compiler repository (27k files, Swift + C++) went from over 3 minutes to about 100 seconds within this release cycle; Rust, Lua, and Java-family projects all see double-digit improvements. +- **Sync is now near-instant** — a save reaches the graph in well under a second, even at compiler scale. The always-on watcher fires after a 300ms quiet window for lone saves (bursts of edits still coalesce), and hands the exact changed paths to sync instead of re-scanning the whole tree — measured save-to-fresh-graph work of ~0.3s on a 4,400-file Java project and ~0.4s on the 27,000-file Swift compiler repository, byte-identical to a full reconciliation. + +Full details in the entries below. + ### New Features - Indexing TypeScript, TSX, JavaScript, JSX, Java, Python, Go, C, C++, Rust, C#, Ruby, PHP, Swift, Kotlin, Scala, Dart, R, Lua, and Luau projects is faster: parsing and symbol extraction now run in a native engine when a prebuilt binary is available for your platform (release bundles include one), producing exactly the same graph — verified byte-for-byte against the previous engine on real repositories, from small libraries up to vscode-, dubbo-, django-, git-, protobuf-, tokio-, rust-analyzer-, jellyfin-, rails-, symfony-, swift-nio-, kotlinx.coroutines-, ggplot2-, Kong-, Scala-3-compiler-, and Flutter-scale codebases (Lombok-generated members, C function-pointer tables, and Unreal-Engine-style macro-heavy headers included; CUDA and Metal sources ride the C++ path). The speedup is largest on resource-constrained machines like CI runners. No setup needed: platforms without the native binary, and individual files with syntax errors, automatically use the previous engine, and `CODEGRAPH_KERNEL=0` turns the native path off entirely. @@ -24,6 +35,7 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). - Indexing very large projects on multi-core machines got faster again: the parallel-resolution workers now periodically refresh their read-only database connections, which lets database housekeeping advance instead of silently building up a backlog behind long-lived readers — a backlog that was taxing the indexer's own writes. Graphs remain byte-for-byte identical; the win is largest at Linux-kernel scale on many-core machines. - Indexing on macOS now uses the machine's real memory headroom when sizing its parallel-resolution workers. macOS deliberately keeps RAM filled with reclaimable cache, so the previous free-memory reading came back tiny (~1GB on an otherwise idle machine) and silently halved the worker pool — a medium Java project's fresh index ran about 15–20% slower than the hardware allowed. Graphs remain byte-for-byte identical; the same fix also lets a memory-driven analysis cache engage fully on macOS for large C codebases. - Fresh indexing got a sizeable across-the-board speedup: during the initial build, the database's secondary lookup indexes are set aside and rebuilt once after parsing instead of being maintained row by row — the same proven trick the later linking phase already used, now applied to the whole parse lane — and the reference-resolution loop likewise stops maintaining lookup indexes it never reads, rebuilding them at the end when almost nothing is left in the table. A medium Java project's parse phase runs about 58% faster and its full fresh index about 19% faster end-to-end; a Linux-kernel-scale index that took ~15 minutes on an 8-core machine now completes in about 11, with the resolution phase alone dropping by a third. Graphs remain byte-for-byte identical, and incremental syncs are unaffected. +- Saving a file now updates the graph almost immediately: the file watcher fires after a 300ms quiet window for one or two changed files (bursts still coalesce under the full debounce, and `CODEGRAPH_WATCH_DEBOUNCE_MS` remains the upper bound), and watcher-triggered syncs reconcile exactly the changed paths instead of stat-walking the entire repository. Directory deletions and event storms still run the full scan-diff, so nothing the events can't describe is ever missed. Measured: save-to-fresh-graph sync work drops to ~0.3s on a 4,400-file project and ~0.4s on a 27,000-file one, with resulting graphs byte-for-byte identical to a full reconciliation. - Indexing Swift and other protocol/interface-heavy codebases got dramatically faster: the conformance walk that checks whether a method lives on a receiver's supertypes (protocols, base classes, extensions) now remembers its answers for the duration of each resolution batch instead of re-querying the graph for every call site — on the Swift compiler repository (27k files) that walk ran nearly a million times per index. A fresh index of that repo drops from about 185 seconds to under 100, with the graph byte-for-byte identical. Method-candidate lookup also gained a per-name owner index, so overload-heavy names (`init` in Swift, `execute` in Java) no longer pay a full candidate scan per receiver type. - Resolving method calls through local variables (`recv.method()`, Lua's `recv:method()`, R's `recv$method()`) got much cheaper on repos where the same receiver is called over and over: the declaration scan that types the receiver now remembers what it has already scanned per scope instead of re-reading the same source lines for every call site, and the regex patterns it scans with are compiled once per receiver instead of per call. Kong's fresh index drops another 8% on top of the require-resolution fix (23% cumulative), with graphs byte-for-byte identical everywhere — including Java projects, where this same scan successfully types tens of thousands of receivers. - Indexing Lua and Luau projects got a sizeable speedup: resolving each `require(...)` no longer rescans the project's entire file list four times — a per-project filename index answers the same lookup instantly, cutting per-require resolution from about a millisecond to microseconds. A fresh index of Kong (1,870 Lua files) runs about 16% faster end-to-end, with the graph byte-for-byte identical. The same housekeeping also closes a latent staleness edge where COBOL copybook lookups could keep serving a cached file list after files changed. diff --git a/README.md b/README.md index dbaa2ad..cdf3fef 100644 --- a/README.md +++ b/README.md @@ -10,7 +10,9 @@ Follow [@getcodegraph](https://x.com/getcodegraph) on X for updates. ### Supercharge Claude Code, Cursor, Codex, OpenCode, Hermes Agent, Gemini, Antigravity, and Kiro with Semantic Code Intelligence -**Surgical context · fewer tool calls · faster answers · 100% local** +**The fastest complete code graph · surgical context · built for how agents actually work · 100% local** + +Rust Kernel powered by Rust — 20 languages parsed natively, scaling itself to your machine's cores and memory ### [Documentation & Website →](https://colbymchenry.github.io/codegraph/) @@ -309,10 +311,26 @@ The reliable, universal payoff is **surgical context and speed**: CodeGraph coll --- +## Built for speed — the Rust kernel + +Rust + +CodeGraph's parsing engine is a **native Rust kernel**: 20 languages — TypeScript, JavaScript, Java, Python, Go, C, C++, Rust, C#, Ruby, PHP, Swift, Kotlin, Scala, Dart, R, Lua, Luau (Metal and CUDA ride the C++ path) — parse in compiled code with one boundary crossing per file. Every language shipped only after its graphs proved **byte-for-byte identical** to the reference engine on real repositories, from small libraries up to the Linux kernel; platforms without a prebuilt binary and files with syntax errors fall back per-file automatically, same graph either way. + +**And it scales itself to the machine it's on.** Worker pools, parallel resolution, and analysis caches are sized from what the system actually has — real core counts (container/cgroup-aware, so a VPS that grants 2 cores gets sized for 2, not the host's 64), honestly-measured available RAM on macOS and Linux, and the measured cost of *your* project's resolution work: + +- **On a workstation:** the full parallel pipeline — native parse workers, a multi-worker resolver pool that engages the moment it pays for itself, memory-gated analysis caches. The Swift compiler repository (27k files of Swift and C++) fresh-indexes in about 100 seconds; a one-file edit re-syncs in ~4. +- **On a 2-core / 6GB VPS:** the same graph, from a pipeline tuned to *finish* — the Linux kernel (70k files, 2M symbols, 6.4M relationships) indexes to completion in under 12 minutes where RAM-first designs run out of memory before reaching 1%. +- **Every day after day one:** saving a file updates the graph in well under a second — the watcher fires 300ms after a lone save and syncs exactly what changed (~0.3s of work on a 4,400-file project, ~0.4s on the 27,000-file Swift compiler repo), never re-scanning the tree. Measured against the fastest competing indexer's re-index-on-change: 2–7× faster on medium and larger repos across a 31-repo, 30-language benchmark — and the gap widens with repo size, because their cost grows with the repository and ours grows with the change. + +--- + ## Key Features | | | |---|---| +| **Native Rust Kernel** | Parsing and extraction run in a compiled Rust engine for 20 languages — with graphs verified byte-for-byte identical to the reference engine, and automatic per-file fallback so nothing ever breaks | +| **Adapts to Your Machine** | Sizes its worker pools and caches from what the system actually has — real core counts (container-aware), honest available RAM, measured per-project cost. A workstation gets the full parallel pipeline; a 2-core VPS gets one tuned to finish reliably | | **Surgical Context** | One tool call returns entry points, related symbols, and code snippets — no slow file-by-file exploration | | **Full-Text Search** | Find code by name instantly across your entire codebase, powered by FTS5 | | **Impact Analysis** | Trace callers, callees, and the full impact radius of any symbol before making changes | @@ -530,7 +548,7 @@ The exact text is `src/mcp/server-instructions.ts` — the single source of trut └───────────────────────────────────────────────────────────────────┘ ``` -1. **Extraction** — [tree-sitter](https://tree-sitter.github.io/) parses source code into ASTs. Language-specific queries extract nodes (functions, classes, methods) and edges (calls, imports, extends, implements). +1. **Extraction** — a native **Rust kernel** parses source with [tree-sitter](https://tree-sitter.github.io/) grammars compiled into it, extracting nodes (functions, classes, methods) and edges (calls, imports, extends, implements) for 20 languages; remaining languages and per-file fallbacks use the same extraction logic on the portable engine, producing identical graphs. 2. **Storage** — Everything goes into a local SQLite database (`.codegraph/codegraph.db`) with FTS5 full-text search. diff --git a/__tests__/sync.test.ts b/__tests__/sync.test.ts index 6e647a0..f26c05e 100644 --- a/__tests__/sync.test.ts +++ b/__tests__/sync.test.ts @@ -757,3 +757,76 @@ describe('Sync Module', () => { }); }); }); + +describe('Scoped sync parity (#watcher-scoped)', () => { + let testDir: string; + let cg: CodeGraph; + + const snapshot = (g: CodeGraph): string => { + // Natural-key snapshot of the whole graph, mirroring dump-graph.mjs at + // unit scale: scoped and full sync must land the DB in the same state. + const nodes = g + .searchNodes('', { limit: 100000 }) + .map((r) => r.node) + .map((n) => `${n.kind}|${n.qualifiedName}|${n.filePath}|${n.startLine}`) + .sort() + .join('\n'); + return nodes; + }; + + beforeEach(async () => { + testDir = fs.mkdtempSync(path.join(os.tmpdir(), 'codegraph-sync-scoped-')); + const srcDir = path.join(testDir, 'src'); + fs.mkdirSync(srcDir); + fs.writeFileSync(path.join(srcDir, 'a.ts'), `export function alpha() { return beta(); }`); + fs.writeFileSync(path.join(srcDir, 'b.ts'), `export function beta() { return 1; }`); + cg = CodeGraph.initSync(testDir); + await cg.indexAll(); + }); + + afterEach(() => { + cg?.destroy(); + if (fs.existsSync(testDir)) fs.rmSync(testDir, { recursive: true, force: true }); + }); + + it('a scoped modify lands the same graph as a full sync of the same edit', async () => { + fs.writeFileSync(path.join(testDir, 'src', 'b.ts'), `export function beta() { return 2; }\nexport function gamma() { return 3; }`); + const scoped = await cg.sync({ paths: ['src/b.ts'] }); + expect(scoped.filesModified).toBe(1); + const scopedSnap = snapshot(cg); + + // Re-apply the same end state through a FULL sync from the same start + // state: revert, full-sync, edit again, full-sync. + fs.writeFileSync(path.join(testDir, 'src', 'b.ts'), `export function beta() { return 1; }`); + await cg.sync(); + fs.writeFileSync(path.join(testDir, 'src', 'b.ts'), `export function beta() { return 2; }\nexport function gamma() { return 3; }`); + const full = await cg.sync(); + expect(full.filesModified).toBe(1); + expect(snapshot(cg)).toBe(scopedSnap); + }); + + it('a scoped delete removes the file and resurrects cross-file refs like a full sync', async () => { + fs.rmSync(path.join(testDir, 'src', 'b.ts')); + const scoped = await cg.sync({ paths: ['src/b.ts'] }); + expect(scoped.filesRemoved).toBe(1); + expect(scoped.filesChecked).toBe(1); // checked paths, not found files (#449 lock signature) + const gone = cg.searchNodes('beta'); + expect(gone.filter((r) => r.node.filePath === 'src/b.ts').length).toBe(0); + }); + + it('a scoped add indexes the new file', async () => { + fs.writeFileSync(path.join(testDir, 'src', 'c.ts'), `export function delta() { return 4; }`); + const scoped = await cg.sync({ paths: ['src/c.ts'] }); + expect(scoped.filesAdded).toBe(1); + expect(cg.searchNodes('delta').length).toBeGreaterThan(0); + }); + + it('scoped sync ignores paths outside the change without touching them', async () => { + fs.writeFileSync(path.join(testDir, 'src', 'a.ts'), `export function alpha() { return beta() + 1; }`); + const scoped = await cg.sync({ paths: ['src/a.ts'] }); + expect(scoped.filesModified).toBe(1); + expect(scoped.filesRemoved).toBe(0); + // b.ts untouched and still present + expect(cg.searchNodes('beta').length).toBeGreaterThan(0); + }); +}); diff --git a/__tests__/watcher.test.ts b/__tests__/watcher.test.ts index 10e1b60..f493a96 100644 --- a/__tests__/watcher.test.ts +++ b/__tests__/watcher.test.ts @@ -31,7 +31,7 @@ import { } from '../src/sync/watcher'; import CodeGraph from '../src/index'; -type SyncFn = () => Promise<{ filesChanged: number; durationMs: number }>; +type SyncFn = (paths?: string[]) => Promise<{ filesChanged: number; durationMs: number }>; /** * Helper to wait for a condition with timeout. Used for assertions that depend @@ -770,4 +770,56 @@ describe('FileWatcher', () => { cg.unwatch(); }); }); + + describe('scoped sync fast path (#watcher-scoped)', () => { + it('passes the exact pending paths to syncFn for plain file events', async () => { + const calls: (string[] | undefined)[] = []; + const syncFn: SyncFn = async (paths?: string[]) => { + calls.push(paths); + return { filesChanged: 1, durationMs: 5 }; + }; + const watcher = newWatcher(syncFn, { debounceMs: 30 }); + expect(watcher.start()).toBe(true); + fs.writeFileSync(path.join(testDir, 'src', 'a.ts'), 'export const a = 1;'); + __emitWatchEventForTests(testDir, 'src/a.ts'); + await new Promise((r) => setTimeout(r, 500)); + watcher.stop(); + expect(calls.length).toBeGreaterThan(0); + expect(calls[0]).toEqual(['src/a.ts']); + }); + + it('falls back to a full sync (undefined paths) after a directory removal event', async () => { + const calls: (string[] | undefined)[] = []; + const syncFn: SyncFn = async (paths?: string[]) => { + calls.push(paths); + return { filesChanged: 0, durationMs: 5 }; + }; + const watcher = newWatcher(syncFn, { debounceMs: 30 }); + expect(watcher.start()).toBe(true); + // A non-source path that does not exist on disk = the #1285 dir-removal shape. + __emitWatchEventForTests(testDir, 'src/removed-dir'); + await new Promise((r) => setTimeout(r, 500)); + watcher.stop(); + expect(calls.length).toBeGreaterThan(0); + expect(calls[0]).toBeUndefined(); + }); + + it('a lone file event fires on the quick window, well before the full debounce', async () => { + const calls: (string[] | undefined)[] = []; + const syncFn: SyncFn = async (paths?: string[]) => { + calls.push(paths); + return { filesChanged: 1, durationMs: 1 }; + }; + // Full debounce is deliberately huge; the quick window (300ms) must win + // for a single pending file. + const watcher = newWatcher(syncFn, { debounceMs: 30_000 }); + expect(watcher.start()).toBe(true); + fs.writeFileSync(path.join(testDir, 'src', 'quick.ts'), 'export const q = 1;'); + __emitWatchEventForTests(testDir, 'src/quick.ts'); + await new Promise((r) => setTimeout(r, 1500)); + watcher.stop(); + expect(calls.length).toBe(1); + expect(calls[0]).toEqual(['src/quick.ts']); + }); + }); }); diff --git a/src/extraction/index.ts b/src/extraction/index.ts index 3c158a1..e89f73b 100644 --- a/src/extraction/index.ts +++ b/src/extraction/index.ts @@ -2487,7 +2487,19 @@ export class ExtractionOrchestrator { * changes. This works in non-git projects and catches committed changes from * `git pull`/`checkout`/`merge`/`rebase` that `git status` cannot see. */ - async sync(onProgress?: (progress: IndexProgress) => void): Promise { + async sync( + onProgress?: (progress: IndexProgress) => void, + /** + * Watcher fast path: the exact project-relative paths the OS reported as + * changed. When provided, reconciliation runs over ONLY these paths — + * per-path logic identical to the full walk (stat pre-filter, hash + * confirm, the #1240 removal/resurrection flow) — skipping the O(repo) + * scan and tracked-load. Callers must pass undefined whenever the change + * set is not exactly known (directory removals, event overflow): the full + * scan-diff remains the ground truth those cases need (#1285). + */ + scopedPaths?: string[] + ): Promise { await initGrammars(); // Initialize WASM runtime (grammars loaded lazily below) const startTime = Date.now(); let filesChecked = 0; @@ -2514,14 +2526,33 @@ export class ExtractionOrchestrator { // changes from `git pull`/`checkout`/`merge`/`rebase` — which `git status` // cannot see, because the working tree is clean afterward. const tSyncScan = Date.now(); - const currentFiles = await scanDirectoryAsync(this.rootDir); - if (process.env.CODEGRAPH_SYNTH_TIMINGS) console.error(`[phase-timing] sync-scan: ${Date.now() - tSyncScan}ms (${currentFiles.length} files)`); - filesChecked = currentFiles.length; - const currentSet = new Set(currentFiles); + let currentFiles: string[]; + let trackedFiles: FileRecord[]; + if (scopedPaths && scopedPaths.length > 0) { + // Scoped reconcile: stat only the reported paths. filesChecked counts + // the PATHS examined (not the files found) — it must stay non-zero even + // when every scoped path was a deletion, because CodeGraph.watch() + // reads `filesChecked === 0 && durationMs === 0` as the + // lock-unavailable signature (#449). + const unique = [...new Set(scopedPaths)]; + currentFiles = unique.filter((p) => fs.existsSync(path.join(this.rootDir, p))); + trackedFiles = []; + for (const p of unique) { + const rec = this.queries.getFileByPath(p); + if (rec) trackedFiles.push(rec); + } + filesChecked = unique.length; + if (process.env.CODEGRAPH_SYNTH_TIMINGS) console.error(`[phase-timing] sync-scoped: ${Date.now() - tSyncScan}ms (${unique.length} paths, ${trackedFiles.length} tracked)`); + } else { + currentFiles = await scanDirectoryAsync(this.rootDir); + if (process.env.CODEGRAPH_SYNTH_TIMINGS) console.error(`[phase-timing] sync-scan: ${Date.now() - tSyncScan}ms (${currentFiles.length} files)`); + filesChecked = currentFiles.length; - const tTracked = Date.now(); - const trackedFiles = this.queries.getAllFiles(); - if (process.env.CODEGRAPH_SYNTH_TIMINGS) console.error(`[phase-timing] sync-tracked-load: ${Date.now() - tTracked}ms (${trackedFiles.length} tracked)`); + const tTracked = Date.now(); + trackedFiles = this.queries.getAllFiles(); + if (process.env.CODEGRAPH_SYNTH_TIMINGS) console.error(`[phase-timing] sync-tracked-load: ${Date.now() - tTracked}ms (${trackedFiles.length} tracked)`); + } + const currentSet = new Set(currentFiles); const trackedMap = new Map(); for (const f of trackedFiles) { trackedMap.set(f.path, f); diff --git a/src/index.ts b/src/index.ts index 56b010b..461ff47 100644 --- a/src/index.ts +++ b/src/index.ts @@ -127,6 +127,8 @@ export interface IndexOptions { /** Enable verbose logging (worker lifecycle, memory, timeouts) */ verbose?: boolean; + /** Watcher fast path: reconcile ONLY these project-relative paths (see ExtractionOrchestrator.sync). */ + paths?: string[]; } /** @@ -778,7 +780,7 @@ export class CodeGraph { try { return this.queries.isNameSegmentVocabEmpty(); } catch { return false; } })(); - const result = await this.orchestrator.sync(options.onProgress); + const result = await this.orchestrator.sync(options.onProgress, options.paths); // Fold the store phase's WAL BEFORE the post-store reads below // (resolution reads on the main thread) — same rationale as @@ -988,8 +990,8 @@ export class CodeGraph { this.watcher = new FileWatcher( this.projectRoot, - async () => { - const result = await this.sync(); + async (paths?: string[]) => { + const result = await this.sync({ paths }); // sync() returns this exact zero-shape iff it failed to acquire the // file lock (a real empty sync always has filesChecked > 0 because // scanDirectory ran). Surface that to the watcher as a typed error diff --git a/src/sync/watcher.ts b/src/sync/watcher.ts index 652a673..fed6ea6 100644 --- a/src/sync/watcher.ts +++ b/src/sync/watcher.ts @@ -59,6 +59,22 @@ const MAX_SYNC_FAILURE_RETRIES = 5; /** Cap on the exponential retry backoff (either mode) so it never sleeps absurdly long. */ const MAX_RETRY_BACKOFF_MS = 30_000; +/** + * Adaptive debounce: a pending set this small fires after the quick quiet + * window instead of the full debounce — a lone save (or editor + test file + * pair) syncs near-instantly, while larger bursts keep the full window and + * coalesce exactly as before. + */ +const QUICK_SYNC_MAX_PENDING = 2; +const QUICK_SYNC_QUIET_MS = 300; + +/** + * Scoped-sync ceiling: above this many pending files a full scan-diff is + * simpler and comparably fast (a branch checkout emits thousands of events), + * and it self-heals anything event coalescing dropped along the way. + */ +const SCOPED_SYNC_MAX_PENDING = 500; + /** Actionable degrade message; both exhaustion paths share it verbatim. */ const EXHAUSTION_REASON = 'OS watch/file limit exhausted; auto-sync disabled. Run `codegraph sync` ' + @@ -273,6 +289,13 @@ export class FileWatcher { /** Test-only inert mode: started, but with no OS watcher installed. */ private inert = false; private debounceTimer: ReturnType | null = null; + /** + * True when the pending set does NOT exactly describe the change (a + * directory removal's children are unknown from the event, #1285) — the + * next sync must be a full scan-diff. Cleared only after a successful FULL + * sync reconciles the tree. + */ + private needsFullScan = false; /** * Files seen by the watcher since the last successful sync — populated on * every change event, cleared at the start of a sync, and re-populated by @@ -314,7 +337,7 @@ export class FileWatcher { private readonly projectRoot: string; private readonly debounceMs: number; - private readonly syncFn: () => Promise<{ filesChanged: number; durationMs: number }>; + private readonly syncFn: (paths?: string[]) => Promise<{ filesChanged: number; durationMs: number }>; private readonly onSyncComplete?: WatchOptions['onSyncComplete']; private readonly onSyncError?: WatchOptions['onSyncError']; private readonly onDegraded?: WatchOptions['onDegraded']; @@ -322,7 +345,7 @@ export class FileWatcher { constructor( projectRoot: string, - syncFn: () => Promise<{ filesChanged: number; durationMs: number }>, + syncFn: (paths?: string[]) => Promise<{ filesChanged: number; durationMs: number }>, options: WatchOptions = {} ) { this.projectRoot = projectRoot; @@ -594,6 +617,7 @@ export class FileWatcher { logDebug('Non-source path removed; scheduling sync for possible directory removal', { path: rel, }); + this.needsFullScan = true; this.scheduleSync(); } @@ -768,10 +792,20 @@ export class FileWatcher { if (this.debounceTimer) { clearTimeout(this.debounceTimer); } + // Adaptive quiet window: a lone save (or a pair — editor + its test file) + // fires fast so the graph feels instant; anything bigger keeps the full + // configured window so an agent's multi-file burst still coalesces into + // one sync exactly as before. Re-arming on each event preserves the + // trailing-edge semantics either way: if more events arrive inside the + // quick window, the reschedule sees the larger pending set and extends to + // the full window. Never exceeds the configured debounce (a user-lowered + // CODEGRAPH_WATCH_DEBOUNCE_MS stays authoritative), floor 100ms. + const quickMs = Math.max(100, Math.min(QUICK_SYNC_QUIET_MS, this.debounceMs)); + const delay = this.pendingFiles.size <= QUICK_SYNC_MAX_PENDING ? quickMs : this.debounceMs; this.debounceTimer = setTimeout(() => { this.debounceTimer = null; this.flush(); - }, this.debounceMs); + }, delay); } /** @@ -808,8 +842,19 @@ export class FileWatcher { this.syncStartedMs = Date.now(); this.syncing = true; + // Scoped fast path: when every pending change is a known file event, hand + // the exact paths to sync and skip its O(repo) scan-diff. Anything the + // events can't fully describe — a directory removal, an empty pending set + // (retry paths), or an event storm past the ceiling — runs the full + // scan-diff, which remains the ground truth. + const scoped = + !this.needsFullScan && this.pendingFiles.size > 0 && this.pendingFiles.size <= SCOPED_SYNC_MAX_PENDING + ? [...this.pendingFiles.keys()] + : undefined; + try { - const result = await this.syncFn(); + const result = await this.syncFn(scoped); + if (!scoped) this.needsFullScan = false; this.lockRetryCount = 0; // a clean sync clears any contention backoff this.syncFailureRetryCount = 0; // ...and any generic-failure backoff // Remove entries whose most recent event predates this sync — those diff --git a/target-linux/.rustc_info.json b/target-linux/.rustc_info.json new file mode 100644 index 0000000..73f9b70 --- /dev/null +++ b/target-linux/.rustc_info.json @@ -0,0 +1 @@ +{"rustc_fingerprint":4293921394394104691,"outputs":{"7971740275564407648":{"success":true,"status":"","code":0,"stdout":"___\nlib___.rlib\nlib___.so\nlib___.so\nlib___.a\nlib___.so\n/usr/local/rustup/toolchains/1.97.1-aarch64-unknown-linux-gnu\noff\npacked\nunpacked\n___\ndebug_assertions\npanic=\"unwind\"\nproc_macro\ntarget_abi=\"\"\ntarget_arch=\"aarch64\"\ntarget_endian=\"little\"\ntarget_env=\"gnu\"\ntarget_family=\"unix\"\ntarget_feature=\"neon\"\ntarget_has_atomic=\"128\"\ntarget_has_atomic=\"16\"\ntarget_has_atomic=\"32\"\ntarget_has_atomic=\"64\"\ntarget_has_atomic=\"8\"\ntarget_has_atomic=\"ptr\"\ntarget_has_atomic_primitive_alignment=\"128\"\ntarget_has_atomic_primitive_alignment=\"16\"\ntarget_has_atomic_primitive_alignment=\"32\"\ntarget_has_atomic_primitive_alignment=\"64\"\ntarget_has_atomic_primitive_alignment=\"8\"\ntarget_has_atomic_primitive_alignment=\"ptr\"\ntarget_os=\"linux\"\ntarget_pointer_width=\"64\"\ntarget_vendor=\"unknown\"\nunix\n","stderr":""},"4054631422912855907":{"success":true,"status":"","code":0,"stdout":"rustc 1.97.1 (8bab26f4f 2026-07-14)\nbinary: rustc\ncommit-hash: 8bab26f4f68e0e26f0bb7960be334d5b520ea452\ncommit-date: 2026-07-14\nhost: aarch64-unknown-linux-gnu\nrelease: 1.97.1\nLLVM version: 22.1.6\n","stderr":""}},"successes":{}} \ No newline at end of file diff --git a/target-linux/CACHEDIR.TAG b/target-linux/CACHEDIR.TAG new file mode 100644 index 0000000..20d7c31 --- /dev/null +++ b/target-linux/CACHEDIR.TAG @@ -0,0 +1,3 @@ +Signature: 8a477f597d28d172789f06886806bc55 +# This file is a cache directory tag created by cargo. +# For information about cache directory tags see https://bford.info/cachedir/