perf(sync): adaptive quick-fire debounce + scoped watcher sync — save-to-graph well under a second at any scale (#1397)
Two changes to the watcher path (the always-on daemon every agent session uses), which previously paid a flat 2s debounce plus a full-tree scan-diff on every save even though the OS events name the exact files: 1. Adaptive debounce: a pending set of ≤2 files fires after a 300ms quiet window; ≥3 keeps the full configured window so agent multi-file bursts coalesce exactly as before. Re-arming preserves trailing-edge semantics; a user-set CODEGRAPH_WATCH_DEBOUNCE_MS remains the authoritative upper bound (quick window never exceeds it, floor 100ms). 2. Scoped sync: watcher-triggered syncs pass their pending paths, and the reconciler stats exactly those — per-path logic identical to the full walk (stat pre-filter, hash confirm, the #1240 removal/resurrection flow) — skipping the O(repo) scan and tracked-load. Strict fallbacks keep the full scan-diff as ground truth: directory removals (#1285 — the events can't name the children), empty pending sets (retry paths), and >500-file storms (branch checkouts, which also self-heal anything event coalescing dropped). filesChecked counts examined PATHS so a deletion-only scoped sync can't mimic the #449 lock-unavailable signature. Measured (warm in-process, the daemon path): dubbo one-file sync work 512→335ms, Swift compiler (27k files) 884→385ms — save-to-fresh-graph ≈0.6-0.7s end-to-end including the quick debounce, from ~2.5-6s perceived before. Gates: scoped-vs-full dumps byte-identical on dubbo AND the Swift compiler; watcher suite 30/30 (3 new: scoped pass-through, dir-removal fallback, quick-fire timing); sync suite 34/34 (4 new scoped-parity cases incl. delete-resurrection and the lock signature); full suite 2,696 ×2 with CODEGRAPH_KERNEL_EXPECT=1. Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Fable 5
parent
157c8e735d
commit
c74e8b05e0
@@ -10,7 +10,9 @@ Follow [@getcodegraph](https://x.com/getcodegraph) on X for updates.
|
||||
|
||||
### Supercharge Claude Code, Cursor, Codex, OpenCode, Hermes Agent, Gemini, Antigravity, and Kiro with Semantic Code Intelligence
|
||||
|
||||
**Surgical context · fewer tool calls · faster answers · 100% local**
|
||||
**The fastest complete code graph · surgical context · built for how agents actually work · 100% local**
|
||||
|
||||
<img src="https://raw.githubusercontent.com/colbymchenry/codegraph/main/assets/languages/rust.svg?v=1" height="18" alt="Rust" align="top"> <sub><b>Kernel powered by Rust</b> — 20 languages parsed natively, scaling itself to your machine's cores and memory</sub>
|
||||
|
||||
### [Documentation & Website →](https://colbymchenry.github.io/codegraph/)
|
||||
|
||||
@@ -309,10 +311,26 @@ The reliable, universal payoff is **surgical context and speed**: CodeGraph coll
|
||||
|
||||
---
|
||||
|
||||
## Built for speed — the Rust kernel
|
||||
|
||||
<img align="right" src="https://raw.githubusercontent.com/colbymchenry/codegraph/main/assets/languages/rust.svg?v=1" width="80" height="80" alt="Rust">
|
||||
|
||||
CodeGraph's parsing engine is a **native Rust kernel**: 20 languages — TypeScript, JavaScript, Java, Python, Go, C, C++, Rust, C#, Ruby, PHP, Swift, Kotlin, Scala, Dart, R, Lua, Luau (Metal and CUDA ride the C++ path) — parse in compiled code with one boundary crossing per file. Every language shipped only after its graphs proved **byte-for-byte identical** to the reference engine on real repositories, from small libraries up to the Linux kernel; platforms without a prebuilt binary and files with syntax errors fall back per-file automatically, same graph either way.
|
||||
|
||||
**And it scales itself to the machine it's on.** Worker pools, parallel resolution, and analysis caches are sized from what the system actually has — real core counts (container/cgroup-aware, so a VPS that grants 2 cores gets sized for 2, not the host's 64), honestly-measured available RAM on macOS and Linux, and the measured cost of *your* project's resolution work:
|
||||
|
||||
- **On a workstation:** the full parallel pipeline — native parse workers, a multi-worker resolver pool that engages the moment it pays for itself, memory-gated analysis caches. The Swift compiler repository (27k files of Swift and C++) fresh-indexes in about 100 seconds; a one-file edit re-syncs in ~4.
|
||||
- **On a 2-core / 6GB VPS:** the same graph, from a pipeline tuned to *finish* — the Linux kernel (70k files, 2M symbols, 6.4M relationships) indexes to completion in under 12 minutes where RAM-first designs run out of memory before reaching 1%.
|
||||
- **Every day after day one:** saving a file updates the graph in well under a second — the watcher fires 300ms after a lone save and syncs exactly what changed (~0.3s of work on a 4,400-file project, ~0.4s on the 27,000-file Swift compiler repo), never re-scanning the tree. Measured against the fastest competing indexer's re-index-on-change: 2–7× faster on medium and larger repos across a 31-repo, 30-language benchmark — and the gap widens with repo size, because their cost grows with the repository and ours grows with the change.
|
||||
|
||||
---
|
||||
|
||||
## Key Features
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| **Native Rust Kernel** | Parsing and extraction run in a compiled Rust engine for 20 languages — with graphs verified byte-for-byte identical to the reference engine, and automatic per-file fallback so nothing ever breaks |
|
||||
| **Adapts to Your Machine** | Sizes its worker pools and caches from what the system actually has — real core counts (container-aware), honest available RAM, measured per-project cost. A workstation gets the full parallel pipeline; a 2-core VPS gets one tuned to finish reliably |
|
||||
| **Surgical Context** | One tool call returns entry points, related symbols, and code snippets — no slow file-by-file exploration |
|
||||
| **Full-Text Search** | Find code by name instantly across your entire codebase, powered by FTS5 |
|
||||
| **Impact Analysis** | Trace callers, callees, and the full impact radius of any symbol before making changes |
|
||||
@@ -530,7 +548,7 @@ The exact text is `src/mcp/server-instructions.ts` — the single source of trut
|
||||
└───────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
1. **Extraction** — [tree-sitter](https://tree-sitter.github.io/) parses source code into ASTs. Language-specific queries extract nodes (functions, classes, methods) and edges (calls, imports, extends, implements).
|
||||
1. **Extraction** — a native **Rust kernel** parses source with [tree-sitter](https://tree-sitter.github.io/) grammars compiled into it, extracting nodes (functions, classes, methods) and edges (calls, imports, extends, implements) for 20 languages; remaining languages and per-file fallbacks use the same extraction logic on the portable engine, producing identical graphs.
|
||||
|
||||
2. **Storage** — Everything goes into a local SQLite database (`.codegraph/codegraph.db`) with FTS5 full-text search.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user