feat: Add file watcher with debounced auto-sync and comprehensive test coverage

Addresses the need for automatic graph synchronization on file changes. Implements FileWatcher using native OS file events (FSEvents/inotify/ReadDirectoryChangesW) with 2-second debouncing to prevent thrashing on rapid saves. Filters changes against include/exclude patterns and ignores .codegraph directory modifications. Integrates with CodeGraph API (watch/unwatch/isWatching methods) and MCP server for automatic activation. Updates documentation to reflect shift from semantic to full-text search and removal of manual hook installation requirements.
This commit is contained in:
Colby McHenry
2026-04-07 16:02:15 -05:00
parent 453c39d774
commit 3da5c96a0b
6 changed files with 599 additions and 30 deletions
+30 -24
View File
@@ -146,8 +146,8 @@ One tool call returns everything Claude needs—entry points, related symbols, a
</td>
<td width="33%" valign="top">
### 🔍 Semantic Search
Find code by meaning, not just text. Search for "authentication" and find `login`, `validateToken`, `AuthService`—even with different naming conventions.
### 🔍 Full-Text Search
Find code by name across your entire codebase instantly. Search for "auth" and find `authenticate`, `AuthService`, `validateToken` — powered by FTS5.
</td>
<td width="33%" valign="top">
@@ -173,7 +173,7 @@ No data leaves your machine. No API keys. No external services. Everything runs
<td width="33%" valign="top">
### ⚡ Always Fresh
Claude Code hooks automatically sync the index as you work. Your code intelligence is always up to date.
The MCP server watches your files and auto-syncs on save — debounced, filtered to source files only, zero config. Your code intelligence is always up to date.
</td>
</tr>
@@ -181,6 +181,12 @@ Claude Code hooks automatically sync the index as you work. Your code intelligen
---
## 📋 Requirements
- **Node.js >= 18.0.0**
---
## 🎯 Quick Start
### 1. Run the Installer
@@ -190,11 +196,10 @@ npx @colbymchenry/codegraph
```
The interactive installer will:
- Prompt to install `codegraph` globally (needed for hooks & MCP server to work)
- Prompt to install `codegraph` globally (needed for the MCP server to work)
- Configure the MCP server in `~/.claude.json`
- Set up auto-allow permissions for CodeGraph tools
- Add global instructions to `~/.claude/CLAUDE.md` (teaches Claude how to use CodeGraph)
- Install Claude Code hooks for automatic index syncing
- Optionally initialize your current project
### 2. Restart Claude Code
@@ -298,12 +303,6 @@ At the start of a session, ask the user if they'd like to initialize CodeGraph:
---
## 📋 Requirements
- Node.js >= 18.0.0
---
## 💻 CLI Usage
```bash
@@ -334,13 +333,12 @@ npx @colbymchenry/codegraph # Run via npx (no global install needed)
```
The installer will:
1. Prompt to install `codegraph` globally (needed for hooks & MCP server)
1. Prompt to install `codegraph` globally (needed for the MCP server)
2. Ask for installation location (global `~/.claude` or local `./.claude`)
3. Optionally set up auto-allow permissions
4. Configure the MCP server in `claude.json`
5. Add global instructions to `~/.claude/CLAUDE.md` (teaches Claude how to use CodeGraph)
6. Install Claude Code hooks for automatic index syncing
7. For local installs: initialize and index the current project
6. For local installs: initialize and index the current project
### `codegraph init [path]`
@@ -422,7 +420,7 @@ codegraph files --json # Output as JSON
### `codegraph context <task>`
Build relevant code context for a task. Uses semantic search to find entry points, then expands through the graph to find related code.
Build relevant code context for a task. Uses full-text search to find entry points, then expands through the graph to find related code.
```bash
codegraph context "fix checkout bug"
@@ -589,9 +587,13 @@ const context = await cg.buildContext('fix login bug', {
// Get impact radius
const impact = cg.getImpactRadius(node.id, 2);
// Sync changes
// Sync changes manually
const syncResult = await cg.sync();
// Or watch for changes and auto-sync
cg.watch(); // uses native OS file events, debounced
cg.unwatch(); // stop watching
// Clean up
cg.close();
```
@@ -615,7 +617,6 @@ All data is stored in a local SQLite database (`.codegraph/codegraph.db`):
- **edges** table: Relationships between nodes
- **files** table: File tracking for incremental updates
- **unresolved_refs** table: References pending resolution
- **vectors** table: Embeddings stored as BLOBs for semantic search
- **nodes_fts**: FTS5 virtual table for full-text search
- **schema_versions** table: Schema version tracking
- **project_metadata** table: Project-level key-value metadata
@@ -629,13 +630,17 @@ After extraction, CodeGraph resolves references:
3. Link class inheritance and interface implementations
4. Apply framework-specific patterns (Express routes, etc.)
### 4. Semantic Search
### 4. File Watching
CodeGraph uses local embeddings (via [@xenova/transformers](https://github.com/xenova/transformers.js)) to enable semantic search:
The MCP server automatically watches your project for file changes using native OS file events (FSEvents on macOS, inotify on Linux, ReadDirectoryChangesW on Windows):
1. Code symbols are embedded using a transformer model
2. Queries are embedded and compared using cosine similarity
3. Results are ranked by relevance
1. File saves are detected instantly via OS-level events — no polling
2. Changes are **debounced** (2-second quiet window) so rapid saves don't thrash
3. Only source files matching your include/exclude patterns trigger a sync
4. Build outputs, node_modules, and `.codegraph/` changes are ignored
5. Incremental sync runs automatically — only changed files are re-parsed
No configuration needed. The graph stays fresh as you code.
### 5. Graph Queries
@@ -650,7 +655,7 @@ The graph structure enables powerful queries:
When you request context for a task:
1. Semantic search finds relevant entry points
1. FTS search finds relevant entry points
2. Graph traversal expands to related code
3. Code snippets are extracted
4. Results are formatted for AI consumption
@@ -730,7 +735,8 @@ Run `codegraph init` in your project directory first.
### Missing symbols in search
- Run `codegraph sync` to pick up recent changes
- The MCP server auto-syncs on file changes — wait a couple seconds after saving
- Run `codegraph sync` manually if needed
- Check if the file's language is supported
- Verify the file isn't excluded by config patterns
+291
View File
@@ -0,0 +1,291 @@
/**
* FileWatcher Tests
*
* Tests for the file watcher that auto-syncs on changes.
*/
import { describe, it, expect, beforeEach, afterEach, vi } from 'vitest';
import * as fs from 'fs';
import * as path from 'path';
import * as os from 'os';
import { FileWatcher } from '../src/sync/watcher';
import type { CodeGraphConfig } from '../src/types';
import CodeGraph from '../src/index';
/**
* Helper to wait for a condition with timeout
*/
function waitFor(
condition: () => boolean,
timeoutMs = 10000,
intervalMs = 100
): Promise<void> {
return new Promise((resolve, reject) => {
const start = Date.now();
const check = () => {
if (condition()) return resolve();
if (Date.now() - start > timeoutMs) return reject(new Error('waitFor timed out'));
setTimeout(check, intervalMs);
};
check();
});
}
describe('FileWatcher', () => {
let testDir: string;
const baseConfig: CodeGraphConfig = {
version: 1,
rootDir: '.',
include: ['**/*.ts', '**/*.js'],
exclude: ['**/node_modules/**', '**/dist/**'],
languages: [],
frameworks: [],
maxFileSize: 1024 * 1024,
extractDocstrings: true,
trackCallSites: true,
};
beforeEach(() => {
testDir = fs.mkdtempSync(path.join(os.tmpdir(), 'codegraph-watcher-'));
// Create a source file so the directory isn't empty
const srcDir = path.join(testDir, 'src');
fs.mkdirSync(srcDir);
fs.writeFileSync(path.join(srcDir, 'index.ts'), 'export const x = 1;');
});
afterEach(() => {
if (fs.existsSync(testDir)) {
fs.rmSync(testDir, { recursive: true, force: true });
}
});
describe('start/stop lifecycle', () => {
it('should start and stop without errors', () => {
const syncFn = vi.fn().mockResolvedValue({ filesChanged: 0, durationMs: 0 });
const watcher = new FileWatcher(testDir, baseConfig, syncFn);
const started = watcher.start();
expect(started).toBe(true);
expect(watcher.isActive()).toBe(true);
watcher.stop();
expect(watcher.isActive()).toBe(false);
});
it('should be idempotent on double start', () => {
const syncFn = vi.fn().mockResolvedValue({ filesChanged: 0, durationMs: 0 });
const watcher = new FileWatcher(testDir, baseConfig, syncFn);
expect(watcher.start()).toBe(true);
expect(watcher.start()).toBe(true); // Should not throw
expect(watcher.isActive()).toBe(true);
watcher.stop();
});
it('should be idempotent on double stop', () => {
const syncFn = vi.fn().mockResolvedValue({ filesChanged: 0, durationMs: 0 });
const watcher = new FileWatcher(testDir, baseConfig, syncFn);
watcher.start();
watcher.stop();
watcher.stop(); // Should not throw
expect(watcher.isActive()).toBe(false);
});
});
describe('debounced sync', () => {
it('should trigger sync after file change', async () => {
const syncFn = vi.fn().mockResolvedValue({ filesChanged: 1, durationMs: 10 });
const watcher = new FileWatcher(testDir, baseConfig, syncFn, { debounceMs: 200 });
watcher.start();
// Create a new file
fs.writeFileSync(path.join(testDir, 'src', 'new.ts'), 'export const y = 2;');
// Wait for debounced sync to fire
await waitFor(() => syncFn.mock.calls.length > 0, 5000);
expect(syncFn).toHaveBeenCalled();
watcher.stop();
});
it('should debounce rapid changes into a single sync', async () => {
const syncFn = vi.fn().mockResolvedValue({ filesChanged: 1, durationMs: 10 });
const watcher = new FileWatcher(testDir, baseConfig, syncFn, { debounceMs: 500 });
watcher.start();
// Rapid-fire changes
for (let i = 0; i < 5; i++) {
fs.writeFileSync(
path.join(testDir, 'src', `file${i}.ts`),
`export const v${i} = ${i};`
);
await new Promise((r) => setTimeout(r, 50));
}
// Wait for the single debounced sync
await waitFor(() => syncFn.mock.calls.length > 0, 5000);
// Should have been called once (debounced), not 5 times
expect(syncFn.mock.calls.length).toBe(1);
watcher.stop();
});
});
describe('filtering', () => {
it('should ignore files not matching include patterns', async () => {
const syncFn = vi.fn().mockResolvedValue({ filesChanged: 0, durationMs: 0 });
const watcher = new FileWatcher(testDir, baseConfig, syncFn, { debounceMs: 200 });
watcher.start();
// Let watcher settle — fs.watch may fire residual events from beforeEach
await new Promise((r) => setTimeout(r, 400));
syncFn.mockClear();
// Create a file that doesn't match include patterns
fs.writeFileSync(path.join(testDir, 'src', 'readme.md'), '# Hello');
// Wait a bit longer than debounce — sync should NOT trigger
await new Promise((r) => setTimeout(r, 500));
expect(syncFn).not.toHaveBeenCalled();
watcher.stop();
});
it('should ignore .codegraph directory changes', async () => {
const syncFn = vi.fn().mockResolvedValue({ filesChanged: 0, durationMs: 0 });
const watcher = new FileWatcher(testDir, baseConfig, syncFn, { debounceMs: 200 });
watcher.start();
// Let watcher settle — fs.watch may fire residual events from beforeEach
await new Promise((r) => setTimeout(r, 400));
syncFn.mockClear();
// Simulate a .codegraph directory change
const cgDir = path.join(testDir, '.codegraph');
fs.mkdirSync(cgDir, { recursive: true });
fs.writeFileSync(path.join(cgDir, 'db.sqlite'), 'fake');
// Wait — sync should NOT trigger
await new Promise((r) => setTimeout(r, 500));
expect(syncFn).not.toHaveBeenCalled();
watcher.stop();
});
});
describe('callbacks', () => {
it('should call onSyncComplete after successful sync', async () => {
const syncFn = vi.fn().mockResolvedValue({ filesChanged: 2, durationMs: 50 });
const onSyncComplete = vi.fn();
const watcher = new FileWatcher(testDir, baseConfig, syncFn, {
debounceMs: 200,
onSyncComplete,
});
watcher.start();
fs.writeFileSync(path.join(testDir, 'src', 'test.ts'), 'export const z = 3;');
await waitFor(() => onSyncComplete.mock.calls.length > 0, 5000);
expect(onSyncComplete).toHaveBeenCalledWith({ filesChanged: 2, durationMs: 50 });
watcher.stop();
});
it('should call onSyncError when sync throws', async () => {
const syncFn = vi.fn().mockRejectedValue(new Error('sync failed'));
const onSyncError = vi.fn();
const watcher = new FileWatcher(testDir, baseConfig, syncFn, {
debounceMs: 200,
onSyncError,
});
watcher.start();
fs.writeFileSync(path.join(testDir, 'src', 'test.ts'), 'export const z = 3;');
await waitFor(() => onSyncError.mock.calls.length > 0, 5000);
expect(onSyncError).toHaveBeenCalled();
expect(onSyncError.mock.calls[0]![0]).toBeInstanceOf(Error);
watcher.stop();
});
});
describe('CodeGraph integration', () => {
let cg: CodeGraph;
afterEach(() => {
if (cg) cg.close();
});
it('should watch and unwatch via CodeGraph API', async () => {
cg = CodeGraph.initSync(testDir, {
config: { include: ['**/*.ts'], exclude: [] },
});
await cg.indexAll();
expect(cg.isWatching()).toBe(false);
const started = cg.watch({ debounceMs: 200 });
expect(started).toBe(true);
expect(cg.isWatching()).toBe(true);
cg.unwatch();
expect(cg.isWatching()).toBe(false);
});
it('should stop watching on close', async () => {
cg = CodeGraph.initSync(testDir, {
config: { include: ['**/*.ts'], exclude: [] },
});
await cg.indexAll();
cg.watch({ debounceMs: 200 });
expect(cg.isWatching()).toBe(true);
cg.close();
// After close, isWatching should be false
// (we can't call isWatching after close since DB is closed,
// but we verify no errors are thrown)
});
it('should auto-sync when files change while watching', async () => {
cg = CodeGraph.initSync(testDir, {
config: { include: ['**/*.ts'], exclude: [] },
});
await cg.indexAll();
const initialStats = cg.getStats();
const initialNodes = initialStats.nodeCount;
cg.watch({ debounceMs: 300 });
// Add a new file with a function
fs.writeFileSync(
path.join(testDir, 'src', 'added.ts'),
'export function added() { return 42; }'
);
// Wait for auto-sync to pick it up
await waitFor(() => {
const stats = cg.getStats();
return stats.nodeCount > initialNodes;
}, 10000);
// The new function should be in the graph
const results = cg.searchNodes('added');
expect(results.length).toBeGreaterThan(0);
cg.unwatch();
});
});
});
+53
View File
@@ -48,6 +48,7 @@ import {
import { GraphTraverser, GraphQueryManager } from './graph';
import { ContextBuilder, createContextBuilder } from './context';
import { Mutex, FileLock } from './utils';
import { FileWatcher, WatchOptions } from './sync';
// Re-export types for consumers
export * from './types';
@@ -77,6 +78,7 @@ export {
defaultLogger,
} from './errors';
export { Mutex, FileLock, processInBatches, debounce, throttle, MemoryMonitor } from './utils';
export { FileWatcher, WatchOptions } from './sync';
export { MCPServer } from './mcp';
/**
@@ -140,6 +142,9 @@ export class CodeGraph {
// File lock for preventing concurrent writes across processes (CLI, MCP, git hooks)
private fileLock: FileLock;
// File watcher for auto-sync on file changes
private watcher: FileWatcher | null = null;
private constructor(
db: DatabaseConnection,
queries: QueryBuilder,
@@ -319,6 +324,7 @@ export class CodeGraph {
* Close the CodeGraph instance and release resources
*/
close(): void {
this.unwatch();
// Release file lock if held
this.fileLock.release();
this.db.close();
@@ -491,6 +497,53 @@ export class CodeGraph {
return this.indexMutex.isLocked();
}
// ===========================================================================
// File Watching
// ===========================================================================
/**
* Start watching for file changes and auto-syncing.
*
* Uses native OS file events (FSEvents on macOS, inotify on Linux 19+,
* ReadDirectoryChangesW on Windows) with debouncing to avoid thrashing.
*
* @param options - Watch options (debounce delay, callbacks)
* @returns true if watching started successfully
*/
watch(options: WatchOptions = {}): boolean {
if (this.watcher?.isActive()) return true;
this.watcher = new FileWatcher(
this.projectRoot,
this.config,
async () => {
const result = await this.sync();
const filesChanged = result.filesAdded + result.filesModified + result.filesRemoved;
return { filesChanged, durationMs: result.durationMs };
},
options
);
return this.watcher.start();
}
/**
* Stop watching for file changes.
*/
unwatch(): void {
if (this.watcher) {
this.watcher.stop();
this.watcher = null;
}
}
/**
* Check if the file watcher is active.
*/
isWatching(): boolean {
return this.watcher?.isActive() ?? false;
}
/**
* Get files that have changed since last index
*/
+27
View File
@@ -116,6 +116,7 @@ export class MCPServer {
try {
this.cg = await CodeGraph.open(resolvedRoot);
this.toolHandler.setDefaultCodeGraph(this.cg);
this.startWatching();
} catch (err) {
// Log the error so transient failures are diagnosable (see issue #47)
const msg = err instanceof Error ? err.message : String(err);
@@ -147,11 +148,37 @@ export class MCPServer {
this.cg = CodeGraph.openSync(resolvedRoot);
this.projectPath = resolvedRoot;
this.toolHandler.setDefaultCodeGraph(this.cg);
this.startWatching();
} catch {
// Still failing — will retry on next tool call
}
}
/**
* Start file watching on the active CodeGraph instance.
* Logs sync activity to stderr for diagnostics.
*/
private startWatching(): void {
if (!this.cg) return;
const started = this.cg.watch({
onSyncComplete: (result) => {
if (result.filesChanged > 0) {
process.stderr.write(
`[CodeGraph MCP] Auto-synced ${result.filesChanged} file(s) in ${result.durationMs}ms\n`
);
}
},
onSyncError: (err) => {
process.stderr.write(`[CodeGraph MCP] Auto-sync error: ${err.message}\n`);
},
});
if (started) {
process.stderr.write('[CodeGraph MCP] File watcher active — graph will auto-sync on changes\n');
}
}
/**
* Stop the server
*/
+2 -6
View File
@@ -4,14 +4,10 @@
* Provides synchronization functionality for keeping the code graph
* up-to-date with file system changes.
*
* Note: Git hooks functionality has been removed. CodeGraph sync is now
* triggered through codegraph's Claude Code hooks integration instead.
*
* Components:
* - FileWatcher: Debounced fs.watch that auto-triggers sync on file changes
* - Content hashing for change detection (in extraction module)
* - Incremental reindexing (in extraction module)
*/
// This module is kept for potential future sync-related exports
// Currently all sync functionality is in the extraction module
export {};
export { FileWatcher, WatchOptions } from './watcher';
+196
View File
@@ -0,0 +1,196 @@
/**
* File Watcher
*
* Watches the project directory for file changes and triggers
* debounced sync operations to keep the code graph up-to-date.
*
* Uses Node.js native fs.watch with recursive mode (macOS FSEvents,
* Windows ReadDirectoryChangesW, Linux inotify on Node 19+).
*/
import * as fs from 'fs';
import { CodeGraphConfig } from '../types';
import { shouldIncludeFile } from '../extraction';
import { logDebug, logWarn } from '../errors';
import { normalizePath } from '../utils';
/**
* Options for the file watcher
*/
export interface WatchOptions {
/**
* Debounce delay in milliseconds.
* After the last file change, wait this long before triggering sync.
* Default: 2000ms
*/
debounceMs?: number;
/**
* Callback when a sync completes (for logging/diagnostics).
*/
onSyncComplete?: (result: { filesChanged: number; durationMs: number }) => void;
/**
* Callback when a sync errors (for logging/diagnostics).
*/
onSyncError?: (error: Error) => void;
}
/**
* FileWatcher monitors a project directory for changes and triggers
* debounced sync operations via a provided callback.
*
* Design goals:
* - Minimal resource usage (native OS file events, no polling)
* - Debounced to avoid thrashing on rapid saves
* - Filters against CodeGraph include/exclude patterns
* - Ignores .codegraph/ directory changes
*/
export class FileWatcher {
private watcher: fs.FSWatcher | null = null;
private debounceTimer: ReturnType<typeof setTimeout> | null = null;
private hasChanges = false;
private syncing = false;
private stopped = false;
private readonly projectRoot: string;
private readonly config: CodeGraphConfig;
private readonly debounceMs: number;
private readonly syncFn: () => Promise<{ filesChanged: number; durationMs: number }>;
private readonly onSyncComplete?: WatchOptions['onSyncComplete'];
private readonly onSyncError?: WatchOptions['onSyncError'];
constructor(
projectRoot: string,
config: CodeGraphConfig,
syncFn: () => Promise<{ filesChanged: number; durationMs: number }>,
options: WatchOptions = {}
) {
this.projectRoot = projectRoot;
this.config = config;
this.syncFn = syncFn;
this.debounceMs = options.debounceMs ?? 2000;
this.onSyncComplete = options.onSyncComplete;
this.onSyncError = options.onSyncError;
}
/**
* Start watching for file changes.
* Returns true if watching started successfully, false otherwise.
*/
start(): boolean {
if (this.watcher) return true; // Already watching
this.stopped = false;
try {
this.watcher = fs.watch(
this.projectRoot,
{ recursive: true },
(_eventType, filename) => {
if (!filename || this.stopped) return;
// Normalize path separators
const normalized = normalizePath(filename);
// Ignore .codegraph/ directory changes (our own DB writes)
if (
normalized === '.codegraph' ||
normalized.startsWith('.codegraph/') ||
normalized.startsWith('.codegraph\\')
) {
return;
}
// Filter against include/exclude patterns
if (!shouldIncludeFile(normalized, this.config)) {
return;
}
logDebug('File change detected', { file: normalized });
this.hasChanges = true;
this.scheduleSync();
}
);
// Handle watcher errors gracefully
this.watcher.on('error', (err) => {
logWarn('File watcher error', { error: String(err) });
// Don't crash — watcher may recover or user can restart
});
logDebug('File watcher started', { projectRoot: this.projectRoot, debounceMs: this.debounceMs });
return true;
} catch (err) {
// Recursive watch not supported (e.g., Linux < Node 19)
logWarn('Could not start file watcher — recursive fs.watch not supported on this platform', { error: String(err) });
return false;
}
}
/**
* Stop watching for file changes.
*/
stop(): void {
this.stopped = true;
if (this.debounceTimer) {
clearTimeout(this.debounceTimer);
this.debounceTimer = null;
}
if (this.watcher) {
this.watcher.close();
this.watcher = null;
}
this.hasChanges = false;
logDebug('File watcher stopped');
}
/**
* Whether the watcher is currently active.
*/
isActive(): boolean {
return this.watcher !== null && !this.stopped;
}
/**
* Schedule a debounced sync.
*/
private scheduleSync(): void {
if (this.debounceTimer) {
clearTimeout(this.debounceTimer);
}
this.debounceTimer = setTimeout(() => {
this.debounceTimer = null;
this.flush();
}, this.debounceMs);
}
/**
* Flush pending changes by running sync.
*/
private async flush(): Promise<void> {
// If already syncing, the post-sync check will re-trigger
if (this.syncing || this.stopped) return;
this.hasChanges = false;
this.syncing = true;
try {
const result = await this.syncFn();
this.onSyncComplete?.(result);
} catch (err) {
const error = err instanceof Error ? err : new Error(String(err));
logWarn('Watch sync failed', { error: error.message });
this.onSyncError?.(error);
} finally {
this.syncing = false;
// If new changes arrived during sync, schedule another
if (this.hasChanges && !this.stopped) {
this.scheduleSync();
}
}
}
}