diff --git a/README.md b/README.md index 3cdef85..6a8835b 100644 --- a/README.md +++ b/README.md @@ -146,8 +146,8 @@ One tool call returns everything Claude needsβ€”entry points, related symbols, a -### πŸ” Semantic Search -Find code by meaning, not just text. Search for "authentication" and find `login`, `validateToken`, `AuthService`β€”even with different naming conventions. +### πŸ” Full-Text Search +Find code by name across your entire codebase instantly. Search for "auth" and find `authenticate`, `AuthService`, `validateToken` β€” powered by FTS5. @@ -173,7 +173,7 @@ No data leaves your machine. No API keys. No external services. Everything runs ### ⚑ Always Fresh -Claude Code hooks automatically sync the index as you work. Your code intelligence is always up to date. +The MCP server watches your files and auto-syncs on save β€” debounced, filtered to source files only, zero config. Your code intelligence is always up to date. @@ -181,6 +181,12 @@ Claude Code hooks automatically sync the index as you work. Your code intelligen --- +## πŸ“‹ Requirements + +- **Node.js >= 18.0.0** + +--- + ## 🎯 Quick Start ### 1. Run the Installer @@ -190,11 +196,10 @@ npx @colbymchenry/codegraph ``` The interactive installer will: -- Prompt to install `codegraph` globally (needed for hooks & MCP server to work) +- Prompt to install `codegraph` globally (needed for the MCP server to work) - Configure the MCP server in `~/.claude.json` - Set up auto-allow permissions for CodeGraph tools - Add global instructions to `~/.claude/CLAUDE.md` (teaches Claude how to use CodeGraph) -- Install Claude Code hooks for automatic index syncing - Optionally initialize your current project ### 2. Restart Claude Code @@ -298,12 +303,6 @@ At the start of a session, ask the user if they'd like to initialize CodeGraph: --- -## πŸ“‹ Requirements - -- Node.js >= 18.0.0 - ---- - ## πŸ’» CLI Usage ```bash @@ -334,13 +333,12 @@ npx @colbymchenry/codegraph # Run via npx (no global install needed) ``` The installer will: -1. Prompt to install `codegraph` globally (needed for hooks & MCP server) +1. Prompt to install `codegraph` globally (needed for the MCP server) 2. Ask for installation location (global `~/.claude` or local `./.claude`) 3. Optionally set up auto-allow permissions 4. Configure the MCP server in `claude.json` 5. Add global instructions to `~/.claude/CLAUDE.md` (teaches Claude how to use CodeGraph) -6. Install Claude Code hooks for automatic index syncing -7. For local installs: initialize and index the current project +6. For local installs: initialize and index the current project ### `codegraph init [path]` @@ -422,7 +420,7 @@ codegraph files --json # Output as JSON ### `codegraph context ` -Build relevant code context for a task. Uses semantic search to find entry points, then expands through the graph to find related code. +Build relevant code context for a task. Uses full-text search to find entry points, then expands through the graph to find related code. ```bash codegraph context "fix checkout bug" @@ -589,9 +587,13 @@ const context = await cg.buildContext('fix login bug', { // Get impact radius const impact = cg.getImpactRadius(node.id, 2); -// Sync changes +// Sync changes manually const syncResult = await cg.sync(); +// Or watch for changes and auto-sync +cg.watch(); // uses native OS file events, debounced +cg.unwatch(); // stop watching + // Clean up cg.close(); ``` @@ -615,7 +617,6 @@ All data is stored in a local SQLite database (`.codegraph/codegraph.db`): - **edges** table: Relationships between nodes - **files** table: File tracking for incremental updates - **unresolved_refs** table: References pending resolution -- **vectors** table: Embeddings stored as BLOBs for semantic search - **nodes_fts**: FTS5 virtual table for full-text search - **schema_versions** table: Schema version tracking - **project_metadata** table: Project-level key-value metadata @@ -629,13 +630,17 @@ After extraction, CodeGraph resolves references: 3. Link class inheritance and interface implementations 4. Apply framework-specific patterns (Express routes, etc.) -### 4. Semantic Search +### 4. File Watching -CodeGraph uses local embeddings (via [@xenova/transformers](https://github.com/xenova/transformers.js)) to enable semantic search: +The MCP server automatically watches your project for file changes using native OS file events (FSEvents on macOS, inotify on Linux, ReadDirectoryChangesW on Windows): -1. Code symbols are embedded using a transformer model -2. Queries are embedded and compared using cosine similarity -3. Results are ranked by relevance +1. File saves are detected instantly via OS-level events β€” no polling +2. Changes are **debounced** (2-second quiet window) so rapid saves don't thrash +3. Only source files matching your include/exclude patterns trigger a sync +4. Build outputs, node_modules, and `.codegraph/` changes are ignored +5. Incremental sync runs automatically β€” only changed files are re-parsed + +No configuration needed. The graph stays fresh as you code. ### 5. Graph Queries @@ -650,7 +655,7 @@ The graph structure enables powerful queries: When you request context for a task: -1. Semantic search finds relevant entry points +1. FTS search finds relevant entry points 2. Graph traversal expands to related code 3. Code snippets are extracted 4. Results are formatted for AI consumption @@ -730,7 +735,8 @@ Run `codegraph init` in your project directory first. ### Missing symbols in search -- Run `codegraph sync` to pick up recent changes +- The MCP server auto-syncs on file changes β€” wait a couple seconds after saving +- Run `codegraph sync` manually if needed - Check if the file's language is supported - Verify the file isn't excluded by config patterns diff --git a/__tests__/watcher.test.ts b/__tests__/watcher.test.ts new file mode 100644 index 0000000..f3638e6 --- /dev/null +++ b/__tests__/watcher.test.ts @@ -0,0 +1,291 @@ +/** + * FileWatcher Tests + * + * Tests for the file watcher that auto-syncs on changes. + */ + +import { describe, it, expect, beforeEach, afterEach, vi } from 'vitest'; +import * as fs from 'fs'; +import * as path from 'path'; +import * as os from 'os'; +import { FileWatcher } from '../src/sync/watcher'; +import type { CodeGraphConfig } from '../src/types'; +import CodeGraph from '../src/index'; + +/** + * Helper to wait for a condition with timeout + */ +function waitFor( + condition: () => boolean, + timeoutMs = 10000, + intervalMs = 100 +): Promise { + return new Promise((resolve, reject) => { + const start = Date.now(); + const check = () => { + if (condition()) return resolve(); + if (Date.now() - start > timeoutMs) return reject(new Error('waitFor timed out')); + setTimeout(check, intervalMs); + }; + check(); + }); +} + +describe('FileWatcher', () => { + let testDir: string; + + const baseConfig: CodeGraphConfig = { + version: 1, + rootDir: '.', + include: ['**/*.ts', '**/*.js'], + exclude: ['**/node_modules/**', '**/dist/**'], + languages: [], + frameworks: [], + maxFileSize: 1024 * 1024, + extractDocstrings: true, + trackCallSites: true, + }; + + beforeEach(() => { + testDir = fs.mkdtempSync(path.join(os.tmpdir(), 'codegraph-watcher-')); + // Create a source file so the directory isn't empty + const srcDir = path.join(testDir, 'src'); + fs.mkdirSync(srcDir); + fs.writeFileSync(path.join(srcDir, 'index.ts'), 'export const x = 1;'); + }); + + afterEach(() => { + if (fs.existsSync(testDir)) { + fs.rmSync(testDir, { recursive: true, force: true }); + } + }); + + describe('start/stop lifecycle', () => { + it('should start and stop without errors', () => { + const syncFn = vi.fn().mockResolvedValue({ filesChanged: 0, durationMs: 0 }); + const watcher = new FileWatcher(testDir, baseConfig, syncFn); + + const started = watcher.start(); + expect(started).toBe(true); + expect(watcher.isActive()).toBe(true); + + watcher.stop(); + expect(watcher.isActive()).toBe(false); + }); + + it('should be idempotent on double start', () => { + const syncFn = vi.fn().mockResolvedValue({ filesChanged: 0, durationMs: 0 }); + const watcher = new FileWatcher(testDir, baseConfig, syncFn); + + expect(watcher.start()).toBe(true); + expect(watcher.start()).toBe(true); // Should not throw + expect(watcher.isActive()).toBe(true); + + watcher.stop(); + }); + + it('should be idempotent on double stop', () => { + const syncFn = vi.fn().mockResolvedValue({ filesChanged: 0, durationMs: 0 }); + const watcher = new FileWatcher(testDir, baseConfig, syncFn); + + watcher.start(); + watcher.stop(); + watcher.stop(); // Should not throw + expect(watcher.isActive()).toBe(false); + }); + }); + + describe('debounced sync', () => { + it('should trigger sync after file change', async () => { + const syncFn = vi.fn().mockResolvedValue({ filesChanged: 1, durationMs: 10 }); + const watcher = new FileWatcher(testDir, baseConfig, syncFn, { debounceMs: 200 }); + + watcher.start(); + + // Create a new file + fs.writeFileSync(path.join(testDir, 'src', 'new.ts'), 'export const y = 2;'); + + // Wait for debounced sync to fire + await waitFor(() => syncFn.mock.calls.length > 0, 5000); + expect(syncFn).toHaveBeenCalled(); + + watcher.stop(); + }); + + it('should debounce rapid changes into a single sync', async () => { + const syncFn = vi.fn().mockResolvedValue({ filesChanged: 1, durationMs: 10 }); + const watcher = new FileWatcher(testDir, baseConfig, syncFn, { debounceMs: 500 }); + + watcher.start(); + + // Rapid-fire changes + for (let i = 0; i < 5; i++) { + fs.writeFileSync( + path.join(testDir, 'src', `file${i}.ts`), + `export const v${i} = ${i};` + ); + await new Promise((r) => setTimeout(r, 50)); + } + + // Wait for the single debounced sync + await waitFor(() => syncFn.mock.calls.length > 0, 5000); + + // Should have been called once (debounced), not 5 times + expect(syncFn.mock.calls.length).toBe(1); + + watcher.stop(); + }); + }); + + describe('filtering', () => { + it('should ignore files not matching include patterns', async () => { + const syncFn = vi.fn().mockResolvedValue({ filesChanged: 0, durationMs: 0 }); + const watcher = new FileWatcher(testDir, baseConfig, syncFn, { debounceMs: 200 }); + + watcher.start(); + + // Let watcher settle β€” fs.watch may fire residual events from beforeEach + await new Promise((r) => setTimeout(r, 400)); + syncFn.mockClear(); + + // Create a file that doesn't match include patterns + fs.writeFileSync(path.join(testDir, 'src', 'readme.md'), '# Hello'); + + // Wait a bit longer than debounce β€” sync should NOT trigger + await new Promise((r) => setTimeout(r, 500)); + expect(syncFn).not.toHaveBeenCalled(); + + watcher.stop(); + }); + + it('should ignore .codegraph directory changes', async () => { + const syncFn = vi.fn().mockResolvedValue({ filesChanged: 0, durationMs: 0 }); + const watcher = new FileWatcher(testDir, baseConfig, syncFn, { debounceMs: 200 }); + + watcher.start(); + + // Let watcher settle β€” fs.watch may fire residual events from beforeEach + await new Promise((r) => setTimeout(r, 400)); + syncFn.mockClear(); + + // Simulate a .codegraph directory change + const cgDir = path.join(testDir, '.codegraph'); + fs.mkdirSync(cgDir, { recursive: true }); + fs.writeFileSync(path.join(cgDir, 'db.sqlite'), 'fake'); + + // Wait β€” sync should NOT trigger + await new Promise((r) => setTimeout(r, 500)); + expect(syncFn).not.toHaveBeenCalled(); + + watcher.stop(); + }); + }); + + describe('callbacks', () => { + it('should call onSyncComplete after successful sync', async () => { + const syncFn = vi.fn().mockResolvedValue({ filesChanged: 2, durationMs: 50 }); + const onSyncComplete = vi.fn(); + const watcher = new FileWatcher(testDir, baseConfig, syncFn, { + debounceMs: 200, + onSyncComplete, + }); + + watcher.start(); + + fs.writeFileSync(path.join(testDir, 'src', 'test.ts'), 'export const z = 3;'); + + await waitFor(() => onSyncComplete.mock.calls.length > 0, 5000); + expect(onSyncComplete).toHaveBeenCalledWith({ filesChanged: 2, durationMs: 50 }); + + watcher.stop(); + }); + + it('should call onSyncError when sync throws', async () => { + const syncFn = vi.fn().mockRejectedValue(new Error('sync failed')); + const onSyncError = vi.fn(); + const watcher = new FileWatcher(testDir, baseConfig, syncFn, { + debounceMs: 200, + onSyncError, + }); + + watcher.start(); + + fs.writeFileSync(path.join(testDir, 'src', 'test.ts'), 'export const z = 3;'); + + await waitFor(() => onSyncError.mock.calls.length > 0, 5000); + expect(onSyncError).toHaveBeenCalled(); + expect(onSyncError.mock.calls[0]![0]).toBeInstanceOf(Error); + + watcher.stop(); + }); + }); + + describe('CodeGraph integration', () => { + let cg: CodeGraph; + + afterEach(() => { + if (cg) cg.close(); + }); + + it('should watch and unwatch via CodeGraph API', async () => { + cg = CodeGraph.initSync(testDir, { + config: { include: ['**/*.ts'], exclude: [] }, + }); + await cg.indexAll(); + + expect(cg.isWatching()).toBe(false); + + const started = cg.watch({ debounceMs: 200 }); + expect(started).toBe(true); + expect(cg.isWatching()).toBe(true); + + cg.unwatch(); + expect(cg.isWatching()).toBe(false); + }); + + it('should stop watching on close', async () => { + cg = CodeGraph.initSync(testDir, { + config: { include: ['**/*.ts'], exclude: [] }, + }); + await cg.indexAll(); + + cg.watch({ debounceMs: 200 }); + expect(cg.isWatching()).toBe(true); + + cg.close(); + // After close, isWatching should be false + // (we can't call isWatching after close since DB is closed, + // but we verify no errors are thrown) + }); + + it('should auto-sync when files change while watching', async () => { + cg = CodeGraph.initSync(testDir, { + config: { include: ['**/*.ts'], exclude: [] }, + }); + await cg.indexAll(); + + const initialStats = cg.getStats(); + const initialNodes = initialStats.nodeCount; + + cg.watch({ debounceMs: 300 }); + + // Add a new file with a function + fs.writeFileSync( + path.join(testDir, 'src', 'added.ts'), + 'export function added() { return 42; }' + ); + + // Wait for auto-sync to pick it up + await waitFor(() => { + const stats = cg.getStats(); + return stats.nodeCount > initialNodes; + }, 10000); + + // The new function should be in the graph + const results = cg.searchNodes('added'); + expect(results.length).toBeGreaterThan(0); + + cg.unwatch(); + }); + }); +}); diff --git a/src/index.ts b/src/index.ts index 6f37e87..0ff1e09 100644 --- a/src/index.ts +++ b/src/index.ts @@ -48,6 +48,7 @@ import { import { GraphTraverser, GraphQueryManager } from './graph'; import { ContextBuilder, createContextBuilder } from './context'; import { Mutex, FileLock } from './utils'; +import { FileWatcher, WatchOptions } from './sync'; // Re-export types for consumers export * from './types'; @@ -77,6 +78,7 @@ export { defaultLogger, } from './errors'; export { Mutex, FileLock, processInBatches, debounce, throttle, MemoryMonitor } from './utils'; +export { FileWatcher, WatchOptions } from './sync'; export { MCPServer } from './mcp'; /** @@ -140,6 +142,9 @@ export class CodeGraph { // File lock for preventing concurrent writes across processes (CLI, MCP, git hooks) private fileLock: FileLock; + // File watcher for auto-sync on file changes + private watcher: FileWatcher | null = null; + private constructor( db: DatabaseConnection, queries: QueryBuilder, @@ -319,6 +324,7 @@ export class CodeGraph { * Close the CodeGraph instance and release resources */ close(): void { + this.unwatch(); // Release file lock if held this.fileLock.release(); this.db.close(); @@ -491,6 +497,53 @@ export class CodeGraph { return this.indexMutex.isLocked(); } + // =========================================================================== + // File Watching + // =========================================================================== + + /** + * Start watching for file changes and auto-syncing. + * + * Uses native OS file events (FSEvents on macOS, inotify on Linux 19+, + * ReadDirectoryChangesW on Windows) with debouncing to avoid thrashing. + * + * @param options - Watch options (debounce delay, callbacks) + * @returns true if watching started successfully + */ + watch(options: WatchOptions = {}): boolean { + if (this.watcher?.isActive()) return true; + + this.watcher = new FileWatcher( + this.projectRoot, + this.config, + async () => { + const result = await this.sync(); + const filesChanged = result.filesAdded + result.filesModified + result.filesRemoved; + return { filesChanged, durationMs: result.durationMs }; + }, + options + ); + + return this.watcher.start(); + } + + /** + * Stop watching for file changes. + */ + unwatch(): void { + if (this.watcher) { + this.watcher.stop(); + this.watcher = null; + } + } + + /** + * Check if the file watcher is active. + */ + isWatching(): boolean { + return this.watcher?.isActive() ?? false; + } + /** * Get files that have changed since last index */ diff --git a/src/mcp/index.ts b/src/mcp/index.ts index b740964..bc3552a 100644 --- a/src/mcp/index.ts +++ b/src/mcp/index.ts @@ -116,6 +116,7 @@ export class MCPServer { try { this.cg = await CodeGraph.open(resolvedRoot); this.toolHandler.setDefaultCodeGraph(this.cg); + this.startWatching(); } catch (err) { // Log the error so transient failures are diagnosable (see issue #47) const msg = err instanceof Error ? err.message : String(err); @@ -147,11 +148,37 @@ export class MCPServer { this.cg = CodeGraph.openSync(resolvedRoot); this.projectPath = resolvedRoot; this.toolHandler.setDefaultCodeGraph(this.cg); + this.startWatching(); } catch { // Still failing β€” will retry on next tool call } } + /** + * Start file watching on the active CodeGraph instance. + * Logs sync activity to stderr for diagnostics. + */ + private startWatching(): void { + if (!this.cg) return; + + const started = this.cg.watch({ + onSyncComplete: (result) => { + if (result.filesChanged > 0) { + process.stderr.write( + `[CodeGraph MCP] Auto-synced ${result.filesChanged} file(s) in ${result.durationMs}ms\n` + ); + } + }, + onSyncError: (err) => { + process.stderr.write(`[CodeGraph MCP] Auto-sync error: ${err.message}\n`); + }, + }); + + if (started) { + process.stderr.write('[CodeGraph MCP] File watcher active β€” graph will auto-sync on changes\n'); + } + } + /** * Stop the server */ diff --git a/src/sync/index.ts b/src/sync/index.ts index b63d94b..51b8b6f 100644 --- a/src/sync/index.ts +++ b/src/sync/index.ts @@ -4,14 +4,10 @@ * Provides synchronization functionality for keeping the code graph * up-to-date with file system changes. * - * Note: Git hooks functionality has been removed. CodeGraph sync is now - * triggered through codegraph's Claude Code hooks integration instead. - * * Components: + * - FileWatcher: Debounced fs.watch that auto-triggers sync on file changes * - Content hashing for change detection (in extraction module) * - Incremental reindexing (in extraction module) */ -// This module is kept for potential future sync-related exports -// Currently all sync functionality is in the extraction module -export {}; +export { FileWatcher, WatchOptions } from './watcher'; diff --git a/src/sync/watcher.ts b/src/sync/watcher.ts new file mode 100644 index 0000000..d3ef24b --- /dev/null +++ b/src/sync/watcher.ts @@ -0,0 +1,196 @@ +/** + * File Watcher + * + * Watches the project directory for file changes and triggers + * debounced sync operations to keep the code graph up-to-date. + * + * Uses Node.js native fs.watch with recursive mode (macOS FSEvents, + * Windows ReadDirectoryChangesW, Linux inotify on Node 19+). + */ + +import * as fs from 'fs'; +import { CodeGraphConfig } from '../types'; +import { shouldIncludeFile } from '../extraction'; +import { logDebug, logWarn } from '../errors'; +import { normalizePath } from '../utils'; + +/** + * Options for the file watcher + */ +export interface WatchOptions { + /** + * Debounce delay in milliseconds. + * After the last file change, wait this long before triggering sync. + * Default: 2000ms + */ + debounceMs?: number; + + /** + * Callback when a sync completes (for logging/diagnostics). + */ + onSyncComplete?: (result: { filesChanged: number; durationMs: number }) => void; + + /** + * Callback when a sync errors (for logging/diagnostics). + */ + onSyncError?: (error: Error) => void; +} + +/** + * FileWatcher monitors a project directory for changes and triggers + * debounced sync operations via a provided callback. + * + * Design goals: + * - Minimal resource usage (native OS file events, no polling) + * - Debounced to avoid thrashing on rapid saves + * - Filters against CodeGraph include/exclude patterns + * - Ignores .codegraph/ directory changes + */ +export class FileWatcher { + private watcher: fs.FSWatcher | null = null; + private debounceTimer: ReturnType | null = null; + private hasChanges = false; + private syncing = false; + private stopped = false; + + private readonly projectRoot: string; + private readonly config: CodeGraphConfig; + private readonly debounceMs: number; + private readonly syncFn: () => Promise<{ filesChanged: number; durationMs: number }>; + private readonly onSyncComplete?: WatchOptions['onSyncComplete']; + private readonly onSyncError?: WatchOptions['onSyncError']; + + constructor( + projectRoot: string, + config: CodeGraphConfig, + syncFn: () => Promise<{ filesChanged: number; durationMs: number }>, + options: WatchOptions = {} + ) { + this.projectRoot = projectRoot; + this.config = config; + this.syncFn = syncFn; + this.debounceMs = options.debounceMs ?? 2000; + this.onSyncComplete = options.onSyncComplete; + this.onSyncError = options.onSyncError; + } + + /** + * Start watching for file changes. + * Returns true if watching started successfully, false otherwise. + */ + start(): boolean { + if (this.watcher) return true; // Already watching + this.stopped = false; + + try { + this.watcher = fs.watch( + this.projectRoot, + { recursive: true }, + (_eventType, filename) => { + if (!filename || this.stopped) return; + + // Normalize path separators + const normalized = normalizePath(filename); + + // Ignore .codegraph/ directory changes (our own DB writes) + if ( + normalized === '.codegraph' || + normalized.startsWith('.codegraph/') || + normalized.startsWith('.codegraph\\') + ) { + return; + } + + // Filter against include/exclude patterns + if (!shouldIncludeFile(normalized, this.config)) { + return; + } + + logDebug('File change detected', { file: normalized }); + this.hasChanges = true; + this.scheduleSync(); + } + ); + + // Handle watcher errors gracefully + this.watcher.on('error', (err) => { + logWarn('File watcher error', { error: String(err) }); + // Don't crash β€” watcher may recover or user can restart + }); + + logDebug('File watcher started', { projectRoot: this.projectRoot, debounceMs: this.debounceMs }); + return true; + } catch (err) { + // Recursive watch not supported (e.g., Linux < Node 19) + logWarn('Could not start file watcher β€” recursive fs.watch not supported on this platform', { error: String(err) }); + return false; + } + } + + /** + * Stop watching for file changes. + */ + stop(): void { + this.stopped = true; + + if (this.debounceTimer) { + clearTimeout(this.debounceTimer); + this.debounceTimer = null; + } + + if (this.watcher) { + this.watcher.close(); + this.watcher = null; + } + + this.hasChanges = false; + logDebug('File watcher stopped'); + } + + /** + * Whether the watcher is currently active. + */ + isActive(): boolean { + return this.watcher !== null && !this.stopped; + } + + /** + * Schedule a debounced sync. + */ + private scheduleSync(): void { + if (this.debounceTimer) { + clearTimeout(this.debounceTimer); + } + this.debounceTimer = setTimeout(() => { + this.debounceTimer = null; + this.flush(); + }, this.debounceMs); + } + + /** + * Flush pending changes by running sync. + */ + private async flush(): Promise { + // If already syncing, the post-sync check will re-trigger + if (this.syncing || this.stopped) return; + + this.hasChanges = false; + this.syncing = true; + + try { + const result = await this.syncFn(); + this.onSyncComplete?.(result); + } catch (err) { + const error = err instanceof Error ? err : new Error(String(err)); + logWarn('Watch sync failed', { error: error.message }); + this.onSyncError?.(error); + } finally { + this.syncing = false; + + // If new changes arrived during sync, schedule another + if (this.hasChanges && !this.stopped) { + this.scheduleSync(); + } + } + } +}