feat(mcp): per-file staleness banner + tunable watcher debounce (#403) (#428)

Two coupled changes addressing the issue's underlying ask — "how does the
agent know when the index lags" — without resorting to a static wait.

Per-file staleness banner
-------------------------
FileWatcher now tracks per-path `pendingFiles` (path, firstSeenMs,
lastSeenMs, indexing) — events since the last successful sync, cleared
only after a sync whose `syncStartedMs >= lastSeenMs` commits. Chokidar
initial-scan events are gated behind a `ready` flag (with `waitUntilReady()`
exposed so tests can deterministically wait through it) so a fresh startup
doesn't falsely flag every existing file as pending.

ToolHandler now wraps every code-returning response (search, context,
callers, callees, impact, trace, explore, node, files) with
`withStalenessNotice`: intersects "files referenced in the response" with
`getPendingFiles()` and emits a hybrid signal —

  * banner at the top for files referenced AND pending (with edit age +
    indexing/pending-sync state, telling the agent to Read those specific
    files directly; the rest of the response stays fresh and codegraph
    stays authoritative for it),
  * compact footer for pending files elsewhere in the project not
    referenced above (capped at 5).

Cost is one boolean check + N substring matches when pending; zero
allocation when idle. `codegraph_status` surfaces the same data as a
first-class `### Pending sync:` section so the agent can ask "is the index
caught up?" in one call.

Cross-project quirk: when an agent passes `projectPath` matching the
default session's project, the staleness wrapper switches from the cached
cross-project CodeGraph (no watcher) to the default one (with watcher) so
the signal still fires. Same fix applied to `handleStatus`.

CODEGRAPH_WATCH_DEBOUNCE_MS
---------------------------
MCP `serve --mcp` now reads `CODEGRAPH_WATCH_DEBOUNCE_MS` and forwards it
to `cg.watch({ debounceMs })`. Clamped to [100ms, 60s]; out-of-range or
non-numeric values fall back to the FileWatcher default (2000ms). Active
value is logged to stderr on watcher startup so it's discoverable. The
docs in `server-instructions.ts`, `installer/instructions-template.ts`,
and `.cursor/rules/codegraph.mdc` no longer claim "~500ms"; they now
describe the banner mechanism instead — since per-file staleness replaces
the "wait N ms" guidance entirely, the docs become accurate at any
debounce value.

Validation
----------
* 847 unit/integration tests pass (added 15 new ones — pending-file
  tracking, banner/footer routing, status section, env-var parsing).
* Direct MCP probe through a real `codegraph serve --mcp` process: edit a
  file, query within the debounce window, banner fires naming the
  edited file with edit-age.
* Real Claude TUI session via `scripts/agent-eval/itrun.sh` with
  `CODEGRAPH_WATCH_DEBOUNCE_MS=10000`: agent edits `math.ts`, calls
  `codegraph_explore`, reads the banner, **and discloses it unprompted in
  its final reply**: "note: symbol index is mid-sync for the new `divide`,
  but the source it returned is verbatim from disk."

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Colby Mchenry
2026-05-25 23:48:10 -05:00
committed by GitHub
co-authored by Claude Opus 4.7
parent 4a4a37d135
commit b48170e69f
12 changed files with 696 additions and 18 deletions
+1 -1
View File
@@ -31,7 +31,7 @@ Use codegraph for **structural** questions — what calls what, what would break
- **Don't grep first** when looking up a symbol by name. `codegraph_search` is faster and returns kind + location + signature in one call.
- **Don't chain `codegraph_search` + `codegraph_node`** when you just want context — `codegraph_context` is one call.
- **Don't loop `codegraph_node` over many symbols** — one `codegraph_explore` call returns several symbols' source grouped in a single capped call, while each separate node/Read call re-reads the whole context and costs far more.
- **Index lag**: the file watcher debounces ~500ms behind writes; don't re-query immediately after editing a file in the same turn.
- **Index lag — check the staleness banner, don't guess a wait.** When a codegraph response starts with "⚠️ Some files referenced below were edited since the last index sync…", the listed files are pending re-index — Read those specific files for accurate content. Files NOT in that banner are fresh and codegraph is authoritative for them. `codegraph_status` also lists pending files under "Pending sync".
### If `.codegraph/` doesn't exist
+22
View File
@@ -34,6 +34,28 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
already attached to the old daemon keep using it while new sessions run
standalone until it idles out — they never mix versions over the socket.
- **Per-file staleness banner — codegraph responses now tell the agent which
files are pending re-index (#403).** When the file watcher has seen edits
since the last successful sync, MCP tool responses (`codegraph_search`,
`context`, `callers`, `callees`, `impact`, `trace`, `explore`, `node`,
`files`) prepend a `⚠️` banner naming the stale files referenced in that
response, with their edit age and indexing state; pending files elsewhere in
the project appear as a small footer. The agent is told which specific files
to Read directly; the rest of the response is fresh and codegraph stays
authoritative for it. No artificial wait, no static "wait ~500ms" guess —
the cost is zero when nothing's pending. `codegraph_status` also surfaces a
`### Pending sync:` section so an agent can ask "is the index caught up?" in
one call.
- **`CODEGRAPH_WATCH_DEBOUNCE_MS` env var lets you tune the file-watcher quiet
window (#403).** Default stays at 2000ms; workspaces with bursty writes
(formatter-on-save chains, multi-file refactors, large generated outputs)
can raise it (e.g. `5000` or `10000`) without touching their agent's command
line. Clamped to `[100ms, 60s]`; out-of-range or non-numeric values fall
back to the default and the active value is logged to stderr on watcher
startup so it's discoverable. Pairs with the staleness banner above: the
banner stays accurate at any debounce value because it's per-file, not a
static "wait N ms" instruction.
### Fixed
- **Git worktrees no longer silently borrow another tree's index (#155).**
When a worktree is nested inside the main checkout — exactly what agent
+47
View File
@@ -0,0 +1,47 @@
/**
* CODEGRAPH_WATCH_DEBOUNCE_MS env override (issue #403).
*
* Lets users tune the watcher quiet window from MCP-launched configs without
* editing the agent's command line — formatter-on-save chains and large
* generated outputs benefit from a longer window. Clamped to [100ms, 60s];
* out-of-range / non-numeric values fall back to the FileWatcher default
* (2000ms) rather than throwing or silently capping a likely typo.
*/
import { describe, it, expect } from 'vitest';
import { parseDebounceEnv } from '../src/mcp/engine';
describe('parseDebounceEnv', () => {
it('returns undefined for unset / empty values', () => {
expect(parseDebounceEnv(undefined)).toBeUndefined();
expect(parseDebounceEnv('')).toBeUndefined();
expect(parseDebounceEnv(' ')).toBeUndefined();
});
it('accepts integer values inside [100, 60000]', () => {
expect(parseDebounceEnv('100')).toBe(100);
expect(parseDebounceEnv('2000')).toBe(2000);
expect(parseDebounceEnv('5000')).toBe(5000);
expect(parseDebounceEnv('60000')).toBe(60000);
});
it('rejects out-of-range values (returns undefined, lets default win)', () => {
expect(parseDebounceEnv('0')).toBeUndefined();
expect(parseDebounceEnv('50')).toBeUndefined(); // below 100
expect(parseDebounceEnv('99')).toBeUndefined();
expect(parseDebounceEnv('60001')).toBeUndefined(); // above 60s
expect(parseDebounceEnv('-500')).toBeUndefined();
});
it('rejects non-integer / non-numeric values', () => {
expect(parseDebounceEnv('abc')).toBeUndefined();
expect(parseDebounceEnv('500.5')).toBeUndefined();
expect(parseDebounceEnv('NaN')).toBeUndefined();
expect(parseDebounceEnv('Infinity')).toBeUndefined();
});
it('accepts scientific notation that resolves to an in-range integer', () => {
// Number('1e3') === 1000, Number.isInteger(1000) === true. Power users
// who write debounce as 1e3 should not be surprised; the clamp still applies.
expect(parseDebounceEnv('1e3')).toBe(1000);
});
});
+153
View File
@@ -0,0 +1,153 @@
/**
* Per-file staleness banner on MCP tool responses (issue #403).
*
* The watcher tracks every file event since the last successful sync; the
* tool dispatcher intersects "files referenced in this response" with that
* pending set and prepends a banner (" Some files referenced below were
* edited since the last index sync") plus an optional footer ("(Note: N
* file(s) elsewhere in this project are pending index sync)").
*
* No auto-flush, no static wait the response is instant and the agent
* decides whether to Read the specific stale file. These tests exercise
* the full real path: real watcher + real CodeGraph index + real
* ToolHandler.execute().
*/
import { describe, it, expect, beforeEach, afterEach } from 'vitest';
import * as fs from 'fs';
import * as path from 'path';
import * as os from 'os';
import CodeGraph from '../src/index';
import { ToolHandler } from '../src/mcp/tools';
function waitFor(condition: () => boolean, timeoutMs = 5000, intervalMs = 50): Promise<void> {
return new Promise((resolve, reject) => {
const start = Date.now();
const tick = () => {
if (condition()) return resolve();
if (Date.now() - start > timeoutMs) return reject(new Error('waitFor timed out'));
setTimeout(tick, intervalMs);
};
tick();
});
}
describe('MCP staleness banner', () => {
let testDir: string;
let cg: CodeGraph;
let handler: ToolHandler;
beforeEach(async () => {
testDir = fs.mkdtempSync(path.join(os.tmpdir(), 'codegraph-stale-banner-'));
fs.mkdirSync(path.join(testDir, 'src'));
// Three isolated files with no cross-references — keeps each test's
// "which path does the response mention?" assertion unambiguous. If the
// files shared imports/calls, codegraph_search responses would surface
// multiple file paths and the banner-vs-footer split would be racy.
fs.writeFileSync(
path.join(testDir, 'src', 'alpha-only.ts'),
'export function alphaOnly() { return 1; }\n',
);
fs.writeFileSync(
path.join(testDir, 'src', 'bravo-only.ts'),
'export function bravoOnly() { return 2; }\n',
);
fs.writeFileSync(
path.join(testDir, 'src', 'charlie-only.ts'),
'export function charlieOnly() { return 3; }\n',
);
cg = CodeGraph.initSync(testDir, { config: { include: ['**/*.ts'], exclude: [] } });
await cg.indexAll();
handler = new ToolHandler(cg);
});
afterEach(() => {
try { cg.unwatch(); } catch { /* ignore */ }
try { cg.close(); } catch { /* ignore */ }
if (fs.existsSync(testDir)) fs.rmSync(testDir, { recursive: true, force: true });
});
it('prepends a stale banner when the response references a pending file', async () => {
// Long debounce so the edit lingers in pendingFiles while we query.
cg.watch({ debounceMs: 4000 });
await cg.waitUntilWatcherReady();
fs.writeFileSync(
path.join(testDir, 'src', 'alpha-only.ts'),
'export function alphaOnly() { return 99; }\n',
);
await waitFor(() => cg.getPendingFiles().some((p) => p.path === 'src/alpha-only.ts'), 8000);
const res = await handler.execute('codegraph_search', { query: 'alphaOnly' });
expect(res.isError).toBeFalsy();
const text = res.content[0].text;
// Banner shape: warning glyph + filename + actionable instruction.
expect(text.startsWith('⚠️')).toBe(true);
expect(text).toContain('src/alpha-only.ts');
expect(text).toMatch(/edited \d+ms ago/);
expect(text).toMatch(/Read them directly/);
// The actual result must still follow the banner.
expect(text).toMatch(/alphaOnly/);
});
it('uses the footer (not the banner) when pending files are not referenced', async () => {
cg.watch({ debounceMs: 4000 });
await cg.waitUntilWatcherReady();
// Edit bravo-only.ts but search for the alphaOnly symbol, whose hit is
// only in alpha-only.ts. The two files share no imports/calls so the
// response text won't mention bravo-only.ts.
fs.writeFileSync(
path.join(testDir, 'src', 'bravo-only.ts'),
'export function bravoOnly() { return 22; }\n',
);
await waitFor(() => cg.getPendingFiles().some((p) => p.path === 'src/bravo-only.ts'), 8000);
const res = await handler.execute('codegraph_search', { query: 'alphaOnly' });
const text = res.content[0].text;
expect(text.startsWith('⚠️')).toBe(false);
expect(text).toMatch(/elsewhere in this project are pending index sync/);
expect(text).toContain('src/bravo-only.ts');
});
it('drops the banner once the sync completes and clears the pending entry', async () => {
cg.watch({ debounceMs: 200 });
await cg.waitUntilWatcherReady();
fs.writeFileSync(
path.join(testDir, 'src', 'alpha-only.ts'),
'export function alphaOnly() { return 7; }\n',
);
await waitFor(() => cg.getPendingFiles().length === 0, 5000);
const res = await handler.execute('codegraph_search', { query: 'alphaOnly' });
const text = res.content[0].text;
expect(text.startsWith('⚠️')).toBe(false);
expect(text).not.toMatch(/elsewhere in this project are pending index sync/);
});
it('lists pending files under "Pending sync" in codegraph_status', async () => {
cg.watch({ debounceMs: 4000 });
await cg.waitUntilWatcherReady();
fs.writeFileSync(
path.join(testDir, 'src', 'charlie-only.ts'),
'export function charlieOnly() { return 33; }\n',
);
await waitFor(() => cg.getPendingFiles().some((p) => p.path === 'src/charlie-only.ts'), 8000);
const res = await handler.execute('codegraph_status', {});
const text = res.content[0].text;
expect(text).toContain('### Pending sync:');
expect(text).toContain('src/charlie-only.ts');
// Status embeds the info first-class, so the auto-banner is suppressed.
expect(text.startsWith('⚠️')).toBe(false);
});
it('returns zero pending files when no watcher is active', () => {
expect(cg.getPendingFiles()).toEqual([]);
});
});
+88
View File
@@ -198,6 +198,94 @@ describe('FileWatcher', () => {
});
});
describe('pending file tracking (#403)', () => {
it('should expose edited paths via getPendingFiles before sync fires', async () => {
// Slow debounce — events arrive but sync hasn't run yet.
const syncFn = vi.fn().mockResolvedValue({ filesChanged: 1, durationMs: 10 });
const watcher = new FileWatcher(testDir, syncFn, { debounceMs: 2000 });
watcher.start();
// Deterministic boundary: wait for chokidar's initial scan to complete
// so any late initial-scan events have fired before we assert. A bare
// sleep is flaky under test-parallelism load.
await watcher.waitUntilReady();
expect(watcher.getPendingFiles()).toEqual([]);
fs.writeFileSync(path.join(testDir, 'src', 'pending.ts'), 'export const p = 1;');
// Allow chokidar to emit, but DON'T let the 2s debounce fire.
await waitFor(() => watcher.getPendingFiles().length > 0, 3000);
const pending = watcher.getPendingFiles();
const paths = pending.map((p) => p.path);
expect(paths).toContain('src/pending.ts');
const entry = pending.find((p) => p.path === 'src/pending.ts')!;
expect(entry.firstSeenMs).toBeGreaterThan(0);
expect(entry.lastSeenMs).toBeGreaterThanOrEqual(entry.firstSeenMs);
// No sync running yet → indexing flag is false.
expect(entry.indexing).toBe(false);
watcher.stop();
});
it('should clear an entry only after a successful sync absorbing that edit', async () => {
const syncFn = vi.fn().mockResolvedValue({ filesChanged: 1, durationMs: 10 });
const watcher = new FileWatcher(testDir, syncFn, { debounceMs: 200 });
watcher.start();
await watcher.waitUntilReady();
fs.writeFileSync(path.join(testDir, 'src', 'fresh.ts'), 'export const f = 1;');
// Watcher saw the change → pendingFiles has the entry. Longer windows
// here because chokidar event delivery on macOS slows under heavy
// parallel test-suite load (4× slower than isolation).
await waitFor(() => watcher.getPendingFiles().some((p) => p.path === 'src/fresh.ts'), 8000);
// Wait through debounce + sync; the entry should drop out.
await waitFor(() => syncFn.mock.calls.length > 0, 8000);
await waitFor(() => !watcher.getPendingFiles().some((p) => p.path === 'src/fresh.ts'), 8000);
expect(watcher.getPendingFiles()).toEqual([]);
watcher.stop();
});
it('should keep entries unchanged when sync fails (rescheduled work sees the same set)', async () => {
// First post-settle sync rejects, second resolves. The initial-scan
// sync (triggered by chokidar's pre-existing add events) is allowed to
// resolve cleanly so it doesn't consume one of our scripted outcomes.
const syncFn = vi
.fn()
.mockResolvedValueOnce({ filesChanged: 0, durationMs: 1 }) // initial scan
.mockRejectedValueOnce(new Error('boom')) // first real edit fails
.mockResolvedValueOnce({ filesChanged: 1, durationMs: 10 }); // retry succeeds
const onSyncError = vi.fn();
const watcher = new FileWatcher(testDir, syncFn, { debounceMs: 200, onSyncError });
watcher.start();
// Wait through chokidar `ready` AND the initial-scan-triggered sync, so
// the next sync corresponds to the explicit edit below.
await watcher.waitUntilReady();
await waitFor(() => syncFn.mock.calls.length >= 1, 5000);
await new Promise((r) => setTimeout(r, 100));
fs.writeFileSync(path.join(testDir, 'src', 'will-fail.ts'), 'export const wf = 1;');
// Wait for the sync that handles the explicit edit to reject.
await waitFor(() => onSyncError.mock.calls.length > 0, 5000);
// The file is STILL in pendingFiles — failure didn't drop it.
const after = watcher.getPendingFiles();
expect(after.some((p) => p.path === 'src/will-fail.ts')).toBe(true);
// Retry resolves; entry clears.
await waitFor(
() => !watcher.getPendingFiles().some((p) => p.path === 'src/will-fail.ts'),
5000,
);
watcher.stop();
});
});
describe('callbacks', () => {
it('should call onSyncComplete after successful sync', async () => {
const syncFn = vi.fn().mockResolvedValue({ filesChanged: 2, durationMs: 50 });
+27 -2
View File
@@ -46,7 +46,7 @@ import {
import { GraphTraverser, GraphQueryManager } from './graph';
import { ContextBuilder, createContextBuilder } from './context';
import { Mutex, FileLock } from './utils';
import { FileWatcher, WatchOptions } from './sync';
import { FileWatcher, WatchOptions, PendingFile } from './sync';
// Re-export types for consumers
export * from './types';
@@ -75,7 +75,7 @@ export {
defaultLogger,
} from './errors';
export { Mutex, FileLock, processInBatches, debounce, throttle, MemoryMonitor } from './utils';
export { FileWatcher, WatchOptions } from './sync';
export { FileWatcher, WatchOptions, PendingFile } from './sync';
export { MCPServer } from './mcp';
/**
@@ -499,6 +499,31 @@ export class CodeGraph {
return this.watcher?.isActive() ?? false;
}
/**
* Files seen by the file watcher since the last successful sync
* the per-file "stale" signal MCP tools attach to responses so an agent
* can fall back to {@link Read} for just the affected file without
* waiting for a debounced sync to complete (issue #403).
*
* Returns an empty list when the watcher isn't active, or no events have
* arrived. Each entry includes `firstSeenMs` and `lastSeenMs` (wall-clock
* `Date.now()` values) so callers can render "edited Nms ago", plus an
* `indexing` flag indicating whether the in-flight sync (if any) will
* absorb that file.
*/
getPendingFiles(): PendingFile[] {
return this.watcher?.getPendingFiles() ?? [];
}
/**
* Resolves once the file watcher has finished its initial chokidar scan.
* Useful for tests that need a deterministic boundary before asserting on
* `getPendingFiles()`. Resolves immediately when no watcher is active.
*/
waitUntilWatcherReady(timeoutMs?: number): Promise<void> {
return this.watcher ? this.watcher.waitUntilReady(timeoutMs) : Promise.resolve();
}
/**
* Get files that have changed since last index
*/
+1 -1
View File
@@ -49,7 +49,7 @@ Use codegraph for **structural** questions — what calls what, what would break
- **Don't grep first** when looking up a symbol by name. \`codegraph_search\` is faster and returns kind + location + signature in one call.
- **Don't chain \`codegraph_search\` + \`codegraph_node\`** when you just want context — \`codegraph_context\` is one call.
- **Don't loop \`codegraph_node\` over many symbols** — one \`codegraph_explore\` call returns several symbols' source grouped in a single capped call, while each separate node/Read call re-reads the whole context and costs far more.
- **Index lag**: the file watcher debounces ~500ms behind writes; don't re-query immediately after editing a file in the same turn.
- **Index lag check the staleness banner, don't guess a wait.** When a codegraph response starts with "⚠️ Some files referenced below were edited since the last index sync…", the listed files are pending re-index Read those specific files for accurate content. Files NOT in that banner are fresh and codegraph is authoritative for them. \`codegraph_status\` also lists pending files under "Pending sync".
### If \`.codegraph/\` doesn't exist
+32
View File
@@ -185,7 +185,18 @@ export class MCPEngine {
return;
}
// Optional override for the debounce window via env var (issue #403).
// Useful for workspaces with bursty writes (formatter-on-save chains,
// large generated outputs) where the 2s default fires too often. Clamped
// to [100ms, 60s]; out-of-range / non-numeric values fall back to the
// FileWatcher default. We log the active value so it's discoverable.
const debounceMs = parseDebounceEnv(process.env.CODEGRAPH_WATCH_DEBOUNCE_MS);
if (debounceMs !== undefined) {
process.stderr.write(`[CodeGraph MCP] File watcher debounce: ${debounceMs}ms (CODEGRAPH_WATCH_DEBOUNCE_MS)\n`);
}
const started = this.cg.watch({
debounceMs,
onSyncComplete: (result) => {
if (result.filesChanged > 0) {
process.stderr.write(
@@ -230,3 +241,24 @@ export class MCPEngine {
});
}
}
/**
* Parse and clamp the CODEGRAPH_WATCH_DEBOUNCE_MS env override.
*
* Issue #403: workspaces with bursty writes (formatter-on-save, multi-file
* refactors) sometimes want a longer quiet window before sync. Returns
* `undefined` for unset / empty / non-numeric / out-of-range values so the
* FileWatcher default (2000ms) takes over never throws.
*
* Clamp range: 100ms (faster would mean a sync per keystroke) to 60s (longer
* and the watcher feels broken). Out-of-range values are treated as "ignore
* this misconfiguration" rather than capped, since silently capping a 0 or
* a typoed value would mask a real config bug.
*/
export function parseDebounceEnv(raw: string | undefined): number | undefined {
if (!raw || !raw.trim()) return undefined;
const n = Number(raw);
if (!Number.isFinite(n) || !Number.isInteger(n)) return undefined;
if (n < 100 || n > 60000) return undefined;
return n;
}
+1 -1
View File
@@ -59,7 +59,7 @@ of calls; a grep/read exploration is dozens.
- **Don't grep first** when looking up a symbol by name \`codegraph_search\` is faster and returns kind + location + signature.
- **Don't chain \`codegraph_search\` + \`codegraph_node\`** when you just want context — \`codegraph_context\` is one round-trip.
- **Don't loop \`codegraph_node\` over many symbols** — one \`codegraph_explore\` call returns them all grouped by file, while each separate call re-reads the whole context and costs far more. Use \`codegraph_node\` for a single symbol.
- **Don't query the index immediately after editing a file** the watcher needs ~500ms to debounce + sync. Wait for the next turn.
- **After editing, check the staleness banner.** When a tool response starts with "⚠️ Some files referenced below were edited since the last index sync…", the listed files are pending re-index Read those specific files for accurate content. Every file NOT in that banner is fresh, so still trust codegraph. \`codegraph_status\` also lists pending files under "Pending sync".
## Limitations
+160 -6
View File
@@ -11,6 +11,7 @@ import {
worktreeMismatchNotice,
type WorktreeIndexMismatch,
} from '../sync/worktree';
import type { PendingFile } from '../sync';
import type { Node, Edge, SearchResult, Subgraph, TaskContext, NodeKind } from '../types';
import { createHash } from 'crypto';
import {
@@ -24,7 +25,7 @@ import {
} from 'fs';
import { clamp, validatePathWithinRoot, validateProjectPath } from '../utils';
import { tmpdir } from 'os';
import { join } from 'path';
import { join, resolve as resolvePath } from 'path';
/** Maximum output length to prevent context bloat (characters) */
const MAX_OUTPUT_LENGTH = 15000;
@@ -264,6 +265,48 @@ function markSessionConsulted(sessionId: string): void {
}
}
/**
* Per-file staleness banner emitted at the top of a tool response when the
* file watcher has pending events for files referenced by the response.
* The agent uses this to fall back to Read for those specific files
* without waiting for the debounced sync (issue #403).
*/
export function formatStaleBanner(stale: PendingFile[]): string {
const now = Date.now();
const lines = stale.map((p) => {
const ageMs = Math.max(0, now - p.lastSeenMs);
const label = p.indexing ? 'indexing in progress' : 'pending sync';
return ` - ${p.path} (edited ${ageMs}ms ago, ${label})`;
});
return (
'⚠️ Some files referenced below were edited since the last index sync — ' +
'their codegraph entries may be stale:\n' +
lines.join('\n') +
'\nFor accurate content of those specific files, Read them directly. ' +
'The rest of this response is fresh.'
);
}
/**
* Compact footer listing pending files that are NOT referenced in this
* response. Gives the agent a complete project-wide freshness picture
* without bloating the main banner.
*/
export function formatStaleFooter(stale: PendingFile[]): string {
const MAX = 5;
const now = Date.now();
const shown = stale.slice(0, MAX);
const lines = shown.map((p) => {
const ageMs = Math.max(0, now - p.lastSeenMs);
return ` - ${p.path} (edited ${ageMs}ms ago)`;
});
const more = stale.length > MAX ? `\n - …and ${stale.length - MAX} more` : '';
return (
`(Note: ${stale.length} file(s) elsewhere in this project are pending index ` +
`sync but were not referenced above:\n${lines.join('\n')}${more})`
);
}
/**
* MCP Tool definition
*/
@@ -802,6 +845,84 @@ export class ToolHandler {
return result;
}
/**
* Annotate a successful read-tool result with per-file staleness the
* non-blocking answer to issue #403. The file watcher tracks every event
* it sees per path; here we intersect "files referenced in this response"
* against that pending set and prepend a compact banner so the agent can
* fall back to Read for those *specific* files without waiting for the
* debounced sync to fire. Other pending files in the project (not
* referenced by this response) get a small footer so the agent has a
* complete picture without bloating the banner.
*
* Cost when nothing is pending the common case is one boolean check.
* No I/O, no parsing of markdown beyond a per-pending-file substring scan.
*/
private withStalenessNotice(result: ToolResult, projectPath?: string): ToolResult {
if (result.isError) return result;
let cg: CodeGraph;
try {
cg = this.getCodeGraph(projectPath);
} catch {
return result; // no default project — leave as is
}
// Cross-project `projectPath` calls open a cached CodeGraph WITHOUT a
// watcher (watchers are only attached to the default session project).
// When the cross-project path happens to be the same project as the
// default cg, the cached instance is the wrong one — its pendingFiles is
// permanently empty. Detect the equal-path case and prefer the default
// cg so the staleness signal still fires when an agent passes the
// explicit projectPath form of its own project.
if (this.cg && cg !== this.cg) {
try {
const sameProject =
resolvePath(this.cg.getProjectRoot()) === resolvePath(cg.getProjectRoot());
if (sameProject) cg = this.cg;
} catch {
/* getProjectRoot may throw on a closed instance — leave cg as is */
}
}
// Defensive: some test fakes inject a partial CodeGraph stub without the
// newer pending-files API. Treat missing/throwing as "no pending files."
let pending: PendingFile[] = [];
try {
pending = cg.getPendingFiles?.() ?? [];
} catch {
return result;
}
if (pending.length === 0) return result;
const [first, ...rest] = result.content;
if (!first || first.type !== 'text') return result;
const text = first.text;
const inResponse: PendingFile[] = [];
const elsewhere: PendingFile[] = [];
for (const p of pending) {
// Substring match against the project-relative POSIX path — that's
// exactly the format both the watcher and every codegraph response
// emit, so a plain includes() is sufficient and avoids regex pitfalls.
if (text.includes(p.path)) inResponse.push(p);
else elsewhere.push(p);
}
let banner = '';
if (inResponse.length > 0) {
banner = formatStaleBanner(inResponse);
}
let footer = '';
if (elsewhere.length > 0) {
footer = formatStaleFooter(elsewhere);
}
if (!banner && !footer) return result;
const composed = [banner, text, footer].filter(Boolean).join('\n\n');
return { ...result, content: [{ type: 'text', text: composed }, ...rest] };
}
/**
* Execute a tool by name
*/
@@ -831,9 +952,12 @@ export class ToolHandler {
if (typeof check === 'object' && check !== undefined) return check;
}
// Read tools resolve through a single result variable so the worktree
// mismatch notice can be prefixed in one place (issue #155). status is
// returned directly — it embeds its own verbose warning.
// Read tools resolve through a single result variable so cross-cutting
// notices — worktree-index mismatch (issue #155) and per-file
// staleness (issue #403) — can be applied in one place. status embeds
// its own verbose worktree warning but still flows through the
// staleness wrapper so its pending-files section stays consistent
// with what the read tools surface.
let result: ToolResult;
switch (toolName) {
case 'codegraph_search':
@@ -851,6 +975,9 @@ export class ToolHandler {
case 'codegraph_node':
result = await this.handleNode(args); break;
case 'codegraph_status':
// status embeds the pending-files list as a first-class section
// (see handleStatus), so we skip the auto-banner wrapper here to
// avoid duplicating the same info at the top of the response.
return await this.handleStatus(args);
case 'codegraph_files':
result = await this.handleFiles(args); break;
@@ -859,7 +986,8 @@ export class ToolHandler {
default:
return this.errorResult(`Unknown tool: ${toolName}`);
}
return this.withWorktreeNotice(result, args.projectPath as string | undefined);
const withWorktree = this.withWorktreeNotice(result, args.projectPath as string | undefined);
return this.withStalenessNotice(withWorktree, args.projectPath as string | undefined);
} catch (err) {
return this.errorResult(`Tool execution failed: ${err instanceof Error ? err.message : String(err)}`);
}
@@ -2016,7 +2144,18 @@ export class ToolHandler {
* Handle codegraph_status
*/
private async handleStatus(args: Record<string, unknown>): Promise<ToolResult> {
const cg = this.getCodeGraph(args.projectPath as string | undefined);
let cg = this.getCodeGraph(args.projectPath as string | undefined);
// Same trick as withStalenessNotice — when an explicit projectPath
// resolves to the same project as the default session cg, prefer the
// default so getPendingFiles() (only populated by the default's watcher)
// is non-empty when there are pending edits.
if (this.cg && cg !== this.cg) {
try {
if (resolvePath(this.cg.getProjectRoot()) === resolvePath(cg.getProjectRoot())) {
cg = this.cg;
}
} catch { /* closed instance — leave as is */ }
}
const stats = cg.getStats();
// Warn when this index actually belongs to a different git working tree
@@ -2073,6 +2212,21 @@ export class ToolHandler {
}
}
// Per-file freshness — the inverse of the auto-prepended staleness banner
// (issue #403). Surfacing it inside `status` gives the agent a single
// place to ask "is the index caught up?" rather than inferring from
// banners on other tool calls.
const pending = cg.getPendingFiles();
if (pending.length > 0) {
lines.push('', '### Pending sync:');
const now = Date.now();
for (const p of pending) {
const ageMs = Math.max(0, now - p.lastSeenMs);
const label = p.indexing ? 'indexing in progress' : 'pending sync';
lines.push(`- ${p.path} (edited ${ageMs}ms ago, ${label})`);
}
}
return this.textResult(lines.join('\n'));
}
+1 -1
View File
@@ -13,7 +13,7 @@
* - Incremental reindexing (in extraction module)
*/
export { FileWatcher, WatchOptions } from './watcher';
export { FileWatcher, WatchOptions, PendingFile } from './watcher';
export { watchDisabledReason, detectWsl } from './watch-policy';
export {
installGitSyncHook,
+163 -6
View File
@@ -45,6 +45,26 @@ export interface WatchOptions {
onSyncError?: (error: Error) => void;
}
/**
* Per-file pending entry tracks a source file the watcher saw an event for
* but hasn't yet synced into the index. Exposed via {@link FileWatcher.getPendingFiles}
* so MCP tool responses can mark stale results without forcing a wait.
*/
export interface PendingFile {
/** Project-relative POSIX path (e.g. "src/foo.ts"). */
path: string;
/** Wall-clock ms at the first event we saw for this path since the last sync. */
firstSeenMs: number;
/** Wall-clock ms at the most recent event we saw for this path. */
lastSeenMs: number;
/**
* True when a sync is currently in flight that began AFTER this file's most
* recent event i.e. the next successful sync will pick it up. False when
* the file is still in the debounce window (no sync running yet).
*/
indexing: boolean;
}
/**
* FileWatcher monitors a project directory for changes and triggers
* debounced sync operations via a provided callback.
@@ -55,13 +75,43 @@ export interface WatchOptions {
* - Debounced to avoid thrashing on rapid saves
* - Filters to supported source files by extension
* - Ignores .codegraph/ and .git/ regardless of .gitignore
* - Tracks per-file pending state so MCP tools can flag stale results
* without blocking on a sync (issue #403)
*/
export class FileWatcher {
private watcher: FSWatcher | null = null;
private debounceTimer: ReturnType<typeof setTimeout> | null = null;
private hasChanges = false;
/**
* Files seen by the watcher since the last successful sync populated on
* every chokidar event, cleared at the start of a sync, and re-populated by
* events that arrive mid-sync (or restored on sync failure). Keyed by the
* same project-relative POSIX path the rest of the codebase uses, so a
* caller can intersect tool-response file paths against this map cheaply.
*/
private pendingFiles = new Map<string, { firstSeenMs: number; lastSeenMs: number }>();
/**
* Wall-clock ms at which the in-flight sync began. Combined with
* {@link pendingFiles}'s `lastSeenMs`, this distinguishes "still in the
* debounce window" (lastSeen > syncStarted, sync hasn't started yet for
* this edit) from "currently being indexed" (lastSeen <= syncStarted).
*/
private syncStartedMs = 0;
private syncing = false;
private stopped = false;
/**
* False until chokidar fires its `ready` event. Gates `pendingFiles`
* insertion so the initial crawl's `add` events (one per pre-existing
* source file) don't pollute the per-file staleness signal. The events
* still flow into `scheduleSync()` to preserve the previous "initial
* scan triggers a reconciling sync" behavior.
*/
private chokidarReady = false;
/**
* Callbacks that resolve when chokidar fires `ready`. Used by tests (and
* any production caller that cares about a clean baseline) to deterministically
* gate on the end of the initial scan instead of guessing at a sleep duration.
*/
private readyWaiters: Array<() => void> = [];
// The shared ignore matcher (built-in defaults + project .gitignore), built
// once at start(). Same source of truth the indexer uses, so watcher scope
// can never diverge from index scope.
@@ -116,6 +166,26 @@ export class FileWatcher {
ignored: (testPath: string, stats?: Stats) => this.shouldIgnore(testPath, stats),
});
// Chokidar emits `add` for every pre-existing source file during its
// initial scan. Those events should still trigger the post-startup
// reconciling sync (preserving prior behavior), but they must NOT land
// in pendingFiles — otherwise every file in the project shows up as
// "edited but not indexed" on startup, which is the opposite of the
// signal #403 is supposed to provide. Flip the flag on chokidar's
// `ready` event; from then on, real edits populate pendingFiles.
//
// We also clear `pendingFiles` here as defense-in-depth: chokidar can
// emit late initial-scan `add` events via setImmediate AFTER the
// `ready` callback runs (observed under test-parallelism load).
// Clearing once at ready guarantees a clean baseline; real subsequent
// edits repopulate the set normally.
this.watcher.on('ready', () => {
this.chokidarReady = true;
this.pendingFiles.clear();
for (const cb of this.readyWaiters) cb();
this.readyWaiters.length = 0;
});
// chokidar emits 'all' for every event type; we only sync source files.
this.watcher.on('all', (_event: string, filePath: string) => {
if (this.stopped) return;
@@ -128,7 +198,17 @@ export class FileWatcher {
if (!isSourceFile(normalized)) return;
logDebug('File change detected', { file: normalized });
this.hasChanges = true;
// Only track events from after chokidar's initial scan as pending
// edits — pre-existing files on disk are already represented by
// (or about to be reconciled by) the index, not a user edit.
if (this.chokidarReady) {
const now = Date.now();
const existing = this.pendingFiles.get(normalized);
this.pendingFiles.set(normalized, {
firstSeenMs: existing?.firstSeenMs ?? now,
lastSeenMs: now,
});
}
this.scheduleSync();
});
@@ -187,7 +267,8 @@ export class FileWatcher {
this.watcher = null;
}
this.hasChanges = false;
this.pendingFiles.clear();
this.chokidarReady = false;
this.ignoreMatcher = null;
logDebug('File watcher stopped');
}
@@ -199,6 +280,30 @@ export class FileWatcher {
return this.watcher !== null && !this.stopped;
}
/**
* Resolves once chokidar has fired its `ready` event (or immediately if
* it has already done so). Useful for tests that need a deterministic
* boundary before asserting on `pendingFiles` guessing a sleep duration
* is flaky under load because chokidar can take longer than expected to
* finish its initial crawl on slow filesystems / parallel test runs.
*
* Production callers don't need this: `pendingFiles` is read continuously,
* the staleness banner is always correct (empty or populated), and the
* initial-scan window is a small one-time startup cost.
*/
waitUntilReady(timeoutMs = 10000): Promise<void> {
if (this.chokidarReady) return Promise.resolve();
return new Promise((resolve, reject) => {
const t = setTimeout(() => {
const idx = this.readyWaiters.indexOf(handler);
if (idx >= 0) this.readyWaiters.splice(idx, 1);
reject(new Error(`FileWatcher.waitUntilReady timed out after ${timeoutMs}ms`));
}, timeoutMs);
const handler = () => { clearTimeout(t); resolve(); };
this.readyWaiters.push(handler);
});
}
/**
* Schedule a debounced sync.
*/
@@ -214,28 +319,80 @@ export class FileWatcher {
/**
* Flush pending changes by running sync.
*
* pendingFiles is NOT cleared at the start of sync entries are removed
* only after sync commits successfully, and only for entries whose
* lastSeenMs <= syncStartedMs. That way, a query that arrives mid-sync
* still sees the affected files marked stale (the DB hasn't been updated
* yet), and an event that lands mid-sync persists into the follow-up.
*
* On sync failure pendingFiles is left untouched every edit is still
* unindexed, and the rescheduled sync will absorb the same set next time.
*/
private async flush(): Promise<void> {
// If already syncing, the post-sync check will re-trigger
if (this.syncing || this.stopped) return;
this.hasChanges = false;
this.syncStartedMs = Date.now();
this.syncing = true;
try {
const result = await this.syncFn();
// Remove entries whose most recent event predates this sync — those
// edits are now in the DB. Entries with lastSeenMs > syncStartedMs
// arrived mid-sync; whether the in-flight sync captured them depends
// on when sync read that file, so we keep them as pending and let
// the follow-up sync handle them. We prefer false positives ("shown
// stale, actually fresh" → at worst one extra Read) over false
// negatives ("shown fresh, actually stale" → misleads the agent).
for (const [filePath, info] of this.pendingFiles) {
if (info.lastSeenMs <= this.syncStartedMs) {
this.pendingFiles.delete(filePath);
}
}
this.onSyncComplete?.(result);
} catch (err) {
const error = err instanceof Error ? err : new Error(String(err));
logWarn('Watch sync failed', { error: error.message });
// Failure: leave pendingFiles untouched. Every edit it tracks is
// still unindexed; the rescheduled sync sees the same set.
this.onSyncError?.(error);
} finally {
this.syncing = false;
// If new changes arrived during sync, schedule another
if (this.hasChanges && !this.stopped) {
// If pending files remain (mid-sync events, or this sync failed),
// schedule another pass.
if (this.pendingFiles.size > 0 && !this.stopped) {
this.scheduleSync();
}
}
}
/**
* Snapshot of files seen by the watcher since the last successful sync.
*
* Used by MCP tool responses to mark stale results without blocking on a
* sync: a tool that returns a hit in `src/foo.ts` while `src/foo.ts` is in
* this list tells the agent "Read this file directly, the index lags."
*
* `indexing` is true when a sync is currently in flight whose start time is
* AFTER this file's most recent event i.e. that sync will absorb the
* edit. False means the file is still inside the debounce window and no
* sync has started yet (a follow-up call a few hundred ms later may show
* `indexing: true` or the file may have left the list entirely).
*
* Cheap: O(pendingFiles.size), no I/O, no locks.
*/
getPendingFiles(): PendingFile[] {
const result: PendingFile[] = [];
for (const [filePath, info] of this.pendingFiles) {
result.push({
path: filePath,
firstSeenMs: info.firstSeenMs,
lastSeenMs: info.lastSeenMs,
indexing: this.syncing && this.syncStartedMs >= info.lastSeenMs,
});
}
return result;
}
}