feat(ui): live refresh and drift banners — the viewer keeps up with the project (CG-53)
`GET /api/events` is a server-sent-event stream the viewer holds open for the life of the page. Two signals, two things the browser could not know: changed source files touched on disk, before any sync — the drift banner index the graph moved, naming what the sync re-indexed — the live refresh The server WATCHES and never syncs: the project tree through the engine's own FileWatcher with a notify-only syncFn, the index through one non-recursive fs.watch on the data directory settled at 400 ms. Both start with the first subscriber and stop with the last, so a viewer nobody has open costs no watch descriptors. Nothing polls, on either side. Drift is now parity with codegraph_node (#1474) rather than an absence. `/api/source?ondrift=current` serves a drifted file's CURRENT bytes flagged `showing: 'current'`, and the three screens that can say so switch off everything anchored to the old line numbering — gutter ports, call-site links, call arcs, the callee rail's anchoring — while keeping the source. The banner is paper-2 with a hairline rule, never amber: amber belongs to the untested badge. Also fixes a stale read this exposed. A long-lived reader holds an LRU of nodes by id that only its own writes invalidate, so `/api/node/<id>` kept answering with a symbol another process's sync had deleted while `/api/search` beside it said it was gone. GraphSession now drops the read caches when the database (or its WAL) has been written, and the Symbol view follows a symbol whose id changed because an edit above it moved its start line, carrying the trail across. Measured on a live viewer: banner 360 ms after a save, toast 440 ms after `codegraph sync` returns, 0 requests in 4 idle seconds, and the client gives up reconnecting after ~90 s with "Not live" rather than hammering a dead port.
This commit is contained in:
Colby McHenry
@@ -0,0 +1,480 @@
|
||||
/**
|
||||
* `GET /api/events` — the viewer's live channel (server-sent events).
|
||||
*
|
||||
* Two questions the open browser cannot answer for itself, and one stream that
|
||||
* answers both:
|
||||
*
|
||||
* - **"has the file I'm looking at changed on disk?"** — the drift banner. The
|
||||
* verdict itself comes from `/api/source` (it hashes the bytes); this stream
|
||||
* only says *when to ask again*, so the banner appears about a third of a
|
||||
* second after a save instead of on the next navigation.
|
||||
* - **"has the index moved?"** — the live refresh. Something else (an agent's
|
||||
* MCP daemon, `codegraph sync`, a git hook) writes the graph; when it does,
|
||||
* every screen the viewer is showing is one round-trip out of date.
|
||||
*
|
||||
* ## This server watches. It never syncs.
|
||||
*
|
||||
* `codegraph ui` is read-only in every sense — the banner it prints says so —
|
||||
* so the obvious implementation (run the engine's watcher, let it sync) is out.
|
||||
* What is left is *observation*, from two independent directions:
|
||||
*
|
||||
* - the project tree, through the engine's own {@link FileWatcher} with a
|
||||
* notify-only `syncFn`. It never writes: the callback that would have run a
|
||||
* sync fans the changed paths out to the browser instead. Everything else
|
||||
* about it — the per-platform watch strategy, the indexer's ignore scope, the
|
||||
* adaptive debounce, the degrade latch — is behaviour we would otherwise have
|
||||
* had to write again, worse.
|
||||
* - the index itself, through one non-recursive `fs.watch` on the data
|
||||
* directory. That is the only cross-process signal there is: the writer is a
|
||||
* different process, and the thing it changes is a file. A settled write is
|
||||
* followed by ONE cheap query (`getIndexRevision`), and only a revision that
|
||||
* actually moved becomes an event.
|
||||
*
|
||||
* **Nothing polls.** Both watchers are edge-triggered, and both start on the
|
||||
* first subscriber and stop with the last one — a viewer nobody has open costs
|
||||
* no watch descriptors, which matters on Linux where the strategy is
|
||||
* per-directory.
|
||||
*
|
||||
* ## Boundary
|
||||
*
|
||||
* A long-lived response sits inside the loopback boundary exactly like every
|
||||
* other route: `Host`, `Origin` and the GET-only rule are already enforced by
|
||||
* `startUiServer` before this module is reached, and nothing here reads the
|
||||
* repository — the paths it names came from the watcher and the index, and the
|
||||
* viewer has to go back through `/api/source` (and therefore through
|
||||
* `resolveProjectFile`) to see a byte of any of them.
|
||||
*/
|
||||
|
||||
import * as fs from 'fs';
|
||||
import type { IncomingMessage, ServerResponse } from 'http';
|
||||
import type { CodeGraph } from '../../index';
|
||||
import { getCodeGraphDir } from '../../directory';
|
||||
import { FileWatcher } from '../../sync/watcher';
|
||||
import type { GraphSession } from './session';
|
||||
|
||||
/**
|
||||
* Paths carried in one event. `total` is always the real number — a burst of
|
||||
* two thousand files still says two thousand, it just does not list them.
|
||||
*/
|
||||
export const MAX_EVENT_FILES = 200;
|
||||
|
||||
/** Comment frame keeping the connection (and the client's idea of it) alive. */
|
||||
export const HEARTBEAT_MS = 25_000;
|
||||
|
||||
/**
|
||||
* Quiet window before an index write is treated as finished.
|
||||
*
|
||||
* A sync writes the WAL continuously, so the *end* of the writing is the signal
|
||||
* — not its start. Long enough that a multi-second sync produces one event
|
||||
* rather than a dozen.
|
||||
*/
|
||||
const INDEX_SETTLE_MS = 400;
|
||||
|
||||
/**
|
||||
* Ceiling on that quiet window. A sync large enough that the WAL never goes
|
||||
* quiet for 400 ms would otherwise hold the first event until it finished; the
|
||||
* cap makes the viewer refresh mid-way instead, which is still true — the graph
|
||||
* really has moved — and costs one query.
|
||||
*/
|
||||
const INDEX_SETTLE_MAX_MS = 3_000;
|
||||
|
||||
/**
|
||||
* Debounce for source-file events. The watcher's own adaptive rule fires a lone
|
||||
* save after `min(300, this)` ms of quiet and keeps the full window for a
|
||||
* burst, so a single edit reaches the browser well inside the one-second bar
|
||||
* while an agent rewriting forty files still arrives as one event.
|
||||
*/
|
||||
const SOURCE_DEBOUNCE_MS = 500;
|
||||
|
||||
/* --------------------------------------------------------------- the wire -- */
|
||||
|
||||
export interface WireIndexRevision {
|
||||
lastIndexedAt: number | null;
|
||||
files: number;
|
||||
}
|
||||
|
||||
/** Sent once, immediately, so a client knows what it is synchronised against. */
|
||||
export interface WireEventHello {
|
||||
type: 'hello';
|
||||
index: WireIndexRevision | null;
|
||||
/** Which of the two observers actually came up. */
|
||||
watching: { source: boolean; index: boolean };
|
||||
/** Non-null when live watching has given up; the client must NOT start polling. */
|
||||
degraded: string | null;
|
||||
heartbeatMs: number;
|
||||
at: number;
|
||||
}
|
||||
|
||||
/** Source files changed on disk. The index has NOT caught up yet. */
|
||||
export interface WireEventChanged {
|
||||
type: 'changed';
|
||||
files: string[];
|
||||
total: number;
|
||||
truncated: boolean;
|
||||
/**
|
||||
* True when the change could not be described file by file (a directory
|
||||
* removal, or a burst past the watcher's scoped ceiling). Treat any open file
|
||||
* as possibly affected.
|
||||
*/
|
||||
scan: boolean;
|
||||
at: number;
|
||||
}
|
||||
|
||||
/** The index moved: some other process finished writing the graph. */
|
||||
export interface WireEventIndex {
|
||||
type: 'index';
|
||||
index: WireIndexRevision;
|
||||
/** Files this sync re-indexed, newest first. Empty when it only deleted. */
|
||||
files: string[];
|
||||
total: number;
|
||||
truncated: boolean;
|
||||
at: number;
|
||||
}
|
||||
|
||||
/** Live watching has stopped for good. Sent once; the stream stays open. */
|
||||
export interface WireEventDegraded {
|
||||
type: 'degraded';
|
||||
reason: string;
|
||||
at: number;
|
||||
}
|
||||
|
||||
export type WireEvent =
|
||||
| WireEventHello
|
||||
| WireEventChanged
|
||||
| WireEventIndex
|
||||
| WireEventDegraded;
|
||||
|
||||
/* ---------------------------------------------------------------- the hub -- */
|
||||
|
||||
interface Client {
|
||||
res: ServerResponse;
|
||||
heartbeat: ReturnType<typeof setInterval>;
|
||||
}
|
||||
|
||||
/**
|
||||
* Fans filesystem and index changes out to every open viewer.
|
||||
*
|
||||
* One hub per server. It owns the watchers, and owns them lazily: they exist
|
||||
* only while somebody is listening.
|
||||
*/
|
||||
export class EventHub {
|
||||
private readonly projectRoot: string;
|
||||
private readonly session: GraphSession;
|
||||
private readonly clients = new Set<Client>();
|
||||
|
||||
private sourceWatcher: FileWatcher | null = null;
|
||||
private indexWatcher: fs.FSWatcher | null = null;
|
||||
private indexTimer: ReturnType<typeof setTimeout> | null = null;
|
||||
/** When the current settle window started, for the {@link INDEX_SETTLE_MAX_MS} cap. */
|
||||
private indexPendingSince = 0;
|
||||
private revision: WireIndexRevision | null = null;
|
||||
private sourceUp = false;
|
||||
private indexUp = false;
|
||||
private degraded: string | null = null;
|
||||
private closed = false;
|
||||
|
||||
constructor(projectRoot: string, session: GraphSession) {
|
||||
this.projectRoot = projectRoot;
|
||||
this.session = session;
|
||||
}
|
||||
|
||||
/**
|
||||
* Attach one browser to the stream.
|
||||
*
|
||||
* Returns `true` in every case — the response is answered here, streaming or
|
||||
* not — so it slots into the API's `switch` like any other endpoint.
|
||||
*/
|
||||
subscribe(req: IncomingMessage, res: ServerResponse, method: string): true {
|
||||
if (this.closed) {
|
||||
// The server is shutting down. Answer, do not attach: a client that got a
|
||||
// stream here would hold the socket open against `close()`.
|
||||
res.writeHead(503, {
|
||||
'Content-Type': 'application/json; charset=utf-8',
|
||||
'Cache-Control': 'no-store',
|
||||
});
|
||||
res.end(method === 'HEAD' ? undefined : JSON.stringify({ error: 'Shutting down.', code: 'internal' }));
|
||||
return true;
|
||||
}
|
||||
|
||||
res.writeHead(200, {
|
||||
'Content-Type': 'text/event-stream; charset=utf-8',
|
||||
'Cache-Control': 'no-store',
|
||||
// Node would otherwise chunk small writes; an event that sits in a buffer
|
||||
// is an event that did not happen.
|
||||
Connection: 'keep-alive',
|
||||
'X-Accel-Buffering': 'no',
|
||||
});
|
||||
|
||||
if (method === 'HEAD') {
|
||||
res.end();
|
||||
return true;
|
||||
}
|
||||
|
||||
// No keep-alive timeout on this socket: the server sets one globally so
|
||||
// Ctrl-C does not wait on browser connections, and it would close a healthy
|
||||
// stream between heartbeats.
|
||||
res.socket?.setTimeout(0);
|
||||
res.socket?.setNoDelay(true);
|
||||
|
||||
this.ensureWatching();
|
||||
|
||||
const client: Client = {
|
||||
res,
|
||||
heartbeat: setInterval(() => {
|
||||
// A comment frame. Not an event, so no client handler ever sees it —
|
||||
// it exists to notice a socket the other end has already dropped.
|
||||
if (!res.writableEnded) res.write(': ping\n\n');
|
||||
}, HEARTBEAT_MS),
|
||||
};
|
||||
// `unref` so a live stream never keeps the process alive on its own.
|
||||
client.heartbeat.unref?.();
|
||||
this.clients.add(client);
|
||||
|
||||
const drop = (): void => this.drop(client);
|
||||
res.on('close', drop);
|
||||
res.on('error', drop);
|
||||
req.on('aborted', drop);
|
||||
|
||||
this.send(client, {
|
||||
type: 'hello',
|
||||
index: this.revision,
|
||||
watching: { source: this.sourceUp, index: this.indexUp },
|
||||
degraded: this.degraded,
|
||||
heartbeatMs: HEARTBEAT_MS,
|
||||
at: Date.now(),
|
||||
});
|
||||
return true;
|
||||
}
|
||||
|
||||
/** Number of attached clients — for tests and for the watchers' lifetime. */
|
||||
get size(): number {
|
||||
return this.clients.size;
|
||||
}
|
||||
|
||||
/** Stop watching and end every open stream. Idempotent. */
|
||||
close(): void {
|
||||
this.closed = true;
|
||||
this.stopWatching();
|
||||
for (const client of [...this.clients]) {
|
||||
clearInterval(client.heartbeat);
|
||||
this.clients.delete(client);
|
||||
try {
|
||||
client.res.end();
|
||||
} catch {
|
||||
/* the socket is already gone */
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/* ------------------------------------------------------------ plumbing -- */
|
||||
|
||||
private drop(client: Client): void {
|
||||
if (!this.clients.delete(client)) return;
|
||||
clearInterval(client.heartbeat);
|
||||
if (this.clients.size === 0) this.stopWatching();
|
||||
}
|
||||
|
||||
private send(client: Client, event: WireEvent): void {
|
||||
if (client.res.writableEnded) return;
|
||||
try {
|
||||
// `retry` on every frame is cheap and means a client that reconnects with
|
||||
// the browser's own EventSource still backs off the way we asked.
|
||||
client.res.write(`retry: 3000\nevent: ${event.type}\ndata: ${JSON.stringify(event)}\n\n`);
|
||||
} catch {
|
||||
this.drop(client);
|
||||
}
|
||||
}
|
||||
|
||||
private broadcast(event: WireEvent): void {
|
||||
for (const client of [...this.clients]) this.send(client, event);
|
||||
}
|
||||
|
||||
/* ------------------------------------------------------------ watching -- */
|
||||
|
||||
private ensureWatching(): void {
|
||||
if (this.closed) return;
|
||||
this.revision ??= this.probe();
|
||||
this.startSourceWatcher();
|
||||
this.startIndexWatcher();
|
||||
}
|
||||
|
||||
private stopWatching(): void {
|
||||
if (this.indexTimer) {
|
||||
clearTimeout(this.indexTimer);
|
||||
this.indexTimer = null;
|
||||
}
|
||||
this.indexPendingSince = 0;
|
||||
try {
|
||||
this.indexWatcher?.close();
|
||||
} catch {
|
||||
/* already closed */
|
||||
}
|
||||
this.indexWatcher = null;
|
||||
this.indexUp = false;
|
||||
this.sourceWatcher?.stop();
|
||||
this.sourceWatcher = null;
|
||||
this.sourceUp = false;
|
||||
}
|
||||
|
||||
/**
|
||||
* The project tree, through the engine's watcher with the sync taken out.
|
||||
*
|
||||
* The `syncFn` is the whole trick: the watcher calls it with exactly the
|
||||
* paths it would have handed to a scoped sync (or `undefined` when the events
|
||||
* could not describe the change), we announce them and report zero files
|
||||
* changed. It succeeds every time, so the watcher's failure ladder — lock
|
||||
* retries, backoff, the degrade latch — is only ever reached by the watch
|
||||
* layer itself, which is precisely the part we do want.
|
||||
*/
|
||||
private startSourceWatcher(): void {
|
||||
if (this.sourceWatcher) return;
|
||||
const watcher = new FileWatcher(
|
||||
this.projectRoot,
|
||||
async (paths?: string[]) => {
|
||||
this.announceChanged(paths);
|
||||
return { filesChanged: 0, durationMs: 0 };
|
||||
},
|
||||
{
|
||||
debounceMs: SOURCE_DEBOUNCE_MS,
|
||||
onDegraded: (reason) => {
|
||||
this.degraded = reason;
|
||||
this.sourceUp = false;
|
||||
this.broadcast({ type: 'degraded', reason, at: Date.now() });
|
||||
},
|
||||
}
|
||||
);
|
||||
this.sourceWatcher = watcher;
|
||||
this.sourceUp = watcher.start();
|
||||
if (!this.sourceUp) {
|
||||
// Watching is off by policy (CODEGRAPH_NO_WATCH, a WSL2 /mnt drive) or
|
||||
// the OS refused. The stream stays — the index watcher is independent —
|
||||
// and `hello` already told the client which half is live.
|
||||
this.sourceWatcher = null;
|
||||
}
|
||||
}
|
||||
|
||||
private announceChanged(paths?: string[]): void {
|
||||
const all = paths ?? [];
|
||||
const files = all.slice(0, MAX_EVENT_FILES);
|
||||
this.broadcast({
|
||||
type: 'changed',
|
||||
files,
|
||||
total: all.length,
|
||||
truncated: files.length < all.length,
|
||||
scan: paths === undefined,
|
||||
at: Date.now(),
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* The index, through one watch on the data directory.
|
||||
*
|
||||
* Non-recursive and on the directory rather than the database file: SQLite
|
||||
* writes land in `codegraph.db-wal`, and a full re-index REPLACES
|
||||
* `codegraph.db` outright (a watch on the file itself would follow the
|
||||
* unlinked inode and never fire again).
|
||||
*/
|
||||
private startIndexWatcher(): void {
|
||||
if (this.indexWatcher) return;
|
||||
const dir = getCodeGraphDir(this.projectRoot);
|
||||
try {
|
||||
const watcher = fs.watch(dir, { persistent: false }, () => this.scheduleProbe());
|
||||
watcher.on('error', () => {
|
||||
// The data directory went away, or the OS dropped the watch. Nothing to
|
||||
// retry against — a client that reloads gets a fresh one.
|
||||
this.indexUp = false;
|
||||
this.indexWatcher = null;
|
||||
try {
|
||||
watcher.close();
|
||||
} catch {
|
||||
/* already closed */
|
||||
}
|
||||
});
|
||||
this.indexWatcher = watcher;
|
||||
this.indexUp = true;
|
||||
} catch {
|
||||
this.indexUp = false;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Wait for the writing to stop, then look once.
|
||||
*
|
||||
* Re-armed by every write, so a sync that takes four seconds produces one
|
||||
* probe at its end — except that {@link INDEX_SETTLE_MAX_MS} caps how long
|
||||
* the first probe can be deferred, so a continuously-writing full index still
|
||||
* refreshes the viewer while it runs.
|
||||
*/
|
||||
private scheduleProbe(): void {
|
||||
if (this.closed) return;
|
||||
const now = Date.now();
|
||||
if (this.indexPendingSince === 0) this.indexPendingSince = now;
|
||||
const remaining = Math.max(0, this.indexPendingSince + INDEX_SETTLE_MAX_MS - now);
|
||||
if (this.indexTimer) clearTimeout(this.indexTimer);
|
||||
const timer = setTimeout(() => {
|
||||
this.indexTimer = null;
|
||||
this.indexPendingSince = 0;
|
||||
this.checkIndex();
|
||||
}, Math.min(INDEX_SETTLE_MS, remaining));
|
||||
timer.unref?.();
|
||||
this.indexTimer = timer;
|
||||
}
|
||||
|
||||
/** One query. An unmoved revision is not an event. */
|
||||
private checkIndex(): void {
|
||||
if (this.closed || this.clients.size === 0) return;
|
||||
const next = this.probe();
|
||||
if (next === null) return;
|
||||
const previous = this.revision;
|
||||
this.revision = next;
|
||||
if (
|
||||
previous !== null &&
|
||||
previous.lastIndexedAt === next.lastIndexedAt &&
|
||||
previous.files === next.files
|
||||
) {
|
||||
return;
|
||||
}
|
||||
|
||||
// Everything re-indexed since the mark we were holding. A sync that only
|
||||
// removed files names nothing here — which is why the revision comparison
|
||||
// above, not this list, decides whether an event happens at all.
|
||||
let files: string[] = [];
|
||||
let total = 0;
|
||||
const since = previous?.lastIndexedAt ?? null;
|
||||
if (since !== null) {
|
||||
try {
|
||||
const changed = this.session.acquire().getFilesIndexedSince(since, MAX_EVENT_FILES);
|
||||
files = changed.paths;
|
||||
total = changed.total;
|
||||
} catch {
|
||||
/* the index went away between the probe and here — the event still stands */
|
||||
}
|
||||
}
|
||||
|
||||
this.broadcast({
|
||||
type: 'index',
|
||||
index: next,
|
||||
files,
|
||||
total: Math.max(total, files.length),
|
||||
truncated: files.length < total,
|
||||
at: Date.now(),
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* The current revision, or null when there is no readable index.
|
||||
*
|
||||
* A missing index is not an error here: `codegraph ui` refuses to start
|
||||
* without one, but a user can delete `.codegraph/` with the viewer open, and
|
||||
* every endpoint already says so in its own words when asked.
|
||||
*/
|
||||
private probe(): WireIndexRevision | null {
|
||||
try {
|
||||
const cg: CodeGraph = this.session.acquire();
|
||||
const revision = cg.getIndexRevision();
|
||||
return { lastIndexedAt: revision.lastIndexedAt, files: revision.fileCount };
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -3,8 +3,10 @@
|
||||
*
|
||||
* Eleven endpoints, one per screen, each answering in a single round-trip — the
|
||||
* same principle as `codegraph_explore`: return enough that the caller does not
|
||||
* have to ask a follow-up question. Everything here is a *reader* of the
|
||||
* existing schema; nothing indexes, resolves, or writes.
|
||||
* have to ask a follow-up question — plus one that does not answer at all and
|
||||
* stays open instead (`/api/events`), so a screen learns that its answer went
|
||||
* stale rather than waiting to be asked again. Everything here is a *reader* of
|
||||
* the existing schema; nothing indexes, resolves, or writes.
|
||||
*
|
||||
* ```
|
||||
* GET /api/stats what this index is and how much to trust it
|
||||
@@ -18,6 +20,7 @@
|
||||
* GET /api/entrypoints where to start reading: routes, roots, hubs
|
||||
* GET /api/map?root=&depth= the module map: modules, links, cycles
|
||||
* GET /api/flow?from=&to= the flow strip: one card per hop
|
||||
* GET /api/events the live channel (SSE): drift and refresh
|
||||
* ```
|
||||
*
|
||||
* It mounts on the `api` seam of `startUiServer`, which means it sits *behind*
|
||||
@@ -43,6 +46,7 @@ import { buildEntryPoints } from './entrypoints';
|
||||
import { buildNodeRefs } from './nodes';
|
||||
import { buildMap } from './map';
|
||||
import { buildFlow } from './flow';
|
||||
import { EventHub } from './events';
|
||||
|
||||
export { GraphSession } from './session';
|
||||
export { ApiError } from './respond';
|
||||
@@ -63,6 +67,15 @@ export type {
|
||||
WireFileCall,
|
||||
WireFileOutsideRef,
|
||||
} from './filecode';
|
||||
export { EventHub, MAX_EVENT_FILES, HEARTBEAT_MS } from './events';
|
||||
export type {
|
||||
WireEvent,
|
||||
WireEventHello,
|
||||
WireEventChanged,
|
||||
WireEventIndex,
|
||||
WireEventDegraded,
|
||||
WireIndexRevision,
|
||||
} from './events';
|
||||
export type {
|
||||
WireMapPayload,
|
||||
WireMapModule,
|
||||
@@ -117,6 +130,11 @@ const API_INDEX = {
|
||||
description: 'The call path between symbols: one hop per card, opened at the calling line.',
|
||||
params: ['from', 'to', 'symbols', 'hop', 'limit'],
|
||||
},
|
||||
{
|
||||
path: '/api/events',
|
||||
description:
|
||||
'Live channel (server-sent events): source files that changed on disk, and the index moving.',
|
||||
},
|
||||
{
|
||||
path: '/api/entrypoints',
|
||||
description: 'Where to start reading: routes, files that run something, and hubs.',
|
||||
@@ -127,10 +145,13 @@ const API_INDEX = {
|
||||
|
||||
export function createGraphApi(options: GraphApiOptions): GraphApi {
|
||||
const session = new GraphSession(options.projectRoot);
|
||||
// Watches nothing until a browser subscribes, and stops again when the last
|
||||
// one goes away — mounting the API costs no watch descriptors.
|
||||
const events = new EventHub(options.projectRoot, session);
|
||||
|
||||
// Async because `/api/source` highlights: everything else answers straight
|
||||
// out of SQLite and resolves on the same tick.
|
||||
const handler: UiApiHandler = async (_req, res, ctx) => {
|
||||
const handler: UiApiHandler = async (req, res, ctx) => {
|
||||
const route = normalize(ctx.pathname);
|
||||
try {
|
||||
switch (route) {
|
||||
@@ -152,6 +173,10 @@ export function createGraphApi(options: GraphApiOptions): GraphApi {
|
||||
return ok(res, await buildSource(session.acquire(), ctx.projectRoot, ctx.query), ctx.method);
|
||||
case '/api/flow':
|
||||
return ok(res, await buildFlow(session.acquire(), ctx.projectRoot, ctx.query), ctx.method);
|
||||
case '/api/events':
|
||||
// Streams instead of answering: it writes its own headers and keeps
|
||||
// the socket open, so it never goes through `ok()`.
|
||||
return events.subscribe(req, res, ctx.method);
|
||||
default:
|
||||
return dispatchPathRoutes(route, res, ctx, session);
|
||||
}
|
||||
@@ -166,7 +191,15 @@ export function createGraphApi(options: GraphApiOptions): GraphApi {
|
||||
}
|
||||
};
|
||||
|
||||
return { handler, close: () => session.close() };
|
||||
return {
|
||||
handler,
|
||||
close: () => {
|
||||
// Streams first: a client still attached would hold the socket open
|
||||
// against the server's own close.
|
||||
events.close();
|
||||
session.close();
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -15,6 +15,15 @@
|
||||
* inode and happily serve a graph that no longer exists on disk. So the file
|
||||
* identity is re-checked on acquisition — one `stat` — and a swapped file
|
||||
* reopens the connection.
|
||||
* - **A sync by ANOTHER process must not be served from memory.** The same
|
||||
* `stat` also notices the database growing, and when it has, the read caches
|
||||
* go (`dropReadCaches`). The query layer holds an LRU of nodes by id which
|
||||
* only a write through *this* instance invalidates, so without it
|
||||
* `/api/node/<id>` keeps answering with a symbol an agent's sync deleted
|
||||
* while `/api/search` — which never caches — correctly says it is gone. That
|
||||
* is not a stale screen, it is two screens contradicting each other; and
|
||||
* because a node's id contains its start line, ANY edit above a symbol
|
||||
* renames it, so this is the common case rather than the corner one.
|
||||
*/
|
||||
|
||||
import * as fs from 'fs';
|
||||
@@ -23,16 +32,44 @@ import { getDatabasePath } from '../../db';
|
||||
import { isInitialized } from '../../directory';
|
||||
import { ApiError } from './respond';
|
||||
|
||||
/** Identity of the database file, so a swap underneath us is detectable. */
|
||||
/**
|
||||
* Identity of the database file, so a swap underneath us is detectable — plus
|
||||
* the marks that say it was WRITTEN to without being replaced.
|
||||
*
|
||||
* The WAL is measured as well as the database: in WAL mode a commit lands in
|
||||
* `codegraph.db-wal` and may not touch `codegraph.db` until a checkpoint, so a
|
||||
* whole sync can go by with the main file's size and mtime unchanged.
|
||||
*/
|
||||
interface FileIdentity {
|
||||
ino: number;
|
||||
birthtimeMs: number;
|
||||
size: number;
|
||||
mtimeMs: number;
|
||||
walSize: number;
|
||||
walMtimeMs: number;
|
||||
}
|
||||
|
||||
function identify(dbPath: string): FileIdentity | null {
|
||||
try {
|
||||
const st = fs.statSync(dbPath);
|
||||
return { ino: st.ino, birthtimeMs: st.birthtimeMs };
|
||||
let walSize = 0;
|
||||
let walMtimeMs = 0;
|
||||
try {
|
||||
const wal = fs.statSync(`${dbPath}-wal`);
|
||||
walSize = wal.size;
|
||||
walMtimeMs = wal.mtimeMs;
|
||||
} catch {
|
||||
// No WAL sidecar: either not in WAL mode, or fully checkpointed. Both are
|
||||
// "nothing pending", which is what zeroes mean here.
|
||||
}
|
||||
return {
|
||||
ino: st.ino,
|
||||
birthtimeMs: st.birthtimeMs,
|
||||
size: st.size,
|
||||
mtimeMs: st.mtimeMs,
|
||||
walSize,
|
||||
walMtimeMs,
|
||||
};
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
@@ -45,6 +82,17 @@ function sameFile(a: FileIdentity | null, b: FileIdentity | null): boolean {
|
||||
return a.ino === b.ino && a.birthtimeMs === b.birthtimeMs;
|
||||
}
|
||||
|
||||
/** Same file, but written to since we last looked. */
|
||||
function sameContent(a: FileIdentity | null, b: FileIdentity | null): boolean {
|
||||
if (a === null || b === null) return false;
|
||||
return (
|
||||
a.size === b.size &&
|
||||
a.mtimeMs === b.mtimeMs &&
|
||||
a.walSize === b.walSize &&
|
||||
a.walMtimeMs === b.walMtimeMs
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Guidance shown when there is no index to read. Deliberately the same three
|
||||
* facts the CLI prints: the viewer never creates an index, `codegraph init`
|
||||
@@ -87,7 +135,17 @@ export class GraphSession {
|
||||
const current = identify(this.dbPath);
|
||||
|
||||
if (this.cg !== null) {
|
||||
if (sameFile(this.identity, current)) return this.cg;
|
||||
if (sameFile(this.identity, current)) {
|
||||
// Same file, but somebody wrote to it. SQLite itself is fine — a WAL
|
||||
// reader sees the new commits — but our in-memory node cache is not,
|
||||
// so it goes. One `stat` already paid for; clearing a bounded Map is
|
||||
// the whole cost.
|
||||
if (!sameContent(this.identity, current)) {
|
||||
this.identity = current;
|
||||
this.cg.dropReadCaches();
|
||||
}
|
||||
return this.cg;
|
||||
}
|
||||
// The database was replaced (a re-index) or removed. Drop the stale
|
||||
// handle; falling through re-opens against whatever is there now.
|
||||
this.closeQuietly();
|
||||
|
||||
+101
-18
@@ -10,13 +10,22 @@
|
||||
* `?file=../../.ssh/id_rsa` is a credential leak over a port the user opened to
|
||||
* read their own code.
|
||||
*
|
||||
* **A file that changed on disk since it was indexed is never sliced.** The
|
||||
* viewer asks for line ranges the *index* recorded; if the file moved on since,
|
||||
* those ranges can point at a different symbol's body, which would be served
|
||||
* under the requested name and look perfectly plausible. So the bytes are
|
||||
* hashed and compared against `files.content_hash`, and on a mismatch the slice
|
||||
* is omitted with `drift: true` — the same call `codegraph_node` makes when it
|
||||
* says "changed on disk after the last index sync".
|
||||
* **A file that changed on disk since it was indexed is never sliced under the
|
||||
* index's numbering.** The viewer asks for line ranges the *index* recorded; if
|
||||
* the file moved on since, those ranges can point at a different symbol's body,
|
||||
* which would be served under the requested name and look perfectly plausible.
|
||||
* So the bytes are hashed and compared against `files.content_hash`, and on a
|
||||
* mismatch the slice is omitted with `drift: true` — the same call
|
||||
* `codegraph_node` makes when it says "changed on disk after the last index
|
||||
* sync".
|
||||
*
|
||||
* A caller that has ALREADY decided the index's numbering is off — a viewer
|
||||
* about to draw a drift banner — asks with `ondrift=current` and gets the
|
||||
* file's CURRENT lines instead of nothing. That is the other half of
|
||||
* `codegraph_node`'s behaviour (issue #1474): a drifted file is served whole
|
||||
* and current rather than omitted, because current bytes are correct by
|
||||
* construction. `showing` says which of the two came back, on every response,
|
||||
* so nothing has to infer it from the presence of `lines`.
|
||||
*
|
||||
* Only files that are IN the index are served. That is a tighter boundary than
|
||||
* the MCP tools take, and it costs the viewer nothing (it only ever renders
|
||||
@@ -230,8 +239,19 @@ export function readFileShape(
|
||||
export interface SourceResult {
|
||||
file: string;
|
||||
language: string;
|
||||
/** The file on disk differs from what was indexed — no slice is served. */
|
||||
/** The file on disk differs from what was indexed. */
|
||||
drift: boolean;
|
||||
/**
|
||||
* Which numbering the returned lines belong to.
|
||||
*
|
||||
* `'indexed'` — the file matches the index, so the two are the same thing.
|
||||
* `'current'` — the file drifted and the caller asked for it anyway
|
||||
* (`ondrift=current`): these are the bytes on disk right now, and NOTHING the
|
||||
* graph holds about this file (symbol ranges, call-site lines, ports) lines
|
||||
* up with them.
|
||||
* `'none'` — the file drifted and no slice is served.
|
||||
*/
|
||||
showing: 'indexed' | 'current' | 'none';
|
||||
contentHash: string;
|
||||
indexedAt: number;
|
||||
generated: boolean;
|
||||
@@ -253,6 +273,32 @@ export interface SourceResult {
|
||||
highlight?: HighlightResult;
|
||||
}
|
||||
|
||||
/**
|
||||
* What to do when the file on disk no longer matches the index.
|
||||
*
|
||||
* `omit` (the default) is the safe answer for a caller that has not decided
|
||||
* anything yet. `current` is for one that has: it is about to say, in the
|
||||
* pixels, that these are the file's CURRENT lines and that nothing the graph
|
||||
* holds about them applies.
|
||||
*/
|
||||
export type OnDrift = 'omit' | 'current';
|
||||
|
||||
/** Said once, so the two places that answer with current bytes cannot diverge. */
|
||||
const DRIFT_CURRENT_REASON =
|
||||
'This file changed on disk after the last index sync. These are its current ' +
|
||||
'lines; the indexed line ranges — symbol bodies, call sites, ports — no longer ' +
|
||||
'match them. The next sync picks it up.';
|
||||
|
||||
export function parseOnDrift(query: URLSearchParams): OnDrift {
|
||||
const raw = query.get('ondrift');
|
||||
if (raw === null || raw === '' || raw === 'omit') return 'omit';
|
||||
if (raw === 'current') return 'current';
|
||||
throw badRequest(
|
||||
`Parameter "ondrift" must be "omit" or "current" (got "${raw}").`,
|
||||
'Omit it to leave a drifted file unsliced; "current" serves the bytes on disk instead.'
|
||||
);
|
||||
}
|
||||
|
||||
export async function buildSource(
|
||||
cg: CodeGraph,
|
||||
projectRoot: string,
|
||||
@@ -267,11 +313,13 @@ export async function buildSource(
|
||||
if (to !== 0 && to < from) {
|
||||
throw badRequest(`Parameter "to" (${to}) must not be before "from" (${from}).`);
|
||||
}
|
||||
const onDrift = parseOnDrift(query);
|
||||
|
||||
const base: SourceResult = {
|
||||
file: storedPath.replace(/\\/g, '/'),
|
||||
language: record.language,
|
||||
drift: false,
|
||||
showing: 'indexed',
|
||||
contentHash: record.contentHash,
|
||||
indexedAt: record.indexedAt,
|
||||
generated: record.generated === true,
|
||||
@@ -283,8 +331,14 @@ export async function buildSource(
|
||||
stats = fs.statSync(absolute);
|
||||
} catch {
|
||||
// Indexed but gone. That IS drift, and the strongest kind: nothing on disk
|
||||
// corresponds to the ranges the graph holds.
|
||||
return { ...base, drift: true, reason: 'The file is in the index but no longer on disk.' };
|
||||
// corresponds to the ranges the graph holds — and `ondrift=current` has
|
||||
// nothing to fall back to either.
|
||||
return {
|
||||
...base,
|
||||
drift: true,
|
||||
showing: 'none',
|
||||
reason: 'The file is in the index but no longer on disk.',
|
||||
};
|
||||
}
|
||||
if (stats.size > MAX_SOURCE_BYTES) {
|
||||
throw badRequest(
|
||||
@@ -306,10 +360,12 @@ export async function buildSource(
|
||||
// string). A touch or a checkout that rewrote the same bytes must not count
|
||||
// as drift, which is exactly what hashing content rather than mtime buys.
|
||||
const hash = createHash('sha256').update(content).digest('hex');
|
||||
if (hash !== record.contentHash) {
|
||||
const drift = hash !== record.contentHash;
|
||||
if (drift && onDrift === 'omit') {
|
||||
return {
|
||||
...base,
|
||||
drift: true,
|
||||
showing: 'none',
|
||||
reason:
|
||||
'This file changed on disk after the last index sync, so the indexed line ' +
|
||||
'ranges no longer reliably match. Source is omitted rather than risk showing ' +
|
||||
@@ -322,10 +378,28 @@ export async function buildSource(
|
||||
// surfacing rather than answering with the last line as if that were meant.
|
||||
// `to` past the end is different — "line 30 to the end, whatever that is" is
|
||||
// an ordinary way to ask, so it clamps.
|
||||
//
|
||||
// The exception is a drifted file the caller asked for anyway: it has already
|
||||
// been told the numbering does not hold, and a save that SHORTENED the file
|
||||
// between the length it was given and this read is an ordinary race, not a
|
||||
// bug. Those get an empty slice.
|
||||
if (from > all.length) {
|
||||
throw badRequest(
|
||||
`Parameter "from" (${from}) is past the end of ${base.file}, which has ${all.length} lines.`
|
||||
);
|
||||
if (!drift) {
|
||||
throw badRequest(
|
||||
`Parameter "from" (${from}) is past the end of ${base.file}, which has ${all.length} lines.`
|
||||
);
|
||||
}
|
||||
return {
|
||||
...base,
|
||||
drift: true,
|
||||
showing: 'current',
|
||||
totalLines: all.length,
|
||||
from,
|
||||
to: from - 1,
|
||||
lines: [],
|
||||
truncated: false,
|
||||
reason: DRIFT_CURRENT_REASON,
|
||||
};
|
||||
}
|
||||
const start = from;
|
||||
const requestedEnd = to === 0 ? all.length : Math.min(to, all.length);
|
||||
@@ -334,17 +408,26 @@ export async function buildSource(
|
||||
|
||||
return {
|
||||
...base,
|
||||
drift,
|
||||
// The bytes are always the ones on disk. What changes with drift is what
|
||||
// they can be *used* for: under `current` the caller must not map anything
|
||||
// the index holds onto these numbers.
|
||||
showing: drift ? 'current' : 'indexed',
|
||||
...(drift ? { reason: DRIFT_CURRENT_REASON } : {}),
|
||||
totalLines: all.length,
|
||||
from: start,
|
||||
to: end,
|
||||
lines: slice,
|
||||
truncated: end < requestedEnd,
|
||||
// Keyed on the content hash, so the cache is invalidated by the file
|
||||
// changing rather than by a clock, and two viewers looking at the same
|
||||
// symbol share one tokenisation.
|
||||
// Keyed on the hash of the bytes ACTUALLY BEING SERVED, so the cache is
|
||||
// invalidated by the file changing rather than by a clock, and two viewers
|
||||
// looking at the same symbol share one tokenisation. It must be the disk
|
||||
// hash rather than the record's: on a drifted file those differ, and keying
|
||||
// current lines under the indexed hash would serve the previous edit's
|
||||
// colours over this one's text.
|
||||
highlight: await highlightLines(slice, {
|
||||
language: record.language,
|
||||
cacheKey: `${record.contentHash}:${start}:${end}`,
|
||||
cacheKey: `${hash}:${start}:${end}`,
|
||||
}),
|
||||
};
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user