Self-contained distribution: bundle Node + node:sqlite, drop better-sqlite3/wasm (closes #238) (#282)
* fix(db): eliminate concurrent-read "database is locked"; add node:sqlite backend (#238) WAL + busy_timeout were already enabled, so the issue's suggested fix was a no-op. The real causes, addressed here: - busy_timeout is now set first (before journal_mode) and lowered 120s -> 5s, so open-time pragmas wait out a lock instead of hanging for two minutes. - getCodeGraph no longer opens a second connection to the default project when a tool passes its own projectPath (the in-process lock amplifier). - The wasm fallback (no WAL) gets a bounded read-retry on SQLITE_BUSY. - New: node:sqlite backend, preferred over wasm, so installs whose native better-sqlite3 build fails land on a real-WAL backend instead of no-WAL wasm. - codegraph status / codegraph_status now report the effective journal mode, so a lock report is triageable (wal vs delete). - CLI hard-blocks Node < 20 to actually enforce the engines floor. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(db)!: node:sqlite is the sole backend; drop better-sqlite3 + wasm Now that distribution will bundle a Node 24 runtime, node:sqlite (real SQLite with WAL + FTS5) is always available. Collapse the three-backend adapter to node:sqlite only and remove the machinery the other two needed: - Remove better-sqlite3 (optionalDependency) and node-sqlite3-wasm (dependency). - Remove WasmDatabaseAdapter, the named->positional param translation, the SQLITE_BUSY read-retry, the wasm fallback banner, the backend env override, and the native/node-sqlite/wasm selection chain. - createDatabase now opens node:sqlite directly, with a clear error pointing at the bundled release / Node 22.5+ when the module is absent. - NodeSqliteAdapter.close() is idempotent and pragma() supports { simple }, to match the better-sqlite3 behavior callers relied on. - status (CLI + MCP) reports the single node:sqlite backend; journal-mode diagnostics and the getCodeGraph single-connection fix are retained. - Tests repointed off better-sqlite3 onto node:sqlite. Net -1044 lines. Running from source now requires Node 22.5+. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(dist): self-contained bundle prototype (vendored Node + install channels) Phase 3 of the node:sqlite migration: ship a vendored Node runtime so CodeGraph runs with no system Node and no native build (node:sqlite is built in). - scripts/build-bundle.sh: build a per-platform archive (official Node + dist + prod deps + launcher). Same recipe per platform; pins Node v24.16.0. - install.sh: curl|sh installer (no Node required) — detects os/arch, pulls the archive from Releases, symlinks onto PATH; re-run to upgrade, --uninstall to remove. The VPS/SSH path. - scripts/npm-shim.js: thin launcher for the npm channel — resolves the per-platform optionalDependency bundle and execs it, so `npm i -g` keeps working and the real work runs on the bundled Node regardless of the user's. - BUNDLING.md: distribution design + release-pipeline TODO (CI matrix, platform packages, code signing, brew, retiring the Node-version gate). Validated end-to-end: darwin-arm64 and linux-x64 bundles both run init + index + status (Backend: node:sqlite, Journal: wal) + FTS query with NO system Node — linux-x64 verified in a clean ubuntu:24.04 amd64 container. Release archives are gitignored; CI will produce and upload them. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(dist): add Windows PowerShell installer (install.ps1) The `irm … | iex` one-liner for Windows, mirroring install.sh: detect arch, pull the matching bundle from Releases, extract to %LOCALAPPDATA%\codegraph, add it to user PATH. Re-run to upgrade. (Windows bundle production in build-bundle.sh is still TODO.) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(dist): release workflow + npm packaging; README/CHANGELOG for bundled distro - .github/workflows/release.yml: manually-triggered (workflow_dispatch) release matrix. Builds a self-contained bundle per platform on its own runner (darwin-arm64/x64, linux-x64/arm64), publishes a GitHub Release with all archives, and publishes the npm thin-installer (shim + per-platform packages). Windows targets are TODO (build-bundle.sh is unix-only). - scripts/pack-npm.sh: assemble the npm packages from built bundles — per-platform packages tagged os/cpu + the main shim package with them as optionalDependencies (esbuild pattern). Proven locally: npm-install the tarballs, run via the shim, resolves the bundle and runs on the bundled Node 24 (node:sqlite / WAL). - README: install section now leads with the no-Node one-liners (curl|sh, irm|iex) then npm/npx; "bundled · none required" badge. - CHANGELOG: standout headline for the self-contained release, plus Added/Changed/ Removed for the install channels, node:sqlite backend, and dropped deps. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(dist): Windows bundles + single-trigger release workflow - build-bundle.sh: add win32-x64 / win32-arm64 targets — download Node's Windows zip, bundle node.exe + a .cmd launcher, output a .zip. Verified structurally (PE32+ node.exe, CRLF .cmd, portable node_modules). Since there are no native addons, any target builds on any OS, so the whole matrix builds on one runner. - pack-npm.sh: handle .zip bundles and win32 packages (os: win32, node.exe). - release.yml: simplified to your spec — manual trigger reads the version from package.json, builds all platform bundles, creates the GitHub Release with notes pulled from CHANGELOG.md, and publishes the npm shim + platform packages. - BUNDLING.md: Windows + build-anywhere notes; release pipeline documented. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Colby Mchenry
co-authored by
Claude Opus 4.7
Claude Opus 4.7
parent
b8aec39abd
commit
ac52fd76c0
+24
-8
@@ -25,7 +25,7 @@ import { getCodeGraphDir, isInitialized } from '../directory';
|
||||
import { createShimmerProgress } from '../ui/shimmer-progress';
|
||||
import { getGlyphs } from '../ui/glyphs';
|
||||
|
||||
import { buildNode25BlockBanner } from './node-version-check';
|
||||
import { buildNode25BlockBanner, buildNodeTooOldBanner, MIN_NODE_MAJOR } from './node-version-check';
|
||||
|
||||
// Lazy-load heavy modules (CodeGraph, runInstaller) to keep CLI startup fast.
|
||||
async function loadCodeGraph(): Promise<typeof import('../index')> {
|
||||
@@ -63,6 +63,16 @@ if (nodeMajor >= 25) {
|
||||
}
|
||||
// Override active — banner shown for visibility, continuing.
|
||||
}
|
||||
// Enforce the supported Node floor. `engines` in package.json only *warns* on
|
||||
// install (unless engine-strict), so hard-block here to actually keep users off
|
||||
// unsupported versions. Mirrors the 25+ block above. See package.json `engines`.
|
||||
if (nodeMajor < MIN_NODE_MAJOR) {
|
||||
process.stderr.write(buildNodeTooOldBanner(nodeVersion) + '\n');
|
||||
if (!process.env.CODEGRAPH_ALLOW_UNSAFE_NODE) {
|
||||
process.exit(1);
|
||||
}
|
||||
// Override active — banner shown for visibility, continuing.
|
||||
}
|
||||
|
||||
// Check if running with no arguments - run installer
|
||||
if (process.argv.length === 2) {
|
||||
@@ -689,6 +699,7 @@ program
|
||||
const stats = cg.getStats();
|
||||
const changes = cg.getChangedFiles();
|
||||
const backend = cg.getBackend();
|
||||
const journalMode = cg.getJournalMode();
|
||||
|
||||
// JSON output mode
|
||||
if (options.json) {
|
||||
@@ -700,6 +711,7 @@ program
|
||||
edgeCount: stats.edgeCount,
|
||||
dbSizeBytes: stats.dbSizeBytes,
|
||||
backend,
|
||||
journalMode,
|
||||
nodesByKind: stats.nodesByKind,
|
||||
languages: Object.entries(stats.filesByLanguage).filter(([, count]) => count > 0).map(([lang]) => lang),
|
||||
pendingChanges: {
|
||||
@@ -724,14 +736,18 @@ program
|
||||
console.log(` Nodes: ${formatNumber(stats.nodeCount)}`);
|
||||
console.log(` Edges: ${formatNumber(stats.edgeCount)}`);
|
||||
console.log(` DB Size: ${(stats.dbSizeBytes / 1024 / 1024).toFixed(2)} MB`);
|
||||
// Surface the active SQLite backend so users can spot the silent
|
||||
// WASM fallback (5-10x slower). better-sqlite3 is in
|
||||
// `optionalDependencies`, so `npm install` succeeds without it
|
||||
// when the native build fails.
|
||||
const backendLabel = backend === 'native'
|
||||
? chalk.green('native')
|
||||
: chalk.yellow(`wasm ${getGlyphs().dash} slower fallback; run \`npm rebuild better-sqlite3\``);
|
||||
// Surface the active SQLite backend (node:sqlite — Node's built-in real
|
||||
// SQLite, full WAL + FTS5, no native build).
|
||||
const backendLabel = chalk.green(`node:sqlite ${getGlyphs().dash} built-in (full WAL)`);
|
||||
console.log(` Backend: ${backendLabel}`);
|
||||
// Effective journal mode: 'wal' means concurrent reads never block on a
|
||||
// writer; anything else means they can ("database is locked"). node:sqlite
|
||||
// supports WAL everywhere, so a non-wal mode means the filesystem can't
|
||||
// (network mounts, WSL2 /mnt). See issue #238.
|
||||
const journalLabel = journalMode === 'wal'
|
||||
? chalk.green('wal')
|
||||
: chalk.yellow(`${journalMode || 'unknown'} ${getGlyphs().dash} WAL inactive; reads can block on writes`);
|
||||
console.log(` Journal: ${journalLabel}`);
|
||||
console.log();
|
||||
|
||||
// Node breakdown
|
||||
|
||||
@@ -37,3 +37,40 @@ export function buildNode25BlockBanner(nodeVersion: string): string {
|
||||
sep,
|
||||
].join('\n');
|
||||
}
|
||||
|
||||
/**
|
||||
* Lowest supported Node.js major version. Matches the `engines` floor in
|
||||
* package.json. Below this, CodeGraph relies on language features / native APIs
|
||||
* that aren't present, and the combination is untested. `engines` alone only
|
||||
* *warns* on install (unless the user set `engine-strict`), so the CLI bootstrap
|
||||
* also hard-blocks here to actually enforce the floor.
|
||||
*/
|
||||
export const MIN_NODE_MAJOR = 20;
|
||||
|
||||
/**
|
||||
* Build the bordered banner shown when CodeGraph detects a Node.js major below
|
||||
* {@link MIN_NODE_MAJOR}. Pinned via unit test so the recovery commands and the
|
||||
* override env var can't be silently stripped by future edits.
|
||||
*
|
||||
* Uses ASCII glyphs to stay readable on Windows OEM-codepage consoles
|
||||
* (see ../ui/glyphs.ts for the rationale).
|
||||
*/
|
||||
export function buildNodeTooOldBanner(nodeVersion: string): string {
|
||||
const sep = '-'.repeat(72);
|
||||
return [
|
||||
sep,
|
||||
`[CodeGraph] Unsupported Node.js version: ${nodeVersion}`,
|
||||
sep,
|
||||
`CodeGraph requires Node.js ${MIN_NODE_MAJOR} or newer. Older versions lack`,
|
||||
'language features and native APIs CodeGraph depends on, and are not',
|
||||
'tested or supported.',
|
||||
'',
|
||||
'Fix: install Node.js 22 LTS:',
|
||||
' nvm install 22 && nvm use 22 # nvm',
|
||||
' brew install node@22 && brew link --overwrite --force node@22 # Homebrew',
|
||||
'',
|
||||
'To override (NOT recommended - unsupported):',
|
||||
' CODEGRAPH_ALLOW_UNSAFE_NODE=1 codegraph ...',
|
||||
sep,
|
||||
].join('\n');
|
||||
}
|
||||
|
||||
+46
-23
@@ -10,7 +10,31 @@ import * as path from 'path';
|
||||
import { SchemaVersion } from '../types';
|
||||
import { runMigrations, getCurrentVersion, CURRENT_SCHEMA_VERSION } from './migrations';
|
||||
|
||||
export { SqliteDatabase, SqliteBackend, WASM_FALLBACK_FIX_RECIPE } from './sqlite-adapter';
|
||||
export { SqliteDatabase, SqliteBackend } from './sqlite-adapter';
|
||||
|
||||
/**
|
||||
* Apply connection-level PRAGMAs. Shared by `initialize` and `open` so the two
|
||||
* paths can't drift.
|
||||
*
|
||||
* `busy_timeout` is set FIRST, before any pragma that might touch the database
|
||||
* file (notably `journal_mode`). If another process holds a write lock at open
|
||||
* time, the later pragmas — and the connection's first query — then wait out
|
||||
* the lock instead of throwing "database is locked" immediately. See issue #238.
|
||||
*
|
||||
* The 5s window (was 120s) rides out a normal incremental sync; the old
|
||||
* 2-minute wait presented as a frozen, hung agent. With WAL, reads never block
|
||||
* on a writer, so this timeout only governs cross-process write contention
|
||||
* (e.g. the git-hook `codegraph sync` running while the MCP server writes).
|
||||
*/
|
||||
function configureConnection(db: SqliteDatabase): void {
|
||||
db.pragma('busy_timeout = 5000'); // MUST be first — see above
|
||||
db.pragma('foreign_keys = ON');
|
||||
db.pragma('journal_mode = WAL'); // node:sqlite supports WAL on every platform
|
||||
db.pragma('synchronous = NORMAL'); // safe with WAL mode
|
||||
db.pragma('cache_size = -64000'); // 64 MB page cache
|
||||
db.pragma('temp_store = MEMORY'); // temp tables in memory
|
||||
db.pragma('mmap_size = 268435456'); // 256 MB memory-mapped I/O
|
||||
}
|
||||
|
||||
/**
|
||||
* Database connection wrapper with lifecycle management
|
||||
@@ -39,17 +63,7 @@ export class DatabaseConnection {
|
||||
// Create and configure database
|
||||
const { db, backend } = createDatabase(dbPath);
|
||||
|
||||
// Enable foreign keys and WAL mode for better performance
|
||||
db.pragma('foreign_keys = ON');
|
||||
db.pragma('journal_mode = WAL');
|
||||
// Wait up to 2 minutes if database is locked by another process
|
||||
// (indexing operations can hold locks for extended periods)
|
||||
db.pragma('busy_timeout = 120000');
|
||||
// Performance tuning
|
||||
db.pragma('synchronous = NORMAL'); // Safe with WAL mode
|
||||
db.pragma('cache_size = -64000'); // 64 MB page cache
|
||||
db.pragma('temp_store = MEMORY'); // Temp tables in memory
|
||||
db.pragma('mmap_size = 268435456'); // 256 MB memory-mapped I/O
|
||||
configureConnection(db);
|
||||
|
||||
// Run schema initialization
|
||||
const schemaPath = path.join(__dirname, 'schema.sql');
|
||||
@@ -77,17 +91,7 @@ export class DatabaseConnection {
|
||||
|
||||
const { db, backend } = createDatabase(dbPath);
|
||||
|
||||
// Enable foreign keys and WAL mode
|
||||
db.pragma('foreign_keys = ON');
|
||||
db.pragma('journal_mode = WAL');
|
||||
// Wait up to 2 minutes if database is locked by another process
|
||||
// (indexing operations can hold locks for extended periods)
|
||||
db.pragma('busy_timeout = 120000');
|
||||
// Performance tuning
|
||||
db.pragma('synchronous = NORMAL');
|
||||
db.pragma('cache_size = -64000');
|
||||
db.pragma('temp_store = MEMORY');
|
||||
db.pragma('mmap_size = 268435456');
|
||||
configureConnection(db);
|
||||
|
||||
// Check and run migrations if needed
|
||||
const conn = new DatabaseConnection(db, dbPath, backend);
|
||||
@@ -123,6 +127,25 @@ export class DatabaseConnection {
|
||||
return this.dbPath;
|
||||
}
|
||||
|
||||
/**
|
||||
* The journal mode actually in effect (e.g. 'wal', 'delete').
|
||||
*
|
||||
* SQLite silently keeps the prior mode if WAL can't be enabled — e.g. on
|
||||
* filesystems without shared-memory support (some network/virtualized mounts,
|
||||
* WSL2 /mnt), and always on the wasm backend. So the effective mode can differ
|
||||
* from what `configureConnection` requested. Surfaced in `codegraph status` so
|
||||
* a "database is locked" report is triageable: 'wal' ⇒ readers never block on a
|
||||
* writer; anything else ⇒ they can. See issue #238.
|
||||
*/
|
||||
getJournalMode(): string {
|
||||
const raw = this.db.pragma('journal_mode');
|
||||
const row = Array.isArray(raw) ? raw[0] : raw;
|
||||
const mode = row && typeof row === 'object'
|
||||
? (row as Record<string, unknown>).journal_mode
|
||||
: row;
|
||||
return String(mode ?? '').toLowerCase();
|
||||
}
|
||||
|
||||
/**
|
||||
* Get current schema version
|
||||
*/
|
||||
|
||||
+56
-184
@@ -1,8 +1,13 @@
|
||||
/**
|
||||
* SQLite Adapter
|
||||
*
|
||||
* Provides a unified interface over better-sqlite3 (native) and
|
||||
* node-sqlite3-wasm (WASM fallback) for universal cross-platform support.
|
||||
* Thin wrapper over Node's built-in `node:sqlite` (`DatabaseSync`), exposed
|
||||
* through a small better-sqlite3-shaped interface so the rest of the codebase
|
||||
* is storage-agnostic.
|
||||
*
|
||||
* CodeGraph ships with a bundled Node runtime, so `node:sqlite` (real SQLite,
|
||||
* with WAL + FTS5) is always available — there is no native build step and no
|
||||
* wasm fallback. When run from source instead, it requires Node >= 22.5.
|
||||
*/
|
||||
|
||||
export interface SqliteStatement {
|
||||
@@ -14,123 +19,34 @@ export interface SqliteStatement {
|
||||
export interface SqliteDatabase {
|
||||
prepare(sql: string): SqliteStatement;
|
||||
exec(sql: string): void;
|
||||
pragma(str: string): any;
|
||||
pragma(str: string, options?: { simple?: boolean }): any;
|
||||
transaction<T>(fn: (...args: any[]) => T): (...args: any[]) => T;
|
||||
close(): void;
|
||||
readonly open: boolean;
|
||||
}
|
||||
|
||||
export type SqliteBackend = 'native' | 'wasm';
|
||||
|
||||
/**
|
||||
* One-line summary of the recovery steps shown when WASM fallback is
|
||||
* active. Single source of truth so the recipe can't drift between the
|
||||
* stderr banner and the MCP status formatter.
|
||||
* The active SQLite backend. Only one now (`node:sqlite`); kept as a named type
|
||||
* so `codegraph status` and the per-instance reporting have a stable shape.
|
||||
*/
|
||||
export const WASM_FALLBACK_FIX_RECIPE =
|
||||
'`xcode-select --install` (macOS) or `apt install build-essential` (Debian/Ubuntu), ' +
|
||||
'then `npm rebuild better-sqlite3`, or `npm install better-sqlite3 --save` to force-include it.';
|
||||
export type SqliteBackend = 'node-sqlite';
|
||||
|
||||
/**
|
||||
* Multi-line banner shown to stderr when `createDatabase` falls back to
|
||||
* WASM. Replaces a one-line `console.warn` that MCP transports (which
|
||||
* take stdout for the protocol) typically swallow, leaving users on a
|
||||
* 5-10x slower backend with no signal.
|
||||
* Wraps Node's built-in `node:sqlite` (`DatabaseSync`) to match the
|
||||
* better-sqlite3 interface the rest of the code expects.
|
||||
*
|
||||
* Exported for unit testing — pinning the recipe content prevents
|
||||
* future edits from silently stripping the recovery commands.
|
||||
* node:sqlite is real SQLite compiled into Node, so it supports WAL, FTS5,
|
||||
* mmap, and `@named` params natively — the only shims needed are the
|
||||
* better-sqlite3 conveniences node:sqlite omits: a `.pragma()` helper, a
|
||||
* `.transaction()` helper, and `open` (node:sqlite exposes `isOpen`).
|
||||
*/
|
||||
export function buildWasmFallbackBanner(nativeError?: string): string {
|
||||
const sep = '─'.repeat(72);
|
||||
const lines = [
|
||||
sep,
|
||||
'[CodeGraph] WASM SQLite fallback active (better-sqlite3 unavailable)',
|
||||
sep,
|
||||
'Indexing and sync will be 5-10x slower than the native backend.',
|
||||
'',
|
||||
'Fix on macOS:',
|
||||
' xcode-select --install # install C build tools',
|
||||
' npm rebuild better-sqlite3 # rebuild native binding for current Node',
|
||||
'',
|
||||
'Fix on Linux:',
|
||||
' sudo apt install build-essential python3 make # Debian/Ubuntu',
|
||||
' # or: sudo yum groupinstall "Development Tools" # RHEL/Fedora',
|
||||
' npm rebuild better-sqlite3',
|
||||
'',
|
||||
'Or force-include as a hard dependency on any platform:',
|
||||
' npm install better-sqlite3 --save',
|
||||
'',
|
||||
'Verify after fix: `codegraph status` should show `Backend: native`.',
|
||||
];
|
||||
if (nativeError) {
|
||||
lines.push('', `Native load error: ${nativeError}`);
|
||||
}
|
||||
lines.push(sep);
|
||||
return lines.join('\n');
|
||||
}
|
||||
|
||||
/**
|
||||
* Translate @named parameters (better-sqlite3 style) to positional ? params
|
||||
* for node-sqlite3-wasm, which only supports positional binding.
|
||||
*
|
||||
* Returns the rewritten SQL and an ordered list of parameter names.
|
||||
* If no named params are found, returns null for paramOrder (positional mode).
|
||||
*/
|
||||
function translateNamedParams(sql: string): { sql: string; paramOrder: string[] | null } {
|
||||
const paramOrder: string[] = [];
|
||||
const rewritten = sql.replace(/@(\w+)/g, (_match, name: string) => {
|
||||
paramOrder.push(name);
|
||||
return '?';
|
||||
});
|
||||
if (paramOrder.length === 0) {
|
||||
return { sql, paramOrder: null };
|
||||
}
|
||||
return { sql: rewritten, paramOrder };
|
||||
}
|
||||
|
||||
/**
|
||||
* Convert better-sqlite3-style params to a positional array for node-sqlite3-wasm.
|
||||
*
|
||||
* Handles three calling conventions:
|
||||
* - Named object: run({ id: '1', name: 'a' }) → positional array via paramOrder
|
||||
* - Positional args: run('a', 'b') → ['a', 'b']
|
||||
* - No args: run() → undefined
|
||||
*/
|
||||
function resolveParams(params: any[], paramOrder: string[] | null): any {
|
||||
if (params.length === 0) return undefined;
|
||||
|
||||
// If paramOrder exists and first arg is a plain object, do named→positional translation
|
||||
if (paramOrder && params.length === 1 && params[0] !== null && typeof params[0] === 'object' && !Array.isArray(params[0]) && !(params[0] instanceof Buffer) && !(params[0] instanceof Uint8Array)) {
|
||||
const obj = params[0];
|
||||
return paramOrder.map(name => obj[name]);
|
||||
}
|
||||
|
||||
// Positional: single value or already an array
|
||||
if (params.length === 1) return params[0];
|
||||
return params;
|
||||
}
|
||||
|
||||
/**
|
||||
* Wraps node-sqlite3-wasm to match the better-sqlite3 interface.
|
||||
*
|
||||
* Key differences handled:
|
||||
* - better-sqlite3 uses @named params; node-sqlite3-wasm uses positional ? only
|
||||
* - better-sqlite3 uses variadic args: stmt.run(a, b, c)
|
||||
* - node-sqlite3-wasm uses a single array/object: stmt.run([a, b, c])
|
||||
* - node-sqlite3-wasm has `isOpen` instead of `open`
|
||||
* - node-sqlite3-wasm doesn't have a `pragma()` method
|
||||
* - node-sqlite3-wasm doesn't have a `transaction()` method
|
||||
*/
|
||||
class WasmDatabaseAdapter implements SqliteDatabase {
|
||||
class NodeSqliteAdapter implements SqliteDatabase {
|
||||
private _db: any;
|
||||
// Track raw WASM statements so we can finalize them on close.
|
||||
// node-sqlite3-wasm won't release its file lock if statements are left open.
|
||||
private _openStmts = new Set<any>();
|
||||
|
||||
constructor(dbPath: string) {
|
||||
// eslint-disable-next-line @typescript-eslint/no-require-imports
|
||||
const { Database } = require('node-sqlite3-wasm');
|
||||
this._db = new Database(dbPath);
|
||||
const { DatabaseSync } = require('node:sqlite');
|
||||
this._db = new DatabaseSync(dbPath);
|
||||
}
|
||||
|
||||
get open(): boolean {
|
||||
@@ -138,25 +54,23 @@ class WasmDatabaseAdapter implements SqliteDatabase {
|
||||
}
|
||||
|
||||
prepare(sql: string): SqliteStatement {
|
||||
const { sql: rewrittenSql, paramOrder } = translateNamedParams(sql);
|
||||
const stmt = this._db.prepare(rewrittenSql);
|
||||
this._openStmts.add(stmt);
|
||||
// node:sqlite matches better-sqlite3's calling convention (variadic
|
||||
// positional args, or a single object for @named params), so params forward
|
||||
// through unchanged.
|
||||
const stmt = this._db.prepare(sql);
|
||||
return {
|
||||
run(...params: any[]) {
|
||||
const resolved = resolveParams(params, paramOrder);
|
||||
const result = resolved !== undefined ? stmt.run(resolved) : stmt.run();
|
||||
const r = stmt.run(...params);
|
||||
return {
|
||||
changes: result?.changes ?? 0,
|
||||
lastInsertRowid: result?.lastInsertRowid ?? 0,
|
||||
changes: Number(r?.changes ?? 0),
|
||||
lastInsertRowid: r?.lastInsertRowid ?? 0,
|
||||
};
|
||||
},
|
||||
get(...params: any[]) {
|
||||
const resolved = resolveParams(params, paramOrder);
|
||||
return resolved !== undefined ? stmt.get(resolved) : stmt.get();
|
||||
return stmt.get(...params);
|
||||
},
|
||||
all(...params: any[]) {
|
||||
const resolved = resolveParams(params, paramOrder);
|
||||
return resolved !== undefined ? stmt.all(resolved) : stmt.all();
|
||||
return stmt.all(...params);
|
||||
},
|
||||
};
|
||||
}
|
||||
@@ -165,41 +79,21 @@ class WasmDatabaseAdapter implements SqliteDatabase {
|
||||
this._db.exec(sql);
|
||||
}
|
||||
|
||||
pragma(str: string): any {
|
||||
pragma(str: string, options?: { simple?: boolean }): any {
|
||||
const trimmed = str.trim();
|
||||
|
||||
// Write pragma: "key = value"
|
||||
// Write pragma ("key = value"): node:sqlite is real SQLite, so every pragma
|
||||
// (WAL, mmap, synchronous, …) applies as-is.
|
||||
if (trimmed.includes('=')) {
|
||||
const eqIdx = trimmed.indexOf('=');
|
||||
const key = trimmed.substring(0, eqIdx).trim();
|
||||
const value = trimmed.substring(eqIdx + 1).trim();
|
||||
|
||||
// WAL is not supported in WASM SQLite — use DELETE journal mode
|
||||
if (key === 'journal_mode' && value.toUpperCase() === 'WAL') {
|
||||
this._db.exec('PRAGMA journal_mode = DELETE');
|
||||
return;
|
||||
}
|
||||
|
||||
// mmap is not available in WASM — silently skip
|
||||
if (key === 'mmap_size') {
|
||||
return;
|
||||
}
|
||||
|
||||
// synchronous = NORMAL is unsafe without WAL — use FULL
|
||||
if (key === 'synchronous' && value.toUpperCase() === 'NORMAL') {
|
||||
this._db.exec('PRAGMA synchronous = FULL');
|
||||
return;
|
||||
}
|
||||
|
||||
this._db.exec(`PRAGMA ${key} = ${value}`);
|
||||
this._db.exec(`PRAGMA ${trimmed}`);
|
||||
return;
|
||||
}
|
||||
|
||||
// Read pragma: "key" — return the value
|
||||
const stmt = this._db.prepare(`PRAGMA ${trimmed}`);
|
||||
const result = stmt.get();
|
||||
stmt.finalize();
|
||||
return result;
|
||||
// Read pragma. Default: the row object (e.g. { journal_mode: 'wal' }).
|
||||
// `{ simple: true }` returns just the single column value, like better-sqlite3.
|
||||
const row = this._db.prepare(`PRAGMA ${trimmed}`).get();
|
||||
if (options?.simple) {
|
||||
return row && typeof row === 'object' ? Object.values(row)[0] : row;
|
||||
}
|
||||
return row;
|
||||
}
|
||||
|
||||
transaction<T>(fn: (...args: any[]) => T): (...args: any[]) => T {
|
||||
@@ -217,51 +111,29 @@ class WasmDatabaseAdapter implements SqliteDatabase {
|
||||
}
|
||||
|
||||
close(): void {
|
||||
// Finalize all tracked statements before closing.
|
||||
// node-sqlite3-wasm won't release its directory-based file lock
|
||||
// if any prepared statements remain open.
|
||||
for (const stmt of this._openStmts) {
|
||||
try { stmt.finalize(); } catch { /* already finalized */ }
|
||||
}
|
||||
this._openStmts.clear();
|
||||
this._db.close();
|
||||
// node:sqlite's DatabaseSync.close() throws if already closed; make it
|
||||
// idempotent to match better-sqlite3 (callers may close more than once).
|
||||
if (this._db.isOpen) this._db.close();
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Create a database connection. Tries native better-sqlite3 first,
|
||||
* falls back to node-sqlite3-wasm. Returns the active backend
|
||||
* alongside the db so each `DatabaseConnection` can report its own
|
||||
* backend per-instance — MCP can open multiple project DBs in one
|
||||
* process (`tools.ts` getCodeGraph cache), so a process-global would
|
||||
* race / overwrite.
|
||||
* Create a database connection backed by `node:sqlite`.
|
||||
*
|
||||
* Returns the active backend alongside the db so each `DatabaseConnection` can
|
||||
* report it per-instance — MCP can open multiple project DBs in one process, so
|
||||
* a process-global would race.
|
||||
*/
|
||||
export function createDatabase(dbPath: string): { db: SqliteDatabase; backend: SqliteBackend } {
|
||||
let nativeError: string | undefined;
|
||||
let wasmError: string | undefined;
|
||||
|
||||
// Try native better-sqlite3 first
|
||||
try {
|
||||
// eslint-disable-next-line @typescript-eslint/no-require-imports
|
||||
const Database = require('better-sqlite3');
|
||||
const db = new Database(dbPath);
|
||||
return { db: db as SqliteDatabase, backend: 'native' };
|
||||
return { db: new NodeSqliteAdapter(dbPath), backend: 'node-sqlite' };
|
||||
} catch (error) {
|
||||
nativeError = error instanceof Error ? error.message : String(error);
|
||||
const msg = error instanceof Error ? error.message : String(error);
|
||||
throw new Error(
|
||||
'Failed to open SQLite via the built-in node:sqlite module.\n' +
|
||||
'CodeGraph requires node:sqlite (Node.js 22.5+). Install the self-contained\n' +
|
||||
'CodeGraph release (it bundles a compatible Node), or run on Node 22.5+.\n' +
|
||||
`Underlying error: ${msg}`
|
||||
);
|
||||
}
|
||||
|
||||
// Fall back to WASM
|
||||
try {
|
||||
const db = new WasmDatabaseAdapter(dbPath);
|
||||
console.warn(buildWasmFallbackBanner(nativeError));
|
||||
return { db, backend: 'wasm' };
|
||||
} catch (error) {
|
||||
wasmError = error instanceof Error ? error.message : String(error);
|
||||
}
|
||||
|
||||
throw new Error(
|
||||
`Failed to load any SQLite backend.\n` +
|
||||
` Native (better-sqlite3): ${nativeError}\n` +
|
||||
` WASM (node-sqlite3-wasm): ${wasmError}`
|
||||
);
|
||||
}
|
||||
|
||||
+13
-4
@@ -613,15 +613,24 @@ export class CodeGraph {
|
||||
}
|
||||
|
||||
/**
|
||||
* Active SQLite backend for this project's connection. `wasm` means
|
||||
* the native better-sqlite3 install failed and the WASM fallback is
|
||||
* serving requests at 5-10x the latency. Surfaced via `codegraph
|
||||
* status` and the `codegraph_status` MCP tool.
|
||||
* Active SQLite backend for this project's connection (`node-sqlite` — Node's
|
||||
* built-in real-SQLite module). Surfaced via `codegraph status` and the
|
||||
* `codegraph_status` MCP tool alongside the effective journal mode.
|
||||
*/
|
||||
getBackend(): import('./db').SqliteBackend {
|
||||
return this.db.getBackend();
|
||||
}
|
||||
|
||||
/**
|
||||
* The journal mode actually in effect ('wal', 'delete', …). 'wal' means
|
||||
* readers never block on a concurrent writer; anything else means they can,
|
||||
* which is the precondition for the "database is locked" failures in issue
|
||||
* #238. Surfaced via `codegraph status` and the `codegraph_status` MCP tool.
|
||||
*/
|
||||
getJournalMode(): string {
|
||||
return this.db.getJournalMode();
|
||||
}
|
||||
|
||||
// ===========================================================================
|
||||
// Node Operations
|
||||
// ===========================================================================
|
||||
|
||||
+24
-9
@@ -11,7 +11,6 @@ import { writeFileSync, readFileSync, existsSync } from 'fs';
|
||||
import { clamp, validatePathWithinRoot } from '../utils';
|
||||
import { tmpdir } from 'os';
|
||||
import { join } from 'path';
|
||||
import { WASM_FALLBACK_FIX_RECIPE } from '../db';
|
||||
|
||||
/** Maximum output length to prevent context bloat (characters) */
|
||||
const MAX_OUTPUT_LENGTH = 15000;
|
||||
@@ -542,6 +541,17 @@ export class ToolHandler {
|
||||
throw new Error(`CodeGraph not initialized in ${projectPath}. Run 'codegraph init' in that project first.`);
|
||||
}
|
||||
|
||||
// If the path resolves to the default project, reuse the already-open
|
||||
// default instance rather than opening a SECOND connection to the same DB.
|
||||
// A duplicate connection serializes reads against the watcher's auto-sync
|
||||
// writes; on the wasm backend (no WAL) that surfaces as intermittent
|
||||
// "database is locked" on concurrent tool calls. See issue #238. Deliberately
|
||||
// not cached under projectPath — the server owns and closes the default
|
||||
// instance, so routing it through projectCache.closeAll() would double-close it.
|
||||
if (this.cg && this.cg.getProjectRoot() === resolvedRoot) {
|
||||
return this.cg;
|
||||
}
|
||||
|
||||
// Check if we already have this resolved root cached (different path, same project)
|
||||
if (this.projectCache.has(resolvedRoot)) {
|
||||
const cg = this.projectCache.get(resolvedRoot)!;
|
||||
@@ -1321,16 +1331,21 @@ export class ToolHandler {
|
||||
`**Database size:** ${(stats.dbSizeBytes / 1024 / 1024).toFixed(2)} MB`,
|
||||
];
|
||||
|
||||
// Surface the active SQLite backend. Without this, users on the
|
||||
// silent WASM fallback (better-sqlite3 install failed) see "slow"
|
||||
// indexing and DB-lock errors with no signal of why.
|
||||
const backend = cg.getBackend();
|
||||
if (backend === 'native') {
|
||||
lines.push(`**Backend:** native (better-sqlite3)`);
|
||||
// Surface the active SQLite backend (node:sqlite, Node's built-in real
|
||||
// SQLite — full WAL + FTS5, no native build).
|
||||
lines.push(`**Backend:** node:sqlite (Node built-in) — full WAL + FTS5`);
|
||||
|
||||
// Effective journal mode. 'wal' ⇒ concurrent reads never block on a writer;
|
||||
// anything else ⇒ they can ("database is locked"). node:sqlite supports WAL
|
||||
// everywhere, so a non-wal mode means the filesystem can't (network/
|
||||
// virtualized mounts, WSL2 /mnt). See issue #238.
|
||||
const journalMode = cg.getJournalMode();
|
||||
if (journalMode === 'wal') {
|
||||
lines.push(`**Journal mode:** wal (concurrent reads safe)`);
|
||||
} else {
|
||||
lines.push(
|
||||
`**Backend:** ⚠ wasm (better-sqlite3 unavailable) — ` +
|
||||
`5-10x slower than native. Fix: ${WASM_FALLBACK_FIX_RECIPE}`
|
||||
`**Journal mode:** ⚠ ${journalMode || 'unknown'} — WAL not active, so reads ` +
|
||||
`can block on a concurrent write (WAL appears unsupported on this filesystem)`
|
||||
);
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user