Self-contained distribution: bundle Node + node:sqlite, drop better-sqlite3/wasm (closes #238) (#282)
* fix(db): eliminate concurrent-read "database is locked"; add node:sqlite backend (#238) WAL + busy_timeout were already enabled, so the issue's suggested fix was a no-op. The real causes, addressed here: - busy_timeout is now set first (before journal_mode) and lowered 120s -> 5s, so open-time pragmas wait out a lock instead of hanging for two minutes. - getCodeGraph no longer opens a second connection to the default project when a tool passes its own projectPath (the in-process lock amplifier). - The wasm fallback (no WAL) gets a bounded read-retry on SQLITE_BUSY. - New: node:sqlite backend, preferred over wasm, so installs whose native better-sqlite3 build fails land on a real-WAL backend instead of no-WAL wasm. - codegraph status / codegraph_status now report the effective journal mode, so a lock report is triageable (wal vs delete). - CLI hard-blocks Node < 20 to actually enforce the engines floor. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(db)!: node:sqlite is the sole backend; drop better-sqlite3 + wasm Now that distribution will bundle a Node 24 runtime, node:sqlite (real SQLite with WAL + FTS5) is always available. Collapse the three-backend adapter to node:sqlite only and remove the machinery the other two needed: - Remove better-sqlite3 (optionalDependency) and node-sqlite3-wasm (dependency). - Remove WasmDatabaseAdapter, the named->positional param translation, the SQLITE_BUSY read-retry, the wasm fallback banner, the backend env override, and the native/node-sqlite/wasm selection chain. - createDatabase now opens node:sqlite directly, with a clear error pointing at the bundled release / Node 22.5+ when the module is absent. - NodeSqliteAdapter.close() is idempotent and pragma() supports { simple }, to match the better-sqlite3 behavior callers relied on. - status (CLI + MCP) reports the single node:sqlite backend; journal-mode diagnostics and the getCodeGraph single-connection fix are retained. - Tests repointed off better-sqlite3 onto node:sqlite. Net -1044 lines. Running from source now requires Node 22.5+. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(dist): self-contained bundle prototype (vendored Node + install channels) Phase 3 of the node:sqlite migration: ship a vendored Node runtime so CodeGraph runs with no system Node and no native build (node:sqlite is built in). - scripts/build-bundle.sh: build a per-platform archive (official Node + dist + prod deps + launcher). Same recipe per platform; pins Node v24.16.0. - install.sh: curl|sh installer (no Node required) — detects os/arch, pulls the archive from Releases, symlinks onto PATH; re-run to upgrade, --uninstall to remove. The VPS/SSH path. - scripts/npm-shim.js: thin launcher for the npm channel — resolves the per-platform optionalDependency bundle and execs it, so `npm i -g` keeps working and the real work runs on the bundled Node regardless of the user's. - BUNDLING.md: distribution design + release-pipeline TODO (CI matrix, platform packages, code signing, brew, retiring the Node-version gate). Validated end-to-end: darwin-arm64 and linux-x64 bundles both run init + index + status (Backend: node:sqlite, Journal: wal) + FTS query with NO system Node — linux-x64 verified in a clean ubuntu:24.04 amd64 container. Release archives are gitignored; CI will produce and upload them. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(dist): add Windows PowerShell installer (install.ps1) The `irm … | iex` one-liner for Windows, mirroring install.sh: detect arch, pull the matching bundle from Releases, extract to %LOCALAPPDATA%\codegraph, add it to user PATH. Re-run to upgrade. (Windows bundle production in build-bundle.sh is still TODO.) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(dist): release workflow + npm packaging; README/CHANGELOG for bundled distro - .github/workflows/release.yml: manually-triggered (workflow_dispatch) release matrix. Builds a self-contained bundle per platform on its own runner (darwin-arm64/x64, linux-x64/arm64), publishes a GitHub Release with all archives, and publishes the npm thin-installer (shim + per-platform packages). Windows targets are TODO (build-bundle.sh is unix-only). - scripts/pack-npm.sh: assemble the npm packages from built bundles — per-platform packages tagged os/cpu + the main shim package with them as optionalDependencies (esbuild pattern). Proven locally: npm-install the tarballs, run via the shim, resolves the bundle and runs on the bundled Node 24 (node:sqlite / WAL). - README: install section now leads with the no-Node one-liners (curl|sh, irm|iex) then npm/npx; "bundled · none required" badge. - CHANGELOG: standout headline for the self-contained release, plus Added/Changed/ Removed for the install channels, node:sqlite backend, and dropped deps. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(dist): Windows bundles + single-trigger release workflow - build-bundle.sh: add win32-x64 / win32-arm64 targets — download Node's Windows zip, bundle node.exe + a .cmd launcher, output a .zip. Verified structurally (PE32+ node.exe, CRLF .cmd, portable node_modules). Since there are no native addons, any target builds on any OS, so the whole matrix builds on one runner. - pack-npm.sh: handle .zip bundles and win32 packages (os: win32, node.exe). - release.yml: simplified to your spec — manual trigger reads the version from package.json, builds all platform bundles, creates the GitHub Release with notes pulled from CHANGELOG.md, and publishes the npm shim + platform packages. - BUNDLING.md: Windows + build-anywhere notes; release pipeline documented. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Colby Mchenry
co-authored by
Claude Opus 4.7
Claude Opus 4.7
parent
b8aec39abd
commit
ac52fd76c0
Executable
+98
@@ -0,0 +1,98 @@
|
||||
#!/usr/bin/env bash
|
||||
#
|
||||
# Build a self-contained CodeGraph bundle: an official Node runtime + the
|
||||
# compiled app + its production deps, so CodeGraph runs with NO system Node and
|
||||
# NO native build — node:sqlite is built into the bundled Node. One archive per
|
||||
# platform.
|
||||
#
|
||||
# Because dropping better-sqlite3 left zero native addons, the recipe is pure
|
||||
# file-packaging (download the target's Node, copy the app, archive) — so any
|
||||
# platform's bundle can be built on any OS. No cross-compile, no native runners.
|
||||
#
|
||||
# Usage:
|
||||
# scripts/build-bundle.sh <target> [node-version]
|
||||
# target: darwin-arm64 | darwin-x64 | linux-x64 | linux-arm64
|
||||
# | win32-x64 | win32-arm64
|
||||
# node-version: e.g. v24.16.0 (default below; pin for reproducible builds)
|
||||
#
|
||||
# Output:
|
||||
# unix: release/codegraph-<target>.tar.gz (launcher: bin/codegraph)
|
||||
# windows: release/codegraph-<target>.zip (launcher: bin/codegraph.cmd)
|
||||
set -euo pipefail
|
||||
|
||||
TARGET="${1:?usage: build-bundle.sh <target> [node-version]}"
|
||||
NODE_VERSION="${2:-v24.16.0}"
|
||||
|
||||
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
|
||||
OUT="$ROOT/release"
|
||||
WORK="$(mktemp -d)"
|
||||
trap 'rm -rf "$WORK"' EXIT
|
||||
|
||||
ARCH="${TARGET##*-}" # x64 | arm64
|
||||
OSFAM="${TARGET%-*}" # darwin | linux | win32
|
||||
|
||||
echo "[bundle] target=${TARGET} node=${NODE_VERSION}"
|
||||
|
||||
# 1. Download + extract the official Node runtime for the target platform.
|
||||
if [ "$OSFAM" = "win32" ]; then
|
||||
NODE_DIST="node-${NODE_VERSION}-win-${ARCH}"
|
||||
NODE_URL="https://nodejs.org/dist/${NODE_VERSION}/${NODE_DIST}.zip"
|
||||
echo "[bundle] downloading ${NODE_URL}"
|
||||
curl -fsSL "$NODE_URL" -o "$WORK/node.zip"
|
||||
if command -v unzip >/dev/null 2>&1; then
|
||||
unzip -q "$WORK/node.zip" -d "$WORK"
|
||||
else
|
||||
tar -xf "$WORK/node.zip" -C "$WORK" # bsdtar can read zip
|
||||
fi
|
||||
NODE_BIN="$WORK/${NODE_DIST}/node.exe"
|
||||
else
|
||||
NODE_DIST="node-${NODE_VERSION}-${TARGET}"
|
||||
NODE_URL="https://nodejs.org/dist/${NODE_VERSION}/${NODE_DIST}.tar.gz"
|
||||
echo "[bundle] downloading ${NODE_URL}"
|
||||
curl -fsSL "$NODE_URL" -o "$WORK/node.tar.gz"
|
||||
tar -xzf "$WORK/node.tar.gz" -C "$WORK"
|
||||
NODE_BIN="$WORK/${NODE_DIST}/bin/node"
|
||||
fi
|
||||
[ -f "$NODE_BIN" ] || { echo "[bundle] error: node binary not found ($NODE_BIN)" >&2; exit 1; }
|
||||
|
||||
# 2. Build the app (compiled JS + copied wasm/schema assets).
|
||||
echo "[bundle] building app"
|
||||
( cd "$ROOT" && npm run build >/dev/null )
|
||||
|
||||
# 3. Stage: app + production-only deps (pure JS/wasm → portable across platforms).
|
||||
STAGE="$WORK/codegraph-${TARGET}"
|
||||
mkdir -p "$STAGE/lib" "$STAGE/bin"
|
||||
cp -R "$ROOT/dist" "$STAGE/lib/dist"
|
||||
cp "$ROOT/package.json" "$ROOT/package-lock.json" "$STAGE/lib/"
|
||||
echo "[bundle] installing production dependencies"
|
||||
( cd "$STAGE/lib" && npm ci --omit=dev --ignore-scripts >/dev/null 2>&1 )
|
||||
rm -f "$STAGE/lib/package-lock.json"
|
||||
|
||||
# 4. Vendored Node + launcher (the launcher uses the bundled Node by relative
|
||||
# path, so no system Node is ever needed).
|
||||
if [ "$OSFAM" = "win32" ]; then
|
||||
cp "$NODE_BIN" "$STAGE/node.exe"
|
||||
printf '@"%%~dp0..\\node.exe" "%%~dp0..\\lib\\dist\\bin\\codegraph.js" %%*\r\n' \
|
||||
> "$STAGE/bin/codegraph.cmd"
|
||||
else
|
||||
cp "$NODE_BIN" "$STAGE/node"
|
||||
cat > "$STAGE/bin/codegraph" <<'LAUNCH'
|
||||
#!/bin/sh
|
||||
DIR="$(cd "$(dirname "$0")/.." && pwd)"
|
||||
exec "$DIR/node" "$DIR/lib/dist/bin/codegraph.js" "$@"
|
||||
LAUNCH
|
||||
chmod +x "$STAGE/bin/codegraph"
|
||||
fi
|
||||
|
||||
# 5. Archive (.zip for Windows, .tar.gz otherwise).
|
||||
mkdir -p "$OUT"
|
||||
if [ "$OSFAM" = "win32" ]; then
|
||||
ARCHIVE="$OUT/codegraph-${TARGET}.zip"
|
||||
rm -f "$ARCHIVE"
|
||||
( cd "$WORK" && zip -rqX "$ARCHIVE" "codegraph-${TARGET}" )
|
||||
else
|
||||
ARCHIVE="$OUT/codegraph-${TARGET}.tar.gz"
|
||||
# --no-xattrs: don't embed macOS xattrs that make GNU tar warn on Linux.
|
||||
tar --no-xattrs -czf "$ARCHIVE" -C "$WORK" "codegraph-${TARGET}"
|
||||
fi
|
||||
echo "[bundle] wrote ${ARCHIVE} ($(du -h "$ARCHIVE" | cut -f1))"
|
||||
Executable
+43
@@ -0,0 +1,43 @@
|
||||
#!/usr/bin/env node
|
||||
'use strict';
|
||||
//
|
||||
// npm thin-installer launcher for CodeGraph.
|
||||
//
|
||||
// The heavy artifact (a vendored Node runtime + the app) ships as a per-platform
|
||||
// optionalDependency: @colbymchenry/codegraph-<platform>-<arch>. npm installs
|
||||
// only the one matching the host, via each package's `os`/`cpu` fields (the
|
||||
// esbuild pattern). This shim — run by the user's OWN Node — locates that bundle
|
||||
// and execs its launcher, so the real work always runs on the bundled Node 24
|
||||
// (with node:sqlite), regardless of the user's Node version. The user's Node is
|
||||
// only ever a launcher; even an ancient version can run this file.
|
||||
//
|
||||
// Wired up at release time as the main package's `bin`:
|
||||
// "bin": { "codegraph": "scripts/npm-shim.js" }
|
||||
// with the platform packages listed in `optionalDependencies`.
|
||||
|
||||
var childProcess = require('child_process');
|
||||
|
||||
var target = process.platform + '-' + process.arch; // e.g. darwin-arm64, linux-x64
|
||||
var pkg = '@colbymchenry/codegraph-' + target;
|
||||
var launcher = process.platform === 'win32' ? 'bin/codegraph.cmd' : 'bin/codegraph';
|
||||
|
||||
var binPath;
|
||||
try {
|
||||
binPath = require.resolve(pkg + '/' + launcher);
|
||||
} catch (e) {
|
||||
process.stderr.write(
|
||||
'codegraph: no prebuilt bundle for ' + target + '.\n' +
|
||||
'Expected the optional package ' + pkg + ' to be installed.\n' +
|
||||
'Try reinstalling: npm i -g @colbymchenry/codegraph\n' +
|
||||
'Or use the standalone installer (no Node required):\n' +
|
||||
' curl -fsSL https://raw.githubusercontent.com/colbymchenry/codegraph/main/install.sh | sh\n'
|
||||
);
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
var res = childProcess.spawnSync(binPath, process.argv.slice(2), { stdio: 'inherit' });
|
||||
if (res.error) {
|
||||
process.stderr.write('codegraph: ' + res.error.message + '\n');
|
||||
process.exit(1);
|
||||
}
|
||||
process.exit(res.status === null ? 1 : res.status);
|
||||
Executable
+95
@@ -0,0 +1,95 @@
|
||||
#!/usr/bin/env bash
|
||||
#
|
||||
# Assemble the npm thin-installer packages from built bundles (esbuild pattern).
|
||||
#
|
||||
# Produces, under release/npm/:
|
||||
# codegraph-<target>/ one per built bundle — the vendored Node + app, tagged
|
||||
# with os/cpu so npm installs only the matching one.
|
||||
# main/ the @colbymchenry/codegraph shim package: a tiny bin
|
||||
# that execs the matching platform bundle, with every
|
||||
# platform package in optionalDependencies.
|
||||
#
|
||||
# The release pipeline then `npm publish`es each dir. This does NOT touch the
|
||||
# repo's package.json — the dev/from-source path keeps working; the *published*
|
||||
# main package's shape is generated here.
|
||||
#
|
||||
# Prereq: run build-bundle.sh for each target first (release/codegraph-*.tar.gz).
|
||||
# Usage: scripts/pack-npm.sh [version] (default: version from package.json)
|
||||
set -euo pipefail
|
||||
|
||||
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
|
||||
VERSION="${1:-$(node -p "require('$ROOT/package.json').version")}"
|
||||
SCOPE="@colbymchenry"
|
||||
REL="$ROOT/release"
|
||||
NPM="$REL/npm"
|
||||
|
||||
rm -rf "$NPM"
|
||||
mkdir -p "$NPM/main"
|
||||
|
||||
shopt -s nullglob
|
||||
archives=("$REL"/codegraph-*.tar.gz "$REL"/codegraph-*.zip)
|
||||
[ ${#archives[@]} -gt 0 ] || { echo "[pack-npm] no bundles in $REL — run build-bundle.sh first" >&2; exit 1; }
|
||||
|
||||
targets=()
|
||||
for archive in "${archives[@]}"; do
|
||||
fname="$(basename "$archive")"
|
||||
case "$fname" in
|
||||
*.tar.gz) base="${fname%.tar.gz}" ;; # codegraph-<target>
|
||||
*.zip) base="${fname%.zip}" ;;
|
||||
esac
|
||||
target="${base#codegraph-}" # <target>, e.g. darwin-arm64 / win32-x64
|
||||
os="${target%-*}" # darwin | linux | win32
|
||||
arch="${target##*-}" # arm64 | x64
|
||||
pkgdir="$NPM/$base"
|
||||
mkdir -p "$pkgdir"
|
||||
case "$fname" in
|
||||
*.zip)
|
||||
tmpx="$(mktemp -d)"
|
||||
unzip -q "$archive" -d "$tmpx"
|
||||
mv "$tmpx/codegraph-${target}"/* "$pkgdir"/
|
||||
rm -rf "$tmpx"
|
||||
nodefile="node.exe"
|
||||
;;
|
||||
*)
|
||||
tar -xzf "$archive" -C "$pkgdir" --strip-components=1
|
||||
nodefile="node"
|
||||
;;
|
||||
esac
|
||||
VERSION="$VERSION" SCOPE="$SCOPE" TARGET="$target" OSV="$os" ARCHV="$arch" NODEFILE="$nodefile" \
|
||||
node -e '
|
||||
const fs=require("fs");
|
||||
fs.writeFileSync(process.argv[1], JSON.stringify({
|
||||
name: `${process.env.SCOPE}/codegraph-${process.env.TARGET}`,
|
||||
version: process.env.VERSION,
|
||||
description: `CodeGraph self-contained bundle for ${process.env.TARGET}`,
|
||||
os: [process.env.OSV], cpu: [process.env.ARCHV],
|
||||
files: [process.env.NODEFILE, "lib", "bin"],
|
||||
license: "MIT"
|
||||
}, null, 2) + "\n");
|
||||
' "$pkgdir/package.json"
|
||||
targets+=("$target")
|
||||
echo "[pack-npm] ${SCOPE}/codegraph-${target}@${VERSION}"
|
||||
done
|
||||
|
||||
# Main shim package.
|
||||
cp "$ROOT/scripts/npm-shim.js" "$NPM/main/npm-shim.js"
|
||||
[ -f "$ROOT/README.md" ] && cp "$ROOT/README.md" "$NPM/main/README.md"
|
||||
VERSION="$VERSION" SCOPE="$SCOPE" TARGETS="${targets[*]}" \
|
||||
node -e '
|
||||
const fs=require("fs");
|
||||
const opt={};
|
||||
for (const t of process.env.TARGETS.split(/\s+/).filter(Boolean))
|
||||
opt[`${process.env.SCOPE}/codegraph-${t}`]=process.env.VERSION;
|
||||
fs.writeFileSync(process.argv[1], JSON.stringify({
|
||||
name: `${process.env.SCOPE}/codegraph`,
|
||||
version: process.env.VERSION,
|
||||
description: "Local-first code intelligence for AI agents (MCP). Self-contained — bundles its own runtime.",
|
||||
bin: { codegraph: "npm-shim.js" },
|
||||
optionalDependencies: opt,
|
||||
files: ["npm-shim.js","README.md"],
|
||||
license: "MIT"
|
||||
}, null, 2) + "\n");
|
||||
' "$NPM/main/package.json"
|
||||
|
||||
echo "[pack-npm] ${SCOPE}/codegraph@${VERSION} (${#targets[@]} platform packages in optionalDependencies)"
|
||||
echo "[pack-npm] output: $NPM"
|
||||
Reference in New Issue
Block a user