feat(kernel): R1 scaffold — napi-rs extraction kernel, buffer contract, routing + fallback, grammar-parity CI

Phase 0 of the Rust extraction-kernel migration (docs/design/
rust-kernel-migration-plan.md, now checked in with §3a recording the
shipped state):

- codegraph-kernel/ napi-rs crate: extractFile(path, content, language)
  → five flat buffers (meta/nodes/edges/refs/arena), one JS boundary
  crossing per file. Node ids computed Rust-side, byte-identical to
  generateNodeId (pinned by test vector). Reserved per-node metrics slot
  for the Arc 3.2 code-metrics work.
- Generic .scm-driven emitter (@def.<kind>/@name/@ref.<kind> captures,
  byte-range scope stack → ::-joined qualified names, contains edges,
  refs attributed to the innermost enclosing symbol). Seed TS/JS queries
  are smoke-level; R2 replaces them with the full port.
- Routing seam in extractFromSource with per-file wasm fallback.
  DEFAULT_ROUTED is empty — no behavior change until a language passes
  its equivalence gate (R3). Dev opt-in: CODEGRAPH_KERNEL_LANGS. Kill
  switch: CODEGRAPH_KERNEL=0. Loader verifies ABI + kind tables before
  routing; EDGE_KINDS became a runtime array because kind order is now
  wire contract.
- Grammar-source parity: vendored TS/TSX/JS wasm grammars built from the
  exact crate revisions (tree-sitter-typescript v0.23.2,
  tree-sitter-javascript v0.25.0, checked-in parser.c, ts-cli 0.25.10) —
  the tree-sitter-wasms builds were 2023-era, which the new
  kernel-grammar-parity test caught on day one. Production TS/JS parsing
  gets 2.5 years of grammar fixes; full suite green (2456 tests).
- Build/release wiring: scripts/build-kernel.sh + npm run build:kernel;
  release.yml kernel prebuild matrix (continue-on-error — the kernel is
  optional everywhere, bundles fall back to the wasm path); bundles stage
  lib/kernel/codegraph-kernel.node; release job runs the kernel suites
  with CODEGRAPH_KERNEL_EXPECT=1.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Colby McHenry
2026-07-16 20:13:51 -05:00
co-authored by Claude Fable 5
parent 4efc6c70e2
commit c5eebe6beb
29 changed files with 2804 additions and 16 deletions
+63
View File
@@ -29,8 +29,45 @@ permissions:
attestations: write # store the GitHub artifact attestations for the bundles
jobs:
# Native extraction-kernel prebuilds (docs/design/rust-kernel-migration-plan.md).
# The kernel is an OPTIONAL per-language speedup: a bundle without a .node
# runs the wasm extraction path unchanged. continue-on-error keeps a Rust
# toolchain flake from ever blocking a release — the release job runs with
# whatever prebuilds succeeded. (Runner images ship rustup; build-kernel.sh
# adds each cross target itself.)
kernel:
continue-on-error: true
strategy:
fail-fast: false
matrix:
include:
- runner: macos-14
targets: aarch64-apple-darwin x86_64-apple-darwin
- runner: ubuntu-22.04 # oldest glibc runner → widest compatibility
targets: x86_64-unknown-linux-gnu
- runner: ubuntu-22.04-arm
targets: aarch64-unknown-linux-gnu
- runner: windows-latest
targets: x86_64-pc-windows-msvc aarch64-pc-windows-msvc
runs-on: ${{ matrix.runner }}
steps:
- uses: actions/checkout@v6
- name: Build kernel prebuilds
shell: bash
run: |
for t in ${{ matrix.targets }}; do
bash scripts/build-kernel.sh --target "$t"
done
ls -R codegraph-kernel/prebuilds
- uses: actions/upload-artifact@v4
with:
name: kernel-${{ matrix.runner }}
path: codegraph-kernel/prebuilds/
if-no-files-found: error
release:
runs-on: ubuntu-latest
needs: kernel
steps:
- uses: actions/checkout@v6
with:
@@ -127,6 +164,32 @@ jobs:
git push origin "HEAD:${GITHUB_REF#refs/heads/}"
fi
- name: Download kernel prebuilds
# Best-effort: whatever platform legs succeeded land in release/kernel/
# (<target>/codegraph-kernel.node); build-bundle.sh includes a target's
# kernel when present and falls back to the wasm path when not.
continue-on-error: true
uses: actions/download-artifact@v4
with:
pattern: kernel-*
merge-multiple: true
path: release/kernel/
- name: Kernel contract + grammar-parity gate
# Asserts the native grammars and the vendored wasm grammars are built
# from the same grammar revisions (node-kind/field tables compared id
# by id) and that the .node speaks the expected wire contract.
# CODEGRAPH_KERNEL_EXPECT=1 turns a missing binary into a FAILURE here
# so the gate can't silently pass by not building the kernel.
run: |
if [ -f release/kernel/linux-x64/codegraph-kernel.node ]; then
mkdir -p codegraph-kernel/prebuilds/linux-x64
cp release/kernel/linux-x64/codegraph-kernel.node codegraph-kernel/prebuilds/linux-x64/
CODEGRAPH_KERNEL_EXPECT=1 npx vitest run __tests__/kernel-scaffold.test.ts __tests__/kernel-grammar-parity.test.ts
else
echo "::warning::no linux-x64 kernel prebuild — skipping kernel gate (bundles ship wasm-only)"
fi
- name: Build all platform bundles
run: |
for t in darwin-arm64 darwin-x64 linux-x64 linux-arm64 win32-x64 win32-arm64; do