feat(kernel): R1 scaffold — napi-rs extraction kernel, buffer contract, routing + fallback, grammar-parity CI
Phase 0 of the Rust extraction-kernel migration (docs/design/ rust-kernel-migration-plan.md, now checked in with §3a recording the shipped state): - codegraph-kernel/ napi-rs crate: extractFile(path, content, language) → five flat buffers (meta/nodes/edges/refs/arena), one JS boundary crossing per file. Node ids computed Rust-side, byte-identical to generateNodeId (pinned by test vector). Reserved per-node metrics slot for the Arc 3.2 code-metrics work. - Generic .scm-driven emitter (@def.<kind>/@name/@ref.<kind> captures, byte-range scope stack → ::-joined qualified names, contains edges, refs attributed to the innermost enclosing symbol). Seed TS/JS queries are smoke-level; R2 replaces them with the full port. - Routing seam in extractFromSource with per-file wasm fallback. DEFAULT_ROUTED is empty — no behavior change until a language passes its equivalence gate (R3). Dev opt-in: CODEGRAPH_KERNEL_LANGS. Kill switch: CODEGRAPH_KERNEL=0. Loader verifies ABI + kind tables before routing; EDGE_KINDS became a runtime array because kind order is now wire contract. - Grammar-source parity: vendored TS/TSX/JS wasm grammars built from the exact crate revisions (tree-sitter-typescript v0.23.2, tree-sitter-javascript v0.25.0, checked-in parser.c, ts-cli 0.25.10) — the tree-sitter-wasms builds were 2023-era, which the new kernel-grammar-parity test caught on day one. Production TS/JS parsing gets 2.5 years of grammar fixes; full suite green (2456 tests). - Build/release wiring: scripts/build-kernel.sh + npm run build:kernel; release.yml kernel prebuild matrix (continue-on-error — the kernel is optional everywhere, bundles fall back to the wasm path); bundles stage lib/kernel/codegraph-kernel.node; release job runs the kernel suites with CODEGRAPH_KERNEL_EXPECT=1. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Fable 5
parent
4efc6c70e2
commit
c5eebe6beb
@@ -0,0 +1,78 @@
|
||||
//! Per-language specs: grammar + `.scm` query + (later) per-language config.
|
||||
//!
|
||||
//! Tier-1 languages are meant to be *mostly* a query file plus a small config
|
||||
//! here; logic queries can't express stays TS-side as a per-language `post()`
|
||||
//! hook over the returned buffers (see `src/extraction/kernel/route.ts`).
|
||||
//!
|
||||
//! Language strings are codegraph `Language` values (src/types.ts), not
|
||||
//! grammar names — `tsx` and `jsx` are separate entries that reuse another
|
||||
//! entry's grammar exactly like `WASM_GRAMMAR_FILES` does on the wasm path.
|
||||
|
||||
use std::sync::OnceLock;
|
||||
use tree_sitter::{Language, Query};
|
||||
|
||||
pub struct LangSpec {
|
||||
/// codegraph Language string (src/types.ts).
|
||||
pub name: &'static str,
|
||||
get_language: fn() -> Language,
|
||||
query_src: &'static str,
|
||||
language: OnceLock<Language>,
|
||||
query: OnceLock<Result<Query, String>>,
|
||||
}
|
||||
|
||||
impl LangSpec {
|
||||
const fn new(name: &'static str, get_language: fn() -> Language, query_src: &'static str) -> Self {
|
||||
LangSpec {
|
||||
name,
|
||||
get_language,
|
||||
query_src,
|
||||
language: OnceLock::new(),
|
||||
query: OnceLock::new(),
|
||||
}
|
||||
}
|
||||
|
||||
pub fn language(&self) -> &Language {
|
||||
self.language.get_or_init(self.get_language)
|
||||
}
|
||||
|
||||
pub fn query(&self) -> Result<&Query, String> {
|
||||
self.query
|
||||
.get_or_init(|| {
|
||||
Query::new(self.language(), self.query_src)
|
||||
.map_err(|e| format!("query compile failed for {}: {e}", self.name))
|
||||
})
|
||||
.as_ref()
|
||||
.map_err(|e| e.clone())
|
||||
}
|
||||
}
|
||||
|
||||
fn ts_language() -> Language {
|
||||
tree_sitter_typescript::LANGUAGE_TYPESCRIPT.into()
|
||||
}
|
||||
|
||||
fn tsx_language() -> Language {
|
||||
tree_sitter_typescript::LANGUAGE_TSX.into()
|
||||
}
|
||||
|
||||
fn js_language() -> Language {
|
||||
tree_sitter_javascript::LANGUAGE.into()
|
||||
}
|
||||
|
||||
static TYPESCRIPT: LangSpec = LangSpec::new(
|
||||
"typescript",
|
||||
ts_language,
|
||||
include_str!("../queries/typescript.scm"),
|
||||
);
|
||||
static TSX: LangSpec = LangSpec::new("tsx", tsx_language, include_str!("../queries/typescript.scm"));
|
||||
static JAVASCRIPT: LangSpec = LangSpec::new(
|
||||
"javascript",
|
||||
js_language,
|
||||
include_str!("../queries/javascript.scm"),
|
||||
);
|
||||
static JSX: LangSpec = LangSpec::new("jsx", js_language, include_str!("../queries/javascript.scm"));
|
||||
|
||||
pub static ALL: [&LangSpec; 4] = [&TYPESCRIPT, &TSX, &JAVASCRIPT, &JSX];
|
||||
|
||||
pub fn spec_for(language: &str) -> Option<&'static LangSpec> {
|
||||
ALL.iter().find(|s| s.name == language).copied()
|
||||
}
|
||||
Reference in New Issue
Block a user