feat(kernel): R2 — full TypeScript/JavaScript extraction port, byte-parity with the wasm path
Replaces the R1 seed .scm emitter with a bespoke Rust walker (codegraph-kernel/src/tsjs/) that mirrors TreeSitterExtractor's TS/JS paths function-for-function: declarations (incl. #808 field/property classification), qualified names, docstrings (#780 wrapper climbs), signatures, imports/re-exports + per-binding refs, calls with receiver-qualified callees (#1230 literal-receiver skip), instantiations, decorators, inheritance, type annotations (#381), type-alias members + tuple contracts (#359/#634), React component recognition (#841 forwardRef/memo/styled), object-of-functions / zustand-through-middleware / RTK Query endpoints + generated hooks / vuex + pinia store shapes, function-as-value capture with the flush gate (#756), and value-reference edges with the shadow prune (#895/#897). The generic query emitter is deleted — extraction parity needs logic .scm can't express; future languages get walkers too (migration plan §4a). Positions and JS string-slice semantics are emitted in UTF-16 code units natively, so kernel output is byte-identical to web-tree-sitter's — no column diff class exists. Parity evidence (macOS): scripts/kernel-parity.mjs (full-object multiset diff per file) — this repo 353/353 files, excalidraw 643/643 (10,650 nodes / 10,726 edges / 68,307 refs), plus torture fixtures checked into __tests__/fixtures/kernel-parity/ and enforced in npm test by kernel-tsjs-parity.test.ts. The strict compare caught one real decoder bug the loose harness missed: refs must NOT carry denormalized filePath/language at the extractFromSource seam (the store fills them). Perf: extraction 2.6× single-thread on excalidraw (487ms vs 1,255ms, identical outputs). Routing stays opt-in (CODEGRAPH_KERNEL_LANGS) until the R3 equivalence gate (large repo, DB dump-diff, retrieval invariants, agent A/B, Linux/Windows) passes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Fable 5
parent
c5eebe6beb
commit
9ad5cd7ba2
@@ -9,13 +9,17 @@
|
||||
//! Calls are synchronous by design: the existing `ParseWorkerPool` workers
|
||||
//! already parallelize per-file, so each worker thread drives its own kernel
|
||||
//! call (do NOT rebuild the pool on the Rust side — see the migration plan §3).
|
||||
//!
|
||||
//! Per-language extraction lives in a dedicated walker module (tsjs/ for
|
||||
//! typescript/tsx/javascript/jsx) that mirrors the TS extractor for behavioral
|
||||
//! parity — verified by scripts/kernel-parity.mjs and the §5 gate.
|
||||
|
||||
#![deny(clippy::all)]
|
||||
|
||||
mod buffers;
|
||||
mod emitter;
|
||||
mod ids;
|
||||
mod langs;
|
||||
mod tsjs;
|
||||
|
||||
use napi::bindgen_prelude::*;
|
||||
use napi_derive::napi;
|
||||
@@ -63,14 +67,13 @@ pub fn contract_info() -> ContractInfo {
|
||||
kernel_version: env!("CARGO_PKG_VERSION").to_string(),
|
||||
node_kinds: buffers::NODE_KINDS.iter().map(|s| s.to_string()).collect(),
|
||||
edge_kinds: buffers::EDGE_KINDS.iter().map(|s| s.to_string()).collect(),
|
||||
languages: langs::ALL.iter().map(|s| s.name.to_string()).collect(),
|
||||
languages: langs::LANGUAGES.iter().map(|s| s.to_string()).collect(),
|
||||
}
|
||||
}
|
||||
|
||||
#[napi]
|
||||
pub fn grammar_info(language: String) -> Option<GrammarInfo> {
|
||||
let spec = langs::spec_for(&language)?;
|
||||
let lang = spec.language();
|
||||
let lang = langs::grammar_for(&language)?;
|
||||
let node_kind_count = lang.node_kind_count();
|
||||
let field_count = lang.field_count();
|
||||
let node_kinds = (0..node_kind_count)
|
||||
@@ -91,9 +94,7 @@ pub fn grammar_info(language: String) -> Option<GrammarInfo> {
|
||||
|
||||
#[napi]
|
||||
pub fn extract_file(file_path: String, content: String, language: String) -> Result<ExtractBuffers> {
|
||||
let spec = langs::spec_for(&language)
|
||||
.ok_or_else(|| Error::from_reason(format!("kernel does not support language: {language}")))?;
|
||||
let out = emitter::extract(&file_path, &content, spec).map_err(Error::from_reason)?;
|
||||
let out = tsjs::extract(&file_path, &content, &language).map_err(Error::from_reason)?;
|
||||
Ok(ExtractBuffers {
|
||||
meta: out.meta.into(),
|
||||
nodes: out.nodes.into(),
|
||||
|
||||
Reference in New Issue
Block a user