feat(kernel): R2 — full TypeScript/JavaScript extraction port, byte-parity with the wasm path

Replaces the R1 seed .scm emitter with a bespoke Rust walker
(codegraph-kernel/src/tsjs/) that mirrors TreeSitterExtractor's TS/JS
paths function-for-function: declarations (incl. #808 field/property
classification), qualified names, docstrings (#780 wrapper climbs),
signatures, imports/re-exports + per-binding refs, calls with
receiver-qualified callees (#1230 literal-receiver skip), instantiations,
decorators, inheritance, type annotations (#381), type-alias members +
tuple contracts (#359/#634), React component recognition (#841
forwardRef/memo/styled), object-of-functions / zustand-through-middleware
/ RTK Query endpoints + generated hooks / vuex + pinia store shapes,
function-as-value capture with the flush gate (#756), and value-reference
edges with the shadow prune (#895/#897). The generic query emitter is
deleted — extraction parity needs logic .scm can't express; future
languages get walkers too (migration plan §4a).

Positions and JS string-slice semantics are emitted in UTF-16 code units
natively, so kernel output is byte-identical to web-tree-sitter's — no
column diff class exists.

Parity evidence (macOS): scripts/kernel-parity.mjs (full-object multiset
diff per file) — this repo 353/353 files, excalidraw 643/643 (10,650
nodes / 10,726 edges / 68,307 refs), plus torture fixtures checked into
__tests__/fixtures/kernel-parity/ and enforced in npm test by
kernel-tsjs-parity.test.ts. The strict compare caught one real decoder
bug the loose harness missed: refs must NOT carry denormalized
filePath/language at the extractFromSource seam (the store fills them).

Perf: extraction 2.6× single-thread on excalidraw (487ms vs 1,255ms,
identical outputs). Routing stays opt-in (CODEGRAPH_KERNEL_LANGS) until
the R3 equivalence gate (large repo, DB dump-diff, retrieval invariants,
agent A/B, Linux/Windows) passes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Colby McHenry
2026-07-16 22:14:40 -05:00
co-authored by Claude Fable 5
parent c5eebe6beb
commit 9ad5cd7ba2
20 changed files with 3382 additions and 419 deletions
+8 -7
View File
@@ -9,13 +9,17 @@
//! Calls are synchronous by design: the existing `ParseWorkerPool` workers
//! already parallelize per-file, so each worker thread drives its own kernel
//! call (do NOT rebuild the pool on the Rust side — see the migration plan §3).
//!
//! Per-language extraction lives in a dedicated walker module (tsjs/ for
//! typescript/tsx/javascript/jsx) that mirrors the TS extractor for behavioral
//! parity — verified by scripts/kernel-parity.mjs and the §5 gate.
#![deny(clippy::all)]
mod buffers;
mod emitter;
mod ids;
mod langs;
mod tsjs;
use napi::bindgen_prelude::*;
use napi_derive::napi;
@@ -63,14 +67,13 @@ pub fn contract_info() -> ContractInfo {
kernel_version: env!("CARGO_PKG_VERSION").to_string(),
node_kinds: buffers::NODE_KINDS.iter().map(|s| s.to_string()).collect(),
edge_kinds: buffers::EDGE_KINDS.iter().map(|s| s.to_string()).collect(),
languages: langs::ALL.iter().map(|s| s.name.to_string()).collect(),
languages: langs::LANGUAGES.iter().map(|s| s.to_string()).collect(),
}
}
#[napi]
pub fn grammar_info(language: String) -> Option<GrammarInfo> {
let spec = langs::spec_for(&language)?;
let lang = spec.language();
let lang = langs::grammar_for(&language)?;
let node_kind_count = lang.node_kind_count();
let field_count = lang.field_count();
let node_kinds = (0..node_kind_count)
@@ -91,9 +94,7 @@ pub fn grammar_info(language: String) -> Option<GrammarInfo> {
#[napi]
pub fn extract_file(file_path: String, content: String, language: String) -> Result<ExtractBuffers> {
let spec = langs::spec_for(&language)
.ok_or_else(|| Error::from_reason(format!("kernel does not support language: {language}")))?;
let out = emitter::extract(&file_path, &content, spec).map_err(Error::from_reason)?;
let out = tsjs::extract(&file_path, &content, &language).map_err(Error::from_reason)?;
Ok(ExtractBuffers {
meta: out.meta.into(),
nodes: out.nodes.into(),