R7b batch 4 #2 (docs/design/lua-luau-kernel-port-checklist.md is the authoritative quirk list). ONE walker for both dialects (ccpp precedent) — the differences are exactly four: luau's type_definition aliases, the `export `-slice isExported hook, the return-type signature suffix, and the grammar handle. Grammar prep is kernel-side only, no wasm change: lua is the SECOND vendored-grammar-C language (the vendored wasm is the v0.4.1 tag, a revision not on crates.io — tag artifacts compiled via build.rs, shas pinned); luau is a plain crate pin =1.2.0 whose tarball is sha-identical to the tag (the swift tag≠crate divergence does not recur). Grammar-parity rows replace the bump gate entirely. Preserved bug-for-bug (all probe-pinned): the require/visitNode-hook ASYMMETRIES (top-level requires — including inside top-level if/for/while — mint import nodes while the identical body-level statement emits `calls "require"`; top-level `local x = foo()` initializers are invisible while global `x = foo()` calls emit), the BFS string-win inside require args (`require(script:WaitForChild("Kid"))` → import Kid) and Roblox instance paths, receiver-QN methods (`M.sub.deep::chained`, `_G::installed`, stack-QN nested globals like `render::leakedGlobal`), the raw-text callee world (colon forms with `self` never stripped, bracket callees, newline-glued chains byte-verbatim, the `(handler)` paren-conversion), LUA_SPEC function-as-value capture with the `M.cb = cb` param-storage skip and first-occurrence dedupe, LuaDoc `---` keeping a leading `- ` plus `--!strict` joining docstring chains (block-comment docstrings keep interior CRLF bytes), variable nodes at the IDENTIFIER with positional value pairing, duplicate same-(kind,name,line) ids, and the lua↔luau isExported wire divergence (lua functions: flag absent; luau functions: present-false; methods: absent in both; variables: present-false in both; `export type`: true). Gates: parity sweeps first-run 0-diff on kong/lazy.nvim/lua-resty-core (lua) + lune/Fusion (luau) — 1,734 clean files byte-parity, deferrals 1/0/0/3/8 matching the survey's both-arm predictions exactly (kong's 1 = a deliberately invalid fixture; luau's = grammar-inherent generic type packs and default type params); full-init dumps byte-identical kernel-vs-wasm ×4 (kong 157,650 dump lines); kernel-lua-parity suite (both torture fixtures + in-memory CRLF variants + glue-chain, duplicate-id, and cross-dialect defer pins + kernel-arm wire-flag pins); full suite 2,647 green ×2 with CODEGRAPH_KERNEL_EXPECT=1. DEFAULT_ROUTED += lua, luau (18 langs). Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
48 KiB
Lua + Luau kernel port (R7b batch 4) — the bug-for-bug checklist
Status: SURVEY COMPLETE (2026-07-20) — one combined checklist for BOTH
languages: languages/luau.ts is 36 lines extending languages/lua.ts, the two
grammars share their node-name vocabulary, and the recommendation is ONE walker
module (codegraph-kernel/src/lua.rs, ccpp-style language-parameterized —
"lua" | "luau" => lua::extract(&file_path, &content, &language) in
lib.rs:214-226). Survey basis: every TS-side branch a .lua/.luau file
exercises, with file:line anchors as of 45a53eb (HEAD at survey time,
clean main). Every grammar-shape claim was probed against the production
vendored wasms (dist/extraction/wasm/tree-sitter-{lua,luau}.wasm) via CST
dumps and childForFieldName truth tables, and every extraction-behavior claim
was pinned against the real dist/ extractor (extract-*.txt ground-truth
dumps in the session scratchpad svy-lua/ — §Probe artifacts), not derived
from code reading alone. Read WITH
docs/design/rust-kernel-migration-plan.md (§0a recipe, §2 boundary, §4
tracker, §5 gates) and the format precedents
(kotlin-kernel-port-checklist.md — the depth bar — plus
swift-kernel-port-checklist.md, ruby-kernel-port-checklist.md).
Blocking findings: none. Three eyes-open items. (1) No grammar bump at
all — both languages are already in VENDORED_WASM_LANGS (grammars.ts:292)
and the vendored wasms are the exact port-target revisions
(../scratchpad/batch4-grammar-probe.md, verified 2026-07-20: lua = the
v0.4.1 tag, table-identical; luau = crate 1.2.0, table-identical). The port is
kernel-side only: lua takes the vendored-grammar-C route (kotlin
mechanism, second use) because v0.4.1 is not on crates.io; luau is a plain
crate pin tree-sitter-luau = "=1.2.0" (csharp-style), and the crate
tarball's parser.c/scanner.c are sha-identical to the v1.2.0 tag
(§Grammar prep — the swift tag≠crate divergence does NOT recur here). (2)
Error incidence is near-zero for lua (kong 0.08% — a single deliberately
invalid fixture; lazy.nvim and lua-resty-core 0.00%) — ruby-style: any
deferral on a lua sweep is a walker-bug signal. Luau real-world incidence is
1.4–7.1% (lune 1.36%, Fusion 7.08%), driven by two grammar-inherent
shapes — generic type packs (A...) -> T and default type parameters
<T = D> — both-arm by construction (same grammar revision on both arms).
(3) The single biggest bug-for-bug surface is the require/visitNode-hook
machinery and its asymmetries: requires at top level (anywhere visitNode
reaches, including inside top-level if/for/while blocks) become import
nodes, while the IDENTICAL statement inside a function body emits a
calls "require" ref (visitFunctionBody never runs the hook); a top-level
local x = foo() initializer emits no calls ref while a top-level global
assignment x = foo() does (§extractVariable). Get these exactly wrong
and sweeps light up on file one.
Grammar prep (kernel-side only — no wasm change, no bump gate)
Both languages already ship vendored wasms (VENDORED_WASM_LANGS,
grammars.ts:291-293; file map lua: 'tree-sitter-lua.wasm' /
luau: 'tree-sitter-luau.wasm' at grammars.ts:39,41; the :255 comment
records WHY lua was vendored — tree-sitter-wasms' ABI-13 build corrupted the
shared WASM heap). Provenance was verified in the batch-4 grammar probe
(../scratchpad/batch4-grammar-probe.md) and re-verified in this survey:
-
lua — vendored wasm ≡ tree-sitter-grammars/tree-sitter-lua v0.4.1, tag commit
816840c592ab973500ae9750763c707b447e7fef(release asset table-identical, 0 positional mismatches). Tag parser.c declares ABI 15, STATE_COUNT 262, SYMBOL_COUNT 137, FIELD_COUNT 22 (verified in the tag clone — matches the wasm's tables). v0.4.1 is NOT on crates.io (only 0.1/0.2/0.5 exist) → vendored-grammar-C route, the kotlin mechanism (codegraph-kernel/grammars/kotlin+ build.rs cc precedent). 0.5.0 adds Lua-5.5global(+1 field) — a future accuracy bump, NOT this batch. -
luau — vendored wasm ≡ tree-sitter-grammars/tree-sitter-luau v1.2.0 ≡ crates.io crate 1.2.0. Survey-verified sha match, tag ↔ crate tarball (the swift lesson checked and CLEAR):
src/parser.c8f25bc1779400fa93d5492310e291fb895363ec051d44753444e724e99a08250(both)src/scanner.ca157bb5210454add058a08ce53eabcccad41aab6dac7006def2554e5cfebd376(both) Tag commita8914d6c1fc5131f8e1c13f769fa704c9f5eb02f. parser.c declares ABI 14, STATE_COUNT 585, SYMBOL_COUNT 197, FIELD_COUNT 21. Crate deps:tree-sitter-language = "0.1"+ccbuild-dep,tree-sitteronly as a dev-dep (0.26.3) → no pin conflict with the kernel's tree-sitter 0.25. Route:tree-sitter-luau = "=1.2.0"in codegraph-kernel/Cargo.toml,"luau" => Some(tree_sitter_luau::LANGUAGE.into())in langs.rs.
-
Lua kernel C vendor (
codegraph-kernel/grammars/lua/), from the v0.4.1 tag's CHECKED-IN generated artifacts (lua HAS an external scanner — comments and long strings):src/parser.cb34a362e43f0311f405721f3089e94f97f31da403b154d456d093e64609a4081src/scanner.c35bbd630b5a7421d46d2e91185eeea09bf78565d44cb676b63ca20d0f1b54bbdsrc/tree_sitter/alloc.hb29c1c9fb7cc82f58c84b376df1297d6e2737a1d655fd356db0859e3c29c2feasrc/tree_sitter/array.h5bdf6ed1a78e3409fd443e085ca967a64c188a5d082aaf7f819bccd53a471c94src/tree_sitter/parser.h180b893c8734778fd32f372dfbc27bd6ad1cd2221f26150b31256ff6716320d2build.rs: extend the kotlin block —cc::Buildwithinclude("grammars/lua"),file(parser.c),file(scanner.c); crib the tag's ownbindings/rust/build.rsflags (.std("c11"), include src, msvc-utf-8);println!("cargo:rerun-if-changed=grammars/lua"). langs.rs:extern "C" { fn tree_sitter_lua() -> *const (); }+"lua" => Some(unsafe { tree_sitter_language::LanguageFn::from_raw(tree_sitter_lua) }.into())(kotlin precedent, langs.rs:17-20,57).LANGUAGES: [&str; 15]→ 17 (langs.rs:24).
-
Grammar-parity rows:
__tests__/kernel-grammar-parity.test.ts:39GRAMMAR_LANGUAGES += 'lua', 'luau'— asserts the C build / crate and the vendored wasm are the same revision (id-by-id kind+field tables, ABI 15 for lua / ABI 14 for luau). This replaces the bump gate entirely: there is no old-vs-new dump-diff because nothing wasm-side changes. -
Error incidence (current wasm, all files ≤1MiB,
error-sweep.cjs):Repo lang files hasError phantoms first-error classes Kong/kong lua 1,309 1 (0.08%) 0 spec/fixtures/invalid-module.lua— deliberately invalidfolke/lazy.nvim lua 65 0 (0.00%) 0 — openresty/lua-resty-core lua 38 0 (0.00%) 0 — lune-org/lune luau 221 3 (1.36%) 0 .d.luautype-pack/typeof declaration shapesdphfox/Fusion luau 113 8 (7.08%) 0 generic type packs (A...) -> T; default type paramstype Scope<C = Fusion>luau-lang/luau (tests/) luau 203 39 (19.21%) 0 STRESS ONLY — deliberate future-syntax tests ( class Point,123i,export local)JohnnyMorganz/StyLua (tests/) lua 419 118 (28.16%) 0 STRESS ONLY — cfxlua dialect ( +=, C comments), luau-in-.lua corporaZERO phantom-hasError files anywhere (hasError with no ERROR/missing node — probed per-file across all 2,368 files, plus 31 targeted snippet probes in
mini-probes-{lua,luau}.out): unlike kotlin, the defer signal is always accompanied by a real ERROR node. Still gate onhas_error()alone — don't scan for ERROR nodes. Error-class probes (errclass*.luau): luau default type parameterstype X<T = D>and generic packs(A...) -> TERROR; luau also rejects@native/@checkedattributes (real-Luau 2024+ syntax the 1.2.0 grammar predates) and lua-only syntax (goto, bitwisea & b); lua rejects luau-isms (type X = …,continue,+=, backtick interpolation) — cfxlua dialect files land here too. All of these defer to wasm, which errors identically (same tables both arms) and still emits its best-effort partial extraction. -
Sweep defaults:
--max-deferral 0.1(kernel-parity.mjs:38) holds everywhere. Expect 0–1 deferrals on lua repos (treat >0 as suspicious, ruby-style) and 1.4–7.1% on luau (typed-framework codebases sit at the high end; a JUMP past ~10% is the bug signal).
Architecture decisions
- One walker module, two grammar entries.
codegraph-kernel/src/lua.rswith a dialect flag (ccpp precedent — lib.rs:219"c" | "cpp" => ccpp::extract(..., &language)). The dialect differences are exactly four (§Extractor config): typeAliasTypes, isExported, getSignature return suffix, grammar handle. Everything else — require hook, receiver methods, variable branch, call shapes, docstrings, fn-refs — is byte-shared. go.rs is the closest crib (receiver-QN methods, extract_type_alias → bool,node_idsvec, fn-ref flush deduped onnode_ids[from]go.rs:1047). - No preParse (neither lua.ts nor luau.ts has the hook; luau spreads
lua) →
preParsedSource(kernel/index.ts:109-112) is a no-op; both arms parse raw bytes. No POST_PASSES entry →tryKernelExtractRawstays eligible (kernel/index.ts:183). - No framework resolver lists lua/luau (grepped
src/resolution/frameworks/*— zero hits) → no decoded-path forcing in parse-worker; all repos ride the raw buffers-to-store transport. No synthesis consumer either:CC_LANGUAGES = {swift, kotlin}(callback-synthesizer.ts:77),NATIVE = {java, kotlin, objc, cpp}(:1649). The only downstream consumers are resolution-side and contract-pinned in §Resolution consumers. .lua→lua,.luau→luaupurely by extension (grammars.ts:122-123,detectLanguage:469 — no content sniffing; projectcodegraph.jsonextension overrides are TS-side and upstream of the kernel). MAX_FILE_SIZE (1 MiB) and generated-file skips are orchestrator-side and shared.- REF_FLAG_FILE_PATH (wire v2) is NOT needed. The lua hook's
addUnresolvedReference(lua.ts:117-123) passes nofilePath; zero refs in any ground-truth dump carried one. The ruby/php bit stays unused. - Value-refs, static-member refs, type-annotation refs, instantiates, extends/implements, decorates: ALL structurally absent for lua/luau — §Dead machinery. The walker needs cheap early-outs that preserve exactly this nothing.
- Deferral policy: per-file
has_error()→defer:(ruby.rs:143 message convention). Expected counts per §Grammar prep. wasm recovery is canonical for erroring files (incl. the cfxlua/@nativedialect files).
Extractor config (languages/lua.ts — 152 lines; languages/luau.ts — 36 lines; read both whole)
luauExtractor = { ...luaExtractor, <4 overrides> } (luau.ts:16-36).
Shared types (lua.ts:62-81): functionTypes=[function_declaration] (covers
global function f, local function f, table function t.f, method
function t:m — ONE node type, form distinguished by the name: child;
anonymous function() end is function_definition, NOT in the list);
classTypes=[] ; methodTypes=[] ; interfaceTypes=[]; structTypes=[];
enumTypes=[]; importTypes=[] (require is a function_call — the hook);
callTypes=[function_call]; variableTypes=[variable_declaration];
nameField=name, bodyField=body, paramsField=parameters.
Luau overrides (luau.ts:20-35): typeAliasTypes=[type_definition] (lua: []);
isExported = source.slice(node.startIndex, node.startIndex + 7) === 'export ' (raw UTF-16 slice of the node's own first 7 chars — export type X
nodes START at export, probed); getSignature = params text + : <text of the named child AFTER parameters> unless that child's type is block
(finds params by startIndex match in namedChildren, so a leading
generic_type_list never disturbs it — probed (v: T): T).
Hooks PRESENT (port each exactly):
- visitNode (lua.ts:105-151) — the require machinery, §Require hook.
- getSignature (lua.ts:83-86) — lua:
getChildByField(node, 'parameters')text or undefined. Probed:(a, b),(...),(). (Luau override above; both read the REALparametersfield — FIELD_COUNT is non-zero here, unlike kotlin: field lookups are LIVE.) - getReceiverType (lua.ts:92-99) —
name:field child of typedot_index_expression→ itstable:field text;method_index_expression→ itstable:field text; else undefined. The table text is VERBATIM dotted source:function M.sub.deep:chained()→ receiverM.sub.deep(probed). Plainfunction f/local function f(name isidentifier) → undefined.
Hooks ABSENT (the walker must NOT invent them): preParse, resolveName,
recoverMangledName, isMisparsedFunction, isConst (every
lua/luau variable is kind variable — constant never occurs),
isAsync/isStatic (both undefined on every node — flag present-bits
stay 0; NOT kotlin's literal false), getVisibility (undefined — visibility
byte 0), getReturnType (returnType never set — luau return types ride
the SIGNATURE string only), extractModifiers (no decorators ever),
classifyClassNode, classifyMethodNode, extractPropertyName,
propertyTypes, fieldTypes, enumMemberTypes, extraClassNodeTypes,
packageTypes/extractPackage (no namespace node, ever — QNs never get
a package prefix), resolveBody (body = getChildByField(node, 'body')
only; empty one-liner function f() end has NO body field — probed
field-truth.out — so no body walk and no endLine extension; the
createNode:1329-1334 extension is structurally a no-op since
function_declaration's extent includes end), extractImport,
extractBareCall, synthesizeMembers, skipBodilessClass,
methodsAreTopLevel, resolveTypeAliasKind, interfaceKind,
isExported for LUA (undefined — see the wire note in §Parity mechanics).
tree-sitter.ts branches (anchors as of 45a53eb)
visitNode dispatch — what each lua/luau node hits (ladder at 936-1303)
| Node | Branch | Behavior |
|---|---|---|
| every node | visitNode hook first (943-953) | function_call that IS a require → import + ref, handled=true → scanFnRefSubtree + STOP; non-require function_call → false, falls to :1248; variable_declaration → emits imports for require initializers, ALWAYS returns false (falls through to variableTypes) |
| every node | maybeCaptureFnRefs (990) | fires for arguments / assignment_statement / field (LUA_SPEC keys) — top-level table registries and call args captured here |
function_declaration |
functionTypes:994 | isInsideClassLikeNode() is ALWAYS false (no class-like kinds exist; :1486-1500 checks the STACK TOP only, and lua mints only file/function/method/variable/import/type_alias) → extractFunction:1517, whose :1522 receiver short-circuit diverts function t.f/t:m to extractMethod:1737. skipChildren |
variable_declaration |
variableTypes:1098 | gate !isInsideClassLikeNode() ∥ isClassScopeConstantAssignment — always passes (first leg). extractVariable:2538 → lua branch :2789. Then scanFnRefSubtree(node, 0) (:1110) + skipChildren → initializer subtrees are NEVER walked (§extractVariable) |
type_definition (luau) |
typeAliasTypes:1071 | extractTypeAlias:2890 → plain path :2967 (no resolveTypeAliasKind) → returns false → children ARE re-visited — how typeof(require(...)) aliases emit an import (§Luau type aliases). Lua: no branch → recursed |
function_call (hook-declined) |
callTypes:1248 | extractCall:3684. NO skipChildren → children recursed → nested/inner calls each get their own ref (glued chains, f()()) |
assignment_statement (top level) |
no branch | maybeCaptureFnRefs('assignment_statement') → RHS candidates; then plain recursion → initializer calls DO emit (globalAssign = topFn(10) → calls topFn — pinned), anon function_definition RHS bodies are walked with the FILE as caller |
update_statement (luau +=) |
no branch, no dispatch key | recursed → RHS calls emit; no fn-ref capture (dispatch has no update_statement key) |
return_statement / if_statement / for_statement / while_statement / do_statement (top level) |
no branch | recursed → nested variable_declarations/function_calls dispatch normally — top-level conditional requires become imports; block-locals become top-level variable nodes (pinned extract-condreq: if ok then local m = require("in.if") end → import in.if + variable m, both contained by FILE) |
comment |
no branch | recursed into content: — nothing matches. Consumed only by getPrecedingDocstring |
hash_bang_line (#! line 1, lua) |
no branch | parses clean (probed); not a comment kind → BREAKS a docstring chain scanning past it |
INSTANTIATION_KINDS (354-361) / impl_item:1274 / property_signature:1282 / export_statement:1219 / swift property:1121 |
never | no lua/luau node kinds among them. Zero instantiates refs, ever |
The require hook (lua.ts:105-151 + requireModule :28-60) — port verbatim
requireModule(callNode): name: field child must be type identifier with
text exactly require (a dotted callee is dot/method_index — never require);
arguments: field child required. Then, in ORDER:
- String win, breadth-first:
findDescendant(args, 'string_content')(BFS over namedChildren, lua.ts:9-17) → trimmed text. This catches plainrequire("a.b"), single-quote and[[...]]requires (string_content is inside the string node), and any require whose argument merely CONTAINS a string anywhere — pinned:require(script:WaitForChild("Kid"))→ import "Kid" (the string beats the method-index path);require("a" .. "b")→ import "a" (first string_content in BFS order — deterministic garbage, preserve). - Fallback
findDescendant(args, 'string')with manual[[ ]]/quote stripping (reached only if a string node had no content child — empty stringrequire("")→ mod falsy → null). - Roblox instance path: first descendant
dot_index_expression(elsemethod_index_expression) → itsfield:(elsemethod:) child text —require(script.Parent.Signal)→ "Signal" (the trailing segment). - Otherwise null (
require(dynName),require()).
Hook dispatch (lua.ts:129-150):
node.type === 'function_call': requireModule ≠ null →emit(node)+ return true (claimed — never double-counted as a call); null → return false → extractCall emitscalls "require"(pinned:require(dynamicTop)at top level → calls require from FILE).node.type === 'variable_declaration': find theassignment_statementchild → itsexpression_list→ for EVERY child of typefunction_call,emit(val)(multi-requirelocal a, b = require("x"), require("y")→ two imports — pinned). Always returns false → extractVariable also runs. NOT emitted from here: requires nested deeper in the initializer ({ mod = require("t") }table value — pinned NO import), andrequire("m").field(the exprList child is dot_index_expression — pinned NO import; theaccessedvariable still mints).emit(callNode):ctx.createNode('import', mod, callNode, { signature: getNodeText(callNode).trim().slice(0, 100) })— import node positioned at the CALL node (name = module string, qualifiedName = same via buildQualifiedName, id line = call's line) — then, if the stack is non-empty,addUnresolvedReference({ fromNodeId: stack top (always the FILE node), referenceName: mod, referenceKind: 'imports', line: call startRow+1, column: call startColumn }). No filePath on the ref.- The hook NEVER runs inside function bodies (visitFunctionBody has no
hook call — §visitFunctionBody): body-level
local lazy = require("app.lazy")emitscalls "require"from the enclosing function and NO import (pinned,extract-bodies-lua.txtL8-9 — the neovim lazy-loading idiom lands here). Port this asymmetry exactly.
Node creation, IDs, qualified names
- createNode (1308): id =
generateNodeId(filePath, kind, name, startRow+1)=`${kind}:${sha256(`${filePath}:${kind}:${name}:${line}`).hex.slice(0,32)}`(tree-sitter-helpers.ts:18-30). FILE node id = literalfile:${filePath}(:509), name = basename, qualifiedName = filePath, endLine =source.split('\n').length, isExported false. - No namespace node (no packageTypes → extractFilePackage:1397 returns
null). buildQualifiedName (1447-1460) joins the NON-file stack nodes' NAMES
with
::— top-level symbols get bare QNs. - Receiver-QN override: table/method functions get
extraProps. qualifiedName = composeReceiverQualifiedName(receiver, name)(1790-1792) =`${receiverType}::${name}`verbatim (1435-1442; namespacePrefix always empty outside C++). Pinned QNs:M::create,M.sub.deep::chained,_G::installed(yes —function _G.installed()mints method_G::installed). The override REPLACES the stack QN even for nested declarations:function M.attached()inside a body →M::attached, NOTrender::attached(pinned). - Nested plain functions get stack QNs from node NAMES:
local function inner()insideM:render→render::inner(the method's NAME, not itsM::renderQN — buildQualifiedName readsnode.name); a GLOBALfunction leakedGlobal()declared inside a body is still stack-scoped →render::leakedGlobal(pinned — quirk, preserve). - Variable nodes are positioned at the IDENTIFIER, not the declaration (§extractVariable) — id line/column = the identifier's.
- contains edge from stack top for every created node (1363); extractModifiers merge (1355-1358) inert (no hook); captureValueRefScope (1374) runs but its output is discarded (§Dead machinery).
- Duplicate IDs are legal output:
local x = 1; local x = 2on one line → twovariable:xrows with the SAME id (pinned, mini-probessameLineTwoLocals) — minified/one-liner lua bundles hit this routinely. The walker emits both rows verbatim (node_idsvec pattern; store upsert handles collapse — parity compares pre-store output).
extractFunction (1517) / extractMethod (1737)
- extractFunction: :1522 receiver short-circuit is THE routing —
getReceiverType ≠ undefined (name is dot/method_index_expression) →
extractMethod. extractName (:97-192): nameField
namefield hit → :136-143 unwrapsdot_index_expression → field:/method_index_expression → method:→ the trailing simple name (dotted,method,chain).<anonymous>unreachable for function_declaration (grammar requires a name;function_definitionnever reaches extractFunction — it's in no type list). - Extras (:1574-1580): docstring (§Docstrings), signature (getSignature),
visibility undefined, isExported: lua undefined / luau false (hook
slice — a function_declaration never starts with
export), isAsync undefined, isStatic undefined, returnType undefined. extractTypeAnnotations (:1594) → early-out (§Dead machinery); extractDecoratorsFor (:1599) → scans children/preceding siblings for decorator/annotation/attribute kinds — lua'sattributenode (<const>) lives inside variable_list, NEVER adjacent to a function → structurally zero output (keep the traversal or early-out equivalently — output is what's pinned). - Body:
resolveBody?.() ?? getChildByField(node, 'body')→ thebody:block (absent on empty one-liners) → visitFunctionBody with the node pushed. - extractMethod: receiverType recomputed (1742); gate :1747 passes via
receiver. Same extras (NOTE: extractMethod passes NO isExported — luau
METHODS have isExported undefined while luau functions have false;
pinned in
extract-torture-luau.txt:method "make"no flag,function "typedTop"isExported=false). QN override :1790-1792. Owner-contains (:1799-1813) NEVER fires: it requires an in-file node namedreceiverTypewith kind ∈ {struct, class, enum, trait} — lua mints none of those kinds, and dotted receivers (M.sub.deep) match no node name anyway. The ONLY containment is the normal stack-top edge (pinned:contains file → method:dotted,contains method:render → method:attached).
extractVariable — the lua/luau branch (2538-2549 head; 2789-2805 branch)
Head: isConst absent → kind ALWAYS variable (:2546-2547); docstring from
the DECLARATION node (:2548); isExported = hook ?? false (:2549) → false
for both languages (luau's slice sees local …).
Branch (:2789-2805): assign = first namedChild of type
assignment_statement ?? node itself (covers bare local x with no =);
varList = assign's variable_list child; exprList = assign's
expression_list child; names = varList's children of type identifier
ONLY (a dot_index_expression/bracket_index_expression LHS — possible only
in bare assignment_statements, which never reach here — and attribute nodes
(<const>/<close>) are skipped without disturbing positions); values =
exprList's namedChildren. Per name, positionally paired:
createNode(kind, name, nameNode, …)— positioned at the IDENTIFIER (pinned:variable "http" L2 C6-10).- signature =
`= ${valueText.slice(0,100)}${'...' if ≥100}`fromvalues[i]; no value at that index → NO signature key at all (pinned:local ok, err = pcall(...)→okhas the signature,errhas none). Signatures keep raw bytes verbatim — embedded\n(= [[long\nstring]]), CRLF\r\n, the full anon-function text (= function(v)\n\treturn hidden(v)\nend). - docstring and isExported repeated onto EVERY name of the declaration.
Then the ladder (:1110-1111) runs scanFnRefSubtree and sets skipChildren →
the initializer subtree is never walked: local fromCall = topFn(3, 4)
emits NO calls ref; local anon = function(v) return hidden(v) end emits no
function node, no calls (pinned). Contrast the shapes that DO walk (no
branch → recursion): top-level x = topFn(10) assignment → calls topFn;
M.assigned = function(z) return topFn(z) end → calls topFn from the
FILE node (pinned L68). In function BODIES both decl forms walk
(§visitFunctionBody) — the asymmetry is top-level-only.
Luau type aliases — extractTypeAlias (2890; plain path :2967-2991)
type_definition → extractName via the name: FIELD: a simple alias's name
child is identifier; a generic alias's name child is generic_type and
the node NAME is its verbatim text — Generic<T>, Map<K, V> (spaces
preserved, pinned). Extras: docstring (works — pinned "doc for Point"),
isExported from the luau slice hook — export type → true (the
type_definition node STARTS at export, CST-probed), else false. Then
:2973's TYPE_ANNOTATION_LANGUAGES.has('luau') is FALSE → no alias-value
refs (doubly dead: the value field is also NULL in this grammar —
field-truth.out). Returns false → the alias's children are RE-VISITED by
the ladder: a typeof(require(...)) alias's function_call child hits the
hook → type_alias node + import node + imports ref, all three (pinned:
FromTypeof → type_alias L11 C0-55 + import "Config" L11 C25-54 + imports
ref from FILE; the alias node is created FIRST). Body-local type X = …
(legal luau) minted NOTHING — visitFunctionBody has no typeAlias branch
(pinned extract-bodies-luau.txt). Lua: typeAliasTypes=[] — a stray
type X = in a .lua file is a parse ERROR anyway (deferred).
extractCall (3684) — the raw-text callee world
Entry: the vbnet (:3698), erlang (:3746), ruby (:3913), arkts (:3996)
branches are language-gated off. Generic path :4312-4313: func = getChildByField(node, 'function') ?? node.namedChild(0) — the function
field is NULL in this grammar (truth-tabled) → always namedChild(0), which
is the name: child (identifier / dot_index / method_index /
parenthesized_expression / function_call in chains).
The member branch (:4364) NEVER fires: dot_index_expression and
method_index_expression are NOT in the accepted type list
(member_expression/attribute/selector_expression/navigation_expression/
field_expression). LITERAL_RECEIVER_TYPES (:373-388), SKIP_RECEIVERS
(:4400), and every re-encode are unreachable. Everything falls to the ELSE
(:4518-4520): calleeName = RAW SOURCE TEXT of the callee node (UTF-16
substring). Pinned shapes:
- bare
topFn(5)→topFn; sugar callsrequire 'x'/f {t}route the same (arguments without parens). - dotted
M.create(2)→M.create; deepcore.util.log("x")→core.util.log. - colon methods keep the COLON:
M:render({})→M:render;M.sub.deep:chained(13)→M.sub.deep:chained; in bodiesself:helperMethod(o)→self:helperMethod,self.field.deep(1)→self.field.deep—selfis NEVER stripped (no SKIP_RECEIVERS on this path). Resolution depends on these exact bytes (§Resolution consumers). - bracket calls:
M.registry[key](3)→M.registry[key];t2[k2](14)→t2[k2](brackets verbatim). - call-result callees:
f2()(15)→ outer reff2()+ inner reff2(children recursed — every chain link emits). - the newline-glue trap (lua's statement ambiguity): a call statement
followed by a line starting
(parses as ONE glued chain —obj:foo():bar()\n\t(helper)(4)emits FOUR refs:obj:foo():bar()\n\t(helper)(raw text with embedded newline/tab — byte-verbatim),obj:foo():bar(),obj:foo():bar,obj:foo(pinnedextract-bodies-lua.txtL19). Same withstring.format("%d", 9)\n("literal"):upper()→ three refs (extract-shapes-lua.txtL29). Reproduce byte-for-byte, including the glued middle links whose "callee" is a whole inner function_call's text. - paren-conversion regex (:4529-4532):
/^\(\s*\*?\s*([A-Za-z_][\w.]*)\s*\)$/applies — a clean single(handler)(16)statement (semicolon-separated or first-in-body, un-glued) → callee text(handler)→ rewrittenhandler(pinned twice: torture paren_conv; mini-probesparenFirst(cb)()→cb). A glued/quoted parenthesized callee (("x"):upper— pinned raw) does NOT match. Port the regex with JS\ssemantics (matches\r). - cpp template-strip (:4542-4548) and fn-ptr fan-out (:4556) are gated off.
- Ref: {fromNodeId = stack top, referenceName, referenceKind 'calls', line = call startRow+1, column = call startColumn (UTF-16)}.
- Luau extra shapes, all pinned: interpolation calls emit in bodies
(
`point {p.x} of {Config.total()}`→ callsConfig.totalat the inner call's position); if-expression arms (if p.x > 0 then bump() else drop()→ both);v += grow()→ calls grow (update_statement recursed).
Docstrings (tree-sitter-helpers.ts:95-127) — LIVE, with byte-level quirks
Both grammars have ONE comment kind, comment (line --, block --[[ ]],
LuaDoc ---, directives --!strict) — comment IS in the accepted sibling
set (:109-121) → docstrings work everywhere getPrecedingDocstring is called
(functions, methods, variables — every name of a multi-name decl —, luau
type aliases, imports never — the hook passes no docstring).
DOCSTRING_WRAPPER_TYPES (:55-62) contains variable_declaration but the
climb inspects PARENTS — a lua declaration's parent is chunk/block → no
climb, no effect. cleanCommentMarkers (:77-90), per comment, then
.join('\n').trim():
--[[ … ]]/--[==[ … ]==]→ the--[-open strip (:80,^--\[=*\[+\]=*\]$) — pinned "Block comment doc\nspanning two lines", "lvl".- line comments →
^--\s?per line (:85). LuaDoc---keeps a leading-(--- Summary→- Summary, pinned in both langs).--!strictbecomes docstring text!strictand JOINS the chain — pinned: torture luau'scoredocstring is"!strict\nheader comment for torture.luau". - Comment runs chain across blank lines (siblings skip whitespace);
the chain BREAKS at any non-comment named sibling — including line 1's
hash_bang_line, which simply terminates the scan (shebang + comment run → run kept, pinned). - CRLF bytes, pinned (
extract-torture-crlf-lua.txt): line-comment docstrings are IDENTICAL to LF (each comment.trim()ed →\rgone, joined with\n); block-comment docstrings KEEP interior\r\n("Block comment doc\r\nspanning two lines") — only the ends are trimmed. Signatures keep raw\r\ntoo (= function(v)\r\n\treturn …). Allgmstrips ridejs_multiline_stripin docstring.rs (#1329) — the lua strips (lua_open/lua_close:49-50,dashes:56) are already in docstring.rs — call the shared module, port nothing.
Function-as-value capture (#756) — LUA_SPEC (function-ref.ts:322-330)
idTypes = {identifier} (bare identifiers only — dotted values never
qualify: { on_make = M.make } captured NOTHING, pinned). dispatch:
arguments → args; assignment_statement → rhs with NO field (RHS = LAST
named child = the expression_list; the param-storage skip :435-443 reads
namedChild(0) = the variable_list, takes its trailing identifier, and
compares to the WHOLE rhs text — M.cb = cb skips (pinned, no candidate);
multi-value RHS never matches); field → value via the value: field
(keyed AND positional table fields — { k = v } and { v1 } both carry
value:, truth-tabled). layers: expression_list → null (fan out). No
special, no unwrap, no ungatedModes, no addressOfOnly.
- Capture points: visitNode:990 (top-level call args + assignments + every
fieldnode the ladder recurses through), visitFunctionBody:5137, and scanFnRefSubtree (hook-consumed require calls + variable_declaration subtrees). scanFnRefSubtree's halt list (:606-614) does NOT includefunction_definition→ the scan DESCENDS into anonymous-function initializer bodies (depth cap 12), so args/fields inside them still produce candidates attributed to the FILE. - Flush (:639-728): definedHere = same-file function/method NAMES;
importedNames — lua module paths are dotted, so QUALIFIED_IMPORT (:665)
contributes the LAST segment (
app.core→core;two.a→a) and SIMPLE_NAME passes Roblox leaves (Signal,Child) — pinned:(helper)glue candidate flushed because importhelperexisted. Nothis./::shapes ever occur → the bare-name gate is the only live path. Dedupe${fromNodeId}|${name}— first occurrence's position wins (pinned: threetopFntable-value candidates → ONE function_ref at the first's line). Emitted refs: {referenceKind:'function_ref'} → wire code 200. - Pinned population (torture.lua):
pcall(topFn, 7, 8)→ topFn (args mode);{ on_start = topFn, on_stop = localFn }+{ cb = topFn, [1] = localFn, nested = { deep_cb = topFn } }→ topFn + localFn once each (field mode + dedupe);skipped = missinggated out;M.cb = cbskipped. Luau:{ plain = typedTop }→ typedTop;M:update(…, print)→ print gated out (not defined/imported).
visitFunctionBody (5129-5286) — the body walker rows
- maybeCaptureFnRefs (5137) per node. No visitNode hook → the require asymmetry (§Require hook).
function_call∈ callTypes → extractCall (5143); no return → children recursed (chains emit every link).- INSTANTIATION_KINDS (5145) never; extractBareCall (5159) absent; cpp declaration/assignment branches (5184-5215) gated off.
- extractStaticMemberRef (5218) — dead on entry (§Dead machinery).
variable_declaratortype-annotation branch (5230) — no such node kind.- Nested
function_declaration(5245) → extractName ≠<anonymous>always → extractFunction → receiver check:local function inner→ function under the enclosing symbol (render::inner);function M.attached→ extractMethod with QNM::attached; globalfunction leakedGlobal→ functionrender::leakedGlobal(all pinned). - classTypes/structTypes/enumTypes/interfaceTypes (5255-5275) — all empty.
variable_declarationin a body: NO branch → plain recursion → the initializer IS walked: bodylocal lazy = require("app.lazy")→ calls require; bodylocal x = topFn(9)→ calls topFn — the INVERSE of top level. No variable node minted for body locals (pinned).type_definitionin a luau body: NO branch → recursed → no alias node; a typeof-require inside it would emitcalls require(hook-less).- Everything else (if/while/repeat/for, return_statement, update_statement, interpolation, cast_expression, parenthesized_expression) is transparent recursion.
Dead machinery — early-outs the walker must reproduce as SILENCE
- Value-reference edges:
VALUE_REF_LANGS(:401) has no lua/luau → flushValueRefs (:777-784) discards everything captureValueRefScope collected. Zeroreferencesedges withvalueRefmetadata. The assignment shadow-prune (:803-878) never runs — itsassignmentcase (:829) is Python's node kind, not lua'sassignment_statement, so even a future opt-in would no-op. (Trap (a) from the survey brief: RESOLVED — structurally dead.) - Static-member refs: STATIC_MEMBER_LANGS (:345-347) excludes lua/luau → extractStaticMemberRef (:4750-4751) returns on entry; MEMBER_ACCESS_TYPES (:323-331) has no lua kinds anyway. Zero.
- Type-annotation refs: TYPE_ANNOTATION_LANGUAGES (:5752-5754) excludes
lua/luau → extractTypeAnnotations (:5788-5790) early-outs; luau parameter
types (
parameter > identifier + type— a node shape lua lacks: lua params are barename: identifierchildren) and return types emit NOTHING; type_alias values emit nothing (§Luau type aliases). Zeroreferencesrefs of any type-flavored kind. - Instantiates: INSTANTIATION_KINDS has no lua kinds; the ladder and
body walker never call extractInstantiation (:4610). Constructor-ish
calls (
setmetatable,M.create(2)) are plaincalls. - Inheritance: extractClass/extractInterface/extractEnum unreachable
(empty type lists) → extractInheritance (:5291) never runs.
setmetatablemetatable-class patterns emit ONLY the calls ref — no class synthesis, no extends (pinned:M.create's body → calls setmetatable, nothing else). - Decorates: extractDecoratorsFor (:4897-5024) runs for every
function/method but no decorator/annotation/attribute node kind ever
appears in the positions it scans (lua's
attribute=<const>/<close>lives inside variable_list — pinned:local x <const> = 99extracts variable x, signature= 99, no ref; luau's real@native/@checkedattributes are grammar ERRORS → deferred files). Zerodecoratesrefs. - file-level
errors: none on clean parses (wasm-arm hasError files add their own — kernel defers those wholesale).
Resolution consumers (TS-side, no port — but they pin the walker's BYTES)
resolveLuaRequire(import-resolver.ts:1637-1671, dispatch :1450-1457) resolves eachimportsref by treating the referenceName as a dotted path (telescope.config→telescope/config.lua) or Roblox leaf (Signal→Signal.luau), trying<p>.lua|.luau|/init.lua|/init.luauas PATH SUFFIXES, longest-shared-prefix-with-the-requiring-file wins, → file-node target at confidence 0.9. Walker obligation: referenceName = the module string/leaf VERBATIM (dots intact, no normalization).- Lua colon-method resolution: extraction emits
lg:log-shaped calls; name-matcher'sluaColonMatch(:1514-1520,/^([\w.]+):(\w+)$/) splits them for local-variable receiver-type inference (#1108), whose lua patterns (:1218-1234) include the #1124 lookahead — the third pattern\b<r>\b\s*:\s*([A-Z][\w.]*)(?![\w.]|\s*[({"'\[])rejects PascalCase METHOD CALLS (lg:Log(), the Roblox convention) as fake "type annotations" because Lua's call syntax is the identicalreceiver:Nameshape. The pre-filter (resolution/index.ts:722-736) keeps single-:refs alive when either side (or the capitalized receiver) names a known symbol. Walker obligation: the SINGLE-COLON byte shape (and[\w.]-only receivers — brackets/glue garbage simply never resolves). - Nothing else consumes lua/luau specially (no framework resolvers, no synthesizers — §arch-3).
Parity mechanics (all have bitten before)
- Emission order per construct, pinned across all dumps: file node →
source-order walk. Per require-decl: import node → imports ref happens
inside the hook, THEN extractVariable's variable nodes (
import:two.a,import:two.b,variable:twoA,variable:twoB— hook loop completes first). Contains edges interleave with creation (createNode). Walk-order refs (imports + calls interleaved by source position), then function_ref refs at flush (:538). No value-ref edges. Store/harness are rowid-order-sensitive — reproduce exactly. - Wire flags (layout.ts:100-106 bit pairs): lua function/method/import/
type_alias-less world — the ONLY flag ever set for lua is variable
isExported=false(present=1, value=0). Luau ADDITIONALLY sets isExported on functions (false) and type_aliases (true forexport type, false otherwise) — but NOT on methods (extractMethod passes none). lua functions: present-bit 0 (undefined). Get the present-vs-value distinction byte-right; it is the one lua↔luau node-payload divergence besides signatures/typeAliases. - visibility byte 0 everywhere; returnType/decorators/typeParameters absent everywhere.
- UTF-16 columns/slices (textutil::col16/slice_utf16): every position
and every getNodeText (callee raw text, signatures, docstring sources,
import signatures +
.trim()). Pinned:local préfixe→ C6-13 (7 UTF-16 units). - CRLF: §Docstrings byte pins; callee raw text and signatures carry
\r\nverbatim; the paren-conversion regex's\s*eats\r(JS ⊇ Rust\s— parity holds). CRLF fixture variants derived in-memory (kernel-tsjs-parity pattern). - Defer policy:
has_error()→defer:(no phantom class observed — but trust the flag anyway, never node-scan). Deferral expectations §arch-7. - generateNodeId inputs: variables hash the IDENTIFIER's line; imports
the CALL's line; functions/methods the declaration's line; luau
type_aliases the type_definition's line (starts at
exportwhen exported). Duplicate-id rows are emitted, not deduped.
Gates (per plan §5 — NO standalone bump gate; grammar-parity rows instead)
- Stage 0 (with or before the walker): kernel C vendor
(grammars/lua, shas above) + build.rs block + langs.rs entries (+ crate
pin tree-sitter-luau "=1.2.0") +
GRAMMAR_LANGUAGES += 'lua','luau'in kernel-grammar-parity — proves C/crate ≡ vendored wasm revision (ABI 15 / ABI 14, kind+field tables id-by-id). No dump-diff needed: the wasm arm is untouched. - Torture fixtures per §Fixtures below, in a new
__tests__/kernel-lua-parity.test.ts(or kernel-lua-luau-parity) with CRLF variants; full suite ×2 green withCODEGRAPH_KERNEL_EXPECT=1. - Parity sweeps (
scripts/kernel-parity.mjs <dir>, default--max-deferral 0.1):…/svy-lua/gate-repos/kong(large lua, 1,309 files — expect ≤1 deferral: the invalid fixture)…/gate-repos/lazy.nvim(small lua, 65 — expect 0)…/gate-repos/lua-resty-core(small lua, 38, LuaJIT/OpenResty idioms — expect 0)…/gate-repos/lune(luau, 221 — expect ~3 deferrals)…/gate-repos/Fusion(luau, 113, heavily typed — expect ~8 deferrals)- luau-lang/luau tests/ + StyLua tests/ as OPTIONAL stress arms only
(deliberate-error corpora — run with
--max-deferral 0.3/0.5, judge only the non-deferred parity). (Re-clone public OSS fresh if the scratchpad is gone — agent-eval policy.) Then full-init dump-diffs byte-identical (kernel arm vsCODEGRAPH_KERNEL=0,scripts/dump-graph.mjs, cmp) on kong + lazy.nvim + lune + Fusion.
DEFAULT_ROUTED += 'lua', 'luau'(kernel/index.ts:37) only after all of the above; changelog rides the existing kernel entry.
Fixtures to build
torture.lua(survey seed + pinned dumpextract-torture-lua.txt— 99 lines, parse-clean). Inventory: shebang + comment-run docstring; require forms — double-quote, Roblox instance path, single-quote,[[ ]],script:WaitForChild("Child")(string-wins),.field-access (NO import), dynamic (NO import), two-in-one-decl (order), pcall-require (nothing); LuaDoc---(leading-quirk); block-comment doc; variable-with-invisible-anon-initializer;function t.f/t:m/a.b.c:m/_G.freceiver QNs; body: require→calls-require, self-colon and self-dot calls, bracket-callee calls, dotted-deep calls, nested local/receiver/global functions (QN quirks),M.assigned = function(file-attributed body calls); table fn-ref registries incl. positional[1] =keys + nested tables + dedupe +missinggate-out +M.cb = cbparam-storage skip; global-assignment call visibility vs local-decl invisibility; call shapes — bare/dotted/colon/bracket/f2()()/(handler)(16)conv-rewrite (the survey torture keeps every statement un-glued — verified: each L77-83 call emits its own single-line ref; the newline-GLUE chain is pinned separately, fixture 5); barelocal x(no signature key — probedbarelocal.lua); one-line multi-statement (local one = 1 local two = 2 print(...)); long strings incl.]==];<const>attribute; non-ASCII line + following symbol (UTF-16 columns); goto/label;return M.torture.luau(seed +extract-torture-luau.txt— 62 lines, parse-clean):--!strictdocstring-joining; requires (string + Roblox); type aliases — plain (doc),export type(isExported=true, LuaDoc), generic (Generic<T>verbatim name),typeof(require(...))(alias+import pair); typed/generic/multi-ret/empty-one-liner function signatures; typed methods (no isExported); body: type-in-body (nothing), require→calls, interpolation call, if-expr arms,+=RHS call, cast,continue; registry table (dotted value dropped, bare captured); unicode line.- CRLF variants of both derived in-memory (kernel-tsjs-parity
pattern) — pin the block-comment
\r\ndocstring byte and CRLF signatures. - Defer fixtures: one lua file with luau syntax (
x += 1) and one luau file with a default type param (type S<T = U> = {}) — kerneldefer:s, wasm-arm output asserted as-is (partial extraction). - Glue fixture (if not folded into torture.lua): call statement +
next-line
(-open statement → the 3-4-ref garbage chain, byte-pinned. - Duplicate-id fixture:
local x = 1; local x = 2one-liner (two identical-id variable rows).
Probe artifacts (session scratchpad svy-lua/)
cst-dump.cjs (CST dumps with fieldNameForNamedChild labels; --all mode
walks anon children via cursor) + cst-shapes-{lua,luau}.txt;
field-truth.cjs + field-truth.out (childForFieldName truth table — the
NULL function field, name/table/field/method/parameters/body fields, luau
generic_type name, positional-field value:); extract-probe.cjs (runs the
REAL dist TreeSitterExtractor; prints nodes/edges/refs in emission order with
every wire-relevant field) + ground truths extract-shapes-lua.txt,
extract-shapes-luau.txt, extract-bodies-{lua,luau}.txt,
extract-requires-lua.txt, extract-condreq (inline),
extract-torture-{lua,luau}.txt, extract-torture-crlf-{lua,luau}.txt;
mini-probes.cjs + mini-probes-{lua,luau}.out (31 snippet probes:
one-liners, same-line duplicate ids, long strings/comments, LuaDoc, shebang,
goto, paren-callee conv, bracket/call-call/paren-string callees, numeric/
generic for, varargs, CRLF, integer-div/bitops, cross-dialect rejects,
unicode, luau type/interp/continue/compound/if-expr/cast/directive/
@native-reject probes); errclass{1-4}.luau (error classification);
error-sweep.cjs + errors-{lua,luau}.txt (per-repo incidence + phantom
scan); fixtures shapes.lua, shapes.luau, bodies.lua, bodies.luau,
requires.lua, condreq.lua, sigs.luau, generic.luau,
torture.{lua,luau} + CRLF twins; grammar/ — tree-sitter-lua (v0.4.1
tag clone), tree-sitter-luau (v1.2.0 tag clone), luau-crate/
(crates.io 1.2.0 tarball, sha-matched), shas in §Grammar prep;
gate-repos/ — kong, lazy.nvim, lua-resty-core, StyLua, luau, Fusion, lune
(shallow clones). Scratch dirs are throwaway — re-derive from this doc if
gone.