Squash danusha2345's PR #1511 at d282f9e8 onto main 8c9c4761,
preserving its nine non-merge commits and main's existing Unreleased notes.
Calls in Kotlin, Java, TS/JS, Scala, Rust and Python declaration initializers
now retain the owner established by the upstream regression expectations.
Include the upstream CFML, dynamic-dispatch summary and viewer follow-ups.
Linux fail-to-pass validation (Node 22.19.0, rebuilt dist and native kernel):
- Before: TS load belonged to file:app.ts; Python/Kotlin/Scala/Rust calls
vanished; Java lost the field-lambda, anonymous override and eager calls.
- After: all six languages PASS; 12 native/WASM LF/CRLF parity checks PASS.
- Focused initializer regressions: 10 passed with CODEGRAPH_KERNEL=0 and
10 passed with the kernel enabled; Kotlin's grammar fallback is recorded.
- Related regression suites: 879 passed, 1 skipped across 15 test files.
- Evidence: /workspace/cg-1510-repro/before and /workspace/cg-1510-repro/after
(combined test output: after/vitest.log).
Fixes #1510
Supersedes #1511
Co-authored-by: Colby McHenry <colbymchenry@users.noreply.github.com>
Co-authored-by: danusha2345 <ewidusoc498@gmail.com>
75 KiB
Kotlin kernel port (R7b) — the bug-for-bug checklist
Status: PORT COMPLETE (2026-07-20) — walker codegraph-kernel/src/kotlin.rs
- the vendored-grammar-C build (codegraph-kernel/grammars/kotlin via build.rs
cc — the mechanism's first use), all gates passed (bump dumps byte-identical
old-vs-new ×3 as predicted; parity sweeps 0-diff okio 299/322 / okhttp
531/580 / kotlinx.coroutines 1031/1082 with exactly the predicted 23/49/51
deferrals; kernel-arm dumps byte-identical ×3; KMP expect/actual synthesis
IDENTICAL — 412 edges both arms; kernel-kotlin-parity suite; DEFAULT_ROUTED
+= kotlin — 15 languages). One fixture note: the survey's torture.kt itself
tripped the PHANTOM-error class (one-line class/object bodies) and deferred —
the checked-in parity fixture reflows those to multi-line, and the phantom
shape is pinned in the defer test instead. Survey basis: every
TS-side branch a
.kt/.ktsfile exercises, with file:line anchors as ofa6c62d7(HEAD at survey time, clean main). Every grammar-shape claim below was probed against both the production tree-sitter-wasms build and a fresh fwcd 0.3.8 tag build (probe scripts + dumps in the session scratchpadsvy-kotlin/— see §Probe artifacts), and every extraction-behavior claim was pinned against the realdist/extractor (extract-*.txtground-truth dumps), not derived from code reading alone. Read WITHdocs/design/rust-kernel-migration-plan.md(§0a recipe, §2 boundary, §4 tracker row "kotlin", §5 gates) and the format precedents (rust-lang-kernel-port-checklist.md,ruby-kernel-port-checklist.md,php-kernel-port-checklist.md,csharp-kernel-port-checklist.md).
Blocking findings: none — but two eyes-open items, one of them a NOVEL
mechanism. (1) The grammar bump is behavior-neutral (rust-style gate:
byte-identical CSTs on all 1,984 gate-repo files, 0 error disagreements) —
but the crates.io crate tree-sitter-kotlin = 0.3.8 is UNUSABLE by the
kernel (it pins tree-sitter >= 0.21, < 0.23; the kernel links 0.25), and
the successor crate tree-sitter-kotlin-ng is a different grammar (8
fields vs 0, 289 vs 357 symbols, renamed kinds — would break every kotlin.ts
branch). The port must take the vendored-grammar-C route: compile the
sha-matched 0.3.8 parser.c+scanner.c inside codegraph-kernel via
build.rs — the FIRST language to exercise the mechanism the §4 tracker
prescribes for vendored grammars (§Grammar prep). (2) Both-arm parse-error
incidence is 4.7–8.5% on the gate repos (fun-interface misparses, phantom
single-line-class-body errors, soft-keyword identifiers, call().prop = x
LHS shapes — all grammar-inherent, all identical across arms). The default
--max-deferral 0.1 HOLDS but with only ~1.2–2× headroom (okhttp 8.45%) —
expect double-digit deferral COUNTS on kotlin sweeps and don't misread them
as walker bugs (§Architecture decisions #6).
Grammar prep (behavior-neutral re-vendor + the vendored-C kernel build)
kotlin is NOT in VENDORED_WASM_LANGS (grammars.ts:291-317) — production
loads node_modules/tree-sitter-wasms/out/tree-sitter-kotlin.wasm
(mapping kotlin: 'tree-sitter-kotlin.wasm' at grammars.ts:35;
tree-sitter-wasms 0.1.13 builds it from npm tree-sitter-kotlin ^0.3.1,
the fwcd lineage; production wasm sha256 b5cb00c8…, 4,052,705 bytes, ABI 14).
-
Lineage decision (investigated, not assumed): two crates exist.
tree-sitter-kotlin0.3.8 (crates.io max_stable, published 2024-08-03; repo fwcd/tree-sitter-kotlin — OUR wasm's lineage; repo still active but no crate release since). Tag0.3.8(annotated tag9e7e624→ commite1a2d5ad1f61f5740677183cd4125bb071cd2f30); sha256-verified crate-tarball ↔ tag, BOTH generated artifacts (kotlin HAS an external scanner):src/parser.c54104a7ef1555c265b746c790e0f8bb953cc17806e9df0c3af82f7f62c06a70asrc/scanner.c27f73337ec357fc341fa57538f34c14277b0346980c3405dc30beab6202ec6d0
tree-sitter-kotlin-ng1.1.0 (tree-sitter-grammars org, 2025-01) — REJECTED: a different grammar, not a re-publish (STATE_COUNT 11432 vs 10155, SYMBOL_COUNT 289 vs 357, FIELD_COUNT 8 vs 0, kinds renamed —additive_expression→binary_expression,call_suffixgone,binding_pattern_kindgone…). Adopting it is an extractor REWRITE, not a port. Do not revisit until/unless the TS side migrates grammars.
-
The 0.3.8 build is behavior-IDENTICAL to the production wasm (this bump is a reproducibility re-vendor, csharp-flavored, not a version change): kind/field tables identical (360 node types, 134 named kinds, 0 fields, ABI 14 both —
table-compare.cjs); the 258-line torture file's full CST dump is byte-identical OLD↔NEW; and the gate-repo sweep (error-sweep.cjs <repo> --sexp) found 0 error disagreements and 0 s-expression mismatches on every clean file across all three repos. Expect the standalone bump gate's old-vs-new full-init dump diff to be byte-identical on all three (rust-style "expect zero", NOT php's enumerate+classify). -
ABI note: the 0.3.8 tag's checked-in parser.c declares
LANGUAGE_VERSION 14— content parity, ABI stays 14 (ruby precedent). kernel-grammar-parity must assert same-revision, not an ABI change. -
FIELD_COUNT 0 is load-bearing for the whole port: every
childForFieldName/getChildByFieldlookup in the kotlin path returns null, which is what makes several TS hooks dead code (§Extractor config). The walker must reproduce the null-field world exactly — do NOT "helpfully" use -ng-style fields that don't exist here. -
Wasm build (from the tag's CHECKED-IN parser.c — never
tree-sitter generate):git clone --depth 1 --branch 0.3.8 https://github.com/fwcd/tree-sitter-kotlin cd tree-sitter-kotlin # the 0.3.8 tag predates tree-sitter.json, which cli 0.25.10 requires — # add the METADATA-ONLY shim (grammar name/scope; nothing regenerated): # {"grammars":[{"name":"kotlin","scope":"source.kotlin","path":".", # "file-types":["kt","kts"]}],"metadata":{"version":"0.3.8","license":"MIT"}} npx -y tree-sitter-cli@0.25.10 build --wasm -o tree-sitter-kotlin.wasm .(brew emcc present; survey artifact sha256
c80c88867a589a1a0959bcea89de84b7e9684b3693b2cdb2944812458e62ff48, 4,052,313 bytes, at scratchpadsvy-kotlin/tree-sitter-kotlin-NEW.wasm. Do NOT use tree-sitter-cli 0.24 — it drops\p{...}classes, the #1164 vbnet lesson, and this grammar's identifiers use them.) -
Kernel side — the NOVEL part (crate pin impossible): the 0.3.8 crate's
[dependencies.tree-sitter] version = ">= 0.21, < 0.23"+ old-stylepub fn language() -> tree_sitter::Languagebindings cannot link against the kernel'stree-sitter = "0.25". Instead of a crate dep, vendor the grammar C into the kernel (the §4 tracker's prescription for vendored-grammar languages — kotlin is the first to need it):- copy the tag's
src/parser.c,src/scanner.c, andsrc/tree_sitter/*.htocodegraph-kernel/grammars/kotlin/(shas above, recorded in a comment); codegraph-kernel/build.rs:cc::Buildcompiling both C files with the crate's own flag set (-Wno-unused-parameter,-Wno-unused-but-set-variable,-Wno-trigraphs; msvc-utf-8— crib the tarball'sbindings/rust/build.rs);- Cargo: add
tree-sitter-language = "0.1"(the version-agnosticLanguageFnshim every modern grammar crate uses) +ccas a build-dependency (if not already present); langs.rs:plusextern "C" { fn tree_sitter_kotlin() -> *const (); } // … "kotlin" => Some(unsafe { tree_sitter_language::LanguageFn::from_raw(tree_sitter_kotlin) }.into()),LANGUAGES+="kotlin"(14 entries).__tests__/kernel-grammar-parity.test.ts:39GRAMMAR_LANGUAGES += 'kotlin'— the id-by-id ABI/kind/field-table compare against the vendored wasm is the proof the C build and the wasm build are the same revision.
- copy the tag's
-
Staging plan (bump PR, before any walker exists): vendor the wasm to
src/extraction/wasm/tree-sitter-kotlin.wasm;VENDORED_WASM_LANGS += 'kotlin'(grammars.ts:291) with an R7b comment (tag + sha-matched note + "crate unusable — kernel compiles vendored C, see codegraph-kernel/grammars/ kotlin"); the kernel C vendor + build.rs + langs.rs + grammar-parity row can land WITH the bump (they're inert until a walker exists) or with the walker — but wasm + C must be same-tag from day one.copy-assetsalready globssrc/extraction/wasm/*.wasm. MIT license (fwcd), same family as the rest. -
Error incidence (both arms, all
.kt/.kts≤1MiB,error-sweep.cjs):Repo files OLD hasError NEW hasError disagreements sexp mismatches (clean files) okio 322 23 (7.14%) 23 (7.14%) 0 0 okhttp 580 49 (8.45%) 49 (8.45%) 0 0 kotlinx.coroutines 1,082 51 (4.71%) 51 (4.71%) 0 0 Error classes (sampled + probed): (a)
fun interface— unsupported by the grammar, ALWAYS errors (okhttp 10/49, kotlinx 6/51; okhttp3's coreCall.kt/Authenticator.kt/Dns.ktare in this class); (b) phantom single-line class bodies —class X { fun f() {} }setshasError=truewith ZERO ERROR/missing nodes and a COMPLETE, correct CST (probed both arms; okio 1, okhttp 7, kotlinx 6); (c) soft-keyword identifiers (var final = false; final = trueerrors —finalis reserved by the grammar); (d)call("x").prop = valuenavigation-off-call assignment LHS (errors; plainobj.prop = xis fine); (e) assorted expect-header and gradle-kts DSL shapes.class Foo private constructor(x)and@Inject constructorparse CLEAN (probed — don't blame ctor visibility). All classes error on BOTH arms → defer-to-wasm keeps parity; only the speedup is lost on those files. -
Probe scripts + outputs live in the survey scratchpad (
…/scratchpad/ svy-kotlin/):table-compare.cjs,shape-probe-kotlin.cjs+torture-{OLD,NEW}.txt(byte-identical),mini-probes.cjs+mini-probes.out(17 targeted shapes, all OLD==NEW),error-sweep.cjs+errors-<repo>.txt,extract-probe.cjs(runs the REAL dist extractor — itsextract-{torture,vref,vref-nopkg,docs,bodiless,crlf,funiface,lfpkg,kts}.txtdumps are the pinned ground truth cited throughout and double as walker test expectations),torture.kt+ the small fixtures, the 0.3.8 tag clone + crate tarball + ng tarball with matching shas. Scratch is throwaway — re-derive from this doc if gone.
Architecture decisions
- No preParse.
kotlinExtractorhas nopreParsehook (languages/ kotlin.ts — whole file, no such key), sopreParsedSource(kernel/index.ts:96) is a no-op — both arms parse raw bytes. Nothing to hoist. NoPOST_PASSESentry either (kernel/index.ts:81) →tryKernelExtractRawstays eligible. - Three framework resolvers can force the DECODED path for kotlin; none of
the gate repos trips any of them (verified). parse-worker.ts:93-100
forces any language with an applicable framework
extract()onto the decodedextractFromSourcepath. Kotlin appears in:springResolver(frameworks/java.ts:13,languages: ['java','kotlin', 'yaml','properties']; extract() at :197 regexes@GetMappingetc. over raw.ktsource) — detect (:23) = pom.xml/build.gradle/build.gradle.kts containingspring-boot/springframework, or Spring annotations in any.javafile. okio/okhttp/kotlinx.coroutines: none match (grepped).expoModulesResolver(frameworks/expo-modules.ts:154,languages: ['swift','kotlin']) — detect = package.jsonexpo-modules-coreor an ExpoModuleDSL source scan. Not present.fabricViewResolver(frameworks/fabric.ts:366, kotlin in languages) — detect needscodegenNativeComponent. Not present. So all three parity repos exercise the raw buffers-to-store transport; a Spring-Boot Kotlin app or an Expo/RN app is the decoded-path smoke check.
- The framework extractors themselves need NO port — regex over raw source, run in extractFromSource:6736-6758 after either arm. §Frameworks pins their input contracts.
- One walker module (suggest
codegraph-kernel/src/kotlin.rs; no crate collision since there is no kotlin crate dep), registered in langs.rs; per-filehas_error()→defer:like every walker. java.rs is the closest crib (JVM package→namespace node viaextractFilePackage, class-like scope stack, methods-in-class-like, dotted imports, STATIC_MEMBER + TYPE_ANNOTATION + VALUE_REF membership, annotation decorators,node_idsvec). Kotlin diverges from it in nine places, each detailed below: (a) avisitNodehook whose PROPERTY branch is the only live-in-kernel part (fun-interface recovery is defer-shielded); (b)getReceiverType— extension functions → receiver-qualified method QNs + the owner-contains fallback (NO ported walker has this surface yet); (c)extractModifiers— expect/actual → node DECORATORS (also a first); (d)resolveBodyby TYPE (zero-field grammar); (e)extraClassNodeTypes(object_declaration); (f) classifyClassNode keyword sniffing (interface/enum reuseclass_declaration); (g) the #750 kotlin re-encode in extractCall (namedChild(0), NOT a function field); (h) a fn-ref spec with EMPTY idTypes +callable_reference/navigation_expressionspecials; (i) dead-field lookups everywhere (signatures, type annotations) that must stay dead. .kt/.kts→kotlinat detectLanguage (grammars.ts:106-107), no content sniffing, no dialect..ktsscripts are ordinary kotlin files whose top-level statements attribute calls to the FILE node (pinned:extract-kts.txt—calls println from=file, top-levelval→constant). MAX_FILE_SIZE (1 MiB, extraction/index.ts:132) and generated-file skips are orchestrator/TS-side and shared.- Deferral expectations: okio 23/322 = 7.14%, okhttp 49/580 = 8.45%,
kotlinx.coroutines 51/1,082 = 4.71% — grammar-inherent, both-arm (§Grammar
prep table). Keep the sweep default
--max-deferral 0.1(it holds on all three) but EXPECT these counts; a kotlin sweep at ~8% deferral is normal, one at >10% means a walker bug. No c/cpp 0.5 exemption. - REF_FLAG_FILE_PATH (wire v2 slot) is NOT needed for kotlin. No kotlin
extraction path emits refs carrying
filePath(the visitNode hook only creates nodes; verified across every ground-truth dump — zero refs printed a filePath). The ruby/php trait-mixin bit stays unused here.
Extractor config (languages/kotlin.ts — 353 lines, read it whole)
Types: functionTypes=[function_declaration]; classTypes=[class_declaration]
(covers class/interface/enum via classifyClassNode); methodTypes=
[function_declaration] (same list — the 994/995 gate routes in-class-like
functions to extractMethod); interfaceTypes=[] ; structTypes=[]; enumTypes=[];
enumMemberTypes=[enum_entry]; typeAliasTypes=[type_alias];
importTypes=[import_header]; callTypes=[call_expression];
variableTypes=[property_declaration]; fieldTypes=[property_declaration]
(both lists — the hook consumes nearly all of them first);
extraClassNodeTypes=[object_declaration]. nameField=simple_identifier,
bodyField=function_body, paramsField=function_value_parameters,
returnField=type.
FIELD_COUNT 0 consequences (the dead-field cluster — reproduce the deadness):
nameField/bodyField/paramsField/returnFieldare node-TYPE names used as FIELD names — everygetChildByFieldon them returns null. Names resolve via extractName's FALLBACK (first direct namedChild of typeidentifier|type_identifier|simple_identifier|constant, tree-sitter.ts:178-189); bodies resolve via theresolveBodyhook (by type); params/return field walks are DEAD (§Type-annotation refs).- getSignature (kotlin.ts:277) is DEAD CODE — always undefined. It reads
getChildByField(node, 'function_value_parameters')→ null → early return. Ground truth: every function/method inextract-torture.txthassig=undefined. The walker must emit NO signature for functions/methods. - The property hook's
typeNode = node.childForFieldName('type')(kotlin.ts:125) is DEAD → propertysignatureis always undefined too (val topVal: Int = 3→ sig undefined — pinned).
Hooks PRESENT (port each exactly):
- visitNode (kotlin.ts:87-215) — runs for EVERY node the main walker
visits (tree-sitter.ts:943-953; NOT in visitFunctionBody). Three branches:
property_declaration(:98-131) — the LIVE branch. varDecl = first namedChild of typevariable_declaration; nameNode = ITS firstsimple_identifier; either missing (destructuring'smulti_variable_declaration) → return false (fall to the ladder). Then the SCOPE WALK up the parent chain, first match wins:function_body|function_declaration|lambda_literal| anonymous_initializer|control_structure_body|getter|setter→ 'local' → return true, extract nothing (this is how init-block/getter-body/ top-level-control-flow locals reached via visitNode recursion are skipped);companion_object|object_declaration→ 'const';class_declaration→ 'instance'; nothing matches (top level) → 'const' (the initial value). Kind: instance →field; elseval(abinding_pattern_kindchild with text exactlyval) →constant,var→variable(const valis just a val; a delegatedby lazy {}property has no=but still a binding → same rule).ctx.createNode(kind, name, node, { signature: undefined })— extra carries ONLY the (always-undefined) signature: no docstring, no visibility, no isStatic, no returnType on kotlin property nodes — but createNode's extractModifiers merge still runs, soexpect val/actual valDO get decorators. Return true → the dispatcher runsscanFnRefSubtree(node, 0)(capture-only, halts at nested function/lambda types) and never descends on its own. The hook itself then walks the property's RHS under the property's scope — the named child after the=token plus aproperty_delegate— viactx.visitFunctionBody, soval SHARED = WidgetK(0),val cb = Runnable { hit() }andby lazy { compute() }all emit their calls FROM the property node (Go's #693 initializer walk, ported). The declaration's own children — modifiers,val/var, the name+type, an extension receiver's type and type parameters,getter/setter— are NOT walked: a same-lineval c get() = f()still emits nothing, a next-line accessor still attributes to the class, and a hook-DECLINED destructuring RHS is still invisible. Consequences pinned inextract-torture.txt.lambda_literalafter a fun-interface ERROR (:139-143) and- fun-interface misparse recovery (:145-214) (ERROR/
function_declaration shapes;
isFunInterfaceNode:46; Pattern 1 walks the sibling lambda'sstatementswith a synthesizedinterfacenode pushed — ground truthextract-funiface.txt: interface node at the ERROR's extent +transformas its method) — both branches are DEFER-SHIELDED in the kernel: everyfun interface(either pattern, probed) makes the treehasError=true, so the kernel defers the whole FILE to wasm before the walker would run. Do NOT port branches 2-3. Walker rule: port branch 1 only; adefer:on has_error covers the rest. (The parity suite still needs a fun-interface fixture asserting the kernel defers and the wasm arm serves the pinned output.)
- resolveBody (kotlin.ts:219-241) — find by TYPE among namedChildren:
first
ERRORchild whose child(0) is{(the fun-interface parent-body case — unreachable on non-erroring files, keep for wasm-parity of the TS side only), else firstfunction_body|class_body|enum_class_body. Used by extractFunction/Method/Class/Enum body resolution AND by createNode's endLine extension (tree-sitter.ts:1329-1333, function/method kinds only). Single-expression bodies (fun f() = expr) are afunction_bodystarting with=— resolved and walked like any body. - classifyClassNode (kotlin.ts:242-255) — scan ALL children (anon
included): child.type
interface→ 'interface';enum→ 'enum'; else 'class'.annotation class/data class/sealed class→ 'class' (theirclass_modifierchildren don't match);sealed interface→ 'interface'. - getReceiverType (kotlin.ts:256-276) — LIVE, the extension-function
surface. Walk ALL children in order: remember the last
user_type; on a.(anon) child WITH a remembered user_type → return that user_type's FIRSTtype_identifierchild's text (else the whole user_type text); onsimple_identifierorfunction_value_parameters→ break (past the name; no receiver). Probed shapes:fun WidgetK.extend()→WidgetK.fun <T> List<T>.genericExt()→List(type_parameters is skipped; generic args live in atype_argumentschild of the user_type, and the FIRST type_identifier is the base).fun com.example.Qualified.qext()→com— a qualified receiver's user_type holds MULTIPLE type_identifiers (com,example,Qualified) and the find takes the FIRST segment. QN becomescom::qext. BUG, PRESERVE.infix fun Int.pow()→Int;operator fun WidgetK.plus()→WidgetK(modifiers don't disturb the walk).
- getVisibility (kotlin.ts:288-301) — for each child of type
modifiers: TEXT.includes('public'|'private'|'protected'|'internal')in that order; no modifiers/no match → 'public'. QUIRKS, PRESERVE: (a) kotlin emits a visibility value no other language does — 'internal'; (b) the probe file'sprivate internal fun(invalid kotlin, parses fine) → 'private' (order); (c) TEXT-includes false positives — annotations live insidemodifiers, so@publicize-style lowercase annotation text containing a keyword flips visibility (e.g. a lone@internalApiannotation → 'internal' instead of 'public'). Match the includes-on-raw-text semantics exactly. - isStatic (kotlin.ts:302-305) — always false (not undefined): every
function/method node carries
isStatic: false. - isAsync (kotlin.ts:306-315) — any
modifierschild whose TEXT.includes('suspend')→ true, else false. The real shape ismodifiers > function_modifier > suspend. TEXT-includes false positive, PRESERVE (probed,mini-probes.outsuspendFalsePos):@suspendMarker fun g()→ isAsync true (the annotation text contains lowercase 'suspend'). - extractModifiers (kotlin.ts:316-338) — expect/actual, the KMP surface.
Scan children for
modifiers→ theirplatform_modifierchildren → their children of NODE TYPEexpect/actual(anon keyword nodes; matched by type, not text) → collect in order; empty → undefined. Runs inside createNode (tree-sitter.ts:1355-1358) for EVERY node kind — mergednewNode.decorators = [...(existing ?? []), ...mods]. Ground truth:expect fun/expect class→ dec=["expect"];actual fun/class/val→ ["actual"];actual typealias PlatformClock→ type_alias node with dec=["actual"] (the synthesizer's KMP_TYPE_KINDS depends on this); members of anexpect classare NOT marked (no platform_modifier of their own) but anactual funinside anactual classIS.decoratorson kotlin nodes come ONLY from this hook — the annotation channel isdecoratesREFS, never the node list (§Decorators). - extractImport (kotlin.ts:339-346) — signature = trimmed
source.substring(node.startIndex, node.endIndex)(UTF-16); moduleName = the first namedChild of typeidentifier's substring (the dotted path). No identifier → null (doesn't occur; evenimport a.b.*has the identifier). No handledRefs → the generic imports ref also fires (§Imports). - packageTypes=[
package_header] + extractPackage (kotlin.ts:347-352) — first namedChild of typeidentifier→ trimmed substring (com.example.torture); none → null. §Namespace capture.
Hooks ABSENT (the walker must NOT do these): preParse, resolveName,
recoverMangledName, isMisparsedFunction, isConst, isExported
(undefined on every node except the file node's literal false),
classifyMethodNode, extractPropertyName, propertyTypes,
interfaceKind (→ kind interface), extractBareCall, synthesizeMembers,
skipBodilessClass (a bodiless class Foo still mints a node — the
1685 comment names Kotlin as the deliberate case), methodsAreTopLevel,
resolveTypeAliasKind.
tree-sitter.ts branches (anchors as of a6c62d7)
visitNode dispatch — what each kotlin node hits (ladder at 936-1303)
| Node | Branch | Behavior |
|---|---|---|
| every node | visitNode hook first (943) | property_declarations (non-destructuring) consumed there; handled → scanFnRefSubtree + STOP |
| every node | maybeCaptureFnRefs (990) | fires for value_arguments/assignment (the KOTLIN_SPEC keys) in visitNode context too — how top-level/class-scope callable refs in call args are captured |
function_declaration |
functionTypes:994 | inside class-like AND ∈ methodTypes → extractMethod:1737; else extractFunction:1517 (which itself diverts to extractMethod when getReceiverType fires — extension fns at any scope). skipChildren |
class_declaration |
classTypes:1005 → classify | 'interface' → extractInterface:1834; 'enum' → extractEnum:1914; else extractClass:1679 |
object_declaration |
extraClassNodeTypes:1022 | extractClass(node) → kind class (objects and sealed-class object members are class nodes; extractInheritance runs → their delegation_specifiers emit extends) |
companion_object |
no branch | recursed → its class_body children visited with the OUTER CLASS still on top: properties → hook ('const' scope → constant/variable under the class, class: parent ⇒ value-ref targets), functions → extractMethod of the outer class. A NAMED companion (companion object Named) is identical — the name mints nothing |
property_declaration (hook-declined = destructuring) |
fieldTypes:1084 (in class-like) else variableTypes:1098 | extractField / extractVariable — both emit NOTHING for kotlin destructuring (no variable_declarator/variable_declaration/identifier direct children; extractVariable's generic fallback :2863-2881 finds no identifier-typed child — kotlin names are simple_identifier). skipChildren + scanFnRefSubtree → the RHS call is invisible too. isClassScopeConstantAssignment (1508) needs node.type assignment → always false for kotlin |
type_alias |
typeAliasTypes:1071 → extractTypeAlias:2890 | plain type_alias node (no resolveTypeAliasKind). QUIRK: the alias-value ref walk reads getChildByField(node,'value') → null (no fields) → NO reference to the aliased type; returns false → the alias's children ARE re-visited (harmless — user_type/modifiers match nothing) |
import_header |
importTypes:1209 → extractImport:3170 | §Imports (the import_list wrapper has no branch and recurses into each header) |
package_header |
consumed by extractFilePackage BEFORE the walk (1397) | during the walk it's recursed, nothing matches |
call_expression (top level / class body / object body / .kts statements) |
callTypes:1248 → extractCall:3684 | attributes to the nodeStack top (file/namespace/class). Note class-BODY calls only occur via init blocks etc. (below) |
anonymous_initializer (init { }) |
no branch | recursed → its statements' calls → calls refs FROM THE CLASS node; its val locals → hook 'local' → nothing (pinned: calls "register" from=class:WidgetK) |
secondary_constructor |
no branch | NO constructor node; recursed → body calls attribute to the CLASS (calls "log" from=class:WidgetK); the constructor_delegation_call's value_arguments still feed fn-ref capture |
getter/setter as SIBLINGS (accessor on its own line) |
no branch | recursed → accessor-body calls attribute to the CLASS (or file). See §Properties for the sibling/child split |
object_literal (object : T { … } initializer) |
no branch anywhere | never a node itself; inside a PROPERTY initializer the hook's walk reaches its funs, which leak out as FUNCTIONS under the property (see §Body walker for the same method-leak quirk) |
file_annotation (@file:JvmName("x")) |
no branch | recursed; its value_arguments feed fn-ref capture (string args → nothing). No decorates ref |
| INSTANTIATION_KINDS (354-361) | no kotlin member | extractInstantiation:4610 is UNREACHABLE for kotlin — constructor calls Foo() are call_expressions → plain calls refs named Foo (capitalized). Kotlin emits zero instantiates refs, ever |
impl_item:1274 / property_signature:1282 / export_statement / swift property:1121 |
never | not kotlin node kinds (the swift property_declaration branch at 1121-1193 is gated language === 'swift' — kotlin property_declarations never enter it) |
Node creation, IDs, qualified names
- createNode (1308): id =
generateNodeId(filePath, kind, name, startRow+1)=`${kind}:${sha256(`${filePath}:${kind}:${name}:${line}`).hex.slice(0,32)}`(tree-sitter-helpers.ts:18-30). FILE node id = literalfile:${filePath}(509), name = basename, qualifiedName = filePath, endLine =source.split('\n').length, isExported false. Dedupe/self-checks compare ID STRINGS (node_idsvec pattern). - endLine extension via resolveBody (1329) is LIVE for kotlin function/method nodes (body found by type; in-range for this grammar, so in practice a no-op extension — but CALL the hook, the ERROR-body branch is part of the contract).
- contains edge from nodeStack top for every created node (1363); extractModifiers merge (1355-1358); captureValueRefScope (1374).
- Namespace capture — extractFilePackage (1397): scan the ROOT's direct
namedChildren for the first
package_header(a leadingfile_annotationor KDoc is skipped by the type filter); extractPackage → dotted text →createNode('namespace', 'com.example.torture', pkgNode)= node #2 after the file node, pushed for the WHOLE walk. Every top-level symbol's qualifiedName =com.example.torture::Name(buildQualifiedName:1447 joins stack names with::; namespacePrefix always empty outside C/C++). No package header (scripts) → no namespace node, bare QNs, file: parents. - Receiver-QN override: extension methods get
extraProps.qualifiedName = composeReceiverQualifiedName(receiverType, name)(1790-1792) =`${receiverType}::${name}`verbatim (1435-1436; prefix empty → pass through) — NO package prefix:fun WidgetK.extendin package com.example.torture has QNWidgetK::extend(pinned). This is the first ported walker with the receiver-QN surface — get the two QN builders' divergence exactly right. - isInsideClassLikeNode (1486): stack-top node kind ∈ {class, struct,
interface, trait, enum, module} —
namespacedoes NOT count (top-level fns under the package namespace stay functions).
extractFunction / extractMethod (1517 / 1737)
- extractFunction: line 1522 — getReceiverType short-circuit is LIVE: any
function_declaration with a receiver (top-level extension fns, and nested
ones inside bodies) diverts to extractMethod. Name via extractName fallback
(first simple_identifier — backtick names keep their backticks:
function "`weird name`").<anonymous>unreachable (grammar requires the name). Extras: docstring (§Docstrings), signature undefined (dead hook), visibility (hook), isExported undefined, isAsync (hook), isStatic false, returnType (hook — §below). extractTypeAnnotations → emits NOTHING (§Type-annotation refs); extractDecoratorsFor → §Decorators. Push, body via resolveBody, visitFunctionBody, pop. - extractMethod (in-class-like functions + receiver-diverted extension fns):
receiverType recomputed (1742); gate 1747 passes via class-like OR
receiver. Same extras. Receiver path (extension fns): QN override
(1790) + the owner-contains fallback (1799-1813) — receiver present AND
not class-like → find the FIRST node in
this.nodeswithname === receiverType && filePath === this.filePath && kind ∈ {struct, class, enum, trait}→ contains edge owner→method. QUIRKS, PRESERVE:interfaceis NOT in the kind set —fun Drawable.ext()never gets an owner edge even with Drawable in-file; source-order dependent (extension above its class → no edge); the qualified-receiver bug (com::qext) looks up a node namedcom(never found). Extension fns keep their normal contains edge from the nodeStack top (namespace/file) REGARDLESS — the owner edge is additive. expect funhas no body → resolveBody null → no body walk; node still minted with dec=["expect"]. Interface bodiless methods likewise.- Nested named
funinside a body → visitFunctionBody:5245 → extractFunction → afunctionnode contained by the enclosing function/method (QN…::caller::localFn), receiver check applies (a nested extension fn becomes a method with owner-contains).
getReturnType = extractKotlinReturnType (kotlin.ts:17-43)
Positional (no fields): iterate namedChildren; before
function_value_parameters → skip; after it, the FIRST user_type |
nullable_type wins; hitting function_body or type_constraints first →
undefined. nullable_type unwraps to its inner user_type (?? child). Name =
the user_type's first type_identifier's text (?? the whole user_type),
trimmed; must match /^[A-Za-z_]\w*$/; Unit/Nothing
(KOTLIN_NON_CLASS_RETURN kotlin.ts:6) → undefined. Pinned: : WidgetK →
WidgetK; : WidgetK? → WidgetK; : Unit → undefined; inferred = expr →
undefined; : (Int) -> Unit (function_type) → undefined; : T (generic
param) → T (leaks as a returnType — preserve); extension receiver types
never mistaken (they sit BEFORE the params). Methods and functions both.
extractClass (1679) — and the bodiless-header asymmetry
resolvedBody = resolveBody (class_body by type; null for bodiless). NO
skipBodilessClass → bodiless classes mint nodes. Extras: docstring,
visibility (hook), isExported undefined; decorators via createNode's
extractModifiers (expect/actual classes). Then extractInheritance (§below) —
BEFORE the body walk, so extends refs precede member emissions. Then
extractCsharpPrimaryCtorParamRefs (no-op — needs language csharp… actually
gated at :5939 by language, cheap early-out) and extractDecoratorsFor
(§Decorators — the @MyMarker class ref). Push, walk, pop.
- Bodied class: body = class_body → ONLY class_body children are visited.
The
primary_constructoris a DIRECT child of class_declaration, NOT of class_body → constructor properties (class Foo(val a: Int)) mint NO field nodes, ctor default-value calls emit NOTHING, anddata classcomponents are invisible (pinned: DataK has zero members). - Bodiless class: body = the class node itself (1714) → the HEADER children
are visited: the primary_constructor recursion reaches default-value
call_expressions →callsrefs from the CLASS node, and delegation_specifier recursion reaches super-ctor argument calls too. Ground truth (extract-bodiless.txt):class Bodiless(val b: Int = initB()) : Base(readCfg2())→ extendsBase+ callsinitB+ callsreadCfg2, all from class:Bodiless; the IDENTICAL bodied class emits ONLY extendsBase. Reproduce the asymmetry exactly; also note the header's value_arguments feed fn-ref capture in the bodiless case only. - Class-body members: hook properties (fields/constants), function_
declarations → extractMethod, nested class/object/enum → their branches,
getter/settersiblings +anonymous_initializer+secondary_ constructor→ plain recursion (calls attribute to the class). - extractInterface (1834): kind
interface; docstring, isExported undefined — NO visibility (extractInterface never calls getVisibility; pinned vis=undefined). extractInheritance runs; body walk with the interface pushed (bodiless member funs still mint method nodes; aval propwith same-line getter → hook →field). Bodiless interface (sealed interface SealedIface) → body ?? node fallback (1856) → header children re-visited (nothing emits — but keep the traversal). - extractEnum (1914): body REQUIRED (resolveBody finds
enum_class_body) — a bodiless enum would mint nothing (doesn't occur). docstring, visibility, isExported undefined. extractInheritance (enum delegation_specifiers). Body loop (1941-1950):enum_entry∈ enumMemberTypes → extractEnumMembers(entry); everything else (function_declaration after the;, companion_object, secondary constructors) → visitNode with the enum pushed → methods of the enum, companion constants under it. - extractEnumMembers (1958):
getChildByField(node,'name')→ null (no fields) → the identifier-children scan (1967-1974): oneenum_membernode per directsimple_identifierchild, positioned at the IDENTIFIER node (createNode('enum_member', text, child)) — one per entry in practice. QUIRKS, PRESERVE: an entry'svalue_arguments(OK(200)) and an entry'sclass_body(OK(200) { override fun label() … }) are NEVER visited — the override methods and any calls inside them are COMPLETELY INVISIBLE (pinned: Http has enum_members OK/ERR + methods label/common/of only).
Properties — the hook rules + the getter-position split (probed)
- Same-line accessor (
val a: Int get() = compute()) → thegetteris a CHILD of property_declaration → the hook consumes everything → the getter body is never walked (no calls refs). - Next-line accessor (
val b: Int\n get() = compute()) → thegetteris a SIBLING (child of class_body / source_file) → after the hook handles the property, the walker visits the getter separately → its body's calls attribute to the CLASS (or file/namespace at top level). Pin BOTH variants. - Top-level properties: scope 'const' →
val→constant /var→variable, contained by the namespace (or file). Class body →field. companion/ object body → constant/variable under the CLASS node (stack top). Interface body →field(class_declaration parent matches 'instance'). - Body-context locals NEVER reach the hook (visitFunctionBody doesn't run
it) — instead they're plain-recursed: a local
val fn = { … }lambda initializer IS walked, so its inner calls attribute to the enclosing function (pinned:printlnfrom caller), unlike hook-consumed properties. Localval x: T = …type annotations emit nothing (§Type-annotation refs). lateinit var svc: Servicein a class → fieldsvc(modifiers don't matter to the hook).
Imports (3170-3236) — and the comment-gluing trap
Hook returns {moduleName: dotted path, signature: trimmed full text}: import
node (name = com.example.other.OtherClass; QN = namespace-prefixed) + the
generic imports ref (3183-3194): {fromNodeId: nodeStack top — the
namespace node when a package exists, else the file node, referenceName:
the dotted path, line: import_header startRow+1, column: startColumn}. NO
kotlin-specific emit pass (the 3197-3234 rust/php/ruby/python emitters are
all gated off). Shapes (probed):
import a.b.C→ identifier texta.b.C.- wildcard
import com.example.wild.*→ theidentifiercovers onlycom.example.wild(the.*is a siblingwildcard_import) → a normal import node/ref namedcom.example.wild(NOT null, unlike rust's wildcard). - alias
import a.b.LongName as Short→ identifier =a.b.LongName(theimport_aliaschild is ignored) — the SOURCE path, the alias binds nothing. - Comment-gluing (grammar quirk, LF and CRLF alike): comments FOLLOWING an
import (or the package header) attach INSIDE the import_header /
package_header node → the node's extent extends over them and the
hook's
signature(trimmed full text) INCLUDES the comment lines verbatim (pinned: sig"import com.example.alias.LongName as Short\n\n// line comment run 1\n// line comment run 2", node L7-10); the namespace node similarly spans to the last glued comment (extract-lfpkg.txt: namespace L1-4). Ref line/column stay at the header START. Downstream: those comments are NOT siblings of the next declaration → its docstring is LOST (§Docstrings). - flushFnRefCandidates' QUALIFIED_IMPORT (665) matches dotted paths → kotlin
imports contribute their LAST segment to importedNames (
OtherClass,helper,wild,LongName) — the fn-ref gate is "defined in this file ∪ imported simple names" (unlike rust/ruby).
extractCall (3684) — the kotlin paths
Entry: not vbnet/erlang/ruby/arkts. func = getChildByField(node,'function') ?? node.namedChild(0) (4313) → always namedChild(0) for kotlin (no fields).
The cpp operator recovery (4324) is language-gated off.
Member branch (4364) — func.type === navigation_expression:
- property (4369-4378): field lookups null →
child1 = func.namedChild(1);navigation_suffix→ its firstsimple_identifier(?? the suffix itself — unreachable in valid code: suffixes carry the identifier;::classsuffixes never appear under a call's function position). methodName = its text. Safe-call suffixes (?.) carry the same simple_identifier —x?.render()emits exactly likex.render(). - receiver = object/operand/argument fields (null) ??
func.namedChild(0)(4385-4389). - LITERAL_RECEIVER_TYPES (4397, set at 373-388): kotlin members that
occur:
string_literal("literal".uppercase()),integer_literal(5.toString()) → emit NOTHING (pinned). Kotlin's other literal kinds (boolean_literal,character_literal,null) also appear in the set — port the WHOLE set verbatim. - receiver
simple_identifier(4401; kotlin's name kind IS in the check list) not in SKIP_RECEIVERS {self,this,cls,super} (by TEXT — kotlin receivers are never those texts as simple_identifiers) →`${recv}.${method}`(w.render,Registry.register,Short.static,instances.add,it.render,anon.draw). - receiver
this_expression/super_expression→ none of the branches → fall to the ELSE → bare methodName (this.toString()→toString;super.hashCode()→hashCode). Same net effect as SKIP, different path. - receiver
call_expression+ kotlin in the gate list (4408-4418) — the #750 re-encode (4429-4442):innerNav = receiver.namedChild(0)(NOT a function field!) → its text with/\s+/gstripped; re-encode ONLY when/^[A-Z]/→`${innerCallee}().${methodName}`. Pinned:WidgetK.create().render()→WidgetK.create().render(+ the innerWidgetK.createfrom recursion);Foo.getInstance().bar()→Foo.getInstance().bar+Foo.getInstance;Foo().bar()would →Foo().bar; lowercase chains fall to bare:lowerFactory().chain()→chain+lowerFactory;w.chainInner().render()→render+w.chainInner;listOf(1).forEach {…}→forEach+listOf. - receiver
navigation_expression(2-hopa.b.method()),postfix_ expression(x!!.draw()), parenthesized, etc. → bare methodName.
Else branch (4518-4520) — calleeName = RAW func text: bare
helper/run/WidgetK (constructor calls are plain capitalized calls
refs — kotlin emits NO instantiates, §dispatch table); backticked
`weird name` verbatim; generic calls generic<Int>(1) → generic
(type_arguments live in the call_suffix, not the callee). QUIRKS, PRESERVE
(all pinned in extract-torture.txt):
- Paren-then-lambda
trailing() { it * 3 }parses as call(call(trailing,()), annotated_lambda) → TWO refs: the outer's callee is the inner call's RAW TEXTtrailing()(garbage, unresolvable) + the innertrailing. A no-paren trailing call (trailing { … },run { … }) is ONE call → one clean ref. - Newline-glued invoke chains:
fn(3)followed by a line starting(continues the expression (kotlin grammar) →fn(3)\n(fn)(4)is one 3-deep call chain emitting calleesfn(3)\n (fn)(raw text with embedded newline),fn(3), andfn. Deterministic garbage — reproduce byte-for-byte. - The parenthesized-conversion regex (4530,
/^\(\s*\*?\s*([A-Za-z_][\w.]*)\s*\)$/) applies to kotlin calleeNames — a clean single-line(handler)(4)(as the FIRST statement of a body, un-glued) has func text(handler)→ rewritten tohandler. Port the regex. - Template strip (4542) + cpp fn-ptr fan-out (4556) are c/cpp-gated — no.
- Final ref: {callerId = stack top, name, line = call startRow+1, column = call startColumn (UTF-16)}. Inner calls of every chain are ALSO visited (the body walker recurses after extractCall — no consumption).
- Calls inside string-template interpolations EMIT (in bodies):
"… ${w.render()} …"→ callsw.renderat the inner call's position (interpolated_expression recursion).$topVal(interpolated_identifier) emits nothing anywhere.
Static-member / value-read refs (4750-4808) — kotlin IS in STATIC_MEMBER_LANGS (345-347)
Called ONLY from the body walker (5218) — top-level/class-scope reads emit
nothing — EXCEPT a property initializer, which the hook now walks through
visitFunctionBody under the property's own scope (§Properties).
navigation_expression ∈ MEMBER_ACCESS_TYPES (326). Mechanics:
- callee-of-call skip (4772-4778): parent ∈ callTypes AND parent.namedChild(0)
starts at this node → skip (
Registry.register(w)'s nav emits no references ref). - recv = object/expression/scope fields (null) ?? namedChild(0); accepted
types include
simple_identifier(4792); text must match/^[A-Z][A-Za-z0-9_]*$/→referencesref at the RECEIVER's position (pushStaticMemberRef 4800). - Pinned:
Registry.count(statement) → referencesRegistry;Color.RED→ referencesColor;com.example.Fq.CONST_READ→ NOTHING (nested navs — the outer recv is a navigation_expression, not accepted; the innermost recvcomis lowercase);listOf(1).size→ nothing (recv is a call). Assignment WRITES emit nothing:Registry.count = 5/+= 1parse asassignment > directly_assignable_expression— that node type is NOT in MEMBER_ACCESS_TYPES (pinned: assignRefs() emits zero refs). - A
Foo.Barnav nested inside a bigger expression is visited on its own as the walker recurses — each nav node is evaluated once.
Decorators — kotlin annotations DO emit decorates (unlike csharp/php), asymmetrically
extractDecoratorsFor (4897) runs for functions/methods/classes (NOT
hook-created properties/fields/constants — the hook never calls it; pinned:
@field:JvmField val fielded → NO ref). Kotlin annotations live at
modifiers > annotation > … — scan #1 (4976-4987) descends into modifiers
children (the comment at 4979 names Kotlin) and consider() accepts node type
annotation (4928):
@JvmStatic/@MyMarker(no args) → annotation >user_type— user_type IS in the target list (4950) → name = its text (<-strip + last-./::-segment normalization at 4959-4962 apply) →decoratesref {from: the decorated node, name, line/col of the ANNOTATION node}. Pinned: decoratesMyMarkerfrom class Annotated,JvmStaticfrom method jvmStatic.@Deprecated("gone", ReplaceWith("new"))(with args) → annotation >constructor_invocation— NOTcall_expression, NOT in the identifier list → NO ref at all (and the argument expressions are never visited by anything — no calls refs either). Pinned: methodoldhas zero decorator refs.- Use-site targets
@field:JvmFieldon a FUNCTION/CLASS would emit (theuse_site_targetchild is skipped, the user_type matches) — but on properties (their usual home) the hook path never runs the extractor, so in practice they're silent. platform_modifierchildren of modifiers (expect/actual) → not accepted types → no decorates refs (they ride node.decorators instead).- The backward-sibling scan (5013-5022) is inert — kotlin annotations are
inside the declaration's
modifiers, never preceding siblings (file_annotation has no branch that reaches consider()).
Inheritance — delegation_specifier (5595-5615)
extractInheritance's child loop runs over the CLASS NODE's direct
namedChildren — kotlin's delegation_specifiers are direct children
(probed; no wrapper node). Per specifier:
- userType = find direct
user_type; ctorInv = find directconstructor_invocation; target = userType ?? ctorInv; none → skip. - typeId: user_type → its FIRST
type_identifier(?? itself); constructor_invocation → its user_type's FIRST type_identifier (?? the user_type ?? the invocation). - ONE
extendsref per specifier {name: typeId text, line/col: the typeId node}. Kotlin NEVER emitsimplements— interfaces ride extends too (pinned: SubK → extends OpenBase + Drawable + Comparable). - QUIRKS, PRESERVE: qualified supertype
: com.example.deep.RemoteBase()→ ref namedcom(first type_identifier of the multi-segment user_type — pinned); generic supertypeComparable<SubK>→Comparable(type_arguments' identifiers aren't direct children);by-delegation (: Drawable by d) emits NOTHING — the specifier's only child isexplicit_delegationand the direct-child finds miss (pinned: DelegatedImpl has zero extends). - Runs for classes, objects (extraClassNodeTypes → extractClass), interfaces,
enums.
object Add : SealedOp()→ extends SealedOp ✓. Anonymousobject_literals never reach it (no class node).
Type-annotation references — kotlin ∈ TYPE_ANNOTATION_LANGUAGES (5753) but the machinery is DEAD
extractTypeAnnotations (5788) for kotlin takes the GENERIC path: params =
getChildByField(node, 'function_value_parameters') (5844) → null (zero
fields); returnType = getChildByField(node, 'type') (5851) → null; the
type_annotation direct-child search (5873) → no such node kind in this
grammar. extractVariableTypeAnnotation (6074) needs a type_annotation
child → dead; the body-walker variable_declarator branch (5230) needs node
kind variable_declarator → kotlin has none. property_signature/
method_signature (1282) are TS-only kinds. Net: kotlin emits ZERO
type-annotation references refs — no param types, no return types, no
property types, no local types (pinned: torture has no such refs).
BUILTIN_TYPES is never consulted for kotlin. The walker needs cheap
early-outs that preserve exactly this nothing.
Docstrings (tree-sitter-helpers.ts:95-127) — KDoc is DROPPED
Kotlin comment node kinds: line_comment (//) and multiline_comment
(/* */ AND KDoc /** */). getPrecedingDocstring accepts only
{comment, line_comment, block_comment, documentation_comment} —
multiline_comment is NOT in the set → KDoc NEVER becomes a docstring,
and a KDoc sitting between a // run and the declaration BREAKS the chain
(it's a non-comment named sibling to the scan). Pinned
(extract-docs.txt): /** KDoc */ + // line one + // line two + fun →
doc = "line one\nline two"; KDoc-only → doc undefined; /** kdoc */ then
// trailing line then fun → "trailing line" only. DOCSTRING_WRAPPER_TYPES
(55-62) contains no kotlin kinds → no anchor climbing. cleanCommentMarkers
(77-90): only the ^\/\/[/!]?\s? per-line strip fires for kotlin line
comments — all gm strips ride js_multiline_strip in docstring.rs (#1329
CRLF semantics) — call the shared docstring.rs, port nothing. Properties
(hook-created) never get docstrings at all. The import/package
comment-gluing (§Imports) eats the docstring of the first declaration after
the import block — pinned: topLevel has doc=undefined despite two //
lines directly above it.
Value-reference edges (398-931) — kotlin IS in VALUE_REF_LANGS (401)
Port the full machinery (crib java.rs/go.rs): CODEGRAPH_VALUE_REFS=0 kill;
MAX_VALUE_REF_NODES = 20,000 caps the prune DFS and each reader scan;
isGeneratedFile skip.
- Targets (captureValueRefScope:735): kind constant|variable, name
length ≥3 AND
/[A-Z_]/(file_tablequalifies via_;countdoes not), parent id prefix ∈ {file:, class:, module:, struct:, enum:}. QUIRK (the php-namespace analogue, pinned): in ANY file with apackageheader, top-level properties' parent is thenamespace:node → NOT accepted → top-level kotlin constants are NEVER value-ref targets. Only un-packaged files (scripts,.kts) keep file-level targets (extract-vref-nopkg.txt), and class/object/companion-scope constants (class: parent) are the working population (extract-vref.txt: readTop → TOP_LIMIT, readBoth → SHARED_TABLE). - Reader scopes: every function/method/constant/variable node (764) —
fields are NOT readers (a class
val's initializer reads nothing; a hook-'const' object property IS a reader — its whole property_declaration subtree incl.by lazy { }lambda contents is DFS'd; the reader DFS has NO lambda halt, unlike the fn-ref scan). - Shadow prune (803-878): the kotlin declarator case is
property_declaration(856-869) — vd = find directvariable_declaration→ its firstsimple_identifier→ bump (the Swift half of that case — name field / value_binding_pattern — is null path for kotlin); destructuring (multi_variable_declaration) bumps NOTHING (a destructured local shadow never prunes — quirk, preserve). bump() countsidentifier/simple_identifiernodes (807 — the comment names Kotlin). Everyval/varANYWHERE in the tree bumps its name: the target's own declarator + any body-local re-declaration → declCountfileScopeCount → target deleted. Pinned: companion
RETRY_MAX+ a method-localval RETRY_MAX = 9→ RETRY_MAX pruned (readBoth emits only SHARED_TABLE). - Emission (880-930): per reader scope DFS (stack-based, namedChildren
pushed in order and POPPED — reverse-source-order visitation, ruby
precedent; edge ORDER follows); reader node type
simple_identifier(906-909 — the comment names Kotlin;identifier/constant/namenever occur in kotlin trees). Any textual occurrence whose text maps to a target: nav members (X.SHARED_TABLE's member half),${TARGET}interpolations (interpolated_expression > simple_identifier) — but NOT$TARGET(node kindinterpolated_identifier, not accepted — pinned). Skip self-id, same-name, dedupe per (scope,target) → EDGE {kind:'references', metadata:{valueRef:true}}, appended AFTER the walk (flush order below). The Dart/Pascal sibling pull (891) is inert — a kotlin property's next sibling can be agetter, which is neitherfunction_bodynorblock.
Function-as-value capture (#756) — KOTLIN_SPEC (function-ref.ts:240-248)
idTypes = EMPTY (bare simple_identifiers are NEVER candidates;
explicitRef always true — irrelevant, no addressOfOnly). dispatch:
value_arguments → args; assignment → rhs with NO field (RHS = LAST
named child; the lhs for the param-storage skip comes from
namedChild(0) — the directly_assignable_expression; the skip
(408:430-443) compares the LHS's trailing identifier to the FULL rhs text —
callable refs start ::, so it effectively never fires for kotlin, but port
the comparison). layers: value_argument → null (fan out namedChildren).
special: {callable_reference, navigation_expression}. No
unwrap/ungatedModes/addressOfOnly.
- The
value_argumentlabel-forward skip (547-557) is DEAD for kotlin — it readsgetChildByField(node,'name')→ null (zero fields), so a named argumentf(cb = cb)is NOT skipped; the fan-out visits both the label and value identifiers (bare ids → nothing anyway, idTypes empty). Only Swift exercises the skip. Reproduce the fan-out. callable_referencespecial (649-665): scan namedChildren — receiver = lasttype_identifierchild, member = lastsimple_identifierchild. No member → [] (String::class— theclassis an anon keyword → member null → nothing). No receiver → bare member (::topLevel→topLevel, gated). Receiver present →/^[A-Z]/on its text:OtherClass::handle→ candidateOtherClass::handle(the::rule at flush:709 — ALWAYS-flush);w::render→ [] — the grammar parses even a lowercase variable receiver astype_identifier, and the CASE regex (not the node type) is what drops it (pinned: no ref).navigation_expressionspecial (671-681): only when the WHOLE node's text startsthis::→ the navigation_suffix starting::→ its LAST named child → candidatethis.<member>(ALWAYS-flush).this::callerpinned. Ordinarya.bnavs in args → [].- Capture points: visitNode:990 (top-level/class-scope call args),
visitFunctionBody:5137, scanFnRefSubtree (hook-consumed property
subtrees —
val x = register(::f)captures via the inner value_arguments; the scan halts atlambda_literal(610), but the hook's own initializer walk (§Properties) covers the same subtree with the PROPERTY on the stack, so refs insideby lazy { }/trailing lambdas are captured there — a shallow::refreachable by BOTH is emitted twice, once from the class and once from the property). NOT captured anywhere: local initializer callable refs — kotlin's dispatch has NO property_declaration/varinit key (unlike SWIFT_SPEC — do not borrow it). Pinned: torture emits exactly three function_refs —topLevel(definedHere),OtherClass::handle,this.caller. - Flush gate (639-728): generated-file skip;
this.-prefixed +::-containing candidates always flush; bare names need definedHere (same-file function/method names) ∪ importedNames (dotted-import last segments — §Imports). Dedupe${fromNodeId}|${name}→ {referenceKind:'function_ref'} (FUNCTION_REF_CODE 200 on the wire).
visitFunctionBody (5129-5286) — kotlin rows
- maybeCaptureFnRefs (5137) per node;
macro_invocationbranch rust-gated. call_expression→ extractCall (5143), NO return → children recursed (chains/args re-visited).- INSTANTIATION_KINDS (5145) — never for kotlin. extractBareCall — absent.
- extractStaticMemberRef (5218) — every body node (§Static-member).
- variable_declarator type-annotation branch (5230) — dead (no such kind).
- Nested
function_declaration(5245) → named → extractFunction (→ receiver check → possibly extractMethod). Local funs becomefunctionnodes (QN…::caller::localFn). - classTypes (5255): a body-local
class LocalClass { … }→ full extractClass (kind via classify), contained by the enclosing function — its methods extract normally (pinned:caller::LocalClass::lm). object_declarationin a body is NOT dispatched (extraClassNodeTypes is not checked in visitForCallsAndStructure) → recursed → its class_body'sfuns hit 5245 → extractFunction: a local object's methods become FUNCTIONS contained by the enclosing function (pinned:caller::om), its properties mint nothing (no hook here). Same forobject_literal(anonymous objects):val anon = object : Drawable { override fun draw() … }in a body → NO class node, NO extends ref,drawleaks out as a function under the enclosing fn with its body calls attributed to it (pinned:caller::draw,calls helper from=function:draw). At TOP-LEVEL/class scope the same object_literal sits inside a hook-consumed property → completely invisible (methods and all — scan halts at nothing relevant but extraction never runs). Pin the asymmetry.- Bodies recursed transparently through when/if/for/try
(control_structure_body), elvis, postfix
!!, labels (label@), lambdas (annotated_lambda/lambda_literal — enclosing-fn attribution), string templates (interpolated_expression emits calls; interpolated_identifier is inert).
Misc shared paths
- Positions:
line = startPosition.row + 1,column = startPosition.column— UTF-16 code units (textutil::col16), as are startIndex/endIndex substrings (getNodeText everywhere) and the import-signature.trim(). - Refs carry NO filePath/language (store denormalizes; §arch-7 — no
REF_FLAG_FILE_PATH use).
function_ref= wire code 200. extract()wrap: file node → namespace node (if package) → walk order →flushFnRefCandidatesthenflushValueRefs(538-539, both while the namespace is still pushed) → pops. Table order: nodes in creation order; contains edges interleaved with creation, value-ref EDGES appended LAST; walk-order refs, then function_ref refs at flush. Store/harness are rowid-order-sensitive — reproduce exactly.- CRLF hazards inventory for the kotlin path: kotlin.ts has NO regexes
over multi-line source (getReturnType's
/^[A-Za-z_]\w*$/and the visibility.includesare single-token); the shared paths' regexes that fire for kotlin are extractCall's parenthesized-conversion (4530, single-name,\s*can eat\r— port as-is with\ssemantics), the #750\s+strip (JS\s⊇\r— Rust regex\smatches\rtoo, parity holds), decorator name normalization (4959-4962), and cleanCommentMarkers'gmstrips →js_multiline_stripin docstring.rs (#1329), call it. Grammar-level CRLF probed clean (identical shapes, no errors,extract-crlf.txtbyte-sane; the comment-gluing reproduces on CRLF identically). - Defer policy: per-file
has_error()→defer:; expected incidence 4.7–8.5% (§arch-6) including the PHANTOM class (hasError with no ERROR/missing node — trust the flag, not node presence). wasm recovery is canonical; fun-interface files always land here. - MAX_FILE_SIZE / generated-file skips: shared, nothing kotlin-specific.
Frameworks & synthesis consumers (stay TS-side — pin the walker's output contract)
- kotlinExpectActualEdges (resolution/callback-synthesizer.ts:987-1026;
doc block :955-985) — the tracker's "expect/actual pairing is
synthesis-side". Reads
queries.iterateNodesByLanguageWithDecorator( 'kotlin','actual')(db/queries.ts:1082 — a LIKE pre-filter over the node DECORATORS column) then exactdecorators.includes('actual'),getNodesByQualifiedNameExact(act.qualifiedName), kind compatibility via KMP_TYPE_KINDS {class, interface, struct, enum, type_alias} (:982 —actual typealiasfulfillment), different file, counterpart NOT marked actual → synthesizedcallsedge decl→actual. Walker obligations: decorators content+order from extractModifiers; EXACT qualifiedNames (package-prefixed —com.example::PlatformFile); kinds; filePath; startLine. Validate on kotlinx.coroutines (the KMP gate repo: 30 expect-files / 95 actual-files at survey time) — spot-checksynthesizedBy: 'kotlin-expect-actual'edge counts are IDENTICAL under kernel and wasm arms after a full init of each. - Closure-collection pass (callback-synthesizer.ts:252-326, CC_LANGUAGES
:77 = {swift, kotlin} — the #1235 gate) — synthesis-side, no port, but
it consumes extraction artifacts: for every method/function node with
language === 'kotlin'it re-reads source and slicessliceLines(content, m.startLine, m.endLine), regexing.forEach { it(dispatchers and.append/.add/.push/.insert(registrars. Walker obligations: method/function node startLine/endLine spans (incl. the resolveBody endLine extension) and the language tag — a truncated endLine silently drops dispatch edges. - rnCrossPlatformEdges (callback-synthesizer.ts:1645+) — kotlin ∈ NATIVE set (:1649); pairs native method/function node NAMES across java/kotlin/objc/cpp with JS callers. Standard node-table obligation only.
- springResolver / expoModulesResolver / fabricViewResolver (§arch-2) —
regex over raw
.ktsource in extractFromSource:6736-6758; their route nodes carry literal ids (route:${filePath}:${line}:…) and their refs carry filePath+language (framework refs, unlike extraction refs). No walker dependency beyond method/class node names for their handler-ref resolution.
Parity mechanics (all have bitten before)
- Emission order per §Misc: file → namespace → source-order walk (per construct: node + contains edge → extends refs BEFORE body members → extractor-order refs) → function_ref refs → value-ref EDGES last.
- generateNodeId inputs: (filePath, kind, name, startRow+1) — name keeps backticks; import nodes are named the dotted path; enum_member line = the IDENTIFIER's line (not the enum_entry's — same line in practice, but the COLUMN and the node's position row both come from the identifier child); namespace line = the package_header start; the glued-comment extents affect endLine/endColumn (and import signatures), never the id line.
- Receiver-QN methods: id hashes the NAME only — the qualifiedName
override (
WidgetK::extend) does not enter generateNodeId. - UTF-16 columns/slices (textutil::col16/slice_utf16): every ref/node column, getNodeText substrings, import-signature trim. Kotlin sources are multibyte-heavy (string templates, KDoc) — the torture fixture needs a non-ASCII line before a symbol.
- CRLF: §Misc inventory; CRLF variants of every fixture derived in-memory (kernel-tsjs-parity pattern).
- Defer policy:
has_error()→defer:— INCLUDING phantom errors (complete CSTs; do not "optimize" by checking for ERROR nodes) and every fun-interface file. Sweep with the default--max-deferral 0.1; expected deferral counts okio 23 / okhttp 49 / kotlinx.coroutines 51. - node-ID-string dedupe:
node_idsvec pattern (same-(kind,name,line) collisions are routine — e.g. one-lineclass X { fun x() … }shapes).
Gates (per plan §5, no exceptions)
- Standalone GRAMMAR-BUMP gate first (rust pattern), before any walker:
vendor the wasm +
VENDORED_WASM_LANGS += 'kotlin'(+ the kernel C vendor/build.rs/langs.rs/grammar-parity row staged with it), then old-vs-new full-init dump-diffs (scripts/dump-graph.mjs, cmp) on the three gate repos with the kernel OFF both arms. Expected: byte-identical on all three (behavior-neutral bump — any hunk at all blocks). Full suite green ×2. - Torture fixtures per
## Fixtures to buildbelow, exercised by a new__tests__/kernel-kotlin-parity.test.ts. - Parity sweeps (
scripts/kernel-parity.mjs <dir>, order-sensitive full-object, default--max-deferral 0.1):/private/tmp/claude-501/-Users-colby-Development-CodeGraph-codegraph/0c11bda1-0b19-4fec-bcd9-d0cb4b2d6e8a/scratchpad/gate-repos/okio(small, 322 kt/kts files)…/gate-repos/okhttp(medium, 580)…/gate-repos/kotlinx.coroutines(large, 1,082 — the KMP/expect-actual gate) (cloned fresh at survey; re-clone public OSS if gone — agent-eval policy). Expect 0-diff on every NON-deferred file and exactly the §arch-6 deferral counts. Then full-init dump-diffs byte-identical (kernel arm vsCODEGRAPH_KERNEL=0,dump-graph.mjs, cmp) on the same three.
- KMP synthesis spot-check (tracker row requirement): after the
kotlinx.coroutines dumps,
select count(*) from edges where json_extract(metadata,'$.synthesizedBy')='kotlin-expect-actual'equal across arms (the dump gate already implies it; assert it explicitly once). - Suite: kernel-kotlin-parity torture + CRLF variants + the defer
fixture (a
fun interfacefile — asserts kerneldefer:+ wasm-served output matchesextract-funiface.txtshape) + a phantom-error fixture (single-line class body — kernel defers despite the complete CST); full suite ×2 green withCODEGRAPH_KERNEL_EXPECT=1. DEFAULT_ROUTED += 'kotlin'(kernel/index.ts:37) only after ALL of the above; changelog rides the existing kernel entry.- Post-route perf sanity: gate repos ride the raw path (§arch-2); a Spring-Boot-Kotlin or Expo repo is the decoded-path smoke check. Deferral costs mean the kotlin speedup lands on ~92-95% of files — measure accordingly.
Fixtures to build
__tests__/fixtures/kernel-parity/torture.kt— the survey'ssvy-kotlin/torture.ktis the seed (itsextract-torture.txtis the expected-output pin). Inventory, by branch: package header + file KDoc +@file:annotation; imports: plain, dotted, wildcard (.*→ package path), alias (as— source path), comment-glued signature (comments after the last import); top-level fn with params/return (ret from user_type; sig undefined); extension fns: plain (WidgetK::extend), generic receiver (List), qualified receiver (com::qextbug), infix (Int::pow), operator; suspend fn (isAsync true) + the@suspendMarkertext-includes false positive;private internal(visibility order) + aninternal fun('internal'); inferred return /Unit/ nullable / lambda return /: Tgeneric leak;expect fun(bodiless + dec) /actual fun; tailrec self-call in expression body; top-levelval/var/const val/by lazy {}(constant/variable kinds, initializer + delegate refs attributed TO the property) + destructuring (val (a,b)→ nothing, both scopes) + next-line-getter top-levelval(getter calls → file/namespace); class with primary ctor (props invisible, defaults not walked), class-body val/var/computed (fields; same-line getter consumed vs next-line getter → class-attributed calls), init block + secondary ctor (class-attributed calls), methods, companion (constants under the class + method + named companionNamedminting nothing); bodiless-vs-bodied header asymmetry (class Bodiless(val b = initB()) : Base(readCfg2())→ extends + 2 class-level calls; the bodied twin → extends only); data class (no members); abstract/open one-liners (phantom-error shapes — but keep the PARITY fixture erroring-free: single-line bodies go in the DEFER fixture instead, since the kernel defers them!); interface (no visibility; bodiless method nodes; default impl;val propw/ same-line getter → field); sealed class + nestedobject/data classmembers (extends SealedOp); sealed interface (bodiless); enum: simple entries (positions = identifier), ctor'd entries, entry-with-body (overrides invisible), post-;methods + companion-in-enum;objectdeclaration (class kind; members; value-ref targets);annotation class; annotated class/method (@MyMarker/@JvmStatic→ decorates;@Deprecated("x", …)→ NOTHING;@field:/@get:on properties → nothing); typealias ×2 (no value refs)actual typealias(dec on type_alias);expect class+actual class(+ marked member); call shapes: bare, constructor (WidgetKcalls ref, NO instantiates),this./super.(bare), member, aliased-import receiver, literal receivers (nothing), capitalized-chain re-encode (WidgetK.create().render+ inner) + lowercase chains (bare + inner), 2-hop nav call, safe-callx?.render()(plain encoding),!!receiver (bare), elvis-arm ctor call, trailing lambda (run {}/w.let {}/listOf(1).forEach {}), paren-then-lambda (trailing() {}→trailing()+trailing), generic call (generic<Int>(1)→generic), backtick call, glued newline-invoke chain (raw-text callee — or a note excluding it if the fixture keeps statements separated; pin ONE of the two deliberately),(handler)(4)first-in-body (conv-regex rewrite), interpolation call ("${w.render()}") +$id(nothing); static-member reads:Registry.count/Color.RED(references at receiver pos),com.example.Fq.X(nothing),Cls.memberas callee (skip), assignment LHS writes (nothing); delegation specifiers: plain + ctor'd + generic + qualified (combug) +bydelegation (nothing); local declarations in bodies: named local fn, local class (full), localobject(methods leak as functions),object : Iface {}literal (function leak; and the top-level-property twin — invisible); callable refs:::topLevel(defined-here gate),OtherClass::handle(import gate irrelevant — always-flush),w::render(dropped),this::caller(this.flush),String::class(nothing),val m = ::caller(NOT captured), assignmentobj.cb = ::handler(captured), named-argf(cb = ::handler)(fan-out capture); value refs: companion/object constants + readers (incl. a${CONST}interpolation read and a$CONSTnon-read), the local-shadow prune,count-style non-target names, and the namespace-drop (packaged file: top-level consts are NOT targets — plus the un-packaged.kts/no-package twin where they ARE); docstrings://runs (kept), KDoc (dropped), KDoc above a//run (run kept), KDoc between run and decl (chain broken), comment-after-imports (lost to gluing); a non-ASCII (UTF-16) line before a symbol; awhen/for/labeled-loop body.
- CRLF variants of the fixtures derived in-memory (kernel-tsjs-parity pattern) — docstring cleaning + comment-gluing + import signatures under CRLF bytes.
.ktsfixture — top-level statements (calls from the FILE node), top-level val (file-parent value-ref target), no package header.- Defer fixture #1:
fun interface(Pattern 1 shape) — kernel defers (defer:), wasm output serves the pinned interface+method recovery (extract-funiface.txt). - Defer fixture #2: phantom error —
abstract class A { abstract fun i(): Int }one-liner — kernel defers on has_error() despite a complete, ERROR-node-free CST; wasm output is byte-normal. - KMP fixture pair —
expect class+funin one file,actual classactual typealiasin another (same QNs, different files) — feeds the expect/actual synthesizer identically under both arms (can fold into the frameworks-integration or synthesizer suites if simpler).
Probe artifacts (session scratchpad svy-kotlin/)
table-compare.cjs (kind/field/ABI tables), shape-probe-kotlin.cjs +
torture-{OLD,NEW}.txt (full CST dumps, byte-identical) + torture.diff
(empty), mini-probes.cjs + mini-probes.out (fun-interface ×2,
destructuring ×2, safe-call/elvis, assignment shapes, file annotation,
delegates, named-arg refs, getter nesting, KDoc runs, CRLF, suspend
false-positive, lateinit, top-level object literal — all OLD==NEW),
error-sweep.cjs + errors-{okio,okhttp,kotlinx.coroutines}.txt,
extract-probe.cjs + extract-{torture,vref,vref-nopkg,docs,bodiless,crlf, funiface,lfpkg,kts}.txt (dist-extractor ground truth), fixtures
(torture.kt, vref.kt, vref-nopkg.kt, docs.kt, bodiless.kt,
crlf.kt, funiface.kt, lfpkg.kt, script.kts), grammar material
(tree-sitter-kotlin/ 0.3.8 tag clone + the tree-sitter.json shim,
crate-extract/ = crates.io 0.3.8 tarball, ng-extract/ = kotlin-ng 1.1.0
tarball, tree-sitter-kotlin-NEW.wasm = the staged-candidate build,
sha256s in §Grammar prep). Scratch dirs are throwaway — re-derive from this
doc if gone.