feat(extraction): capture function-as-value — callback registration sites in callers/impact (#756) (#807)

A function name used as a VALUE — passed as an argument
(signal(SIGINT, handler), qsort(..., compare)), assigned to a function
pointer or field (ops->recv_cb = my_cb, OnClick := Handler), or placed in
a struct initializer / handler table ({ .recv_cb = my_cb },
{ "get", getCommand }) — produced no edge in ANY of the 19 tree-sitter
languages, so registered callbacks looked dead and their registration
sites were invisible to callers/impact.

This adds table-driven function-as-value capture across all 19 languages
(plus the wrapper forms: &fn, &Cls::method, Java Class::m, Kotlin ::f,
Swift #selector, ObjC @selector, Ruby method(:sym), Scala eta, Pascal
@Handler), gated at extraction (same-file definitions + imported
bindings; C-family file-scope initializers are constant-expression
contexts and skip the gate, which is how redis-style cross-file command
tables resolve), and resolved by a dedicated strategy: function/method
targets only, same-file first, unique-or-drop cross-file, no fuzzy
fallback ever. Edges persist as kind 'references' with metadata.fnRef,
so getCallers/getImpactRadius surface them with zero graph-layer
changes; MCP callers/callees label them "via callback registration".

Precision rules bought by real-repo false positives (full A/B record in
docs/design/function-ref-capture.md): C++ is &-explicit outside
file-scope tables (fmt's begin/out/size collisions; out-of-line member
defs are function-kind); TS/JS/Python bare ids resolve to functions only
(TS class fields extract as method-kind — pre-existing quirk); Swift
refuses same-file method overload-families; param-forward shapes
(this.x = x, value: value) and destructuring are skipped; minified
bundles (*.min.js) produce no candidates.

Validated on 17 public OSS repos (redis, excalidraw, gin, bytes, okhttp,
okio, Alamofire, flask, sinatra, Newtonsoft.Json, scopt, provider,
busted, Fusion, AFNetworking, PascalCoin, fmt): node counts identical,
zero calls edges lost or gained, references strictly additive
(+3,200 registration edges total), precision spot-checked by reading
sampled source lines (redis 30/30, flask 8/8). Deliberately NOT covered:
indirect-dispatch resolution (o->cb(x) → impl) — that needs data-flow
through struct fields, and a wrong edge is worse than none.

EXTRACTION_VERSION 18 → 19 (re-index to benefit).

Closes #756

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Colby Mchenry
2026-06-11 14:20:27 -05:00
committed by GitHub
co-authored by Claude Opus 4.8
parent 0df9246752
commit 8a114ba53c
13 changed files with 1706 additions and 16 deletions
+1 -1
View File
@@ -21,4 +21,4 @@
* turns the re-index hint into noise — keep it honest (see CLAUDE.md, "Honesty
* in the product is load-bearing").
*/
export const EXTRACTION_VERSION = 18;
export const EXTRACTION_VERSION = 19;
+644
View File
@@ -0,0 +1,644 @@
/**
* Function-as-value capture (#756) — registration-linking for callbacks.
*
* A function name used as a VALUE — passed as a call argument
* (`register_handler(target_cb)`, `signal(SIGINT, handler)`), assigned to a
* field or function pointer (`o->cb = target_cb`, `OnFire := TargetCb`),
* placed in a struct/object initializer (`{ .recv_cb = my_cb }`,
* `{ recv: targetCb }`, `Ops{Cb: targetCb}`), or listed in a function table
* (`static cb_t table[] = { cb_a, cb_b }`) — is a real dependency that static
* call extraction misses entirely: `callers(target_cb)` showed nothing but
* direct calls, so every callback looked dead and its registration sites were
* invisible to impact analysis.
*
* This module captures those value positions during the AST walk as
* `function_ref` candidates. Capture is table-driven per language (the value
* positions and wrapper forms differ per grammar — `&fn` in C, `Main::fn` in
* Java, `::fn` in Kotlin, `#selector(fn)` in Swift, `@TargetCb` in Pascal,
* `method(:fn)` in Ruby). Candidates are GATED at end-of-file extraction
* (see `TreeSitterExtractor.flushFnRefCandidates`): only names matching a
* same-file function/method or an imported binding survive, which bounds
* volume and keeps precision high. Resolution then matches survivors against
* function/method nodes ONLY (`matchFunctionRef` in
* `src/resolution/name-matcher.ts`) and persists them as `references` edges,
* which `callers`/`impact` already traverse.
*
* Deliberately NOT covered (resolving the *dispatch* — `o->cb(x)` → the
* registered function — needs data-flow through struct fields; a wrong edge
* is worse than none): indirect-call resolution, PHP string callables,
* Ruby bare symbols outside `method(:sym)`, and `obj.method` member values
* where `obj` isn't `this`/`self`.
*/
import type { Node as SyntaxNode } from 'web-tree-sitter';
import { getNodeText, getChildByField } from './tree-sitter-helpers';
export interface FnRefCandidate {
name: string;
line: number;
column: number;
/** Which capture position produced this candidate (gate policy keys on it). */
mode: CaptureMode;
/**
* True when the value was an explicit reference form (`&fn`, `&Cls::m`,
* `::fn`, `#selector`, `method(:sym)`) rather than a bare identifier —
* C++'s flush policy keys on it.
*/
explicitRef: boolean;
}
/** How to pull candidate value nodes out of a dispatched container node. */
type CaptureMode =
| 'args' // every named child is a potential value (call argument lists)
| 'rhs' // the assignment right-hand side (named field, else last named child)
| 'value' // the `value` field of a keyed pair (object/struct/table initializers)
| 'list' // every named child (array / initializer-list / table positional elements)
| 'varinit'; // a variable declarator's initializer value
interface CaptureRule {
mode: CaptureMode;
/** Field holding the value for rhs/value/varinit (defaults per mode). */
field?: string;
}
export interface FnRefSpec {
/** Bare identifier node types that can act as a function value. */
idTypes: Set<string>;
/** Container node type → how to extract candidate values from it. */
dispatch: Map<string, CaptureRule>;
/**
* Transparent wrapper layers between a container and its values
* (`argument`, `value_argument`, `literal_element`, `expression_list`…).
* Value: the field to descend into, or null for "named children".
* `expression_list` fans out to ALL named children (Go multi-assign).
*/
layers?: Map<string, string | null>;
/**
* Unary wrappers whose operand is the function value — C/C++ `&fn`
* (pointer_expression), Pascal `@Fn` (exprUnary), Scala eta `fn _`
* (postfix_expression). Value: operand field, or null for first named child.
*/
unwrap?: Map<string, string | null>;
/**
* Whole-node reference forms needing bespoke name extraction —
* `method_reference` (Java), `callable_reference` / `navigation_expression`
* (Kotlin), `selector_expression` (Swift `#selector` / ObjC `@selector`),
* Ruby `method(:sym)` calls, and `this.method` member forms.
*/
special?: Set<string>;
/**
* Capture modes whose candidates skip the same-file/import gate and rely on
* resolution's unique-or-drop rule instead. C-family only: an initializer
* value, function-pointer assignment RHS, or table element is a
* function-pointer position by construction, and C has no symbol imports —
* the dominant repo-scale pattern (`server.c`'s command table naming
* handlers defined across files) would otherwise be invisible. Call
* arguments stay gated everywhere (locals passed as args dwarf callbacks).
*/
ungatedModes?: Set<CaptureMode>;
/**
* C++ only: in args/rhs/varinit positions, accept ONLY explicit reference
* forms (`&fn`, `&Cls::method`) — never bare identifiers. C++ codebases are
* dense with generic free-function/accessor names (`begin`, `end`, `out`,
* `size`, `data`) that collide with parameters and locals, and out-of-line
* member definitions extract as function-kind nodes — bare-id matching on
* fmt was mostly wrong edges. File-scope initializer tables (value/list)
* still accept bare identifiers, same as C.
*/
addressOfOnly?: boolean;
}
/** Names that are never function references even when grammars call them identifiers. */
const NAME_STOPLIST = new Set([
'this',
'self',
'super',
'null',
'nil',
'true',
'false',
'undefined',
'new',
'NULL',
'nullptr',
'None',
]);
// ---------------------------------------------------------------------------
// Per-language specs. Node types verified against each grammar (probe fixtures
// in the #756 investigation; see docs/design/function-ref-capture.md).
// ---------------------------------------------------------------------------
/** C / C++ / Objective-C share the C-family initializer & assignment shapes. */
function cFamilySpec(extra?: { special?: string[]; addressOfOnly?: boolean }): FnRefSpec {
return {
idTypes: new Set(['identifier']),
dispatch: new Map<string, CaptureRule>([
['argument_list', { mode: 'args' }],
['assignment_expression', { mode: 'rhs', field: 'right' }],
['init_declarator', { mode: 'varinit', field: 'value' }],
['initializer_list', { mode: 'list' }],
['initializer_pair', { mode: 'value', field: 'value' }],
]),
unwrap: new Map([['pointer_expression', 'argument']]),
special: new Set(extra?.special ?? []),
// C has no symbol imports, and callbacks are registered cross-file at repo
// scale (redis: server.c's command table names handlers from t_*.c) — so
// initializer positions bypass the gate and lean on resolution's
// unique-or-drop rule. ONLY 'value'/'list' (struct/array initializers),
// and the flush additionally requires FILE scope: a C file-scope
// initializer is a constant-expression context, so a bare identifier
// there can only be a function address (or enum/macro, which the
// function-kind filter drops) — never a variable. 'rhs'/'varinit' were
// tried and produced false edges (`prev = next`, `*str = field` — data
// assignments matching a unique same-named function elsewhere), so
// assignments stay gated to same-file/import.
ungatedModes: new Set<CaptureMode>(['value', 'list']),
addressOfOnly: extra?.addressOfOnly,
};
}
// NOTE: deliberately NO `member_expression` (`this.handleClick`) capture for
// TS/JS. Class fields with type annotations are extracted as method-kind
// nodes (pre-existing extractor behavior), so `this.X` value positions —
// which in real code are mostly DATA reads (`setCursor(this.canvas)`) —
// resolved to those field nodes and produced wrong "registration" edges
// (excalidraw A/B finding). Revisit if/when TS field classification is fixed.
const TS_JS_SPEC: FnRefSpec = {
idTypes: new Set(['identifier']),
dispatch: new Map<string, CaptureRule>([
['arguments', { mode: 'args' }],
['assignment_expression', { mode: 'rhs', field: 'right' }],
['variable_declarator', { mode: 'varinit', field: 'value' }],
['pair', { mode: 'value', field: 'value' }],
['array', { mode: 'list' }],
]),
};
const PYTHON_SPEC: FnRefSpec = {
idTypes: new Set(['identifier']),
dispatch: new Map<string, CaptureRule>([
['argument_list', { mode: 'args' }],
['assignment', { mode: 'rhs', field: 'right' }],
['keyword_argument', { mode: 'value', field: 'value' }], // Thread(target=worker)
['pair', { mode: 'value', field: 'value' }],
['list', { mode: 'list' }],
]),
special: new Set(['attribute']),
};
const GO_SPEC: FnRefSpec = {
idTypes: new Set(['identifier']),
dispatch: new Map<string, CaptureRule>([
['argument_list', { mode: 'args' }],
['assignment_statement', { mode: 'rhs', field: 'right' }],
['short_var_declaration', { mode: 'rhs', field: 'right' }],
['var_spec', { mode: 'varinit', field: 'value' }],
['keyed_element', { mode: 'value' }], // value = last literal_element child
['literal_value', { mode: 'list' }], // positional composite literals
]),
layers: new Map<string, string | null>([
['literal_element', null],
['expression_list', null],
]),
};
const RUST_SPEC: FnRefSpec = {
idTypes: new Set(['identifier']),
dispatch: new Map<string, CaptureRule>([
['arguments', { mode: 'args' }],
['assignment_expression', { mode: 'rhs', field: 'right' }],
['field_initializer', { mode: 'value', field: 'value' }],
['array_expression', { mode: 'list' }],
['static_item', { mode: 'varinit', field: 'value' }],
['let_declaration', { mode: 'varinit', field: 'value' }],
]),
};
const JAVA_SPEC: FnRefSpec = {
// No bare-identifier function values in Java — only method references.
idTypes: new Set<string>(),
dispatch: new Map<string, CaptureRule>([
['argument_list', { mode: 'args' }],
['assignment_expression', { mode: 'rhs', field: 'right' }],
['variable_declarator', { mode: 'varinit', field: 'value' }],
]),
special: new Set(['method_reference']),
};
const KOTLIN_SPEC: FnRefSpec = {
idTypes: new Set<string>(),
dispatch: new Map<string, CaptureRule>([
['value_arguments', { mode: 'args' }],
['assignment', { mode: 'rhs' }], // RHS = last named child (no field in grammar)
]),
layers: new Map<string, string | null>([['value_argument', null]]),
special: new Set(['callable_reference', 'navigation_expression']),
};
const CSHARP_SPEC: FnRefSpec = {
idTypes: new Set(['identifier']),
dispatch: new Map<string, CaptureRule>([
['argument_list', { mode: 'args' }],
['assignment_expression', { mode: 'rhs', field: 'right' }], // covers `+=` event subscription
['initializer_expression', { mode: 'list' }],
['variable_declarator', { mode: 'varinit' }],
]),
layers: new Map<string, string | null>([['argument', null]]),
special: new Set(['member_access_expression']),
};
const RUBY_SPEC: FnRefSpec = {
// Bare identifiers in Ruby args are method CALLS or locals, never function
// values — only the `method(:name)` idiom (and `&method(:name)`) qualifies.
idTypes: new Set<string>(),
dispatch: new Map<string, CaptureRule>([
['argument_list', { mode: 'args' }],
['pair', { mode: 'value', field: 'value' }],
]),
layers: new Map<string, string | null>([['block_argument', null]]),
special: new Set(['call']),
};
const SWIFT_SPEC: FnRefSpec = {
idTypes: new Set(['simple_identifier']),
dispatch: new Map<string, CaptureRule>([
['value_arguments', { mode: 'args' }],
['assignment', { mode: 'rhs', field: 'result' }],
['array_literal', { mode: 'list' }],
['property_declaration', { mode: 'varinit', field: 'value' }],
]),
layers: new Map<string, string | null>([['value_argument', 'value']]),
special: new Set(['selector_expression']),
};
const SCALA_SPEC: FnRefSpec = {
idTypes: new Set(['identifier']),
dispatch: new Map<string, CaptureRule>([
['arguments', { mode: 'args' }],
['assignment_expression', { mode: 'rhs', field: 'right' }],
['val_definition', { mode: 'varinit', field: 'value' }],
]),
unwrap: new Map<string, string | null>([['postfix_expression', null]]), // eta-expansion `fn _`
};
const DART_SPEC: FnRefSpec = {
idTypes: new Set(['identifier']),
dispatch: new Map<string, CaptureRule>([
['arguments', { mode: 'args' }],
['assignment_expression', { mode: 'rhs', field: 'right' }],
['pair', { mode: 'value', field: 'value' }],
['list_literal', { mode: 'list' }],
['static_final_declaration', { mode: 'varinit' }],
]),
layers: new Map<string, string | null>([['argument', null]]),
};
const LUA_SPEC: FnRefSpec = {
idTypes: new Set(['identifier']),
dispatch: new Map<string, CaptureRule>([
['arguments', { mode: 'args' }],
['assignment_statement', { mode: 'rhs' }], // RHS expression_list children carry `value` fields
['field', { mode: 'value', field: 'value' }], // table fields, keyed AND positional
]),
layers: new Map<string, string | null>([['expression_list', null]]),
};
const PASCAL_SPEC: FnRefSpec = {
idTypes: new Set(['identifier']),
dispatch: new Map<string, CaptureRule>([
['exprArgs', { mode: 'args' }],
['assignment', { mode: 'rhs', field: 'rhs' }], // OnClick := Handler
]),
unwrap: new Map<string, string | null>([['exprUnary', 'operand']]), // @Handler
};
/**
* Capture specs by language. PHP is deliberately absent: its first-class
* callable `fn(...)` already extracts as a `calls` edge, and string callables
* (`'fn_name'`) are a precision risk left for a follow-up.
*/
export const FN_REF_SPECS: Record<string, FnRefSpec | undefined> = {
c: cFamilySpec(),
cpp: cFamilySpec({ addressOfOnly: true }),
objc: cFamilySpec({ special: ['selector_expression'] }),
typescript: TS_JS_SPEC,
tsx: TS_JS_SPEC,
javascript: TS_JS_SPEC,
jsx: TS_JS_SPEC,
python: PYTHON_SPEC,
go: GO_SPEC,
rust: RUST_SPEC,
java: JAVA_SPEC,
kotlin: KOTLIN_SPEC,
csharp: CSHARP_SPEC,
ruby: RUBY_SPEC,
swift: SWIFT_SPEC,
scala: SCALA_SPEC,
dart: DART_SPEC,
lua: LUA_SPEC,
luau: LUA_SPEC,
pascal: PASCAL_SPEC,
};
// ---------------------------------------------------------------------------
// Capture
// ---------------------------------------------------------------------------
/**
* Extract candidate names from a dispatched container node. Returns the
* (name, position) pairs of every function-value-shaped expression found.
*/
export function captureFnRefCandidates(
container: SyntaxNode,
rule: CaptureRule,
spec: FnRefSpec,
source: string
): FnRefCandidate[] {
const valueNodes: SyntaxNode[] = [];
switch (rule.mode) {
case 'args':
case 'list': {
for (let i = 0; i < container.namedChildCount; i++) {
const child = container.namedChild(i);
if (child) valueNodes.push(child);
}
break;
}
case 'rhs': {
const rhs = rule.field
? getChildByField(container, rule.field)
: container.namedChild(container.namedChildCount - 1);
if (rhs) {
// Param-storage skip: `this.status = status` / `o->cb = cb` — when
// the assigned member's name EQUALS the RHS identifier, the RHS is a
// local/parameter being stored, and the function it holds (if any)
// is unknowable statically. A same-named function elsewhere would
// resolve to the WRONG target (excalidraw A/B finding), so skip.
const lhs =
getChildByField(container, 'left') ??
getChildByField(container, 'lhs') ??
getChildByField(container, 'target') ??
(container.namedChildCount >= 2 ? container.namedChild(0) : null);
const lhsText = lhs ? getNodeText(lhs, source) : '';
const lhsLastName = lhsText.match(/([A-Za-z_$][A-Za-z0-9_$]*)\s*$/)?.[1];
const rhsText = getNodeText(rhs, source).trim();
if (lhsLastName && lhsLastName === rhsText) break;
valueNodes.push(rhs);
}
break;
}
case 'value': {
let value = rule.field ? getChildByField(container, rule.field) : null;
// Keyed containers without a value field (Go keyed_element): the value
// is the LAST named child (the first is the key).
if (!value && container.namedChildCount > 0) {
value = container.namedChild(container.namedChildCount - 1);
}
if (value) valueNodes.push(value);
break;
}
case 'varinit': {
// Destructuring (`const { center } = ellipse`) extracts DATA from the
// RHS — never a function alias. Without this skip, a parameter that
// shadows a same-named imported function produced a wrong edge.
const nameNode =
getChildByField(container, 'name') ?? getChildByField(container, 'pattern');
if (nameNode && (nameNode.type === 'object_pattern' || nameNode.type === 'array_pattern' ||
nameNode.type === 'tuple_pattern' || nameNode.type === 'struct_pattern')) {
break;
}
if (rule.field) {
const value = getChildByField(container, rule.field);
if (value) valueNodes.push(value);
} else {
// No value field in this grammar (C# variable_declarator, Dart
// static_final_declaration): the initializer is the last named child —
// but a declarator WITHOUT an initializer has its NAME there instead.
// Require ≥2 named children and never pick the name/pattern child.
const value = container.namedChild(container.namedChildCount - 1);
const nameChild =
getChildByField(container, 'name') ?? getChildByField(container, 'pattern');
if (
value &&
container.namedChildCount >= 2 &&
(!nameChild || value.id !== nameChild.id)
) {
valueNodes.push(value);
}
}
break;
}
}
const out: FnRefCandidate[] = [];
for (const v of valueNodes) {
// A bare identifier is one that normalizes without passing through an
// unwrap/special reference form. C++'s addressOfOnly policy (applied at
// flush, where file scope is known) drops bare ids outside file-scope
// initializer tables.
const explicitRef = !spec.idTypes.has(v.type);
for (const { name, node } of normalizeValue(v, spec, source, 0)) {
if (!name || NAME_STOPLIST.has(name)) continue;
out.push({
name,
line: node.startPosition.row + 1,
column: node.startPosition.column,
mode: rule.mode,
explicitRef,
});
}
}
return out;
}
/**
* Normalize one value expression to zero or more function names. Recursion is
* bounded (wrapper layers only); anything that isn't a recognized
* function-value shape yields [].
*/
function normalizeValue(
node: SyntaxNode,
spec: FnRefSpec,
source: string,
depth: number
): Array<{ name: string; node: SyntaxNode }> {
if (depth > 4) return [];
const type = node.type;
// Bare identifier
if (spec.idTypes.has(type)) {
return [{ name: getNodeText(node, source), node }];
}
// Transparent layers (argument, value_argument, literal_element,
// expression_list, block_argument). expression_list fans out (Go `a, b = f, g`).
const layerField = spec.layers?.get(type);
if (spec.layers?.has(type)) {
// Labeled-argument param-forward skip (Swift/Kotlin): `value: value` /
// `delay: delay` — when the label EQUALS the value identifier, the value
// is a forwarded local/parameter, not a function reference (Alamofire
// A/B finding; same rationale as the `this.x = x` assignment skip).
if (type === 'value_argument') {
const label = getChildByField(node, 'name');
const value = getChildByField(node, 'value') ?? node.namedChild(node.namedChildCount - 1);
if (
label &&
value &&
getNodeText(label, source).trim() === getNodeText(value, source).trim()
) {
return [];
}
}
if (layerField) {
const inner = getChildByField(node, layerField);
return inner ? normalizeValue(inner, spec, source, depth + 1) : [];
}
const results: Array<{ name: string; node: SyntaxNode }> = [];
for (let i = 0; i < node.namedChildCount; i++) {
const child = node.namedChild(i);
if (child) results.push(...normalizeValue(child, spec, source, depth + 1));
}
return results;
}
// Unary wrappers: &fn / @Fn / `fn _`
const unwrapField = spec.unwrap?.get(type);
if (spec.unwrap?.has(type)) {
// C-family `pointer_expression` covers BOTH `&x` (address-of — a function
// value) and `*x` (dereference — a data read, never a function value).
// Only `&` qualifies; without this, fmt's `*begin` reads resolved to its
// free `begin()` functions.
if (type === 'pointer_expression' && node.child(0)?.type !== '&') return [];
const inner = unwrapField ? getChildByField(node, unwrapField) : node.namedChild(0);
if (!inner) return [];
// C++ `&Widget::on_click` — keep the QUALIFIED name. Resolution scopes the
// method to that class (more precise than a bare-name match, and exempt
// from the cpp bare-ids-are-free-functions rule since `&Cls::m` is an
// explicit member-pointer).
if (inner.type === 'qualified_identifier') {
const text = getNodeText(inner, source).trim();
return /^[A-Za-z_][\w:]*$/.test(text) ? [{ name: text, node: inner }] : [];
}
return normalizeValue(inner, spec, source, depth + 1);
}
// Special whole-node reference forms
if (spec.special?.has(type)) {
return normalizeSpecial(node, type, source);
}
return [];
}
/** Rightmost descendant-or-self named child of one of the given types. */
function lastNamedOfType(node: SyntaxNode, types: Set<string>): SyntaxNode | null {
let found: SyntaxNode | null = null;
for (let i = 0; i < node.namedChildCount; i++) {
const child = node.namedChild(i);
if (!child) continue;
if (types.has(child.type)) found = child;
const deeper = lastNamedOfType(child, types);
if (deeper) found = deeper;
}
return found;
}
function normalizeSpecial(
node: SyntaxNode,
type: string,
source: string
): Array<{ name: string; node: SyntaxNode }> {
switch (type) {
// Java `Main::targetCb` / `this::run0` — last identifier child is the method.
case 'method_reference': {
let last: SyntaxNode | null = null;
for (let i = 0; i < node.namedChildCount; i++) {
const child = node.namedChild(i);
if (child && child.type === 'identifier') last = child;
}
return last ? [{ name: getNodeText(last, source), node: last }] : [];
}
// Kotlin `::targetCb` — the simple_identifier child.
case 'callable_reference': {
for (let i = 0; i < node.namedChildCount; i++) {
const child = node.namedChild(i);
if (child && child.type === 'simple_identifier') {
return [{ name: getNodeText(child, source), node: child }];
}
}
return [];
}
// Kotlin `this::fire` parses as navigation_expression with a `::fire`
// navigation_suffix. Ordinary `a.b` navigation MUST yield nothing.
case 'navigation_expression': {
for (let i = 0; i < node.namedChildCount; i++) {
const child = node.namedChild(i);
if (child && child.type === 'navigation_suffix' && getNodeText(child, source).startsWith('::')) {
const id = child.namedChild(child.namedChildCount - 1);
if (id) return [{ name: getNodeText(id, source), node: id }];
}
}
return [];
}
// Swift `#selector(Holder.fire)` → fire. ObjC `@selector(storeImage:)` →
// `storeImage:` verbatim (ObjC method nodes keep their selector colons).
case 'selector_expression': {
const inner = node.namedChild(0);
if (!inner) return [];
if (inner.type === 'identifier' || inner.type === 'simple_identifier') {
return [{ name: getNodeText(inner, source), node: inner }];
}
// Swift dotted form: rightmost simple_identifier. ObjC keyword selector:
// text as-is.
const last = lastNamedOfType(node, new Set(['simple_identifier']));
if (last) return [{ name: getNodeText(last, source), node: last }];
return [{ name: getNodeText(inner, source).trim(), node: inner }];
}
// Ruby `method(:target_cb)` — a `call` whose method is literally `method`
// with a single symbol argument.
case 'call': {
const method = getChildByField(node, 'method');
if (!method || getNodeText(method, source) !== 'method') return [];
const args = getChildByField(node, 'arguments');
if (!args || args.namedChildCount !== 1) return [];
const sym = args.namedChild(0);
if (!sym || sym.type !== 'simple_symbol') return [];
const name = getNodeText(sym, source).replace(/^:/, '');
return name ? [{ name, node: sym }] : [];
}
// `self.handle_click` (Python) — object must be EXACTLY `self`.
case 'attribute': {
const obj = getChildByField(node, 'object');
const attr = getChildByField(node, 'attribute');
if (obj && attr && obj.type === 'identifier' && getNodeText(obj, source) === 'self') {
return [{ name: getNodeText(attr, source), node: attr }];
}
return [];
}
// `this.Run0` (C#) — receiver must be EXACTLY `this`. Two grammar shapes:
// newer tree-sitter-c-sharp exposes an `expression` field holding a
// `this_expression`; the vendored grammar keeps `this` as an anonymous
// token (only the `name` field is a named child), so fall back to the
// node text.
case 'member_access_expression': {
const name = getChildByField(node, 'name');
if (!name) return [];
const expr = getChildByField(node, 'expression');
const isThisReceiver = expr
? expr.type === 'this_expression' || expr.type === 'this'
: getNodeText(node, source).startsWith('this.');
return isThisReceiver ? [{ name: getNodeText(name, source), node: name }] : [];
}
default:
return [];
}
}
+3
View File
@@ -41,6 +41,9 @@ const GENERATED_PATTERNS: ReadonlyArray<RegExp> = [
/\.pb\.[jt]s$/,
/_pb\.[jt]s$/,
/_grpc_pb\.[jt]s$/,
// Minified bundles vendored into a repo (docs sites, examples). Their
// single-letter symbols make name-based edges pure noise.
/\.min\.m?js$/,
// Python — protobuf / gRPC / openapi-codegen
/_pb2(_grpc)?\.py$/,
/_pb2\.pyi$/,
+183 -1
View File
@@ -17,6 +17,8 @@ import {
} from '../types';
import { getParser, detectLanguage, isLanguageSupported, isFileLevelOnlyLanguage } from './grammars';
import { generateNodeId, getNodeText, getChildByField, getPrecedingDocstring } from './tree-sitter-helpers';
import { FN_REF_SPECS, captureFnRefCandidates, type FnRefSpec, type FnRefCandidate } from './function-ref';
import { isGeneratedFile } from './generated-detection';
import type { LanguageExtractor, ExtractorContext } from './tree-sitter-types';
import { EXTRACTORS } from './languages';
import { LiquidExtractor } from './liquid-extractor';
@@ -222,12 +224,18 @@ export class TreeSitterExtractor {
private extractor: LanguageExtractor | null = null;
private nodeStack: string[] = []; // Stack of parent node IDs
private methodIndex: Map<string, string> | null = null; // lookup key → node ID for Pascal defProc lookup
// Function-as-value capture (#756): per-language spec + candidates collected
// during the walk, gated & flushed into unresolvedReferences at end-of-file
// (see flushFnRefCandidates).
private fnRefSpec: FnRefSpec | undefined;
private fnRefCandidates: Array<FnRefCandidate & { fromNodeId: string }> = [];
constructor(filePath: string, source: string, language?: Language) {
this.filePath = filePath;
this.source = source;
this.language = language || detectLanguage(filePath, source);
this.extractor = EXTRACTORS[this.language] || null;
this.fnRefSpec = FN_REF_SPECS[this.language];
}
/**
@@ -314,6 +322,10 @@ export class TreeSitterExtractor {
this.visitNode(this.tree.rootNode);
// Gate + flush function-as-value candidates (#756) while the file's
// nodes and import refs are complete and the file node is still pushed.
this.flushFnRefCandidates();
if (packageNodeId) this.nodeStack.pop();
this.nodeStack.pop();
} catch (error) {
@@ -352,6 +364,136 @@ export class TreeSitterExtractor {
};
}
/**
* Function-as-value capture (#756): if this node is one of the language's
* value-position containers (call arguments, assignment RHS, struct/object
* initializer, array/table literal), collect candidate function names from
* it. Candidates are gated & flushed at end-of-file (flushFnRefCandidates).
*/
private maybeCaptureFnRefs(node: SyntaxNode, nodeType: string): void {
const spec = this.fnRefSpec;
if (!spec) return;
const rule = spec.dispatch.get(nodeType);
if (!rule || this.nodeStack.length === 0) return;
const fromNodeId = this.nodeStack[this.nodeStack.length - 1];
if (!fromNodeId) return;
for (const cand of captureFnRefCandidates(node, rule, spec, this.source)) {
this.fnRefCandidates.push({ ...cand, fromNodeId });
}
}
/**
* Candidates-only scan of a subtree the main walkers won't traverse
* (top-level variable initializers). No extraction side effects. Halts at
* nested function definitions: their bodies are walked — and their
* candidates attributed — by extractFunction's own body walk.
*/
private scanFnRefSubtree(node: SyntaxNode, depth: number): void {
if (!this.fnRefSpec || depth > 12) return;
const nodeType = node.type;
if (depth > 0 && (
this.extractor?.functionTypes.includes(nodeType) ||
nodeType === 'arrow_function' ||
nodeType === 'function_expression' ||
nodeType === 'lambda_literal' ||
nodeType === 'lambda_expression'
)) {
return;
}
this.maybeCaptureFnRefs(node, nodeType);
for (let i = 0; i < node.namedChildCount; i++) {
const child = node.namedChild(i);
if (child) this.scanFnRefSubtree(child, depth + 1);
}
}
/**
* Gate captured function-as-value candidates and push survivors as
* `function_ref` unresolved references.
*
* The gate bounds volume and protects precision: a candidate survives only
* if its name matches a function/method DEFINED IN THIS FILE or a name this
* file imports/references. Everything else (locals, params, fields passed
* as arguments) is dropped before it ever reaches the database. Resolution
* then matches survivors against function/method nodes only
* (matchFunctionRef) and emits `references` edges — which callers/impact
* already traverse.
*
* Known v1 limit, deliberate: a C/C++ callback registered in a DIFFERENT
* translation unit than its definition (extern, no symbol imports to match)
* is not captured. Same-file registration — the dominant C pattern (static
* callback + same-file ops struct) — is.
*/
private flushFnRefCandidates(): void {
if (this.fnRefCandidates.length === 0) return;
const candidates = this.fnRefCandidates;
this.fnRefCandidates = [];
// Generated/minified files (vendored jquery.min.js and friends): their
// function-as-value edges are noise — single-letter minified symbols
// resolve everywhere. Same policy as the callback synthesizer.
if (isGeneratedFile(this.filePath)) return;
const definedHere = new Set<string>();
for (const n of this.nodes) {
if (n.kind === 'function' || n.kind === 'method') definedHere.add(n.name);
}
// Import-binding names only (all binding emitters push kind 'imports').
// Deliberately NOT 'references': those carry type-annotation and
// interface-member names, which let local variables that share a type
// member's name slip through the gate (excalidraw A/B finding).
const SIMPLE_NAME = /^[A-Za-z_$][A-Za-z0-9_$]*$/;
const importedNames = new Set<string>();
for (const r of this.unresolvedReferences) {
if (r.referenceKind === 'imports' && SIMPLE_NAME.test(r.referenceName)) {
importedNames.add(r.referenceName);
}
}
const ungated = this.fnRefSpec?.ungatedModes;
const addressOfOnly = this.fnRefSpec?.addressOfOnly === true;
const seen = new Set<string>();
for (const c of candidates) {
const atFileScope = c.fromNodeId.startsWith('file:');
// C++ (addressOfOnly): a BARE identifier qualifies only inside a
// file-scope initializer table. Everywhere else — args, assignments,
// local braced-init lists like `{begin, size}` — only explicit `&`
// forms count (fmt A/B finding: generic names `begin`/`out`/`size`
// collide with locals and members).
if (
addressOfOnly &&
!c.explicitRef &&
!(atFileScope && (c.mode === 'value' || c.mode === 'list'))
) {
continue;
}
// C-family file-scope initializers skip the gate (constant-expression
// context — a bare identifier there is a function address, never a
// variable; see FnRefSpec.ungatedModes). Local initializers and
// everything else require a same-file/import match.
const skipGate = ungated?.has(c.mode) === true && atFileScope;
// Qualified C++ member-pointers (`Widget::on_click`) gate on the member
// name; everything else on the full name.
const gateName = c.name.includes('::')
? c.name.slice(c.name.lastIndexOf('::') + 2)
: c.name;
if (!skipGate && !definedHere.has(gateName) && !importedNames.has(gateName)) {
continue;
}
const key = `${c.fromNodeId}|${c.name}`;
if (seen.has(key)) continue;
seen.add(key);
this.unresolvedReferences.push({
fromNodeId: c.fromNodeId,
referenceName: c.name,
referenceKind: 'function_ref',
line: c.line,
column: c.column,
});
}
}
/**
* Visit a node and extract information
*/
@@ -365,7 +507,14 @@ export class TreeSitterExtractor {
if (this.extractor.visitNode) {
const ctx = this.makeExtractorContext();
const handled = this.extractor.visitNode(node, ctx);
if (handled) return;
if (handled) {
// The hook consumed this subtree, so the walkers below never descend
// into it — scan it for function-as-value candidates (#756). Scala's
// hook handles val/var definitions (`val table = Seq(targetCb)`), for
// example. The scan is capture-only and halts at nested functions.
this.scanFnRefSubtree(node, 0);
return;
}
}
// Pascal-specific AST handling
@@ -374,6 +523,11 @@ export class TreeSitterExtractor {
if (skipChildren) return;
}
// Function-as-value capture (#756) — independent of the dispatch ladder
// below (the captured container types have no other handler there), so it
// can never shadow or be shadowed by an extraction branch.
this.maybeCaptureFnRefs(node, nodeType);
// Check for function declarations
// For Python/Ruby, function_definition inside a class should be treated as method
if (this.extractor.functionTypes.includes(nodeType)) {
@@ -437,17 +591,33 @@ export class TreeSitterExtractor {
// Check for class properties (e.g. C# property_declaration)
else if (this.extractor.propertyTypes?.includes(nodeType) && this.isInsideClassLikeNode()) {
this.extractProperty(node);
// Property initializers aren't walked — scan for function-as-value
// candidates (#756): Scala `val table = Seq(targetCb)` in an object,
// Kotlin `val cb = ::handler` class properties.
this.scanFnRefSubtree(node, 0);
skipChildren = true;
}
// Check for class fields (e.g. Java field_declaration, C# field_declaration)
else if (this.extractor.fieldTypes?.includes(nodeType) && this.isInsideClassLikeNode()) {
this.extractField(node);
// Field initializers aren't walked — scan for function-as-value
// candidates (#756): Java `List<IntConsumer> table = List.of(Main::cb)`,
// C# `List<Action<int>> table = new() { TargetCb }`.
this.scanFnRefSubtree(node, 0);
skipChildren = true;
}
// Check for variable declarations (const, let, var, etc.)
// Only extract top-level variables (not inside functions/methods)
else if (this.extractor.variableTypes.includes(nodeType) && !this.isInsideClassLikeNode()) {
this.extractVariable(node);
// extractVariable doesn't walk every initializer shape (object literals
// are deliberately skipped; Python/Ruby don't walk at all), so scan the
// declaration subtree for function-as-value candidates — `const routes =
// { home: renderHome }`, `handlers = {"recv": target_cb}`. The scan halts
// at nested function definitions (their bodies are walked — and
// attributed — separately) and flush-time dedup absorbs any overlap with
// initializers extractVariable DOES walk.
this.scanFnRefSubtree(node, 0);
skipChildren = true; // extractVariable handles children
}
// Swift stored properties inside a type. Swift instance properties aren't
@@ -3086,6 +3256,10 @@ export class TreeSitterExtractor {
const visitForCallsAndStructure = (node: SyntaxNode): void => {
const nodeType = node.type;
// Function-as-value capture (#756) — function bodies are walked here,
// not in visitNode, so the capture hook must fire in both walkers.
this.maybeCaptureFnRefs(node, nodeType);
// Rocket route-registration macros (`routes![…]` / `catchers![…]`): the
// handler paths live in a raw token tree the call walker can't see.
if (nodeType === 'macro_invocation') this.extractRustRouteMacro(node);
@@ -4461,8 +4635,16 @@ export class TreeSitterExtractor {
for (let i = 0; i < node.namedChildCount; i++) {
const child = node.namedChild(i);
if (!child) continue;
// Function-as-value capture (#756): Pascal bodies are walked here, not
// in visitNode/visitForCallsAndStructure, so the capture hook fires here
// — assignment RHS is the Delphi event-wiring idiom (`OnFire := Handler`).
this.maybeCaptureFnRefs(child, child.type);
if (child.type === 'exprCall') {
this.extractPascalCall(child);
// The walker doesn't descend into a call's arguments — dispatch the
// argument container directly (`RegisterHandler(TargetCb)` / `(@Cb)`).
const args = child.namedChildren.find((c: SyntaxNode) => c.type === 'exprArgs');
if (args) this.maybeCaptureFnRefs(args, 'exprArgs');
} else if (child.type === 'exprDot') {
// A STATEMENT-level bare exprDot is a paren-less call (`Obj.Free;`,
// `TFoo.GetInstance.DoIt;`). Anywhere else (assignment side, condition,