feat(extraction): index Erlang escripts and OTP app resource files (#635, #648) (#1169)

escripts (.escript) index like any module — the ELP grammar has a
first-class shebang node, so no source transform is needed; main/1 and its
helpers get full function/call extraction.

OTP application resource files (<app>.app.src and compiled <app>.app) join
the graph as Erlang terms the grammar parses natively. They route by full
suffix (their last-dot extension, .src, is far too generic for the
extension map). The application tuple yields structure: {mod, {Mod, _}}
links the app to its callback module — the app's entry point — and
{applications, [...]} / {included_applications, [...]} connect umbrella
sibling apps, resolving through the OTP app-name == module-name convention;
kernel/stdlib and other out-of-repo apps stay unresolved.

App-file refs resolve only ever to MODULES: validation on emqx caught the
ssl OTP-app dependency resolving to a test helper FUNCTION named ssl (the
same defect class as the earlier -behaviour gate), so the matchReference
module-only gate now covers every ref an .app/.app.src file emits.

Validated on emqx: 2 app.src + 6 escripts indexed, entry-module and
umbrella-dependency edges all namespace-targeted post-gate, escript
functions extracted; a stray legacy/module.src stays unknown.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

CHANGELOG.md

@@ -12,7 +12,7 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 ### New Features
 
 - Erlang behaviour dispatch is now followed through the graph: a framework call through a variable module — cowboy's `Handler:init`/`Middleware:execute` folds, a plugin manager's `Mod:callback(...)` — links to the repo's implementations of the behaviour that declares that callback, so flow traces and impact cross the OTP callback boundary instead of stopping at it. The links are precision-gated: the callback arity must match, exactly one behaviour may own that callback shape (a collision stays unlinked rather than guessed), the implementer must actually export the callback, and the fan-out is bounded — a behaviour with hundreds of implementers stays a visibly dynamic boundary. Every bridged hop is labeled as dynamic dispatch with its wiring site, never shown as a plain static call.
-- CodeGraph now indexes **Erlang** (`.erl`, `.hrl`) — functions, with clauses and arities of the same name grouped as one symbol spanning all of them, plus records with their fields, `-type`/`-opaque` aliases, `-define` macros, and `-spec` signatures attached to every function. Cross-module `mod:fn(...)` calls resolve to the target module's function, `fun name/arity` values are captured as references (so callback registrations like `lists:foreach(fun submit/1, ...)` link up), `-include`/`-include_lib` connect to the header files they pull in, `-behaviour` declarations link a callback module to its behaviour (and only ever to a module — a same-named macro or function elsewhere in the repo is never mistaken for one), and `-export` lists (plus `-compile(export_all)`) drive each function's public/private flag. OTP's indirection idioms are followed where the target is static: `spawn`/`apply`/`proc_lib`/`timer`/`rpc` calls that name their target as `(Module, Function, Args)` arguments produce call edges, and `gen_server:call`/`cast` connects to the target module's `handle_call`/`handle_cast` — its own when targeting `?MODULE` (including the `-define(SERVER, ?MODULE)` idiom), and the named module when a registered name follows OTP's name-the-server-after-its-module convention (`gen_server:call(other_mod, ...)`, directly or through a `-define(STORE, other_mod)` macro); a registered name that matches no module stays unlinked. Macros participate in the graph too: a `-define` body's calls belong to the macro, each `?MACRO(...)` use site links into the call chain (and bare `?CONSTANT` reads are tracked as references), so a call path hidden behind a macro — `set_password → ?SQL_UPSERT_T → sql_query_t` — traces end-to-end and "where is this macro used" is answerable. Truly dynamic dispatch (`Mod:handle(...)`, message sends, var-module spawns) is deliberately left unlinked rather than guessed. `codegraph_explore` also understands Erlang-native symbol spelling in queries — `mod:fn/3` and `init/2` find the symbols they name. (#635, #648)
+- CodeGraph now indexes **Erlang** (`.erl`, `.hrl`) — functions, with clauses and arities of the same name grouped as one symbol spanning all of them, plus records with their fields, `-type`/`-opaque` aliases, `-define` macros, and `-spec` signatures attached to every function. Cross-module `mod:fn(...)` calls resolve to the target module's function, `fun name/arity` values are captured as references (so callback registrations like `lists:foreach(fun submit/1, ...)` link up), `-include`/`-include_lib` connect to the header files they pull in, `-behaviour` declarations link a callback module to its behaviour (and only ever to a module — a same-named macro or function elsewhere in the repo is never mistaken for one), and `-export` lists (plus `-compile(export_all)`) drive each function's public/private flag. OTP's indirection idioms are followed where the target is static: `spawn`/`apply`/`proc_lib`/`timer`/`rpc` calls that name their target as `(Module, Function, Args)` arguments produce call edges, and `gen_server:call`/`cast` connects to the target module's `handle_call`/`handle_cast` — its own when targeting `?MODULE` (including the `-define(SERVER, ?MODULE)` idiom), and the named module when a registered name follows OTP's name-the-server-after-its-module convention (`gen_server:call(other_mod, ...)`, directly or through a `-define(STORE, other_mod)` macro); a registered name that matches no module stays unlinked. Macros participate in the graph too: a `-define` body's calls belong to the macro, each `?MACRO(...)` use site links into the call chain (and bare `?CONSTANT` reads are tracked as references), so a call path hidden behind a macro — `set_password → ?SQL_UPSERT_T → sql_query_t` — traces end-to-end and "where is this macro used" is answerable. escripts index like any module (the shebang line is understood), and OTP application resource files (`.app.src`, `.app`) join the graph: `{mod, ...}` links an app to its callback module and `{applications, [...]}` connects umbrella sibling apps — resolving only ever to modules, so an OTP app name like `ssl` is never mistaken for a same-named function. Truly dynamic dispatch (`Mod:handle(...)`, message sends, var-module spawns) is deliberately left unlinked rather than guessed. `codegraph_explore` also understands Erlang-native symbol spelling in queries — `mod:fn/3` and `init/2` find the symbols they name. (#635, #648)
 - CodeGraph now indexes **Visual Basic .NET** (`.vb`) — classes, Modules, interfaces, structures, enums, properties, events, `MustOverride` abstract members, and `Declare` P/Invoke signatures, with `Inherits`/`Implements` hierarchy edges, call edges (resolved through VB's ambiguous call-vs-index parentheses), and `New`/`As New` instantiation links. Real-world VB styles parse cleanly: WinForms designer files, interpolated and multi-line strings, XML literals (embedded `<%= %>` expressions included), single-line and multi-line LINQ queries, multi-line lambdas, `Handles`/`WithEvents` event wiring, Custom Events, date literals, classic type-character identifiers (`i%`, `name$`), and non-English (Unicode) identifiers. (#648, #639, #170)
 - CodeGraph now indexes **COBOL** (`.cbl`, `.cob`, `.cpy`) — programs, sections and paragraphs with `PERFORM`/`GO TO` call edges, `CALL` cross-program calls, `COPY` copybook imports (standalone copybooks included), and DATA DIVISION records with 88-level condition names, in both fixed and free source format. Impact queries work on data items: every `MOVE`/`ADD`/`COMPUTE`/`SUBTRACT` write-site links back to the field it changes, so "what touches this copybook field" answers across programs. CICS flows connect too: `EXEC CICS LINK`/`XCTL` program targets, `EXEC SQL INCLUDE` copybooks, and pseudo-conversational `RETURN TRANSID(...)` hops resolve to the program owning the transaction id. (#590, #648)
 - CodeGraph now indexes **CFML** (`.cfc`, `.cfm`, `.cfs`) — both the classic tag-based style (`<cfcomponent>`/`<cffunction>`) and modern bare-script `component { ... }` syntax, including `extends`/`implements`, embedded `<cfscript>` blocks (at any nesting depth, including inside `<cfif>`/`<cfloop>`/`<cftry>`), call edges, and calls embedded in `#hash#` expressions inside `<cfquery>` SQL bodies. Files saved with a UTF-8 byte-order mark and tags with unquoted attribute values — both common in long-lived CFML codebases — are handled too. Thanks @ghedwards. (#1118)

README.md

@@ -718,7 +718,7 @@ is written):
 | CFML | `.cfc`, `.cfm`, `.cfs` | Full support (tag-based `<cfcomponent>`/`<cffunction>` and bare-script `component { ... }` styles, `extends`/`implements`, embedded `<cfscript>` delegation, call edges) |
 | COBOL | `.cbl`, `.cob`, `.cpy` | Full support (programs, sections/paragraphs with PERFORM/GO TO call edges, CALL 'literal' cross-program calls, COPY copybook imports — including standalone `.cpy` files — DATA DIVISION records/fields/88-levels, EXEC CICS LINK/XCTL and EXEC SQL INCLUDE targets; fixed and free format) |
 | Visual Basic .NET | `.vb` | Full support (classes, Modules, interfaces, structures, enums, properties, events, `Declare` P/Invoke, `Handles`/`WithEvents`, `Inherits`/`Implements` edges, call edges through VB's call/index paren ambiguity, `As New` instantiation, interpolated strings, LINQ, Unicode identifiers) |
-| Erlang | `.erl`, `.hrl` | Full support (functions with multi-clause/multi-arity grouping, `-spec` signatures, records with fields, `-type`/`-opaque` aliases, `-define` macros, `-include`/`-include_lib`/`-import` edges, local and `mod:fn` remote call edges, `fun name/arity` references, `spawn`/`apply`/`proc_lib`/`timer`/`rpc` MFA-argument call edges, `gen_server:call/cast(?MODULE)` → own `handle_call`/`handle_cast` links, `-behaviour` links, `-export`-based visibility) |
+| Erlang | `.erl`, `.hrl`, `.escript`, `.app.src`, `.app` | Full support (functions with multi-clause/multi-arity grouping, `-spec` signatures, records with fields, `-type`/`-opaque` aliases, `-define` macros, `-include`/`-include_lib`/`-import` edges, local and `mod:fn` remote call edges, `fun name/arity` references, `spawn`/`apply`/`proc_lib`/`timer`/`rpc` MFA-argument call edges, `gen_server:call/cast(?MODULE)` → own `handle_call`/`handle_cast` links, `-behaviour` links, `-export`-based visibility) |
 
 ## Measured cross-file coverage
 

__tests__/extraction.test.ts

@@ -110,6 +110,14 @@ describe('Language Detection', () => {
   it('should detect Erlang files', () => {
     expect(detectLanguage('src/my_server.erl')).toBe('erlang');
     expect(detectLanguage('include/records.hrl')).toBe('erlang');
+    expect(detectLanguage('bin/release_tool.escript')).toBe('erlang');
+    // OTP app resource files route by full suffix — `.src` alone is too generic.
+    expect(detectLanguage('src/myapp.app.src')).toBe('erlang');
+    expect(detectLanguage('ebin/myapp.app')).toBe('erlang');
+    expect(detectLanguage('legacy/module.src')).toBe('unknown');
+    expect(isSourceFile('src/myapp.app.src')).toBe(true);
+    expect(isSourceFile('ebin/myapp.app')).toBe(true);
+    expect(isSourceFile('legacy/module.src')).toBe(false);
   });
 
   it('should return unknown for unsupported extensions', () => {
@@ -9134,6 +9142,54 @@ second(X) -> X.
     });
   });
 
+  describe('escript and app resource files', () => {
+    it('should extract functions and calls from an escript behind a shebang', () => {
+      const code = `#!/usr/bin/env escript
+%%! -smp enable
+
+main([Path]) ->
+    Result = analyze(Path),
+    io:format("~p~n", [Result]).
+
+analyze(Path) ->
+    {ok, Bin} = file:read_file(Path),
+    byte_size(Bin).
+`;
+      const result = extractFromSource('bin/tool.escript', code);
+      const fns = result.nodes.filter((n) => n.kind === 'function').map((n) => n.name);
+      expect(fns).toContain('main');
+      expect(fns).toContain('analyze');
+      const calls = result.unresolvedReferences.filter((r) => r.referenceKind === 'calls').map((r) => r.referenceName);
+      expect(calls).toContain('analyze');
+      expect(calls).toContain('io::format');
+    });
+
+    it('should link an app resource file to its callback module and dependency apps', () => {
+      const code = `{application, sample, [
+    {description, "Sample application"},
+    {vsn, "1.0.0"},
+    {registered, [sample_server]},
+    {mod, {sample_app, []}},
+    {applications, [kernel, stdlib, sample_core]},
+    {included_applications, [sample_extra]},
+    {env, [{limit, 100}]},
+    {modules, []}
+]}.
+`;
+      const result = extractFromSource('src/sample.app.src', code);
+      const refs = result.unresolvedReferences.map((r) => `${r.referenceKind}:${r.referenceName}`);
+      // The application-callback module is the app's entry point.
+      expect(refs).toContain('references:sample_app');
+      // Dependencies resolve to umbrella siblings; kernel/stdlib just drop.
+      expect(refs).toContain('imports:kernel');
+      expect(refs).toContain('imports:sample_core');
+      expect(refs).toContain('imports:sample_extra');
+      // Registered names, env values, and the like carry no graph structure.
+      expect(refs.filter((r) => r.endsWith(':sample_server'))).toHaveLength(0);
+      expect(refs.filter((r) => r.endsWith(':limit'))).toHaveLength(0);
+    });
+  });
+
   describe('Macro linkage', () => {
     it('should attribute macro-body calls to the macro and link function-like uses into the chain', () => {
       const code = `-module(m).

__tests__/resolution.test.ts

@@ -142,6 +142,35 @@ describe('Resolution Module', () => {
       // In-repo behaviour module resolves to its namespace.
       const resolved = matchReference(mkRef('my_behaviour'), context);
       expect(resolved?.targetNodeId).toBe(behaviourModule.id);
+
+      // The same module-only rule covers refs emitted by .app/.app.src
+      // resource files: on emqx, the `ssl` OTP app dependency resolved to a
+      // test helper FUNCTION named ssl. A colliding non-module name stays
+      // unresolved; a real umbrella-sibling module resolves.
+      nodes.push({
+        id: 'function:test/ldap_SUITE.erl:ssl:12',
+        kind: 'function',
+        name: 'ssl',
+        qualifiedName: 'ldap_SUITE::ssl',
+        filePath: 'test/ldap_SUITE.erl',
+        language: 'erlang',
+        startLine: 12,
+        endLine: 14,
+        startColumn: 0,
+        endColumn: 0,
+        updatedAt: Date.now(),
+      });
+      const appRef = (name: string) => ({
+        fromNodeId: 'file:src/myapp.app.src',
+        referenceName: name,
+        referenceKind: 'imports' as const,
+        line: 6,
+        column: 0,
+        filePath: 'src/myapp.app.src',
+        language: 'erlang' as const,
+      });
+      expect(matchReference(appRef('ssl'), context)).toBeNull();
+      expect(matchReference(appRef('my_behaviour'), context)?.targetNodeId).toBe(behaviourModule.id);
     });
 
     it('should prefer same-module candidates over cross-module matches', () => {

src/extraction/grammars.ts

@@ -140,6 +140,10 @@ export const EXTENSION_MAP: Record<string, Language> = {
   // tree-sitter-erlang grammar (the ELP grammar).
   '.erl': 'erlang',
   '.hrl': 'erlang',
+  // escripts parse natively — the grammar has a first-class `shebang` node.
+  // (`.app`/`.app.src` resource files route via isErlangAppFile below: their
+  // last-dot extension is too generic for this map.)
+  '.escript': 'erlang',
   // Spring config: `application.properties` / `application-*.properties`. Same
   // shape as the `.yml` variants — the YAML/properties extractor emits one node
   // per leaf key, and the Spring resolver links `@Value("${k}")` references.
@@ -158,6 +162,7 @@ export const EXTENSION_MAP: Record<string, Language> = {
 export function isSourceFile(filePath: string, overrides?: Record<string, Language>): boolean {
   if (isPlayRoutesFile(filePath)) return true; // Play `conf/routes` is extensionless
   if (isShopifyLiquidJson(filePath)) return true; // Shopify OS 2.0 JSON templates / section groups
+  if (isErlangAppFile(filePath)) return true; // OTP `.app`/`.app.src` resource files
   const dot = filePath.lastIndexOf('.');
   if (dot < 0) return false;
   const ext = filePath.slice(dot).toLowerCase();
@@ -175,6 +180,18 @@ export function isShopifyLiquidJson(filePath: string): boolean {
   return /(^|\/)(templates|sections)\/.+\.json$/i.test(filePath);
 }
 
+/**
+ * OTP application resource file: `<app>.app.src` (checked into every rebar3/
+ * erlang.mk app) or its compiled `<app>.app`. Erlang TERMS, not forms — the
+ * grammar parses them as top-level expressions, and the Erlang extractor's
+ * application-tuple handler turns `{mod, {Mod, _}}` and `{applications, […]}`
+ * into entry-module and dependency edges. Routed by full suffix because the
+ * last-dot extension (`.src`) is far too generic for EXTENSION_MAP.
+ */
+export function isErlangAppFile(filePath: string): boolean {
+  return /\.app(?:\.src)?$/i.test(filePath);
+}
+
 /**
  * Play Framework routes file: the extensionless `conf/routes` (and included
  * `conf/*.routes`). No grammar — route extraction is done by the Play framework
@@ -322,6 +339,9 @@ export function detectLanguage(filePath: string, source?: string, overrides?: Re
   // Shopify OS 2.0 JSON templates / section groups → the Liquid extractor (it
   // links each section `"type"` to its `sections/<type>.liquid`).
   if (isShopifyLiquidJson(filePath)) return 'liquid';
+  // OTP `.app`/`.app.src` resource files — Erlang terms the grammar parses as
+  // top-level expressions (last-dot ext `.src` is too generic for the map).
+  if (isErlangAppFile(filePath)) return 'erlang';
   const lang = (overrides && overrides[ext]) || EXTENSION_MAP[ext] || 'unknown';
 
   // .h files could be C, C++, or Objective-C — check source content

src/extraction/languages/erlang.ts

@@ -213,6 +213,52 @@ function handleBehaviour(node: SyntaxNode, ctx: ExtractorContext): boolean {
   return true;
 }
 
+/**
+ * OTP application resource file (`<app>.app.src` / `<app>.app`): a single
+ * `{application, Name, Props}.` term the grammar parses as a top-level
+ * expression. Two properties carry graph structure — `{mod, {Mod, _Args}}`
+ * names the application-callback module (the app's entry point), and
+ * `{applications, [...]}` / `{included_applications, [...]}` declare the apps
+ * this one depends on. In an umbrella repo those resolve to the sibling app's
+ * module of the same name (the OTP convention); kernel/stdlib and other
+ * out-of-repo apps stay unresolved.
+ */
+function handleAppResourceTuple(node: SyntaxNode, ctx: ExtractorContext): boolean {
+  const parentId = ctx.nodeStack[ctx.nodeStack.length - 1];
+  const props = node.namedChildren[2];
+  if (!parentId || props?.type !== 'list') return true;
+  const ref = (nameNode: SyntaxNode, kind: 'references' | 'imports'): void => {
+    const name = atomText(nameNode, ctx.source);
+    if (!name) return;
+    ctx.addUnresolvedReference({
+      fromNodeId: parentId,
+      referenceName: name,
+      referenceKind: kind,
+      line: nameNode.startPosition.row + 1,
+      column: nameNode.startPosition.column,
+    });
+  };
+  for (const prop of props.namedChildren) {
+    if (prop.type !== 'tuple' || prop.namedChildren.length < 2) continue;
+    const key = prop.namedChildren[0];
+    const value = prop.namedChildren[1];
+    if (!key || key.type !== 'atom' || !value) continue;
+    const keyName = atomText(key, ctx.source);
+    if (keyName === 'mod' && value.type === 'tuple') {
+      const mod = value.namedChildren[0];
+      if (mod?.type === 'atom') ref(mod, 'references');
+    } else if (
+      (keyName === 'applications' || keyName === 'included_applications') &&
+      value.type === 'list'
+    ) {
+      for (const app of value.namedChildren) {
+        if (app.type === 'atom') ref(app, 'imports');
+      }
+    }
+  }
+  return true; // nothing else in an app term carries graph structure
+}
+
 export const erlangExtractor: LanguageExtractor = {
   functionTypes: ['fun_decl'], // dispatched via visitNode (name lives on the clause)
   classTypes: [],
@@ -282,6 +328,18 @@ export const erlangExtractor: LanguageExtractor = {
       case 'spec':
       case 'callback':
         return true;
+      // `{application, Name, Props}.` at the top of an .app/.app.src resource
+      // file (never a valid form in a module, so the gate is file + position).
+      case 'tuple':
+        if (
+          node.parent?.type === 'source_file' &&
+          /\.app(?:\.src)?$/i.test(ctx.filePath) &&
+          node.namedChildren[0]?.type === 'atom' &&
+          atomText(node.namedChildren[0]!, ctx.source) === 'application'
+        ) {
+          return handleAppResourceTuple(node, ctx);
+        }
+        return false;
       default:
         return false;
     }

src/resolution/name-matcher.ts

@@ -1758,8 +1758,14 @@ export function matchReference(
   // `-behaviour(supervisor)` resolved to a `-define(supervisor, …)` macro
   // constant in an unrelated app. Resolve only to the behaviour module's
   // namespace; an out-of-repo behaviour (OTP's gen_server/supervisor) stays
-  // unresolved rather than guessed.
-  if (ref.language === 'erlang' && ref.referenceKind === 'implements') {
+  // unresolved rather than guessed. The same module-only rule applies to every
+  // ref an `.app`/`.app.src` resource file emits — its `{mod, …}` callback and
+  // `{applications, …}` dependency names can only mean modules, and on emqx
+  // the `ssl` OTP app otherwise resolved to a test helper FUNCTION named ssl.
+  if (
+    ref.language === 'erlang' &&
+    (ref.referenceKind === 'implements' || /\.app(?:\.src)?$/i.test(ref.filePath))
+  ) {
     const modules = context
       .getNodesByName(ref.referenceName)
       .filter((n) => n.language === 'erlang' && n.kind === 'namespace');