feat(extraction): add Visual Basic .NET language support (.vb) (#648, #639, #170) (#1164)

Vendored patched govindbanura/tree-sitter-vbnet grammar (MIT, ~20-fix patch
+ new external scanner for XML literals and multi-line LINQ continuation;
provenance + rebuild instructions in docs/grammars/tree-sitter-vbnet.md),
vbnet extractor with VB-specific call/index disambiguation, Inherits/
Implements heritage, As New instantiation, events, Declare P/Invoke, and
MustOverride abstract members.

Parse health on five real repos: PolicyPlus 100%, CompactGUI 100%,
staxrip 95.2%, SCrawler 87.2%, PCL 87.5% (upstream grammar: 3-18%).
Retrieval A/B (sonnet): 26-43% faster with 0-5 file reads vs 7-20 without.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Colby Mchenry
2026-07-03 11:55:45 -05:00
committed by GitHub
co-authored by Claude Fable 5
parent d7afc8cc1f
commit 63e1b5a23a
12 changed files with 1953 additions and 5 deletions
+23
View File
@@ -446,5 +446,28 @@
"files": "~270",
"question": "How does an incoming player chat message travel from packet handling to being broadcast to the other connected players? Name the programs on the path in order."
}
],
"VB.NET": [
{
"name": "policyplus",
"repo": "https://github.com/Fleex255/PolicyPlus",
"size": "Small",
"files": "~94",
"question": "When the user toggles a policy to Enabled in the policy-setting editor and clicks OK, how does the new state end up written into the loaded policy source (POL file or registry)? Trace the path from the EditSetting dialog to the concrete write."
},
{
"name": "scrawler",
"repo": "https://github.com/AAndyProgram/SCrawler",
"size": "Medium",
"files": "~320",
"question": "When a user download is started for a Reddit user, how does the request flow from the user-level download entry point through the shared downloader base into the Reddit site plugin, and where do downloaded media items get appended to the user's content list?"
},
{
"name": "staxrip",
"repo": "https://github.com/staxrip/staxrip",
"size": "Medium",
"files": "~145",
"question": "When a job finishes video encoding, how does staxrip decide which muxer runs and how does the muxer command line get built and executed? Trace from job processing to the mkvmerge invocation."
}
]
}
+1
View File
@@ -11,6 +11,7 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
### New Features
- CodeGraph now indexes **Visual Basic .NET** (`.vb`) — classes, Modules, interfaces, structures, enums, properties, events, `MustOverride` abstract members, and `Declare` P/Invoke signatures, with `Inherits`/`Implements` hierarchy edges, call edges (resolved through VB's ambiguous call-vs-index parentheses), and `New`/`As New` instantiation links. Real-world VB styles parse cleanly: WinForms designer files, interpolated and multi-line strings, XML literals (embedded `<%= %>` expressions included), single-line and multi-line LINQ queries, multi-line lambdas, `Handles`/`WithEvents` event wiring, Custom Events, date literals, classic type-character identifiers (`i%`, `name$`), and non-English (Unicode) identifiers. (#648, #639, #170)
- CodeGraph now indexes **COBOL** (`.cbl`, `.cob`, `.cpy`) — programs, sections and paragraphs with `PERFORM`/`GO TO` call edges, `CALL` cross-program calls, `COPY` copybook imports (standalone copybooks included), and DATA DIVISION records with 88-level condition names, in both fixed and free source format. Impact queries work on data items: every `MOVE`/`ADD`/`COMPUTE`/`SUBTRACT` write-site links back to the field it changes, so "what touches this copybook field" answers across programs. CICS flows connect too: `EXEC CICS LINK`/`XCTL` program targets, `EXEC SQL INCLUDE` copybooks, and pseudo-conversational `RETURN TRANSID(...)` hops resolve to the program owning the transaction id. (#590, #648)
- CodeGraph now indexes **CFML** (`.cfc`, `.cfm`, `.cfs`) — both the classic tag-based style (`<cfcomponent>`/`<cffunction>`) and modern bare-script `component { ... }` syntax, including `extends`/`implements`, embedded `<cfscript>` blocks (at any nesting depth, including inside `<cfif>`/`<cfloop>`/`<cftry>`), call edges, and calls embedded in `#hash#` expressions inside `<cfquery>` SQL bodies. Files saved with a UTF-8 byte-order mark and tags with unquoted attribute values — both common in long-lived CFML codebases — are handled too. Thanks @ghedwards. (#1118)
- CFML inheritance written as a component path now links to the right component. `extends="coldbox.system.web.Controller"` names its supertype by dotted path and `extends="../base"` by relative path (the FW/1 style) — both previously produced no inheritance edge at all, which on framework-style CFML apps hid most of the type hierarchy from impact and blast-radius analysis (on ColdBox's own core, over 90% of inheritance was invisible). Resolution is deliberately conservative: the target's directory layout must corroborate the declared path — so a supertype that lives in an out-of-repo library (testbox, mxunit, an installed framework) correctly stays unlinked rather than being guessed at, and an ambiguous path produces no edge rather than a wrong one. (#1152)
+2 -1
View File
@@ -244,7 +244,7 @@ The reliable, universal payoff is **surgical context and speed**: CodeGraph coll
| **Full-Text Search** | Find code by name instantly across your entire codebase, powered by FTS5 |
| **Impact Analysis** | Trace callers, callees, and the full impact radius of any symbol before making changes |
| **Always Fresh** | File watcher uses native OS events (FSEvents/inotify/ReadDirectoryChangesW) with debounced auto-sync — the graph stays current as you code, zero config |
| **20+ Languages** | TypeScript, JavaScript, Python, Go, Rust, Java, C#, PHP, Ruby, C, C++, Objective-C, Metal, Swift, Kotlin, Scala, Dart, Lua, Luau, R, CFML, COBOL, Svelte, Vue, Astro, Liquid, Pascal/Delphi |
| **20+ Languages** | TypeScript, JavaScript, Python, Go, Rust, Java, C#, VB.NET, PHP, Ruby, C, C++, Objective-C, Metal, Swift, Kotlin, Scala, Dart, Lua, Luau, R, CFML, COBOL, Svelte, Vue, Astro, Liquid, Pascal/Delphi |
| **Framework-aware Routes** | Recognizes web-framework routing files and links URL patterns to their handlers across 17 frameworks |
| **Mixed iOS / React Native / Expo** | Closes cross-language flows that static parsing misses: Swift ↔ ObjC bridging, React Native legacy bridge + TurboModules + Fabric view components, native → JS event emitters, Expo Modules |
| **100% Local** | No data leaves your machine. No API keys. No external services. SQLite database only |
@@ -717,6 +717,7 @@ is written):
| Luau | `.luau` | Full support (everything in Lua, plus `type`/`export type` aliases, typed signatures, and Roblox instance-path `require`) |
| CFML | `.cfc`, `.cfm`, `.cfs` | Full support (tag-based `<cfcomponent>`/`<cffunction>` and bare-script `component { ... }` styles, `extends`/`implements`, embedded `<cfscript>` delegation, call edges) |
| COBOL | `.cbl`, `.cob`, `.cpy` | Full support (programs, sections/paragraphs with PERFORM/GO TO call edges, CALL 'literal' cross-program calls, COPY copybook imports — including standalone `.cpy` files — DATA DIVISION records/fields/88-levels, EXEC CICS LINK/XCTL and EXEC SQL INCLUDE targets; fixed and free format) |
| Visual Basic .NET | `.vb` | Full support (classes, Modules, interfaces, structures, enums, properties, events, `Declare` P/Invoke, `Handles`/`WithEvents`, `Inherits`/`Implements` edges, call edges through VB's call/index paren ambiguity, `As New` instantiation, interpolated strings, LINQ, Unicode identifiers) |
## Measured cross-file coverage
+217
View File
@@ -8536,3 +8536,220 @@ DO-WORK.
expect(result.nodes.find((n) => n.kind === 'function')?.name).toBe('DO-WORK');
});
});
// =============================================================================
// VB.NET (.vb) — vendored patched govindbanura/tree-sitter-vbnet grammar
// =============================================================================
describe('VB.NET Extraction', () => {
it('should detect .vb as vbnet', () => {
expect(detectLanguage('Service.vb')).toBe('vbnet');
expect(detectLanguage('app/Forms/MainForm.vb')).toBe('vbnet');
expect(isSourceFile('Service.vb')).toBe(true);
});
const SAMPLE = `Imports System
Imports System.Collections.Generic
Namespace Acme.Billing
Public Interface IRepository
Function GetById(ByVal id As Integer) As Invoice
End Interface
Public Enum InvoiceState
Draft = 0
Sent
Paid
End Enum
Public Structure Money
Public Amount As Decimal
End Structure
Public MustInherit Class EntityBase
Public Property Id As Integer
End Class
Public Class Invoice
Inherits EntityBase
Implements IRepository
Private ReadOnly _lines As New List(Of String)
Public Const MaxLines As Integer = 100
Public Event Paid(ByVal amount As Decimal)
Public Property State As InvoiceState
Public Sub New(ByVal id As Integer)
Me.Id = id
End Sub
Public Function GetById(ByVal id As Integer) As Invoice Implements IRepository.GetById
Return New Invoice(id)
End Function
Public Sub AddLine(ByVal description As String)
_lines.Add(description)
Validate(description)
End Sub
Private Sub Validate(ByVal text As String)
If text.Length > MaxLines Then Throw New ArgumentException("too long")
End Sub
End Class
' lowercase keywords: VB is case-insensitive
public module Helpers
public function Twice(byval n as integer) as integer
return n * 2
end function
Public Sub Run()
Dim inv = New Invoice(1)
inv.AddLine("widget")
Dim d As New Dictionary(Of String, Integer)
Helpers.Twice(21)
End Sub
end module
End Namespace
`;
it('should extract classes, modules, interfaces, structures, and enums', () => {
const result = extractFromSource('Invoice.vb', SAMPLE);
const kinds = (kind: string) => result.nodes.filter((n) => n.kind === kind).map((n) => n.name);
expect(kinds('class')).toEqual(expect.arrayContaining(['EntityBase', 'Invoice', 'Helpers']));
expect(kinds('interface')).toContain('IRepository');
expect(kinds('struct')).toContain('Money');
expect(kinds('enum')).toContain('InvoiceState');
expect(kinds('enum_member')).toEqual(expect.arrayContaining(['Draft', 'Sent', 'Paid']));
});
it('should extract methods, constructors, properties, fields, and events (case-insensitive keywords)', () => {
const result = extractFromSource('Invoice.vb', SAMPLE);
const methods = result.nodes.filter((n) => n.kind === 'method').map((n) => n.name);
expect(methods).toEqual(expect.arrayContaining(['GetById', 'AddLine', 'Validate', 'Twice', 'Run']));
const props = result.nodes.filter((n) => n.kind === 'property').map((n) => n.name);
expect(props).toEqual(expect.arrayContaining(['Id', 'State']));
const fields = result.nodes.filter((n) => n.kind === 'field' || n.kind === 'constant').map((n) => n.name);
expect(fields).toEqual(expect.arrayContaining(['_lines', 'MaxLines']));
// Event declarations index as findable members
expect(fields).toContain('Paid');
});
it('should qualify types with their namespace', () => {
const result = extractFromSource('Invoice.vb', SAMPLE);
const invoice = result.nodes.find((n) => n.kind === 'class' && n.name === 'Invoice');
expect(invoice?.qualifiedName).toContain('Acme.Billing');
});
it('should emit Inherits as extends and Implements as implements references', () => {
const result = extractFromSource('Invoice.vb', SAMPLE);
const extendsRefs = result.unresolvedReferences.filter((r) => r.referenceKind === 'extends');
expect(extendsRefs.map((r) => r.referenceName)).toContain('EntityBase');
const implementsRefs = result.unresolvedReferences.filter((r) => r.referenceKind === 'implements');
expect(implementsRefs.map((r) => r.referenceName)).toContain('IRepository');
});
it('should extract calls through both invocation and index-shaped parens', () => {
const result = extractFromSource('Invoice.vb', SAMPLE);
const calls = result.unresolvedReferences.filter((r) => r.referenceKind === 'calls').map((r) => r.referenceName);
// `_lines.Add(description)` parses as array_access (non-empty parens) — still a call site
expect(calls).toContain('_lines.Add');
// bare call with args
expect(calls).toContain('Validate');
// qualified module call
expect(calls).toContain('Helpers.Twice');
});
it('should emit instantiates for New, with VB generic syntax stripped', () => {
const result = extractFromSource('Invoice.vb', SAMPLE);
const insts = result.unresolvedReferences.filter((r) => r.referenceKind === 'instantiates').map((r) => r.referenceName);
expect(insts).toContain('Invoice');
// `As New Dictionary(Of String, Integer)` → bare type name, not `Dictionary(Of ...)`
expect(insts.some((n) => n.includes('(') || /\bOf\b/.test(n))).toBe(false);
});
it('should extract Imports as import nodes', () => {
const result = extractFromSource('Invoice.vb', SAMPLE);
const imports = result.nodes.filter((n) => n.kind === 'import').map((n) => n.name);
expect(imports).toEqual(expect.arrayContaining(['System', 'System.Collections.Generic']));
});
it('should parse a file without a trailing newline (preParse guard)', () => {
const code = 'Class Tail\n Sub Go()\n Log("x")\n End Sub\nEnd Class';
const result = extractFromSource('Tail.vb', code);
expect(result.nodes.find((n) => n.kind === 'class')?.name).toBe('Tail');
expect(result.nodes.find((n) => n.kind === 'method')?.name).toBe('Go');
});
});
describe('VB.NET Extraction — scanner-backed constructs', () => {
it('should parse XML literals as opaque literals without breaking siblings', () => {
const code = `Class Muxer
Function WriteTags() As Object
Dim xml = <Tags>
<%= From tag In Tags Select <Tag><Name><%= tag.Name %></Name></Tag> %>
</Tags>
Return xml
End Function
Sub After()
Log("still extracted")
End Sub
End Class
`;
const result = extractFromSource('Muxer.vb', code);
const methods = result.nodes.filter((n) => n.kind === 'method').map((n) => n.name);
expect(methods).toEqual(expect.arrayContaining(['WriteTags', 'After']));
});
it('should parse multi-line LINQ query clauses', () => {
const code = `Class T
Function Big() As Integer
Dim big = From l In _lines
Where l.Length > 3
Select l.Length
Return big.Sum()
End Function
End Class
`;
const result = extractFromSource('Linq.vb', code);
const calls = result.unresolvedReferences.filter((r) => r.referenceKind === 'calls').map((r) => r.referenceName);
expect(calls).toContain('big.Sum');
expect(result.nodes.find((n) => n.kind === 'method')?.name).toBe('Big');
});
it('should extract MustOverride members without derailing following members', () => {
const code = `MustInherit Class VideoEncoder
MustOverride ReadOnly Property OutputExt As String
Public MustOverride Sub ShowConfigDialog(Optional param As Object = Nothing)
MustOverride Function GetError() As String
Sub New()
CanEdit = True
End Sub
End Class
`;
const result = extractFromSource('VideoEncoder.vb', code);
const methods = result.nodes.filter((n) => n.kind === 'method').map((n) => n.name);
expect(methods).toEqual(expect.arrayContaining(['ShowConfigDialog', 'GetError', 'New']));
const props = result.nodes.filter((n) => n.kind === 'property').map((n) => n.name);
expect(props).toContain('OutputExt');
});
it('should parse nullable declarator shorthand (Dim x? = expr)', () => {
const code = `Class T
Sub M(folderInfo As Object)
Dim SteamFolderData? = Parser.GetSteamNameAndID(folderInfo)
Use(SteamFolderData)
End Sub
End Class
`;
const result = extractFromSource('Factory.vb', code);
const calls = result.unresolvedReferences.filter((r) => r.referenceKind === 'calls').map((r) => r.referenceName);
expect(calls).toContain('Parser.GetSteamNameAndID');
});
});
+171
View File
@@ -0,0 +1,171 @@
# tree-sitter-vbnet.wasm — provenance & rebuild
`src/extraction/wasm/tree-sitter-vbnet.wasm` is built from
[govindbanura/tree-sitter-vbnet](https://github.com/govindbanura/tree-sitter-vbnet)
(MIT) at commit `538b7087bf80e86004531b392fe1186379c0a2b5` with the patch in
`tree-sitter-vbnet.patch` applied. The patch carries two files: `grammar.js`
(edits) and `src/scanner.c` (a new external scanner; upstream has none). The
upstream repo checks in no generated `src/`, so everything else is produced by
`tree-sitter generate`.
Alternatives considered: `CodeAnt-AI/tree-sitter-vb-dotnet` (22★) has **no
license file** and its git history stopped in July 2025 — unusable for
vendoring; `gabriel-gubert/tree-sitter-vbnet` is a 470-line VBScript-flavored
toy. The Roslyn-based approach (PR #627) was withdrawn by its author in favor
of tree-sitter — a Roslyn sidecar would add a .NET runtime dependency to a
local-first npm tool.
## What the patch adds
Upstream parses textbook VB.NET but fails on the constructs that dominate real
codebases (measured: 318% of files parsed clean across PolicyPlus, CompactGUI,
and staxrip before patching). Each item below was found by parse-error census
on those repos plus SCrawler and PCL:
1. **Generic type arguments in dotted names** — `System.Collections.Generic.
Dictionary(Of K, V)`, `Implements IRepository(Of Invoice)`, and
method-level `Implements I(Of T).Member` (generic segments were only
accepted unqualified). Open generic types (`GetType(LoaderTask(Of ,))`)
parse too.
2. **Interpolated strings** `$"… {expr[,align][:fmt]} …"` with `""`/`{{`/`}}`
escapes — including multi-line bodies and content pieces that begin with an
apostrophe: the pieces carry lexical precedence 101 (above `comment`'s 100)
because the comment **extra** otherwise fires *inside* the string rule and
eats the rest of the line, closing quote included.
3. **Date/time literals** `#1/15/2020#` — previously lexed as a preprocessor
directive that swallowed to end-of-line. Directives are now constrained to
`#` + letter (`#If`, `#Region`, …), which real directives always satisfy.
4. **VB 14 multi-line string literals** (a `"…"` literal may span lines since
VS 2015) and single-token `string_literal`/`character_literal` (`"["c`) —
the old multi-token form let extras interleave mid-string.
5. **Numeric literal forms** — hex/octal/binary (`&HFF`, `&O777`, `&B1010`),
digit separators (`1_000`), type characters (`6.0!`, `50.0#`, `1.5@`,
`123&`, `7%`), and lowercase `f/r/d` suffixes. WinForms `.Designer.vb`
files are full of `6.0!`.
6. **Identifier type characters and Unicode identifiers**`Dim i% = 0`,
`Dim r$ = …` (classic VB style, pervasive in SCrawler) and full Unicode
identifiers (`CrashReason.Java虚拟机参数有误` — PCL is written in Chinese).
The identifier token is now `[\p{L}\p{Nl}_][\p{L}\p{Nl}\p{Nd}\p{Mn}\p{Pc}]*
[%$&!#@]?` with the `u` regex flag. **The `u` flag requires
tree-sitter-cli ≥ 0.25** — 0.24.x silently drops the `\p{…}` classes.
7. **`As New T(args)` initializer clauses** — `as_clause` embeds a full
`object_creation_expression` for the `As New` form, so `Dim x As New
StringBuilder` / `Property P As New List(Of String)` produce instantiation
nodes. `Dim x? = expr` nullable declarators parse as well.
8. **Statement separators and single-line forms**`:` as a statement
terminator and block opener (`Class X : Inherits Y`, `Case 1 : Return "X"`),
single-line `If … Then stmt Else stmt` (via terminator-less inline statement
variants, aliased to the normal statement node names), inline `RaiseEvent`,
and optional `Then` on block `If` and `ElseIf` (legal VB, used in staxrip).
9. **Multi-line lambdas**`Sub(…) … End Sub` / `Function(…) … End Function`
bodies (upstream had a statement-block body with no `End` closer, so every
block lambda broke its surrounding argument list), `Async`/`Iterator`
lambda modifiers, `ByVal`/`ByRef` lambda parameters, and single-line
`Sub() If cond Then …` statement bodies.
10. **Member declarations** — `Declare [Auto|Ansi|Unicode] Sub/Function … Lib
"dll" [Alias "…"]` P/Invoke declarations, `Custom Event … AddHandler/
RemoveHandler/RaiseEvent … End Event`, stacked attribute lines above one
member, property `= initializer` before `Implements`, type-less
auto-properties, and **`MustOverride` body-less methods and properties**:
`MustOverride` lexes as a dedicated token (removed from the
`member_modifier` alternation) that only `abstract_method_declaration` /
`abstract_property_declaration` accept, making the body-less parse
deterministic. (A GLR body-less alternative on `method_declaration` was
tried first and measurably poisoned error recovery — 100%→60% clean on
PolicyPlus — before being replaced with the token split.)
11. **Expressions** — VB 15 tuple literals `(a, b)`, array literals
`{1, 2, 3}` (plus nested `{{k, v}, …}` dictionary groups, replacing the
ambiguous upstream `dictionary_initializer`), omitted argument slots
(`f(a,, b)` — Optional parameters passed positionally), `TypeOf x IsNot T`,
generic method calls without parens (`items.OfType(Of Panel)`),
null-conditional indexing `x?(0)`, and `Global.`-qualified type names.
12. **LINQ queries** — query expressions no longer require a trailing
`Select`/`Group` clause, `Aggregate`-led queries, and
`Distinct`/`Skip`/`Take` clauses.
### External scanner (`src/scanner.c`, new)
Two constructs are not LR(1)-parseable with tree-sitter's newline-as-extra
treatment; both get external tokens:
- **`QUERY_CLAUSE_CONTINUATION`** — multi-line LINQ (`From x In xs`
`Where …`). At a clause boundary the newline alone cannot distinguish
"query continues on the next line" from "statement ends here". The scanner
looks past the newline run at the next word and emits the continuation
token only when it is a query-clause keyword (with a `Select Case`
guard), so the decision is made by the lexer instead of the LR table.
- **`XML_LITERAL`** — whole VB XML literals (`<Tags><Tag/></Tags>`) consumed
as one opaque token: element nesting, attributes, comments, CDATA,
processing instructions, and **nested** `<%= … %>` embedded expressions
(the staxrip `WriteTagfile` shape). Valid only where a literal can begin an
expression, so a relational `<` (which always *follows* an expression)
never collides. The scanner never skips a leading newline (it must remain
available as a statement terminator).
The scanner is stateless (serialize/deserialize are no-ops).
The `_eof` hack upstream (a literal-`$` token) cannot match a real
end-of-file, so files whose last line has no trailing newline would end with a
MISSING-newline error; the extractor's `preParse` appends a trailing newline
instead of patching that in the grammar.
## Measured parse health (at vendoring time)
| Corpus | Clean parses |
|---|---|
| Fleex255/PolicyPlus (94 `.vb`) | 94/94 (100%) — upstream: 3/94 |
| IridiumIO/CompactGUI (66) | 66/66 (100%) — upstream: 12/66 |
| staxrip/staxrip (145) | 138/145 (95.2%) — upstream: 22/145 |
| AAndyProgram/SCrawler (320) | 279/320 (87.2%) |
| Meloong-Git/PCL (112, Chinese identifiers) | 98/112 (87.5%) |
Known remaining gap (localized ERROR regions, deliberately unpatched):
- **Column-0 GoTo labels** (`Recheck:` at the start of a line inside indented
code — the classic VB label style, used heavily in PCL). The `word:`
keyword-extraction token interacts badly with a newline immediately followed
by a word at column 0, consuming the newline and dropping the previous
statement's terminator. Removing `word:` fixes labels but reintroduces
keyword-prefix identifier bugs corpus-wide (measured: staxrip 95%→28%), so
`word:` stays and column-0 labels keep a localized error; indented labels
parse fine. Worth an upstream tree-sitter investigation eventually.
## Rebuild
```bash
git clone https://github.com/govindbanura/tree-sitter-vbnet
cd tree-sitter-vbnet
git checkout 538b7087bf80e86004531b392fe1186379c0a2b5
git apply path/to/tree-sitter-vbnet.patch # patches grammar.js, adds src/scanner.c
# tree-sitter needs a tree-sitter.json (upstream ships none); grammar name is
# `vbnet` (C symbols tree_sitter_vbnet*):
cat > tree-sitter.json <<'JSON'
{
"grammars": [
{ "name": "vbnet", "camelcase": "Vbnet", "scope": "source.vbnet",
"path": ".", "file-types": ["vb"] }
],
"metadata": { "version": "0.1.0", "license": "MIT",
"description": "VB.NET grammar for tree-sitter",
"links": { "repository": "https://github.com/govindbanura/tree-sitter-vbnet" } }
}
JSON
npm install tree-sitter-cli@0.25.10 # ≥0.25 REQUIRED: the /u regex flag (Unicode
# identifiers) is dropped silently by 0.24.x
npx tree-sitter generate # src/scanner.c from the patch is picked up
npx tree-sitter build --wasm -o tree-sitter-vbnet.wasm # needs emscripten or Docker
```
Upstream's checked-in `test/corpus` expectations predate its own grammar.js
(every corpus test fails at the pinned commit, before any patching), so the
five-repo parse-health sweep above — plus 16 construct repros and the
`__tests__/extraction.test.ts` VB.NET block — is the regression baseline.
## Upstreaming
Not yet sent. The patch is one large, coherent "parse real-world VB.NET"
change; if upstream shows signs of life it can be offered as a PR the same way
the COBOL patch was ([tree-sitter-cobol#41](https://github.com/yutaro-sakamoto/tree-sitter-cobol/pull/41)),
with the corpus numbers above as the motivation. Until then,
`git apply tree-sitter-vbnet.patch` on upstream commit `538b708` reproduces
the vendored grammar exactly.
File diff suppressed because it is too large Load Diff
+6 -1
View File
@@ -43,6 +43,7 @@ const WASM_GRAMMAR_FILES: Record<GrammarLanguage, string> = {
cfscript: 'tree-sitter-cfscript.wasm',
cfquery: 'tree-sitter-cfquery.wasm',
cobol: 'tree-sitter-cobol.wasm',
vbnet: 'tree-sitter-vbnet.wasm',
};
/**
@@ -131,6 +132,9 @@ export const EXTENSION_MAP: Record<string, Language> = {
'.cob': 'cobol',
'.cobol': 'cobol',
'.cpy': 'cobol',
// VB.NET: vendored grammar (patched govindbanura/tree-sitter-vbnet) — classes,
// modules, interfaces, structures, properties, events, Handles clauses, LINQ.
'.vb': 'vbnet',
// Spring config: `application.properties` / `application-*.properties`. Same
// shape as the `.yml` variants — the YAML/properties extractor emits one node
// per leaf key, and the Spring resolver links `@Value("${k}")` references.
@@ -249,7 +253,7 @@ export async function loadGrammarsForLanguages(languages: Language[]): Promise<v
// `class Foo(...)` as an ERROR that swallows the whole class (#237); we
// vendor the upstream ABI-15 tree-sitter-c-sharp 0.23.5 wasm, which parses
// primary constructors natively.
const wasmPath = (lang === 'pascal' || lang === 'scala' || lang === 'lua' || lang === 'luau' || lang === 'csharp' || lang === 'r' || lang === 'cfml' || lang === 'cfscript' || lang === 'cfquery' || lang === 'cobol')
const wasmPath = (lang === 'pascal' || lang === 'scala' || lang === 'lua' || lang === 'luau' || lang === 'csharp' || lang === 'r' || lang === 'cfml' || lang === 'cfscript' || lang === 'cfquery' || lang === 'cobol' || lang === 'vbnet')
? path.join(__dirname, 'wasm', wasmFile)
: require.resolve(`tree-sitter-wasms/out/${wasmFile}`);
const language = await WasmLanguage.load(wasmPath);
@@ -468,6 +472,7 @@ export function getLanguageDisplayName(language: Language): string {
cfscript: 'CFScript',
cfquery: 'CFQuery (SQL)',
cobol: 'COBOL',
vbnet: 'Visual Basic .NET',
unknown: 'Unknown',
};
return names[language] || language;
+2
View File
@@ -30,6 +30,7 @@ import { objcExtractor } from './objc';
import { cfscriptExtractor } from './cfscript';
import { cfqueryExtractor } from './cfquery';
import { cobolExtractor } from './cobol';
import { vbnetExtractor } from './vbnet';
export const EXTRACTORS: Partial<Record<Language, LanguageExtractor>> = {
typescript: typescriptExtractor,
@@ -57,4 +58,5 @@ export const EXTRACTORS: Partial<Record<Language, LanguageExtractor>> = {
cfscript: cfscriptExtractor,
cfquery: cfqueryExtractor,
cobol: cobolExtractor,
vbnet: vbnetExtractor,
};
+137
View File
@@ -0,0 +1,137 @@
import type { Node as SyntaxNode } from 'web-tree-sitter';
import { getNodeText } from '../tree-sitter-helpers';
import type { LanguageExtractor } from '../tree-sitter-types';
/**
* The vendored VB.NET grammar has no true end-of-file token (its `_eof` rule is
* a literal-`$` placeholder that never matches real input), so a file whose
* last line lacks a trailing newline ends every parse with a MISSING-newline
* error on the final statement. Appending a newline is offset-preserving for
* all existing content.
*/
export function ensureTrailingNewline(source: string): string {
return source.endsWith('\n') ? source : source + '\n';
}
/** Case-insensitive member-modifier scan (VB keywords are case-insensitive). */
function hasModifier(node: SyntaxNode, re: RegExp): boolean {
for (let i = 0; i < node.childCount; i++) {
const child = node.child(i);
if (child?.type === 'member_modifier' && re.test(child.text)) return true;
}
return false;
}
/**
* A VB.NET method's declared return type (`Function Foo(...) As Bar`),
* normalized to the bare class name a chained `Foo.Create().Bar()` could be
* called on (the #645/#608 mechanism). The type lives in the method's
* `as_clause` child; predefined types (Integer/String/) and arrays yield
* undefined, generics `List(Of Foo)` unwrap to the base type, and a dotted
* `Ns.Foo` reduces to the simple name. Subs have no as_clause undefined.
*/
function extractVbnetReturnType(node: SyntaxNode, source: string): string | undefined {
const asClause = node.namedChildren.find((c: SyntaxNode) => c.type === 'as_clause');
if (!asClause) return undefined;
const typeNode = asClause.childForFieldName('declared_type');
if (!typeNode || typeNode.type === 'predefined_type' || typeNode.type === 'array_type') return undefined;
let t = getNodeText(typeNode, source).trim();
t = t.replace(/\?+$/, ''); // nullable `Foo?`
t = t.replace(/\(\s*Of\b[^)]*\)/gi, ''); // generics `List(Of Foo)` → `List`
const last = t.split('.').pop()?.trim();
if (!last || !/^[A-Za-z_]\w*$/.test(last)) return undefined;
return last;
}
export const vbnetExtractor: LanguageExtractor = {
preParse: ensureTrailingNewline,
functionTypes: [],
// VB Modules are static containers (Shared members, no instantiation) —
// indexed as classes so their members get normal containment/qualification.
classTypes: ['class_declaration', 'module_declaration'],
methodTypes: [
'method_declaration',
'constructor_declaration',
// `Declare Function GetWindowLong Lib "user32" ...` (P/Invoke)
'external_method_declaration',
// Interface members are distinct node types in this grammar (unlike C#).
'interface_method_declaration',
// `MustOverride Sub/Function ...` — body-less abstract members.
'abstract_method_declaration',
],
interfaceTypes: ['interface_declaration'],
structTypes: ['structure_declaration'],
enumTypes: ['enum_declaration'],
enumMemberTypes: ['enum_member_declaration'],
typeAliasTypes: ['delegate_declaration'],
packageTypes: ['namespace_declaration'],
extractPackage: (node: SyntaxNode, source: string) => {
const name = node.childForFieldName('name');
return name ? getNodeText(name, source) : null;
},
importTypes: ['imports_statement'],
// VB uses parentheses for BOTH calls and indexing, so the grammar can only
// split them heuristically (empty parens → invocation, args → array access;
// even Roslyn parses both as InvocationExpression and disambiguates during
// binding). Both are treated as call sites — extractCall has a vbnet branch
// — and name matching simply never resolves an index read on a collection.
callTypes: ['invocation_expression', 'array_access_expression', 'generic_invocation_expression'],
variableTypes: ['declaration_statement'],
fieldTypes: ['field_declaration'],
propertyTypes: ['property_declaration', 'interface_property_declaration', 'abstract_property_declaration'],
nameField: 'name',
bodyField: 'body',
paramsField: 'parameters',
// Method/property statements are direct children of the declaration node
// (this grammar has no body wrapper), so the node is its own body — without
// this, calls inside every Sub/Function would be skipped.
resolveBody: (node: SyntaxNode) => node,
getReturnType: extractVbnetReturnType,
getVisibility: (node) => {
if (hasModifier(node, /^private$/i)) return 'private';
if (hasModifier(node, /^protected(\s+friend)?$/i)) return 'protected';
if (hasModifier(node, /^friend$/i)) return 'internal';
return 'public'; // VB members default to Public in practice
},
isStatic: (node) => hasModifier(node, /^shared$/i),
isConst: (node) => hasModifier(node, /^const$/i) || (hasModifier(node, /^shared$/i) && hasModifier(node, /^readonly$/i)),
isAsync: (node) => hasModifier(node, /^async$/i),
extractImport: (node, source) => {
const importText = source.substring(node.startIndex, node.endIndex).trim();
// `Imports System.Collections.Generic` / `Imports Alias = Some.Namespace` /
// `Imports Global.Company.Product`. The name reference is the last
// qualified/simple/global name child (skips the alias identifier).
const nameNode = [...node.namedChildren]
.reverse()
.find((c: SyntaxNode) =>
c.type === 'qualified_name' || c.type === 'simple_name' || c.type === 'global_qualified_name' || c.type === 'identifier'
);
if (nameNode) {
return { moduleName: getNodeText(nameNode, source), signature: importText };
}
return null;
},
visitNode: (node, ctx) => {
// Events are indexed so `RaiseEvent X` / `Handles obj.X` flows have a
// findable declaration (WinForms/WPF code is built around them).
if (node.type === 'event_declaration' || node.type === 'custom_event_declaration') {
const nameNode = node.childForFieldName('name');
if (nameNode) {
ctx.createNode('field', getNodeText(nameNode, ctx.source), node);
}
return true;
}
// `Sub New(...)` lexes as one token with no name field — without this,
// constructors index as `<anonymous>`.
if (node.type === 'constructor_declaration') {
const ctor = ctx.createNode('method', 'New', node);
if (ctor) {
ctx.pushScope(ctor.id);
ctx.visitFunctionBody(node, ctor.id);
ctx.popScope();
}
return true;
}
return false;
},
};
+99 -2
View File
@@ -1153,7 +1153,7 @@ export class TreeSitterExtractor {
// produce an `instantiates` reference. Children still walked so
// nested calls inside the constructor args (`new Foo(bar())`) get
// their own `calls` refs.
else if (INSTANTIATION_KINDS.has(nodeType)) {
else if (INSTANTIATION_KINDS.has(nodeType) || this.isVbnetConstructorShapedArrayCreation(node)) {
this.extractInstantiation(node);
// Java/C# `new T(...) { ... }` — anonymous class with body. Without
// extracting it as a class node + its methods, the interface→impl
@@ -3498,6 +3498,51 @@ export class TreeSitterExtractor {
const callerId = this.nodeStack[this.nodeStack.length - 1];
if (!callerId) return;
// VB.NET: `foo(args)` is syntactically ambiguous between a call and an
// index read, so the grammar parses non-empty parens as
// array_access_expression (field `array`, not `function`) — even Roslyn
// parses both as InvocationExpression and resolves during binding. Treat
// all three shapes as call sites: the callee is the member/identifier
// under the array/function field, qualified with a simple-identifier
// receiver for resolution. Index reads on collections simply never
// resolve to a callable, so they cost nothing.
if (
this.language === 'vbnet' &&
(node.type === 'array_access_expression' ||
node.type === 'invocation_expression' ||
node.type === 'generic_invocation_expression')
) {
const fn = getChildByField(node, 'function') || getChildByField(node, 'array');
if (!fn) return;
let calleeName = '';
if (fn.type === 'member_access_expression') {
const member = getChildByField(fn, 'member');
const memberName = member ? getNodeText(member, this.source) : '';
if (!memberName) return;
const receiver = getChildByField(fn, 'object');
const SKIP = new Set(['me', 'mybase', 'myclass']);
if (receiver && receiver.type === 'identifier' && !SKIP.has(getNodeText(receiver, this.source).toLowerCase())) {
calleeName = `${getNodeText(receiver, this.source)}.${memberName}`;
} else {
calleeName = memberName;
}
} else if (fn.type === 'identifier') {
calleeName = getNodeText(fn, this.source);
} else {
return; // parenthesized/chained receivers: no static name to link
}
if (calleeName) {
this.unresolvedReferences.push({
fromNodeId: callerId,
referenceName: calleeName,
referenceKind: 'calls',
line: node.startPosition.row + 1,
column: node.startPosition.column,
});
}
return;
}
// Ruby `call` nodes use `receiver` + `method` fields (tree-sitter-ruby), not
// the `object`/`name`/`function` fields the branches below expect — so
// without this they fell through to the generic path, which took the
@@ -3904,6 +3949,24 @@ export class TreeSitterExtractor {
* Children are still walked so nested calls inside the constructor
* arguments (`new Foo(bar())`) get their own `calls` references.
*/
/**
* VB.NET `New Invoice(1)` is syntactically ambiguous between constructing
* Invoice with an argument and allocating an Invoice array of bound 1; the
* grammar parses the parenthesized form as array_creation_expression. A
* user-defined type with no `{...}` array initializer is overwhelmingly a
* constructor call, so treat it as an instantiation. Predefined element
* types (`New Byte(1023)`) and brace-initialized forms stay arrays.
*/
private isVbnetConstructorShapedArrayCreation(node: SyntaxNode): boolean {
if (this.language !== 'vbnet' || node.type !== 'array_creation_expression') return false;
const typeNode = getChildByField(node, 'type');
if (!typeNode || typeNode.type === 'predefined_type' || typeNode.type === 'array_type') return false;
for (const child of node.namedChildren) {
if (child?.type === 'array_initializer') return false;
}
return true;
}
private extractInstantiation(node: SyntaxNode): void {
if (this.nodeStack.length === 0) return;
const fromId = this.nodeStack[this.nodeStack.length - 1];
@@ -3965,6 +4028,13 @@ export class TreeSitterExtractor {
// because no class is named with the angle-bracket suffix.
const ltIdx = className.indexOf('<');
if (ltIdx > 0) className = className.slice(0, ltIdx);
// VB.NET spells generics with parentheses: `New List(Of String)` /
// `New Dictionary(Of K, V)(cap)` — strip from the `(` so the bare
// type name is what resolution matches.
if (this.language === 'vbnet') {
const parenIdx = className.indexOf('(');
if (parenIdx > 0) className = className.slice(0, parenIdx);
}
// For namespaced/qualified constructors (`new ns.Foo()`,
// `new ns::Foo()`) keep the trailing identifier — that's what
// matches a class node in the index.
@@ -4374,7 +4444,7 @@ export class TreeSitterExtractor {
if (this.extractor!.callTypes.includes(nodeType)) {
this.extractCall(node);
} else if (INSTANTIATION_KINDS.has(nodeType)) {
} else if (INSTANTIATION_KINDS.has(nodeType) || this.isVbnetConstructorShapedArrayCreation(node)) {
// `new Foo()` inside a function body — emit an `instantiates`
// reference. Without this branch the body walker only knew
// about `call_expression`, so constructor invocations
@@ -4747,6 +4817,33 @@ export class TreeSitterExtractor {
}
}
// VB.NET: `Inherits Base` / `Implements IFoo, IBar(Of T)` are STATEMENTS
// inside the class body (children of the class node), not header clauses.
// Each name is a simple/qualified/generic reference; generics unwrap to
// the base identifier and dotted paths keep the trailing segment.
if (
this.language === 'vbnet' &&
(child.type === 'inherits_statement' || child.type === 'implements_statement')
) {
const kind = child.type === 'inherits_statement' ? 'extends' : 'implements';
for (const ref of child.namedChildren) {
if (!ref || (ref.type !== 'simple_name' && ref.type !== 'qualified_name' && ref.type !== 'generic_name' && ref.type !== 'global_qualified_name')) continue;
let name = getNodeText(ref, this.source);
name = name.replace(/\(\s*Of\b[^)]*\)/gi, '');
const lastDot = name.lastIndexOf('.');
if (lastDot >= 0) name = name.slice(lastDot + 1);
name = name.trim();
if (!name) continue;
this.unresolvedReferences.push({
fromNodeId: classId,
referenceName: name,
referenceKind: kind,
line: ref.startPosition.row + 1,
column: ref.startPosition.column,
});
}
}
// C#: `class Movie : BaseItem, IPlugin` → base_list with identifier children
// base_list combines both base class and interfaces in a single colon-separated list.
// We emit all as 'extends' since the syntax doesn't distinguish them.
Binary file not shown.
+1
View File
@@ -99,6 +99,7 @@ export const LANGUAGES = [
'cfscript',
'cfquery',
'cobol',
'vbnet',
'unknown',
] as const;