main
278
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  
|
09e301bbfa |
feat(kernel): R7b Swift walker — swift module, tree-sitter-swift 0.7.3 bump, swift default-routed (#1381)
Fifth R7b batch-3 port, checklist-first recipe (docs/design/swift-kernel-port-checklist.md, 1,056 lines — the largest of the arc, with a built-extractor-validated emission pin and a childForFieldName truth table). Grammar bump first, validated standalone: tree-sitter-wasms ^0.4.0 (ABI 13) → crate 0.7.3 — with a provenance twist: the wasm is built from the CRATE TARBALL's src/ (alex-pinkus keeps generated files off main and the 0.7.3-with-generated-files tag ships an older ABI-14 generation that can never sha-match; grammar.json rules are JSON-equal; the tarball is byte-for-byte what the kernel's cargo build compiles — table identity by construction). Older crates evaluated and rejected: clean-parse shapes are byte-identical on 0.7.3 (53-line CST battery diff, all inert), so an older pin buys nothing and loses the macro-era wins. Delta = error-set membership (63 old-error files parse clean: swift-testing #expect, #Preview/#GET macros, package access, typed throws — vapor 23.1%→9.3%; 21 NEW-only regressions in 3 probed construct classes) + two gate-found categories: docstring boundaries near #if directives (7 clean files, docstring-field-only — verified mechanically) and array-literal-callee call refs (2 refs, 1 file). Every hunk classified via the error-union rule + parked-ref↔edge ripple pairing. Walker (the arc's biggest) centers on the #1020 DEDICATED property branch: computed properties → property nodes with the getter walked under the property (SwiftUI body), static let/var → constant/variable, stored → field, decorator/type-annotation/@Siblings-attr-arg refs all attached to the ENCLOSING TYPE, stored initializer calls attributed to the class. Preserved bug-for-bug: the never-resolving 'parameter' field (zero param type refs, zero signatures), present-false isAsync, open→internal visibility, everything-is-extends inheritance (first type_identifier per specifier), no instantiates refs ever, subscript reads as `calls arr`, `defer` as `calls defer`, multi-case enum entries minting only the first case, /** */ block docs ignored AND chain-breaking, init/deinit/subscript minting no nodes with visitNode-routed bodies (calls → class, static reads → nothing), multi- segment extension resolveName, sugar extension names, the #selector shapes, and the value_argument label-forward skip. ONE fix found by the sweep (then pinned in the fixture + checklist): the shared `assignment` shadow-prune case is swift-live — declared-then-assigned `let X: T` prunes X as a value-ref target. Gates: sweeps 0-diff Alamofire 89/98, vapor 224/247, swift-nio 407/554 (--max-deferral 0.3 — swift error incidence is 9–27% on BOTH arms, structural; every deferral count matches the survey's table exactly); full-init dumps byte-identical ×3 (31.9k/20.7k/126.3k lines); the Alamofire census reproduces property=348 (the #1020 number) on the kernel arm; kernel-swift-parity suite (206-line torture + CRLF + the #if-between-enum- cases defer fixture) + swift grammar-parity row; full suite 2,626 green ×2 under CODEGRAPH_KERNEL_EXPECT=1. DEFAULT_ROUTED += swift (14 langs). Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
a6c62d77df |
feat(kernel): R7b PHP walker — php module, tree-sitter-php 0.24.2 bump, php default-routed (#1380)
Fourth and final R7b batch-2 port, checklist-first recipe (docs/design/php-kernel-port-checklist.md). Grammar bump first, validated standalone with the diff ENUMERATED + CLASSIFIED (unlike rust/ruby the php bump is NOT graph-neutral): tree-sitter-php ^0.22 (tree-sitter-wasms, 2023) → v0.24.2, the full HTML-interleaving `php` variant (the walker calls LANGUAGE_PHP, never PHP_ONLY) — crate pinned =0.24.2, wasm built from tag 5b5627f's checked-in php/src/parser.c + scanner.c + shared common/scanner.h (all sha-matched against the crates.io tarball, ABI 14→15). Old-vs-new full-init diffs decompose completely into: (1) the anonymous_class wrapper shape (anon-class nodes/methods re-shape — 2,532 rows), (2) grouped nested-clause skip (absent in the gate repos, fixture-pinned), (3) 32 formerly-erroring files parsing clean (monolog Level.php, symfony Request/Response with 8.4 property hooks), (4) a survey-missed category found at gate time: the 8.4 parenthesis-free `new X()->m()` chaining misparse fix (86 garbage instantiates refs disappear, precision-positive), plus resolution RIPPLE proven mechanically (every remaining ref-table flip pairs 1:1 with a resolved edge on the opposite side; node rows byte-stable outside 1/3/4). Walker (java.rs chassis + the php specifics) preserves bug-for-bug: the visitNode hook (const_declaration at ANY scope → bare `constant` nodes, values never walked; trait-use → implements refs WITH filePath via the ruby port's REF_FLAG_FILE_PATH wire slot), FIRST-namespace whole-file scoping (braced namespaces scope nothing; namespaced files DROP top-level const value-ref targets), the import trio (single/aliased/grouped incl. the nested-clause skip, include/require static-literal-only, `Foo\Bar::Baz` use refs), the call-encoding zoo (DOT-joined scoped calls, `this->prop.m` #1251 encoding, `Cls::factory().m` fluent with inner args dropped, nullsafe `?->` emitting nothing, unsuppressed literal receivers), interface multi-extends first-base-only drop, anon-class methods as file-level functions (top) or vanishing (in-body), property type-hints emitting no field refs, the final-modifier-as-type signature quirk, HOF-gated string callables (skipGate) + array callables, and the `name`-node value-ref reader. Gates: sweeps 0-diff monolog 217/217, laravel-framework 3007/3008, symfony 10726/10737 (13,950 files byte-parity; 12 deferrals = exactly the predicted genuinely-broken fixtures, ≈0–0.1%); full-init dumps byte-identical ×3 (16.1k/354.2k/702.8k lines); kernel-php-parity suite (torture + drupal .module + leading-HTML fixtures, CRLF variants, wire-flag pin, defer) + php grammar-parity row; full suite 2,622 green ×2 under CODEGRAPH_KERNEL_EXPECT=1. DEFAULT_ROUTED += php (13 languages). Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
1909931238 |
feat(kernel): R7b Ruby walker — ruby module, tree-sitter-ruby 0.23.1 bump, ref-flag wire slot, ruby default-routed (#1379)
Third R7b port, checklist-first recipe (docs/design/ruby-kernel-port-checklist.md). Grammar bump first, validated standalone (the rust pattern): tree-sitter-ruby ^0.20.1 (tree-sitter-wasms, 2024-02) → v0.23.1 — crate pinned =0.23.1, wasm built from tag 71bd32f's checked-in parser.c/scanner.c (both sha-matched against the crates.io tarball; content bump, ABI stays 14). Old-vs-new full-init dumps: sinatra/jekyll byte-identical; rails = exactly the one classified hunk (the `recv&.!=` safe-nav operator misparse fix, `table_name.!` → `table_name.!=`, precision-positive). Walker (python.rs chassis + the six ruby divergences) preserves bug-for-bug: the importTypes:['call'] funnel (class-body DSL — attr_accessor, has_many, define_method incl. its block, sinatra route blocks — emits NOTHING at non-body scope), hook-handled module multiply-capture (nested modules re-scan their subtree per level after popping — `this.hooked` fn-refs from class AND module AND file), the sibling-scan visibility trio (bare `private` invisible; `private :sym`/`private def` poison all later defs; the inner def stays public), bare-call statements (do…end body_statement emits, brace-block block_body doesn't), `.new` instantiates with last-`::`-segment names, constant-receiver references refs, require/require_relative path refs (posix-normalized, `.rb`-suffixed, `Kernel.require` and interpolated-path quirks included), `=begin` docstring marker survival, and the reverse-order value-ref DFS. Wire v2: the hook's mixin `implements` refs carry `filePath: ctx.filePath` — the ONE extraction-ref denormalized field (php's trait-use refs share the shape). RefRow's first pad byte becomes a flags slot (REF_FLAG_FILE_PATH); decode re-attaches its own filePath parameter; KERNEL_ABI_VERSION 1→2 on both sides (mismatched dist/.node pairs degrade to wasm, as designed). Gates: sweeps 0-diff sinatra 147/147, jekyll 164/164, rails 3452/3452 (3,763 files, 0 deferrals — ruby error incidence 0.00%, any deferral = walker bug); full-init dumps byte-identical ×3 (7.2k/9.4k/375.6k lines); kernel-ruby-parity suite (torture + CRLF + wire-flag pin + defer) + ruby grammar-parity row; full suite 2,613 green ×2 under CODEGRAPH_KERNEL_EXPECT=1 (one unrelated mcp-initialize timing flake under parallel load, passes solo 3/3 ×3). DEFAULT_ROUTED += ruby (12 langs). Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
286e9ccc2d |
feat(kernel): R7b C# walker — csharp module, tree-sitter-c-sharp 0.23.5 pin, csharp default-routed (#1378)
Second R7b port, checklist-first recipe (docs/design/csharp-kernel-port-checklist.md; parity passed FIRST RUN again). No grammar bump — the #717 vendored wasm verified table-identical to crate 0.23.5 (ABI 15, STATE_COUNT 8053, node-kind + field tables); first port with no grammar-prep step. The #237 #if-blanking preParse stays TS-side via the existing route-point hoist. Walker preserves bug-for-bug: the single-namespace-node quirks (second namespace nests under the first, nested namespaces leave no trace, import refs hang off the namespace node), raw member-access callee texts (this./base./literal receivers, multi-line fluent chains) with unconditional chain re-encode, deliberate emission holes (property accessor/expression bodies, ctor initializers, delegates/events/ operators/indexers/local functions, top-level locals), garbage extends refs ((repo) primary-ctor args, BaseDto(Name) record bases, enum : byte), the alias- import moduleName quirks, nameof-as-call, CSHARP fn-ref spec (+= subscription, this.X bare-name form, argument layer, initializer lists), C# type-ref engine (nested-generic returnType failure included), and value-ref shadow pruning. Gates: sweeps 0-diff serilog 211/216 / Newtonsoft.Json 914/945 / jellyfin 2104/2105 (deferrals match the survey's per-repo predictions — both-arm #if damage; default --max-deferral 0.1 holds, no c/cpp exemption); full-init dumps byte-identical ×3 (14.0k/109.1k/210.8k lines); kernel-csharp-parity suite (torture ×3 + CRLF variants + 8 micro-pins + defer) + csharp grammar-parity row; full suite 2,608 ×2 under CODEGRAPH_KERNEL_EXPECT=1. DEFAULT_ROUTED += csharp (11 langs). Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
f1ca991943 |
feat(kernel): R7b Rust walker — rustlang module, tree-sitter-rust 0.24.2 bump, rust default-routed (#1371)
First R7b language port. Grammar: tree-sitter-rust pinned =0.24.2 + wasm vendored from tag 77a3747 (parser.c/scanner.c sha-matched against the crates.io tarball), replacing the 2023 ABI-14 tree-sitter-wasms build — the bump alone is precision-positive on the wasm path (receiver-qualified instance-method resolutions replace ambiguous bare-name matches; node sections byte-identical on ripgrep/tokio). Walker mirrors the TS reference bug-for-bug per docs/design/rust-lang-kernel-port-checklist.md (survey artifact): dead-code isAsync, impl-pushes-no-scope, the impl-Trait-for-Generic<T> trait-receiver quirk, phantom const identifiers, use-binding triple emission, wildcard-use-emits-nothing, scoped-supertrait drop, chained-call re-encode gated on scoped_identifier, Rocket route macros body-only. Gates: parity sweeps 0 diffs — ripgrep 101/101, tokio 790/790, rust-analyzer 1217/1488 (271 deferrals are token-macro-table sources that error on BOTH arms — grammar-inherent); full-init dump-diffs byte-identical on all three (3,857 / 13,440 / 39,030 nodes); kernel-rustlang-parity suite (torture + CRLF + defer) in npm test; full suite green x2 with CODEGRAPH_KERNEL_EXPECT=1. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
69ea438bac |
perf(kernel): cFnPtr native extraction sweep — step 2, pass 230→151s across the arc (§7a.10) (#1365)
Task #5 step 2. The fuse-then-link refactor (#1364) left the extraction sweep as a clean per-file boundary: raw text in → collected facts out. This ports that sweep to the native kernel: `cfnptr_scan_files` (codegraph-kernel/src/cfnptr.rs) strips and scans a batch of 16 files per NAPI call, and the TS side only reads files, ships batches, interns the returned facts, and resolves include paths. The JS sweep remains as the fallback (no binary, feature detection against older binaries, CODEGRAPH_KERNEL=0, or CODEGRAPH_KERNEL_CFNPTR=0). Parity discipline: the JS regexes are the spec, so the scanners are hand-rolled byte machines reproducing that engine — ASCII \w/\b next to UNICODE \s (NBSP/U+2000-200A/FEFF decoded from UTF-8), alternation order, lastIndex resume, and the observable backtracking dimensions (INIT/ARRAY modifier and struct/star/bracket optionals, DISPATCH's greedy segment loop); greedy shortcuts only where backtracking provably can't rescue a match. The native stripper blanks per UTF-16 code unit, so its output is string-identical to the TS stripper — pinned by a new kernel arm on the strip differential oracle (fixtures + 500 seeded random cases). Gates, all green: new differential suite (adversarial fixture project — CRLF, NBSP, continuations, decoy strings, unterminated comments, backtracking shapes — indexed native-vs-JS: identical edge streams, plus a record-level scanner check); repo differential on git/redis/vim/SameBoy (identical, 705/852/433/180 edges); probe-hash on the live linux kernel DB reproduced f6e1713d… (279,335 rows); linux init counts exact 2,049,153/6,413,518; dump sha 6dd1185b… reproduced (10,446,478 lines); full suite green ×2 (153 files / 2588 tests). Measured (8c cg1212, quiet host): cFnPtr sub A=47.9s B=1.1 C=40.9 D=24.1 E=36.8 = 150.9s vs step 1's 179s and the pre-arc 230s (−34% cumulative); the sweep itself halved (94.5→47.9s, JS strips 132.4k→68.9k). callback-synthesis phase 199.9→171.1s. E's attributed wall grew from overlap shift under parallel synthesis; the phase total is the honest number. Full record: plan §7a.10. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
b9d0f57a64 |
feat(extraction): C deferral round 2 — 8 new preParse passes, linux kernel/+mm/ deferral 58.6%→33.9% (#1353)
Census-driven cut of the top-ranked post-R7a lever. All passes TS-side, C-only (preParseCSource), shared by both arms: - parameterized-annotation whole-blank (__free/__printf/__counted_by/ __bpf_md_ptr…; extends through a stranded field `;`) - type-keyword-arg scanner (kzalloc_obj(struct T), list_entry, multi-line continuations behind nested-paren args; bounded hand scanner, head exclusions + call-vs-declaration guard; blanks trailing stars) - static/extern CAPS-macro declaration lines at any scope; the initialized form is REWRITTEN to its expansion (name/tail keep exact offsets) - va_arg qualified-type blank; GNU named-variadic #define dots-only blank (post-restore); sandwiched notrace-family; C23 auto; multi-line iterator-macro spans (hlist_for_each_entry_rcu + lockdep arg) - word list += cacheline family (2- and 4-underscore spellings) + 10 more census-confirmed annotations Gates: five-repo parity sweeps 0 diffs (git deferral 16.1→12.2%, redis 25.3→24.1%, fmt/protobuf unchanged); linux full-tree both arms 2,049,153 nodes / 6,413,518 edges (+858/+6,585 vs R7a) with byte-identical dumps (10,446,478 lines, sha256 6dd1185b); kernel-arm parse-loop 356→306s at 2c; suite 2517 green under CODEGRAPH_KERNEL_EXPECT=1. Honesty note recorded in the docs: error recovery was already salvaging most SYMBOLS on deferred files — the graph win is relationships + phantom cleanup, and the unreleased CHANGELOG entry was rewritten off the sweep-subset framing. Also records §7a.5: post-R7a 8-core cg1212 re-run 16.4min (was 18.3min); 8c parse sits on the single-writer floor, so the <10min-on-8c gap re-ranks to the per-ref resolution path. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
2d72891b59 |
feat(kernel): R7a C/C++ walker — dual-lang ccpp module, preParse hoist, 7 new blanks, c/cpp default-routed (#1346)
Parity: 0 diffs on redis/git/fmt/protobuf/ALS sweeps; full-init dumps byte-identical on all five + linux at kernel scale (10.4M dump lines, same sha256 both arms). Linux 2c/6GB envelope: kernel-arm 19.1min vs wasm-arm 22.9min (parse 356s vs 435s) on a much richer graph (the new blanks recover error-swallowed code: git 2x nodes, linux kernel/+mm/ 3x). Deferral guard corrected by measurement (C/C++ error incidence 9-42%; --max-deferral flag); defer-reuse memo kills the 3x re-blank/re-parse cost deferred files paid. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
44561b6aad |
feat(extraction): vendor current C/C++ grammars (R7a prep) — c v0.24.2 + cpp v0.23.4, sha-matched (#1345)
tree-sitter-wasms shipped 2023-era C/C++ grammars; the kernel walker must compile the crates.io versions, so production wasm upgrades FIRST and in isolation (the R2 pattern). Built with tree-sitter-cli 0.25.10 from each tag's CHECKED-IN parser.c: tree-sitter-c v0.24.2 (b780e47, parser.c f2883ff9), tree-sitter-cpp v0.23.4 (f41e1a0, parser.c 2a35a43b, scanner.c cf60387d) — sha-matched against the crates.io tarballs. .metal/.cu map to language 'cpp' so the dialects ride the same coherent grammar. Full suite green with the upgraded grammars (2,490; one unrelated daemon idle-timeout flake passed solo 9/9) — incl. the UE-macro, misparse-guard, Metal, and CUDA coverage. Checklist updated with the vendored revs and the metal/cuda routing correction. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
ca88d3bd15 |
fix(db,resolution): WAL file cap + cgroup cache credit + pool/parse sizing corrections from the instrumented kernel-scale runs (#1335)
Four §7a.1 instrumented-run findings, each measured: 1. File-size trigger + truncate-at-barrier: a fully-backfilled WAL still grows the FILE without bound — the writer only restarts at frame 0 when a commit finds zero reader marks, which the instrumented run showed never happens (file marched 361→721MB through two COMPLETE backfills; 22GB by phase end). backpressure() now also trips at 4× the soft cap on raw file size and TRUNCATEs at the parked barrier; the timer path truncates opportunistically after complete backfills. Dubbo peak: 251MB → 69MB at the same 16MB valve; dumps byte-identical under aggressive folding. 2. cgroup memory credit: memory.current counts reclaimable page cache — a post-parse container read 57MB of headroom on a 6GB box and silently disabled the pool. inactive_file is credited back (the docker-stats working-set convention); the same run now reads a sane 4.4GB budget. 3. Pool at 2 cores reversed: sequential resolution measured FASTER than pooled-6-on-2 at kernel scale (853s vs 1,150s), and synthesis is Amdahl-bound by cFnPtrEdges (306s of 358s) so pooling it bought nothing. cpuCap = min(ap−1, 6), no floor: ap=2 → sequential is the fast path. 4. Parse floor of 2: one parse worker at a 2-cpuset measured 34% slower (493s vs 369s) — main + store-worker don't fill the second core. Floor restores the baseline (373.5s measured). Plus the observability §7a.1 burned three 25-minute cycles for: valve armed/fire/timer-pass/heartbeat lines, checkpoint-worker error capture, pool sizing decisions (incl. the disabled path), backpressure-hook presence — all behind CODEGRAPH_SYNTH_TIMINGS / CODEGRAPH_WAL_VALVE_DEBUG. Suite: 2,490 passed / 4 skipped (kernel required). Kernel-scale record runs with this build follow in the migration plan §7a.1. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
b8833fec57 |
feat(resolution): memory-aware, cgroup-honest worker-pool sizing + CODEGRAPH_RESOLVE_WORKERS (#1333)
Pool sizing used os.cpus().length, which enumerates the HOST's CPUs: inside a 2-CPU cpuset it sized 6 resolver workers (the §7a.1 false-'sequential' premise) and 8 parse workers, and at true 8-core concurrency six ~1GB workers OOM-killed a 7GB container (oom_kill=5) mid-synthesis — sizing had no memory term and no override knob. resolvePoolSize (pure, matrix-tested): explicit CODEGRAPH_RESOLVE_WORKERS override (0 disables, cap 16); CPU term max(2, min(availableParallelism-1, 6)) — cpuset-honest, floored at 2 so true 2-core boxes keep pooled synthesis's ~2×; memory term floor(budget*0.7 / clamp(0.2*dbSize, 256MB, 1.5GB)) with budget = min(freemem, cgroup v2/v1 headroom). Parse pool's core input switches to availableParallelism. Dev machines are unchanged (still 6 workers); the 8c/7GB kernel-scale container now sizes 4. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Colby McHenryandClaude Fable 5
|
c2503e2bee |
feat(kernel): R5 — Python and Go ports, gates passed, default-on
Python (codegraph-kernel/src/python.rs) and Go (src/go.rs) join the native kernel, mirroring the wasm extractors bug-for-bug. Python: decorated_definition docstrings/decorators (decorates refs only for bare-identifier decorators — the call-kind quirk), function-in-class → method, module-level assignments always extract as variable, from-import per-name binding refs, self.x fn-ref candidates as bare names. Go: receiver methods with Recv::name qualified names + contains edges to the first earlier struct of that name, type_spec struct/interface classification with embedding→extends and interface method nodes, composite-literal instantiates keeping the package qualifier, top-level var/const initializer walks attributed to the declared symbol (#693), 2-hop field chains (#1276), New().Method() re-encode (#645/#608), and the GO_SPEC fn-ref layers. Grammars: tree-sitter-python 0.23.6 + tree-sitter-go 0.23.4 crates, with wasm vendored from the same tags (parser.c sha-matched) — both were 2023-era in tree-sitter-wasms. Gates: extraction sweeps 100% (flask 83/83, django 3,035/3,038 +3 error-file deferrals, gin 99/99, prometheus 978/979 +1); full-init dump-diffs byte-identical on flask (10,833 rows), gin (17,540), django (360,794), and prometheus (213,758); torture fixtures enforced in npm test. DEFAULT_ROUTED now covers typescript/tsx/javascript/jsx/java/ python/go. Full suite: 2,471 tests pass. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Colby McHenryandClaude Fable 5
|
28068fa0f1 |
perf(kernel): direct-to-store decode — buffers flow to the store worker, main thread never materializes nodes
Kernel-routed files ship their flat tables from the parse worker to the store worker as buffers (tryKernelExtractRaw → kernelBuffers on the result → KernelStoreBundle); the store worker decodes and finalizes (finalizeStoreBundle shared with the object path so filter semantics can never drift). Files with applicable framework extract() hooks keep the decoded path; the no-writer fallback materializes via materializeKernelResult. Byte-identical dumps re-verified on dubbo, excalidraw, express, gson; full suite green (2,467). Measurement (plan §4d): dubbo's parse-loop wall is 94% store-writer busy time — the many-core fresh-index wall is single-writer SQLite ingest, not extraction or main-thread work. d2s improves the writer lane ~11% (structured-clone deserialization avoided on the writer) and frees the main thread; the remaining many-core gap is a store-architecture arc (deferred index builds, multi-file transactions, buffer→bind), out of the kernel project's scope. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Colby McHenryandClaude Fable 5
|
03d54e47a1 |
feat(kernel): R4 — Java port with Lombok synthesis, gate passed, default-on
Java joins the native kernel (codegraph-kernel/src/java.rs), mirroring the wasm extractor's Java paths bug-for-bug: package namespaces, imports, javadoc, annotations→decorates, type_list inheritance, static-final constants, enum constants, anonymous classes (including the TS side's 0-based-line quirk on the extends ref), method_invocation calls with the this.field unwrap and the Foo.getInstance().bar() chain encoding (#645/#608), static-member value reads, method-reference fn-refs (#756), value-reference edges, and the full Lombok member synthesizer (#912: Getter/Setter/Data/Value/Builder/ToString/ EqualsAndHashCode/Slf4j-family with taken-member dedup). The shared docstring/textutil modules moved to crate level. Grammar: tree-sitter-java 0.23.5, with the wasm grammar vendored from the same tag (parser.c sha-matched) replacing tree-sitter-wasms' 2023-era build. Gate (plan §4c): extraction sweeps 100% — gson 262/262, retrofit 341/341, dubbo 4,048/4,048 — plus a Java torture fixture in npm test; full-init dump-diffs byte-identical on gson (49,766 rows), retrofit (62,735), and dubbo (441,266 rows); all R2/R3 repos re-verified; Linux container runs all 23 kernel tests green under CODEGRAPH_KERNEL_EXPECT=1. The gate caught a real cross-language bug: fn-ref dedupe and value-ref self-target checks must compare node ID STRINGS, not node-table rows — ids collide for same-(kind, name, line) nodes, which minified one-line bundles hit routinely (retrofit's website JS exposed it; latent in the TS/JS walker since R2, never released). Fixed in both walkers. Benchmark honesty: dubbo fresh-init on an 11-core Mac is ~flat (parse-loop wall 5,020→4,394ms; total ~11.3s both arms) because that wall is main-thread-bound (reads + store), not worker-CPU-bound — the §6 expectation assumed otherwise. Where worker CPU binds the kernel delivers: dubbo on a 2-CPU/6GB container drops 27.8-28.6s → 22.3-22.8s (~1.25×). The identified lever for the many-core headline is decoding kernel buffers directly into store rows (skipping per-node JS object materialization); the buffer contract already carries everything. DEFAULT_ROUTED now includes java. Full suite: 2,467 tests pass. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Colby McHenryandClaude Fable 5
|
c8cca9a601 |
feat(kernel): R3 — TS/JS equivalence gate passed, kernel default-on
Gate evidence (docs/design/rust-kernel-migration-plan.md §4b):
- Graph parity, byte-identical (stronger than the §5 ≤0.5% bar): full
codegraph-init dump-diffs kernel-vs-wasm on express (13,712 rows),
excalidraw (89,898), and vscode (2,378,238 rows) — identical bytes.
Python control repo (flask) identical + timing unchanged. The parity
harness is now ORDER-sensitive (emission order drives rowids, which
drive resolution order) and dumps come from the new
scripts/dump-graph.mjs (natural keys, no rowids/timestamps).
- The one real find, caught by the vscode tier: tree-sitter error
RECOVERY is encoding-dependent — byte-identical grammar sources and
the same core (0.25.10) recover erroring files differently under
UTF-8 (native) vs UTF-16 (web-tree-sitter) parsing; proven by
reproducing the wasm tree with a native UTF-16 parse. Policy: the
kernel defers any file whose tree has_error() to the wasm extractor
(silent 'defer:' signal, per file) — parity by construction on
erroring files (incidence 0-0.42% across the gate repos), and the
harness fails if deferrals exceed 10% so a broken kernel can't hide
behind the fallback.
- Retrieval invariants: canonical excalidraw flow (mutateElement →
renderStaticScene) connects end-to-end on the kernel-indexed graph;
synthesized-edge families present. Agent A/B is vacuous under
byte-identical DBs (same justification as #1320-#1322).
- Perf: vscode init 105.4s → 82.1s (1.28×) on an 11-core Mac;
excalidraw on a 2-CPU/6GB Linux container (the CI-runner envelope)
6.2-7.1s → 4.3-4.8s (~1.5×). Linux arm64 in-container build: all 22
kernel tests green under CODEGRAPH_KERNEL_EXPECT=1. Windows VM leg
deferred (VM stopped; prlctl start needs Parallels Pro) — benign: a
missing .node falls back to wasm, and the release matrix builds and
gates the win32 prebuilds.
- Full suite: 2,465 tests pass WITH default-on routing, so the entire
extraction corpus now exercises the kernel for TS/JS wherever a
.node is staged.
DEFAULT_ROUTED = {typescript, tsx, javascript, jsx}. Override:
CODEGRAPH_KERNEL_LANGS (replaces the set) / CODEGRAPH_KERNEL=0 (kill).
Changelog entry added under [Unreleased].
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Colby McHenryandClaude Fable 5
|
9ad5cd7ba2 |
feat(kernel): R2 — full TypeScript/JavaScript extraction port, byte-parity with the wasm path
Replaces the R1 seed .scm emitter with a bespoke Rust walker (codegraph-kernel/src/tsjs/) that mirrors TreeSitterExtractor's TS/JS paths function-for-function: declarations (incl. #808 field/property classification), qualified names, docstrings (#780 wrapper climbs), signatures, imports/re-exports + per-binding refs, calls with receiver-qualified callees (#1230 literal-receiver skip), instantiations, decorators, inheritance, type annotations (#381), type-alias members + tuple contracts (#359/#634), React component recognition (#841 forwardRef/memo/styled), object-of-functions / zustand-through-middleware / RTK Query endpoints + generated hooks / vuex + pinia store shapes, function-as-value capture with the flush gate (#756), and value-reference edges with the shadow prune (#895/#897). The generic query emitter is deleted — extraction parity needs logic .scm can't express; future languages get walkers too (migration plan §4a). Positions and JS string-slice semantics are emitted in UTF-16 code units natively, so kernel output is byte-identical to web-tree-sitter's — no column diff class exists. Parity evidence (macOS): scripts/kernel-parity.mjs (full-object multiset diff per file) — this repo 353/353 files, excalidraw 643/643 (10,650 nodes / 10,726 edges / 68,307 refs), plus torture fixtures checked into __tests__/fixtures/kernel-parity/ and enforced in npm test by kernel-tsjs-parity.test.ts. The strict compare caught one real decoder bug the loose harness missed: refs must NOT carry denormalized filePath/language at the extractFromSource seam (the store fills them). Perf: extraction 2.6× single-thread on excalidraw (487ms vs 1,255ms, identical outputs). Routing stays opt-in (CODEGRAPH_KERNEL_LANGS) until the R3 equivalence gate (large repo, DB dump-diff, retrieval invariants, agent A/B, Linux/Windows) passes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Colby McHenryandClaude Fable 5
|
c5eebe6beb |
feat(kernel): R1 scaffold — napi-rs extraction kernel, buffer contract, routing + fallback, grammar-parity CI
Phase 0 of the Rust extraction-kernel migration (docs/design/ rust-kernel-migration-plan.md, now checked in with §3a recording the shipped state): - codegraph-kernel/ napi-rs crate: extractFile(path, content, language) → five flat buffers (meta/nodes/edges/refs/arena), one JS boundary crossing per file. Node ids computed Rust-side, byte-identical to generateNodeId (pinned by test vector). Reserved per-node metrics slot for the Arc 3.2 code-metrics work. - Generic .scm-driven emitter (@def.<kind>/@name/@ref.<kind> captures, byte-range scope stack → ::-joined qualified names, contains edges, refs attributed to the innermost enclosing symbol). Seed TS/JS queries are smoke-level; R2 replaces them with the full port. - Routing seam in extractFromSource with per-file wasm fallback. DEFAULT_ROUTED is empty — no behavior change until a language passes its equivalence gate (R3). Dev opt-in: CODEGRAPH_KERNEL_LANGS. Kill switch: CODEGRAPH_KERNEL=0. Loader verifies ABI + kind tables before routing; EDGE_KINDS became a runtime array because kind order is now wire contract. - Grammar-source parity: vendored TS/TSX/JS wasm grammars built from the exact crate revisions (tree-sitter-typescript v0.23.2, tree-sitter-javascript v0.25.0, checked-in parser.c, ts-cli 0.25.10) — the tree-sitter-wasms builds were 2023-era, which the new kernel-grammar-parity test caught on day one. Production TS/JS parsing gets 2.5 years of grammar fixes; full suite green (2456 tests). - Build/release wiring: scripts/build-kernel.sh + npm run build:kernel; release.yml kernel prebuild matrix (continue-on-error — the kernel is optional everywhere, bundles fall back to the wasm path); bundles stage lib/kernel/codegraph-kernel.node; release job runs the kernel suites with CODEGRAPH_KERNEL_EXPECT=1. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
a2f3c31a97 |
perf(resolution): defer checkpoints and double-buffer persist during resolution, byte-identical graphs (#1320)
Fresh init on a 4,402-file Java repo (dubbo): 18.2s -> 13.5s (-26%), with the resolution phase going 12.6s -> 7.9s (-38%). Graphs verified byte-identical on both the pool path (dubbo) and the sequential path (excalidraw). Two changes: 1. The fastInit+pool path restored WAL for the resolver workers but left wal_autocheckpoint at its default, so the persist loop inline-checkpointed hot pages all phase long (#1231's pathology inside resolution — measured at 58% of resolution wall). Checkpointing is now deferred behind the bounded valve and folded once at maintenance, mirroring the deferWal path. 2. The resolution loop is double-buffered: batch k+1 is prefetched (OFFSET past batch k's still-pending rows, under an explicit ORDER BY rowid) and fanned out across the pool while batch k's ref cleanup runs on the main thread. Batch settle-waits dropped 2572ms -> 117ms. Correctness invariant found by the byte-identical gate and now documented in the loop: batch k+1's resolution READS batch k's edges (resolveMethodOnType walks supertype chains over extends/implements edges that resolution itself inserts), so edges must persist BEFORE the next batch fans out; only the ref cleanup overlaps. Also extends the CODEGRAPH_SYNTH_TIMINGS instrumentation with phase labels (grammar-init, parse-loop, fts-rebuild, resolver-reinit, resolution, callback-synthesis) and pool timings (worker open/resolve, per-batch mode, persist), so the next profile is one env var away. Suite green (2444). Sync path timings unchanged. Pool floor re-validated: forced-on at 40k refs is still net-slower, so the 150k threshold stands. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
a5a8942d1c |
fix(scan): includeIgnored child patterns revive repos under a gitignored parent (#1318)
`.gitignore: /repos/` lists `repos/` as ONE ignored entry, while the CLI hint (#1156) suggests `includeIgnored: ["repos/a/", "repos/b/"]` — the child spelling. findIgnoredEmbeddedRepos tested the opt-in matcher against the PARENT path only, which a child pattern never matches, so the documented opt-in silently indexed nothing and init looped the byte-identical suggestion back at the user (#1295). Ignored dirs that don't match as a whole are now descended (the walk was already bounded: depth 4 / 2000 entries, and only runs when includeIgnored is configured) and each nested repo root is matched individually — parent spelling opts in everything under the dir, child spelling exactly the named repos. findUnindexedIgnoredRepos gets the same per-repo check so the hint stops nagging about repos that are already configured while still naming unopted siblings. Fixes #1295 Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
c472cfb52e |
fix(resolution): literal-receiver builtins and nested locals stop fabricating call edges (#1317)
", ".join(sorted(x)) resolved by bare name to a project function named join — one nested inside a DIFFERENT function, so scope alone rules the edge out. Both defects from #1230, fixed independently: 1. Extraction: a member call on a LITERAL receiver (string, number, collection, regex — across grammars) emits no call ref at all. A literal's methods are the language's builtins, never project symbols; the bare-name fallback let them exact-match any same-named project function. Silent miss, never a wrong edge. 2. Resolution: matchByExactName filters out candidates nested inside a same-file FUNCTION container unless the ref originates within that container's line range. Class members (parent is a class-like node), top-level symbols, and C++ namespace prefixes (no parent node) are untouched. requests re-index: byte-identical (813 calls edges). excalidraw: -27 edges, all literal-receiver refs by construction. The issue's repro is pinned: join has exactly one caller (format_fields), report_missing has zero project callees. Fixes #1230 Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
41c2029798 |
fix(go): field-chain calls resolve via validated type inference, never bare-name guessing (#1316)
target.conn.Exec("insert") with `conn *sql.DB` emitted a BARE `Exec`
ref (the receiver chain was dropped for non-identifier receivers), and
exact-match then bound it to the only local `Exec` — an unrelated
interface's method — fabricating an internal dependency (#1276).
Extraction now keeps Go 2-hop selector chains (`base.field.Method`),
and a dedicated matcher resolves them EXCLUSIVELY via two inference
hops: base's type from the enclosing scope (#1108 machinery), field's
declared type from the struct's own declaration lines (comment-
stripped, per-line — chi's "the tree router" doc comment otherwise
donates a phantom type). resolveMethodOnType validates the target.
Package-qualified field types are followed only when the package is
in-module — `handler http.Handler` must not bind a same-named local
decoy. Failure at any hop leaves the ref unresolved: chained Go
receivers never fall through to the bare-name strategies (they were
never emitted before, so no prior recall depends on that path).
chi before/after: node count stable (1,181); 8 correct field-chain
edges gained (mx.tree.FindRoute/InsertRoute/routes, validated,
including the unexported `node` type); the removed edges are the
prior bare-name guesses on external receivers.
Fixes #1276
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
|
b6a05d155b |
fix(c): blank leading attribute macros so functions index under real names (#1311)
SEC_ATTR UINT32 LostName(VOID) — an unknown attribute macro before a
typedef'd return type — misparses in tree-sitter's C grammar: the macro
becomes the type, the return type the declarator, and the PARAMETER
LIST is stored as the function name ("(VOID)"). The C++ grammar
recovers this shape via recoverMangledCppName, but in C the real name
never reaches the mangled string, so only a pre-parse blank can help.
Attribute macros are project-specific, so the blank keys on structure:
line-leading ALL-CAPS token followed by TWO identifiers then `(` — the
`MACRO Ret name(` definition shape. Plain typedef'd returns, ALL-CAPS
calls, #define lines, multi-word builtin returns, and mid-line uses are
all rejected by construction. Offset-preserving like the C++ blanks.
curl re-index: 5,531 C functions before and after, zero name changes;
7 nodes in memdebug.c improve start-line accuracy by 1 (the macro line
no longer counts as part of the definition).
Fixes #1211
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
|
e437918026 |
fix(cpp): compose namespace prefix into out-of-line method qualified names (#1310)
An out-of-line member definition inside a namespace block takes its
qualifiedName from the declarator's receiver, which is spelled RELATIVE
to the enclosing namespace — so `namespace simulator {
ManifestStartup::Output ManifestStartup::Apply(...) {} }` indexed as
ManifestStartup::Apply while the class node carried
simulator::ManifestStartup. Fully-qualified call sites
(simulator::ManifestStartup::Apply(...)) never resolved; callers and
file impact came up empty (#1291).
The receiver-based qualifiedName now composes the active namespace
prefix, anchored at the first prefix segment the receiver re-spells
(so `namespace sim { void sim::M::f() {} }` doesn't double-prefix).
namespacePrefix is only ever non-empty for C++ — Go/Rust/Kotlin/Lua
receivers pass through unchanged.
leveldb re-index: node count byte-stable (3,044), calls edges +6,
namespace-qualified method names 947 -> 1,252.
Fixes #1291
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
|
4dd29ea5c1 |
fix(cpp): strip template args from out-of-line method receiver qualifiers (#1309)
template<typename T> T Box<T>::get() stored qualified_name Box<T>::get — the <T> qualifier never matched the class node indexed as Box, so the method didn't link to its class, while the inline form of the same method produced Box::get. ICU-shaped multi-line template parameter lists leaked whole <…> blocks (newlines included) into qualified_name, exceeding NAME_MAX for downstream consumers. extractCppReceiverType now applies stripCppTemplateArgs (the #1043 normalization for base-class refs) to the receiver qualifier. fmt re-index: template-arg-in-qualifier names 25 -> 4 (remaining are a FMT_BEGIN_EXPORT misparse artifact and gmock conversion-operator names, both distinct pre-existing shapes), node count byte-stable at 7,536. Fixes #1286 Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
5736e24bb6 |
perf(index): faster fresh indexing + parallel reference resolution, byte-identical graphs (#1305)
* perf(index): ~34% faster fresh indexing, byte-identical graphs Profiling a fresh init on a medium TS repo (excalidraw, 657 files) showed the main thread as the critical path: per-row SQLite statement calls, repeated import-resolution walks, and per-row FTS trigger firings, with the parse workers ~75% idle behind it. This lands the semantics-preserving tranche of fixes: - Multi-row batched INSERTs (nodes/edges/unresolved refs/name segments) behind cached per-batch-size prepared statements; row order preserved, so rowid-based resolution determinism (#1015) is unchanged. - storeFileBundle: one transaction per file instead of four; nested transaction() calls now flatten (BEGIN-in-BEGIN previously threw, so no caller depended on nested rollback). - Dedicated store-writer thread for the fresh-DB bulk path (bundles applied in file order on a single writer connection; main thread does no DB work during the parse loop). Kill switch: CODEGRAPH_NO_STORE_WORKER=1. - Bulk FTS mode: drop the nodes_fts sync triggers during the bulk load, rebuild once at the end; crash inside the window self-heals on the next open. - Per-context memos for resolveImportPath/findExportedSymbol + a per-file exported-symbol index, invalidated exactly where clearCaches() already resets the resolver's own caches. - Fast-init on completely fresh DBs (journal in memory, no fsync until the index completes; interrupted init re-runs from scratch). Kill switch: CODEGRAPH_NO_FAST_INIT=1. - MaybeYield returns undefined on the not-due path so per-ref yield checks stop paying a promise + microtask hop each. - Parse pool prewarm for bulk indexing; compile-cache enabled at CLI and worker entry points. Excalidraw fresh init: 5.11s -> 3.36s median (n=5, warm cache, M-series). Graph dumps byte-identical across init, re-index, and sync paths; full suite green (2403 passed). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * perf(resolution): parallel reference resolution with canonical admission Fan resolution batches across a pool of read-only worker threads, each hosting a full ReferenceResolver over its own SQLite connection; results are admitted on the main thread in chunk order, so edge insertion order, row cleanup, failure parking, and deferred post-pass queues are exactly the sequence the single-threaded loop produces. Per-ref inputs match the baseline because the sequential path already resolves each batch against the state committed BEFORE that batch. Validated byte-identical on excalidraw (pool forced on) and apache/dubbo (4,048 Java files): dubbo full index 39s -> 19s (2.05x) with identical graph dumps (91,495 nodes / 223,953 edges). The pool only engages when total pending refs clear a threshold (default 150k, CODEGRAPH_PARALLEL_RESOLVE_MIN to tune, CODEGRAPH_NO_PARALLEL_RESOLVE=1 to disable): measured on a ~58k-ref repo the workers' boot CPU contends with resolution on the same cores and makes indexing slower, so small repos keep the sequential path. When fast-init left the DB in memory-journal mode, WAL is restored before resolution only when the pool will run (readers + rollback-journal writers don't mix). Also: sqlite adapter readOnly open support. TreeCursor spine rewrite of the body walker was built, measured neutral on real repos and equal in a 20k-child microbench (web-tree-sitter's namedChild(i) is not quadratic in this binding), and rejected — per-node JS<->WASM marshaling is the floor, which a traversal swap cannot remove. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
ad5300a601 |
fix(ui): show synthesis as a 'Linking dynamic dispatch' phase; mute node:sqlite warning spam (#1299)
Two first-run UX bugs surfaced by indexing a real 1,342-file C repo: 1. After 'Resolving refs' hit 100%, the ~40 dynamic-dispatch synthesis passes ran with no progress surface, so the bar sat frozen at 100% long enough to read as a hang (the C fn-pointer pass alone can hold for a while on C-heavy repos). Synthesis now reports per-pass progress through a new 'linking' IndexProgress phase, rendered as 'Linking dynamic dispatch'. The step total is pinned by a test to the synthesizer's actual __mark() count so adding a pass without bumping it fails loudly. 2. node:sqlite's ExperimentalWarning is emitted once per THREAD, so the main process plus every parse worker printed it mid-index, interleaved with the progress UI. All launch paths now pass --disable-warning=ExperimentalWarning: both bundle launchers, the Windows npm-shim invocation, and the CLI self-relaunch (NODE_RUNTIME_FLAGS, deliberately excluded from the re-exec gate so an older installed launcher never triggers a pointless re-exec, and version-gated off nodes older than the flag). Verified end-to-end on the same repo: zero warnings, live linking bar, byte-identical graph (50,520 nodes / 148,232 edges). Full suite green. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
6103f5e228 |
fix(cpp): resolve explicit operator calls (a.operator+(b)) to the operator method (#1268)
* fix(cpp): resolve explicit operator calls (a.operator+(b)) to the operator method (#1247) tree-sitter-cpp can't parse an operator_name in field position: the call_expression carries `function: <receiver>` plus an ERROR child wrapping the operator_name instead of a field_expression callee, so the extractor emitted a calls ref named just the receiver (`a`) and the edge never resolved — while the operator method itself indexed fine. Two-part fix, scoped to the explicit call form (infix `a + b` / `a[i]` need receiver type inference and are tracked in #1258): - extraction: recover the operator_name from the ERROR child and emit `<receiver>.operator+` (`->` receivers normalized, `this->` emits the bare name), like any other member call - resolution: matchMethodCall's dot pattern now admits an operator method part (cpp-gated; symbol chars failed the \w match), so receiver-type inference + resolveMethodOnType validate the target — a same-named operator on an unrelated class can't capture the edge Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(cpp): harden explicit operator-call recovery against real-world shapes (#1247) Validated on nlohmann/json (dozens of explicit operator[] / operator* / operator< call sites). Two refinements the synthetic fixtures missed: - normalize spaced call-site operator names (`it.operator * ()`, `other.operator < (*this)`) to the compact form definitions index as - drop the ref for a complex receiver (`obj()->operator+`, member chains ending in a call) instead of emitting a bare operator name: exact-name fallback GUESSED among unrelated same-named operators (linked a std::map operator[] call to an in-repo operator[]) — silent miss, never a wrong edge Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
9d0cd3a7d1 |
fix(sync): resolve cross-file refs when an edit adds or removes the satisfying symbol (#1240) (#1249)
* chore: ignore .kommandr/ directory * fix(sync): resolve cross-file refs when an edit adds or removes the satisfying symbol (#1240) Incremental sync scoped reference resolution to the changed files' own refs, and a completed pass deleted every ref it failed to resolve — so a symbol change in one file could never repair references in UNCHANGED files, in either direction, until a full re-index: - New-export case: a.ts imports/calls `greet` before b.ts defines it. The failed refs were deleted at index time; when b.ts later gained `greet`, nothing revisited a.ts — the calls/imports edges stayed missing while status reported a clean index. - Removal case: when a re-index (or file deletion) dropped a symbol, the incoming edges cascade-deleted and the callers — whose resolved refs had been consumed — never got a chance to rebind to an alternative definition or reconnect when the symbol returned. Fix, sharing one lifecycle: - Schema v8: unresolved_refs gains status ('pending'/'failed') and name_tail (last dotted segment, so `h.greet` is findable by `greet`). Both resolver persist paths now park unresolvable refs as failed instead of deleting them. All pending-work readers (batched drain, non-progress guard, #1187 orphan sweep, status pendingRefs) filter to pending, preserving their invariants and keeping status honest. - Sync retry: after scoped resolution, failed refs whose name tail matches a symbol name now present in the changed files are re-resolved through a per-ref-yielding path (watchdog-safe, #1091 class). Names matching >500 failed refs are skipped as external/builtin noise (#999 rationale). - Removal side: createEdges stamps each resolution edge with its originating reference (metadata.refName, + refKind when kind promotion rewrote it). When the #899 restore misses a target or sync deletes a file, the dropped edge is resurrected as exactly that ref — re-resolved in the same sync (rebinding to an alternative definition) or parked failed until the symbol reappears. Edges without the stamp (pre-upgrade, synthesized) still drop silently: reconstructing from the target's plain name would strip receiver context and risk a rebind a full re-index would never make. - Pure-removal syncs clear resolver caches so a long-lived daemon can't resolve resurrected refs against the pre-removal graph. Validated: issue repro now yields a graph byte-identical to a full re-index; move/remove-readd/file-deletion scenarios all rebind or heal; baseline-vs-new A/B on express and gin shows identical node/edge counts and no timing regression (DB grows ~25% from the parked ref rows — pure cache, reset by any full re-index). 8 regression tests added. Fixes #1240 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
a11a439002 |
fix(indexing): HDD-class storage — false parse timeouts, dropped files, and WAL checkpoint write-back (#1231) (#1242)
Parse timeouts are now judged by the worker's own clock: the base timer only marks a job late (after a long synchronous store stall, Node runs the timers phase before the poll phase, so the timer fired before an already-delivered result was processed — killing workers over parses that took milliseconds, even on 0-byte files); a result arriving before a 3× hard-kill backstop is accepted, timed-out files are retried, and CODEGRAPH_PARSE_TIMEOUT_MS overrides the budget. Grammar WASM bytes are pre-read once on the main thread and handed to every worker, so spawns/respawns load grammars from memory instead of re-reading a saturated disk. Bulk indexing defers WAL auto-checkpointing for the whole run: the default 1000-page interval re-writes hot B-tree/FTS pages into the main DB file over and over — ~95% of all disk I/O under throttled measurement. A WalCheckpointValve bounds WAL growth with off-thread PASSIVE backfill passes (never blocking the writer or the #850 watchdog heartbeat), pauses the writer for a full backfill if the disk truly can't keep up, and folds the WAL at the parse→resolution boundary so post-parse reads never page a bulk-write-sized WAL. Opt out with CODEGRAPH_NO_WAL_DEFER=1; tune with CODEGRAPH_WAL_VALVE_MB. Measured at 150 IOPS (HDD class): commons-lang 1526s → 59s with 0 dropped files (was 8); guava-scale completes in 7.6 min with a full graph where v1.3.1 needed 25 min for a repo 5× smaller. Unthrottled: no change. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
a3f90089e8 |
fix(indexing): bounded-memory yielding pipeline tail + daemon session fixes (#1212) (#1226)
Large-codebase indexing died at the end of "Resolving refs" two ways: watchdog kills of healthy work (24k-file Java on Windows, #1212 — third iteration of the #1091/#1122 class) and hard OOMs (Linux kernel scale, where v1.3.0 could not complete at any watchdog setting). Root causes: ~31 of 37 dynamic-edge synthesis passes ran start-to-finish with no yield points, several materialized whole-graph snapshots (kotlin expect/actual opened with getAllNodes() — 2M nodes in one array; the C fn-pointer pass retained every C file's contents twice plus every function node), and the post-index WAL checkpoint ran minutes of synchronous IO on the main thread, killing even a successful index at the finish line. The pipeline tail now follows the same discipline as the rest: never hold O(graph) in the heap, yield everywhere. - All synthesis passes stream node-kind scans (cursors, not arrays) and yield on time-budgeted checkpoints; language gates skip passes whose filters a project's file languages provably can't satisfy. - kotlin expect/actual filters SQL-side; c-fnptr caches are LRU-bounded, units stream one file at a time, and the all-functions array + write-only id map are gone; spring reads each .java once, not twice. - runMaintenance moved to a worker thread (own SQLite connection); per-file store commits chunk with yields behind a serialized flush chain (preserving #1015 file-order determinism); resolver warm-up streams the DISTINCT name set; resolution batch-tail and merged-edge inserts run in bounded sub-transactions. - Daemon: fixed a socket-handoff race that could leave a fresh MCP session permanently silent (client-hello tail unshifted into a flowing stream with zero listeners — the long-standing #662 test flake was this real bug); first tool call no longer queues behind the query pool's cold start (pool.ready gate). Validation: Linux kernel (70,129 files, 2.05M nodes, 6.4M edges) fully indexes in 27m8s on a 2-core/6GB container at default heap + default watchdog; llvm-project (180k files) completes under 1GB RSS including kill-and-sync recovery; synthesized-edge and full-graph parity are byte-identical vs baseline on elasticsearch/redis/vim; the ex-flaky daemon test passed 25/25 under load. Env-gated diagnostics kept: CODEGRAPH_SYNTH_TIMINGS pass/phase timings, CODEGRAPH_MCP_DEBUG hop tracing. Design record: docs/design/main-thread-stall-followup.md. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
e65a39746c |
fix(init): surface and offer to opt in gitignored child repos on an empty index (#1156) (#1208)
A Git super-repo whose `.gitignore` excludes its child repositories indexed ~nothing at the parent: CodeGraph respects `.gitignore` by default (#970, #1065), so the excluded children were skipped and `codegraph init` printed "Done" with 0 nodes — even though `init` inside each child worked fine. The empty index was silent and unexplained. `init`/`index` now detect the gitignored child repos they skipped when an index comes up empty of symbols, name them, and — in an interactive terminal — offer to index them (writing an `includeIgnored` entry to codegraph.json and re-indexing on the spot); non-interactive runs print the exact codegraph.json snippet to add. Gated on nodesCreated === 0, so a project that deliberately keeps gitignored reference clones out of a working index is never nagged. - extraction: findUnindexedIgnoredRepos — the inverse of discoverEmbeddedRepoRoots (bounded, skips default-ignored dirs, respects existing includeIgnored) - project-config: addIncludeIgnoredPatterns — create/merge codegraph.json, idempotent, refuses to clobber malformed JSON - cli: wire the detect-name-offer flow into both `init` and `index` - tests: +13 covering detection, config writing, and the no-nag gate Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
c049d9eb0d |
fix(extraction): detect export-macro-annotated class in .h language check (#1159) (#1207)
Carries #1133 forward onto current main (rebased for conflicts). A lean Unreal-Engine-style `.h` whose only C++ signal is `class ENGINE_API Foo : public Bar` (no public:/virtual/namespace/template) was misdetected as C and its class + inheritance edge silently dropped; looksLikeCpp now recognizes the export-macro-annotated class/struct shape, matching what blankCppExportMacros already recovers. Fixes #1159. Co-Authored-By: robertyluo <robertyluo@tencent.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
 xyyxrandGitHub
|
8e697a0d09 |
fix(extraction): recover heavily-reflected Unreal Engine C++ classes (in-body reflection macros) (#1158)
Heavily-reflected UE C++ classes (`UCharacterMovementComponent`, `AActor`, `UWorld`, …) were dropped from the index: in-body no-semicolon reflection macros (`UPROPERTY`/`UFUNCTION`/`GENERATED_BODY`/`UE_DEPRECATED_*`), member-level `*_API` prefixes, and mid-line `UMETA`/`UPARAM`/`UE_DEPRECATED` accumulate tree-sitter parse errors until the enclosing class_specifier collapses into an ERROR node. Three offset-preserving, C++-only pre-parse passes (`blankCppAnnotationMacroCalls`, `blankCppApiPrefixMacros`, `blankCppInlineAnnotationMacros`) blank the markup before parsing so the class survives. Validated on the real Epic engine headers (CharacterMovementComponent.h / Actor.h / World.h): each main class + its base clause goes from DROPPED on main to recovered with the fix. Closes #1160. Thanks @luoyxy for the report and root-cause analysis. |
||
|
xyyxrandGitHub
|
2a06d9a71f |
feat(config): codegraph.json include to force gitignored first-party source into the index (#1063)
Adds an `include` list to the root `codegraph.json` that forces gitignored first-party source (second-VCS / SVN / Perforce dual-tracked repos) into the index — discovered directly off disk on the full index, incremental sync, and file-watching, on both git and non-git projects. Gitignore-style patterns, root-relative; explicit `exclude` still wins and built-in skips (node_modules, dist, .git) are never re-included. Complements `exclude` and `includeIgnored`. Closes #1163. Thanks @luoyxy for the contribution. |
||
|
f5edf8cf49 |
fix(mybatis): quote/comment robustness, iBatis <sqlMap> coverage, dup-id collision (#1182) (#1204)
Four gaps in the MyBatis mapper extractor, all reported and reproduced by @ESPINS in #1182 and verified against main: 1. Single-quoted attribute values (namespace/id/refid/resultType/parameterType) were dropped — the regexes hardcoded double quotes. Now accept either quote via a backreference. 2. Tags inside <!-- ... --> produced phantom statement/include symbols. A length-preserving, CDATA-aware pre-pass blanks comments before scanning, keeping offsets/line numbers intact. 3. Legacy iBatis 2 <sqlMap> files had zero statement coverage (the root finder gated on a <mapper namespace> root). It now also recognizes <sqlMap> (namespaced and namespace-less DAO.method ids) and iBatis's extra <statement>/<procedure> verbs — closing the gap with no new dependency (option (c) from the issue; the batis-xml parser route is declined). 4. Two statements sharing a qualifiedName AND a start line (a vendor-split databaseId pair on one line) collided on the node id, so INSERT OR REPLACE silently dropped one. The id-hash now folds in the statement's byte offset; the stored qualifiedName/startLine are unchanged so the Java<->XML bridge is untouched. Gaps 1 and 2 follow @ESPINS's fix-mybatis-quotes-comments branch. Tests add extractor-level coverage for all four gaps plus a DB-level e2e that proves iBatis statements land and both vendor-split nodes survive a real indexAll. Co-authored-by: Jimin Lee <dlwlalsggg@gmail.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
7f325134e0 |
feat(extraction): add Nix language support with module-system option wiring (#324, #332 via #648 — carries #1084) (#1190)
Carries @TyceHerrman's #1084 as the functional base. Extraction + file wiring (imports/modules lists, callPackage), module-system option-path synthesizer, lexical-scope resolution gates, ABI-15 wasm rebuilt from upstream source. Validated on agenix, nix-darwin, home-manager, and nixpkgs (44,368 files, 3m49s, 1.30M nodes). Co-authored-by: Tyce Herrman <Tyce.Herrman@pm.me> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
99152212a9 |
feat(extraction): add ArkTS language support with ArkUI dispatch bridges (#396, #512, #890 via #648) (#1186)
Adds ArkTS (.ets, HarmonyOS/OpenHarmony) as a first-class language: full TypeScript-grade extraction via the harmony-contrib tree-sitter grammar (MIT, vendored byte-identical from the tree-sitter-arkts 0.2.0 npm tarball), plus the ArkUI constructs that make HarmonyOS apps traceable: - @Component/@ComponentV2 structs with decorators from both grammar positions; members extract as class members with qualified names. - build() component trees: child instantiation edges via arkui_component_expression, no synthesizer needed. - Attribute chains emitted dot-prefixed and resolved ONLY against @Extend/@Styles/@AnimatableExtend/@Builder helpers (unique-or-drop) — bare-name fallthrough produced 36,840 wrong edges (17% of calls) on the OpenHarmony samples monorepo. All four grammar chain shapes handled, including the detached-chain forms. - .onClick(this.handler) method-reference bindings. - ohpm workspace modules: bare imports follow oh-package.json5 file: deps (ambiguous names dropped), honoring each module's main entry — which also lets .ts consumers resolve .ets modules. - ArkUI dynamic-dispatch bridges, all provenance:'heuristic' with wiring-site metadata: assignment-gated state->build() re-render (V1 @State family + V2 @Local/@Provider/@Consumer), @ohos.events.emitter emit->subscriber pairing on static event keys (numeric ids same-file, named constants same-module, fan-out capped), and router.pushUrl literal urls -> the target page's @Entry struct. - $r/$rawfile resource intrinsics treated as built-ins; arkts joins the web language family, value-reference edges, re-export chase, and the other TS-applicable gates. Also ships a language-agnostic index-completeness guard: indexAll stamps index_state (indexing -> complete/partial/failed), reconciles discovered vs accounted files (a loaded run silently dropped 37 files), and codegraph status surfaces truncated/partial indexes in human and --json output. Validated on HarmoneyOpenEye (82 files), CoolMallArkTS (528, modular ohpm + ArkUI V2), and openharmony/applications_app_samples (11,693 files, 202,890 nodes stable across re-index, attribute false-positive audit 36,840 -> 588 residual all-plausible). Supersedes PRs #656 and #988 with credit — both informed this implementation. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
f8cdbe3c67 |
feat(terraform): remote-state bridge, provider aliases, moved/import/check refs (#1174)
Follow-ups noted in #1173: - cloudposse/atmos remote-state: module.M.outputs.X emits a scoped module.M:remote-output.X candidate; the resolver bridges it to the target COMPONENT's own output when every gate holds — the module source is the stack-config remote-state module, the component name is static (a literal, or component = var.X whose variable declares a literal default in the same directory), and exactly one directory in the repo matches the component name and declares that output. Dynamic (each.value) or ambiguous wiring stays unlinked. On cloudposse/terraform-aws-components: 254 remote-state bridge edges, every one re-derived from a matching source declaration (789/789 cross-directory output edges explained: 528 local-module + 254 remote-state + 7 checker-artifact false alarms under deprecated/); coverage 66.4% -> 69.1%. - provider aliases: provider "aws" { alias = "east" } is addressed as provider.aws.east so aliased and default configurations stop colliding; provider = aws.east on a resource/data block (and the values of a module's providers map) reference the selected configuration, resolved same-directory first then up the module tree — the one construct Terraform genuinely inherits from parents. The selection is no longer misread as a resource reference (aws.east). - moved/import/removed blocks reference the resource addresses they name (anchored to the file node — no phantom symbols), so a refactor's paper trail joins the graph; check-assert conditions contribute their references while check-scoped data blocks keep indexing as before. Scoped module candidates are suppressed there: module.a.aws_x.b names a resource inside a module instance, not an output. +91 edges on cloud-foundation-fabric's moved-heavy stages. Also fixes a latent test bug from #1173: cg.getNodeById is not public API (cg.getNode is) — it only passed because the asserted edge list was empty. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
6c24f4bddf |
feat(extraction): add Terraform/OpenTofu language support with module-boundary bridging (#83, #310, #648 — carries #706) (#1173)
* feat(extraction): add Terraform and OpenTofu language support Index .tf, .tfvars, and .tofu files via the tree-sitter-terraform dialect of HCL (vendored from @tree-sitter-grammars/tree-sitter-hcl, Apache-2.0). Symbols extracted: - resource / data → class (qualified "type.name" / "data.type.name") - module → module (qualified "module.name") - variable → variable (qualified "var.name") - output → variable (qualified "output.name") - provider → namespace - locals → constant per attribute (qualified "local.key") References resolved cross-file: - var.X, local.X, module.M[.out], data.T.N[.attr], <type>.<name>[.attr] - built-ins skipped: each.*, count.*, self.*, path.*, terraform.workspace The Terraform framework resolver disambiguates same-named candidates across modules by preferring the one in the same directory as the reference site, then by closest common-ancestor path, falling back to the generic name matcher only when neither applies. Validated on two Terraform monorepos (277 and 470 .tf files): indexing runs in 1.3s and 2.4s respectively, query latency stays under 200ms, and cross-module references resolve to the correct module 100% of the time on inspected samples. 18 new extraction tests; full suite 1146/1148 green (2 pre-existing flaky skips, 0 regressions). * feat(terraform): bridge the module boundary and enforce directory scoping Builds on #706. The module declaration was a dead end: module.M.out resolved to the declaration and stopped, module inputs never reached the child module's variables, and impact could not cross the boundary — on real multi-module repos that breaks the core blast-radius question ("what breaks upstream if I change this module's variable/output"). - module blocks now wire across the boundary through :-scoped refs only the Terraform resolver understands: module.M:var.<input> → the child's variable node, module.M:output.<o> → the child's output node (emitted alongside the module.M declaration ref), and module.M:file → the local source directory's entry file (imports). Registry/git sources emit no file ref and resolve nothing — an out-of-repo module stays a visible boundary instead of a guess. - .tfvars top-level assignments reference the variable they set, walking up to the nearest ancestor directory (envs/prod.tfvars → root vars). - Resolution now enforces Terraform's real scoping: same-directory only (no cross-module fallback by common path prefix, no single-candidate anywhere-in-tree binding), and terraform refs never fall through to the generic name matcher — var.X can never legally bind outside its module directory, so the fallback could only add wrong edges. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(terraform): README language table + changelog entry + agent-eval corpus Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Javier Rodríguez Fernández <jfernandez@freepik.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
e1a8d888e5 |
feat(extraction): add CUDA language support (.cu/.cuh) (#387, #648) (#1172)
CUDA rides the C++ grammar via the Metal (#1121) dialect pattern: blankCudaConstructs (offset-preserving) blanks execution-space specifiers (__global__ family), __launch_bounds__(...), and <<<grid, block>>> launch configs — which otherwise lex as shift operators and destroy the host→kernel call edge entirely. Gated by .cu/.cuh extension OR by content (looksLikeCudaSource), because much real CUDA lives in .h/.hpp headers: cutlass launches most kernels from headers and flash-attention's launch templates are .h. Safe by construction — no CUDA marker is valid C++ anywhere, and the launch blank is bounded + brace-balance-checked so a stray <<< (committed merge-conflict markers) can never blank real code. All real-world launch styles connect: plain, templated (k<T, 256><<<...>>>), function-pointer (auto kernel = &fn<...>; with branch reassignments each linked), dim3{...} brace-init configs, and kernels defined through name-in-first-argument macros (DEFINE_FLASH_FORWARD_KERNEL style — gtest TEST_F / PYBIND11_MODULE shapes deliberately excluded by the two-lone-identifiers rule). Two general C++ resolution wins the flow validation forced out: - namespace blocks now prefix contained symbols' qualifiedNames (prefix-only — no namespace nodes, avoiding #1093-style crowd-out), so ns::fn(...) calls resolve; previously every namespace-qualified C++ call was a permanently dead edge. cutlass: +30,864 edges (~10%), node count byte-identical. - templated callees (fn<T, 256>(args)) strip template args at extraction (mirroring #1043 for base classes), so they match their definitions. Validated on llm.c (165 host→kernel launch edges, was 0), flash-attention (run_flash_fwd → flash_fwd_kernel → compute_attn traces in one codegraph_explore call), and NVIDIA CUTLASS; fmt as the plain-C++ control (unchanged). A/B n=2/arm: Read/Grep displacement decisive on all three repos (flash-attention Reads 29,13 → 5,2). Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
1441933a26 |
feat(extraction): add Solidity language support (.sol) (#374, #648) (#1170)
Contracts/libraries/interfaces, structs, enums, modifiers, events, errors, state variables; call edges for emit/revert/modifier guards/base-constructor chains/library calls; is-inheritance with implements reclassification; import resolution. Validated on solmate, solady, openzeppelin-contracts. Lands #667. Co-authored-by: naiba <hi@nai.ba> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
a0208feaac |
feat(extraction): index Erlang escripts and OTP app resource files (#635, #648) (#1169)
escripts (.escript) index like any module — the ELP grammar has a
first-class shebang node, so no source transform is needed; main/1 and its
helpers get full function/call extraction.
OTP application resource files (<app>.app.src and compiled <app>.app) join
the graph as Erlang terms the grammar parses natively. They route by full
suffix (their last-dot extension, .src, is far too generic for the
extension map). The application tuple yields structure: {mod, {Mod, _}}
links the app to its callback module — the app's entry point — and
{applications, [...]} / {included_applications, [...]} connect umbrella
sibling apps, resolving through the OTP app-name == module-name convention;
kernel/stdlib and other out-of-repo apps stay unresolved.
App-file refs resolve only ever to MODULES: validation on emqx caught the
ssl OTP-app dependency resolving to a test helper FUNCTION named ssl (the
same defect class as the earlier -behaviour gate), so the matchReference
module-only gate now covers every ref an .app/.app.src file emits.
Validated on emqx: 2 app.src + 6 escripts indexed, entry-module and
umbrella-dependency edges all namespace-targeted post-gate, escript
functions extracted; a stray legacy/module.src stays unknown.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
|
a5b8cd8e25 |
feat(extraction): Erlang macro-body call linkage (#635, #648) (#1168)
Calls hidden inside -define bodies were invisible: the extractor consumed pp_define without walking the replacement, and macro use sites produced no edges, so a call path routed through a macro (ejabberd's SQL upsert macros, logging wrappers) was completely dark. The macro's constant node now participates in the graph. The -define body's calls are attributed to the MACRO — true exactly once, instead of a per-use duplicate that would explode on logging macros — and each use site links in: ?MACRO(...) with arguments emits a `calls` ref (inlined code joins the call chain), a bare ?CONSTANT read emits `references` (answering "where is this macro used" without polluting call paths). Compiler-predefined macros (?MODULE, ?LINE, ?FUNCTION_NAME, ...) are excluded, macro-use arguments keep walking so a call nested in ?assertEqual(ok, do_thing()) still attributes to the enclosing function, and macro-to-macro chains connect. Validated: node counts unchanged on cowboy/ejabberd/emqx; edges +26/+7.3K/ +42K with honest hub shapes (?T i18n, ?SLOG logging, ?QOS_1 protocol constants); 40/40 sampled edges precise; +1.3s index cost on emqx's 2,273 files. The payoff chain on ejabberd: set_password_scram_t → ?SQL_UPSERT_T → ejabberd_sql:sql_query_t — database writes through SQL macros now trace end-to-end. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
7e3d44fa96 |
feat(extraction): Erlang gen_server registered-name dispatch targets (#635, #648) (#1167)
gen_server:call/cast/send_request now connects to the TARGET module's
handle_call/handle_cast for every statically-named target, not just self:
a bare atom reaches the module of that name (OTP's {local, ?MODULE}
convention names a server after its module), and a macro defined as a bare
atom (-define(STORE, kv_store)) resolves the same way, alongside the
existing ?MODULE / -define(SERVER, ?MODULE) self paths. A registered name
that matches no module emits a qualified ref that never resolves — silent,
never guessed. Pid, var, and tuple targets ({global, Name}, {Name, Node})
stay unlinked.
Validated on emqx: 53 new edges, 53/53 precise (each source line is a real
registered-name gen_server request; each target module self-registers under
that name, macro-indirected registrations included). Nearly all are
test-suite → handler links — production code goes through API wrappers the
self path already covers — which is exactly the tests-exercising-this-
handler linkage blast-radius and test-gap reporting consume. ejabberd
yields zero (it always wraps): no false positives invented.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
|
6511722250 |
feat(extraction): add Erlang language support (.erl/.hrl) (#635, #648) (#1165)
Vendored WhatsApp/tree-sitter-erlang 0.19 (the ELP grammar, ABI 14) with an Erlang-shaped extractor: multi-clause/multi-arity functions merged into one symbol, -spec signatures, records with fields, -type/-opaque aliases, -define macros, -include/-include_lib file edges, and -export-driven visibility. Modules wrap in a namespace so remote mod:fn(...) calls resolve through the existing qualified-name matcher as mod::fn with zero resolver changes. -behaviour declarations link to the behaviour module — gated to namespace targets only (bare-name fallthrough linked -behaviour(supervisor) to an unrelated macro constant on emqx). OTP indirection with static targets is followed: spawn/apply/proc_lib/timer/rpc MFA-argument callees, and gen_server:call/cast(?MODULE | ?SERVER) to the module's own handle_call/handle_cast. Var-module dispatch and message sends stay deliberately unlinked. codegraph_explore also normalizes Erlang-native query spelling (mod:fn/3, init/2) so named symbols resolve as typed. Benchmarked on cowboy (189 files), ejabberd (414), emqx (2,447): extraction PASS on all three; with-codegraph arms reached 2/2/0 file Reads vs 10/5+/19 without, fastest on the largest repo. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
63e1b5a23a |
feat(extraction): add Visual Basic .NET language support (.vb) (#648, #639, #170) (#1164)
Vendored patched govindbanura/tree-sitter-vbnet grammar (MIT, ~20-fix patch + new external scanner for XML literals and multi-line LINQ continuation; provenance + rebuild instructions in docs/grammars/tree-sitter-vbnet.md), vbnet extractor with VB-specific call/index disambiguation, Inherits/ Implements heritage, As New instantiation, events, Declare P/Invoke, and MustOverride abstract members. Parse health on five real repos: PolicyPlus 100%, CompactGUI 100%, staxrip 95.2%, SCrawler 87.2%, PCL 87.5% (upstream grammar: 3-18%). Retrieval A/B (sonnet): 26-43% faster with 0-5 file reads vs 7-20 without. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
41620c60fa |
feat(extraction): add COBOL language support (.cbl/.cob/.cpy) (#590, #648) (#1161)
Programs, sections/paragraphs (reconstructed extents over the grammar's flat header stream), PERFORM/THRU/GO TO/CALL call edges, COPY copybook imports incl. standalone .cpy fragments, DATA DIVISION records/fields/ 88-levels with write-site impact references, and CICS flows: EXEC LINK/XCTL program targets (literal + same-file VALUE deref), EXEC SQL INCLUDE, and pseudo-conversational RETURN/START TRANSID hops resolved to the owning program via a CICS framework resolver. Fixed and free source format (free format via a scanner wide-mode sentinel). Grammar: vendored wasm built from a patched yutaro-sakamoto/ tree-sitter-cobol (EXEC blocks as an external-scanner token, copybook fragment entry point, single-quote continuation, COPY REPLACING pseudo-text, NOT=, CALL GIVING, ENTRY, FREE, bitwise ops, abbreviated relations, COBOL-2002 usages, and more). Patch + provenance + upstream PR draft in docs/grammars/. Parse health: AWS CardDemo 43/44 native (upstream: 9/31), 44/44 through preParse; copybooks 28/29; CobolCraft free-format 17/17 (upstream: 0); NIST COBOL85 unchanged at 373/382. Copybook members resolve to files like C includes (basename index, name-matcher short-circuit so compiler-supplied members stay honestly unresolved): CardDemo imports 5 -> 285. Impact proof: ACCT-CURR-BAL (CVACT01Y copybook) surfaces its 4 writer programs cross-file. Also: run-all.sh now neutralizes the ambient prompt-hook in both A/B arms (CODEGRAPH_NO_PROMPT_HOOK=1); COBOL corpus entries for agent-eval. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
7d624ecfac |
feat(resolution): CFML receiver-type inference for locals, typed args, and component properties (#1155)
CFML joins the #1108 receiver-inference family: new/createObject/typed-arg/property(inject) declarations type the receiver, variables./this. fields scan whole-file, method QNs re-scoped to Class::member in all three extraction paths. 1,649 typed edges on fw1/ColdBox/CFWheels, 1,649/1,649 audit-consistent, inherited methods resolve via #1152 extends edges. Co-authored-by: ghedwards <125586+ghedwards@users.noreply.github.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
816bacb7f2 |
feat(extraction): add CFML language support (.cfc/.cfm/.cfs) (#1118) (#1153)
Tag-based and bare-script CFML, extends/implements, <cfscript>/<cfquery> delegation, BOM + unquoted-attribute handling. Wasm grammars verified bit-for-bit reproducible from cfmleditor/tree-sitter-cfml. Validated on FW/1, ColdBox, CFWheels. Follow-up: #1152. Co-authored-by: ghedwards <125586+ghedwards@users.noreply.github.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
cc89146454 |
feat(extraction): index Metal shader files (.metal) via the C++ grammar (#1121) (#1151)
.metal was absent from EXTENSION_MAP, so Metal Shading Language files were silently skipped. MSL ≈ C++14, and the C++ grammar extracts its functions, structs, type aliases, and call edges at parity with plain C++ — except MSL's post-declarator [[attribute]] annotations, which misparse struct fields into spurious extends refs from the struct to the field's own type (a wrong inheritance edge whenever the repo typedefs float3/float4x4 itself, common in shared ShaderTypes.h). blankMetalAttributes blanks them pre-parse, offset-preserving, following the blankCppExportMacros pattern (#1061), gated to .metal files only — in regular C++ the attribute position is legal syntax the grammar parses natively. The preParse hook gains an optional filePath param to support the gate. Validated on llama.cpp's ggml-metal.metal (10.7k lines: 130 kernels vs 113 `kernel void` ground-truth lines, rope_yarn resolves its 4 kernel callers) and SDL's shaders (PQtoLinear ← GetOutputColor), 0 bogus extends edges. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |