A C++ class annotated with an export/visibility macro between `class`/`struct`
and the type name — `class MYMODULE_API UMyComponent : public UActorComponent`
(the standard Unreal-Engine `*_API` pattern), or the equivalent `*_EXPORT`/
`*_ABI` macros in Qt, Boost, LLVM, etc. — makes tree-sitter read `class MACRO`
as an elaborated type and the whole declaration as a function. #946 dropped the
resulting phantom function, but that also discarded the recoverable class name,
members, and base-class edge, so the class never entered the graph and
"find subclasses" / type-hierarchy / impact-through-inheritance returned
nothing for effectively every gameplay class in a UE project.
Add `blankCppExportMacros` as `cppExtractor.preParse`: it blanks the macro with
equal-length spaces before parsing (offset-preserving, like C#'s
`blankCsharpPreprocessorDirectives`/#237), so the declaration parses as a normal
class_specifier and existing extraction emits the node, members, and `extends`
edge. Generalized past UE `*_API` to any all-caps export macro, with two
false-positive guards: the trailing `[:{]` definition-guard (leaves elaborated
var decls like `struct FOO var;` alone) and requiring the macro to be followed
by the real name (leaves an all-caps class NAME such as `class FOO : public Base`
alone). C++-only, so C's heavier `struct TAG var;` never reaches it. The #946
drop stays as the fallback for any residual misparse the blanking doesn't catch.
Validated on google/leveldb (LEVELDB_EXPORT, 134 files): class/struct nodes
266→293, extends edges 292→359, phantom functions 588→513; every export-macro
real definition flips function→class and `EnvWrapper extends Env` goes
absent→present.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(explore): surface a named method's buried signature type (change surface) (#1064)
#1064: a natural-language query like "what do I need to change if I add a new
parameter to NewClient" dropped the file the answer lives in (grpc-go's
dialoptions.go, which defines NewClient's DialOption) and surfaced lexical
namesakes instead, so the agent fell back to grep.
Root cause (instrumented on grpc-go): the answer file is lexically dissimilar
to the query and reachable only structurally, so it scores ~0 on every text
and centrality signal and never renders.
Two-part fix, both bounded and validated to not perturb flow queries:
1. Change surface — read each named method's signature-type edges from the full
graph and, ONLY when the type's file is genuinely BURIED (≈0 graph mass AND
no term hits), inject + rank + gate-keep + tier it. A well-connected type
file is left to rank on its own merit, so this never displaces a flow file.
2. Tier de-noise by centrality — still seed every <=3-def name (RWR/flow ranking
unchanged), but the named-first tier admits only the most-substantive def
plus co-named defs of comparable centrality (>=25% of the top def's caller
count). This keeps real overloads/wrappers (excalidraw's `mutateElement` in
three files, callers 74/58/40) while dropping vastly-less-central namesakes
(Go's `NewClient`: real 492 callers vs xds-pool 11, test-fake 3) that would
otherwise crowd the answer file out of the tier.
Validation:
- Deterministic probe: dialoptions.go goes from dropped to surfaced (with the
full option field set via defaultDialOptions).
- Broader-repo regression check (Alamofire, excalidraw, axios): 5/6 control
flow queries byte-identical to baseline; the 1 shift (Alamofire "how a request
gets validated": Validation.swift -> DataRequest.swift) is lateral —
DataRequest defines validate() and is a directly-relevant answer.
- grpc-go agent A/B (n=2, sonnet): grep fallback eliminated (0 vs baseline 4,2).
- 1845 unit tests pass.
* docs(changelog): note explore signature-type surfacing (#1064)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The gitlink discovery added in #1031/#1033 indexed a tracked 160000 gitlink even when the parent .gitignore excludes its directory, pulling a gitignored reference/benchmark corpus of git add'ed clones into the index (one report: ~138k files, 4.8 GiB, wedged "Resolving refs" watchdog).
Gate both gitlink-discovery sites on the same rule the untracked-embedded path already uses: skip a gitignored gitlink unless codegraph.json includeIgnored opts it in; index non-ignored gitlinks as before. Validated real-world on macOS, Linux, and Windows.
Closes#1065
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Release the four CLI/indexing fixes triaged from @jcrabapple's reports
(#1044 node -f, #1045 query %, #1046 explore count, #1047 Android res XML)
plus the rest of [Unreleased]. The Release workflow promotes the
changelog and publishes.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
An Android `res/` tree (layouts, value bags, drawables, menus, navigation
graphs) holds only non-code resources that yield zero symbols, yet on an
Android app it dominates the file count (one report: 26k XML = 97% of
files, 0 symbols) — bloating the DB, slowing indexing, and padding
explore/search results and file counts with entries that have nothing to
find.
Default-ignore the Android resource type directories (`res/layout/`,
`res/values/`, `res/drawable/`, … and their `-<qualifier>` variants) at
discovery, via DEFAULT_IGNORE_PATTERNS so it applies uniformly to the git
index, the non-git walk, and change detection. The `res/<type>/`
structure is self-identifying, so non-Android projects are untouched, and
the only XML that carries symbols — MyBatis mappers under
`src/main/resources/` — never lives under `res/`, so nothing useful is
dropped. `res/raw/` is deliberately kept (arbitrary bundled assets), and
a `.gitignore` negation re-includes anything.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
codegraph_explore's "Found N symbols across M files." header reported
`subgraph.nodes.size` / `fileGroups.size` — the raw FTS gather. A broad
natural-language query ("publish status to the API") matches a huge pool
(260 symbols / 124 files on a 636-file repo) while only a handful clear
the relevance gate + budget and render, so the header read as "260
results to wade through" even though the correctly-ranked answer was the
few files shown.
Report instead the files whose source actually SURVIVES in the final
output (after the hard-ceiling truncation that can drop trailing
sections), summing their relevant symbols. Gather, ranking, gate, budget,
and rendering are untouched — only the header string changes. Overflow
relevant files are still named under "Not shown above", so nothing is
hidden. Adds a regression test locking header-count == rendered-sections.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`codegraph query` printed `(score * 100)%` next to each hit, but `score`
is an unbounded BM25/FTS relevance magnitude (relative-ranking only), so
it rendered as values like "12042%" that made the output look broken.
Results already arrive in rank order, so drop the score from the
human-readable output entirely — matching the MCP search tool, which
shows no score. The raw `score` stays in `--json` for programmatic
sorting/thresholding. Also corrects the SearchResult.score doc comment,
which wrongly claimed a 0-1 range. Adds an end-to-end regression test.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`codegraph node` was defined with a required `<name>` positional, so
commander.js rejected `codegraph node -f <file>` with "missing required
argument 'name'" before the action ran — making file-read mode (the CLI
face of the codegraph_node MCP tool's file mode) unreachable. The action
body already handled an absent name.
Make `name` optional (`[name]`), validate that a symbol or a file is
supplied (friendly usage hint instead of a cryptic commander error when
neither is), and guard the name-based arg branches so they never run on
undefined. Adds an end-to-end regression test across all four paths.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`insertEdge` has always used `INSERT OR IGNORE`, but the edges table carried
no UNIQUE constraint — only an autoincrement PK and non-unique indexes — so
`OR IGNORE` had nothing to conflict on and behaved like a plain INSERT.
Whenever two extraction/resolution passes emitted the same edge (e.g. a
return type captured by both a type-reference and a value-reference pass),
the graph stored byte-identical duplicate rows: ~527 on this repo, inflating
edge counts and letting callers/impact list the same relationship twice.
Add a UNIQUE identity index on (source, target, kind, IFNULL(line,-1),
IFNULL(col,-1)) — in schema.sql for fresh databases and migration v6 (dedup
existing rows, then create the index) for existing ones. IFNULL folds the
nullable line/col so coordinate-less edges (synthesized / file-level) dedup
too; SQLite otherwise treats each NULL as distinct. Distinct call sites
(same source/target/kind, different line/col) are preserved — only
byte-identical structural duplicates collapse. This is the storage-layer
invariant the reporter identified: it makes OR IGNORE keep its promise and
catches every double-emit, present and future, rather than chasing each
emitting pass.
Migration v6 is deterministic (keeps the lowest id per identity group) and
idempotent (IF NOT EXISTS index; no-op DELETE once unique). The DELETE's
GROUP BY matches the index expression exactly so creation can't fail on a
leftover pair.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`instantiates` edges came only from heap `new Calculator(0)` (a
new_expression) and copy-init `Calculator c = Calculator(0)` (a
call_expression). Stack direct-init `Calculator calc(0)` and brace-init
`Widget w{1, 2}` parse as a `declaration` whose constructor arguments hang
directly off the declarator as an argument_list / initializer_list — there
is no call/new node — so the function-body walker saw no constructor
invocation and emitted no edge. A function that built objects with the
ordinary stack syntax looked like it didn't construct them, and the
dependency was missing from impact / callers.
In the body walker, a C++ `declaration` that is a stack/brace construction
now reuses extractInstantiation (a declaration's `type` field IS the
constructed class name, and extractInstantiation already strips template
args / namespace and emits the `instantiates` ref). Gated by
isCppStackConstruction, which requires BOTH a class-like type
(type_identifier / template_type / qualified_identifier — so `int x(0)`
and `auto z = …` are excluded) AND a declarator carrying args
(argument_list / initializer_list — so default `Calculator c;` and the
most-vexing-parse `Calculator c();` are excluded). The edge targets the
class node, not the same-named constructor method.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A C++ class deriving from a template — `class Derived : public Base<int>`,
a CRTP base `class App : public CRTPBase<App>`, a struct inheriting a
template, or a templated base mixed into a multi-base clause — recorded its
base as the full instantiation text (`Base<int>`). That never name-matched
the template, which is indexed as the bare node `Base`, so the `extends`
edge never resolved and the derived class looked like it inherited from
nothing — callers/impact analysis stopped at the boundary.
Strip the template arguments from the base-type reference name in the
`base_class_clause` handler via a new `stripCppTemplateArgs` helper: it
removes every balanced `<…>` group (any nesting/position), so `Base<int>`
→ `Base` and `ns::Tpl<int>` → `ns::Tpl`. The remaining qualified head is
exactly what the non-templated base case already produces, so resolution
treats templated and non-templated bases identically; a name with no
template args passes through unchanged.
Covers same-file and same-namespace bases (the dominant real-world
patterns). A base in a different namespace referenced with its qualifier
(`other_ns::Tpl<int>`) still doesn't resolve, but that's a pre-existing,
orthogonal namespace-resolution gap — the non-templated `other_ns::Plain`
fails identically — not a template issue.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
On Windows, calling process.exit() while a recursive fs.watch handle is still
tearing down aborts the daemon with a libuv UV_HANDLE_CLOSING assertion
(0xC0000409) — reproducible whenever the indexed tree contains a nested repo
(submodule / embedded clone), since that's what keeps a watch active at shutdown.
A small exit delay doesn't help; only letting the loop drain is clean (verified
on a real Windows VM: close()+exit() and close()+setTimeout(exit) both abort,
while letting the loop drain exits 0).
finalizeDaemonExit() now exits immediately on POSIX (unchanged) but on Windows
marks success (exitCode=0) and lets the loop drain to a natural exit, with an
unref'd backstop that force-exits only if a stray handle would otherwise hang
shutdown. The daemon's own timers are already unref'd and its PPID watchdog lives
in the proxy, so nothing keeps the loop alive past the closing watch handles —
natural drain is fast. Pure + platform-injected so both branches unit-test off-Windows.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Indexing a super-repo now descends into its submodules and gitlinked clones, so
a query run from inside one resolves up to the parent's unified index — whose
graph DOES contain that nested repo's files. But the git-worktree-mismatch
warning still fired, telling the agent the results were from "a different working
tree" and to run `codegraph init -i` — which would split the submodule back into
its own index and undo the unified view. A false positive carrying harmful advice.
Distinguish a genuine borrowed worktree (the SAME repository on a different
branch — shares a git common dir with the index root) from a submodule/embedded
clone (a DIFFERENT repository — its own common dir), and suppress the warning
only for the latter. Add gitCommonDir() for the check. The issue-#155 linked-worktree
case is unchanged.
Verified end-to-end: the warning no longer fires for a submodule-rooted MCP
session and still fires for a real linked worktree. Edit-sync (manual sync +
the live watcher) keeps the nested repo's files current on both macOS and Linux
(active-submodule and bare-gitlink shapes), so suppressing the warning is safe.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A nested git repo tracked as a gitlink (mode 160000) — a clone `git add`ed
into the super-repo without a `.gitmodules` entry, or a submodule that
isn't active/initialized in this checkout — fell through both file-collection
passes: it's tracked, so the untracked `-o` listing skips it, but it's not
an active submodule, so `--recurse-submodules` won't expand it. Indexing the
top level therefore pulled in only the outer repo's own files and stopped at
the nested repo's boundary (one report: ~10 files at the root).
Switch the tracked scan to `ls-files -s` to expose file modes, collect the
unexpanded 160000 entries, and recurse into each that has a real working tree
on disk as its own embedded repo. Mirror the same discovery in
discoverEmbeddedRepoRoots so the watcher's scope stays equal to the indexer's.
Active submodules (#147) and untracked nested clones (#193) are unchanged;
gitlinks under default-ignored dirs (vendor/, node_modules/) stay excluded
(#407); an uninitialized submodule with no checkout on disk is left alone.
Adds four-shape coverage in extraction.test.ts.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
All codegraph_* tools are query-only — they read the pre-built index and
never mutate the workspace — but they advertised no MCP annotations, so
Cursor's Ask mode (and any client that gates on read-only tools) blocked
every call with "you are in ask mode and cannot run non read-only tools."
Add a shared READ_ONLY_ANNOTATIONS constant (readOnlyHint: true,
destructiveHint: false, idempotentHint: true, openWorldHint: false) and
reference it from each of the 8 tool definitions. The field flows through
every tools/list path: the live getTools() (including explore's
spread-rewritten description), the static proxy getStaticTools(), and the
no-default withRequiredProjectPath schema clone.
The annotations field is additive, so it ships without bumping the
negotiated 2024-11-05 protocol version: clients that gate on it read it
regardless, and older clients ignore it.
Closes#1018
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
indexAll parsed every file through a single worker thread, so a full `codegraph index` used one core no matter the machine. Add ParseWorkerPool (src/extraction/parse-pool.ts), modeled on the shipped QueryPool: indexAll now parses across clamp(cores-1,1,8) workers. CODEGRAPH_PARSE_WORKERS overrides the count; 1 reproduces the previous single-worker path exactly (the rollback).
Parses run concurrently but results commit to SQLite in file order. This matters: the post-index resolution phase selects among ambiguous same-named candidates by node DB-insertion order, so a stable commit order keeps the graph deterministic — byte-identical to the serial path — instead of drifting with parse-completion timing. A bounded reorder buffer (backpressure on dispatched-but-uncommitted count) keeps memory flat even if a file is slow at the commit cursor.
Crash/timeout of a worker rejects only that file's parse (feeding the existing retry pass) and respawns; per-worker recycle every 250 parses reclaims WASM heap. In-process fallback unchanged when the compiled worker is absent (tests).
Validated on real OSS (django +9%, redis +17%; modest and parse-fraction-dependent), graph byte-identical across worker counts, peak memory flat-to-lower since workers recycle independently — so the #320 OOM concern doesn't materialize. Adds 11 pool unit tests.
Closes#1015. Refs #320.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Swift in-class properties are extracted by a dedicated branch in
TreeSitterExtractor.visitNode, not the generic nameField/variableTypes path
swift.ts declares. That branch had a `!isComputed` gate that dropped computed
properties entirely, so `codegraph query`/`codegraph_explore` returned "No
results found" for them — including a SwiftUI view's `var body: some View`,
the most important symbol in any SwiftUI app, and the heavily-read
`var isCloudProxy: Bool` from the report.
Stored properties were already fixed in #708 (v1.0.0); the reporter tested
v0.9.9 and confirmed "still present on main" by inspecting swift.ts only,
missing the dedicated branch — so only the computed-property half was real.
- Computed properties now index as `property` nodes; the getter is walked via
visitFunctionBody so its calls attribute to the property (a SwiftUI `body`'s
subview tree becomes the property's callees — the render flow is traceable
through it), not flattened onto the enclosing type.
- Protocol property requirements (`var x: T { get }`) — a third never-indexed
category — index as `property` too.
- Routing the getter through visitFunctionBody also stops getter-local
`let`/`var` declarations from being wrongly node-ified as struct fields
(the generic child-walk used to do this): Alamofire property 0→348, field
618→588, idempotent.
Stored/static behavior is unchanged.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
deleteResolvedReferences bound every id into a single unbounded
`IN (...)`, so a list longer than SQLITE_MAX_VARIABLE_NUMBER (32766 on
the bundled node:sqlite) threw "too many SQL variables" — the one IN-list
in queries.ts that #540 missed. It's reachable only through the exported
QueryBuilder (library use): the internal resolution path uses
deleteSpecificResolvedReferences, which binds per-row and is immune, so
the CLI/MCP indexing pipeline was never affected. Wrap it in the same
SQLITE_PARAM_CHUNK_SIZE loop every sibling query uses, and add a
regression test (33k ids, past the real 32766 ceiling) that throws
without the fix.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A project kept on an ExFAT/FAT external volume (or some network mounts /
WSL2 DrvFs) broke the background auto-sync daemon at two points, both
because the filesystem lacks POSIX features the daemon relied on:
1. Lock acquisition hard-links a temp file onto .codegraph/daemon.pid for
race-free exclusivity (#411) — these filesystems have no hard links.
2. The Unix-domain socket listen() fails regardless of path length, so the
old length-only tmpdir fallback never triggered.
Both surface as a capability error, but each OS reports a DIFFERENT errno
for the same gap (macOS ENOTSUP, Linux EPERM, Windows EISDIR), so the fix
is policy-based rather than an enumerated code-set:
- Lock: fall back to an O_EXCL create on any non-EEXIST link error. The
temp write already proved the directory is writable, so the fallback
either succeeds (still atomic + exclusive, "first writer wins") or
surfaces its own genuine error.
- Socket: an ordered candidate list [in-project, tmpdir] walked by BOTH
the daemon (binds) and the proxy (connects) — they converge on the
fallback with zero coordination. Relocate past any non-EADDRINUSE bind
error; EADDRINUSE still rethrows, preserving the #974 contract.
Normal repos are unaffected: the in-project candidate binds first, and the
hard-link lock path is unchanged.
Validated end-to-end on real removable-drive filesystems: macOS ExFAT
(hdiutil image), Linux FAT32 (Docker loop mount), Windows exFAT (diskpart
VHD) — each acquires the lock, relocates (or binds a named pipe on
Windows), and serves a real client.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#1009 added `exclude` (keep git-tracked dirs out of the index) but didn't
document it. Add an "Excluding a tracked directory" section to the site config
page (parallel to includeIgnored) and a brief note + example to the README,
covering the committed-theme/SDK case .gitignore can't handle.
Three fixes for a repo that commits a large JS/TS theme/SDK (Metronic under
static/, ~1,600 tracked files):
1. A SECOND "Resolving refs" quadratic that #915 didn't cover. #915 capped
import-name collisions; this caps method-name collisions (init/update/render
re-declared on every widget), which flow through matchMethodCall Strategy 3
and findBestMatch instead. New AMBIGUOUS_NAME_CEILING (default 500, env
CODEGRAPH_AMBIGUOUS_NAME_CEILING): above it the fuzzy strategies decline
rather than score K candidates — no proximity score can pick the one true
target among thousands anyway. Resolving drops from O(K^2) to linear in refs
(e.g. 900-file synthetic: 28.7s -> 3.4s), edge counts unchanged, and the cap
never fires on normal repos (max real method-collision ~40).
2. A new `exclude` array in codegraph.json keeps git-TRACKED paths out of the
index, which .gitignore can't do (enumeration is `git ls-files`). Mirrors the
existing includeIgnored plumbing across the git, sync, and non-git-walk
paths.
3. `index`/`init` now install the #850 liveness + #277 ppid watchdogs (which
were serve-only), so a wedged or orphaned indexer self-terminates instead of
pinning a core. The --liftoff-only relaunch's spawnSync can't forward
signals, so killing the parent shim used to orphan the worker.
Tests: ubiquitous-name ceiling, exclude (incl. tracked-file exclusion on git +
non-git), orphan self-termination (POSIX), and ppid-parser units. Shared the
ppid parsers out of mcp/index.ts into mcp/ppid-watchdog.ts.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
When the server runs with no default project to fall back to — a gateway
server started outside any repo, or a monorepo root whose .codegraph/
indexes live only in sub-projects — every tool call must carry an explicit
projectPath. Previously projectPath was always optional, so an agent talking
to such a server would omit it, get success-shaped "pass projectPath"
guidance, and not reliably retry; the user had to nudge it by hand.
getTools() now marks projectPath required in the exposed tool schemas on the
no-default-project branch (a high-salience channel clients surface/validate,
unlike the instructions prose the reporter found too weak). When a default
project is open, projectPath stays optional and a bare call falls back to it.
The fix lives at the MCP schema layer, not the Claude-only front-load hook:
the hook is local-filesystem-based and never runs for the reporter (they're
on AGENTS.md / Codex-opencode). The proxy/getStaticTools path is untouched —
index.ts forces direct mode whenever resolveDaemonRoot is null, so the
no-default case never reaches the proxy.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The UserPromptSubmit hook's structural-prompt gate was English-only, so a
structural question written in Chinese — or any non-Latin script — silently
injected nothing: JS `\b` is ASCII-only and never matches between Han
characters, so the keyword regex couldn't fire (and couldn't be extended in
place). To the user the hook looked unwired, with no error to explain why.
Make the gate language-aware, split into tested helpers in directory.ts:
- hasStructuralKeyword: English (\b-guarded) + CJK structural keywords.
- extractCodeTokens: identifier-shaped tokens (camelCase / snake_case /
name() / a.b) in any language — verified against the index via
getNodesByName before firing, so a tech brand like `JavaScript` that looks
like a symbol but isn't one here doesn't inject ~16KB of spurious context.
- isStructuralPrompt: the cheap candidate gate (keyword OR code-token).
Adds 21 unit tests for the gate (previously untested) covering the reporter's
verification table plus the false-positive guards.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(c/c++): resolve macro-built function-pointer command tables (#991)
C/C++ commands dispatched through macro-built function-pointer tables were
dead-ends in the graph: redis' `call` never showed up as a caller of any
command (`c->cmd->proc(c)`), because the table is generated into a #included
`.def`, the handler is buried inside `MAKE_CMD(...)`, the struct type is itself
a macro alias, the `proc` field uses a function-TYPE typedef, and the receiver
is a chained field access. #954 deferred exactly this shape.
Six composable additions to c-fnptr-synthesizer.ts close it:
- function-type typedefs (`typedef RET T(...)` + `T *f`) flag the field as a
function pointer;
- multi-declarator fields (`struct redisCommand *cmd, *last`) each count as a
slot/type (needed for positional alignment and the chain walk);
- chained/array receivers (`c->cmd->proc`) resolve through field types across
all same-named struct layouts (redis has two unrelated `client` structs);
- `#include "x"` directives are followed (from raw source) so a non-indexed
`.def` is read as a registration unit with the includer's effective macro env;
- function-like + object-like macros are expanded (params->args, type aliases)
before positional/designated registration;
- a macro that expands to a brace-wrapped element (sqlite `FUNCTION(...)`) has
one outer brace layer peeled.
Validated on two independent macro-table lineages at 100% target precision:
redis (209 commands via redisCommand.proc, `call`->every command) and sqlite
(69 FuncDef.xSFunc targets). No regression on the controls: git (cmd_struct.fn,
138 builtins), curl (Curl_cftype.*), lua (0). 0 non-function targets across all
five; +3 synthetic fixtures; full suite green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(c/c++): resolve conditional-compilation command tables (vim) (#991)
Vim's `:ex` and normal-mode command tables are the hardest fn-pointer-table
shape: the struct is defined INLINE with the array, the whole thing is behind
`#ifdef DO_DECLARE_EXCMD`/`DO_DECLARE_NVCMD` (switched on by the includer), built
by a macro the file conditionally redefines (`EXCMD`/`NVCMD` = the table element
under the switch, a bare enum id otherwise), and dispatched by a parenthesized
array subscript through a file-scope table: `(cmdnames[i].cmd_func)(&ea)`.
Four more composable additions on top of the macro-table work:
- a focused `#ifdef`/`#ifndef`/`#if defined`/`#else`/`#elif`/`#endif` evaluator
drops inactive arms (unevaluable `#if EXPR` keeps its body); an indexed header
is re-scanned in an includer's context only when that includer #defines a
switch the header guards, with the include's macros re-read from the resolved
text (the plain last-wins parse picks the wrong, enum, arm);
- inline `struct TAG {…} var[] = {…}` tables whose struct never became a node are
parsed in place and registered;
- array-subscript receivers (`tbl[i].f`) strip the subscript and resolve the
base through a global-var → struct-type map;
- an optional `)` before the call covers the parenthesized `(….f)(args)` form.
Validated on vim: 273 `:ex` commands (`do_one_cmd`→every command) + 67
normal-mode commands, 0 non-function targets, 0 cross-table misroute (registering
both tables is what stops `normal_cmd`'s `nv_cmds[i].cmd_func` from falling back
to the `cmdname` owner of the shared field name). Controls unchanged at 0
non-function (redis/sqlite/git/curl gain coverage from array/global dispatch, lua
still 0); +1 synthetic fixture; full suite green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(c/c++): resolve bare arrays of function pointers (#991)
The C/C++ fn-pointer synthesizer keyed everything on (struct type,
fn-pointer field), so a dispatch through a bare array of function
pointers — no struct, no field — was unbridged: an opcode/handler table
like `static op_t *opcodes[256] = {nop,…}` invoked `opcodes[op](…)` left
every handler with zero callers. Closes the last #991 deferred item.
Keyed by the array VARIABLE name (a new `arrayReg`, parallel to the
struct `reg`). Registration detects an array whose element type is a
function typedef — a function-TYPE typedef element (`opcode_t *ops[]`,
the `*` making it an array of pointers) or a function-pointer typedef
element (`zend_rc_dtor_func_t t[]`) — and reads its literal entries,
whether positional (`fn`/`&fn`), designated by index (`[IDX]=fn`), or
cast-wrapped (`(cast)fn`). Dispatch is `tbl[i](…)` / `(*tbl[i])(…)`,
gated on `tbl` being a known fn-pointer array (the precision anchor);
the fan-out reaches the whole set (a runtime subscript hits any entry),
like a command table. The same-file table wins on a name collision, so
two file-local `static opcodes[256]` (SameBoy's CPU + disassembler)
never cross. The fn-pointer typedef/field regexes now also tolerate a
calling-convention macro before the `*` (`(ZEND_FASTCALL *name)`), which
hardens the existing struct-field path too.
Validated on two independent lineages: SameBoy (GB emulator) — 147 edges
via `opcodes[]`, 0 cross-file leak; php-src (Zend) — 54 edges across 7
tables in the designated+cast+CC-typedef form. Control: lua 0 — its
`lua_CFunction searchers[]` is pushed into the VM, never C-dispatched, so
the call-gate fires nothing. No regression on the #991 corpus: redis
(835) / sqlite (683) struct edges byte-identical, git +3 / curl +20
legitimate new bare-array edges, vim 433 with all guards holding; 0
non-function targets across all. + 4 fixtures.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The shared daemon served every session on one event loop with synchronous
node:sqlite. codegraph_explore is CPU-bound work stitched together by microtask
awaits, so N concurrent explores keep the microtask queue continuously full and
starve the macrotask phases — timers AND socket I/O. The transport freezes: no
response can flush until the whole batch drains, so with ~10 subagents on a large
repo clients routinely time out (reported via X by @symbolic2020).
Move the heavy read-tool dispatch onto a worker-thread pool. Each worker holds
its own WAL read connection (verified: a worker reader sees the main writer's
committed catch-up/watcher writes); the single watcher/writer, the catch-up gate,
codegraph_status, and the staleness/worktree notices stay on the main thread.
Concurrent reads now run in true parallel up to core count and the main loop
stays free for the MCP transport, so responses flush incrementally instead of
all-at-once after the batch drains. Enabled for the shared daemon only; direct
(single-stdio-client) mode is unchanged.
- crash recovery: respawn + retry-once, with a circuit breaker that falls back
to in-process dispatch if workers can't run on this platform
- graceful backstop: an overloaded pool returns success-shaped "busy, retry"
guidance, never isError (so it can't teach the agent to abandon codegraph)
- pending-aware growth + capped concurrent cold-starts avoid a startup
thundering herd (N simultaneous module-loads + DB opens could stall the loop)
- config: CODEGRAPH_QUERY_POOL_SIZE (default clamp(cores-1, 1, 16); 0 disables
→ in-process), CODEGRAPH_QUERY_BUSY_TIMEOUT_MS (default 45s)
10 concurrent explores on vscode (10.5k files): 31s → ~9s, staggered flush,
0 timeouts, byte-identical output; scales with cores (≈3.3× on 8, 1.8× on 2).
Full suite passes plus 10 new query-pool tests (fake-worker injection so the
scheduling logic is covered without spawning threads).
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The client-facing MCP proxy could exit with "Transport closed" when its
connection to the shared daemon hit a socket 'error' with no listener
attached — common on WSL2 /mnt (DrvFs), where AF_UNIX is flaky. The global
fatal handler turned that uncaughtException into process.exit(1), which the
MCP client saw as a bare transport close even though the index was healthy.
proxy.ts now keeps an 'error' listener on the daemon socket for its whole
life (and skips a socket destroyed in the connect window), so a stray error
degrades to the existing in-process fallback instead of crashing. daemon.ts
releases the lockfile it acquired when it fails to bind, so the next launch
doesn't spin on a stale lock (the duplicate serve --mcp pileup).
No default behavior change for anyone; WSL /mnt users who still hit trouble
can set CODEGRAPH_NO_DAEMON=1 to skip the shared daemon entirely. Validated
on macOS (unit + live serve probe) and Linux (Docker, --init): 64/64 across
the daemon/socket/lifecycle suites, incl. real AF_UNIX.
Closes#974
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
PR #980 merged after v1.1.0 was already promoted and published, so the
squash-merge's 3-way merge auto-placed its CHANGELOG entry under the
released [1.1.0] section. Move it to [Unreleased] so the published 1.1.0
notes stay accurate and the next release (1.1.1) promotes the fix.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#514 (v1.0.0) began walking into gitignored directories to discover and
index the git repos nested inside them. That broke users who rely on
.gitignore to exclude a directory: a gitignored folder of cloned
reference repos blew graphs up (one report went 10k to 500k edges, #976)
and stalled indexing on multi-gigabyte trees of clones (#970).
Respect .gitignore by default again. Discovering embedded repos inside a
gitignored directory is now opt-in via codegraph.json:
{ "includeIgnored": ["packages/", "services/"] }
The single choke point findIgnoredEmbeddedRepos now returns nothing
unless a gitignored dir matches the project's includeIgnored patterns,
and the matcher is threaded from the scan root through the full-index,
incremental-sync, and watcher-scope paths. Downstream ScopeIgnore and the
watcher are unchanged: they key off the discovered embedded roots, so
gating discovery fixes the indexer, sync, and watcher together. Untracked
embedded repos (#193) stay indexed by default.
This restores the super-repo-of-clones behavior (#622, #699) for the
people who want it, while making the default match what every other tool
(and CodeGraph's own git ls-files foundation) does: .gitignore excludes.
project-config.ts now parses codegraph.json once (loadParsedConfig) and
exposes loadIncludeIgnoredPatterns alongside the existing extension map.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The MCP server gated tool availability on whether the server root had a
.codegraph/ index, so in a monorepo where only sub-projects are indexed the
agent saw zero tools — and couldn't reach an indexed sub-project even by
projectPath. A session started before `codegraph init` also never surfaced the
tools afterward. The Claude front-load hook had the mirror gap: it only walked
UP for an index, so it stayed silent at a monorepo root.
MCP server:
- Always expose the tool surface; when the root isn't indexed, send a
per-project instructions variant (pass projectPath) instead of the
"inactive" note. Safety comes from response SHAPE (success-shaped guidance,
never isError), not from hiding tools.
- Reword the no-default-project guidance to be per-project, not per-session,
and sharpen the projectPath schema description.
Front-load hook (UserPromptSubmit):
- Scan DOWN (bounded depth, workspace-root-gated) for indexed sub-projects and
shape the injection by topology: front-load the one the prompt names, nudge
about the rest, or list them when ambiguous.
Verified: full suite (1703 passed); a live two-package monorepo run confirms the
hook front-loads the correct sub-project with no cross-package leakage. The
front-load's net speed effect is the existing multi-file-vs-single-file
tradeoff, unchanged by this work.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* perf(resolution): resolve imports to definitions, not sibling import nodes (#915)
"Resolving refs" crawled (tens of minutes) on large projects — most painfully
ones mixing a big front-end and back-end. An external package or module imported
across hundreds/thousands of files (react, a shared UI package, Python
logging/typing) is re-declared as an `import` node in every importing file, so
its unresolved import ref fell through to the exact-name matcher, which scored
all K same-named import nodes via findBestMatch — K refs x K candidates = O(K^2)
per package, producing only meaningless import->import edges.
Fix: exclude `import`-kind nodes as name-match targets (they're statements, not
definitions; real import->definition resolution is the import resolver's job).
Plus two safe constant-factor wins in findBestMatch: hoist the per-candidate
ref.filePath split, and skip cross-language candidates when a same-language one
exists (provably the same winner — same-language scores >=50, cross-language
maxes at 35).
Measured: superset (Py+TS) candidates scored 7.5M -> 833K (9x), non-import edges
preserved (+1618 now resolve to real defs), ~22K useless import->import edges
removed; kubernetes (Go) computePathProximity 37.2s -> 5.0s; synthetic 8k-file
mixed repo (K=4000) resolution 16.0s -> 1.7s. Full suite green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: correct stale better-sqlite3/wasm references to node:sqlite
The SQLite backend has been Node's built-in node:sqlite (real SQLite, WAL + FTS5,
from the bundled runtime) for a while — there is no native build step and no
node-sqlite3-wasm fallback. README and the docs site were already updated; this
catches the stragglers:
- CLAUDE.md: the src/db/ backend description and the sqlite-backend test note.
- src/db/index.ts, src/mcp/tools.ts: two code comments that still blamed "the
wasm backend" for non-WAL behavior (reworded to "when WAL isn't in effect").
Leaves tree-sitter grammar wasm (web-tree-sitter / --liftoff-only) untouched —
that's a different, still-current use of wasm.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(telemetry): drop the dead sqlite_backend field (schema v2)
node:sqlite is now the only backend, so the `index` event's `sqlite_backend`
field was a constant ("native") carrying no signal — and the `install` event
never actually sent it. Remove the field and the backendKind() helper, bump the
telemetry SCHEMA_VERSION 1 -> 2, and update TELEMETRY.md + docs/design/telemetry.md.
The ingest worker is deliberately left tolerant: `index` doesn't require the
field and schema_version validates as nonNegInt(99), so v2 events ingest fine and
old clients still sending v1 + sqlite_backend keep validating too. Added a legacy
comment there explaining it's safe to drop once old-client share is negligible.
telemetry.test.ts: the assertion pinning schema_version and a stale-claim fixture
line updated 1 -> 2. All telemetry tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
GoFrame's standard router binds routes reflectively (group.Bind(ctrl)): the path
and method live in a g.Meta struct tag on a request type, and the controller
method that serves it is matched by that request type at runtime — so there was
no path string and no edge from a route to its handler, and "where is this route
handled / where are routes bound to controllers?" could only be answered
lexically (issue #720's report).
- frameworks/goframe.ts: detect gogf/gf in go.mod, extract each path-bearing
g.Meta into a route node (requires path:, so response mime:-only tags are
skipped), encoding the package-qualified request type for the join.
- goframe-synthesizer.ts: join each route -> the controller method whose
signature takes that request type — NOT by name (DeptSearchReq is served by
List) — keyed pkg.Type to disambiguate the many identical bare names a large
app defines one-per-module, with an addon-root tiebreak for cloned demo addons.
Edge kind calls, provenance heuristic, synthesizedBy goframe-route, surfaced as
a dynamic-dispatch hop in codegraph_explore.
Validated on real repos: gf-demo-user 7/7, gfast 65/68 (3 genuinely
handler-less), hotgo 242/247 (98%) — 100% precision (0 non-controller handlers,
0 core/addon cross-binding), node count stable. Agent A/B (gfast, sonnet/high,
2 runs/arm): with codegraph 1 explore call / 0 Read / ~20s vs without 7.5 Read
avg + grep-hunting for the non-existent literal route string / ~42s; same correct
answer.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The directory walk deliberately follows an in-root symlink whose target
lives outside the repo root (the standard Dota custom-game layout, where
`game/` and `content/` link into the SDK tree) and enumerates the files
under it. But the read path then rejected every one of them via the
strict symlink-escape guard, logging `Path traversal blocked in batch
reader` and indexing nothing — discovery and the reader disagreed.
Add an opt-in `allowSymlinkEscape` to validatePathWithinRoot that waives
only the realpath-escape rejection (the lexical `../` guard still
applies) and pass it at the three indexing read sites (batch reader,
indexFile, indexFileWithContent). The content-serving sinks
(ContextBuilder, MCP tools) keep the strict guard, so this stays inside
the #527 model: indexing now follows the symlink, getCode still refuses
to serve out-of-root contents.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The extension → language table was hardcoded, so a codebase using a
non-standard extension for a supported language (e.g. `.dota_lua` for Lua)
had those files silently skipped — no way to opt them in short of patching
the source.
Add an opt-in, project-scoped `codegraph.json` at the repo root:
{ "extensions": { ".dota_lua": "lua", ".tpl": "php" } }
Mappings merge on top of the built-in defaults and take precedence (so a
built-in can be re-pointed, e.g. `.h` → `cpp`). Absent or malformed config
is the zero-config default — byte-identical to prior behavior; an invalid
target language or unparseable file is warned-and-skipped, never fatal.
Implementation:
- New `src/project-config.ts` — `loadExtensionOverrides(rootDir)`, validated
against `isLanguageSupported`, mtime-cached per root.
- `detectLanguage` / `isSourceFile` gain an optional `overrides` arg
(omitting it is the existing behavior).
- Overrides threaded per-operation through every extraction call site
(scan/walk gates, git change-detection, grammar selection, extraction,
the file watcher), resolved from the project root — no process-global
state, so the multi-project daemon stays isolated. The parse worker
receives the resolved language in its message.
Tests: 13 new cases (unit, loader validation/normalization/caching, and a
full-index integration proving a custom-extension file is extracted while
the zero-config path indexes nothing). Worker path smoke-tested via the
built CLI.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
C/C++ polymorphism is the function pointer: a struct fn-pointer field, concrete
functions registered into it through a table (`{"add", cmd_add}`), a designated
initializer (`.handler = on_open`), or an assignment, then dispatched indirectly
(`p->fn(argv)`). Static extraction captures neither the registration→field
binding nor the indirect call, so the dispatcher→handler edge was missing — git's
run_builtin looked like it called nothing, a vtable's implementations had no
callers, and the hook_demo.c in the issue was unreachable.
Add a resolution-layer synthesizer keyed by (struct type, fn-pointer field). It
reads source (the established Celery/Sidekiq/Spring pattern — C extraction has no
struct fields or indirect-call edges to build on) in passes: collect fn-pointer
typedefs, parse struct field layouts, collect registrations (positional matched
by field index, designated, and assignment), propagate field←field assignments
(so a generic hook slot reassigned from a registry — the hook_demo.c
`h->func = found->fn` shape — inherits the registry field's handlers), then link
each indirect dispatch site to the registered handlers. Receiver type resolves
from the enclosing function's params/locals, falling back to a field name unique
to one struct. Covers both the command-table idiom (git, redis) and the
ops-struct/vtable idiom (curl content-encoders, protocol handlers).
Pure edge synthesis (no node growth); high precision via the (struct, field) key.
Validated: git 502 edges (run_builtin→cmd_* plus git_hash_algo/archiver/reftable
vtables), redis 357 (dictType.hashFunction, connection + reply-object vtables),
curl 478 (Curl_cwtype.do_init → deflate/gzip/brotli/zstd); 0 non-function targets
on all three; node-stable; 0 on the lua control (its {name,fn} tables register
into the Lua VM, with no C indirect call to bridge). Full suite 1665 pass.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Lombok generates getters/setters, builder(), equals/hashCode/toString, and
the @Slf4j log field at compile time, so they never appear in the source AST.
Static extraction missed them entirely, so a bean.getName() / User.builder() /
log.info() call resolved to nothing and call-chain analysis broke silently —
the agent would conclude the method didn't exist.
Add a synthesizeMembers hook on LanguageExtractor, called at the end of class
extraction (class still on the scope stack, real members already extracted), and
a Java implementation that synthesizes the mechanical members for @Getter,
@Setter, @Data, @Value, @Builder/@SuperBuilder, @ToString, @EqualsAndHashCode,
and the @Log* family. Each node is anchored on the field/class name-token leaf
(so it pulls in no spurious value-reference scope), marked with a `lombok`
decorator and a docstring naming the generating annotation, and never overrides
a member the source already declares. Methods and fields are deduped separately
since they're distinct namespaces in Java (a boolean field `isRunning` and its
generated getter `isRunning()` coexist).
Deliberately not synthesized: constructors (new X() already links via
instantiates, and overloaded @NoArgs/@AllArgs/@RequiredArgs ctors would collide
on a synthetic node id), fluent builder setters, and @Accessors(fluent=true).
Validated on eladmin (274 Java files, Lombok-heavy): 100% accessor precision
(878/878 map to a real field), 722 previously-broken calls now resolve;
spring-petclinic (no Lombok) control synthesizes nothing.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
MCP tool results used Markdown ATX headings (##/###/####) for section
headers — the status summary, each search hit, every file section in an
exploration — which Markdown-rendering clients (e.g. the Claude Code
VSCode extension) blow up to H1–H4 font size, filling the transcript with
oversized lines (worst on search/explore, where the noise scales with
result count). Swap them all for bold labels, which render at body size
while keeping the same structure. CLI/TTY output (ContextBuilder) is
unchanged — the issue notes it's fine.
The format is parse-coupled, so kept in sync:
- The explore truncation boundary and the offload chunker
(reasoning/reasoner.ts) both key off the per-file header, now a unique
`**`-prefixed marker emitted via a shared fileSectionHeader() helper.
- Updated the offload strip regexes and switched the opt-in report-style
prompt off ATX headings (same client, same rendering issue).
- Updated test helpers (sectionFor, sourcedFiles, the callers
section-boundary scan) that scanned the old markers.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
On a very large repo (the report is a ~93k-file / 5.7GB-DB Java monorepo) the
first MCP `tools/call` after a fresh `serve --mcp` could hang for 10+ minutes
with zero output, and with the liveness watchdog on, the daemon was SIGKILLed
mid-query instead. Root cause: the post-open catch-up reconcile that the first
tool call is gated on does ~2*N synchronous `fs.existsSync`/`fs.statSync` calls
plus a load-all-files query in two non-yielding loops. On a huge repo that wedges
the event loop for minutes, which (a) trips the 60s watchdog (it SIGKILLs a
process whose loop stops turning) and (b) blocks the first call the whole time.
Two complementary fixes:
- Make the reconcile yield. `ExtractionOrchestrator.sync()` now uses the
yielding `scanDirectoryAsync`, and both O(files) reconcile loops
`await setImmediate` every SYNC_RECONCILE_YIELD_INTERVAL (1000) files. The loop
can no longer wedge the main thread, so the watchdog stays fed and the socket /
any concurrent read stays responsive while a big reconcile runs. Results are
unchanged — only yield points are added.
- Time-box the catch-up gate. The first `tools/call` now waits on the reconcile
for at most CODEGRAPH_CATCHUP_GATE_TIMEOUT_MS (default 3000ms), then serves and
lets the reconcile finish in the background (which now yields, so the served
call runs concurrently). `=0` restores the old unbounded wait. On a normal repo
the reconcile finishes well under the budget, so behavior is unchanged.
Tests: adds two time-box cases to mcp-catchup-gate (serves promptly when the
reconcile runs long; `=0` restores the unbounded wait). Full suite green
(1655 passed). Validated end-to-end through the real daemon: first call returns
at the ~3s time-box instead of waiting an injected 8s reconcile; no-delay control
unchanged; `=0` opt-out waits the full reconcile.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Change detection's git fast path (collectGitStatus) consumed `git status`
output with only an isSourceFile filter, on the assumption that git already
omits ignored paths. It doesn't: gitignore is a no-op for *tracked* files, and
the built-in default excludes (vendor/, node_modules/) aren't gitignore at all.
So a tracked file inside a committed dependency dir, or under a .gitignored
dir, surfaced as a change the full index never tracks — `codegraph status`
reported phantom pending changes that `sync` (a filtered filesystem reconcile)
never cleared, and the public getChangedFiles() API returned the same wrong
list.
Apply buildDefaultIgnore(repoDir) per recursion level, matching repo-relative
paths — structurally equivalent to the full-index path's ScopeIgnore (each
embedded repo judged by its own rules) with no extra git subprocess calls.
Deletions stay unfiltered: getChangedFiles acts on one only when the path is
already tracked in the DB, where removal is always correct, and that lets a
newly-excluded dir's stale rows clean themselves up.
Unblocks #699 (an .ignore overlay inherits this leak unless change detection
consults the same matcher as enumeration).
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
github-actions[bot]