codegraph

978 Commits 1 Branch 0 Tags

3 Commits

Include renames

Author	SHA1	Message	Date
Colby McHenry	ad91c8fdd8	feat(ui): classify code from the engine's own tree-sitter parse, retiring Shiki (CG-57) The viewer ran a second highlighter over source the engine had already parsed with a real grammar: Shiki, plus 56 pruned TextMate grammars shipped in dist/textmate/. The classification now comes off that tree instead, so a file is read by exactly the grammar that decided what its symbols are. The swap is complete rather than flagged: @shikijs/core, @shikijs/engine-javascript and @shikijs/langs are off the dependency list, scripts/prune-grammars.mjs and `npm run build:textmate` are deleted, and check-ui-build.mjs asserts the tree-sitter grammars in dist/extraction/wasm instead of dist/textmate. The wire contract is unchanged — `[classId, text]` pairs with the class names alongside — so the viewer's decoder and code blocks did not have to be rewritten. Two classes are added to the six: `type` (a named type reference, painted at plain ink) and `def` (the name a definition declares, weight 600), the latter taken from the extractors' own definition tables so it cannot drift from what indexing calls a definition. Three differences are not cosmetic: * Interpolations (`${…}`, `#{…}`, `$"{…}"`, f-strings) are classified as code, not as string. The call-site overlay refuses to claim a token classed string, so calls written inside interpolated strings now link. * Built-in type words are emitted whole and classed `type` in every language. The grammars disagree about whether `string` is a type_identifier or an anonymous token inside a predefined_type, and TextMate scoped them inconsistently too. * 3 000 lines of TypeScript cost 24-41 ms instead of ~700 ms. Given up deliberately: Liquid, Razor, YAML, Twig, XML and .properties render plain. .svelte/.vue/.astro are classified through their <script> blocks, the same delegation the SFC extractors do. Pulling html/css/vue out of tree-sitter-wasms would cover them, but those ABI-13 builds are the known cause of shared-WASM-heap corruption for every other language in the same process. Measured parity, per-language before/after screenshots and the reproduction recipe: docs/design/cg57-highlighting-parity.md.	2026-08-27 06:22:11 -05:00
Colby McHenry	2ad836d935	feat(ui): highlight source server-side with a near-monochrome Shiki theme (CG-43) The viewer's code block stops lexing with a hand-rolled dialect table and reads real TextMate grammars instead, run once in `/api/source`. Three things make that safe to depend on: * Highlighting never fails a request. A missing grammar, an oversized slice, an ESM import that did not resolve — every one of them answers `engine: 'plain'` with a reason and the source still goes out. * Identifiers survive whatever token boundaries a grammar chose. Every code token is split into identifier runs before it goes on the wire, so the graph's call-site overlay claims a token the highlighter produced rather than re-cutting the line. `assignRefs` now matches on a token's text rather than on the class a grammar gave it, so a language that scopes type names as `storage.type` still links. * The theme classifies rather than colours: its foregrounds are sentinels the server maps back to class names, and the viewer paints them from CSS custom properties — one token stream serves light and dark with no refetch, and the ramp lives only in app.css. Comments move from --ink-3 to a new --code-comment. --ink-3 measures 3.46:1 on paper and 3.00:1 on the hot-line tint, both under AA for 12.5px text; --code-comment is the smallest step along the same ramp that clears 4.5:1 on every background a code line can have, and stays quieter than the strings and numbers above it. Shipping: @shikijs/core and @shikijs/engine-javascript are runtime dependencies (no wasm, no native module); @shikijs/langs stays a devDependency and `npm run build:textmate` writes only the closure the engine's 40-odd languages reach — 56 grammars, 2.6 MB, against 11 MB for all 722. check-ui-build.mjs asserts the tree after every build and inside every release archive.	2026-08-27 01:23:10 -05:00
Colby McHenry	a72f22a6d3	feat(ui): scaffold the codegraph ui viewer as a Svelte 5 + Vite workspace (CG-40) Adds `ui/` as an npm workspace (Svelte 5.56 + Vite 7, devDependencies only — the engine's runtime dependencies are untouched) and chains its build into `npm run build`, so the browser viewer ships inside `dist/` with everything else: `build-bundle.sh` already copies `dist` wholesale and `pack-npm.sh` packs that bundle. Output is `dist/viewer/`, NOT `dist/ui/`: `src/ui/` is the engine's terminal ui (shimmer progress + its worker) and tsc compiles it to `dist/ui/`, so emitting there both deletes those modules — the CLI then dies at startup with `Cannot find module '../ui/shimmer-progress'` — and would leave the static server handing out compiled engine internals. The design spec is corrected to match. `scripts/check-ui-build.mjs` is the release guard: index.html must exist, be non-trivial, and every local asset it references must be on disk, and the compiled engine next door must still be intact. It runs after every UI build, again in `build-bundle.sh` once the bundle stage has copied `dist`, and again in `pack-npm.sh` once each archive is unpacked — so a broken viewer fails the release instead of shipping a CLI that serves a 404. `vite build` does not override an ambient NODE_ENV, so a shell or runner with NODE_ENV=development silently shipped dev-mode Svelte (~13 kB of dev-only runtime checks, warning in the user's console). The config now pins production for `command === 'build'`; macOS and Windows ARM64 then emit byte-identical bundle hashes. The shell itself follows docs/design/codegraph-ui-design-spec.md §2–§3.1: design tokens as CSS custom properties (light on bare `:root`, dark under both `prefers-color-scheme` and `[data-theme="dark"]`), square corners, hairline rules, one oxblood accent; top bar 48px / trail bar 34px / main; a hash router over `#/s/<id>`, `#/file/<path>`, with `#/map` and `#/flow` reserved for phase 2. Fonts are vendored through @fontsource rather than fetched, so a local reader works offline and never announces the project to a CDN. Verified: clean `npm run build` from an empty dist on macOS and on the Windows ARM64 VM (forward-slash asset URLs, CLI still starts, both assertion failure modes exit 1); `dist/viewer` present in a real darwin-arm64 bundle and in the packed npm platform package; shell geometry, tokens, all seven routes, both themes and font loading checked in headless Chromium with no console errors; `npm test` unaffected.	2026-08-26 15:55:10 -05:00

Author

SHA1

Message

Date

Colby McHenry

ad91c8fdd8

feat(ui): classify code from the engine's own tree-sitter parse, retiring Shiki (CG-57)

The viewer ran a second highlighter over source the engine had already parsed
with a real grammar: Shiki, plus 56 pruned TextMate grammars shipped in
dist/textmate/. The classification now comes off that tree instead, so a file is
read by exactly the grammar that decided what its symbols are.

The swap is complete rather than flagged: @shikijs/core, @shikijs/engine-javascript
and @shikijs/langs are off the dependency list, scripts/prune-grammars.mjs and
`npm run build:textmate` are deleted, and check-ui-build.mjs asserts the
tree-sitter grammars in dist/extraction/wasm instead of dist/textmate.

The wire contract is unchanged — `[classId, text]` pairs with the class names
alongside — so the viewer's decoder and code blocks did not have to be rewritten.
Two classes are added to the six: `type` (a named type reference, painted at
plain ink) and `def` (the name a definition declares, weight 600), the latter
taken from the extractors' own definition tables so it cannot drift from what
indexing calls a definition.

Three differences are not cosmetic:

* Interpolations (`${…}`, `#{…}`, `$"{…}"`, f-strings) are classified as code,
  not as string. The call-site overlay refuses to claim a token classed string,
  so calls written inside interpolated strings now link.
* Built-in type words are emitted whole and classed `type` in every language.
  The grammars disagree about whether `string` is a type_identifier or an
  anonymous token inside a predefined_type, and TextMate scoped them
  inconsistently too.
* 3 000 lines of TypeScript cost 24-41 ms instead of ~700 ms.

Given up deliberately: Liquid, Razor, YAML, Twig, XML and .properties render
plain. .svelte/.vue/.astro are classified through their <script> blocks, the same
delegation the SFC extractors do. Pulling html/css/vue out of tree-sitter-wasms
would cover them, but those ABI-13 builds are the known cause of shared-WASM-heap
corruption for every other language in the same process.

Measured parity, per-language before/after screenshots and the reproduction
recipe: docs/design/cg57-highlighting-parity.md.

2026-08-27 06:22:11 -05:00

Colby McHenry

2ad836d935

feat(ui): highlight source server-side with a near-monochrome Shiki theme (CG-43)

The viewer's code block stops lexing with a hand-rolled dialect table and
reads real TextMate grammars instead, run once in `/api/source`.

Three things make that safe to depend on:

* Highlighting never fails a request. A missing grammar, an oversized
  slice, an ESM import that did not resolve — every one of them answers
  `engine: 'plain'` with a reason and the source still goes out.
* Identifiers survive whatever token boundaries a grammar chose. Every
  code token is split into identifier runs before it goes on the wire, so
  the graph's call-site overlay claims a token the highlighter produced
  rather than re-cutting the line. `assignRefs` now matches on a token's
  text rather than on the class a grammar gave it, so a language that
  scopes type names as `storage.type` still links.
* The theme classifies rather than colours: its foregrounds are sentinels
  the server maps back to class names, and the viewer paints them from
  CSS custom properties — one token stream serves light and dark with no
  refetch, and the ramp lives only in app.css.

Comments move from --ink-3 to a new --code-comment. --ink-3 measures
3.46:1 on paper and 3.00:1 on the hot-line tint, both under AA for 12.5px
text; --code-comment is the smallest step along the same ramp that clears
4.5:1 on every background a code line can have, and stays quieter than
the strings and numbers above it.

Shipping: @shikijs/core and @shikijs/engine-javascript are runtime
dependencies (no wasm, no native module); @shikijs/langs stays a
devDependency and `npm run build:textmate` writes only the closure the
engine's 40-odd languages reach — 56 grammars, 2.6 MB, against 11 MB for
all 722. check-ui-build.mjs asserts the tree after every build and inside
every release archive.

2026-08-27 01:23:10 -05:00

Colby McHenry

a72f22a6d3

feat(ui): scaffold the codegraph ui viewer as a Svelte 5 + Vite workspace (CG-40)

Adds `ui/` as an npm workspace (Svelte 5.56 + Vite 7, devDependencies only —
the engine's runtime dependencies are untouched) and chains its build into
`npm run build`, so the browser viewer ships inside `dist/` with everything
else: `build-bundle.sh` already copies `dist` wholesale and `pack-npm.sh`
packs that bundle.

Output is `dist/viewer/`, NOT `dist/ui/`: `src/ui/` is the engine's terminal
ui (shimmer progress + its worker) and tsc compiles it to `dist/ui/`, so
emitting there both deletes those modules — the CLI then dies at startup with
`Cannot find module '../ui/shimmer-progress'` — and would leave the static
server handing out compiled engine internals. The design spec is corrected to
match.

`scripts/check-ui-build.mjs` is the release guard: index.html must exist, be
non-trivial, and every local asset it references must be on disk, and the
compiled engine next door must still be intact. It runs after every UI build,
again in `build-bundle.sh` once the bundle stage has copied `dist`, and again
in `pack-npm.sh` once each archive is unpacked — so a broken viewer fails the
release instead of shipping a CLI that serves a 404.

`vite build` does not override an ambient NODE_ENV, so a shell or runner with
NODE_ENV=development silently shipped dev-mode Svelte (~13 kB of dev-only
runtime checks, warning in the user's console). The config now pins production
for `command === 'build'`; macOS and Windows ARM64 then emit byte-identical
bundle hashes.

The shell itself follows docs/design/codegraph-ui-design-spec.md §2–§3.1:
design tokens as CSS custom properties (light on bare `:root`, dark under both
`prefers-color-scheme` and `[data-theme="dark"]`), square corners, hairline
rules, one oxblood accent; top bar 48px / trail bar 34px / main; a hash router
over `#/s/<id>`, `#/file/<path>`, with `#/map` and `#/flow` reserved for phase
2. Fonts are vendored through @fontsource rather than fetched, so a local
reader works offline and never announces the project to a CDN.

Verified: clean `npm run build` from an empty dist on macOS and on the Windows
ARM64 VM (forward-slash asset URLs, CLI still starts, both assertion failure
modes exit 1); `dist/viewer` present in a real darwin-arm64 bundle and in the
packed npm platform package; shell geometry, tokens, all seven routes, both
themes and font loading checked in headless Chromium with no console errors;
`npm test` unaffected.

2026-08-26 15:55:10 -05:00