feat(ui): classify code from the engine's own tree-sitter parse, retiring Shiki (CG-57)
The viewer ran a second highlighter over source the engine had already parsed
with a real grammar: Shiki, plus 56 pruned TextMate grammars shipped in
dist/textmate/. The classification now comes off that tree instead, so a file is
read by exactly the grammar that decided what its symbols are.
The swap is complete rather than flagged: @shikijs/core, @shikijs/engine-javascript
and @shikijs/langs are off the dependency list, scripts/prune-grammars.mjs and
`npm run build:textmate` are deleted, and check-ui-build.mjs asserts the
tree-sitter grammars in dist/extraction/wasm instead of dist/textmate.
The wire contract is unchanged — `[classId, text]` pairs with the class names
alongside — so the viewer's decoder and code blocks did not have to be rewritten.
Two classes are added to the six: `type` (a named type reference, painted at
plain ink) and `def` (the name a definition declares, weight 600), the latter
taken from the extractors' own definition tables so it cannot drift from what
indexing calls a definition.
Three differences are not cosmetic:
* Interpolations (`${…}`, `#{…}`, `$"{…}"`, f-strings) are classified as code,
not as string. The call-site overlay refuses to claim a token classed string,
so calls written inside interpolated strings now link.
* Built-in type words are emitted whole and classed `type` in every language.
The grammars disagree about whether `string` is a type_identifier or an
anonymous token inside a predefined_type, and TextMate scoped them
inconsistently too.
* 3 000 lines of TypeScript cost 24-41 ms instead of ~700 ms.
Given up deliberately: Liquid, Razor, YAML, Twig, XML and .properties render
plain. .svelte/.vue/.astro are classified through their <script> blocks, the same
delegation the SFC extractors do. Pulling html/css/vue out of tree-sitter-wasms
would cover them, but those ABI-13 builds are the known cause of shared-WASM-heap
corruption for every other language in the same process.
Measured parity, per-language before/after screenshots and the reproduction
recipe: docs/design/cg57-highlighting-parity.md.
This commit is contained in:
@@ -56,6 +56,12 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||
|
||||
Any row that names a symbol can start a **flow**: press `Flow ›`, then type a second symbol or press `→ here` on another row, and you get the path between them — so "how does `POST /v1/payroll/cycles/{cycleID}/run` reach the database" is two clicks. Typing into the search box now finds entry points too, under their own heading below the symbol matches, so a URL comes back with its handler attached instead of on its own.
|
||||
|
||||
- **Syntax colouring in `codegraph ui` now comes from CodeGraph's own reading of your code.** The viewer used to run a second syntax highlighter over source CodeGraph had already parsed, with its own separate set of grammars. It doesn't any more: the colouring is taken straight from the parse that built your graph, so a file is coloured by exactly the grammar that decided what its symbols are. Three things you will notice — the name a definition declares now stands out on the line that declares it, wherever it appears; calls written inside a string (`${user.name()}`, `#{...}`, `$"{...}"`) are read as code and are now clickable links like every other call site; and built-in type words such as `string`, `int` and `void` look the same in every language instead of one way in Go and another in TypeScript. A big file paints far faster, most visibly in TypeScript, which was by a wide margin the slowest before.
|
||||
|
||||
Two formats change for the worse and it is worth saying so: Liquid, Razor, YAML, Twig, XML and `.properties` files are shown without colouring now, and in `.svelte`, `.vue` and `.astro` files the `<script>` block is coloured but the surrounding markup is not. Nothing about navigation changes there — call sites in those files still link, exactly as before.
|
||||
|
||||
This also takes about 3 MB of grammar files and two dependencies out of the install.
|
||||
|
||||
### Fixes
|
||||
|
||||
- Fixed a long-running `codegraph ui` session serving a symbol that a sync had already deleted. The viewer keeps one connection to your index open, and its in-memory lookup didn't notice when another process — your agent's sync, or `codegraph sync` — rewrote the file underneath it, so a symbol screen could keep showing a body with no callers while search correctly reported it had moved. Because a symbol's identity includes the line it starts on, this happened after almost any edit above it.
|
||||
|
||||
Reference in New Issue
Block a user