fix(extraction): capture & clean docstrings across all README languages (#780) (#806)

* fix(extraction): capture docstrings for export/const/decorator-wrapped symbols (#780)

getPrecedingDocstring walked previousNamedSibling from the EMITTED
declaration node, so it only found a leading comment when the comment was
a direct sibling of that node. For a declaration nested under a wrapper —
`export class X` / `export const f = () => {}` (export_statement /
lexical_declaration), a plain const arrow (variable_declarator), or a
decorated Python def/class (decorated_definition) — the comment is a
sibling of the WRAPPER, so the inner node had no preceding comment and
the docstring was stored as NULL.

Climb out through the wrapper node(s) before scanning for the comment.
Each wrapper holds exactly one declaration, so this can't mis-attribute a
comment to a sibling (verified: an uncommented method does NOT inherit its
class's comment). Also strip leading `#` from Python/Ruby/shell line
comments, which the cleanup chain missed (Python docstrings used to keep
their `#`).

Query/extraction-layer change to a parse helper; re-index to pick up
docstrings on already-indexed files. Verified on the reporter's JS/TS and
Python repros (8/8 now captured) plus over-walk controls; +3 tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(extraction): clean comment markers across all supported languages (#780)

Validating docstring capture across every README language surfaced that
the marker cleanup only knew C-style `//` and `/* */`, plus the `#` added
earlier this branch. Doc comments in other styles were captured but left
their markers in the stored text:

  - Rust/Swift/Kotlin doc lines `///` and `//!`  -> leading `/` / `!` leaked
  - Lua/Luau `--` and `--[[ ]]`                  -> not stripped
  - Pascal `{ }` and `(* *)`                     -> not stripped

Extract the cleanup into cleanCommentMarkers() and handle every style.
Paired block delimiters are stripped only when the comment OPENS with one,
so a line comment that happens to end with `}` / `*)` / `]]` is never
truncated; per-line markers stay anchored at line start.

Validated end-to-end (extract -> index -> codegraph_node output) across
all 19 tree-sitter code languages plus Svelte/Vue `<script>` blocks: every
one now stores and returns a clean docstring. +1 cross-language test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

src/extraction/tree-sitter-helpers.ts

@@ -43,11 +43,66 @@ export function getChildByField(node: SyntaxNode, fieldName: string): SyntaxNode
   return node.childForFieldName(fieldName);
 }
 
+/**
+ * Node types that *wrap* a declaration so a leading comment is a sibling of the
+ * wrapper, not of the emitted (inner) declaration node. CodeGraph emits the
+ * inner node, so before looking for its preceding comment we climb out through
+ * these. Examples: `export class X {}` (export_statement), `@dec\ndef f()`
+ * (decorated_definition), `const f = () => {}` (lexical_declaration →
+ * variable_declarator). Each wraps exactly one declaration, so climbing can't
+ * mis-attribute a comment to a sibling. (#780)
+ */
+const DOCSTRING_WRAPPER_TYPES = new Set([
+  'export_statement', // JS/TS: export class/function/const ...
+  'decorated_definition', // Python: @decorator over def/class
+  'lexical_declaration', // JS/TS: const/let x = () => {}
+  'variable_declaration', // JS/TS: var x = ...
+  'variable_declarator', // JS/TS: the `x = () => {}` inside the declaration
+  'ambient_declaration', // TS: declare ...
+]);
+
+/**
+ * Strip comment-syntax markers from a raw comment so the stored docstring is
+ * just the prose. Covers the marker styles across every supported language:
+ * C-family line and block comments and their doc variants, Rust/Swift/Kotlin
+ * triple-slash and bang doc lines, hash lines (Python/Ruby/shell), Lua/Luau
+ * line and long-bracket comments, and Pascal brace and paren-star comments.
+ * (#780)
+ *
+ * Paired block delimiters are stripped only when the comment OPENS with one,
+ * so a line comment that merely happens to END with a closing delimiter is
+ * never truncated. The per-line markers are anchored at line start, so
+ * they're safe to apply to any comment.
+ */
+function cleanCommentMarkers(comment: string): string {
+  let c = comment.trim();
+  if (c.startsWith('/*')) c = c.replace(/^\/\*+!?/, '').replace(/\*+\/$/, '');
+  else if (c.startsWith('--[')) c = c.replace(/^--\[=*\[/, '').replace(/\]=*\]$/, '');
+  else if (c.startsWith('(*')) c = c.replace(/^\(\*/, '').replace(/\*\)$/, '');
+  else if (c.startsWith('{')) c = c.replace(/^\{/, '').replace(/\}$/, '');
+  return c
+    .replace(/^\/\/[/!]?\s?/gm, '') // // , and Rust/Swift doc lines /// //!
+    .replace(/^--\s?/gm, '') //        Lua/Luau line comments
+    .replace(/^#\s?/gm, '') //         Python/Ruby/shell line comments
+    .replace(/^\s*\*\s?/gm, '') //     block-comment continuation (* foo)
+    .trim();
+}
+
 /**
  * Get the docstring/comment preceding a node
  */
 export function getPrecedingDocstring(node: SyntaxNode, source: string): string | undefined {
-  let sibling = node.previousNamedSibling;
+  // Climb out of any wrapper(s) so a comment preceding the WHOLE construct
+  // (export-, decorator-, or const-arrow-wrapped) is reachable as a sibling.
+  // The emitted node's own `previousNamedSibling` is empty (export/const) or a
+  // decorator (Python) in those cases, so without this the docstring was
+  // dropped. (#780)
+  let anchor = node;
+  while (anchor.parent && DOCSTRING_WRAPPER_TYPES.has(anchor.parent.type)) {
+    anchor = anchor.parent;
+  }
+
+  let sibling = anchor.previousNamedSibling;
   const comments: string[] = [];
 
   while (sibling) {
@@ -66,15 +121,6 @@ export function getPrecedingDocstring(node: SyntaxNode, source: string): string
 
   if (comments.length === 0) return undefined;
 
-  // Clean up comment markers
-  return comments
-    .map((c) =>
-      c
-        .replace(/^\/\*\*?|\*\/$/g, '')
-        .replace(/^\/\/\s?/gm, '')
-        .replace(/^\s*\*\s?/gm, '')
-        .trim()
-    )
-    .join('\n')
-    .trim();
+  // Strip each comment's syntax markers (language-aware), then join.
+  return comments.map(cleanCommentMarkers).join('\n').trim();
 }