feat(extraction): universal recovery of macro-mangled C/C++ function names (#1102)

* feat(extraction): universal recovery of macro-mangled C/C++ function names

The curated inline-macro blank list (#1100/#1101) can't enumerate every
library's macro. Add a universal post-parse net so a function is findable by
name regardless of which macro decorates it, plus a batch of common libraries
to the curated list for full name+return-type recovery.

- recoverMangledCppName: after extraction, recover the real identifier from a
  name still mangled by an un-blanked macro (`MACRO Ret name(…)` misparses to
  "Ret name"). It's a new `recoverMangledName` extractor hook wired only onto
  C/C++, applied to every name they produce. Safe by construction: it only
  touches an already-mangled name (an internal space that isn't a legit
  `operator …`/destructor), so a clean name is returned unchanged; guarded
  against the `Ret (name)` parenthesized-name idiom and bare primitives. Scoped
  to C/C++ so Kotlin/Scala backtick identifiers (which legitimately contain
  spaces) are never touched.
- Curated list extended past UE/pugixml/Godot/Boost to Qt (Q_INVOKABLE, …),
  Folly, Abseil, LLVM, V8, Eigen, and rapidjson.

Validated on CARLA (large UE project, 1131 C++/h files) vs the pre-fix baseline:
function-name mangles 440 -> 6, 431 fixed, and — critically — 0 regressions
(the salvage also recovers names that the pre-parse's own non-local error-recovery
shifts would otherwise re-mangle, erasing the 7 shifts seen in #1101). The 6
residual are all the moodycamel `Ret (name)` idiom, correctly left alone. On a
made-up macro with no list entry (`WEBKIT_EXPORT WTFString compute()`), the name
`compute` is still recovered. Full suite green; eleven regression/safety tests added.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(changelog): note universal C++ macro-mangled name recovery (#1102)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Colby Mchenry
2026-07-01 09:29:20 -05:00
committed by GitHub
co-authored by Claude Opus 4.8
parent a164ceae8b
commit cb20a3bf7f
5 changed files with 122 additions and 1 deletions
+47
View File
@@ -123,6 +123,8 @@ function extractCppReturnType(node: SyntaxNode, source: string): string | undefi
}
export const cExtractor: LanguageExtractor = {
// Universal net: recover a real name from any macro-mangled function name.
recoverMangledName: recoverMangledCppName,
functionTypes: ['function_definition'],
classTypes: [],
methodTypes: [],
@@ -275,6 +277,15 @@ const CPP_INLINE_MACROS = [
'_ALWAYS_INLINE_', '_FORCE_INLINE_',
// Boost
'BOOST_FORCEINLINE', 'BOOST_NOINLINE',
// Qt (per-method markers + inline)
'Q_INVOKABLE', 'Q_SCRIPTABLE', 'Q_ALWAYS_INLINE', 'Q_SLOT', 'Q_SIGNAL',
// Folly / Abseil / LLVM / V8 / Eigen / rapidjson
'FOLLY_ALWAYS_INLINE', 'FOLLY_NOINLINE',
'ABSL_ATTRIBUTE_ALWAYS_INLINE', 'ABSL_ATTRIBUTE_NOINLINE',
'LLVM_ATTRIBUTE_ALWAYS_INLINE', 'LLVM_ATTRIBUTE_NOINLINE',
'V8_INLINE', 'V8_NOINLINE',
'EIGEN_STRONG_INLINE', 'EIGEN_ALWAYS_INLINE', 'EIGEN_DEVICE_FUNC',
'RAPIDJSON_FORCEINLINE',
// Common cross-ecosystem inline/attribute hints
'ALWAYS_INLINE', 'FORCE_INLINE', 'NOINLINE',
] as const;
@@ -288,6 +299,40 @@ export function blankCppInlineMacros(source: string): string {
return source.replace(CPP_INLINE_MACRO_RE, (m) => ' '.repeat(m.length));
}
// Bare C/C++ type/qualifier tokens that must never be taken as a recovered
// function name (guards `recoverMangledCppName` against the `Ret (name)` idiom,
// where the token before the params is the return type, not the name).
const CPP_PRIMITIVE_NAMES = new Set([
'bool', 'void', 'int', 'char', 'short', 'long', 'float', 'double', 'unsigned',
'signed', 'wchar_t', 'char8_t', 'char16_t', 'char32_t', 'char_t', 'size_t',
'auto', 'const', 'struct', 'class', 'enum', 'union', 'typename',
]);
/**
* Universal fallback (any macro, no list) for a C/C++ function name still mangled
* because a macro we don't blank sat in front of the return type: `MACRO Ret
* name(…)` / `Ret MACRO name(…)` misparse so the return type is glued onto the
* name ("Ret name", "char_t* to_str(double v)"). Recover the real identifier —
* the token immediately before the parameter list (or the last token). This runs
* AFTER the curated pre-parse blank, so it only ever sees the residual tail that
* blanking didn't already fix cleanly (which also recovers the return type).
*
* Safe by construction: only touches an ALREADY-mangled name — one with an
* internal space that isn't a legit `operator …`/destructor — so a well-formed
* name is returned unchanged. Guarded against the two ways it could mis-pick:
* the `Ret (name)` parenthesized-name idiom (left as-is, ambiguous), and a token
* that is a bare primitive/keyword rather than a real identifier.
*/
export function recoverMangledCppName(name: string): string {
if (!/\s/.test(name) || name.startsWith('operator') || name.startsWith('~')) return name;
if (/^\S+\s+\([A-Za-z_]\w*\)/.test(name)) return name; // `Ret (name)` idiom — leave alone
const beforeParams = name.includes('(') ? name.slice(0, name.indexOf('(')) : name;
const tokens = beforeParams.trim().split(/\s+/);
const candidate = tokens[tokens.length - 1];
if (!candidate || !/^[A-Za-z_]\w*$/.test(candidate) || CPP_PRIMITIVE_NAMES.has(candidate)) return name;
return candidate;
}
/** C/C++ source pre-processing before tree-sitter: recover both macro-annotated
* class definitions and macro-prefixed function definitions. Offset-preserving. */
function preParseCppSource(source: string): string {
@@ -299,6 +344,8 @@ export const cppExtractor: LanguageExtractor = {
// #1061/#946) and macro-prefixed functions (`FORCEINLINE FString Foo()`, #1093
// follow-up) that tree-sitter otherwise misparses.
preParse: preParseCppSource,
// Universal net for any macro the curated blank list misses.
recoverMangledName: recoverMangledCppName,
functionTypes: ['function_definition'],
classTypes: ['class_specifier'],
// A bodiless `class_specifier` is a forward declaration (`class Foo;`) or an
+10
View File
@@ -133,6 +133,16 @@ export interface LanguageExtractor {
/** Override symbol name extraction (e.g. ObjC multi-part selectors). */
resolveName?: (node: SyntaxNode, source: string) => string | undefined;
/**
* Post-process an already-extracted name to recover a real identifier from a
* name still mangled by a macro the pre-parse didn't blank (C/C++:
* `MACRO Ret name(` misparses to the name "Ret name"). Applied to every name
* this extractor produces, so it MUST be a no-op on a well-formed name — only
* C/C++ set it, because a mangled name there is unambiguous (an internal space),
* whereas e.g. Kotlin/Scala backtick identifiers legitimately contain spaces.
*/
recoverMangledName?: (name: string) => string;
/** Extract property name when the generic name walk fails (e.g. ObjC @property). */
extractPropertyName?: (node: SyntaxNode, source: string) => string | null;
+8
View File
@@ -63,6 +63,14 @@ const VUE_STORE_FILE_SIGNAL = /\bdefineStore\b|\bcreateStore\b|\bVuex\b|\bmutati
* Extract the name from a node based on language
*/
function extractName(node: SyntaxNode, source: string, extractor: LanguageExtractor): string {
const name = extractNameRaw(node, source, extractor);
// Universal fallback: recover a real identifier from a name still mangled by a
// macro the pre-parse didn't blank (C/C++ only — see recoverMangledName). A
// no-op on well-formed names, so a clean name is never altered.
return extractor.recoverMangledName ? extractor.recoverMangledName(name) : name;
}
function extractNameRaw(node: SyntaxNode, source: string, extractor: LanguageExtractor): string {
const hookName = extractor.resolveName?.(node, source);
if (hookName) return hookName;