feat: support macros by gennaroprota · Pull Request #1192 · cppalliance/mrdocs

gennaroprota · 2026-04-30T13:44:40Z

This adds support for preprocessing macros: #define directives are captured via a clang::PPCallbacks subclass and exposed as MacroSymbol instances at the corpus root (since macros aren't in any C++ scope). Include guard macros are recognized and skipped.

Filters apply to macros, too:

extract-all: when off, undocumented macros are dropped.
exclude-symbols, include-symbols, implementation-defined, see-below: matched against the macro name.
File-pattern filters: matched against the directive's source location.

Multipage mode produces a macros.{ext} page at the output root listing every macro with a link to its per-symbol page, and the global-namespace page gains a "See also: Macros" navigation hint (multipage-only) so the new page is discoverable. Macro @param blocks are validated.

Changes

Source: Macro capture in a new clang::PPCallbacks subclass, exposed as a corpus-root MacroSymbol. The Handlebars generators render the macro synopsis, the single-page section heading, the multipage macros page, and the "See also: Macros" link from templates.
Golden tests: Macro fixtures covering extraction (happy path, edge cases, extraction modes, multi-file inputs, embedded macros, include guards), the filter paths, rendering, and multipage output.
Build: mrdocs.rnc gains a <macro> element, a sibling of <namespace>, with <source> carrying the verbatim definition.
Breaking changes: None for projects without macros. The output schema gains a new top-level <macro> element, so downstream tooling that enumerates top-level elements should add a clause for it.

Testing

Covered by the golden tests above. No CI workflow changes are needed.

Documentation

No user-facing documentation in this PR. A reference page describing what is captured, how the filters apply, and the new macros page is worth a follow-up so the feature is discoverable from the docs rather than the schema.

Closes #1127.

github-actions · 2026-04-30T13:45:09Z

✨ Highlights

🧪 New golden tests added

🧾 Changes by Scope

Scope	Lines Δ%	Lines Δ	Lines +	Lines -	Files Δ	Files +	Files ~	Files ↔	Files -
🥇 Golden Tests	75%	2778	2778	-	60	60	-	-	-
🛠️ Source	22%	812	734	78	32	4	28	-	-
📄 Docs	3%	116	116	-	3	-	3	-	-
🏗️ Build	<1%	14	12	2	1	-	1	-	-
Total	100%	3720	3640	80	96	64	32	-	-

Legend: Files + (added), Files ~ (modified), Files ↔ (renamed), Files - (removed)

🔝 Top Files

test-files/golden-tests/symbols/macro/macros.xml (Golden Tests): 743 lines Δ (+743 / -0)
test-files/golden-tests/generator/hbs/macros/macros.xml (Golden Tests): 243 lines Δ (+243 / -0)
src/lib/AST/ASTVisitor.cpp (Source): 188 lines Δ (+186 / -2)

Generated by 🚫 dangerJS against 3801993

codecov · 2026-04-30T13:45:35Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.16%. Comparing base (27f3488) to head (3801993).

Additional details and impacted files

@@           Coverage Diff            @@
##           develop    #1192   +/-   ##
========================================
  Coverage    83.16%   83.16%           
========================================
  Files           35       35           
  Lines         3660     3660           
  Branches       844      844           
========================================
  Hits          3044     3044           
  Misses         409      409           
  Partials       207      207

Flag	Coverage Δ
bootstrap	`83.16% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

cppalliance-bot · 2026-04-30T13:53:57Z

An automated preview of the documentation is available at https://1192.mrdocs.prtest2.cppalliance.org/index.html

If more commits are pushed to the pull request, the docs will rebuild at the same URL.

2026-07-29 03:09:00 UTC

alandefreitas · 2026-05-02T00:19:08Z

Thanks for exploring this. This is a complex issue.

This adds support for preprocessing macros: #define directives are captured via a clang::PPCallbacks

Interesting. I thought it would be easier to have a preprocessor step just for macros.

so we look up a preceding doc-comment by source location

This seems extremely problematic because the counterexamples where it doesn't work would be kind of obvious. Clang doesn't even attach comments to anything if there's not close to a symbol, which is probably the case when there's a macro.

My intuition is we can't really implement this feature unless clang wants to attach comments to macros or we implement our own parser (which we also intend to).

Filters apply to macros, too

How do these compare to doxygen? Shouldn't we have some special options for macros? I'm not sure. I assume macros are somewhat special in terms of what's typically hidden or shown, but I haven't thought too much about it.

I think @mizvekov would probably be more qualified to review this issue and PR.

gennaroprota · 2026-05-04T06:43:15Z

This adds support for preprocessing macros: #define directives are captured via a clang::PPCallbacks

Interesting. I thought it would be easier to have a preprocessor step just for macros.

Hmm... I'm listening for MacroDefined events on the preprocessor instance that Clang is already running for the AST pass, which seems natural.

so we look up a preceding doc-comment by source location

This seems extremely problematic because the counterexamples where it doesn't work would be kind of obvious. Clang doesn't even attach comments to anything if there's not close to a symbol, which is probably the case when there's a macro.

My intuition is we can't really implement this feature unless clang wants to attach comments to macros or we implement our own parser (which we also intend to).

Why? The current condition for accepting a preceding comment is that it is separated from the macro definition by blank lines only (and it is a doc-comment). We could tighten that to "no lines at all between the comment and the macro definition"; or maybe we can accept at most one line that is blank.

Filters apply to macros, too

How do these compare to doxygen? Shouldn't we have some special options for macros? I'm not sure. I assume macros are somewhat special in terms of what's typically hidden or shown, but I haven't thought too much about it.

I surveyed Doxygen's options. They are:

ENABLE_PREPROCESSING/MACRO_EXPANSION: master switches for Doxygen's preprocessor. MrDocs always runs a preprocessing pass.
PREDEFINED: pre-defines macros. MrDocs has the defines config option.
EXPAND_ONLY_PREDEF/EXPAND_AS_DEFINED: restrict macro expansion to predefined macros plus those of an explicit list. MrDocs has no equivalent.
SKIP_FUNCTION_MACROS: heuristic that drops some function-like macros; it exists to keep some macros from confusing the Doxygen parser.

Would you like to add any of these to MrDocs?

alandefreitas · 2026-05-12T03:29:29Z

Why? The current condition for accepting a preceding comment is that it is separated from the macro definition by blank lines only (and it is a doc-comment).

Oh... OK. That's good. I had only made the comment because your previous comment only said by "source location", which implied there could be content between the macro and the source location. But what you mentioned sounds great.

Would you like to add any of these to MrDocs?

No. Great survey. Thanks. I think my original comment was more specific about filtering macros. But that’s great. If there are no specific options to filter macros by name, let's just go with what we have. Sounds great.

One more question: if clang doesn't attach comments to macros like it does to the Decls, how do we find these comments? Do we look for comments attached to something else that should be attached to macros and then move them to the macros? I don't really understand it yet.

gennaroprota · 2026-05-12T07:54:11Z

Good question. Clang's ASTContext stores every comment it lexes in ASTContext::Comments (which, despite the type name RawCommentList, is really a per-file table). In the case of declarations, it also builds a second-layer mapping (Decl -> RawComment) by walking from each Decl's location backward with some proximity rules; in the case of macros it doesn't, but the underlying per-file map still has every comment in the TU, including the ones above #defines. We just do the location-based lookup ourselves. This logic is in findPrecedingMacroComment, in ASTVisitor.cpp.

mizvekov · 2026-05-19T20:22:49Z

FYI the need for this approach will soon be made obsolete by llvm/llvm-project#198452

gennaroprota · 2026-05-20T07:09:22Z

@mizvekov: Thanks for the heads-up! @alandefreitas: I think we can use the current approach for the moment being, and update it when the upstream patch lands and we switch to a new Clang version.

alandefreitas · 2026-05-21T05:02:32Z

I think we can use the current approach for the moment being, and update it when the upstream patch lands and we switch to a new Clang version.

I think the current approach could be reasonable. But I also think if llvm/llvm-project#198452 gets merged in two weeks or so, using the next version will be easier and clearer. We won’t notice the difference; we have about 10 PRs in line.

gennaroprota · 2026-05-21T07:53:14Z

I think the current approach could be reasonable. But I also think if llvm/llvm-project#198452 gets merged in two weeks or so, using the next version will be easier and clearer. We won’t notice the difference; we have about 10 PRs in line.

Yes, if we also plan to switch to a new Clang version as soon as it is released.

alandefreitas · 2026-05-22T23:02:13Z

new Clang version

It's more like a commit than a version, but yes. It'd be the same major. It's about time to upgrade anyway.

gennaroprota · 2026-05-28T08:21:27Z

@alandefreitas: The patch has been merged upstream.

alandefreitas · 2026-05-28T20:15:58Z

@alandefreitas: The patch has been merged upstream.

🚀🔥

alandefreitas · 2026-06-08T19:45:20Z

How's the situation with LLVM going so far?

The new pin includes llvm/llvm-project#198452, which attaches documentation comments to macro definitions. That is the prerequisite for reading them from Clang directly instead of scanning the source, as we were initially doing in PR cppalliance#1192. The pin was about six months behind, so this also adapts MrDocs to the Clang API changes across that range: - USRGeneration.h moved from clang/Index to clang/UnifiedSymbolResolution; the `clang::index` namespace and signatures are unchanged. - `clang::DiagnosticConsumer::finish()` was removed, so the collecting diagnostic consumer relies on the forwarded `EndSourceFile()` instead. - The driver option table moved to clang/Options/Options.h, with its enumerators now in `clang::options` and `getDriverOptTable()` in `clang`. - `cl::getRegisteredOptions()` now returns a `DenseMap`, so its entries expose `.second` instead of `getValue()`. - `SubstituteConstraintExpressionWithoutSatisfaction` was re-synced with Clang's current implementation. `getRawCommentForDeclNoCache`, the old per-declaration comment lookup, is no longer public. `getDocumentation` now uses the public `getRawCommentForAnyRedecl`, and a new `ASTVisitor` pass (`attachComments`) attaches a comment to every declaration up front via `ASTContext::attachCommentsToJustParsedDecls`, so that lookup returns a declaration's own comment instead of collapsing its redeclaration chain to a single comment. Parsing the newer libc++ also needs a new vcruntime_new.h stub and C++ guards so stdbool.h, stdalign.h, and threads.h stop redefining `bool`, `alignas`, and `thread_local`, which are keywords, not macros, in C++. The regenerated goldens reflect two effects and are otherwise unchanged: - `getRawCommentForAnyRedecl` follows the redeclaration chain, which the old lookup did not, so a declaration with no comment of its own now resolves to a documented redeclaration's comment. This adds `<documented>` markers on redeclaration locations (merge-params, merge-tparams, friend-fn-*, no-extract-all, class-template-array-spec), gives inherited members their base's doc comment (template-base), and shows an operator's written brief in place of a synthesized one (inherited-members-specialization). Declarations that each carry their own distinct comment keep it, thanks to the up-front attach pass. - Clang's output drifted over the six months: constrained templates and destructors get different symbol IDs because USR generation changed, though their structure is identical (requires-clause, dtor-overloads), and a few declaration locations shifted by a line or column (auto, param-types, function-template).

The new pin includes llvm/llvm-project#198452, which attaches documentation comments to macro definitions. That is the prerequisite for reading them from Clang directly instead of scanning the source, as we were initially doing in PR cppalliance#1192. The pin was about six months behind, so this also adapts MrDocs to the Clang API changes across that range: - USRGeneration.h moved from clang/Index to clang/UnifiedSymbolResolution; the `clang::index` namespace and signatures are unchanged. - `clang::DiagnosticConsumer::finish()` was removed, so the collecting diagnostic consumer relies on the forwarded `EndSourceFile()` instead. - The driver option table moved to clang/Options/Options.h, with its enumerators now in `clang::options` and `getDriverOptTable()` in `clang`. - `cl::getRegisteredOptions()` now returns a `DenseMap`, so its entries expose `.second` instead of `getValue()`. - `SubstituteConstraintExpressionWithoutSatisfaction` was re-synced with Clang's current implementation. `getRawCommentForDeclNoCache`, the per-declaration comment lookup `getDocumentation` relied on, is no longer public. Its only public replacement, `getRawCommentForAnyRedecl`, follows the redeclaration chain and so cannot return a single declaration's own comment. `getDocumentation` keeps the per-declaration behavior by porting Clang's declaration-path comment search (`getLocsForCommentSearch` and `getRawCommentNoCacheImpl`) as file-local helpers. Parsing the newer libc++ also needs a new vcruntime_new.h stub and C++ guards so stdbool.h, stdalign.h, and threads.h stop redefining `bool`, `alignas`, and `thread_local`, which are keywords, not macros, in C++. The LLVM commit hash also feeds the CI cache key, so ci-matrix.yml and the test_cache_keys.py unit test that mirrors it are bumped to match; otherwise CI restores the previous libc++ install and builds the new one over it. The regenerated goldens change only because Clang's output drifted over the six months: constrained templates and destructors get different symbol IDs from USR generation, though their structure is identical (requires-clause, dtor-overloads), and a few declaration locations shifted by a line or column (auto, param-types, function-template).

The new pin includes llvm/llvm-project#198452, which attaches documentation comments to macro definitions. That is the prerequisite for reading them from Clang directly instead of scanning the source, as we were initially doing in PR cppalliance#1192. The pin was about six months behind, so this also adapts MrDocs to the Clang API changes across that range: - USRGeneration.h moved from clang/Index to clang/UnifiedSymbolResolution; the `clang::index` namespace and signatures are unchanged. - `clang::DiagnosticConsumer::finish()` was removed, so the collecting diagnostic consumer relies on the forwarded `EndSourceFile()` instead. - The driver option table moved to clang/Options/Options.h, with its enumerators now in `clang::options` and `getDriverOptTable()` in `clang`. - `cl::getRegisteredOptions()` now returns a `DenseMap`, so its entries expose `.second` instead of `getValue()`. - `SubstituteConstraintExpressionWithoutSatisfaction` was re-synced with Clang's current implementation. `getRawCommentForDeclNoCache`, the per-declaration comment lookup `getDocumentation` relied on, is no longer public. Its only public replacement, `getRawCommentForAnyRedecl`, follows the redeclaration chain and so cannot return a single declaration's own comment. `getDocumentation` keeps the per-declaration behavior by porting Clang's declaration-path comment search (`getLocsForCommentSearch` and `getRawCommentNoCacheImpl`) as file-local helpers. Parsing the newer libc++ also needs a new vcruntime_new.h stub and C++ guards so stdbool.h, stdalign.h, and threads.h stop redefining `bool`, `alignas`, and `thread_local`, which are keywords, not macros, in C++. The regenerated goldens change only because Clang's output drifted over the six months: constrained templates and destructors get different symbol IDs from USR generation, though their structure is identical (requires-clause, dtor-overloads), and a few declaration locations changed.

The new pin includes llvm/llvm-project#198452, which attaches documentation comments to macro definitions. That is the prerequisite for reading them from Clang directly instead of scanning the source, as we were initially doing in PR cppalliance#1192. The pin was about six months behind, so this also adapts MrDocs to the Clang API changes across that range: - USRGeneration.h moved from clang/Index to clang/UnifiedSymbolResolution; the `clang::index` namespace and signatures are unchanged. - `clang::DiagnosticConsumer::finish()` was removed, so the collecting diagnostic consumer relies on the forwarded `EndSourceFile()` instead. - The driver option table moved to clang/Options/Options.h, with its enumerators now in `clang::options` and `getDriverOptTable()` in `clang`. - `cl::getRegisteredOptions()` now returns a `DenseMap`, so its entries expose `.second` instead of `getValue()`. - `SubstituteConstraintExpressionWithoutSatisfaction` was re-synced with Clang's current implementation. Parsing the newer libc++ also needs a new vcruntime_new.h stub and C++ guards so stdbool.h, stdalign.h, and threads.h stop redefining `bool`, `alignas`, and `thread_local`, which are keywords, not macros, in C++. The regenerated goldens change only because Clang's output drifted over the six months: constrained templates and destructors get different symbol IDs from USR generation, though their structure is identical (requires-clause, dtor-overloads), and a few declaration locations changed.

The new pin includes llvm/llvm-project#198452, which attaches documentation comments to macro definitions. That is the prerequisite for reading them from Clang directly instead of scanning the source, as we were initially doing in PR cppalliance#1192. The pin was about six months behind, so this also adapts MrDocs to the Clang API changes across that range: - USRGeneration.h moved from clang/Index to clang/UnifiedSymbolResolution; the `clang::index` namespace and signatures are unchanged. - `clang::DiagnosticConsumer::finish()` was removed, so the collecting diagnostic consumer relies on the forwarded `EndSourceFile()` instead. - The driver option table moved to clang/Options/Options.h, with its enumerators now in `clang::options` and `getDriverOptTable()` in `clang`. - `cl::getRegisteredOptions()` now returns a `DenseMap`, so its entries expose `.second` instead of `getValue()`. - `SubstituteConstraintExpressionWithoutSatisfaction` was re-synced with Clang's current implementation. Parsing the newer libc++ also needs a new vcruntime_new.h stub, since its align_val_t.h now includes vcruntime_new.h under the vcruntime ABI. The regenerated goldens change only because Clang's output drifted over the six months: constrained templates and destructors get different symbol IDs from USR generation, though their structure is identical (requires-clause, dtor-overloads), and a few declaration locations changed.

The new pin includes llvm/llvm-project#198452, which attaches documentation comments to macro definitions. That is the prerequisite for reading them from Clang directly instead of scanning the source, as we were initially doing in PR cppalliance#1192. The pin was about six months behind, so this also adapts MrDocs to the Clang API changes across that range: - USRGeneration.h moved from clang/Index to clang/UnifiedSymbolResolution; the `clang::index` namespace and signatures are unchanged. - `clang::DiagnosticConsumer::finish()` was removed, so the collecting diagnostic consumer relies on the forwarded `EndSourceFile()` instead. - The driver option table moved to clang/Options/Options.h, with its enumerators now in `clang::options` and `getDriverOptTable()` in `clang`. - `cl::getRegisteredOptions()` now returns a `DenseMap`, so its entries expose `.second` instead of `getValue()`. - `SubstituteConstraintExpressionWithoutSatisfaction` was re-synced with Clang's current implementation. Parsing the newer libc++ also needs a new vcruntime_new.h stub (its align_val_t.h now includes it under the vcruntime ABI) and C++ guards so stdbool.h, stdalign.h, and threads.h stop redefining some C++ keywords as macros. The regenerated goldens change only because Clang's output drifted over the six months: constrained templates and destructors get different symbol IDs from USR generation, though their structure is identical (requires-clause, dtor-overloads), and a few declaration locations changed.

The new pin includes llvm/llvm-project#198452, which attaches documentation comments to macro definitions. That is the prerequisite for reading them from Clang directly instead of scanning the source, as we were initially doing in PR cppalliance#1192. The pin was about six months behind, so this also adapts MrDocs to the Clang API changes across that range: - USRGeneration.h moved from clang/Index/ to clang/UnifiedSymbolResolution/; the `clang::index` namespace and signatures are unchanged. - `clang::DiagnosticConsumer::finish()` was removed, so the collecting diagnostic consumer relies on the forwarded `EndSourceFile()` instead. - The driver option table moved to clang/Options/Options.h, with its enumerators now in `clang::options` and `getDriverOptTable()` in `clang`. - `cl::getRegisteredOptions()` now returns a `DenseMap`, so its entries expose `.second` instead of `getValue()`. - `SubstituteConstraintExpressionWithoutSatisfaction` was re-synced with Clang's current implementation. Parsing the newer libc++ also needs a new vcruntime_new.h stub (its align_val_t.h now includes it under the vcruntime ABI) and C++ guards so stdbool.h, stdalign.h, and threads.h stop redefining some C++ keywords as macros. The regenerated goldens change only because Clang's output drifted over the six months: constrained templates and destructors get different symbol IDs from USR generation, though their structure is identical (requires-clause, dtor-overloads), and a few declaration locations changed.

This adds support for preprocessing macros: `#define` directives are captured via a `clang::PPCallbacks` subclass and exposed as `MacroSymbol` instances at the corpus root (macros aren't in any C++ scope, so they sit alongside the global namespace rather than under it). Include guard macros are recognized and skipped. Filters apply to macros, too: - `extract-all`: when off, undocumented macros are dropped. - `exclude-symbols`, `include-symbols`, `implementation-defined`, `see-below`: matched against the macro name. - File-pattern filters: matched against the directive's source location. New output pages: multipage mode now produces `macros.{ext}` at the output root, listing every macro with a link to its per-symbol page. The global-namespace page gains a "See also: Macros" navigation hint (multipage-only) so the new page is discoverable. Macro `@param` blocks are validated. Closes cppalliance#1127.

gennaroprota force-pushed the feat/support_macros branch 5 times, most recently from c1ce891 to a42ad72 Compare May 1, 2026 16:00

gennaroprota force-pushed the feat/support_macros branch from a42ad72 to 32e9ccf Compare May 4, 2026 05:58

gennaroprota force-pushed the feat/support_macros branch from 32e9ccf to 81953ee Compare May 20, 2026 07:11

gennaroprota mentioned this pull request Jul 2, 2026

chore: bump LLVM to 77e43ec1 (May 21, 2026) #1241

Merged

gennaroprota force-pushed the feat/support_macros branch 2 times, most recently from 95096eb to 3970523 Compare July 28, 2026 15:17

gennaroprota and others added 3 commits July 28, 2026 17:40

refactor: macros use regular symbol pipeline

94abbc5

feat: macro extraction options

3801993

alandefreitas force-pushed the feat/support_macros branch from 3970523 to 3801993 Compare July 29, 2026 03:00

alandefreitas added this pull request to the merge queue Jul 29, 2026

Merged via the queue into cppalliance:develop with commit 3fd5e35 Jul 29, 2026
32 checks passed

Uh oh!

Conversation

gennaroprota commented Apr 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Testing

Documentation

Uh oh!

github-actions Bot commented Apr 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✨ Highlights

🧾 Changes by Scope

🔝 Top Files

Uh oh!

codecov Bot commented Apr 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

cppalliance-bot commented Apr 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alandefreitas commented May 2, 2026

Uh oh!

gennaroprota commented May 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alandefreitas commented May 12, 2026

Uh oh!

gennaroprota commented May 12, 2026

Uh oh!

mizvekov commented May 19, 2026

Uh oh!

gennaroprota commented May 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alandefreitas commented May 21, 2026

Uh oh!

gennaroprota commented May 21, 2026

Uh oh!

alandefreitas commented May 22, 2026

Uh oh!

gennaroprota commented May 28, 2026

Uh oh!

alandefreitas commented May 28, 2026

Uh oh!

alandefreitas commented Jun 8, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

gennaroprota commented Apr 30, 2026 •

edited

Loading

github-actions Bot commented Apr 30, 2026 •

edited

Loading

codecov Bot commented Apr 30, 2026 •

edited

Loading

cppalliance-bot commented Apr 30, 2026 •

edited

Loading

gennaroprota commented May 4, 2026 •

edited

Loading

gennaroprota commented May 20, 2026 •

edited

Loading