Skip to content

Use _quarto.modules registry instead of redundant module requires in filters#14702

Merged
cderv merged 6 commits into
mainfrom
feature/sweep-redundant-module-requires
Jul 17, 2026
Merged

Use _quarto.modules registry instead of redundant module requires in filters#14702
cderv merged 6 commits into
mainfrom
feature/sweep-redundant-module-requires

Conversation

@cderv

@cderv cderv commented Jul 16, 2026

Copy link
Copy Markdown
Member

When packaging the Lua filter bundle, Pandoc's Lua interpreter enforces a 200 top-level-local-variable limit (MAXVARS) across all inlined filter files. main is already at that ceiling with zero headroom, which surfaced while evaluating room to bump the bundled Pandoc to 3.10.

modules/import_all.lua already centralizes most modules/* requires into a single _quarto.modules registry table, at no additional local-slot cost no matter how many modules get registered. But ~24 inlined filter files still redundantly re-required the same modules (constants, patterns, lightbox, scope, typst) as fresh top-level locals instead of referencing the existing registry - each costing one slot in the shared budget.

This replaces those redundant local X = require("modules/Y") aliases with direct _quarto.modules.Y access at every call site. Pure access-path change to an already-loaded singleton table - no logic or API change.

26 alias declarations removed across 24 files. Packaged bundle main.lua top-level local count drops from ~200 to 173.

Verified manually: packaged quarto check passes with no "too many local variables" error, packaged renders of lightbox (HTML) and typst (PDF) fixtures succeed, and smoke-all chunks covering the edited filters (2022, 2023/01) pass locally.

Follow up on #14664 where we hit the limit

@posit-snyk-bot

posit-snyk-bot commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator

Snyk checks have passed. No issues have been found so far.

Status Scan Engine Critical High Medium Low Total (0)
Open Source Security 0 0 0 0 0 issues
Licenses 0 0 0 0 0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

editor-support crossref runs quarto-init.lua then crossref.lua as two
separate pandoc --lua-filter args. quarto-init.lua's Meta handler
(read_includes) runs before crossref.lua ever imports import_all.lua,
so _quarto.modules was nil at that point. Revert to a local require.
@cscheid

cscheid commented Jul 16, 2026

Copy link
Copy Markdown
Member

This is a really great LLM task! Maybe it's worth taking a longer look at the Lua files and trying to move more of those files out.

Although, make sure to remember that these Lua changes are not truly tested in the test suite, and that we should probably do a bit of manual testing with the actual software bundle.

@cderv

cderv commented Jul 17, 2026

Copy link
Copy Markdown
Member Author

Although, make sure to remember that these Lua changes are not truly tested in the test suite, and that we should probably do a bit of manual testing with the actual software bundle.

You mean testing the built quarto.exe right ? Not just the dev version ?

This is a really great LLM task! Maybe it's worth taking a longer look at the Lua files and trying to move more of those files out.

Yes it was definitely a good task, and I plan to follow up. We have a little room, but it seems LLM had ideas to improve it with more refactor. Though I wanted to split tasks to be more manageable in testing. It seems this specific PR refactor is safe.

@cderv

cderv commented Jul 17, 2026

Copy link
Copy Markdown
Member Author

I triggered the build action to see if at least all bundles can be built: https://github.com/quarto-dev/quarto-cli/actions/runs/29572067222

And we can download artifacts to test

@cderv

cderv commented Jul 17, 2026

Copy link
Copy Markdown
Member Author

All green - artifacts are building - I'll manually test.

@cderv

cderv commented Jul 17, 2026

Copy link
Copy Markdown
Member Author

With help of 🤖 , I tested with the packaged Windows build from that run. All good! SO I am merging.

details

The registry swap is a straight lookup replacement (constants.kFoo_quarto.modules.constants.kFoo, same cached table require() returned before), so the only possible regression is a hard Lua error on a nil field, never silently wrong output. A plain quarto render can't exercise that failure mode either, since main.lua always loads the module registry (import_all.lua) before any filter runs. The path that actually broke (includes.lua) is only reachable through quarto editor-support crossref, which runs quarto-init.lua on its own without loading import_all first.

So I focused testing there:

  • quarto editor-support crossref on a few fixtures touching content-hidden, citations/tables, code-annotation, and raw LaTeX tables - all ran clean, valid JSON out, no _quarto.modules nil-index errors.
  • render a content-hidden doc to html - output correct.
  • render to typst (one static doc, one with an executed Python cell via great-tables), compiling through to PDF - no crash in typst.lua / typst-css-property-processing.lua.

Everything held up. Side note for anyone testing on Windows: extracting the build zip into a deeply nested path trips Windows' MAX_PATH and looks exactly like a Lua "module not found" error - nothing to do with this PR, just keep the extraction path short.

@cderv
cderv merged commit 45caede into main Jul 17, 2026
69 checks passed
@cderv
cderv deleted the feature/sweep-redundant-module-requires branch July 17, 2026 11:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants