Unify wasm-bindgen output under -sWASM_BINDGEN, add -sWASM_BINDGEN=auto - #27208
Unify wasm-bindgen output under -sWASM_BINDGEN, add -sWASM_BINDGEN=auto#27208guybedford wants to merge 2 commits into
Conversation
cddd117 to
b7e9291
Compare
b7e9291 to
c10cb62
Compare
c10cb62 to
918ef09
Compare
|
@walkingeyerobot has now also confirmed that this unification approach works in his testing too. |
cc70816 to
cc29009
Compare
|
@sbc100 Sam, this looks good to me, but I'd appreciate it if you could give it a quick look before we hit the merge button. Thanks! |
| // The final EXPORTED_FUNCTIONS set, including additions made by JS | ||
| // libraries (e.g. wasm-bindgen self-registering its exports), so the | ||
| // caller can re-derive which library symbols were exported. | ||
| exportedFunctions: Array.from(EXPORTED_FUNCTIONS), |
There was a problem hiding this comment.
This new feature I think maybe requires a little attention.
I assume this is to that JS librarys can do things like:
EXPORTED_FUNCTIONS.add("foo")
?
I'm not sure how common this need is going to be. I wonder if there are any use cases in the emscirpten itself where we had to hack around the lack of this feature?
Could you share an example of exactly how/when wasm-bindgen adds to this list?
Normally settings flow in single direction "emcc -> js compiler" having settings flow back like this is maybe a little scary.
At the very least, I think we should land this as a separate PR adding this as a new feature with it own specific test case in test/test_jslib.js
There was a problem hiding this comment.
In Wasm Bindgen you can declare, e.g. a JS class export:
use wasm_bindgen::prelude::*;
#[wasm_bindgen]
pub struct Counter {
count: u32,
name: String,
}
#[wasm_bindgen]
impl Counter {
#[wasm_bindgen(constructor)]
pub fn new(name: String) -> Counter {
Counter { count: 0, name }
}
#[wasm_bindgen(getter)]
pub fn count(&self) -> u32 {
self.count
}
pub fn increment(&mut self) -> u32 {
self.count += 1;
self.count
}
}This is then provided as import { Counter } from 'mod' where Counter.prototype is configured correctly.
The name of the prototype is handled via configuration (with name modifers such as js_name and js_namespace), and is distinct from the name of the Rust mangled constructor function.
So the actual wasm bindgen exports are an entirely different layer to the Wasm module exports.
This allows us to drive this information back into the Emscripten linker.
The naive implementation is not possible because we currently spill all internal exports as public so do need to add filtering in this PR.
There was a problem hiding this comment.
I've gone ahead and split this out into #27436. This will be a prerequisite for this PR still.
| // Provide the aggregate exports object for code that reaches the wasm exports by | ||
| // name (e.g. wasm-bindgen's glue) via a namespace import. Emscripten's own named | ||
| // imports are unaffected and remain tree-shakable. | ||
| import * as wasmExports from './{{{ WASM_BINARY_FILE }}}'; |
There was a problem hiding this comment.
One of them primary reason we are pushing for WASM_ESM_INTEGRATION is to avoid this kind of thing, since I assume once your do this it makes DCE in the bundlers a lot harder/more complicated.
Presumably most programs don't actually need access to the full set of wasmExports, so perhaps we could at least narrow this down a litte?
There was a problem hiding this comment.
I've restricted this to the WASM_BINDGEN mode only.
Note that namespaces in JS bundlers are still fully tree-shakable as they are themselves fully statically analyzable so far as they do not escape analysis paths (which they usually don't so long as they aren't passed around).
…TIONS JS library code can already mutate the compile-time EXPORTED_FUNCTIONS set at library load time, which under MODULARIZE=instance causes jsifier to emit the symbol with an `export` declaration. This makes that flow fully work by forwarding the final EXPORTED_FUNCTIONS set back from the JS compiler so the linker can derive which JS library symbols were exported, and have the WASM_ESM_INTEGRATION wrapper re-export them. This is used by binding layers (e.g. wasm-bindgen) that define a public JS API surface distinct from the wasm export names, registering it from their generated JS library.
wasm-bindgen 0.2.126 (wasm-bindgen/wasm-bindgen#5210) hoists the clean exported API (functions, classes, enums) into top-level library symbols that self-register into EXPORTED_FUNCTIONS via its `--js-library`. The user-facing API now comes from wasm-bindgen's library in every flow, so emscripten no longer needs to guess or own the export set. This collapses the previous staticlib-only handling into a single -sWASM_BINDGEN path that works whether cargo/rustc drives the link (bin crate, emcc as the linker, rustc supplies -sEXPORTED_FUNCTIONS) or emcc drives it (staticlib, exports discovered locally): - The exports the wasm-bindgen expansion reaches by name - the supplied EXPORTED_FUNCTIONS (method shims, the __wbindgen_* runtime, the marker, main) plus anything its expansion adds - are internal glue, not a user API. They are captured and kept off every export layer: the ESM wrapper (user_requested_exports), the factory Module attachment (EXPORTED_FUNCTIONS, via should_export), and the keepalive pass in finalize_wasm. `main` still runs automatically on init; `_main` isn't surfaced. - A genuine EMSCRIPTEN_KEEPALIVE C/C++ export is not in that internal set and remains surfaced, so a hand-written native export composes with wasm-bindgen's API in the same module. Human-supplied EXPORTED_FUNCTIONS are not preserved through wasm-bindgen linkage yet (the rustc-supplied set is indistinguishable from glue); that can be revisited later. - Strip the placeholder symbols wasm-bindgen consumes (__wbindgen_describe*, __externref_*, ...) so they aren't reported as undefined exports. - Only run nm-based export discovery for explicit -sWASM_BINDGEN when no driver supplied EXPORTED_FUNCTIONS; the rustc-driven link already lists them exactly. - Wire imported JS: feed library_bindgen.extern-pre.js as extern-pre-js and copy the snippets/ dir next to the output so relative imports resolve. - The WASM_ESM_INTEGRATION wrapper re-exports the JS library symbols that were exported (MODULARIZE=instance), and provides wasmExports via a namespace import of the wasm so by-name export access works. Add -sWASM_BINDGEN=auto: run wasm-bindgen only when the linked wasm carries wasm-bindgen's __wasm_bindgen_emscripten_marker custom section, which is how cargo/rustc opts in when driving emcc as the linker (addressing the request to replace implicit marker detection with an explicit flag); otherwise it is a no-op and wasm-bindgen need not be installed. Both output modes then expose only the clean API (e.g. a `Greeter` class). Add an end-to-end test parameterized over the ESM and factory output modes (built via cargo with -sWASM_BINDGEN=auto), a no-marker test asserting auto is a no-op for an ordinary build, extend the staticlib integration test to assert an EMSCRIPTEN_KEEPALIVE export survives alongside the wasm-bindgen API, and install a pinned wasm-bindgen-cli alongside rust in CI so the flow is always exercised.
cc29009 to
b9865b9
Compare
This unifies wasm-bindgen output generation under a single
-sWASM_BINDGENpath and adds-sWASM_BINDGEN=auto, building on the wasm-bindgen side change in wasm-bindgen/wasm-bindgen#5210 (shipped in 0.2.126). It supersedes #27179.Previously
-sWASM_BINDGENwas effectively a staticlib-only flow where emcc had to discover and own the export set itself. With #5210, wasm-bindgen hoists its clean exported API (functions, classes, enums) into top-level library symbols that self-register intoEXPORTED_FUNCTIONSvia its--js-library. The user-facing API now comes from wasm-bindgen's own library in every flow, so emscripten no longer needs to guess or own exports — it just surfaces what wasm-bindgen registered. That lets one-sWASM_BINDGENpath serve both whether cargo/rustc drives the link (bin crate, emcc as the linker, rustc supplies-sEXPORTED_FUNCTIONS) or emcc drives it (staticlib, exports discovered locally).The model for a public export is the union across both systems: wasm-bindgen's self-registered API, plus emscripten's own
EMSCRIPTEN_KEEPALIVEexports — minus wasm-bindgen's internal expansion glue, which must not spill into the public surface.What's implemented:
EXPORTED_FUNCTIONS(method shims, the__wbindgen_*runtime, the marker,main) plus anything its expansion adds — are captured as internal glue and kept off every export layer: the ESM wrapper (user_requested_exports), the factoryModuleattachment (EXPORTED_FUNCTIONS, viashould_export), and the keepalive pass infinalize_wasm.mainstill runs automatically on init;_mainisn't surfaced.EMSCRIPTEN_KEEPALIVEC/C++ export is not in that internal set and remains surfaced, so a hand-written native export composes with wasm-bindgen's API in the same module.__wbindgen_describe*,__externref_*, ...) are stripped so they aren't reported as undefined exports.-sWASM_BINDGENwhen no driver suppliedEXPORTED_FUNCTIONS; the rustc-driven link already lists them exactly.library_bindgen.extern-pre.jsis fed as extern-pre-js and thesnippets/dir is copied next to the output so relative imports resolve.WASM_ESM_INTEGRATION,wasmExportsis provided via a namespace import of the wasm so wasm-bindgen's by-name export access works. This is gated behindWASM_BINDGENso plain ESM integration keeps per-symbol named imports and their tree-shakability. (The wrapper re-export of exported JS library symbols itself comes from Support JS libraries self-registering their exports via EXPORTED_FUNCTIONS #27436.)-sWASM_BINDGEN=auto(replacing #27179's implicit marker detection, per review feedback there): runs wasm-bindgen only when the linked wasm carries wasm-bindgen's__wasm_bindgen_emscripten_markercustom section — how cargo/rustc opts in when driving emcc as the linker. Otherwise it's a no-op and wasm-bindgen need not be installed, so rustc can always pass it via-Clink-argwithout affecting ordinary builds.Both output modes then expose only the clean API (e.g. a
Greeterclass).Testing:
-sWASM_ESM_INTEGRATION) and factory (-sMODULARIZE -sEXPORT_ES6) output modes, built viacargo buildwith-sWASM_BINDGEN=auto(marker auto-detected), asserting the cleanGreeterAPI works,mainruns on init, and none of the raw wasm exports leak.-sWASM_BINDGEN=autois a no-op for an ordinaryhello_world.cbuild (doesn't require wasm-bindgen installed).EMSCRIPTEN_KEEPALIVEnative export survives alongside the wasm-bindgen API (fails under a blanket export wipe).wasm-bindgen-clialongside rust so the flow is always exercised.Not in scope (follow-up): preserving human-supplied
EXPORTED_FUNCTIONSthrough wasm-bindgen linkage (the rustc-supplied set is currently indistinguishable from glue, so explicit exports aren't surfaced).