[release/10.0][browser][mono] Make sure we treat the 4GB ptrs as unsigned

[lewing]

Fixes Issue #58382

main PR #121221

Description

The wasm/JS boundary does not preserve signedness — a wasm i32 arrives in JavaScript as a signed number, so every pointer above the 2GB mark shows up negative. Most of the resulting damage is silent rather than a crash: a negative subarray start is clamped relative to the end of the heap, and a negative HEAPU8 index yields undefined.

This is a backport of #121221, which coerces the pointers to unsigned at the JavaScript boundary: crypto buffers, debugger buffers, WebSocket buffers, event-pipe buffers, jiterpreter traces, JS MemoryView/Span, the various memory JS APIs, asset loading and string marshaling.

Relative to the original PR, the two src/native/libs/System.Native.Browser/* files are dropped — that is CoreCLR-on-wasm, which does not exist on this branch. Everything under src/mono/browser/runtime/ is taken as-is; the only textual adaptation is in crypto.ts, where the function is still named mono_wasm_browser_entropy here.

This also brings over the test from #131360 so the behaviour is covered going forward.

The two predecessor fixes in this area, #104662 and #109079, are already in release/10.0, and #124633 is CoreCLR-on-wasm only, so #121221 is the remaining piece.

Customer Impact

Requested by a customer who needs a mono wasm heap larger than 2GB on .NET 10.

Without this fix, an app configured with <EmccMaximumHeapSize> above 2GB corrupts data silently once allocations cross the boundary. Measured on the shipped 10.0.9 runtime pack with 2.2GB of heap in use, RandomNumberGenerator.Fill returns an all-zero buffer with no error — a security-relevant failure — and the jiterpreter emits invalid wasm modules (code generation failed: CompileError … extra bits in varint), silently losing tier-up. The remaining affected areas (debugger, WebSocket, event-pipe, asset loading, memory views) fail in similarly quiet ways.

Support above 2GB stays opt-in and "use at your own risk", but today it is quietly wrong rather than unsupported.

Regression

No. This has never worked; it is a long-standing limitation tracked by #58382.

Testing

Measured against the shipped runtime packs using a Microsoft.NET.Sdk.WebAssembly app with EmccMaximumHeapSize=4294901760, WasmBuildNative=true, and 2.2GB / 3.6GB of heap wasted in preRun before the runtime starts:

runtime pack	result
11.0.0-preview.6 @ 2.2GB and 3.6GB	all interop checks pass, no jiterpreter errors
10.0.9 @ 2.2GB	RandomNumberGenerator.Fill silently returns an all-zero buffer; jiterpreter logs code generation failed
10.0.9 with the crypto coercion from this PR hand-patched into dotnet.runtime.js	crypto check passes

The included MemoryTests scenario asserts data integrity — not just the absence of exceptions — across strings, byte[]/int[]/double[] in both directions, memory views, JSObject, exceptions, Task<int>, the JSExport direction, crypto and globalization, with every allocation above 2GB. It also fails if allocations did not actually land above 2GB, and asserts the run produced no jiterpreter code generation failures.

Risk

Low. The change is confined to the browser mono runtime JavaScript and is inert below 2GB: the added >>> 0 / fixupPointer coercions are no-ops for pointers that already fit in a signed 32-bit integer, which is every pointer in the default (2GB-capped) configuration. Reaching a heap above 2GB additionally requires <EmccMaximumHeapSize> plus a native relink via the wasm-tools workload, so the affected configuration is opt-in.

The one behavioural change beyond coercion is in the jiterpreter, where support for the unused constantSlots feature was dropped rather than fixed.

Note

This pull request was authored with GitHub Copilot.

[azure-pipelines]

Azure Pipelines:
Successfully started running 3 pipeline(s).
13 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[Copilot]

Pull request overview

Backports the >2GB (4GB address space) pointer signedness fixes for the Mono browser/WASM JavaScript runtime to release/10.0, and strengthens validation by turning the large-heap scenario into an interop conformance test that checks data integrity (not just “no exceptions”) above the 2GB boundary.

Changes:

• Coerce pointer/offset values at the JS/wasm boundary to unsigned (via >>> 0 / fixupPointer) across key runtime subsystems (strings, websockets, debugger, diagnostics, startup, marshaling, and jiterpreter).
• Update the jiterpreter wasm generation paths to avoid emitting invalid encodings when pointer-like values appear negative in JS.
• Expand the AllocateLargeHeapThenInterop test scenario into a richer interop conformance run and assert success + absence of jiterpreter codegen failures.

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
src/mono/wasm/Wasm.Build.Tests/MemoryTests.cs	Strengthens the large-heap test by asserting success output and no jiterpreter “code generation failed” messages.
src/mono/wasm/testassets/WasmBasicTestApp/App/wwwroot/main.js	Adds JS imports/exports used by the expanded interop conformance test and invokes the new async managed entrypoint.
src/mono/wasm/testassets/WasmBasicTestApp/App/MemoryTest.cs	Reworks the scenario into a conformance-style test covering many interop surfaces above 2GB.
src/mono/browser/runtime/web-socket.ts	Fixes buffer/view creation and stored pointers to treat offsets as unsigned.
src/mono/browser/runtime/strings.ts	Ensures string decoding/copying paths treat heap indices/pointers as unsigned.
src/mono/browser/runtime/startup.ts	Fixes assembly/pdb byte view creation to use unsigned offsets.
src/mono/browser/runtime/roots.ts	Ensures root buffer/external root addresses are treated as unsigned offsets in JS.
src/mono/browser/runtime/memory.ts	Adds fixupPointer and applies unsigned coercion for byte-indexed heap reads/writes and zeroing.
src/mono/browser/runtime/marshal.ts	Fixes pointer extraction and memory-view handling to use unsigned pointer values.
src/mono/browser/runtime/jiterpreter.ts	Coerces key pointer-like values to unsigned and updates builder usage.
src/mono/browser/runtime/jiterpreter-trace-generator.ts	Uses pointer-safe constant emission and unsigned offsets where required for wasm encoding correctness.
src/mono/browser/runtime/jiterpreter-support.ts	Removes constant-slot machinery and emits pointer constants in a way compatible with signed wasm immediates while fixing unsigned encodings where needed.
src/mono/browser/runtime/jiterpreter-jit-call.ts	Coerces pointer-like trampoline args to unsigned before use.
src/mono/browser/runtime/jiterpreter-interp-entry.ts	Ensures imethod/pointer-like values are handled as unsigned.
src/mono/browser/runtime/diagnostics/index.ts	Fixes diagnostic message buffer views to use unsigned offsets.
src/mono/browser/runtime/diagnostics/common.ts	Fixes writes back into HEAPU8 to use unsigned offsets.
src/mono/browser/runtime/debug.ts	Fixes debugger message buffer views to use unsigned offsets.
src/mono/browser/runtime/crypto.ts	Coerces entropy buffer pointer to unsigned before slicing heap views.

[dotnet-policy-service]

Tagging subscribers to 'arch-wasm': @lewing, @pavelsavara
See info in area-owners.md if you want to be subscribed.