Add -plugin-package-db for cross-compilation plugin support - #172
Open
angerman wants to merge 95 commits into
Open
Add -plugin-package-db for cross-compilation plugin support#172angerman wants to merge 95 commits into
angerman wants to merge 95 commits into
Conversation
This change reverts part of !14544, which forces the bootstrap compiler to have ghc-internal. As such it breaks booting with ghc 9.8.4. A better solution would be to make this conditional on the ghc version in the cabal file!
…ernal If the boot compiler doesn't have ghc-internal use "<unavailble>" as the `cGhcInternalUnitId`. This allows booting with older compilers. The subsequent stage2 compilers will have the proper ghc-internal id from their stage1 compiler, that boots them.
Make the first simple optimization pass after desugaring a real CoreToDo pass. This allows CorePlugins to decide whether they want to be executed before or after this pass.
It's more user-friendly to directly print the right thing instead of requiring the user to retry with the additional `-dppr-debug` flag.
mermaid is a common diagram format that can be inlined in markdown files, and e.g. github will even render it. This change adds support for mermaid diagram output to ghc-pkg.
This adds support to ghc-pkg to infer a package-db from a target name.
Add a new optional unitDataDir field to GhcPkg.InstalledPackageInfo and populate it in ghc-pkg's convertPackageInfoToCacheFormat. This is needed by the WASM linker to locate per-package data files (specifically WasmGlobalRegs.S) via GHC.Unit.Database.unitDataDir at link time, without requiring a file-system search.
By mistake we tried to use deriveConstant without passing `--gcc-flag -fcommon` (which Hadrian does) and it failed. This patch adds deriveConstant support for constants stored in the .bss section so that deriveConstant works without passing `-fcommon` to the C compiler.
Apple's LLVM toolchain uses `arm64` as the canonical architecture name for AArch64 on Apple platforms, while GNU config.sub normalises to `aarch64`. This mismatch causes `--target=aarch64-apple-darwin` to be passed to clang, which conflicts with toolchain wrappers (e.g. nix cc-wrapper) that expect `arm64-apple-darwin`. The result is thousands of test failures on aarch64-darwin because the cc-wrapper warning pollutes compiler output and the target flag interaction breaks compilation. Fix by adding normaliseLlvmTarget that rewrites `aarch64-apple-*` to `arm64-apple-*` for the LLVM target triple, matching Apple conventions and the existing llvm-targets file which already uses arm64-apple-darwin.
Add AC_ARG_WITH([compiler]) to allow specifying the Haskell compiler via the --with-compiler flag, consistent with standard autoconf practices for tool configuration. This provides an alternative to setting the GHC environment variable, making the build system more flexible and consistent with other configure scripts that accept --with-* options for tools.
Add entries to prevent AI agent config files from being accidentally committed. These files contain project-specific instructions for various AI coding assistants and should remain local. Covers: Claude Code, GitHub Copilot, Cursor, Gemini CLI/Jules, OpenAI Codex, and JetBrains Junie. See: https://agents.md/ for the AGENTS.md standard
Fixes #26434 In detail, this does a number of things: * Makes GHC aware of 'extra-libraries-static' (this changes the package database format). * Adds a switch '-static-external' that will honour 'extra-libraries-static' to link external system dependencies statically. * Adds a new field to settings/targets: "ld supports verbatim namespace". This field is used by '-static-external' to conditionally use '-l:foo.a' syntax during linking, which is more robust than trying to find the absolute path to an archive on our own. * Adds a switch '-fully-static' that is meant as a high-level interface for e.g. cabal. This also honours 'extra-libraries-static'. This also attempts to clean up the confusion around library search directories. At the moment, we have 3 types of directories in the package database format: * library-dirs * library-dirs-static * dynamic-library-dirs However, we only have two types of linking: dynamic or static. Given the existing logic in 'mungeDynLibFields', this patch assumes that 'library-dirs' is really just nothing but a fallback and always prefers the more specific variants if they exist and are non-empty. Conceptually, we should be ok with even just one search dirs variant. Haskell libraries are named differently depending on whether they're static or dynamic, so GHC can conveniently pick the right one depending on the linking needs. That means we don't really need to play tricks with search paths to convince the compiler to do linking as we want it. For system C libraries, the convention has been anyway to place static and dynamic libs next to each other, so we need to deal with that issue anyway and it is outside of our control. But this is out of the scope of this patch. This patch is backwards compatible with cabal. Cabal should however be patched to use the new '-fully-static' switch.
"Executable" seems more appropriate.
This patch teaches GHC how to build the external interpreter program when it is missing. As long as we have the `ghci` library, doing this is trivial so most of this patch is refactoring for doing it sanely. (cherry picked from commit 55eab80)
The comment still referenced the old `linkBinary` name after the rename to `linkExecutable` in 55ff022.
GHC and ghc-iserv load Haskell shared libraries dynamically for Template Haskell and GHCi. These libraries reference RTS symbols (e.g., stg_INTLIKE_closure) that are linked into the executable. Without special linker flags, those symbols aren't visible to dlopen'd libraries. This commit adds platform-specific linker flags to export these symbols: - Linux/FreeBSD: -rdynamic (passes --export-dynamic to ld) - macOS: -flat_namespace (makes all symbols visible across namespaces) - Windows: Cannot use --export-all-symbols due to 65535 symbol limit See Note [ghc-iserv and dynamic symbol export] in ghc-iserv.cabal.in for detailed explanation of the approach and alternatives considered.
Replace legacy __sync_fetch_and_* builtins with their modern __atomic_fetch_*
equivalents. This simplifies the code significantly, particularly for the nand
operation which previously required extensive workarounds for compiler
compatibility issues.
Changes:
- Replace __sync_fetch_and_{add,sub,and,or,xor} with __atomic_fetch_*
- Replace __sync_fetch_and_nand with __atomic_fetch_nand
- Remove CAS-based fallback for nand operations
- Remove compiler-specific warning suppressions for -Wsync-nand
- Remove volatile qualifiers (not needed with __atomic builtins)
- Update comments to reflect modern atomics usage
All operations maintain __ATOMIC_SEQ_CST memory ordering for sequential
consistency, matching the original behavior.
Co-authored-by: Andrea Bedini <andrea@andreabedini.com>
Add libffi-clib as a vendored library, replacing the external libffi dependency. Wire into compiler linker, RTS, and all build stages.
This commit adds build system support for creating dynamic GHC builds, including Makefile targets, bindist generation, and utility configurations. Key changes: 1. Makefile enhancements - Add DYNAMIC=1 build variable support - Create dylib symlinks for macOS dynamic builds - Use concrete file target for testsuite-timeout - Include ghc-iserv-dyn in tarballs for all targets - Proper bindist generation for dynamic builds 2. Utility cabal files (hp2ps.cabal, unlit.cabal) - Configure for dynamic linking support - Ensure utilities work with dynamic GHC 3. ghc-iserv infrastructure (iservmain.c) - Updates for dynamic interpreter server - Proper initialization for dynamic linking context 4. Test expectations for Stable Haskell - Update bug report URL in test expectations Usage: make DYNAMIC=1 _build/bindist # Build dynamic GHC bindist
This commit adds infrastructure for RTS sublibrary loading in dynamic builds, enabling the split RTS architecture to work with shared library linking. Key changes: 1. RTS sublibrary infrastructure (rts/rts.cabal) - Define separate sublibraries for RTS components - Add proper library dependencies and visibility - Configure shared library generation for RTS parts 2. Configure support for dynamic builds (rts/configure.ac) - Detect platform-specific dynamic linking requirements - Set appropriate linker flags for each sublibrary - Handle symbol visibility for exported functions 3. API updates for sublibrary boundaries (rts/include/RtsAPI.h) - Adjust exported symbol declarations - Ensure proper visibility across sublibrary boundaries 4. AutoApply support for interpreter (rts/AutoApply*.cmm) - Add AutoApply.cmm and vector variants (V16, V32, V64) - Required for dynamic bytecode interpreter operation 5. Cabal project configuration - cabal.project.stage1: Add no-ghc-internal flag for stage1 builds - cabal.project.stage2: Configure full RTS with all sublibraries 6. Thread infrastructure (rts/Threads.h) - Updates for sublibrary thread handling
This commit updates the testsuite to handle the split RTS architecture and dynamic GHC build configuration. Key changes: 1. testlib.py improvements - More robust test driver for dynamic builds - Better handling of shared library paths - Improved error detection and reporting 2. Test infrastructure (boilerplate.mk) - Configure tests for dynamic linking environment - Set proper library paths for test execution 3. Test adjustments for RTS split - T18072debug: Update grep to match cabal-based RTS naming - T23142.hs: Revert module name to fix -Di debug output test - keep-cafs-fail.stdout: Update expected output 4. Dynamic linking test updates - ghci/linking/dyn/all.T: Adjust for dynamic GHC - T2228: Restore expect_broken(7298) for dynamic builds - T11531.stderr: Update expected error messages 5. Platform-specific adjustments - T10458: Skip on musl with dynamic GHC - T11223 tests: Update stderr expectations for Windows 6. Test configuration - .gitignore: Add patterns for dynamic test artifacts - dynlibs/Makefile: Update for dynamic build testing - perf/size/all.T: Adjust size expectations
Add "Stable Haskell Edition" branding to user-visible output while maintaining drop-in compatibility with upstream GHC: - ghc --version: Append "(Stable Haskell Edition)" suffix - ghc -v2 banner: Add edition to verbose compiler banner - GHCi welcome: Add edition and update URL to GitHub repo - ghc --info: Add new "Edition" field (keeps "Project name" unchanged) - Bug reports: Redirect all URLs to github.com/stable-haskell/ghc/issues All internal identifiers (cProjectVersion, unit IDs, etc.) remain unchanged to preserve ABI and tool compatibility.
The branding commit changed the bug report URL from haskell.org/ghc/reportabug to github.com/stable-haskell/ghc/issues. Update test expectation files to match the new URL output. Fixes CI failures in T11223_link_order_a_b_2_fail and T11223_simple_duplicate_lib tests across all platforms.
This commit extends the CI/CD pipeline to build and test dynamic GHC configurations alongside the existing static builds. Key changes: 1. ci.yml - Main CI workflow - Add DYNAMIC=1 to build matrix - Configure dynamic build jobs for Linux and macOS - Run ghci-ext tests on dynamic builds (require interpreter) - Parallel execution of static and dynamic builds 2. reusable-release.yml - Release workflow - Add dynamic GHC builds to release artifacts - Generate separate bindists for dynamic configuration - Include ghc-iserv-dyn in release tarballs - Re-enable release workflow on pull requests for testing The dynamic build matrix allows testing of: - Template Haskell with dynamic code loading - GHCi interactive features - Dynamic library loading and linking - Interpreter-based test suites (ghci-ext) Build configurations: - Static (default): DYNAMIC=0 or unset - Dynamic: DYNAMIC=1
Add GitHub Actions release workflow with reusable build/test jobs, artifact upload/download, and proper _build/dist output paths.
Add QUIET mode for reduced output, per-phase timing instrumentation, metrics collection scripts, and matplotlib-based build phase plots. Includes macOS fixes and review feedback.
Since we now ignore loading ANY rts dependency.
- Add AR/RANLIB variables for wasm32-wasi-ar and wasm32-wasi-ranlib (explicit tool paths for cross-compilation) - Add --disable-libffi-adjustors to GHC_TOOLCHAIN_ARGS (wasm32 has no native adjustors, so ghc-toolchain defaults to libffi, but WASI cannot support ffi_closure_alloc which requires W^X memory; note: +use-system-libffi for general FFI is separate and correct) - Add -fno-exceptions to CXX_OPTS (wasi-sdk libc++ has no exception support)
* hsc2hs: batch cross-compilation for massive speedup Replace hackage hsc2hs-0.68.10 with stable-haskell/hsc2hs fork that includes batch cross-compilation support. This reduces C compiler invocations from hundreds/thousands per .hsc file to just 1-2 total by batching all constant-like directives (#const, #size, #alignment, step with graceful fallback to per-directive compilation. Key changes in the fork (bf966e8): - Batch collection of all batchable directives with conditional stack tracking - Single C file generation with all constants as global variables - Assembly parsing via ATTParser to extract all values at once - Graceful try/catch around ATT.parse for compilers producing non-AT&T assembly (e.g. emcc for WebAssembly), falling through to per-directive path - --no-batch flag for debugging/fallback Performance impact per .hsc file: - Before (non-via-asm): ~28 compilations per directive (e.g. 4089 for Flags.hsc) - Before (via-asm): 1 compilation per directive (e.g. 147 for Flags.hsc) - After (batch): 2 compilations total per file (validity check + batch) Fork: https://github.com/stable-haskell/hsc2hs/tree/feat/batch-cross-compilation * fix: update hsc2hs fork with Windows CodeView debug section fix Update hsc2hs fork reference to include fix for ATT parser crash on Windows where Clang emits CodeView .debug$S sections even with -g0. The parser now filters out debug sections and handles unrecognized instruction patterns gracefully.
.nix-wasm-bin/ — symlink directory created by flake.nix pointing
into the Nix store for wasi-sdk tools; local only
.nixos-remote-build.conf — per-machine config for the remote Linux build
helper script; not part of the source tree
angerman
force-pushed
the
stable-ghc-9.14
branch
from
March 6, 2026 01:06
05e1a0c to
7968d7c
Compare
When cross-compiling (host != target), plugins are built for the host
platform but GHC's UnitState contains target packages. Loading target
libraries into GHC's host process fails.
This adds a -plugin-package-db flag that points GHC at host-side
package databases. When specified, a separate UnitState is built from
these databases for plugin loading, while TH/bytecode continues to
use the target UnitState via the external interpreter.
Architecture:
No -plugin-package-db:
plugins use InternalInterp + existing target UnitState (unchanged)
With -plugin-package-db:
plugins use InternalInterp + host plugin UnitState (from host DBs)
TH/GHCi uses ExternalInterp + target UnitState (unchanged)
Changes:
- DynFlags: add pluginPackageDBFlags field
- Session: parse -plugin-package-db and -clear-plugin-package-dbs flags
- Unit/State: add initPluginUnitConfig using hostPlatformArchOS
- Driver/Env: add hscSetCurrentUnitState helper
- Runtime/Loader: extend withPluginInterp to swap UnitState when
plugin DBs are configured; replace checkExternalInterpreter with
withPluginInterp (creates local InternalInterp for plugin loading
alongside ExternalInterp for TH, see #14335)
T14335 previously tested that plugins + -fexternal-interpreter produced an error. With the new dual-interpreter architecture, plugins load via InternalInterp while TH uses the external interpreter, so this combination now succeeds. Change compile_fail → compile and remove the .stderr file.
The plugin now runs successfully with -fexternal-interpreter, producing its normal output (plugin passes queried, options, pass run). Add the expected stderr to match.
angerman
force-pushed
the
feat/plugin-package-db
branch
from
March 6, 2026 01:09
f838dbf to
e1ab30b
Compare
angerman
force-pushed
the
stable-ghc-9.14
branch
from
August 29, 2026 02:25
2c2ebfa to
a4f2bdf
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
-plugin-package-dband-clear-plugin-package-dbsflags so GHC can point at host-side package databases for plugin resolution during cross-compilation-plugin-package-dbis specified, builds a separateUnitStatefrom host package DBs usinghostPlatformArchOS, so plugins load host libraries while TH/bytecode continues using the targetUnitStatevia the external interpretercheckExternalInterpreter(which errored on-fexternal-interpreter+ plugins) withwithPluginInterp, which creates a localInternalInterpfor plugin loading alongside the external interpreter (#14335)Architecture
Files changed
compiler/GHC/Driver/DynFlags.hspluginPackageDBFlagsfield + recompilation checkcompiler/GHC/Driver/Session.hs-plugin-package-db,-clear-plugin-package-dbscompiler/GHC/Unit/State.hsinitPluginUnitConfig(host platform, plugin DBs)compiler/GHC/Driver/Env.hshscSetCurrentUnitStatehelpercompiler/GHC/Runtime/Loader.hswithPluginInterpto swap in hostUnitStateTest plan
-plugin-package-dbspecified)-fexternal-interpreter+ plugins (native): plugins load viaInternalInterp, TH via iserv — no regression-plugin-package-db /path/to/host/db: plugins resolve from host DB, load host libraries-fplugin-library: still works (independent path)