RISC-V: minimal SBI runtime; PolarFire SoC boots 4-CPU SMP Yocto Linux

Adds a minimal M-mode SBI runtime (src/riscv_sbi.c) so wolfBoot can boot an S-mode OS with no external firmware: SBI BASE/TIME/IPI/RFENCE/HSM/DBCN plus the legacy calls; per-hart M-mode trap stacks switched via mscratch with a re-entrant trap entry (the OS stack pointer is virtual at trap time) and a wolfBoot gp reload in the handler; rdtime emulation for S and U mode (these harts have no time CSR) and byte-wise misaligned load/store emulation via MPRV (misaligned traps are not delegatable); MPRV-based guest memory access for console buffers; HSM hart_start with per-hart start mailboxes and remote RFENCE fence fan-out with completion wait.

PolarFire SoC platform wiring: CLINT MTIME time-base enable (SYSREG RTC divider, HSS parity) with all comparators parked (the reset value left MTIP pending, so parked harts' WFI never slept); eNVM clock (ENVM_CR) programmed before the AHB clock raise (HSS parity; unprogrammed it ran eNVM reads out of spec and corrupted L2-scratch code); mailbox-check-before-wfi fix in the U54 park loop; secondary harts set gp in the eNVM wake stub; cross-hart SBI state and start mailboxes live in the E51 DTIM and the M-mode trap stacks in the nomap-reserved hss-buffer DDR region, because cacheable stores to the L2 scratchpad can be lost on cache-line eviction; STACK_SIZE_PER_HART unified between the linker script and CFLAGS (the mismatch placed woken harts' stacks inside the E51 stack); watchdog policy (the MSS WDTs always count and reset at timeout: one final all-hart refresh at handoff, the parked E51 monitor keeps petting, OS watchdog nodes disabled in the dtb); SDHCI software-reset-all at handoff so the OS driver finds a clean controller; dtb fixups (no memory-node override -- the 1GB 32-bit window must not be widened over the non-cached alias; root on mmcblk0p3); DEBUG_DDR off by default (training no longer depends on printf timing thanks to the auto-init-disable reorder and the WRCALIB all-4-lane gate); obsolete bring-up diagnostics removed and the boot log reduced to milestones and errors.

Verified on the MPFS250T Video Kit: 'smp: Brought up 1 node, 4 CPUs', repeated clean cold boots to the systemd login prompt in ~40 s (10/10 and 5/5 gates), and 12-minute idle soaks with no reset.

Author: dgarske

.github/workflows/test-configs.yml

@@ -335,7 +335,6 @@ jobs:
     with:
       arch: riscv64
       config-file: ./config/examples/polarfire_mpfs250_m_qspi.config
-      make-args: LIBERO_FPGA_CONFIG_DIR=tools/ci/mpfs_libero_stub
 
   raspi3_test:
     uses: ./.github/workflows/test-build.yml

arch.mk

@@ -742,6 +742,10 @@ ifeq ($(ARCH),RISCV64)
   ifeq ($(RISCV_MMODE),1)
     # Machine Mode: Running directly from eNVM/L2 SRAM
     CFLAGS+=-DWOLFBOOT_RISCV_MMODE -DWOLFBOOT_DUALBOOT
+    # Minimal SBI runtime: services S-mode ecalls / timer / IPI when booting
+    # an S-mode OS (Linux).  Compiles to nothing unless WOLFBOOT_MMODE_SMODE_BOOT
+    # is also defined (see src/riscv_sbi.c).
+    OBJS+=src/riscv_sbi.o
     # Use M-mode specific linker script
     LSCRIPT_IN:=hal/$(TARGET)-m.ld
     # MPFS DDR init pulls LIBERO_SETTING_* values from a Libero/HSS-generated

config/examples/polarfire_mpfs250_m.config

@@ -45,7 +45,7 @@ OPTIMIZATION_LEVEL=1
 RISCV_MMODE?=1
 
 # Stack size per hart (L2 SRAM constraints)
-CFLAGS_EXTRA+=-DSTACK_SIZE_PER_HART=8192
+CFLAGS_EXTRA+=-DSTACK_SIZE_PER_HART=4096
 
 # E51 core lacks RISC-V crypto extensions (Zknh), use portable C
 # RISC-V SHA256/384/512 assembly (wolfcrypt port/riscv) enabled by default.
@@ -154,20 +154,8 @@ CFLAGS_EXTRA+=-DDEBUG_BOOT
 #CFLAGS_EXTRA+=-DDEBUG_SDHCI
 #CFLAGS_EXTRA+=-DDEBUG_DISK
 #CFLAGS_EXTRA+=-DDISK_TEST
-# DDR pattern test (256 KB triple-write at cached 0x82000000) is
-# DISABLED once disk-load works.  Its CPU writes thrash L2 cache
-# enough to corrupt L2 Scratch (where the M-mode stack lives) on the
-# return from mpfs_ddr_init().  Now that mpfs_clear_bootup_cache_ways
-# does PDMA pre-fill via the non-cached path, the pattern test is
-# redundant -- the MTC 256 B sanity inside the retry loop already
-# verifies DDR works end-to-end before pre-fill runs.
-#CFLAGS_EXTRA+=-DMPFS_DDR_PATTERN_TEST
-# Verbose register-level DDR tracing.  KEEP ENABLED until the timing
-# sensitivity in run_training is fixed -- the DBG_DDR printf delays
-# during the post-ZQ-cal phase appear to be required for TIP to
-# reach train_stat=0x1D consistently.  Removing -DDEBUG_DDR causes
-# train_stat to stick at 0x1.
-CFLAGS_EXTRA+=-DDEBUG_DDR
-# Phase 3.9 ruled out: kicking PHY_TRAINING_START=1 does not advance
-# TIP past BCLK_SCLK.  Macro left for posterity, off by default.
-#CFLAGS_EXTRA+=-DMPFS_DDR_KICK_TRAINING_START
+# Verbose register-level DDR tracing (development aid).  Historically the
+# DBG_DDR printf delays appeared load-bearing for TIP training; that was
+# resolved by the auto-init-disable training reorder and the WRCALIB
+# all-4-lane accept gate with retry, so this is safe to leave disabled.
+#CFLAGS_EXTRA+=-DDEBUG_DDR

docs/Targets.md

@@ -890,8 +890,12 @@ Six ready-to-use config templates cover all supported boot mode / storage / memo
 | **M-Mode + DDR** | `polarfire_mpfs250_m.config` | M-mode (E51, no HSS) | SD Card | LPDDR4 (DDR) | No |
 
 The **M-Mode + DDR** configuration brings up the LPDDR4 controller from
-the E51 in M-mode (no HSS), then loads a signed FIT image from SD card
-and hands off to a U54 hart in S-mode for Linux. Because all
+the E51 in M-mode (no HSS), then loads a signed FIT image from SD card,
+verifies it (SHA384 + ECC384) and hands off to a U54 hart in S-mode.
+wolfBoot includes a minimal SBI runtime (`src/riscv_sbi.c`) so the
+hand-off target can be a Linux kernel: tested booting 4-CPU SMP Yocto
+Linux to a login prompt in ~40 s from power-on on the MPFS250T Video
+Kit. Because all
 LIBERO_SETTING_\* values are board-specific, this build pulls them from
 a Libero/HSS-generated `fpga_design_config.h` pointed at by the
 `LIBERO_FPGA_CONFIG_DIR` makefile variable - typical sources are an
@@ -1135,8 +1139,7 @@ or DDR required. This is the simplest bring-up path.
 * Executes from L2 Scratchpad SRAM (256 KB at `0x0A000000`)
 * Loads signed application from SC QSPI flash to L2 Scratchpad (`0x0A010200`)
 * No HSS or DDR required — boots entirely from on-chip memory
-* Wakes and manages secondary U54 harts via IPI
-* Per-hart UART output (each hart uses its own MMUART)
+* Parks and releases secondary U54 harts via CLINT IPI
 * ECC384 + SHA384 signature verification
 
 **Relevant files:**
@@ -1241,8 +1244,51 @@ Booting at 0x...
 - **Strip debug symbols** before signing the test-app ELF. The debug build is ~150 KB but the
   stripped ELF is ~5 KB. L2 Scratchpad has ~150 KB available between wolfBoot code and the stack:
   `riscv64-unknown-elf-strip --strip-debug test-app/image.elf`
-- **DDR support:** DDR initialization is available on the `polarfire_ddr` branch for use cases
-  that require loading larger applications to DDR memory.
+- **DDR support:** software LPDDR4 initialization is included via the **M-Mode + DDR**
+  configuration (`polarfire_mpfs250_m.config`, requires `LIBERO_FPGA_CONFIG_DIR`) for use cases
+  that require loading larger images (e.g. a Linux FIT) to DDR memory. See the next section.
+
+### PolarFire SoC M-Mode + DDR: booting Linux (wolfSBI)
+
+The **M-Mode + DDR** configuration (`config/examples/polarfire_mpfs250_m.config`) replaces both
+HSS and OpenSBI: wolfBoot performs the LPDDR4 init/training on the E51, loads and verifies a
+signed Yocto FIT image (kernel + dtb) from SD card into DDR, applies device-tree fixups, releases
+U54 hart 1 into S-mode at the kernel entry, and then remains resident as a minimal M-mode SBI
+runtime. Validated on the MPFS250T Video Kit: 4-CPU SMP Yocto Linux to login in ~40 s from
+power-on.
+
+**wolfSBI runtime** (`src/riscv_sbi.c`, generic RISC-V with HAL hooks; enabled by
+`WOLFBOOT_MMODE_SMODE_BOOT`):
+* SBI v0.2 extensions: BASE, TIME (per-hart `mtimecmp`, MTIP-to-STIP injection), IPI (SSIP
+  injection via CLINT MSIP), RFENCE (remote `fence.i` / `sfence.vma` with completion wait),
+  HSM (`hart_start`/`hart_stop`/`hart_status` backed by per-hart start mailboxes), DBCN and the
+  legacy console putchar (shared with the wolfBoot UART), SRST.
+* `rdtime` emulation: the U54/E51 have no `time` CSR, so reads trap as illegal instruction from
+  S/U-mode and are emulated from CLINT MTIME (enabled via SYSREG `RTC_CLOCK_CR`, 1 MHz to match
+  the device tree `timebase-frequency`).
+* Misaligned load/store emulation (not delegatable on these harts) via MPRV byte accesses,
+  including compressed forms; the kernel relies on this for unaligned copy tails.
+* Per-hart M-mode trap stacks live in the `hss-buffer` reserved (nomap) DDR region; cross-hart
+  state (HSM mailboxes, IPI flags, the hart-release gate flag) lives in the E51 DTIM at
+  `0x01000000`, which is uncached and coherent for all harts. Cacheable L2-scratchpad memory
+  must not be used for cross-hart signalling (stores can be lost on dirty-line eviction).
+
+**Device-tree fixups** applied to the loaded dtb (`hal/mpfs250.c`): bootargs/root device,
+MAC addresses from the device serial number, and all five MSS watchdog nodes are disabled.
+
+**Watchdog policy:** the MSS watchdogs always count and reset the chip on timeout (they cannot
+be disabled in hardware, and `CONTROL=0` does not prevent the reset). After hand-off the parked
+E51 acts as a monitor and refreshes all five watchdogs; the OS watchdog driver is disabled via
+the dtb fixup so the two never conflict.
+
+**Hand-off / SMP flow:** secondary harts park in eNVM until the E51 signals image-copy
+completion (DTIM gate flag), then park in a WFI loop. The boot hart is released with a staged
+mailbox {entry, dtb} plus MSIP; Linux brings up the remaining harts through SBI HSM
+`hart_start`, which uses the same mailbox + MSIP path. The release path must stay fast
+(no UART access): the kernel allows roughly one second for a started hart to come online.
+
+**DEBUG_DDR note:** DDR training no longer depends on `DEBUG_DDR` console timing; the flag is
+purely diagnostic and off by default.
 
 ### PolarFire testing
 

hal/mpfs250-m.ld

@@ -122,9 +122,12 @@ PROVIDE(_start_heap = _end);
  *
  * Total stack area: STACK_SIZE + 4 * STACK_SIZE_PER_HART
  */
-/* M-mode: only E51 (hart 0) runs; secondary harts park in eNVM WFI loop.
- * Set to 0 so no L2 Scratch is wasted on phantom secondary stacks. */
-PROVIDE(STACK_SIZE_PER_HART = 0);
+/* Per-hart stacks for the secondary (U54) park/wake path.  MUST match the
+ * STACK_SIZE_PER_HART value in CFLAGS (the wake asm computes sp with the
+ * C macro while this symbol places the region): the historical 0-vs-8192
+ * mismatch placed the woken harts' stacks INSIDE the E51 stack region,
+ * smashing the monitor when SBI HSM started the secondary harts. */
+PROVIDE(STACK_SIZE_PER_HART = 4096);
 
 /* End of L2 scratchpad */
 PROVIDE(_l2_scratch_end = ORIGIN(L2_SCRATCH) + LENGTH(L2_SCRATCH));