fs/xipfs: writable execute-in-place file system for downloadable modules#19536
Open
casaroli wants to merge 8 commits into
Open
fs/xipfs: writable execute-in-place file system for downloadable modules#19536casaroli wants to merge 8 commits into
casaroli wants to merge 8 commits into
Conversation
casaroli
requested review from
Donny9,
GUIDINGLI,
acassis,
gustavonihei,
hartmannathan,
jerpelea,
masayuki2009,
raiden00pl and
xiaoxiang781216
as code owners
July 25, 2026 15:08
acassis
previously approved these changes
Jul 25, 2026
ROMFS is the usual way to carry executables on a NOMMU target with memory mapped NOR flash: it can hand out a real flash pointer from mmap(), so the NXFLAT loader maps a module's text in place instead of copying it into RAM. But a ROMFS image is built on the host and is read only, so a module cannot be downloaded onto the board at run time. xipfs is a writable file system with the same in-place property. Each file is stored as one physically contiguous, erase-block aligned extent, so an mmap() of it resolves to flash_base + extent_offset and a loader can execute the file where it already lies. This needs the underlying MTD driver to answer BIOC_XIPBASE; on the RP2350 rp23xx_flash_mtd.c does. Files are write once. A file is created, its size is declared, it is written sequentially, closed, and is thereafter immutable until it is deleted. That is the whole life cycle of a downloaded module, and it is what licenses the design: the exact extent is reserved at create time, so no file ever grows, moves, or fragments internally. Random writes, appends and truncation of a written file are not supported and are refused. The only source of fragmentation is therefore free space holes left by deletes. Allocation fails with -ENOSPC when no single contiguous run is large enough, and never defragments on its own; the caller decides whether to compact and retry, through XIPFSIOC_DEFRAG. Defragmentation is manual, best effort and interruptible: it is a loop of atomic single-extent relocations, each one copy, commit, erase, so every stop point -- a time budget, a pinned extent, an erase error -- leaves a consistent layout that is simply less compact. It reports the largest contiguous run it achieved, which is what tells the caller whether the retry will fit. Metadata is committed power safely. Two metadata block sets are used in ping-pong, each generation carrying a sequence number and a CRC, and every state change is ordered as write the new data, flip the metadata reference, then erase what the old one referenced. Mount scans both sets and selects the last fully valid generation, so a torn write costs the interrupted operation and nothing else. A mapping takes a pin on the extent, and the pin lives on the extent rather than on the file descriptor, so three running instances of one module hold three pins and the extent becomes movable only when the last one goes. Defragmentation skips pinned extents, which is what stops it relocating code that is executing. The pin is released by munmap() or by the task teardown walk, so a task that dies without unmapping does not leak it. Directories are records in that same generation, carrying their own identity and the identity of the directory holding them; the root is implicit and owns identity zero. They are deliberately NOT objects in the data region, which is what keeps the commit story in one piece: mkdir and rmdir add or remove a record and commit one generation, exactly as create and unlink do, so there is never a multi-object update to journal or an orphan to collect at mount. An empty directory therefore exists, survives a remount, and costs one entry out of the volume's fixed supply and no flash blocks at all. A name is one path component; depth comes from the parent, so XIPFS_NAME_MAX bounds a component, which is what statfs reports it as. Mount rebuilds the tree and checks that it is one: identities unique, names unique within a directory, every parent a live directory, and following parents reaching the root -- a cycle on the medium would otherwise hang a path walk rather than merely answering wrongly. '.' and '..' are refused as components, since an entry stored under either could never be reached again. The commands that act on the volume rather than on one file -- XIPFSIOC_DEFRAG and XIPFSIOC_LISTPINNED -- are reached through the ioctldir method, on a descriptor for the mountpoint directory. They are accepted on a descriptor for a file inside the volume too, but that route holds the file open for the duration and an open extent cannot be relocated, so a pass asked for that way is obstructed by the act of asking. mmap() falls back to the generic RAM copy for ordinary readers when the media cannot be addressed directly. A module loader must not silently get a RAM copy, so MAP_XIP_STRICT is added: with it the mapping either resolves in place or fails with -ENXIO, which the caller can turn into defragment and retry. Assisted-by: Claude Code:claude-opus-5 Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>
A file system that answers statfs with a magic nothing maps to shows up as "Unrecognized" in df. Give xipfs its constant alongside the others in sys/statfs.h and the case in fs_gettype that turns it into a name. Assisted-by: Claude Code:claude-opus-5 Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>
The flash MTD device answers BIOC_XIPBASE, which is what xipfs needs to serve mappings straight out of the memory mapped QSPI flash. Mount it at /mnt/xipfs when both are configured, formatting on first boot, so a board comes up with somewhere to download and run a module from. Assisted-by: Claude Code:claude-opus-5 Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>
rammtd answers BIOC_XIPBASE with the base of its RAM buffer, so it is a usable stand-in for memory mapped NOR: extents are directly addressable and the in-place mmap path can be exercised end to end without any flash. Mount xipfs on it when it is the configured file system, and add a configuration that runs the xipfs test suite, fault injection included. Assisted-by: Claude Code:claude-opus-5 Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>
Give the QEMU boards the same directly addressable media the sim configuration has: rammtd answers BIOC_XIPBASE with the base of its buffer, so xipfs layered on it hands out real pointers and the in-place mmap path can be exercised on an ARM target with no flash present. Registered as /dev/rammtd and mounted at /mnt/xipfs when xipfs is configured. Both mps2-an500 (Cortex-M7, armv7e-m) and mps2-an521 (Cortex-M33, armv8-m) get it, which is what makes the filesystem testable on two different core generations without either one needing flash. Each board also gets a xipfs configuration that runs the test suite, so the bringup above is exercised rather than only compiled. Both run the suite to completion under QEMU 10.1, 90 checks apiece, the power loss sweeps included. The an521 configuration carries CONFIG_CMSDK_UART0_RX_IRQ=48 and _TX_IRQ=49 rather than the reversed pair the an521 nsh configuration uses. That is the SSE-200 order, receive first, and the one consistent both with the overflow interrupt at 63 that nsh already has and with mps2-an500, which puts RX at 16 and TX at 17. With the pair reversed the TX interrupt reaches uart_cmsdk_rx_interrupt, which acknowledges only UART_INTSTATUS_RX, so the board live-locks on its first console write. Correcting nsh is left to a separate change. Assisted-by: Claude Code:claude-opus-5 Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>
Describe the write-once usage model, the strict in-place mmap and what MAP_XIP_STRICT is for, extent pinning, manual defragmentation and how to read its result, the power-loss ordering, the on-media layout, and the limitations. The NXFLAT page said ROMFS was the only file system able to serve the XIP mappings its loader needs. That is now one of two, so point at both, and at what the writable one adds: a module can arrive at run time instead of being baked into a host-built image. Assisted-by: Claude Code:claude-opus-5 Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>
A configuration for the Pimoroni Pico Plus 2 that mounts xipfs on the flash MTD and builds everything that exercises it: the test suite with fault injection, the xipfs command, and the NXFLAT execute-in-place demo. Building an NXFLAT module needs mknxflat and ldnxflat, so name them in the rp23xx board Make.defs files, which already carry the rest of the NXFLAT flags but not these. Assisted-by: Claude Code:claude-opus-5 Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>
Pages for the three applications that come with xipfs: the command that compacts a volume and prints its block map, the test suite and what each of its sections covers, and the demo that downloads an NXFLAT module into a volume and runs two instances of it in place. Assisted-by: Claude Code:claude-opus-5 Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds XIPFS, a writable file system that can serve execute-in-place
mappings, so a loadable module can arrive on the board at run time and still
be executed out of flash rather than copied into RAM.
Today ROMFS is the only in-tree file system able to serve the XIP mappings
the NXFLAT loader needs (
Documentation/components/nxflat.rstsays soexplicitly). A ROMFS image is built on the host and is read-only, so on a
NOMMU target with memory-mapped NOR there is no way to download a module and
run it in place. XIPFS closes that gap; this PR also updates the NXFLAT page,
which no longer has only one answer to point at.
How it works:
erase-block-aligned run, so
mmap()resolves toflash_base + offsetandthe loader executes the file where it already lies. This needs the MTD
driver underneath to answer
BIOC_XIPBASE.closed, and is immutable until deleted — the whole life cycle of a
downloaded module. That is what licenses reserving the exact extent up
front, which makes intra-file fragmentation impossible by construction.
Random writes, appends and truncation of a written file are refused.
-ENOSPCand never compacts on its own; the caller decides, viaXIPFSIOC_DEFRAGon a descriptor for the mountpoint directory (using theioctldirmethod added in fs/vfs: Add ioctldir for volume ioctls via the mountpoint directory. #19512). A pass is a loop of atomicsingle-extent relocations, so every stop point is a consistent layout, and
it reports the largest contiguous run it achieved — which is what tells the
caller whether a retry will fit.
write the header that carries the sequence number and CRC. That single
header program is the commit point, so a torn write costs the interrupted
operation and nothing else; mount selects the last fully valid generation.
the descriptor, so N running instances of one module hold N pins and the
extent becomes movable only when the last goes. Defragmentation skips
pinned extents, which is what stops it relocating code that is executing.
Pins are released by
munmap()or by task teardown, so a module thatfaults without unmapping does not leak one.
objects in the data region. That placement is deliberate:
mkdirandrmdiradd or remove a record and commit one generation, exactly as createand unlink do, so there is no multi-object update to journal and no orphan
to collect at mount. An empty directory therefore exists and survives a
remount, and costs one entry out of the volume's fixed supply and no flash
blocks. Mount rebuilds the tree and verifies it is a tree — unique
identities, unique names per directory, every parent a live directory, and
parents reaching the root — because a cycle on the medium would otherwise
hang a path walk rather than merely answer wrongly.
MAP_XIP_STRICTis added tosys/mman.h: with it a mapping either resolvesin place or fails with
-ENXIO, never silently falling back to theCONFIG_FS_RAMMAPcopy. A module loader must not get a RAM copy by accident,since that defeats the entire point.
The applications that go with this — the
xipfscompaction command, the testsuite whose results are quoted below, and the NXFLAT execute-in-place demo —
are in the companion PR:
Companion PR: apache/nuttx-apps#3665
Both are needed together: the test suite and demo in that PR are what exercise
this one, and its Kconfig options depend on
CONFIG_FS_XIPFSfrom here.Impact
CONFIG_FS_XIPFSdefaults tonanddepends on
MTDand!DISABLE_MOUNTPOINT. Nothing changes for existingconfigurations.
first writable one. The
Limitationssection of the docs is explicit aboutwhat XIPFS is not: no random writes, no appends, no growth, no rename, and a
file occupies a whole number of erase blocks.
include/sys/mman.h— one new flag,MAP_XIP_STRICT(bit 27).include/sys/statfs.handfs/mount/fs_gettype.c— a magic and a case sodfnames the file system instead of printing "Unrecognized".fs/mount/fs_mount.c— registersxipfsamong the MTD-backed filesystems.
Documentation/components/nxflat.rst— the "ROMFS is the only XIP-capablefile system" limitation is now two, and says which one is writable.
Make.defsfiles gainMKNXFLAT/LDNXFLAT, which everyother ARM board with NXFLAT support already names.
developed against is already upstream (rp23xx: add an MTD driver over the unused QSPI flash #19531).
fs/xipfs/; the fsCMakeLists.txtpicksthe directory up automatically.
refuses anything else with
-EFTYPE, whichautoformatturns into areformat where the mount asked for it.
Testing
Host: macOS 15 (arm64),
arm-none-eabi-gcc 14.2.rel1.Targets:
sim:xipfs— new configuration, XIPFS onrammtd(which answersBIOC_XIPBASE, so the in-place path is exercised without flash).pimoroni-pico-2-plus:xipfs— new configuration, XIPFS on the real QSPINOR via
/dev/rpflash, i.e. genuine erases and programs throughout.Both were verified with the test suite from the companion PR. Every run below
was bracketed by a
uname -acheck of the git revision and board name, beforethe first command and after the last, so the logs cannot be from a different
build.
pimoroni-pico-2-plus:xipfs— full suite, real flashThe power-loss sections fail the Nth flash write or erase, remount, and assert
the volume is consistent and every committed file byte-for-byte intact — with
the failing operation left torn (half a page programmed, half a sector
erased) as well as cleanly refused, because a torn generation is what forces
the mount-time CRC to do real work.
sim:xipfsExecute-in-place, end to end on hardware
An NXFLAT module written into the volume at run time and run twice
concurrently:
Both instances report the same text address — inside the flash window, equal
to where the file lies on the media — and different stacks.
Defragmentation on real NOR
Directories from the shell
tools/checkpatch.sh -c -u -m -gpasses on every commit in the branch.