diff --git a/Documentation/applications/examples/nxflatxip/index.rst b/Documentation/applications/examples/nxflatxip/index.rst new file mode 100644 index 0000000000000..f3f2244d9768b --- /dev/null +++ b/Documentation/applications/examples/nxflatxip/index.rst @@ -0,0 +1,87 @@ +================================================= +``nxflatxip`` NXFLAT Executed In Place from XIPFS +================================================= + +Writes an NXFLAT module into a :doc:`XIPFS ` +volume at run time, the way a download would, and runs two instances of it +concurrently. The module's text is executed directly out of flash and shared +between the instances; each instance gets its own data. + +The same module in a ROMFS image would execute in place just as well. What it +could not do is arrive after the firmware was built, which is the point of +the exercise. + +Usage:: + + nxflatxip [bench | defrag] + +With no argument it stages the module, reports the extent it landed in and +the flash address the file system will map it at, then runs both instances:: + + nsh> nxflatxip + === NXFLAT module executed in place from xipfs === + + staged 268 bytes to /mnt/xipfs/xipmod + extent: block 6 x1, size 268, flash addr 0x10106000 + + running two concurrent instances... + + [while running] pin count on the shared text = 2 + + instance seed 1: text 0x10106070, stack 0x20006860, sum 2080 -- data private and intact + instance seed 2: text 0x10106070, stack 0x200074b0, sum 4160 -- data private and intact + [after exit] pin count on the shared text = 2 (released when this task exits) + +The two instances report the same text address, inside the flash window and +equal to where the file lies on the media, and different stack addresses. +While they run the extent carries one pin per instance, which is what stops +the defragmenter relocating code that is executing. + +The pins outlive the instances: ``binfmt`` maps the text in the context of +whoever called ``exec()``, so the mappings belong to the demo's address space +and are released when it exits. Run ``xipfs -n`` afterwards to see the count +back at zero. + +Subcommands +=========== + +``bench`` + Times a bulk read out of the mapped flash before and after a write. On a + target whose driver has to tear down and restore execute-in-place around + each erase, the difference is the cost of whichever restore path it used. + Needs a large file to read; the command says how to create one. + +``defrag`` + Fills the volume, deletes every other file until an allocation genuinely + cannot be satisfied, compacts, retries the same allocation, then verifies + every relocated file byte for byte and again after a remount. Prints a + block map at each step. + +Building the module +=================== + +The module is built from ``module/xipmod.c`` at build time, exactly the way +:doc:`../nxflat/index` builds its test programs, so it needs the same two +host tools from the NuttX toolchain, ``mknxflat`` and ``ldnxflat``, and the +board's ``Make.defs`` must name them. + +The module has no static data and no string constants, and reports through a +callback into the firmware rather than formatting its own output. The comment +at the top of ``module/xipmod.c`` records why: the ``ldnxflat`` in +circulation miscomputes both of those cases. The callback also exercises the +other direction of the interface, firmware code entered with the module's +data base still live in its PIC register. + +Configuration +============= + +``CONFIG_EXAMPLES_NXFLATXIP`` + Enable the example. Requires ``CONFIG_FS_XIPFS``, ``CONFIG_NXFLAT`` and + ``CONFIG_LIBC_EXECFUNCS``. + +``CONFIG_EXAMPLES_NXFLATXIP_MOUNTPT`` + The XIPFS volume to use. + +``CONFIG_EXAMPLES_NXFLATXIP_MTD`` + The MTD device backing that volume, used only by the ``defrag`` + subcommand, which remounts to show that what it did survives. diff --git a/Documentation/applications/system/xipfs/index.rst b/Documentation/applications/system/xipfs/index.rst new file mode 100644 index 0000000000000..487062969d873 --- /dev/null +++ b/Documentation/applications/system/xipfs/index.rst @@ -0,0 +1,68 @@ +========================================= +``xipfs`` XIPFS Compaction and Block Map +========================================= + +Compacts a :doc:`XIPFS ` volume and reports how +its erase blocks are laid out. + +Because a XIPFS file is a single contiguous extent, an allocation can fail +for lack of a contiguous run while plenty of free space remains. Compaction +coalesces the holes that deletes leave behind. It is never automatic: an +allocation simply fails with ``ENOSPC`` and the caller decides whether +compacting is worth its cost in erases. This command is that decision made +by hand. + +Usage:: + + xipfs [-n] [-t ] [] + +``-n`` + Survey and report only; move nothing. + +``-t `` + Give the pass a time budget. It stops at the first extent boundary after + the budget is spent, which is always a consistent layout. + +```` + The volume to act on. Defaults to ``CONFIG_SYSTEM_XIPFS_MOUNTPOINT``. + +The compaction itself is asked for through a descriptor for the mountpoint +directory, so no file inside the volume is open while it runs and one pass +can reach every extent. Without ``-n`` the command surveys, compacts, and +surveys again, so the map it prints last is the one on the media:: + + nsh> xipfs -n + /mnt/xipfs: 250 blocks of 4096 bytes (1000 KB) + + file start blocks bytes pin + xipmod 0 1 268 0 + + 0 |#.................................................| + 50 |..................................................| + + '.' free '#' in use 'P' pinned by a live mapping + + used 1, free 249, largest free run 249 blocks (996 KB) + 1 free run, fragmentation 0% + +The fragmentation figure is the share of free space a single allocation +cannot reach, ``(free - largest_run) / free``. At 0% the largest possible +file already fits however scattered the map looks, which is the question a +caller facing ``ENOSPC`` actually has. + +Files are listed by their path relative to the mount: the command walks the +volume's directories, which hold no blocks of their own and so do not appear +in the map. + +A ``P`` marks an extent held by a live execute-in-place mapping. Compaction +skips those, because relocating them would move code that is executing, so +they bound what any pass can reclaim. + +Configuration +============= + +``CONFIG_SYSTEM_XIPFS`` + Enable the command. Requires ``CONFIG_FS_XIPFS``. + +``CONFIG_SYSTEM_XIPFS_MOUNTPOINT`` + The volume used when none is named on the command line. diff --git a/Documentation/applications/testing/xipfs/index.rst b/Documentation/applications/testing/xipfs/index.rst new file mode 100644 index 0000000000000..7b68753c03f0d --- /dev/null +++ b/Documentation/applications/testing/xipfs/index.rst @@ -0,0 +1,98 @@ +================================== +``xipfs`` XIPFS File System Test +================================== + +Exercises the :doc:`XIPFS ` file system: the +routine VFS paths, the write-once rules, both flavours of mapping, and then +the two properties that are easy to get wrong and quiet when they are -- +release of execute-in-place pins, and power-loss atomicity of the metadata +commit. + +Usage:: + + xipfs_test [
] + +With no argument every section runs. The power-loss sweeps take minutes while +everything else takes seconds, hence the selector:: + + nsh> xipfs_test xip + XIPFS test suite on /mnt/xipfs (/dev/rpflash) + -- XIP mapping -- + PASS create module image + PASS media is memory mapped + PASS strict XIP mmap succeeds + PASS mapping points into flash + PASS mapped bytes match file contents + PASS mapping does not cost RAM proportional to the file + PASS mapping takes a pin + PASS munmap + PASS munmap drops the pin + ==== 9 passed, 0 failed ==== + +Sections +======== + +``basic`` + Create, read back, stat, readdir, unlink, and that all of it survives a + remount. + +``writeonce`` + That reopening a closed file for writing, appending, and seeking during a + write are refused, and that a sequential write still works afterwards. + +``strict`` + That a mapping past the end of a file is refused and a valid one is not. + +``xip`` + That a mapping lands inside the media window rather than the heap, that the + mapped bytes are the file's, that it costs no RAM proportional to the file, + and that it takes and releases a pin. + +``multimap`` + That N mappings of one file produce N pins on one extent and alias the same + address. + +``crosstask`` + That one task dying does not release another task's pin, and that the last + holder does. + +``teardown`` + That a task killed without calling ``munmap`` still has its pin released. + This is the one that matters for a module that faults. + +``dirs`` + That directories behave like a real tree: an empty one exists and survives + a remount, ``readdir`` reports each level, ``rmdir`` refuses a directory + that still holds something and succeeds once emptied, and a path through a + missing directory or through a file fails rather than being created. + +``defrag`` + That compaction relocates extents, keeps their contents, survives a + remount, distinguishes an open file from a pinned one, and never relocates + a pinned extent. + +``powerloss``, ``powertorn``, ``unlink``, ``defraglos`` + Fail the Nth flash operation, remount, and check the volume is consistent + and every committed file byte-for-byte intact -- sweeping N across create, + unlink and compaction. The ``torn`` variant leaves the failing write + partially programmed rather than cleanly refused, which is what a real + power loss mid-program leaves behind. These need + ``CONFIG_FS_XIPFS_FAULT_INJECT``. + +Configuration +============= + +``CONFIG_TESTING_FS_XIPFS`` + Enable the test. Requires ``CONFIG_FS_XIPFS``. + +``CONFIG_TESTING_FS_XIPFS_MOUNTPT`` + The volume to test. The suite remounts it, so nothing else should be using + it. + +``CONFIG_TESTING_FS_XIPFS_MTD`` + The MTD device backing that volume, needed for the remounts. + +.. note:: + + The suite writes and erases the volume continuously, the power-loss + sections most of all. Point it at a volume whose contents do not matter. diff --git a/Documentation/components/filesystem/index.rst b/Documentation/components/filesystem/index.rst index 1e44ca94a656b..bb3fe88d99f72 100644 --- a/Documentation/components/filesystem/index.rst +++ b/Documentation/components/filesystem/index.rst @@ -587,6 +587,7 @@ NuttX provides support for a variety of file systems out of the box. tmpfs.rst unionfs.rst userfs.rst + xipfs.rst zipfs.rst inotify.rst nuttxfs.rst @@ -603,7 +604,7 @@ they require: 1. They require a block device driver. They include vfat, romfs, smartfs, and littlefs. -2. They require MTD drivers. They include romfs, spiffs, littlefs. +2. They require MTD drivers. They include romfs, spiffs, littlefs, xipfs. 3. They require neither block nor MTD drivers. They include nxffs, tmpfs, nfs binfs, procfs, userfs, hostfs, cromfs, unionfs, rpmsgfs, and zipfs. diff --git a/Documentation/components/filesystem/xipfs.rst b/Documentation/components/filesystem/xipfs.rst new file mode 100644 index 0000000000000..eef89c5d394c6 --- /dev/null +++ b/Documentation/components/filesystem/xipfs.rst @@ -0,0 +1,272 @@ +====================================== +XIPFS Contiguous Execute-In-Place FS +====================================== + +XIPFS stores each file as a single physically **contiguous**, erase-block +aligned extent on memory-mapped flash, and exposes a true execute-in-place +``mmap`` that returns a direct pointer into that flash rather than a RAM +copy. + +It exists for one purpose: letting a NOMMU target run downloadable modules +out of NOR flash so that their read-only text and rodata never consume RAM. +Only a running module's writable segment is copied. + +Configuration +============= + +``CONFIG_FS_XIPFS`` + Enable the filesystem. Requires ``CONFIG_MTD``. + +``CONFIG_FS_XIPFS_FAULT_INJECT`` + Test-only. Adds a countdown to the flash write and erase paths so a test + can fail an arbitrary operation and then remount, modelling a power loss + at that exact point. Do not enable in production. + +XIPFS mounts on an **MTD** device:: + + mount -t xipfs /dev/rammtd /mnt/xipfs + +Pass ``autoformat`` as the mount data to format an unrecognised volume +rather than failing with ``EFTYPE``. + +For XIP mappings to be possible at all, the underlying MTD driver must +implement the ``BIOC_XIPBASE`` ioctl, returning the directly addressable +base of the media. ``rammtd`` does, which makes it a usable stand-in for +memory-mapped NOR during development. A driver that does not is still +perfectly usable as a filesystem; it simply cannot serve XIP mappings. + +Usage model +=========== + +XIPFS is **not** a general purpose read-write filesystem, and does not +pretend to be. Files are **write once**: + +* A file is created, its size is declared, it is written sequentially, and + it is closed. +* Thereafter it is immutable until it is deleted. Reopening for writing, + appending, and seeking during a write are all refused. + +Declaring the size up front with ``ftruncate()`` is the preferred path, +because it lets the filesystem reserve exactly the right contiguous extent:: + + fd = open("/mnt/xipfs/module.bin", O_WRONLY | O_CREAT | O_TRUNC, 0644); + ftruncate(fd, module_size); + write(fd, module_data, module_size); + close(fd); + +If the size is not declared, the largest available free run is reserved and +trimmed back at close. That keeps ordinary sequential writes -- including +shell redirection, and a file arriving over a serial transfer -- working, at +the cost of temporarily reserving more space than needed. + +Such a reservation is erased **lazily**, one block ahead of the writer, +rather than in full when it is taken. Erasing it up front would make the +cost of writing a small file proportional to the free space instead of to +the file: on an empty 1 MB volume that was roughly 250 sector erases, about +13 seconds on an RP2350, with interrupts disabled for each one. Long enough +that the far end of a serial transfer gives up. The blocks the trim at +close hands back were never erased at all. + +An exact reservation from ``ftruncate()`` is already the right size, so +there is nothing to defer and it is still erased when taken. + +Directories +----------- + +Directories are real: they are records in the metadata generation, so an +empty one exists, survives a remount, and is removed by ``rmdir``. What they +are **not** is objects in the data region. A directory costs no data block +and no erase; it costs one entry out of the volume's fixed supply, and +``statfs`` reports that supply as ``f_files``/``f_ffree``. + +That placement is deliberate, and it is what keeps the power-safety story in +one piece: ``mkdir`` and ``rmdir`` add or remove a record and commit a single +generation, exactly as ``open(O_CREAT)`` and ``unlink`` do. There is no +multi-object update to journal and no orphan to collect at mount. + +Each entry carries its own identity and the identity of the directory that +holds it; the root is implicit and owns identity zero. A name is therefore +one path component, and ``XIPFS_NAME_MAX`` bounds a component rather than a +whole path -- which is what ``statfs`` reports as ``f_namelen``. + +Mount rebuilds the tree from those records and checks that it *is* a tree: +identities must be unique, names must be unique within a directory, every +parent must name a live directory, and following parents must reach the +root. A cycle on the medium would otherwise hang a path walk rather than +merely giving a wrong answer. + +``.`` and ``..`` are refused as components with ``EINVAL`` rather than +interpreted. An entry stored under either name could never be reached again, +and the VFS has already resolved the ones that were meant navigationally. + +Unlike a filesystem that synthesises directories from names, a path through +something that is not a directory fails rather than being created: writing +to ``bin/hello`` when ``bin`` does not exist gives ``ENOENT``, and when +``bin`` is a file, ``ENOTDIR``. Create the directories first. + +Execute-in-place mapping +======================== + +A normal ``mmap()`` returns a direct flash pointer when it can and falls +back to the VFS RAM copy when it cannot. That fallback is convenient for +data readers but fatal for a module loader, which would silently lose the +entire benefit of executing in place. + +Loaders must therefore pass ``MAP_XIP_STRICT``:: + + addr = mmap(NULL, len, PROT_READ | PROT_EXEC, + MAP_SHARED | MAP_XIP_STRICT, fd, 0); + +With that flag, a mapping that cannot be resolved in place fails with +``ENXIO`` instead of being copied. The usual response is to defragment and +retry, or to refuse the load. + +Pinning +------- + +A mapping that aliases flash takes a **pin** on the extent, and the pin +lives on the extent rather than on the file descriptor. Three running +instances of one module therefore produce a pin count of three, and the +extent becomes movable again only when the last of them goes away. + +An extent with a non-zero pin count is never relocated or erased by the +defragmenter, which is what allows a module to keep executing from flash +while the filesystem is being compacted around it. + +Pins are released on ``munmap`` **or** on task teardown. A module that +faults or is killed without unmapping still drops its pin, because +``mm_map_destroy()`` walks the dying task's mapping list and invokes each +mapping's unmap callback. Nothing relies on the application behaving well. + +Defragmentation +=============== + +Because files are created at a known size and never grow, a file can never +fragment internally. The only thing that fragments is free space, through +the holes deletes leave behind. + +Compaction is therefore **manual, best effort and pin aware**. It is never +invoked from inside a failing allocation: allocation simply returns +``ENOSPC`` and the caller decides whether compacting is worth it. + +:: + + struct xipfs_defrag_arg_s arg = { .max_ms = 50 }; + int fd = open("/mnt/xipfs", O_RDONLY | O_DIRECTORY); + ioctl(fd, XIPFSIOC_DEFRAG, &arg); + /* arg.result.largest_free_run says whether a retry can now succeed */ + +The pass is a loop of atomic relocations. Each one copies an extent into +free space, commits new metadata naming the new location, and only then +erases the vacated blocks, so every iteration boundary is a fully +consistent layout. Any reason for stopping -- a time budget, a pinned +extent in the way, a media error, a power cut -- lands on one of those +boundaries. Nothing is ever left half moved. + +The result reports what was achieved and why it stopped: + +``XIPFS_DEFRAG_DONE`` + Nothing left to compact. +``XIPFS_DEFRAG_BLOCKED_PINS`` + A live XIP mapping is in the way. Resolve by unloading a module; + ``XIPFSIOC_LISTPINNED`` reports which. +``XIPFS_DEFRAG_BLOCKED_OPEN`` + A merely open file is in the way. Resolve by closing a descriptor. +``XIPFS_DEFRAG_BLOCKED_RAM`` + Transient resource shortage. +``XIPFS_DEFRAG_TIME_BUDGET`` + The caller's ``max_ms`` was reached. +``XIPFS_DEFRAG_ERROR`` + A media error stopped the pass cleanly. + +.. note:: + + Issue the volume commands -- ``XIPFSIOC_DEFRAG``, ``XIPFSIOC_LISTPINNED`` + and ``XIPFSIOC_FAULTINJECT`` -- on a descriptor for the **mountpoint + directory**, obtained with ``open(mountpoint, O_RDONLY | O_DIRECTORY)``. + They reach the file system through its ``ioctldir`` method. + + A descriptor for a file inside the volume is also accepted, but it holds + that file open for the duration, and an open extent cannot be relocated. + A pass asked for that way is therefore obstructed by the act of asking, + and normally reports ``XIPFS_DEFRAG_BLOCKED_OPEN`` with the caller's own + file as the obstruction. + + ``XIPFSIOC_EXTENTINFO``, ``XIPFSIOC_PIN`` and ``XIPFSIOC_UNPIN`` name a + file rather than the volume, so those do take a descriptor for the file + itself. + +The ``xipfs`` command +--------------------- + +``apps/system/xipfs`` wraps the ioctl and reports the layout:: + + xipfs [-n] [-t ] [] + +``-n`` surveys and prints without moving anything: the per-file table, a +block map, and how much of the free space a single allocation can reach. +``-t`` passes a time budget to the pass. Without ``-n`` it surveys, +compacts, and surveys again, so the final map is the one on the media:: + + /mnt/xipfs: 250 blocks of 4096 bytes (1000 KB) + + 0 |#####.............................................| + + '.' free '#' in use 'P' pinned by a live mapping + + used 5, free 245, largest free run 245 blocks (980 KB) + 1 free run, fragmentation 0% + +The fragmentation figure is the share of free space that a single +allocation cannot reach, ``(free - largest_run) / free``. At 0% the +largest possible file already fits however scattered the map looks, which +is the question a caller facing ``ENOSPC`` actually has. + +Power-loss behaviour +==================== + +Metadata is a ping-pong ring of whole generations. Each commit erases the +next ring slot, writes the directory body, and finally writes the header +carrying the sequence number and a CRC over header plus body. + +That final header write is the commit point, and it is a single read/write +block program -- the smallest unit the media can tear at. A power loss +before it leaves a slot whose header is still erased, so the generation +does not exist. A power loss during it leaves a header failing its own +CRC. Either way mount falls back to the previous generation, which is +untouched because it lives in a different slot. + +Every state change follows the same shape: write the new data, flip the +metadata reference, then erase the old. The reference flip is always the +commit point, and the old copy is never erased before the new reference is +durable. + +On-media layout +=============== + +:: + + block 0 superblock copy A + block 1 superblock copy B + block 2..5 metadata ring (4 generations, ping-pong). A generation is + a header plus one 64-byte record per file and per directory + block 6.. data region, contiguous erase-block aligned extents. Only + files live here; a directory is a record and nothing more + +Limitations +=========== + +* A directory costs one of the volume's fixed number of entries, since it is + a metadata record. On a 4 KiB sector with 256-byte pages that supply is 60 + entries, files and directories together. +* No random writes, appends, growth, or rename. +* A file occupies a whole number of erase blocks, so a small file consumes + a full block. This is accepted deliberately: it means no block ever + holds a mix of live and dead data, which is what makes delete, relocate + and erase clean operations and shrinks the defrag staging buffer to a + single program page. +* Erase slicing, erase-suspend and running the erase routine from RAM are + not implemented. On a single NOR die without read-while-write these are + needed to bound interrupt latency; they belong in + ``fs/xipfs/xipfs_flash.c``, through which every media access already + funnels. diff --git a/Documentation/components/nxflat.rst b/Documentation/components/nxflat.rst index e6d70f53f0a05..755af04e43177 100644 --- a/Documentation/components/nxflat.rst +++ b/Documentation/components/nxflat.rst @@ -59,18 +59,22 @@ module to the base code. Limitations ----------- - - **ROMFS (or RAM mapping) Only**: + - **XIP-Capable File System (or RAM mapping) Only**: The current NXFLAT release will work only with either (1) NXFLAT - executable modules residing on a ROMFS file system, or (2) executables - residing on other file systems provided that CONFIG_FS_RAMMAP is - defined. This limitation is because the loader depends on the capability - to mmap() the code segment. See the NuttX User Guide for further information. + executable modules residing on a file system that supports execute in + place, or (2) executables residing on other file systems provided that + CONFIG_FS_RAMMAP is defined. This limitation is because the loader depends + on the capability to mmap() the code segment. See the NuttX User Guide for + further information. NUTTX does not provide any general kind of file mapping capability. In fact, true file mapping is only possible with MCUs that provide an MMU1. Without an MMU, file system may support eXecution In Place (XIP) to mimic - file mapping. Only the ROMFS file system supports that kind of XIP execution - need by NXFLAT. + file mapping. Two file systems support that kind of XIP execution needed + by NXFLAT: :doc:`ROMFS `, whose image is built on the + host and is read-only, and :doc:`XIPFS `, which is + writable, so a module can be downloaded onto the board at run time and + still be executed out of flash. It is also possible to simulate file mapping by allocating memory, copying the NXFLAT binary file into memory, and executing from the copy of the diff --git a/boards/arm/mps/mps2-an500/configs/xipfs/defconfig b/boards/arm/mps/mps2-an500/configs/xipfs/defconfig new file mode 100644 index 0000000000000..8727ce5d5cb4e --- /dev/null +++ b/boards/arm/mps/mps2-an500/configs/xipfs/defconfig @@ -0,0 +1,88 @@ +# +# This file is autogenerated: PLEASE DO NOT EDIT IT. +# +# You can use "make menuconfig" to make any modifications to the installed .config file. +# You can then do "make savedefconfig" to generate a new defconfig file that includes your +# modifications. +# +CONFIG_ARCH="arm" +CONFIG_ARCH_BOARD="mps2-an500" +CONFIG_ARCH_BOARD_MPS2_AN500=y +CONFIG_ARCH_CHIP="mps" +CONFIG_ARCH_CHIP_MPS2_AN500=y +CONFIG_ARCH_CHIP_MPS=y +CONFIG_ARCH_INTERRUPTSTACK=2048 +CONFIG_ARCH_STACKDUMP=y +CONFIG_ARMV7M_SYSTICK=y +CONFIG_ARM_MPU=y +CONFIG_ARM_MPU_NREGIONS=1 +CONFIG_ARM_MPU_RESET=y +CONFIG_BUILTIN=y +CONFIG_CMSDK_UART0=y +CONFIG_CMSDK_UART0_BASE=0x40004000 +CONFIG_CMSDK_UART0_CLOCK=25000000 +CONFIG_CMSDK_UART0_OV_IRQ=28 +CONFIG_CMSDK_UART0_RX_IRQ=16 +CONFIG_CMSDK_UART0_SERIAL_CONSOLE=y +CONFIG_CMSDK_UART0_TX_IRQ=17 +CONFIG_CMSDK_UART=y +CONFIG_CXX_STANDARD="c++17" +CONFIG_DEBUG_ASSERTIONS=y +CONFIG_DEBUG_BUSFAULT=y +CONFIG_DEBUG_FEATURES=y +CONFIG_DEBUG_HARDFAULT_ALERT=y +CONFIG_DEBUG_SCHED=y +CONFIG_DEBUG_SCHED_ERROR=y +CONFIG_DEBUG_SCHED_INFO=y +CONFIG_DEBUG_SCHED_WARN=y +CONFIG_DEBUG_SYMBOLS=y +CONFIG_DEBUG_USAGEFAULT=y +CONFIG_DEFAULT_TASK_STACKSIZE=4096 +CONFIG_EXPERIMENTAL=y +CONFIG_FRAME_POINTER=y +CONFIG_FS_PROCFS=y +CONFIG_FS_ROMFS=y +CONFIG_FS_TMPFS=y +CONFIG_FS_XIPFS=y +CONFIG_FS_XIPFS_FAULT_INJECT=y +CONFIG_HAVE_CXX=y +CONFIG_HAVE_CXXINITIALIZE=y +CONFIG_IDLETHREAD_STACKSIZE=4096 +CONFIG_INIT_ENTRYPOINT="nsh_main" +CONFIG_INTELHEX_BINARY=y +CONFIG_LIBC_MEMFD_ERROR=y +CONFIG_LIBC_STRERROR=y +CONFIG_LIBC_STRERROR_ERRNUM=y +CONFIG_MTD=y +CONFIG_NSH_BUILTIN_APPS=y +CONFIG_NSH_FILEIOSIZE=512 +CONFIG_NSH_READLINE=y +CONFIG_PREALLOC_TIMERS=4 +CONFIG_PROFILE_ALL=y +CONFIG_PROFILE_MINI=y +CONFIG_RAMLOG=y +CONFIG_RAMMTD=y +CONFIG_RAMMTD_FLASHSIM=y +CONFIG_RAM_SIZE=2097152 +CONFIG_RAM_START=0x60000000 +CONFIG_RAW_BINARY=y +CONFIG_READLINE_CMD_HISTORY=y +CONFIG_RR_INTERVAL=200 +CONFIG_SCHED_HPWORK=y +CONFIG_SCHED_HPWORKPRIORITY=192 +CONFIG_SCHED_IRQMONITOR=y +CONFIG_SPINLOCK=y +CONFIG_STACK_COLORATION=y +CONFIG_STANDARD_SERIAL=y +CONFIG_START_DAY=25 +CONFIG_START_MONTH=4 +CONFIG_START_YEAR=2023 +CONFIG_SYMTAB_ORDEREDBYNAME=y +CONFIG_SYSTEM_GPROF=y +CONFIG_SYSTEM_NSH=y +CONFIG_SYSTEM_SYSTEM=y +CONFIG_SYSTEM_XIPFS=y +CONFIG_TESTING_FS_XIPFS=y +CONFIG_TIMER=y +CONFIG_TIMER_ARCH=y +CONFIG_USEC_PER_TICK=1000 diff --git a/boards/arm/mps/mps2-an500/src/mps2_bringup.c b/boards/arm/mps/mps2-an500/src/mps2_bringup.c index 8eca057051824..3d80e17ea9c67 100644 --- a/boards/arm/mps/mps2-an500/src/mps2_bringup.c +++ b/boards/arm/mps/mps2-an500/src/mps2_bringup.c @@ -31,10 +31,20 @@ #include +#ifdef CONFIG_RAMMTD +# include +# include +# include +#endif + /**************************************************************************** * Pre-processor Definitions ****************************************************************************/ +#ifdef CONFIG_RAMMTD +# define MPS2_RAMMTD_SIZE (256 * 1024) +#endif + /**************************************************************************** * Private Functions ****************************************************************************/ @@ -73,6 +83,54 @@ static int mps2_bringup(void) } #endif +#ifdef CONFIG_RAMMTD + /* A RAM backed MTD device standing in for memory mapped NOR. rammtd + * answers BIOC_XIPBASE with the base of its buffer, so xipfs layered on it + * can hand out real, directly executable pointers -- which is what lets a + * module be executed in place rather than copied. This gives the xipfs + * test suite a target with no real flash. + */ + + { + FAR uint8_t *ramstart = kmm_malloc(MPS2_RAMMTD_SIZE); + FAR struct mtd_dev_s *mtd; + + if (ramstart == NULL) + { + syslog(LOG_ERR, "ERROR: Failed to allocate RAM MTD\n"); + } + else if ((mtd = rammtd_initialize(ramstart, MPS2_RAMMTD_SIZE)) == NULL) + { + syslog(LOG_ERR, "ERROR: rammtd_initialize failed\n"); + kmm_free(ramstart); + } + else + { + mtd->ioctl(mtd, MTDIOC_BULKERASE, 0); + + ret = register_mtddriver("/dev/rammtd", mtd, 0755, NULL); + if (ret < 0) + { + syslog(LOG_ERR, "ERROR: register_mtddriver failed: %d\n", ret); + } + +#ifdef CONFIG_FS_XIPFS + else + { + ret = nx_mount("/dev/rammtd", "/mnt/xipfs", "xipfs", 0, + "autoformat"); + if (ret < 0) + { + syslog(LOG_ERR, + "ERROR: Failed to mount xipfs at /mnt/xipfs: %d\n", + ret); + } + } +#endif + } + } +#endif + return ret; } diff --git a/boards/arm/mps/mps2-an521/configs/xipfs/defconfig b/boards/arm/mps/mps2-an521/configs/xipfs/defconfig new file mode 100644 index 0000000000000..b5fee089c6668 --- /dev/null +++ b/boards/arm/mps/mps2-an521/configs/xipfs/defconfig @@ -0,0 +1,78 @@ +# +# This file is autogenerated: PLEASE DO NOT EDIT IT. +# +# You can use "make menuconfig" to make any modifications to the installed .config file. +# You can then do "make savedefconfig" to generate a new defconfig file that includes your +# modifications. +# +# CONFIG_DEBUG_WARN is not set +CONFIG_ARCH="arm" +CONFIG_ARCH_BOARD="mps2-an521" +CONFIG_ARCH_BOARD_MPS2_AN521=y +CONFIG_ARCH_CHIP="mps" +CONFIG_ARCH_CHIP_MPS2_AN521=y +CONFIG_ARCH_CHIP_MPS=y +CONFIG_ARCH_INTERRUPTSTACK=2048 +CONFIG_ARCH_STACKDUMP=y +CONFIG_ARMV8M_SYSTICK=y +CONFIG_ARM_MPU=y +CONFIG_ARM_MPU_NREGIONS=1 +CONFIG_ARM_MPU_RESET=y +CONFIG_BUILTIN=y +CONFIG_CMSDK_UART0=y +CONFIG_CMSDK_UART0_BASE=0x50200000 +CONFIG_CMSDK_UART0_CLOCK=25000000 +CONFIG_CMSDK_UART0_OV_IRQ=63 +CONFIG_CMSDK_UART0_RX_IRQ=48 +CONFIG_CMSDK_UART0_SERIAL_CONSOLE=y +CONFIG_CMSDK_UART0_TX_IRQ=49 +CONFIG_CMSDK_UART=y +CONFIG_DEBUG_ASSERTIONS=y +CONFIG_DEBUG_FEATURES=y +CONFIG_DEBUG_SCHED=y +CONFIG_DEBUG_SCHED_ERROR=y +CONFIG_DEBUG_SYMBOLS=y +CONFIG_DEFAULT_TASK_STACKSIZE=4096 +CONFIG_EXPERIMENTAL=y +CONFIG_FS_PROCFS=y +CONFIG_FS_ROMFS=y +CONFIG_FS_TMPFS=y +CONFIG_FS_XIPFS=y +CONFIG_FS_XIPFS_FAULT_INJECT=y +CONFIG_HAVE_CXX=y +CONFIG_HAVE_CXXINITIALIZE=y +CONFIG_IDLETHREAD_STACKSIZE=4096 +CONFIG_INIT_ENTRYPOINT="nsh_main" +CONFIG_INTELHEX_BINARY=y +CONFIG_LIBC_MEMFD_ERROR=y +CONFIG_LIBC_STRERROR=y +CONFIG_LIBC_STRERROR_ERRNUM=y +CONFIG_MTD=y +CONFIG_NSH_BUILTIN_APPS=y +CONFIG_NSH_FILEIOSIZE=512 +CONFIG_NSH_READLINE=y +CONFIG_PREALLOC_TIMERS=4 +CONFIG_RAMLOG=y +CONFIG_RAMMTD=y +CONFIG_RAMMTD_FLASHSIM=y +CONFIG_RAM_SIZE=2097152 +CONFIG_RAM_START=0x38000000 +CONFIG_RAW_BINARY=y +CONFIG_READLINE_CMD_HISTORY=y +CONFIG_RR_INTERVAL=200 +CONFIG_SCHED_HPWORK=y +CONFIG_SCHED_HPWORKPRIORITY=192 +CONFIG_SPINLOCK=y +CONFIG_STACK_COLORATION=y +CONFIG_STANDARD_SERIAL=y +CONFIG_START_DAY=25 +CONFIG_START_MONTH=4 +CONFIG_START_YEAR=2023 +CONFIG_SYMTAB_ORDEREDBYNAME=y +CONFIG_SYSTEM_NSH=y +CONFIG_SYSTEM_SYSTEM=y +CONFIG_SYSTEM_XIPFS=y +CONFIG_TESTING_FS_XIPFS=y +CONFIG_TIMER=y +CONFIG_TIMER_ARCH=y +CONFIG_USEC_PER_TICK=1000 diff --git a/boards/arm/mps/mps2-an521/src/mps2_bringup.c b/boards/arm/mps/mps2-an521/src/mps2_bringup.c index e5bcb5a578087..d1a1483a3f3ce 100644 --- a/boards/arm/mps/mps2-an521/src/mps2_bringup.c +++ b/boards/arm/mps/mps2-an521/src/mps2_bringup.c @@ -31,10 +31,20 @@ #include +#ifdef CONFIG_RAMMTD +# include +# include +# include +#endif + /**************************************************************************** * Pre-processor Definitions ****************************************************************************/ +#ifdef CONFIG_RAMMTD +# define MPS2_RAMMTD_SIZE (256 * 1024) +#endif + /**************************************************************************** * Private Functions ****************************************************************************/ @@ -73,6 +83,54 @@ static int mps2_bringup(void) } #endif +#ifdef CONFIG_RAMMTD + /* A RAM backed MTD device standing in for memory mapped NOR. rammtd + * answers BIOC_XIPBASE with the base of its buffer, so xipfs layered on it + * can hand out real, directly executable pointers -- which is what lets a + * module be executed in place rather than copied. This gives the xipfs + * test suite a target with no real flash. + */ + + { + FAR uint8_t *ramstart = kmm_malloc(MPS2_RAMMTD_SIZE); + FAR struct mtd_dev_s *mtd; + + if (ramstart == NULL) + { + syslog(LOG_ERR, "ERROR: Failed to allocate RAM MTD\n"); + } + else if ((mtd = rammtd_initialize(ramstart, MPS2_RAMMTD_SIZE)) == NULL) + { + syslog(LOG_ERR, "ERROR: rammtd_initialize failed\n"); + kmm_free(ramstart); + } + else + { + mtd->ioctl(mtd, MTDIOC_BULKERASE, 0); + + ret = register_mtddriver("/dev/rammtd", mtd, 0755, NULL); + if (ret < 0) + { + syslog(LOG_ERR, "ERROR: register_mtddriver failed: %d\n", ret); + } + +#ifdef CONFIG_FS_XIPFS + else + { + ret = nx_mount("/dev/rammtd", "/mnt/xipfs", "xipfs", 0, + "autoformat"); + if (ret < 0) + { + syslog(LOG_ERR, + "ERROR: Failed to mount xipfs at /mnt/xipfs: %d\n", + ret); + } + } +#endif + } + } +#endif + return ret; } diff --git a/boards/arm/rp23xx/common/src/rp23xx_common_bringup.c b/boards/arm/rp23xx/common/src/rp23xx_common_bringup.c index 33645f1649090..0d16de61a5376 100644 --- a/boards/arm/rp23xx/common/src/rp23xx_common_bringup.c +++ b/boards/arm/rp23xx/common/src/rp23xx_common_bringup.c @@ -434,6 +434,18 @@ int rp23xx_common_bringup(void) { serr("ERROR: Failed to register /dev/rpflash: %d\n", ret); } +#ifdef CONFIG_FS_XIPFS + else + { + ret = nx_mount("/dev/rpflash", "/mnt/xipfs", "xipfs", 0, + "autoformat"); + if (ret < 0) + { + serr("ERROR: Failed to mount xipfs at /mnt/xipfs: %d\n", + ret); + } + } +#endif } } #endif diff --git a/boards/arm/rp23xx/pimoroni-pico-2-plus/configs/xipfs/defconfig b/boards/arm/rp23xx/pimoroni-pico-2-plus/configs/xipfs/defconfig new file mode 100644 index 0000000000000..d1876a5ba4700 --- /dev/null +++ b/boards/arm/rp23xx/pimoroni-pico-2-plus/configs/xipfs/defconfig @@ -0,0 +1,54 @@ +# +# This file is autogenerated: PLEASE DO NOT EDIT IT. +# +# You can use "make menuconfig" to make any modifications to the installed .config file. +# You can then do "make savedefconfig" to generate a new defconfig file that includes your +# modifications. +# +# CONFIG_NSH_ARGCAT is not set +# CONFIG_NSH_CMDOPT_HEXDUMP is not set +# CONFIG_NSH_DISABLE_DATE is not set +# CONFIG_NSH_DISABLE_LOSMART is not set +# CONFIG_STANDARD_SERIAL is not set +CONFIG_ARCH="arm" +CONFIG_ARCH_BOARD="pimoroni-pico-2-plus" +CONFIG_ARCH_BOARD_COMMON=y +CONFIG_ARCH_BOARD_PIMORONI_PICO_2_PLUS=y +CONFIG_ARCH_CHIP="rp23xx" +CONFIG_ARCH_CHIP_RP23XX=y +CONFIG_ARCH_RAMVECTORS=y +CONFIG_ARCH_STACKDUMP=y +CONFIG_BOARDCTL_RESET=y +CONFIG_BOARD_LOOPSPERMSEC=10450 +CONFIG_BUILTIN=y +CONFIG_DEBUG_FULLOPT=y +CONFIG_DEBUG_SYMBOLS=y +CONFIG_DISABLE_POSIX_TIMERS=y +CONFIG_EXAMPLES_HELLO=y +CONFIG_EXAMPLES_NXFLATXIP=y +CONFIG_FS_PROCFS=y +CONFIG_FS_PROCFS_REGISTER=y +CONFIG_FS_XIPFS=y +CONFIG_FS_XIPFS_FAULT_INJECT=y +CONFIG_INIT_ENTRYPOINT="nsh_main" +CONFIG_LIBC_EXECFUNCS=y +CONFIG_MTD=y +CONFIG_NFILE_DESCRIPTORS_PER_BLOCK=6 +CONFIG_NSH_BUILTIN_APPS=y +CONFIG_NSH_READLINE=y +CONFIG_NXFLAT=y +CONFIG_RAM_SIZE=532480 +CONFIG_RAM_START=0x20000000 +CONFIG_READLINE_CMD_HISTORY=y +CONFIG_RP23XX_FLASH_MTD=y +CONFIG_RR_INTERVAL=200 +CONFIG_SCHED_WAITPID=y +CONFIG_START_DAY=9 +CONFIG_START_MONTH=2 +CONFIG_START_YEAR=2021 +CONFIG_SYSLOG_CONSOLE=y +CONFIG_SYSTEM_NSH=y +CONFIG_SYSTEM_XIPFS=y +CONFIG_TESTING_FS_XIPFS=y +CONFIG_TESTING_FS_XIPFS_MTD="/dev/rpflash" +CONFIG_UART0_SERIAL_CONSOLE=y diff --git a/boards/arm/rp23xx/pimoroni-pico-2-plus/scripts/Make.defs b/boards/arm/rp23xx/pimoroni-pico-2-plus/scripts/Make.defs index 8772d6efc31b3..fa4bfae9c517b 100644 --- a/boards/arm/rp23xx/pimoroni-pico-2-plus/scripts/Make.defs +++ b/boards/arm/rp23xx/pimoroni-pico-2-plus/scripts/Make.defs @@ -38,6 +38,9 @@ CXXFLAGS := $(ARCHCXXFLAGS) $(ARCHOPTIMIZATION) $(ARCHCPUFLAGS) $(ARCHXXINCLUDES CPPFLAGS := $(ARCHINCLUDES) $(ARCHDEFINES) $(EXTRAFLAGS) AFLAGS := $(CFLAGS) -D__ASSEMBLY__ +MKNXFLAT = mknxflat +LDNXFLAT = ldnxflat + NXFLATLDFLAGS1 = -r -d -warn-common NXFLATLDFLAGS2 = $(NXFLATLDFLAGS1) -T$(TOPDIR)/binfmt/libnxflat/gnu-nxflat-pcrel.ld -no-check-sections LDNXFLATFLAGS = -e main -s 2048 diff --git a/boards/arm/rp23xx/raspberrypi-pico-2/scripts/Make.defs b/boards/arm/rp23xx/raspberrypi-pico-2/scripts/Make.defs index 3a74f74f1001f..c50a89026bca2 100644 --- a/boards/arm/rp23xx/raspberrypi-pico-2/scripts/Make.defs +++ b/boards/arm/rp23xx/raspberrypi-pico-2/scripts/Make.defs @@ -38,6 +38,9 @@ CXXFLAGS := $(ARCHCXXFLAGS) $(ARCHOPTIMIZATION) $(ARCHCPUFLAGS) $(ARCHXXINCLUDES CPPFLAGS := $(ARCHINCLUDES) $(ARCHDEFINES) $(EXTRAFLAGS) AFLAGS := $(CFLAGS) -D__ASSEMBLY__ +MKNXFLAT = mknxflat +LDNXFLAT = ldnxflat + NXFLATLDFLAGS1 = -r -d -warn-common NXFLATLDFLAGS2 = $(NXFLATLDFLAGS1) -T$(TOPDIR)/binfmt/libnxflat/gnu-nxflat-pcrel.ld -no-check-sections LDNXFLATFLAGS = -e main -s 2048 diff --git a/boards/arm/rp23xx/xiao-rp2350/scripts/Make.defs b/boards/arm/rp23xx/xiao-rp2350/scripts/Make.defs index e14338d6cc868..ccf52abfac7ac 100644 --- a/boards/arm/rp23xx/xiao-rp2350/scripts/Make.defs +++ b/boards/arm/rp23xx/xiao-rp2350/scripts/Make.defs @@ -38,6 +38,9 @@ CXXFLAGS := $(ARCHCXXFLAGS) $(ARCHOPTIMIZATION) $(ARCHCPUFLAGS) $(ARCHXXINCLUDES CPPFLAGS := $(ARCHINCLUDES) $(ARCHDEFINES) $(EXTRAFLAGS) AFLAGS := $(CFLAGS) -D__ASSEMBLY__ +MKNXFLAT = mknxflat +LDNXFLAT = ldnxflat + NXFLATLDFLAGS1 = -r -d -warn-common NXFLATLDFLAGS2 = $(NXFLATLDFLAGS1) -T$(TOPDIR)/binfmt/libnxflat/gnu-nxflat-pcrel.ld -no-check-sections LDNXFLATFLAGS = -e main -s 2048 diff --git a/boards/sim/sim/sim/configs/xipfs/defconfig b/boards/sim/sim/sim/configs/xipfs/defconfig new file mode 100644 index 0000000000000..53c3eb0119289 --- /dev/null +++ b/boards/sim/sim/sim/configs/xipfs/defconfig @@ -0,0 +1,33 @@ +# +# This file is autogenerated: PLEASE DO NOT EDIT IT. +# +# You can use "make menuconfig" to make any modifications to the installed .config file. +# You can then do "make savedefconfig" to generate a new defconfig file that includes your +# modifications. +# +CONFIG_ARCH="sim" +CONFIG_ARCH_BOARD="sim" +CONFIG_ARCH_BOARD_SIM=y +CONFIG_ARCH_CHIP="sim" +CONFIG_ARCH_SIM=y +CONFIG_BOARDCTL_POWEROFF=y +CONFIG_BOARD_LOOPSPERMSEC=0 +CONFIG_BUILTIN=y +CONFIG_DEBUG_SYMBOLS=y +CONFIG_FS_PROCFS=y +CONFIG_FS_XIPFS=y +CONFIG_FS_XIPFS_FAULT_INJECT=y +CONFIG_IDLETHREAD_STACKSIZE=8192 +CONFIG_INIT_ENTRYPOINT="nsh_main" +CONFIG_LIBC_STRERROR=y +CONFIG_MTD=y +CONFIG_NSH_BUILTIN_APPS=y +CONFIG_NSH_READLINE=y +CONFIG_RAMMTD=y +CONFIG_RAMMTD_FLASHSIM=y +CONFIG_SCHED_WAITPID=y +CONFIG_START_MONTH=6 +CONFIG_START_YEAR=2008 +CONFIG_SYSTEM_NSH=y +CONFIG_SYSTEM_XIPFS=y +CONFIG_TESTING_FS_XIPFS=y diff --git a/boards/sim/sim/sim/src/sim_bringup.c b/boards/sim/sim/sim/src/sim_bringup.c index c839d22333220..3064b1e1a39f4 100644 --- a/boards/sim/sim/sim/src/sim_bringup.c +++ b/boards/sim/sim/sim/src/sim_bringup.c @@ -212,6 +212,22 @@ int sim_bringup(void) smart_initialize(0, mtd, NULL); +#elif defined(CONFIG_FS_XIPFS) + /* Mount the XIPFS file system. rammtd answers BIOC_XIPBASE with + * the base of its RAM buffer, which makes it a usable stand-in + * for memory mapped NOR: extents are directly addressable, so + * the XIP mmap path can be exercised end to end. + */ + + ret = nx_mount("/dev/rammtd", "/mnt/xipfs", "xipfs", 0, + "autoformat"); + if (ret < 0) + { + syslog(LOG_ERR, + "ERROR: Failed to mount XIPFS at /mnt/xipfs: %d\n", + ret); + } + #elif defined(CONFIG_FS_SPIFFS) /* Mount the SPIFFS file system */ diff --git a/fs/Kconfig b/fs/Kconfig index 3631734c3e5ed..db8e7ded3100c 100644 --- a/fs/Kconfig +++ b/fs/Kconfig @@ -181,4 +181,5 @@ source "fs/hostfs/Kconfig" source "fs/rpmsgfs/Kconfig" source "fs/zipfs/Kconfig" source "fs/mnemofs/Kconfig" +source "fs/xipfs/Kconfig" source "fs/v9fs/Kconfig" diff --git a/fs/Makefile b/fs/Makefile index 342e0ea12e58d..264080b76af46 100644 --- a/fs/Makefile +++ b/fs/Makefile @@ -65,6 +65,7 @@ include littlefs/Make.defs include rpmsgfs/Make.defs include zipfs/Make.defs include mnemofs/Make.defs +include xipfs/Make.defs include v9fs/Make.defs endif diff --git a/fs/mount/fs_gettype.c b/fs/mount/fs_gettype.c index ff01f781830c3..1b97aca7d32d8 100644 --- a/fs/mount/fs_gettype.c +++ b/fs/mount/fs_gettype.c @@ -149,6 +149,12 @@ FAR const char *fs_gettype(FAR struct statfs *statbuf) break; #endif +#ifdef CONFIG_FS_XIPFS + case XIPFS_MAGIC: + fstype = "xipfs"; + break; +#endif + #ifdef CONFIG_FS_ZIPFS case ZIPFS_MAGIC: fstype = "zipfs"; diff --git a/fs/mount/fs_mount.c b/fs/mount/fs_mount.c index 67edf7b6b3a87..0f4758c43a916 100644 --- a/fs/mount/fs_mount.c +++ b/fs/mount/fs_mount.c @@ -61,7 +61,8 @@ /* These file systems require MTD drivers */ #if (defined(CONFIG_FS_SPIFFS) || defined(CONFIG_FS_LITTLEFS) || \ - defined(CONFIG_FS_MNEMOFS)) && defined(CONFIG_MTD) + defined(CONFIG_FS_MNEMOFS) || defined(CONFIG_FS_XIPFS)) && \ + defined(CONFIG_MTD) # define MDFS_SUPPORT 1 #endif @@ -136,6 +137,9 @@ extern const struct mountpt_operations g_littlefs_operations; #ifdef CONFIG_FS_MNEMOFS extern const struct mountpt_operations g_mnemofs_operations; #endif +#ifdef CONFIG_FS_XIPFS +extern const struct mountpt_operations g_xipfs_operations; +#endif static const struct fsmap_t g_mdfsmap[] = { @@ -147,6 +151,9 @@ static const struct fsmap_t g_mdfsmap[] = #endif #ifdef CONFIG_FS_MNEMOFS { "mnemofs", &g_mnemofs_operations }, +#endif +#ifdef CONFIG_FS_XIPFS + { "xipfs", &g_xipfs_operations }, #endif { NULL, NULL }, }; diff --git a/fs/xipfs/CMakeLists.txt b/fs/xipfs/CMakeLists.txt new file mode 100644 index 0000000000000..7bcae69132248 --- /dev/null +++ b/fs/xipfs/CMakeLists.txt @@ -0,0 +1,26 @@ +# ############################################################################## +# fs/xipfs/CMakeLists.txt +# +# SPDX-License-Identifier: Apache-2.0 +# +# Licensed to the Apache Software Foundation (ASF) under one or more contributor +# license agreements. See the NOTICE file distributed with this work for +# additional information regarding copyright ownership. The ASF licenses this +# file to you under the Apache License, Version 2.0 (the "License"); you may not +# use this file except in compliance with the License. You may obtain a copy of +# the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations under +# the License. +# +# ############################################################################## + +if(CONFIG_FS_XIPFS) + target_sources(fs PRIVATE xipfs_alloc.c xipfs_defrag.c xipfs_flash.c + xipfs_meta.c xipfs_mmap.c xipfs_vfs.c) +endif() diff --git a/fs/xipfs/Kconfig b/fs/xipfs/Kconfig new file mode 100644 index 0000000000000..fbdad9240a0ce --- /dev/null +++ b/fs/xipfs/Kconfig @@ -0,0 +1,42 @@ +# +# For a description of the syntax of this configuration file, +# see the file kconfig-language.txt in the NuttX tools repository. +# + +config FS_XIPFS + bool "XIPFS contiguous execute-in-place file system" + default n + depends on !DISABLE_MOUNTPOINT && MTD + ---help--- + Enable support for XIPFS, a file system that stores each file as a + single physically contiguous, erase-block aligned extent on memory + mapped flash. + + Because files are contiguous and the media is directly addressable, + XIPFS can return a real pointer into flash from mmap(), so a module + can be executed in place without its read-only text and rodata ever + being copied into RAM. This requires the underlying MTD driver to + implement the BIOC_XIPBASE ioctl. + + Files are write once: a file is created at a known size, written + sequentially, closed, and is immutable thereafter until it is + deleted. Random writes, appends and growth are not supported. + + Directories are supported. They are records in the same metadata + generation the files are, not objects in the data region, so an + empty one costs one entry out of the volume's fixed supply and no + flash blocks at all. + +if FS_XIPFS + +config FS_XIPFS_FAULT_INJECT + bool "XIPFS flash fault injection" + default n + ---help--- + Add a countdown to the XIPFS flash write and erase paths so that a + test can fail an arbitrary flash operation and then remount, which + models a power loss at that exact point. + + This is a test-only facility. Do not enable it in production. + +endif # FS_XIPFS diff --git a/fs/xipfs/Make.defs b/fs/xipfs/Make.defs new file mode 100644 index 0000000000000..c1bc18c6207ce --- /dev/null +++ b/fs/xipfs/Make.defs @@ -0,0 +1,38 @@ +############################################################################ +# fs/xipfs/Make.defs +# +# SPDX-License-Identifier: Apache-2.0 +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. The +# ASF licenses this file to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance with the +# License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. +# +############################################################################ + +ifeq ($(CONFIG_FS_XIPFS),y) + +# Add the xipfs C files to the build + +CSRCS += xipfs_alloc.c +CSRCS += xipfs_defrag.c +CSRCS += xipfs_flash.c +CSRCS += xipfs_meta.c +CSRCS += xipfs_mmap.c +CSRCS += xipfs_vfs.c + +# Add the xipfs directory to the build + +DEPPATH += --dep-path xipfs +VPATH += :xipfs +endif diff --git a/fs/xipfs/xipfs.h b/fs/xipfs/xipfs.h new file mode 100644 index 0000000000000..4e93738ffe107 --- /dev/null +++ b/fs/xipfs/xipfs.h @@ -0,0 +1,392 @@ +/**************************************************************************** + * fs/xipfs/xipfs.h + * + * SPDX-License-Identifier: Apache-2.0 + * + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. The + * ASF licenses this file to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance with the + * License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + * + ****************************************************************************/ + +#ifndef __FS_XIPFS_XIPFS_H +#define __FS_XIPFS_XIPFS_H + +/**************************************************************************** + * Included Files + ****************************************************************************/ + +#include + +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include + +/**************************************************************************** + * Pre-processor Definitions + ****************************************************************************/ + +/* On-media identification */ + +#define XIPFS_SBLK_MAGIC XIPFS_MAGIC /* "XIPF", see sys/statfs.h */ +#define XIPFS_GEN_MAGIC 0x5847454e /* "XGEN" */ +#define XIPFS_VERSION 2 + +/* Volume geometry. + * + * Block 0 and block 1 hold the two superblock copies. The next + * XIPFS_META_NBLOCKS blocks form the metadata ring. Everything after that + * is the data region, allocated in whole erase blocks. + */ + +#define XIPFS_SBLK_A 0 +#define XIPFS_SBLK_B 1 +#define XIPFS_META_START 2 +#define XIPFS_META_NBLOCKS 4 +#define XIPFS_DATA_START (XIPFS_META_START + XIPFS_META_NBLOCKS) + +/* The smallest volume that can hold the superblocks, the metadata ring and + * at least one data block. + */ + +#define XIPFS_MIN_BLOCKS (XIPFS_DATA_START + 1) + +/* Directory entry size */ + +#define XIPFS_DIRENT_SIZE 64 + +/* Dirent flags */ + +#define XIPFS_DIRENT_DIR (1 << 0) /* The entry is a directory */ + +/* The root directory is implicit: it owns id 0 and has no record of its + * own, so it always exists and cannot be removed. + */ + +#define XIPFS_ROOT_ID 0 + +/**************************************************************************** + * Public Types + ****************************************************************************/ + +/* On-media superblock. Written once by mkfs into both block 0 and block 1. + * The CRC covers every byte preceding the crc field. + */ + +begin_packed_struct struct xipfs_sblk_s +{ + uint32_t magic; + uint32_t version; + uint32_t erasesize; + uint32_t blocksize; + uint32_t nblocks; /* Total erase blocks in the volume */ + uint32_t meta_start; /* First block of the metadata ring */ + uint32_t meta_nblocks; /* Length of the metadata ring */ + uint32_t data_start; /* First block of the data region */ + uint32_t data_nblocks; /* Length of the data region */ + uint32_t crc; +} end_packed_struct; + +/* On-media directory entry. One per file and one per directory, stored in + * the body of a metadata generation. + * + * Directories are records here and nowhere else. They are not objects in + * the data region, which is what keeps the power-safety story intact: mkdir + * and rmdir add or remove a record and commit one generation, exactly as + * create and unlink do, so there is never a multi-object update to journal + * or an orphan to collect at mount. + * + * 'name' is one path component, not a path. Depth comes from 'parent', so + * XIPFS_NAME_MAX bounds a component, which is what statfs reports it as. + * + * Padded to XIPFS_DIRENT_SIZE so that a whole number of dirents fits in a + * read/write block, which keeps the body write loop aligned without any + * straddling entries. + */ + +begin_packed_struct struct xipfs_dirent_s +{ + char name[XIPFS_NAME_MAX + 1]; /* 32 bytes, one component */ + uint32_t size; /* File size in bytes; 0 for a dir */ + uint32_t start_block; /* Extent block; 0 for a directory */ + uint32_t nblocks; /* Extent length; 0 for a directory */ + uint32_t flags; /* XIPFS_DIRENT_DIR */ + uint16_t id; /* Identity, unique, never 0 */ + uint16_t parent; /* Containing directory's id */ + uint8_t reserved[12]; /* Pad to XIPFS_DIRENT_SIZE */ +} end_packed_struct; + +/* The padding above is maintained by hand, and the body read loop indexes + * the generation at a fixed XIPFS_DIRENT_SIZE stride. A field added without + * taking it out of 'reserved' would therefore not fail loudly: it would + * reparse every existing volume at the wrong offset. + */ + +static_assert(sizeof(struct xipfs_dirent_s) == XIPFS_DIRENT_SIZE, + "xipfs dirent layout mismatch"); + +/* On-media metadata generation header. + * + * This lives in the first read/write block of a metadata ring block; the + * dirent array follows in the blocks after it. The body is written first + * and the header last, so the single header write IS the commit point: a + * torn body leaves no valid header, and a torn header fails its own CRC. + * Either way mount falls back to the previous generation. + */ + +begin_packed_struct struct xipfs_genhdr_s +{ + uint32_t magic; + uint32_t seq; /* Monotonic generation number */ + uint32_t nentries; /* Number of dirents in body */ + uint32_t crc; /* CRC over header + body */ +} end_packed_struct; + +/* In-RAM directory entry. One per file and one per directory; a directory + * carries no extent, so its start_block, nblocks and size are all zero and + * everything that accounts for blocks has to skip it. + * + * For a file this is also the object the pin count lives on -- deliberately + * NOT the fd and NOT the open file struct, so that N mappings of one module + * produce a pin count of N and the extent only becomes movable when the + * last one goes away. + */ + +struct xipfs_extent_s +{ + FAR struct xipfs_extent_s *flink; + + /* Back pointer to the owning mount. The task teardown unmap path can + * only reach the extent, and must not consult the current task's group, + * so the extent has to be able to name its own filesystem. + */ + + FAR struct xipfs_mount_s *fs; + + /* One path component, not a path; depth comes from 'parent' */ + + char name[XIPFS_NAME_MAX + 1]; + + uint16_t id; /* Identity, unique, never 0 */ + uint16_t parent; /* Containing directory; XIPFS_ROOT_ID */ + uint32_t size; + uint32_t start_block; + uint32_t nblocks; + uint32_t pincount; /* Mappings that alias flash (true XIP) */ + uint32_t openrefs; /* Open file descriptors */ + bool isdir; /* A directory record, with no extent */ + bool writing; /* Create-time write still in progress */ + bool unlinked; /* Detached from directory, awaiting free */ +}; + +/* In-RAM mount state */ + +struct xipfs_mount_s +{ + FAR struct inode *driver; + FAR struct mtd_dev_s *mtd; + struct mtd_geometry_s geo; + + /* Direct address of the media, from BIOC_XIPBASE. NULL when the + * underlying driver cannot expose one, in which case XIP mappings are + * impossible and strict requests fail with -ENXIO. + */ + + FAR uint8_t *xipbase; + + uint32_t meta_start; + uint32_t meta_nblocks; + uint32_t meta_slot; /* Ring slot holding the live generation */ + uint32_t meta_seq; /* Sequence number of that generation */ + uint32_t data_start; + uint32_t data_nblocks; + + FAR unsigned long *bitmap; /* Free bitmap over the data region */ + size_t bitmapsize; + + /* Staging buffer for defrag relocation. Reserved once at bind time so + * that a relocation can never fail part way through because a transient + * allocation did not succeed. + */ + + FAR uint8_t *stage; + + /* Scratch for building and verifying a metadata generation. Separate + * from 'stage' because defrag commits a generation while its relocation + * copy is in flight. + */ + + FAR uint8_t *metabuf; + + uint32_t entries_per_blk; /* Dirents in one read/write block */ + uint32_t max_entries; /* Dirents that fit in one ring block */ + + FAR struct xipfs_extent_s *extents; + uint32_t nextents; + + /* Single guard over extent metadata, the allocator and the pin counts. + * The map path holds it across "range check -> resolve -> take pin" and + * defrag holds it across "observe pincount == 0 -> relocate", so a new + * mapping cannot slip in between the compactor's check and its move. + */ + + rmutex_t lock; + + bool unmounted; + +#ifdef CONFIG_FS_XIPFS_FAULT_INJECT + bool media_dead; + int32_t fault_countdown; /* Negative disables injection */ + uint8_t fault_mode; /* XIPFS_FAULT_CLEAN or XIPFS_FAULT_TORN */ +#endif +}; + +/* Per open file state, hung off filep->f_priv */ + +struct xipfs_file_s +{ + FAR struct xipfs_extent_s *ext; + + /* Create-time write state. NOR programs whole pages, so bytes are + * accumulated here and flushed a page at a time. + */ + + FAR uint8_t *wrbuf; + uint32_t wrpos; /* Bytes buffered but not yet programmed */ + uint32_t written; /* Bytes already programmed to flash */ + uint32_t erased; /* Blocks of the extent already erased */ + bool writing; + bool greedy; /* Extent was over-reserved, trim at close */ +}; + +/* Directory enumeration state */ + +/* An open directory: which directory it is, and how far the walk has got */ + +struct xipfs_dir_s +{ + struct fs_dirent_s base; + uint16_t dirid; + uint32_t index; +}; + +/**************************************************************************** + * Public Data + ****************************************************************************/ + +#undef EXTERN +#if defined(__cplusplus) +#define EXTERN extern "C" +extern "C" +{ +#else +#define EXTERN extern +#endif + +EXTERN const struct mountpt_operations g_xipfs_operations; + +/**************************************************************************** + * Public Function Prototypes + ****************************************************************************/ + +/* xipfs_flash.c ************************************************************/ + +int xipfs_flash_probe(FAR struct xipfs_mount_s *fs); +int xipfs_flash_read(FAR struct xipfs_mount_s *fs, uint32_t block, + uint32_t offset, FAR void *buffer, size_t nbytes); +int xipfs_flash_write(FAR struct xipfs_mount_s *fs, uint32_t block, + uint32_t offset, FAR const void *buffer, + size_t nbytes); +int xipfs_flash_erase(FAR struct xipfs_mount_s *fs, uint32_t block, + uint32_t nblocks); + +/* Direct address of a block, or NULL when the media is not memory mapped */ + +FAR uint8_t *xipfs_flash_addr(FAR struct xipfs_mount_s *fs, uint32_t block); + +/* xipfs_meta.c *************************************************************/ + +int xipfs_meta_format(FAR struct xipfs_mount_s *fs); +int xipfs_meta_mount(FAR struct xipfs_mount_s *fs); +int xipfs_meta_commit(FAR struct xipfs_mount_s *fs); +void xipfs_meta_freeextents(FAR struct xipfs_mount_s *fs); + +FAR struct xipfs_extent_s *xipfs_meta_find(FAR struct xipfs_mount_s *fs, + uint16_t parent, + FAR const char *name, + size_t namelen); +FAR struct xipfs_extent_s *xipfs_meta_add(FAR struct xipfs_mount_s *fs, + uint16_t parent, + FAR const char *name, + size_t namelen, bool isdir, + uint32_t start_block, + uint32_t nblocks, uint32_t size); +bool xipfs_meta_haschildren(FAR struct xipfs_mount_s *fs, uint16_t dirid); +void xipfs_meta_path(FAR struct xipfs_mount_s *fs, + FAR struct xipfs_extent_s *ext, + FAR char *buf, size_t buflen); +void xipfs_meta_detach(FAR struct xipfs_mount_s *fs, + FAR struct xipfs_extent_s *ext); + +/* xipfs_alloc.c ************************************************************/ + +int xipfs_alloc_init(FAR struct xipfs_mount_s *fs); +void xipfs_alloc_rebuild(FAR struct xipfs_mount_s *fs); +int xipfs_alloc(FAR struct xipfs_mount_s *fs, uint32_t nblocks, + FAR uint32_t *start_block); +void xipfs_free(FAR struct xipfs_mount_s *fs, uint32_t start_block, + uint32_t nblocks); +uint32_t xipfs_alloc_largestrun(FAR struct xipfs_mount_s *fs); +uint32_t xipfs_alloc_freeblocks(FAR struct xipfs_mount_s *fs); + +/* xipfs_mmap.c *************************************************************/ + +int xipfs_mmap(FAR struct file *filep, FAR struct mm_map_entry_s *map); + +/* xipfs_defrag.c ***********************************************************/ + +int xipfs_defrag(FAR struct xipfs_mount_s *fs, uint32_t max_ms, + FAR struct xipfs_defrag_result_s *result); +int xipfs_listpinned(FAR struct xipfs_mount_s *fs, + FAR struct xipfs_pinned_entry_s *entries, + size_t nentries, FAR size_t *count); + +/**************************************************************************** + * Inline Functions + ****************************************************************************/ + +static inline int xipfs_lock(FAR struct xipfs_mount_s *fs) +{ + return nxrmutex_lock(&fs->lock); +} + +static inline void xipfs_unlock(FAR struct xipfs_mount_s *fs) +{ + nxrmutex_unlock(&fs->lock); +} + +#undef EXTERN +#if defined(__cplusplus) +} +#endif + +#endif /* __FS_XIPFS_XIPFS_H */ diff --git a/fs/xipfs/xipfs_alloc.c b/fs/xipfs/xipfs_alloc.c new file mode 100644 index 0000000000000..108bf466da6a7 --- /dev/null +++ b/fs/xipfs/xipfs_alloc.c @@ -0,0 +1,272 @@ +/**************************************************************************** + * fs/xipfs/xipfs_alloc.c + * + * SPDX-License-Identifier: Apache-2.0 + * + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. The + * ASF licenses this file to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance with the + * License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + * + ****************************************************************************/ + +/**************************************************************************** + * Included Files + * + * Allocation is in whole erase blocks and always contiguous. Because a + * file's full size is known when it is created, the exact extent is + * reserved up front and never grows, so a file can never become internally + * fragmented. The only source of fragmentation is the holes left behind by + * deletes, which is what the defragmenter coalesces. + * + * The free map is a bitmap over the data region, derived entirely from the + * committed directory: it is state that can always be rebuilt, never state + * that has to be recovered. + * + ****************************************************************************/ + +#include + +#include +#include +#include +#include +#include + +#include +#include + +#include "xipfs.h" + +/**************************************************************************** + * Public Functions + ****************************************************************************/ + +/**************************************************************************** + * Name: xipfs_alloc_init + * + * Description: + * Allocate the free bitmap. Sized for the data region only. + * + ****************************************************************************/ + +int xipfs_alloc_init(FAR struct xipfs_mount_s *fs) +{ + fs->bitmapsize = BITS_TO_LONGS(fs->data_nblocks) * sizeof(unsigned long); + fs->bitmap = kmm_zalloc(fs->bitmapsize); + + if (fs->bitmap == NULL) + { + return -ENOMEM; + } + + return OK; +} + +/**************************************************************************** + * Name: xipfs_alloc_rebuild + * + * Description: + * Recompute the free bitmap from the extent list. Called after mount and + * after any change that moves extents. + * + ****************************************************************************/ + +void xipfs_alloc_rebuild(FAR struct xipfs_mount_s *fs) +{ + FAR struct xipfs_extent_s *ext; + uint32_t index; + + memset(fs->bitmap, 0, fs->bitmapsize); + + for (ext = fs->extents; ext != NULL; ext = ext->flink) + { + /* A directory record owns no blocks, and its start_block lies below + * the data region, so the index must not be formed for one. + */ + + if (ext->nblocks == 0) + { + continue; + } + + index = ext->start_block - fs->data_start; + + DEBUGASSERT(index + ext->nblocks <= fs->data_nblocks); + bitmap_set(fs->bitmap, index, ext->nblocks); + } +} + +/**************************************************************************** + * Name: xipfs_alloc + * + * Description: + * Reserve a contiguous run of 'nblocks' erase blocks using best fit. + * + * This deliberately does NOT invoke the defragmenter on failure. A + * caller that hits -ENOSPC gets to decide whether compacting is worth it + * right now, and the defragmenter therefore always runs from a known + * clean state rather than from inside a half-finished allocation. + * + * Returned Value: + * OK with *start_block set, or -ENOSPC if no single contiguous run of + * the requested size exists. + * + ****************************************************************************/ + +int xipfs_alloc(FAR struct xipfs_mount_s *fs, uint32_t nblocks, + FAR uint32_t *start_block) +{ + uint32_t best_start = 0; + uint32_t best_len = 0; + uint32_t run_start = 0; + uint32_t run_len = 0; + uint32_t i; + int ret; + + if (nblocks == 0) + { + return -EINVAL; + } + + for (i = 0; i <= fs->data_nblocks; i++) + { + bool used = (i == fs->data_nblocks) || test_bit(i, fs->bitmap) != 0; + + if (!used) + { + if (run_len == 0) + { + run_start = i; + } + + run_len++; + continue; + } + + /* End of a free run. Keep it if it is the tightest fit so far. */ + + if (run_len >= nblocks && (best_len == 0 || run_len < best_len)) + { + best_len = run_len; + best_start = run_start; + } + + run_len = 0; + } + + if (best_len == 0) + { + return -ENOSPC; + } + + /* The scan already proved this run free under the mount lock, so -EBUSY + * is not a full volume: the bitmap and the extent list disagree. + */ + + ret = bitmap_allocate_region(fs->bitmap, best_start, nblocks); + if (ret < 0) + { + ferr("ERROR: Free run %" PRIu32 "+%" PRIu32 " is already allocated\n", + best_start, nblocks); + return ret; + } + + *start_block = fs->data_start + best_start; + return OK; +} + +/**************************************************************************** + * Name: xipfs_free + * + * Description: + * Release a previously allocated run. Coalescing is implicit: the run + * simply becomes part of whatever free neighbours surround it. + * + ****************************************************************************/ + +void xipfs_free(FAR struct xipfs_mount_s *fs, uint32_t start_block, + uint32_t nblocks) +{ + /* Freeing nothing is a no-op. This is not a corner case to tolerate but + * the correct answer for an extent whose reservation never landed: a + * create cut short before xipfs_reserve succeeds leaves start_block at 0, + * which is below the data region, and the cleanup path still hands that + * extent here. A live extent always has nblocks > 0, so guarding on the + * length is exactly equivalent to guarding on "was this ever reserved". + */ + + if (nblocks == 0) + { + return; + } + + DEBUGASSERT(start_block >= fs->data_start); + DEBUGASSERT(start_block + nblocks <= fs->data_start + fs->data_nblocks); + + bitmap_clear(fs->bitmap, start_block - fs->data_start, nblocks); +} + +/**************************************************************************** + * Name: xipfs_alloc_largestrun + * + * Description: + * Length in blocks of the largest contiguous free run. This is what + * tells a caller whose allocation just failed whether a retry after + * defragmenting can possibly succeed. + * + ****************************************************************************/ + +uint32_t xipfs_alloc_largestrun(FAR struct xipfs_mount_s *fs) +{ + uint32_t best = 0; + uint32_t run = 0; + uint32_t i; + + for (i = 0; i < fs->data_nblocks; i++) + { + if (test_bit(i, fs->bitmap)) + { + run = 0; + continue; + } + + run++; + if (run > best) + { + best = run; + } + } + + return best; +} + +/**************************************************************************** + * Name: xipfs_alloc_freeblocks + ****************************************************************************/ + +uint32_t xipfs_alloc_freeblocks(FAR struct xipfs_mount_s *fs) +{ + uint32_t count = 0; + uint32_t i; + + for (i = 0; i < fs->data_nblocks; i++) + { + if (!test_bit(i, fs->bitmap)) + { + count++; + } + } + + return count; +} diff --git a/fs/xipfs/xipfs_defrag.c b/fs/xipfs/xipfs_defrag.c new file mode 100644 index 0000000000000..586a95c5455cb --- /dev/null +++ b/fs/xipfs/xipfs_defrag.c @@ -0,0 +1,468 @@ +/**************************************************************************** + * fs/xipfs/xipfs_defrag.c + * + * SPDX-License-Identifier: Apache-2.0 + * + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. The + * ASF licenses this file to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance with the + * License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + * + ****************************************************************************/ + +/**************************************************************************** + * Included Files + * + * Because files are written once at a known size and never grow, a file can + * never fragment internally. The only thing that fragments is free space, + * via the holes deletes leave behind. So this is not a continuous + * compactor: it is an occasional coalescing pass, invoked explicitly. + * + * It is structured as a loop of atomic relocations. Each relocation copies + * one closed, unpinned extent into free space, commits a new metadata + * generation naming the new location, and only then erases the blocks it + * vacated. Every iteration boundary is therefore a fully consistent + * layout, merely a less compact one, and every way of stopping -- a time + * budget, a pinned extent in the way, a media error, a power cut -- lands + * on one of those boundaries. Nothing is ever left half moved. + * + * The pass is best effort by design. It reports what it achieved and why + * it stopped rather than succeeding or failing, because the caller asked + * for it after an allocation did not fit and what it actually needs to know + * is whether retrying that allocation can now work. + * + ****************************************************************************/ + +#include + +#include +#include +#include +#include +#include + +#include + +#include "xipfs.h" + +/**************************************************************************** + * Private Functions + ****************************************************************************/ + +/**************************************************************************** + * Name: xipfs_defrag_findlowestfree + * + * Description: + * Find the lowest addressed free run in the data region. + * + * Returned Value: + * true if a free run exists, with its start and length reported. + * + ****************************************************************************/ + +static bool xipfs_defrag_findlowestfree(FAR struct xipfs_mount_s *fs, + FAR uint32_t *start, + FAR uint32_t *len) +{ + FAR struct xipfs_extent_s *ext; + uint32_t block; + uint32_t run; + bool used; + + for (block = fs->data_start; + block < fs->data_start + fs->data_nblocks; + block++) + { + used = false; + + for (ext = fs->extents; ext != NULL; ext = ext->flink) + { + if (block >= ext->start_block && + block < ext->start_block + ext->nblocks) + { + used = true; + break; + } + } + + if (used) + { + continue; + } + + /* Measure the run starting here */ + + run = 0; + while (block + run < fs->data_start + fs->data_nblocks) + { + bool inuse = false; + + for (ext = fs->extents; ext != NULL; ext = ext->flink) + { + if (block + run >= ext->start_block && + block + run < ext->start_block + ext->nblocks) + { + inuse = true; + break; + } + } + + if (inuse) + { + break; + } + + run++; + } + + *start = block; + *len = run; + return true; + } + + return false; +} + +/**************************************************************************** + * Name: xipfs_defrag_pickvictim + * + * Description: + * Choose the unpinned extent that starts immediately above the given free + * run and fits inside it. Moving that one down is what merges the hole + * with whatever free space follows the extent. + * + * Returned Value: + * The extent to relocate, or NULL if none qualifies. 'blocked' is set + * when the only candidates were rejected because they are pinned, which + * is the difference between "nothing left to do" and "cannot proceed + * until something is unloaded". + * + ****************************************************************************/ + +static FAR struct xipfs_extent_s * +xipfs_defrag_pickvictim(FAR struct xipfs_mount_s *fs, uint32_t free_start, + uint32_t free_len, FAR bool *pin_blocked, + FAR bool *open_blocked, + FAR uint32_t *pinned_blocks) +{ + FAR struct xipfs_extent_s *best = NULL; + FAR struct xipfs_extent_s *ext; + + *pin_blocked = false; + *open_blocked = false; + + for (ext = fs->extents; ext != NULL; ext = ext->flink) + { + if (ext->isdir) + { + /* A directory is a metadata record with no blocks to move */ + + continue; + } + + if (ext->start_block <= free_start) + { + continue; + } + + if (ext->nblocks > free_len) + { + continue; + } + + if (ext->pincount > 0) + { + /* Pinned extents sitting mid-flash split the free space and bound + * how much can ever be reclaimed. That is inherent to executing + * in place, not a defect: the module is running from those very + * blocks. + */ + + *pin_blocked = true; + *pinned_blocks += ext->nblocks; + continue; + } + + if (ext->writing || ext->openrefs > 0) + { + /* Merely open, not mapped. Still not movable -- a reader holds a + * file position against it -- but the caller resolves this by + * closing a descriptor rather than by unloading a module, so it + * is reported separately. + */ + + *open_blocked = true; + continue; + } + + /* Prefer the extent closest to the hole */ + + if (best == NULL || ext->start_block < best->start_block) + { + best = ext; + } + } + + return best; +} + +/**************************************************************************** + * Name: xipfs_defrag_relocate + * + * Description: + * Perform one atomic relocation of 'ext' to 'dest'. + * + * The order is load bearing: copy the data in full, commit the metadata + * that names the new location, and only then erase the old blocks. A + * power loss before the commit leaves the old generation live and the old + * data intact, so the copy is simply forgotten. A power loss after it + * leaves the new location live and the old blocks merely stale, which the + * next mount reclaims because nothing references them. + * + ****************************************************************************/ + +static int xipfs_defrag_relocate(FAR struct xipfs_mount_s *fs, + FAR struct xipfs_extent_s *ext, + uint32_t dest) +{ + uint32_t old_start = ext->start_block; + uint32_t nblocks = ext->nblocks; + uint32_t pages_per_block; + uint32_t block; + uint32_t page; + int ret; + + pages_per_block = fs->geo.erasesize / fs->geo.blocksize; + + /* The destination must already be erased. Erasing it here rather than + * assuming keeps the invariant local and cheap: it is free space, so + * nothing can be lost by erasing it. + */ + + ret = xipfs_flash_erase(fs, dest, nblocks); + if (ret < 0) + { + return ret; + } + + /* Copy a page at a time. Block granular allocation means no source block + * ever holds a mix of live and dead data, so the staging buffer is one + * program page rather than a whole erase block. + */ + + for (block = 0; block < nblocks; block++) + { + for (page = 0; page < pages_per_block; page++) + { + uint32_t offset = page * fs->geo.blocksize; + + ret = xipfs_flash_read(fs, old_start + block, offset, fs->stage, + fs->geo.blocksize); + if (ret < 0) + { + return ret; + } + + ret = xipfs_flash_write(fs, dest + block, offset, fs->stage, + fs->geo.blocksize); + if (ret < 0) + { + return ret; + } + } + } + + /* Commit point. Until this succeeds the extent still officially lives at + * its old location and everything written above is invisible. + */ + + ext->start_block = dest; + + ret = xipfs_meta_commit(fs); + if (ret < 0) + { + ext->start_block = old_start; + return ret; + } + + /* The move is now durable. Erasing the vacated blocks is pure hygiene: + * if it fails or is interrupted, the layout is still correct and the + * blocks are simply unreferenced until something reuses them. + */ + + xipfs_alloc_rebuild(fs); + xipfs_flash_erase(fs, old_start, nblocks); + + return OK; +} + +/**************************************************************************** + * Public Functions + ****************************************************************************/ + +/**************************************************************************** + * Name: xipfs_defrag + * + * Description: + * Run one best-effort compaction pass. Called with the mount unlocked. + * + * Returned Value: + * OK on any clean stop, including one that made no progress at all. A + * negative errno only for a failure that prevented the pass from running. + * + ****************************************************************************/ + +int xipfs_defrag(FAR struct xipfs_mount_s *fs, uint32_t max_ms, + FAR struct xipfs_defrag_result_s *result) +{ + FAR struct xipfs_extent_s *victim; + clock_t start_tick; + uint32_t free_start; + uint32_t free_len; + uint32_t pinned_blocks; + bool pin_blocked; + bool open_blocked; + int ret; + + memset(result, 0, sizeof(*result)); + result->reason = XIPFS_DEFRAG_DONE; + + ret = xipfs_lock(fs); + if (ret < 0) + { + return ret; + } + + start_tick = clock_systime_ticks(); + + for (; ; ) + { + /* Check the caller's budget between moves, never during one */ + + if (max_ms > 0 && + TICK2MSEC(clock_systime_ticks() - start_tick) >= max_ms) + { + result->reason = XIPFS_DEFRAG_TIME_BUDGET; + break; + } + + if (!xipfs_defrag_findlowestfree(fs, &free_start, &free_len)) + { + /* No free space at all; nothing to compact into */ + + break; + } + + pinned_blocks = 0; + victim = xipfs_defrag_pickvictim(fs, free_start, free_len, + &pin_blocked, &open_blocked, + &pinned_blocks); + if (victim == NULL) + { + /* A live mapping is the more serious obstruction, so it wins the + * report when both are present. + */ + + if (pin_blocked) + { + result->reason = XIPFS_DEFRAG_BLOCKED_PINS; + result->blocks_pinned = pinned_blocks; + } + else if (open_blocked) + { + result->reason = XIPFS_DEFRAG_BLOCKED_OPEN; + } + + break; + } + + ret = xipfs_defrag_relocate(fs, victim, free_start); + if (ret < 0) + { + /* Stopped at a valid intermediate layout. Nothing is partially + * applied, so this is a clean stop rather than an error the + * caller has to recover from. + */ + + result->reason = (ret == -ENOMEM) ? XIPFS_DEFRAG_BLOCKED_RAM + : XIPFS_DEFRAG_ERROR; + break; + } + + result->extents_moved++; + result->blocks_reclaimed += victim->nblocks; + } + + result->largest_free_run = (size_t)xipfs_alloc_largestrun(fs) * + fs->geo.erasesize; + + finfo("xipfs: defrag moved %" PRIu32 " extents, largest run %zu, " + "reason %d\n", result->extents_moved, result->largest_free_run, + result->reason); + + xipfs_unlock(fs); + return OK; +} + +/**************************************************************************** + * Name: xipfs_listpinned + * + * Description: + * Report the extents currently holding XIP pins. Without this, a caller + * told that defrag is blocked by pins has no way to find out which module + * to unload, and the report is a dead end. + * + ****************************************************************************/ + +int xipfs_listpinned(FAR struct xipfs_mount_s *fs, + FAR struct xipfs_pinned_entry_s *entries, + size_t nentries, FAR size_t *count) +{ + FAR struct xipfs_extent_s *ext; + size_t n = 0; + int ret; + + ret = xipfs_lock(fs); + if (ret < 0) + { + return ret; + } + + for (ext = fs->extents; ext != NULL; ext = ext->flink) + { + if (ext->pincount == 0) + { + continue; + } + + if (n < nentries) + { + xipfs_meta_path(fs, ext, entries[n].path, + sizeof(entries[n].path)); + entries[n].start_block = ext->start_block; + entries[n].nblocks = ext->nblocks; + entries[n].pincount = ext->pincount; + } + + n++; + } + + /* Report the true total even when the caller's array was too small, so a + * short buffer is detectable rather than silently truncating. + */ + + *count = n; + + xipfs_unlock(fs); + return n > nentries ? -ENOSPC : OK; +} diff --git a/fs/xipfs/xipfs_flash.c b/fs/xipfs/xipfs_flash.c new file mode 100644 index 0000000000000..08a1cde0daa8d --- /dev/null +++ b/fs/xipfs/xipfs_flash.c @@ -0,0 +1,455 @@ +/**************************************************************************** + * fs/xipfs/xipfs_flash.c + * + * SPDX-License-Identifier: Apache-2.0 + * + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. The + * ASF licenses this file to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance with the + * License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + * + ****************************************************************************/ + +/**************************************************************************** + * Included Files + * + * Every media access in xipfs funnels through this file. That is + * deliberate: on a single NOR die without read-while-write, an erase stalls + * all fetches from the die, so erase slicing, running the erase routine + * from RAM and erase-suspend/resume all have to be introduced here once the + * actual NOR part is known. Keeping the chokepoint from the start means + * adding them later is a local change rather than a refactor. + * + ****************************************************************************/ + +#include + +#include +#include +#include +#include +#include + +#include +#include + +#include "xipfs.h" + +/**************************************************************************** + * Private Functions + ****************************************************************************/ + +/**************************************************************************** + * Name: xipfs_flash_fault + * + * Description: + * Test hook modelling a power loss at an arbitrary flash operation. When + * the countdown reaches zero the operation fails and the medium latches + * dead for the remainder of the run, which is what a real power cut looks + * like to the code above: no further write or erase ever completes. + * + * A real cut tears exactly the one operation that was in flight at the + * instant power dropped; everything nominally after it simply never runs. + * So only the first failing operation reports XIPFS_OP_FAIL_TORN, and only + * when the caller armed torn mode -- every operation past the dead latch + * fails clean, having never touched the medium. + * + ****************************************************************************/ + +#ifdef CONFIG_FS_XIPFS_FAULT_INJECT +enum xipfs_fault_e +{ + XIPFS_OP_PROCEED = 0, /* Not the injected point; run the operation */ + XIPFS_OP_FAIL_CLEAN, /* Fail with the medium left untouched */ + XIPFS_OP_FAIL_TORN /* Fail after half of the operation has landed */ +}; + +static enum xipfs_fault_e xipfs_flash_fault(FAR struct xipfs_mount_s *fs) +{ + if (fs->media_dead) + { + return XIPFS_OP_FAIL_CLEAN; + } + + if (fs->fault_countdown < 0) + { + return XIPFS_OP_PROCEED; + } + + if (fs->fault_countdown == 0) + { + finfo("xipfs: injected %s media failure\n", + fs->fault_mode == XIPFS_FAULT_TORN ? "torn" : "clean"); + fs->media_dead = true; + return fs->fault_mode == XIPFS_FAULT_TORN ? + XIPFS_OP_FAIL_TORN : XIPFS_OP_FAIL_CLEAN; + } + + fs->fault_countdown--; + return XIPFS_OP_PROCEED; +} + +/**************************************************************************** + * Name: xipfs_flash_tear_write + * + * Description: + * Model a NOR page program cut short by power loss: the first half of the + * page reaches the medium and the rest is left at the erased value, so a + * reader sees a page that is neither its old contents nor the intended new + * ones -- which is what makes a torn metadata generation fail its CRC + * rather than look valid. Every xipfs write is a single read/write block, + * so the tear is expressed by reprogramming that block with its tail + * replaced by the erased value; a transient allocation failure simply + * leaves the operation a clean one, still a legitimate power-loss outcome. + * + ****************************************************************************/ + +static void xipfs_flash_tear_write(FAR struct xipfs_mount_s *fs, + uint32_t block, uint32_t offset, + FAR const void *buffer, size_t nbytes) +{ + off_t byteoff = (off_t)block * fs->geo.erasesize + offset; + size_t half = nbytes / 2; + FAR uint8_t *torn; + + torn = kmm_malloc(nbytes); + if (torn == NULL) + { + return; + } + + memcpy(torn, buffer, half); + memset(torn + half, 0xff, nbytes - half); + + MTD_BWRITE(fs->mtd, byteoff / fs->geo.blocksize, + nbytes / fs->geo.blocksize, torn); + + kmm_free(torn); +} + +/**************************************************************************** + * Name: xipfs_flash_tear_erase + * + * Description: + * Model a sector erase cut short: the leading half of the sector reaches + * the erased state while the rest still holds its old contents. Expressed + * by programming the erased value over the first half of the read/write + * blocks, which is a simulation the RAM-backed test medium accepts even + * though real NOR could not be erased this way. + * + ****************************************************************************/ + +static void xipfs_flash_tear_erase(FAR struct xipfs_mount_s *fs, + uint32_t block) +{ + uint32_t blks_per_sector = fs->geo.erasesize / fs->geo.blocksize; + off_t startblk = (off_t)block * blks_per_sector; + uint32_t half = blks_per_sector / 2; + FAR uint8_t *ones; + uint32_t i; + + if (half == 0) + { + return; + } + + ones = kmm_malloc(fs->geo.blocksize); + if (ones == NULL) + { + return; + } + + memset(ones, 0xff, fs->geo.blocksize); + + for (i = 0; i < half; i++) + { + MTD_BWRITE(fs->mtd, startblk + i, 1, ones); + } + + kmm_free(ones); +} +#endif + +/**************************************************************************** + * Public Functions + ****************************************************************************/ + +/**************************************************************************** + * Name: xipfs_flash_probe + * + * Description: + * Read the MTD geometry and resolve the direct address of the media. + * + * A media that cannot answer BIOC_XIPBASE is still perfectly usable as a + * filesystem; it simply cannot serve execute-in-place mappings, and + * strict XIP requests against it fail loudly rather than silently + * degrading into a RAM copy. + * + ****************************************************************************/ + +int xipfs_flash_probe(FAR struct xipfs_mount_s *fs) +{ + FAR void *xipbase = NULL; + int ret; + + ret = MTD_IOCTL(fs->mtd, MTDIOC_GEOMETRY, + (unsigned long)(uintptr_t)&fs->geo); + if (ret < 0) + { + ferr("ERROR: MTDIOC_GEOMETRY failed: %d\n", ret); + return ret; + } + + if (fs->geo.blocksize == 0 || fs->geo.erasesize == 0 || + (fs->geo.erasesize % fs->geo.blocksize) != 0) + { + ferr("ERROR: Bad geometry: blocksize=%" PRIu32 " erasesize=%" PRIu32 + "\n", fs->geo.blocksize, fs->geo.erasesize); + return -EINVAL; + } + + if (fs->geo.neraseblocks < XIPFS_MIN_BLOCKS) + { + ferr("ERROR: Volume too small: %" PRIu32 " blocks\n", + fs->geo.neraseblocks); + return -EINVAL; + } + + /* The metadata generation header occupies the first read/write block of a + * ring block and the dirents follow it, so a ring block must hold at + * least the header plus one dirent. + */ + + if (fs->geo.erasesize <= fs->geo.blocksize) + { + ferr("ERROR: erasesize must exceed blocksize for the metadata ring\n"); + return -EINVAL; + } + + ret = MTD_IOCTL(fs->mtd, BIOC_XIPBASE, (unsigned long)(uintptr_t)&xipbase); + if (ret >= 0 && xipbase != NULL) + { + fs->xipbase = (FAR uint8_t *)xipbase; + finfo("xipfs: XIP base %p\n", fs->xipbase); + } + else + { + fs->xipbase = NULL; + fwarn("xipfs: media is not memory mapped; XIP unavailable\n"); + } + + return OK; +} + +/**************************************************************************** + * Name: xipfs_flash_addr + * + * Description: + * Return the direct address of an erase block, or NULL when the media is + * not memory mapped. + * + ****************************************************************************/ + +FAR uint8_t *xipfs_flash_addr(FAR struct xipfs_mount_s *fs, uint32_t block) +{ + if (fs->xipbase == NULL) + { + return NULL; + } + + return fs->xipbase + (size_t)block * fs->geo.erasesize; +} + +/**************************************************************************** + * Name: xipfs_flash_read + * + * Description: + * Read nbytes at 'offset' within erase block 'block'. + * + ****************************************************************************/ + +int xipfs_flash_read(FAR struct xipfs_mount_s *fs, uint32_t block, + uint32_t offset, FAR void *buffer, size_t nbytes) +{ + off_t byteoff; + ssize_t nread; + + DEBUGASSERT(offset + nbytes <= fs->geo.erasesize); + + if (nbytes == 0) + { + return OK; + } + + byteoff = (off_t)block * fs->geo.erasesize + offset; + + /* Prefer the byte oriented read when the driver offers one. Otherwise + * fall back to reading whole read/write blocks through the staging + * buffer. + */ + + if (fs->mtd->read != NULL) + { + nread = MTD_READ(fs->mtd, byteoff, nbytes, buffer); + if (nread < 0) + { + return (int)nread; + } + + return nread == (ssize_t)nbytes ? OK : -EIO; + } + + /* Block oriented fallback. Only whole read/write blocks can be + * transferred, so the caller's range must be aligned. Every internal + * caller either reads a whole block or reads from a byte capable driver, + * so this is an assertion rather than a supported partial path. + */ + + if ((offset % fs->geo.blocksize) != 0 || + (nbytes % fs->geo.blocksize) != 0) + { + return -EINVAL; + } + + nread = MTD_BREAD(fs->mtd, byteoff / fs->geo.blocksize, + nbytes / fs->geo.blocksize, buffer); + if (nread < 0) + { + return (int)nread; + } + + return nread == (ssize_t)(nbytes / fs->geo.blocksize) ? OK : -EIO; +} + +/**************************************************************************** + * Name: xipfs_flash_write + * + * Description: + * Write nbytes at 'offset' within erase block 'block'. Both must be + * multiples of the read/write block size: NOR is programmed in whole + * pages, and every xipfs write path is structured to respect that. + * + ****************************************************************************/ + +int xipfs_flash_write(FAR struct xipfs_mount_s *fs, uint32_t block, + uint32_t offset, FAR const void *buffer, + size_t nbytes) +{ + off_t byteoff; + ssize_t nwritten; + + DEBUGASSERT(offset + nbytes <= fs->geo.erasesize); + + if (nbytes == 0) + { + return OK; + } + + if ((offset % fs->geo.blocksize) != 0 || + (nbytes % fs->geo.blocksize) != 0) + { + ferr("ERROR: Unaligned write: offset=%" PRIu32 " nbytes=%zu\n", + offset, nbytes); + return -EINVAL; + } + +#ifdef CONFIG_FS_XIPFS_FAULT_INJECT + switch (xipfs_flash_fault(fs)) + { + case XIPFS_OP_FAIL_TORN: + xipfs_flash_tear_write(fs, block, offset, buffer, nbytes); + + /* Fall through: half the page has landed; report the cut. */ + + case XIPFS_OP_FAIL_CLEAN: + return -EIO; + + default: + break; + } +#endif + + byteoff = (off_t)block * fs->geo.erasesize + offset; + + nwritten = MTD_BWRITE(fs->mtd, byteoff / fs->geo.blocksize, + nbytes / fs->geo.blocksize, buffer); + if (nwritten < 0) + { + return (int)nwritten; + } + + return nwritten == (ssize_t)(nbytes / fs->geo.blocksize) ? OK : -EIO; +} + +/**************************************************************************** + * Name: xipfs_flash_erase + * + * Description: + * Erase nblocks erase blocks starting at 'block'. + * + * On a single NOR die this is the operation that stalls instruction + * fetches, so this is where erase slicing and erase-suspend belong once + * the part is known. Callers must already have guaranteed that no live + * XIP mapping covers the range: defrag does so by refusing to touch a + * pinned extent. + * + ****************************************************************************/ + +int xipfs_flash_erase(FAR struct xipfs_mount_s *fs, uint32_t block, + uint32_t nblocks) +{ + uint32_t i; + int ret; + + if (nblocks == 0) + { + return OK; + } + + DEBUGASSERT(block + nblocks <= fs->geo.neraseblocks); + + /* Erase one block at a time. On real hardware this loop is the natural + * place to bound interrupt latency; keeping the granularity here from the + * beginning means the callers already tolerate a partial erase followed + * by an error. + */ + + for (i = 0; i < nblocks; i++) + { +#ifdef CONFIG_FS_XIPFS_FAULT_INJECT + switch (xipfs_flash_fault(fs)) + { + case XIPFS_OP_FAIL_TORN: + xipfs_flash_tear_erase(fs, block + i); + + /* Fall through: half the sector is erased; report the cut. */ + + case XIPFS_OP_FAIL_CLEAN: + return -EIO; + + default: + break; + } +#endif + + ret = MTD_ERASE(fs->mtd, block + i, 1); + if (ret < 0) + { + ferr("ERROR: Erase of block %" PRIu32 " failed: %d\n", + block + i, ret); + return ret; + } + } + + return OK; +} diff --git a/fs/xipfs/xipfs_meta.c b/fs/xipfs/xipfs_meta.c new file mode 100644 index 0000000000000..8f842edc0703a --- /dev/null +++ b/fs/xipfs/xipfs_meta.c @@ -0,0 +1,873 @@ +/**************************************************************************** + * fs/xipfs/xipfs_meta.c + * + * SPDX-License-Identifier: Apache-2.0 + * + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. The + * ASF licenses this file to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance with the + * License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + * + ****************************************************************************/ + +/**************************************************************************** + * Included Files + * + * Power-loss atomicity of the metadata commit lives here. The scheme is a + * ping-pong ring of whole generations: each commit erases the next ring + * slot, writes the dirent body, and finally writes the header block that + * carries the sequence number and the CRC over header plus body. + * + * That final header write is the commit point, and it is a single + * read/write block program -- the smallest unit the media can tear at. A + * power loss before it leaves a slot whose header is still erased, so the + * generation simply does not exist. A power loss during it leaves a header + * that fails its own CRC. Either way mount falls back to the previous + * generation, which is untouched because it lives in a different slot. + * + * The workload is tens of modules, so a whole directory fits in one erase + * block. Rewriting all of it per commit is what buys the single-write + * commit point and removes any need for journal replay logic. + * + ****************************************************************************/ + +#include + +#include +#include +#include +#include +#include + +#include +#include + +#include "xipfs.h" + +/**************************************************************************** + * Private Functions + ****************************************************************************/ + +/**************************************************************************** + * Name: xipfs_meta_bodyblocks + * + * Description: + * Number of read/write blocks the dirent body of a generation occupies. + * + ****************************************************************************/ + +static uint32_t xipfs_meta_bodyblocks(FAR struct xipfs_mount_s *fs, + uint32_t nentries) +{ + if (nentries == 0) + { + return 0; + } + + return (nentries + fs->entries_per_blk - 1) / fs->entries_per_blk; +} + +/**************************************************************************** + * Name: xipfs_meta_verify + * + * Description: + * Read the generation in ring slot 'slot' and verify it. On success + * returns OK and reports the sequence number and entry count. + * + ****************************************************************************/ + +static int xipfs_meta_verify(FAR struct xipfs_mount_s *fs, uint32_t slot, + FAR uint32_t *seq, FAR uint32_t *nentries) +{ + FAR struct xipfs_genhdr_s *hdr; + uint32_t hdrfields[3]; + uint32_t running; + uint32_t nbody; + uint32_t crc; + uint32_t i; + int ret; + + ret = xipfs_flash_read(fs, fs->meta_start + slot, 0, fs->metabuf, + fs->geo.blocksize); + if (ret < 0) + { + return ret; + } + + hdr = (FAR struct xipfs_genhdr_s *)fs->metabuf; + if (hdr->magic != XIPFS_GEN_MAGIC) + { + return -EINVAL; + } + + if (hdr->nentries > fs->max_entries) + { + return -EINVAL; + } + + /* Snapshot the header before the body reads reuse the buffer */ + + hdrfields[0] = hdr->magic; + hdrfields[1] = hdr->seq; + hdrfields[2] = hdr->nentries; + crc = hdr->crc; + + /* Recompute the CRC over the header fields followed by every body block. + * An empty generation is legitimate and simply has no body blocks. + */ + + running = crc32part((FAR const uint8_t *)hdrfields, sizeof(hdrfields), 0); + nbody = xipfs_meta_bodyblocks(fs, hdrfields[2]); + + for (i = 0; i < nbody; i++) + { + ret = xipfs_flash_read(fs, fs->meta_start + slot, + (i + 1) * fs->geo.blocksize, fs->metabuf, + fs->geo.blocksize); + if (ret < 0) + { + return ret; + } + + running = crc32part(fs->metabuf, fs->geo.blocksize, running); + } + + if (running != crc) + { + return -EINVAL; + } + + *seq = hdrfields[1]; + *nentries = hdrfields[2]; + return OK; +} + +/**************************************************************************** + * Name: xipfs_meta_validate + * + * Description: + * Check that the entries just loaded form a tree. Two identities must not + * collide, every parent must name a live directory, and following parents + * must reach the root -- a cycle on the medium would otherwise hang the + * path walk rather than merely returning the wrong answer. Bounding the + * climb by the entry count is what makes that terminate. + * + ****************************************************************************/ + +static int xipfs_meta_validate(FAR struct xipfs_mount_s *fs) +{ + FAR struct xipfs_extent_s *ext; + FAR struct xipfs_extent_s *other; + FAR struct xipfs_extent_s *up; + uint32_t steps; + + for (ext = fs->extents; ext != NULL; ext = ext->flink) + { + for (other = ext->flink; other != NULL; other = other->flink) + { + if (other->id == ext->id) + { + ferr("ERROR: Duplicate id %u ('%s' and '%s')\n", + ext->id, ext->name, other->name); + return -EFTYPE; + } + + if (other->parent == ext->parent && + strcmp(other->name, ext->name) == 0) + { + ferr("ERROR: Duplicate name '%s' in directory %u\n", + ext->name, ext->parent); + return -EFTYPE; + } + } + + up = ext; + steps = 0; + + while (up->parent != XIPFS_ROOT_ID) + { + if (++steps > fs->nextents) + { + ferr("ERROR: Directory cycle above '%s'\n", ext->name); + return -EFTYPE; + } + + for (other = fs->extents; other != NULL; other = other->flink) + { + if (other->id == up->parent) + { + break; + } + } + + if (other == NULL || !other->isdir) + { + ferr("ERROR: '%s' has no parent directory %u\n", + up->name, up->parent); + return -EFTYPE; + } + + up = other; + } + } + + return OK; +} + +/**************************************************************************** + * Name: xipfs_meta_load + * + * Description: + * Build the in-RAM entry list from the generation in ring slot 'slot'. + * + ****************************************************************************/ + +static int xipfs_meta_load(FAR struct xipfs_mount_s *fs, uint32_t slot, + uint32_t nentries) +{ + FAR struct xipfs_dirent_s *dirent; + FAR struct xipfs_extent_s *ext; + uint32_t nbody; + uint32_t idx = 0; + uint32_t i; + uint32_t j; + int ret; + + xipfs_meta_freeextents(fs); + + nbody = xipfs_meta_bodyblocks(fs, nentries); + + for (i = 0; i < nbody; i++) + { + ret = xipfs_flash_read(fs, fs->meta_start + slot, + (i + 1) * fs->geo.blocksize, fs->metabuf, + fs->geo.blocksize); + if (ret < 0) + { + return ret; + } + + for (j = 0; j < fs->entries_per_blk && idx < nentries; j++, idx++) + { + dirent = (FAR struct xipfs_dirent_s *) + (fs->metabuf + j * XIPFS_DIRENT_SIZE); + + bool isdir = (dirent->flags & XIPFS_DIRENT_DIR) != 0; + + /* The mount path is exactly where a corrupted medium must be + * caught rather than trusted. Every entry needs an identity of + * its own, a directory must carry no extent, and a file's extent + * must lie inside the data region. + */ + + if (dirent->id == XIPFS_ROOT_ID || + dirent->name[0] == '\0' || + strchr(dirent->name, '/') != NULL) + { + ferr("ERROR: Corrupt dirent, id %u name '%s'\n", + dirent->id, dirent->name); + return -EFTYPE; + } + + if (isdir) + { + if (dirent->start_block != 0 || dirent->nblocks != 0 || + dirent->size != 0) + { + ferr("ERROR: Directory '%s' carries an extent\n", + dirent->name); + return -EFTYPE; + } + } + else if (dirent->start_block < fs->data_start || + dirent->nblocks == 0 || + dirent->start_block + dirent->nblocks > + fs->data_start + fs->data_nblocks) + { + ferr("ERROR: Corrupt dirent '%s' at %" PRIu32 "+%" PRIu32 "\n", + dirent->name, dirent->start_block, dirent->nblocks); + return -EFTYPE; + } + + ext = kmm_zalloc(sizeof(struct xipfs_extent_s)); + if (ext == NULL) + { + return -ENOMEM; + } + + strlcpy(ext->name, dirent->name, sizeof(ext->name)); + ext->fs = fs; + ext->id = dirent->id; + ext->parent = dirent->parent; + ext->isdir = isdir; + ext->size = dirent->size; + ext->start_block = dirent->start_block; + ext->nblocks = dirent->nblocks; + + ext->flink = fs->extents; + fs->extents = ext; + fs->nextents++; + } + } + + return xipfs_meta_validate(fs); +} + +/**************************************************************************** + * Name: xipfs_meta_newid + * + * Description: + * The lowest identity not in use. Identities are stored on the medium + * because parent references them, so they have to survive a remount; they + * are reused once nothing refers to them. Zero is reserved for the root. + * + ****************************************************************************/ + +static uint16_t xipfs_meta_newid(FAR struct xipfs_mount_s *fs) +{ + FAR struct xipfs_extent_s *ext; + uint16_t id; + + for (id = 1; id != 0; id++) + { + for (ext = fs->extents; ext != NULL; ext = ext->flink) + { + if (ext->id == id) + { + break; + } + } + + if (ext == NULL) + { + return id; + } + } + + return 0; +} + +/**************************************************************************** + * Public Functions + ****************************************************************************/ + +/**************************************************************************** + * Name: xipfs_meta_freeextents + ****************************************************************************/ + +void xipfs_meta_freeextents(FAR struct xipfs_mount_s *fs) +{ + FAR struct xipfs_extent_s *ext; + FAR struct xipfs_extent_s *next; + + for (ext = fs->extents; ext != NULL; ext = next) + { + next = ext->flink; + kmm_free(ext); + } + + fs->extents = NULL; + fs->nextents = 0; +} + +/**************************************************************************** + * Name: xipfs_meta_commit + * + * Description: + * Write the current in-RAM directory as a new generation into the next + * ring slot. Returns only after the commit point has landed, so a + * successful return means the new state is durable and a failure means + * the previous generation is still the live one. + * + ****************************************************************************/ + +int xipfs_meta_commit(FAR struct xipfs_mount_s *fs) +{ + FAR struct xipfs_genhdr_s *hdr; + FAR struct xipfs_extent_s *ext; + FAR struct xipfs_dirent_s *dirent; + uint32_t hdrfields[3]; + uint32_t nentries = 0; + uint32_t slot; + uint32_t nbody; + uint32_t crc; + uint32_t i; + uint32_t j; + int ret; + + /* Count the entries that will be committed. Unlinked extents are + * already detached from the list, so they are simply absent. + */ + + for (ext = fs->extents; ext != NULL; ext = ext->flink) + { + nentries++; + } + + if (nentries > fs->max_entries) + { + return -ENOSPC; + } + + slot = (fs->meta_slot + 1) % fs->meta_nblocks; + nbody = xipfs_meta_bodyblocks(fs, nentries); + + /* Erase the target slot. Until the header write below lands, this slot + * holds nothing valid and the live generation remains the old one. + */ + + ret = xipfs_flash_erase(fs, fs->meta_start + slot, 1); + if (ret < 0) + { + return ret; + } + + hdrfields[0] = XIPFS_GEN_MAGIC; + hdrfields[1] = fs->meta_seq + 1; + hdrfields[2] = nentries; + crc = crc32part((FAR const uint8_t *)hdrfields, sizeof(hdrfields), 0); + + /* Write the body first, accumulating the CRC as we go */ + + ext = fs->extents; + for (i = 0; i < nbody; i++) + { + memset(fs->metabuf, 0xff, fs->geo.blocksize); + + for (j = 0; j < fs->entries_per_blk && ext != NULL; j++) + { + dirent = (FAR struct xipfs_dirent_s *) + (fs->metabuf + j * XIPFS_DIRENT_SIZE); + + memset(dirent, 0, sizeof(*dirent)); + strlcpy(dirent->name, ext->name, sizeof(dirent->name)); + dirent->size = ext->size; + dirent->start_block = ext->start_block; + dirent->nblocks = ext->nblocks; + dirent->flags = ext->isdir ? XIPFS_DIRENT_DIR : 0; + dirent->id = ext->id; + dirent->parent = ext->parent; + + ext = ext->flink; + } + + crc = crc32part(fs->metabuf, fs->geo.blocksize, crc); + + ret = xipfs_flash_write(fs, fs->meta_start + slot, + (i + 1) * fs->geo.blocksize, fs->metabuf, + fs->geo.blocksize); + if (ret < 0) + { + return ret; + } + } + + /* The commit point: a single read/write block program */ + + memset(fs->metabuf, 0xff, fs->geo.blocksize); + hdr = (FAR struct xipfs_genhdr_s *)fs->metabuf; + hdr->magic = hdrfields[0]; + hdr->seq = hdrfields[1]; + hdr->nentries = hdrfields[2]; + hdr->crc = crc; + + ret = xipfs_flash_write(fs, fs->meta_start + slot, 0, fs->metabuf, + fs->geo.blocksize); + if (ret < 0) + { + return ret; + } + + fs->meta_slot = slot; + fs->meta_seq = hdrfields[1]; + return OK; +} + +/**************************************************************************** + * Name: xipfs_meta_format + * + * Description: + * Lay down a fresh filesystem: erase the volume, write both superblock + * copies, then commit an empty generation. + * + ****************************************************************************/ + +int xipfs_meta_format(FAR struct xipfs_mount_s *fs) +{ + FAR struct xipfs_sblk_s *sblk; + int ret; + + ret = xipfs_flash_erase(fs, 0, fs->geo.neraseblocks); + if (ret < 0) + { + return ret; + } + + memset(fs->metabuf, 0xff, fs->geo.blocksize); + sblk = (FAR struct xipfs_sblk_s *)fs->metabuf; + + memset(sblk, 0, sizeof(*sblk)); + sblk->magic = XIPFS_SBLK_MAGIC; + sblk->version = XIPFS_VERSION; + sblk->erasesize = fs->geo.erasesize; + sblk->blocksize = fs->geo.blocksize; + sblk->nblocks = fs->geo.neraseblocks; + sblk->meta_start = XIPFS_META_START; + sblk->meta_nblocks = XIPFS_META_NBLOCKS; + sblk->data_start = XIPFS_DATA_START; + sblk->data_nblocks = fs->geo.neraseblocks - XIPFS_DATA_START; + sblk->crc = crc32((FAR const uint8_t *)sblk, + offsetof(struct xipfs_sblk_s, crc)); + + ret = xipfs_flash_write(fs, XIPFS_SBLK_A, 0, fs->metabuf, + fs->geo.blocksize); + if (ret < 0) + { + return ret; + } + + ret = xipfs_flash_write(fs, XIPFS_SBLK_B, 0, fs->metabuf, + fs->geo.blocksize); + if (ret < 0) + { + return ret; + } + + /* Adopt the geometry we just wrote, then commit generation 1 */ + + fs->meta_start = XIPFS_META_START; + fs->meta_nblocks = XIPFS_META_NBLOCKS; + fs->data_start = XIPFS_DATA_START; + fs->data_nblocks = fs->geo.neraseblocks - XIPFS_DATA_START; + fs->meta_slot = fs->meta_nblocks - 1; + fs->meta_seq = 0; + + xipfs_meta_freeextents(fs); + + return xipfs_meta_commit(fs); +} + +/**************************************************************************** + * Name: xipfs_meta_mount + * + * Description: + * Read the superblock, scan the metadata ring and adopt the last fully + * valid generation. + * + ****************************************************************************/ + +int xipfs_meta_mount(FAR struct xipfs_mount_s *fs) +{ + struct xipfs_sblk_s sblk; + uint32_t best_slot = 0; + uint32_t best_seq = 0; + uint32_t best_nentries = 0; + bool found = false; + uint32_t copy; + uint32_t slot; + uint32_t seq; + uint32_t nentries; + int ret; + + /* Superblock: try copy A, fall back to copy B */ + + ret = -EFTYPE; + for (copy = XIPFS_SBLK_A; copy <= XIPFS_SBLK_B; copy++) + { + ret = xipfs_flash_read(fs, copy, 0, fs->metabuf, fs->geo.blocksize); + if (ret < 0) + { + continue; + } + + memcpy(&sblk, fs->metabuf, sizeof(sblk)); + + if (sblk.magic != XIPFS_SBLK_MAGIC || + sblk.version != XIPFS_VERSION) + { + ret = -EFTYPE; + continue; + } + + if (crc32((FAR const uint8_t *)&sblk, + offsetof(struct xipfs_sblk_s, crc)) != sblk.crc) + { + fwarn("xipfs: superblock copy %" PRIu32 " failed CRC\n", copy); + ret = -EFTYPE; + continue; + } + + /* The geometry recorded at format time must still match the medium, + * otherwise every block index in the directory means something + * different from what it meant when it was written. + */ + + if (sblk.erasesize != fs->geo.erasesize || + sblk.blocksize != fs->geo.blocksize || + sblk.nblocks != fs->geo.neraseblocks) + { + ferr("ERROR: Superblock geometry does not match the medium\n"); + return -EFTYPE; + } + + ret = OK; + break; + } + + if (ret < 0) + { + return ret; + } + + fs->meta_start = sblk.meta_start; + fs->meta_nblocks = sblk.meta_nblocks; + fs->data_start = sblk.data_start; + fs->data_nblocks = sblk.data_nblocks; + + if (fs->meta_nblocks < 2 || + fs->meta_start + fs->meta_nblocks > fs->data_start || + fs->data_start + fs->data_nblocks > fs->geo.neraseblocks) + { + ferr("ERROR: Superblock describes an inconsistent layout\n"); + return -EFTYPE; + } + + /* Scan the ring for the highest sequence number that fully verifies */ + + for (slot = 0; slot < fs->meta_nblocks; slot++) + { + if (xipfs_meta_verify(fs, slot, &seq, &nentries) < 0) + { + continue; + } + + if (!found || seq > best_seq) + { + found = true; + best_seq = seq; + best_slot = slot; + best_nentries = nentries; + } + } + + if (!found) + { + ferr("ERROR: No valid metadata generation found\n"); + return -EFTYPE; + } + + fs->meta_slot = best_slot; + fs->meta_seq = best_seq; + + finfo("xipfs: mounted generation %" PRIu32 " from slot %" PRIu32 + " with %" PRIu32 " entries\n", best_seq, best_slot, best_nentries); + + return xipfs_meta_load(fs, best_slot, best_nentries); +} + +/**************************************************************************** + * Name: xipfs_meta_find + ****************************************************************************/ + +FAR struct xipfs_extent_s *xipfs_meta_find(FAR struct xipfs_mount_s *fs, + uint16_t parent, + FAR const char *name, + size_t namelen) +{ + FAR struct xipfs_extent_s *ext; + + for (ext = fs->extents; ext != NULL; ext = ext->flink) + { + if (ext->parent == parent && + strncmp(ext->name, name, namelen) == 0 && + ext->name[namelen] == '\0') + { + return ext; + } + } + + return NULL; +} + +/**************************************************************************** + * Name: xipfs_meta_path + * + * Description: + * Compose an entry's path relative to the mountpoint by climbing its + * parents. Written backwards from the end of the buffer, so a path too + * long to fit loses its leading components rather than the name that + * identifies it. The climb is bounded by the entry count, which mount has + * already proved cycle-free, so this cannot spin. + * + * The caller must hold the lock. + * + ****************************************************************************/ + +void xipfs_meta_path(FAR struct xipfs_mount_s *fs, + FAR struct xipfs_extent_s *ext, + FAR char *buf, size_t buflen) +{ + FAR struct xipfs_extent_s *up = ext; + FAR struct xipfs_extent_s *cur; + uint32_t steps = 0; + size_t pos; + + DEBUGASSERT(buflen > 0); + + pos = buflen - 1; + buf[pos] = '\0'; + + for (; ; ) + { + size_t len = strlen(up->name); + + if (len > pos) + { + break; + } + + pos -= len; + memcpy(&buf[pos], up->name, len); + + if (up->parent == XIPFS_ROOT_ID || pos == 0) + { + break; + } + + for (cur = fs->extents; cur != NULL; cur = cur->flink) + { + if (cur->id == up->parent) + { + break; + } + } + + if (cur == NULL || ++steps > fs->nextents) + { + break; + } + + buf[--pos] = '/'; + up = cur; + } + + if (pos > 0) + { + memmove(buf, &buf[pos], buflen - pos); + } +} + +/**************************************************************************** + * Name: xipfs_meta_haschildren + * + * Description: + * Whether anything names 'dirid' as its parent, which is what stops rmdir + * removing a directory that still holds something. + * + ****************************************************************************/ + +bool xipfs_meta_haschildren(FAR struct xipfs_mount_s *fs, uint16_t dirid) +{ + FAR struct xipfs_extent_s *ext; + + for (ext = fs->extents; ext != NULL; ext = ext->flink) + { + if (ext->parent == dirid) + { + return true; + } + } + + return false; +} + +/**************************************************************************** + * Name: xipfs_meta_add + * + * Description: + * Attach a new extent to the in-RAM directory. The caller commits. + * + ****************************************************************************/ + +FAR struct xipfs_extent_s *xipfs_meta_add(FAR struct xipfs_mount_s *fs, + uint16_t parent, + FAR const char *name, + size_t namelen, bool isdir, + uint32_t start_block, + uint32_t nblocks, uint32_t size) +{ + FAR struct xipfs_extent_s *ext; + uint16_t id; + + if (namelen > XIPFS_NAME_MAX) + { + return NULL; + } + + id = xipfs_meta_newid(fs); + if (id == 0) + { + return NULL; + } + + ext = kmm_zalloc(sizeof(struct xipfs_extent_s)); + if (ext == NULL) + { + return NULL; + } + + memcpy(ext->name, name, namelen); + ext->name[namelen] = '\0'; + + ext->fs = fs; + ext->id = id; + ext->parent = parent; + ext->isdir = isdir; + ext->start_block = start_block; + ext->nblocks = nblocks; + ext->size = size; + + ext->flink = fs->extents; + fs->extents = ext; + fs->nextents++; + + return ext; +} + +/**************************************************************************** + * Name: xipfs_meta_detach + * + * Description: + * Remove an extent from the in-RAM directory without freeing it. The + * object stays alive while opens or pins reference it; the caller frees + * it once the last reference goes away. + * + ****************************************************************************/ + +void xipfs_meta_detach(FAR struct xipfs_mount_s *fs, + FAR struct xipfs_extent_s *ext) +{ + FAR struct xipfs_extent_s **prev; + + for (prev = &fs->extents; *prev != NULL; prev = &(*prev)->flink) + { + if (*prev == ext) + { + *prev = ext->flink; + ext->flink = NULL; + ext->unlinked = true; + fs->nextents--; + return; + } + } +} diff --git a/fs/xipfs/xipfs_mmap.c b/fs/xipfs/xipfs_mmap.c new file mode 100644 index 0000000000000..580b271c43c79 --- /dev/null +++ b/fs/xipfs/xipfs_mmap.c @@ -0,0 +1,253 @@ +/**************************************************************************** + * fs/xipfs/xipfs_mmap.c + * + * SPDX-License-Identifier: Apache-2.0 + * + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. The + * ASF licenses this file to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance with the + * License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + * + ****************************************************************************/ + +/**************************************************************************** + * Included Files + * + * This is what distinguishes xipfs from littlefs, FAT or SPIFFS: the mmap + * operation hands back a direct pointer into memory mapped flash rather + * than a RAM copy, so a module's text and rodata are executed and read + * where they already live. + * + * Two properties make that safe. + * + * First, the mapping takes a pin on the extent, and the pin lives on the + * extent object rather than on the file descriptor. Three running + * instances of one module therefore produce a pin count of three, and the + * extent only becomes movable again when the last of them goes away. + * + * Second, the pin is acquired under the same lock the defragmenter holds + * while it decides whether an extent is movable. Without that, a new + * mapping could slip in between the compactor observing "pin count zero" + * and the compactor starting the move, and the module would be executing + * out of blocks that were being erased underneath it. + * + ****************************************************************************/ + +#include + +#include + +#include +#include +#include + +#include +#include + +#include "xipfs.h" + +/**************************************************************************** + * Private Function Prototypes + ****************************************************************************/ + +static int xipfs_munmap(FAR struct task_group_s *group, + FAR struct mm_map_entry_s *entry, + FAR void *start, size_t length); + +/**************************************************************************** + * Private Functions + ****************************************************************************/ + +/**************************************************************************** + * Name: xipfs_munmap + * + * Description: + * Release an XIP mapping and drop its pin. + * + * This runs in two situations: an explicit munmap by the module loader, + * and the task teardown walk that mm_map_destroy performs when a task + * dies. The second case is the one that matters most -- a module that + * faults or is killed without unmapping must still release its pin, or + * the extent stays immovable forever and the defragmenter slowly stops + * being able to reclaim anything. + * + * Because it can run from teardown, this function must not consult + * this_task()->group: the group is being dismantled and is passed as + * NULL. Everything it needs is reachable from the entry itself. + * + ****************************************************************************/ + +static int xipfs_munmap(FAR struct task_group_s *group, + FAR struct mm_map_entry_s *entry, + FAR void *start, size_t length) +{ + FAR struct xipfs_extent_s *ext; + FAR struct xipfs_mount_s *fs; + int ret; + + DEBUGASSERT(entry != NULL && entry->priv.p != NULL); + + /* Only whole mappings can be released. A partial unmap of an XIP window + * has no meaning -- there is no allocation to shrink, only a refcount -- + * and accepting one would leave the pin count wrong in a way that is + * very hard to trace back later. + */ + + if (start != entry->vaddr || length != entry->length) + { + ferr("ERROR: Partial unmap of an XIP mapping is not supported\n"); + return -EINVAL; + } + + ext = (FAR struct xipfs_extent_s *)entry->priv.p; + fs = ext->fs; + + ret = xipfs_lock(fs); + if (ret < 0) + { + /* Failing here would leak the pin permanently, and the caller has no + * way to retry from the teardown path. Drop the count anyway: the + * lock only orders us against defrag, and an unpinned-but-still- + * mapped extent cannot arise because the mapping is going away. + */ + + fwarn("xipfs: unmap could not take the lock; dropping pin anyway\n"); + } + + DEBUGASSERT(ext->pincount > 0); + ext->pincount--; + + /* An extent that was unlinked while mapped is freed once the last + * reference goes away. + */ + + if (ext->unlinked && ext->pincount == 0 && ext->openrefs == 0) + { + xipfs_free(fs, ext->start_block, ext->nblocks); + kmm_free(ext); + } + + if (ret >= 0) + { + xipfs_unlock(fs); + } + + return mm_map_remove(get_group_mm(group), entry); +} + +/**************************************************************************** + * Public Functions + ****************************************************************************/ + +/**************************************************************************** + * Name: xipfs_mmap + * + * Description: + * Map a range of a file directly onto the underlying memory mapped media. + * + * The error returned when that is impossible is significant. The core + * mmap path falls back to copying the file into RAM only when the file + * system answers -ENOTTY, so returning anything else suppresses the + * fallback. A caller that passed MAP_XIP_STRICT -- a module loader, for + * which a silent RAM copy would defeat the entire point of executing in + * place -- gets -ENXIO instead, which it can turn into "defragment and + * retry" or into a refusal to load. Ordinary data readers that did not + * ask for strict behaviour still get the convenience of the copy. + * + ****************************************************************************/ + +int xipfs_mmap(FAR struct file *filep, FAR struct mm_map_entry_s *map) +{ + FAR struct xipfs_mount_s *fs; + FAR struct xipfs_file_s *xf; + FAR struct xipfs_extent_s *ext; + FAR uint8_t *addr; + bool strict; + int ret; + + DEBUGASSERT(filep->f_priv != NULL && filep->f_inode != NULL); + + xf = filep->f_priv; + fs = filep->f_inode->i_private; + strict = (map->flags & MAP_XIP_STRICT) != 0; + + ret = xipfs_lock(fs); + if (ret < 0) + { + return ret; + } + + ext = xf->ext; + + /* A file still being written has no stable contents to map yet */ + + if (ext->writing) + { + ret = strict ? -ENXIO : -ENOTTY; + goto errout_with_lock; + } + + if (map->offset < 0 || map->length == 0 || + (uint64_t)map->offset + map->length > ext->size) + { + ret = -EINVAL; + goto errout_with_lock; + } + + addr = xipfs_flash_addr(fs, ext->start_block); + if (addr == NULL) + { + /* The media cannot be addressed directly, so there is no in-place + * mapping to be had at any price. + */ + + ret = strict ? -ENXIO : -ENOTTY; + goto errout_with_lock; + } + + map->vaddr = addr + map->offset; + + /* Take the pin before releasing the lock. From this moment the extent + * is frozen in place: the defragmenter will skip it, so a mapping can + * never be relocated out from under a running module. + */ + + ext->pincount++; + + /* Record what kind of mapping this is now, rather than trying to work it + * out again at unmap time. priv.p carries the pinned extent and offset + * carries the mount, which is all the teardown path can rely on. + */ + + map->priv.p = ext; + map->offset = (off_t)(uintptr_t)fs; + map->munmap = xipfs_munmap; + map->msync = NULL; + + ret = mm_map_add(get_current_mm(), map); + if (ret < 0) + { + ext->pincount--; + goto errout_with_lock; + } + + finfo("xipfs: XIP map '%s' at %p len %zu pins %" PRIu32 "\n", + ext->name, map->vaddr, map->length, ext->pincount); + + xipfs_unlock(fs); + return OK; + +errout_with_lock: + xipfs_unlock(fs); + return ret; +} diff --git a/fs/xipfs/xipfs_vfs.c b/fs/xipfs/xipfs_vfs.c new file mode 100644 index 0000000000000..33291a2082c84 --- /dev/null +++ b/fs/xipfs/xipfs_vfs.c @@ -0,0 +1,1893 @@ +/**************************************************************************** + * fs/xipfs/xipfs_vfs.c + * + * SPDX-License-Identifier: Apache-2.0 + * + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. The + * ASF licenses this file to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance with the + * License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + * + ****************************************************************************/ + +/**************************************************************************** + * Included Files + * + * The VFS surface is deliberately narrow. Files here are written once at + * creation and are immutable thereafter, so the write path supports exactly + * that shape and refuses anything else rather than half implementing POSIX + * random write semantics that would break the contiguity guarantee the + * whole design rests on. + * + * The namespace is flat: modules are files, there are no directories. + * + ****************************************************************************/ + +#include + +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include + +#include "xipfs.h" + +/**************************************************************************** + * Private Function Prototypes + ****************************************************************************/ + +static int xipfs_open(FAR struct file *filep, FAR const char *relpath, + int oflags, mode_t mode); +static int xipfs_close(FAR struct file *filep); +static ssize_t xipfs_read(FAR struct file *filep, FAR char *buffer, + size_t buflen); +static ssize_t xipfs_write(FAR struct file *filep, FAR const char *buffer, + size_t buflen); +static off_t xipfs_seek(FAR struct file *filep, off_t offset, int whence); +static int xipfs_volume_cmd(FAR struct xipfs_mount_s *fs, int cmd, + unsigned long arg); +static int xipfs_ioctl(FAR struct file *filep, int cmd, + unsigned long arg); +static int xipfs_ioctldir(FAR struct inode *mountpt, + FAR struct fs_dirent_s *dir, + int cmd, unsigned long arg); +static int xipfs_truncate(FAR struct file *filep, off_t length); +static int xipfs_sync(FAR struct file *filep); +static int xipfs_dup(FAR const struct file *oldp, + FAR struct file *newp); +static int xipfs_fstat(FAR const struct file *filep, + FAR struct stat *buf); +static int xipfs_opendir(FAR struct inode *mountpt, + FAR const char *relpath, + FAR struct fs_dirent_s **dir); +static int xipfs_closedir(FAR struct inode *mountpt, + FAR struct fs_dirent_s *dir); +static int xipfs_readdir(FAR struct inode *mountpt, + FAR struct fs_dirent_s *dir, + FAR struct dirent *entry); +static int xipfs_rewinddir(FAR struct inode *mountpt, + FAR struct fs_dirent_s *dir); +static int xipfs_bind(FAR struct inode *driver, FAR const void *data, + FAR void **handle); +static int xipfs_unbind(FAR void *handle, FAR struct inode **driver, + unsigned int flags); +static int xipfs_statfs(FAR struct inode *mountpt, + FAR struct statfs *buf); +static int xipfs_unlink(FAR struct inode *mountpt, + FAR const char *relpath); +static int xipfs_mkdir(FAR struct inode *mountpt, + FAR const char *relpath, mode_t mode); +static int xipfs_rmdir(FAR struct inode *mountpt, + FAR const char *relpath); +static int xipfs_stat(FAR struct inode *mountpt, + FAR const char *relpath, FAR struct stat *buf); + +/**************************************************************************** + * Public Data + ****************************************************************************/ + +const struct mountpt_operations g_xipfs_operations = +{ + xipfs_open, /* open */ + xipfs_close, /* close */ + xipfs_read, /* read */ + xipfs_write, /* write */ + xipfs_seek, /* seek */ + xipfs_ioctl, /* ioctl */ + xipfs_mmap, /* mmap */ + xipfs_truncate, /* truncate */ + NULL, /* poll */ + NULL, /* readv */ + NULL, /* writev */ + xipfs_sync, /* sync */ + xipfs_dup, /* dup */ + xipfs_fstat, /* fstat */ + NULL, /* fchstat */ + xipfs_opendir, /* opendir */ + xipfs_closedir, /* closedir */ + xipfs_readdir, /* readdir */ + xipfs_rewinddir, /* rewinddir */ + xipfs_bind, /* bind */ + xipfs_unbind, /* unbind */ + xipfs_statfs, /* statfs */ + xipfs_unlink, /* unlink */ + xipfs_mkdir, /* mkdir */ + xipfs_rmdir, /* rmdir */ + NULL, /* rename */ + xipfs_stat, /* stat */ + NULL, /* chstat */ + NULL, /* syncfs */ + xipfs_ioctldir /* ioctldir */ +}; + +/**************************************************************************** + * Private Functions + ****************************************************************************/ + +/**************************************************************************** + * Name: xipfs_blocksneeded + ****************************************************************************/ + +static uint32_t xipfs_blocksneeded(FAR struct xipfs_mount_s *fs, + uint32_t nbytes) +{ + if (nbytes == 0) + { + return 1; + } + + return (nbytes + fs->geo.erasesize - 1) / fs->geo.erasesize; +} + +/**************************************************************************** + * Name: xipfs_component_ok + * + * Description: + * Validate one path component. "." and ".." are refused rather than + * interpreted: a stored entry by either name could never be reached again, + * and the VFS has already resolved the ones that were meant to navigate. + * + ****************************************************************************/ + +static int xipfs_component_ok(FAR const char *name, size_t len) +{ + if (len == 0) + { + return -EINVAL; + } + + if (len > XIPFS_NAME_MAX) + { + return -ENAMETOOLONG; + } + + if (name[0] == '.' && (len == 1 || (len == 2 && name[1] == '.'))) + { + return -EINVAL; + } + + return OK; +} + +/**************************************************************************** + * Name: xipfs_walk + * + * Description: + * Resolve every component of 'relpath' but the last, and report the + * directory that would contain it along with where the last component + * starts and how long it is. + * + * Trailing separators are ignored, so "bin/" resolves the same as "bin". + * An empty path is the root, reported as a zero length leaf. + * + * The caller must hold the lock. + * + ****************************************************************************/ + +static int xipfs_walk(FAR struct xipfs_mount_s *fs, FAR const char *relpath, + FAR uint16_t *parent, FAR const char **leaf, + FAR size_t *leaflen) +{ + FAR struct xipfs_extent_s *ext; + FAR const char *p = relpath; + size_t len; + int ret; + + *parent = XIPFS_ROOT_ID; + *leaf = p; + *leaflen = 0; + + if (p == NULL) + { + return OK; + } + + for (; ; ) + { + while (*p == '/') + { + p++; + } + + len = 0; + while (p[len] != '\0' && p[len] != '/') + { + len++; + } + + if (len == 0) + { + /* Nothing further, so what was found last time is the leaf */ + + return OK; + } + + /* Is this the last component? Anything after it is separators only */ + + if (strspn(p + len, "/") == strlen(p + len)) + { + ret = xipfs_component_ok(p, len); + if (ret < 0) + { + return ret; + } + + *leaf = p; + *leaflen = len; + return OK; + } + + /* An interior component, which has to be an existing directory */ + + ext = xipfs_meta_find(fs, *parent, p, len); + if (ext == NULL) + { + return -ENOENT; + } + + if (!ext->isdir) + { + return -ENOTDIR; + } + + *parent = ext->id; + p += len; + } +} + +/**************************************************************************** + * Name: xipfs_lookup + * + * Description: + * Resolve a whole path. On success *ext is the entry, or NULL for the + * root, which has no record of its own. + * + * The caller must hold the lock. + * + ****************************************************************************/ + +static int xipfs_lookup(FAR struct xipfs_mount_s *fs, + FAR const char *relpath, + FAR struct xipfs_extent_s **ext) +{ + FAR const char *leaf; + size_t leaflen; + uint16_t parent; + int ret; + + ret = xipfs_walk(fs, relpath, &parent, &leaf, &leaflen); + if (ret < 0) + { + return ret; + } + + if (leaflen == 0) + { + *ext = NULL; /* The root */ + return OK; + } + + *ext = xipfs_meta_find(fs, parent, leaf, leaflen); + return *ext != NULL ? OK : -ENOENT; +} + +/**************************************************************************** + * Name: xipfs_readdata + * + * Description: + * Read from an extent's data at a byte offset. When the media is memory + * mapped this is a straight memcpy from flash; otherwise it falls back to + * block reads. + * + ****************************************************************************/ + +static int xipfs_readdata(FAR struct xipfs_mount_s *fs, + FAR struct xipfs_extent_s *ext, off_t offset, + FAR uint8_t *buffer, size_t nbytes) +{ + FAR uint8_t *addr; + uint32_t block; + uint32_t blkoff; + size_t chunk; + int ret; + + addr = xipfs_flash_addr(fs, ext->start_block); + if (addr != NULL) + { + memcpy(buffer, addr + offset, nbytes); + return OK; + } + + while (nbytes > 0) + { + block = ext->start_block + offset / fs->geo.erasesize; + blkoff = offset % fs->geo.erasesize; + chunk = fs->geo.erasesize - blkoff; + + if (chunk > nbytes) + { + chunk = nbytes; + } + + ret = xipfs_flash_read(fs, block, blkoff, buffer, chunk); + if (ret < 0) + { + return ret; + } + + buffer += chunk; + offset += chunk; + nbytes -= chunk; + } + + return OK; +} + +/**************************************************************************** + * Name: xipfs_flushpage + * + * Description: + * Write the accumulated partial page out to flash. Called when the + * buffer fills and once more at close to flush the tail. + * + ****************************************************************************/ + +static int xipfs_flushpage(FAR struct xipfs_mount_s *fs, + FAR struct xipfs_file_s *xf) +{ + FAR struct xipfs_extent_s *ext = xf->ext; + uint32_t block; + uint32_t blkoff; + uint32_t relblk; + int ret; + + if (xf->wrpos == 0) + { + return OK; + } + + /* NOR is programmed in whole pages, so pad the tail with the erased + * value rather than leaving it undefined. + */ + + if (xf->wrpos < fs->geo.blocksize) + { + memset(xf->wrbuf + xf->wrpos, 0xff, fs->geo.blocksize - xf->wrpos); + } + + relblk = xf->written / fs->geo.erasesize; + block = ext->start_block + relblk; + blkoff = xf->written % fs->geo.erasesize; + + /* Under a greedy reservation the run was not erased when it was taken, so + * each block has to be erased just before its first page is programmed. + * Writes are sequential, so this advances one block at a time. + */ + + if (relblk >= xf->erased) + { + ret = xipfs_flash_erase(fs, ext->start_block + xf->erased, + relblk - xf->erased + 1); + if (ret < 0) + { + return ret; + } + + xf->erased = relblk + 1; + } + + ret = xipfs_flash_write(fs, block, blkoff, xf->wrbuf, fs->geo.blocksize); + if (ret < 0) + { + return ret; + } + + xf->written += fs->geo.blocksize; + xf->wrpos = 0; + return OK; +} + +/**************************************************************************** + * Name: xipfs_reserve + * + * Description: + * Reserve the extent for a file being created. + * + * When the size is known up front -- which is the normal case for a + * downloaded module, and what ftruncate communicates -- exactly that many + * blocks are reserved and the file can never fragment. When it is not, + * the largest free run is reserved greedily and trimmed back at close. + * Either way the file occupies one contiguous run for its whole life. + * + ****************************************************************************/ + +static int xipfs_reserve(FAR struct xipfs_mount_s *fs, + FAR struct xipfs_file_s *xf, uint32_t nblocks, + bool greedy) +{ + FAR struct xipfs_extent_s *ext = xf->ext; + uint32_t start; + int ret; + + ret = xipfs_alloc(fs, nblocks, &start); + if (ret < 0) + { + return ret; + } + + /* A greedy reservation covers the largest free run, which on an empty + * filesystem is very nearly the whole partition. Erasing all of it here + * would cost time proportional to the free space rather than to the file + * -- seconds, with interrupts disabled and the other core parked -- for a + * file that may turn out to be 2 KB. Erase lazily instead, one block + * ahead of the writer, and let the trim at close hand back the tail that + * was never touched. + * + * When the size is known the reservation is already exact, so there is + * nothing to gain by deferring: erase it up front and be done. + */ + + if (greedy) + { + xf->erased = 0; + } + else + { + ret = xipfs_flash_erase(fs, start, nblocks); + if (ret < 0) + { + xipfs_free(fs, start, nblocks); + return ret; + } + + xf->erased = nblocks; + } + + ext->start_block = start; + ext->nblocks = nblocks; + xf->greedy = greedy; + + return OK; +} + +/**************************************************************************** + * Name: xipfs_finalize + * + * Description: + * Complete a create: flush the tail page, trim any greedy reservation + * down to what was actually written, and commit. + * + ****************************************************************************/ + +static int xipfs_finalize(FAR struct xipfs_mount_s *fs, + FAR struct xipfs_file_s *xf) +{ + FAR struct xipfs_extent_s *ext = xf->ext; + uint32_t needed; + int ret; + + ret = xipfs_flushpage(fs, xf); + if (ret < 0) + { + return ret; + } + + needed = xipfs_blocksneeded(fs, ext->size); + + if (needed < ext->nblocks) + { + /* Give back the tail of a greedy reservation. The extent keeps its + * start, so it stays exactly as contiguous as it was. + */ + + xipfs_free(fs, ext->start_block + needed, ext->nblocks - needed); + ext->nblocks = needed; + } + + ext->writing = false; + xf->writing = false; + + ret = xipfs_meta_commit(fs); + if (ret < 0) + { + return ret; + } + + xipfs_alloc_rebuild(fs); + return OK; +} + +/**************************************************************************** + * Name: xipfs_release + * + * Description: + * Drop one open reference, freeing an unlinked extent once nothing at all + * references it any more. + * + ****************************************************************************/ + +static void xipfs_release(FAR struct xipfs_mount_s *fs, + FAR struct xipfs_extent_s *ext) +{ + DEBUGASSERT(ext->openrefs > 0); + ext->openrefs--; + + if (ext->unlinked && ext->openrefs == 0 && ext->pincount == 0) + { + xipfs_free(fs, ext->start_block, ext->nblocks); + kmm_free(ext); + } +} + +/**************************************************************************** + * Name: xipfs_open + ****************************************************************************/ + +static int xipfs_open(FAR struct file *filep, FAR const char *relpath, + int oflags, mode_t mode) +{ + FAR struct xipfs_mount_s *fs; + FAR struct xipfs_extent_s *ext; + FAR struct xipfs_file_s *xf; + FAR const char *leaf; + size_t leaflen; + uint16_t parent; + int ret; + + fs = filep->f_inode->i_private; + + ret = xipfs_lock(fs); + if (ret < 0) + { + return ret; + } + + ret = xipfs_walk(fs, relpath, &parent, &leaf, &leaflen); + if (ret < 0) + { + goto errout_with_lock; + } + + if (leaflen == 0) + { + ret = -EISDIR; /* The mountpoint itself */ + goto errout_with_lock; + } + + ext = xipfs_meta_find(fs, parent, leaf, leaflen); + + if (ext != NULL && ext->isdir) + { + ret = -EISDIR; + goto errout_with_lock; + } + + if ((oflags & O_WRONLY) != 0 || (oflags & O_RDWR) != 0) + { + /* Files are write once. Reopening an existing file for writing is + * not an append or an overwrite here -- it has no meaning under the + * immutability assumption, so it is refused rather than silently + * doing something surprising. + */ + + if (ext != NULL) + { + if ((oflags & O_TRUNC) == 0 || (oflags & O_CREAT) == 0) + { + ret = -EACCES; + goto errout_with_lock; + } + + /* O_CREAT|O_TRUNC on an existing file means "replace it". Detach + * the old extent; it is freed once nothing references it. + */ + + if (ext->pincount > 0) + { + ret = -EBUSY; + goto errout_with_lock; + } + + xipfs_meta_detach(fs, ext); + if (ext->openrefs == 0) + { + xipfs_free(fs, ext->start_block, ext->nblocks); + kmm_free(ext); + } + + ext = NULL; + } + + if ((oflags & O_CREAT) == 0) + { + ret = -ENOENT; + goto errout_with_lock; + } + + ext = xipfs_meta_add(fs, parent, leaf, leaflen, false, 0, 0, 0); + if (ext == NULL) + { + ret = -ENOMEM; + goto errout_with_lock; + } + + ext->writing = true; + } + else + { + if (ext == NULL) + { + ret = -ENOENT; + goto errout_with_lock; + } + + if (ext->writing) + { + ret = -EBUSY; + goto errout_with_lock; + } + } + + xf = kmm_zalloc(sizeof(struct xipfs_file_s)); + if (xf == NULL) + { + ret = -ENOMEM; + goto errout_with_extent; + } + + xf->ext = ext; + xf->writing = ext->writing; + + if (xf->writing) + { + xf->wrbuf = kmm_malloc(fs->geo.blocksize); + if (xf->wrbuf == NULL) + { + kmm_free(xf); + ret = -ENOMEM; + goto errout_with_extent; + } + } + + ext->openrefs++; + filep->f_priv = xf; + filep->f_pos = 0; + + xipfs_unlock(fs); + return OK; + +errout_with_extent: + if (ext != NULL && ext->writing) + { + xipfs_meta_detach(fs, ext); + kmm_free(ext); + } + +errout_with_lock: + xipfs_unlock(fs); + return ret; +} + +/**************************************************************************** + * Name: xipfs_close + ****************************************************************************/ + +static int xipfs_close(FAR struct file *filep) +{ + FAR struct xipfs_mount_s *fs; + FAR struct xipfs_file_s *xf; + int ret; + + fs = filep->f_inode->i_private; + xf = filep->f_priv; + + ret = xipfs_lock(fs); + if (ret < 0) + { + return ret; + } + + if (xf->writing) + { + ret = xipfs_finalize(fs, xf); + if (ret < 0) + { + /* The create did not become durable. Discard it entirely rather + * than leaving a half written file visible. Detaching marks the + * extent unlinked; xipfs_release below is what actually reclaims + * its blocks once this last reference drops, so freeing here as + * well would hand the same run back twice. + */ + + ferr("ERROR: Failed to finalize '%s': %d\n", xf->ext->name, ret); + xipfs_meta_detach(fs, xf->ext); + } + } + + if (xf->wrbuf != NULL) + { + kmm_free(xf->wrbuf); + } + + xipfs_release(fs, xf->ext); + kmm_free(xf); + filep->f_priv = NULL; + + xipfs_unlock(fs); + return ret; +} + +/**************************************************************************** + * Name: xipfs_read + ****************************************************************************/ + +static ssize_t xipfs_read(FAR struct file *filep, FAR char *buffer, + size_t buflen) +{ + FAR struct xipfs_mount_s *fs; + FAR struct xipfs_file_s *xf; + FAR struct xipfs_extent_s *ext; + size_t remaining; + int ret; + + fs = filep->f_inode->i_private; + xf = filep->f_priv; + + ret = xipfs_lock(fs); + if (ret < 0) + { + return ret; + } + + ext = xf->ext; + + if (ext->writing) + { + xipfs_unlock(fs); + return -EBUSY; + } + + if (filep->f_pos >= (off_t)ext->size) + { + xipfs_unlock(fs); + return 0; + } + + remaining = ext->size - filep->f_pos; + if (buflen > remaining) + { + buflen = remaining; + } + + ret = xipfs_readdata(fs, ext, filep->f_pos, (FAR uint8_t *)buffer, + buflen); + if (ret < 0) + { + xipfs_unlock(fs); + return ret; + } + + filep->f_pos += buflen; + + xipfs_unlock(fs); + return buflen; +} + +/**************************************************************************** + * Name: xipfs_write + * + * Description: + * Append to a file being created. Sequential only: this is the whole + * write model, not a subset of a richer one. + * + ****************************************************************************/ + +static ssize_t xipfs_write(FAR struct file *filep, FAR const char *buffer, + size_t buflen) +{ + FAR struct xipfs_mount_s *fs; + FAR struct xipfs_file_s *xf; + FAR struct xipfs_extent_s *ext; + size_t written = 0; + size_t chunk; + int ret; + + fs = filep->f_inode->i_private; + xf = filep->f_priv; + + ret = xipfs_lock(fs); + if (ret < 0) + { + return ret; + } + + ext = xf->ext; + + if (!xf->writing) + { + /* Files are immutable once closed */ + + ret = -EACCES; + goto errout_with_lock; + } + + if (filep->f_pos != (off_t)ext->size) + { + ferr("ERROR: xipfs supports sequential writes only\n"); + ret = -EINVAL; + goto errout_with_lock; + } + + /* No extent yet means the size was never declared. Reserve the largest + * run available and trim it back at close. + */ + + if (ext->nblocks == 0) + { + uint32_t largest = xipfs_alloc_largestrun(fs); + + if (largest == 0) + { + ret = -ENOSPC; + goto errout_with_lock; + } + + ret = xipfs_reserve(fs, xf, largest, true); + if (ret < 0) + { + goto errout_with_lock; + } + } + + while (written < buflen) + { + if (ext->size + 1 > ext->nblocks * fs->geo.erasesize) + { + ret = -ENOSPC; + break; + } + + chunk = fs->geo.blocksize - xf->wrpos; + if (chunk > buflen - written) + { + chunk = buflen - written; + } + + if (chunk > ext->nblocks * fs->geo.erasesize - ext->size) + { + chunk = ext->nblocks * fs->geo.erasesize - ext->size; + } + + memcpy(xf->wrbuf + xf->wrpos, buffer + written, chunk); + xf->wrpos += chunk; + ext->size += chunk; + written += chunk; + + if (xf->wrpos == fs->geo.blocksize) + { + ret = xipfs_flushpage(fs, xf); + if (ret < 0) + { + break; + } + } + } + + if (written == 0 && ret < 0) + { + goto errout_with_lock; + } + + filep->f_pos = ext->size; + + xipfs_unlock(fs); + return written; + +errout_with_lock: + xipfs_unlock(fs); + return ret; +} + +/**************************************************************************** + * Name: xipfs_seek + ****************************************************************************/ + +static off_t xipfs_seek(FAR struct file *filep, off_t offset, int whence) +{ + FAR struct xipfs_mount_s *fs; + FAR struct xipfs_file_s *xf; + off_t pos; + int ret; + + fs = filep->f_inode->i_private; + xf = filep->f_priv; + + ret = xipfs_lock(fs); + if (ret < 0) + { + return ret; + } + + switch (whence) + { + case SEEK_SET: + pos = offset; + break; + + case SEEK_CUR: + pos = filep->f_pos + offset; + break; + + case SEEK_END: + pos = xf->ext->size + offset; + break; + + default: + xipfs_unlock(fs); + return -EINVAL; + } + + if (pos < 0) + { + xipfs_unlock(fs); + return -EINVAL; + } + + /* Seeking while creating would create a hole, and a hole cannot be + * expressed in a write once, exactly sized extent. + */ + + if (xf->writing && pos != filep->f_pos) + { + xipfs_unlock(fs); + return -EINVAL; + } + + filep->f_pos = pos; + + xipfs_unlock(fs); + return pos; +} + +/**************************************************************************** + * Name: xipfs_truncate + * + * Description: + * Declare the final size of a file being created. This is how a caller + * that already knows how big the module is -- which is the normal case -- + * gets an exactly sized extent instead of a greedy reservation. + * + ****************************************************************************/ + +static int xipfs_truncate(FAR struct file *filep, off_t length) +{ + FAR struct xipfs_mount_s *fs; + FAR struct xipfs_file_s *xf; + int ret; + + fs = filep->f_inode->i_private; + xf = filep->f_priv; + + ret = xipfs_lock(fs); + if (ret < 0) + { + return ret; + } + + if (!xf->writing || xf->ext->size != 0 || xf->ext->nblocks != 0) + { + /* Only meaningful once, before any data has been written */ + + ret = -EINVAL; + goto errout_with_lock; + } + + if (length < 0) + { + ret = -EINVAL; + goto errout_with_lock; + } + + ret = xipfs_reserve(fs, xf, xipfs_blocksneeded(fs, length), false); + +errout_with_lock: + xipfs_unlock(fs); + return ret; +} + +/**************************************************************************** + * Name: xipfs_volume_cmd + * + * Description: + * The commands that act on the volume rather than on any one file. They + * are reachable both from a descriptor for a file inside the volume and + * from one for the mountpoint directory; only the second is free of the + * side effect that matters, which is that the file used to carry the + * command is itself open, and an open file cannot be relocated. + * + ****************************************************************************/ + +static int xipfs_volume_cmd(FAR struct xipfs_mount_s *fs, int cmd, + unsigned long arg) +{ + switch (cmd) + { + case XIPFSIOC_DEFRAG: + { + FAR struct xipfs_defrag_arg_s *dfg = + (FAR struct xipfs_defrag_arg_s *)((uintptr_t)arg); + + if (dfg == NULL) + { + return -EINVAL; + } + + return xipfs_defrag(fs, dfg->max_ms, &dfg->result); + } + + case XIPFSIOC_LISTPINNED: + { + FAR struct xipfs_pinned_arg_s *pin = + (FAR struct xipfs_pinned_arg_s *)((uintptr_t)arg); + + if (pin == NULL) + { + return -EINVAL; + } + + return xipfs_listpinned(fs, pin->entries, pin->nentries, + &pin->count); + } + +#ifdef CONFIG_FS_XIPFS_FAULT_INJECT + case XIPFSIOC_FAULTINJECT: + { + FAR struct xipfs_fault_s *fault = + (FAR struct xipfs_fault_s *)((uintptr_t)arg); + int ret; + + if (fault == NULL) + { + return -EINVAL; + } + + ret = xipfs_lock(fs); + if (ret < 0) + { + return ret; + } + + fs->fault_countdown = fault->count; + fs->fault_mode = fault->mode; + fs->media_dead = false; + + xipfs_unlock(fs); + return OK; + } +#endif + + default: + return -ENOTTY; + } +} + +/**************************************************************************** + * Name: xipfs_ioctldir + * + * Description: + * ioctl method for a descriptor on the mountpoint directory, obtained + * with open(mountpoint, O_RDONLY | O_DIRECTORY). This is the route the + * volume commands want: no file inside the volume is open, so nothing is + * held immovable by the act of asking, and a compaction pass can reach + * every extent. The directory itself is not needed -- the volume comes + * from the mountpoint inode -- so dir is unused. + * + ****************************************************************************/ + +static int xipfs_ioctldir(FAR struct inode *mountpt, + FAR struct fs_dirent_s *dir, + int cmd, unsigned long arg) +{ + FAR struct xipfs_mount_s *fs; + + UNUSED(dir); + + DEBUGASSERT(mountpt != NULL && mountpt->i_private != NULL); + fs = mountpt->i_private; + + return xipfs_volume_cmd(fs, cmd, arg); +} + +/**************************************************************************** + * Name: xipfs_ioctl + ****************************************************************************/ + +static int xipfs_ioctl(FAR struct file *filep, int cmd, unsigned long arg) +{ + FAR struct xipfs_mount_s *fs; + FAR struct xipfs_file_s *xf; + int ret; + + fs = filep->f_inode->i_private; + xf = filep->f_priv; + + switch (cmd) + { + case XIPFSIOC_PIN: + { + /* Pin the extent and report its flash address. + * + * A module loader needs this rather than mmap because the + * lifetime is wrong for mmap: the mapping would be recorded + * against whichever task happened to call the loader, but the + * module lives until the *spawned* task exits, and the release + * therefore has to happen from a different task entirely. An + * explicit pin expresses that ownership directly. + */ + + FAR uintptr_t *addrp = (FAR uintptr_t *)((uintptr_t)arg); + FAR uint8_t *addr; + + if (addrp == NULL) + { + return -EINVAL; + } + + ret = xipfs_lock(fs); + if (ret < 0) + { + return ret; + } + + addr = xipfs_flash_addr(fs, xf->ext->start_block); + if (addr == NULL || xf->ext->writing) + { + xipfs_unlock(fs); + return -ENXIO; + } + + xf->ext->pincount++; + *addrp = (uintptr_t)addr; + + xipfs_unlock(fs); + return OK; + } + + case XIPFSIOC_UNPIN: + { + ret = xipfs_lock(fs); + if (ret < 0) + { + return ret; + } + + if (xf->ext->pincount == 0) + { + xipfs_unlock(fs); + return -EINVAL; + } + + xf->ext->pincount--; + + xipfs_unlock(fs); + return OK; + } + + case XIPFSIOC_EXTENTINFO: + { + FAR struct xipfs_extent_info_s *info = + (FAR struct xipfs_extent_info_s *)((uintptr_t)arg); + FAR uint8_t *addr; + + if (info == NULL) + { + return -EINVAL; + } + + ret = xipfs_lock(fs); + if (ret < 0) + { + return ret; + } + + addr = xipfs_flash_addr(fs, xf->ext->start_block); + + info->start_block = xf->ext->start_block; + info->nblocks = xf->ext->nblocks; + info->erasesize = fs->geo.erasesize; + info->size = xf->ext->size; + info->pincount = xf->ext->pincount; + info->data_start = fs->data_start; + info->data_nblocks = fs->data_nblocks; + info->xipaddr = (uintptr_t)addr; + + xipfs_unlock(fs); + return OK; + } + + case FIOC_FILEPATH: + return -ENOTTY; + + default: + + /* Not one of the per-file commands. The volume commands are also + * accepted here, for a caller that has a file open and nothing + * else, but the mountpoint directory is the better route: it does + * not leave a file open across the operation. + */ + + return xipfs_volume_cmd(fs, cmd, arg); + } +} + +/**************************************************************************** + * Name: xipfs_sync + ****************************************************************************/ + +static int xipfs_sync(FAR struct file *filep) +{ + /* Every committed operation is already durable when it returns; there is + * no write back cache to flush. + */ + + UNUSED(filep); + return OK; +} + +/**************************************************************************** + * Name: xipfs_dup + ****************************************************************************/ + +static int xipfs_dup(FAR const struct file *oldp, FAR struct file *newp) +{ + FAR struct xipfs_mount_s *fs; + FAR struct xipfs_file_s *oldxf; + FAR struct xipfs_file_s *newxf; + int ret; + + fs = newp->f_inode->i_private; + oldxf = oldp->f_priv; + + if (oldxf->writing) + { + /* Two descriptors appending to one write once file has no coherent + * meaning. + */ + + return -EACCES; + } + + ret = xipfs_lock(fs); + if (ret < 0) + { + return ret; + } + + newxf = kmm_zalloc(sizeof(struct xipfs_file_s)); + if (newxf == NULL) + { + xipfs_unlock(fs); + return -ENOMEM; + } + + newxf->ext = oldxf->ext; + newxf->ext->openrefs++; + newp->f_priv = newxf; + + xipfs_unlock(fs); + return OK; +} + +/**************************************************************************** + * Name: xipfs_fstat + ****************************************************************************/ + +static int xipfs_fstat(FAR const struct file *filep, FAR struct stat *buf) +{ + FAR struct xipfs_mount_s *fs; + FAR struct xipfs_file_s *xf; + int ret; + + fs = filep->f_inode->i_private; + xf = filep->f_priv; + + ret = xipfs_lock(fs); + if (ret < 0) + { + return ret; + } + + memset(buf, 0, sizeof(struct stat)); + buf->st_mode = S_IFREG | 0444; + buf->st_size = xf->ext->size; + buf->st_blksize = fs->geo.erasesize; + buf->st_blocks = xf->ext->nblocks; + + xipfs_unlock(fs); + return OK; +} + +/**************************************************************************** + * Name: xipfs_opendir + ****************************************************************************/ + +static int xipfs_opendir(FAR struct inode *mountpt, FAR const char *relpath, + FAR struct fs_dirent_s **dir) +{ + FAR struct xipfs_mount_s *fs = mountpt->i_private; + FAR struct xipfs_extent_s *ext; + FAR struct xipfs_dir_s *xdir; + int ret; + + ret = xipfs_lock(fs); + if (ret < 0) + { + return ret; + } + + ret = xipfs_lookup(fs, relpath, &ext); + if (ret >= 0 && ext != NULL && !ext->isdir) + { + ret = -ENOTDIR; + } + + xipfs_unlock(fs); + + if (ret < 0) + { + return ret; + } + + xdir = kmm_zalloc(sizeof(struct xipfs_dir_s)); + if (xdir == NULL) + { + return -ENOMEM; + } + + xdir->dirid = ext != NULL ? ext->id : XIPFS_ROOT_ID; + + *dir = &xdir->base; + return OK; +} + +/**************************************************************************** + * Name: xipfs_closedir + ****************************************************************************/ + +static int xipfs_closedir(FAR struct inode *mountpt, + FAR struct fs_dirent_s *dir) +{ + UNUSED(mountpt); + kmm_free(dir); + return OK; +} + +/**************************************************************************** + * Name: xipfs_readdir + ****************************************************************************/ + +static int xipfs_readdir(FAR struct inode *mountpt, + FAR struct fs_dirent_s *dir, + FAR struct dirent *entry) +{ + FAR struct xipfs_mount_s *fs; + FAR struct xipfs_dir_s *xdir; + FAR struct xipfs_extent_s *ext; + uint32_t i; + int ret; + + fs = mountpt->i_private; + xdir = (FAR struct xipfs_dir_s *)dir; + + ret = xipfs_lock(fs); + if (ret < 0) + { + return ret; + } + + /* Resume where the last call stopped and report the next entry this + * directory holds. The list is in no particular order, so neither is a + * directory listing. + */ + + ext = fs->extents; + for (i = 0; i < xdir->index && ext != NULL; i++) + { + ext = ext->flink; + } + + for (; ext != NULL; ext = ext->flink, xdir->index++) + { + if (ext->parent != xdir->dirid) + { + continue; + } + + entry->d_type = ext->isdir ? DTYPE_DIRECTORY : DTYPE_FILE; + strlcpy(entry->d_name, ext->name, sizeof(entry->d_name)); + xdir->index++; + + xipfs_unlock(fs); + return OK; + } + + xipfs_unlock(fs); + return -ENOENT; +} + +/**************************************************************************** + * Name: xipfs_rewinddir + ****************************************************************************/ + +static int xipfs_rewinddir(FAR struct inode *mountpt, + FAR struct fs_dirent_s *dir) +{ + UNUSED(mountpt); + ((FAR struct xipfs_dir_s *)dir)->index = 0; + return OK; +} + +/**************************************************************************** + * Name: xipfs_bind + ****************************************************************************/ + +static int xipfs_bind(FAR struct inode *driver, FAR const void *data, + FAR void **handle) +{ + FAR struct xipfs_mount_s *fs; + FAR const char *options = data; + bool autoformat; + int ret; + + if (driver == NULL || handle == NULL) + { + return -EINVAL; + } + + if (!INODE_IS_MTD(driver) || driver->u.i_mtd == NULL) + { + ferr("ERROR: xipfs requires an MTD driver\n"); + return -ENODEV; + } + + autoformat = options != NULL && + strstr(options, "autoformat") != NULL; + + fs = kmm_zalloc(sizeof(struct xipfs_mount_s)); + if (fs == NULL) + { + return -ENOMEM; + } + + nxrmutex_init(&fs->lock); + fs->driver = driver; + fs->mtd = driver->u.i_mtd; + +#ifdef CONFIG_FS_XIPFS_FAULT_INJECT + fs->fault_countdown = -1; + fs->fault_mode = XIPFS_FAULT_CLEAN; +#endif + + ret = xipfs_flash_probe(fs); + if (ret < 0) + { + goto errout_with_fs; + } + + fs->entries_per_blk = fs->geo.blocksize / XIPFS_DIRENT_SIZE; + fs->max_entries = (fs->geo.erasesize / fs->geo.blocksize - 1) * + fs->entries_per_blk; + + if (fs->entries_per_blk == 0) + { + ferr("ERROR: blocksize %" PRIu32 " is too small for a dirent\n", + fs->geo.blocksize); + ret = -EINVAL; + goto errout_with_fs; + } + + /* Both scratch buffers are reserved once, here. In particular the defrag + * staging buffer must never be allocated at the moment defrag runs: that + * would turn a transient memory shortage into a relocation that cannot + * finish. + */ + + fs->metabuf = kmm_malloc(fs->geo.blocksize); + fs->stage = kmm_malloc(fs->geo.blocksize); + + if (fs->metabuf == NULL || fs->stage == NULL) + { + ret = -ENOMEM; + goto errout_with_buffers; + } + + ret = xipfs_meta_mount(fs); + if (ret == -EFTYPE && autoformat) + { + finfo("xipfs: no valid filesystem found, formatting\n"); + ret = xipfs_meta_format(fs); + } + + if (ret < 0) + { + goto errout_with_buffers; + } + + ret = xipfs_alloc_init(fs); + if (ret < 0) + { + goto errout_with_buffers; + } + + xipfs_alloc_rebuild(fs); + + *handle = fs; + return OK; + +errout_with_buffers: + xipfs_meta_freeextents(fs); + kmm_free(fs->metabuf); + kmm_free(fs->stage); + +errout_with_fs: + nxrmutex_destroy(&fs->lock); + kmm_free(fs); + return ret; +} + +/**************************************************************************** + * Name: xipfs_unbind + ****************************************************************************/ + +static int xipfs_unbind(FAR void *handle, FAR struct inode **driver, + unsigned int flags) +{ + FAR struct xipfs_mount_s *fs = handle; + FAR struct xipfs_extent_s *ext; + int ret; + + ret = xipfs_lock(fs); + if (ret < 0) + { + return ret; + } + + /* Refuse while anything is still mapped. An XIP mapping is a raw pointer + * into flash held by code that has no idea the filesystem is going away, + * and the extent object behind it must outlive the mapping. + */ + + for (ext = fs->extents; ext != NULL; ext = ext->flink) + { + if (ext->pincount > 0 || ext->openrefs > 0) + { + xipfs_unlock(fs); + return -EBUSY; + } + } + + if (driver != NULL) + { + *driver = fs->driver; + } + + xipfs_meta_freeextents(fs); + kmm_free(fs->bitmap); + kmm_free(fs->metabuf); + kmm_free(fs->stage); + + xipfs_unlock(fs); + nxrmutex_destroy(&fs->lock); + kmm_free(fs); + + return OK; +} + +/**************************************************************************** + * Name: xipfs_statfs + ****************************************************************************/ + +static int xipfs_statfs(FAR struct inode *mountpt, FAR struct statfs *buf) +{ + FAR struct xipfs_mount_s *fs = mountpt->i_private; + int ret; + + ret = xipfs_lock(fs); + if (ret < 0) + { + return ret; + } + + memset(buf, 0, sizeof(struct statfs)); + buf->f_type = XIPFS_SBLK_MAGIC; + buf->f_bsize = fs->geo.erasesize; + buf->f_blocks = fs->data_nblocks; + buf->f_bfree = xipfs_alloc_freeblocks(fs); + buf->f_bavail = buf->f_bfree; + buf->f_files = fs->nextents; + buf->f_ffree = fs->max_entries - fs->nextents; + buf->f_namelen = XIPFS_NAME_MAX; + + xipfs_unlock(fs); + return OK; +} + +/**************************************************************************** + * Name: xipfs_unlink + ****************************************************************************/ + +static int xipfs_unlink(FAR struct inode *mountpt, FAR const char *relpath) +{ + FAR struct xipfs_mount_s *fs = mountpt->i_private; + FAR struct xipfs_extent_s *ext; + int ret; + + ret = xipfs_lock(fs); + if (ret < 0) + { + return ret; + } + + ret = xipfs_lookup(fs, relpath, &ext); + if (ret < 0) + { + goto errout_with_lock; + } + + if (ext == NULL || ext->isdir) + { + ret = -EISDIR; + goto errout_with_lock; + } + + if (ext->writing) + { + ret = -EBUSY; + goto errout_with_lock; + } + + /* Detach first, then commit. If the commit fails the extent is put back, + * because the on-media directory is the authority and it still names it. + */ + + xipfs_meta_detach(fs, ext); + + ret = xipfs_meta_commit(fs); + if (ret < 0) + { + ext->unlinked = false; + ext->flink = fs->extents; + fs->extents = ext; + fs->nextents++; + goto errout_with_lock; + } + + /* The blocks are only reusable once nothing references the extent. A + * pinned extent stays exactly where it is until the last mapping goes, + * because a running module is still executing out of it. + */ + + if (ext->openrefs == 0 && ext->pincount == 0) + { + xipfs_free(fs, ext->start_block, ext->nblocks); + kmm_free(ext); + } + +errout_with_lock: + xipfs_unlock(fs); + return ret; +} + +/**************************************************************************** + * Name: xipfs_mkdir + * + * Description: + * Create a directory, which is one record in the next metadata generation + * and nothing else. It costs no data block and no erase in the data + * region, only an entry out of the volume's fixed supply. + * + ****************************************************************************/ + +static int xipfs_mkdir(FAR struct inode *mountpt, FAR const char *relpath, + mode_t mode) +{ + FAR struct xipfs_mount_s *fs = mountpt->i_private; + FAR struct xipfs_extent_s *ext; + FAR const char *leaf; + size_t leaflen; + uint16_t parent; + int ret; + + UNUSED(mode); + + ret = xipfs_lock(fs); + if (ret < 0) + { + return ret; + } + + ret = xipfs_walk(fs, relpath, &parent, &leaf, &leaflen); + if (ret < 0) + { + goto errout_with_lock; + } + + if (leaflen == 0) + { + ret = -EEXIST; /* The root is always there */ + goto errout_with_lock; + } + + if (xipfs_meta_find(fs, parent, leaf, leaflen) != NULL) + { + ret = -EEXIST; + goto errout_with_lock; + } + + ext = xipfs_meta_add(fs, parent, leaf, leaflen, true, 0, 0, 0); + if (ext == NULL) + { + ret = -ENOSPC; + goto errout_with_lock; + } + + ret = xipfs_meta_commit(fs); + if (ret < 0) + { + /* The medium is the authority and it does not name the directory, so + * take it back out again. + */ + + xipfs_meta_detach(fs, ext); + kmm_free(ext); + } + +errout_with_lock: + xipfs_unlock(fs); + return ret; +} + +/**************************************************************************** + * Name: xipfs_rmdir + ****************************************************************************/ + +static int xipfs_rmdir(FAR struct inode *mountpt, FAR const char *relpath) +{ + FAR struct xipfs_mount_s *fs = mountpt->i_private; + FAR struct xipfs_extent_s *ext; + int ret; + + ret = xipfs_lock(fs); + if (ret < 0) + { + return ret; + } + + ret = xipfs_lookup(fs, relpath, &ext); + if (ret < 0) + { + goto errout_with_lock; + } + + if (ext == NULL) + { + ret = -EBUSY; /* The mountpoint itself */ + goto errout_with_lock; + } + + if (!ext->isdir) + { + ret = -ENOTDIR; + goto errout_with_lock; + } + + if (xipfs_meta_haschildren(fs, ext->id)) + { + ret = -ENOTEMPTY; + goto errout_with_lock; + } + + xipfs_meta_detach(fs, ext); + + ret = xipfs_meta_commit(fs); + if (ret < 0) + { + /* Put it back: the committed generation still names it */ + + ext->unlinked = false; + ext->flink = fs->extents; + fs->extents = ext; + fs->nextents++; + goto errout_with_lock; + } + + kmm_free(ext); + +errout_with_lock: + xipfs_unlock(fs); + return ret; +} + +/**************************************************************************** + * Name: xipfs_stat + ****************************************************************************/ + +static int xipfs_stat(FAR struct inode *mountpt, FAR const char *relpath, + FAR struct stat *buf) +{ + FAR struct xipfs_mount_s *fs = mountpt->i_private; + FAR struct xipfs_extent_s *ext; + int ret; + + ret = xipfs_lock(fs); + if (ret < 0) + { + return ret; + } + + memset(buf, 0, sizeof(struct stat)); + + ret = xipfs_lookup(fs, relpath, &ext); + if (ret < 0) + { + xipfs_unlock(fs); + return ret; + } + + if (ext == NULL || ext->isdir) + { + buf->st_mode = S_IFDIR | 0555; + buf->st_blksize = fs->geo.erasesize; + xipfs_unlock(fs); + return OK; + } + + buf->st_mode = S_IFREG | 0444; + buf->st_size = ext->size; + buf->st_blksize = fs->geo.erasesize; + buf->st_blocks = ext->nblocks; + + xipfs_unlock(fs); + return OK; +} diff --git a/include/nuttx/fs/ioctl.h b/include/nuttx/fs/ioctl.h index f6ec263ff3d1a..4533ec6922d05 100644 --- a/include/nuttx/fs/ioctl.h +++ b/include/nuttx/fs/ioctl.h @@ -251,6 +251,37 @@ * OUT: None */ +/* XIPFS specific commands. See include/nuttx/fs/xipfs.h for the argument + * structures. + */ + +#define XIPFSIOC_DEFRAG _FIOC(0x0019) /* IN: FAR struct + * xipfs_defrag_arg_s * + * OUT: Defragmentation result + */ +#define XIPFSIOC_LISTPINNED _FIOC(0x001a) /* IN: FAR struct + * xipfs_pinned_arg_s * + * OUT: Extents holding XIP pins + */ +#define XIPFSIOC_EXTENTINFO _FIOC(0x001b) /* IN: FAR struct + * xipfs_extent_info_s * + * OUT: Physical extent placement + */ + +/* IN: FAR struct xipfs_fault_s * (op countdown and tear mode) + * OUT: None + */ + +#define XIPFSIOC_FAULTINJECT _FIOC(0x001c) + +#define XIPFSIOC_PIN _FIOC(0x001d) /* IN: FAR uintptr_t * + * OUT: Flash address; pins the + * extent in place + */ +#define XIPFSIOC_UNPIN _FIOC(0x001e) /* IN: None + * OUT: Releases one pin + */ + /* NuttX character driver ioctl definitions *********************************/ #define _DIOCVALID(c) (_IOC_TYPE(c)==_DIOCBASE) diff --git a/include/nuttx/fs/xipfs.h b/include/nuttx/fs/xipfs.h new file mode 100644 index 0000000000000..5ecd217dbdf34 --- /dev/null +++ b/include/nuttx/fs/xipfs.h @@ -0,0 +1,168 @@ +/**************************************************************************** + * include/nuttx/fs/xipfs.h + * + * SPDX-License-Identifier: Apache-2.0 + * + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. The + * ASF licenses this file to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance with the + * License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + * + ****************************************************************************/ + +#ifndef __INCLUDE_NUTTX_FS_XIPFS_H +#define __INCLUDE_NUTTX_FS_XIPFS_H + +/**************************************************************************** + * Included Files + ****************************************************************************/ + +#include + +#include +#include + +/**************************************************************************** + * Pre-processor Definitions + ****************************************************************************/ + +/* Maximum length of one path component, not including the NUL terminator. + * Depth comes from the directory an entry belongs to rather than from its + * name, so this bounds a component and not a whole path -- which is what + * statfs reports it as, in f_namelen. + */ + +#define XIPFS_NAME_MAX 31 + +/* Longest path reported back to an application, separators included. A path + * is bounded only by the depth of the tree, so this is a reporting limit and + * not a filesystem one: a path longer than this is truncated from the front, + * which keeps the part that identifies the file. + */ + +#define XIPFS_PATH_MAX 127 + +/* Reason codes returned in struct xipfs_defrag_result_s.reason. + * + * The distinction that matters to the caller is transient (PINS, RAM, + * BUDGET -- retrying later may help) versus permanent (FULL -- it will + * not). The caller normally asked because an allocation returned + * -ENOSPC, so largest_free_run tells it directly whether a retry of that + * allocation can now succeed. + */ + +#define XIPFS_DEFRAG_DONE 0 /* Nothing left to compact */ +#define XIPFS_DEFRAG_BLOCKED_PINS 1 /* Blocked by a live XIP mapping */ +#define XIPFS_DEFRAG_BLOCKED_RAM 2 /* Transient resource shortage */ +#define XIPFS_DEFRAG_TIME_BUDGET 3 /* Ran out of the caller's budget */ +#define XIPFS_DEFRAG_ERROR 4 /* Media error; stopped cleanly */ +#define XIPFS_DEFRAG_BLOCKED_OPEN 5 /* Blocked by a merely open file */ + +/* BLOCKED_PINS and BLOCKED_OPEN are deliberately distinct because the + * caller resolves them differently: a pinned extent needs a module to be + * unloaded, whereas an open one only needs a descriptor to be closed. Issue + * the ioctl on a descriptor for the mountpoint directory to avoid the + * self-inflicted case: a descriptor for a file inside the volume holds that + * file open, and the pass then reports BLOCKED_OPEN for the caller's own + * file. + */ + +/* Fault-injection mode carried in the XIPFSIOC_FAULTINJECT argument. + * + * CLEAN models a power loss that stops the failing write or erase before it + * perturbs the medium at all, leaving the target exactly as it was. TORN + * models the harder real case: an interrupted NOR program leaves the first + * half of a page written and the rest unprogrammed, and an interrupted erase + * leaves the first half of a sector erased and the rest holding old + * contents. A torn generation is what forces the mount-time CRC to do real + * work -- reject a half-formed generation rather than trust it -- which the + * clean model, where an operation either happens whole or not at all, never + * exercises. + */ + +#define XIPFS_FAULT_CLEAN 0 +#define XIPFS_FAULT_TORN 1 + +/**************************************************************************** + * Public Types + ****************************************************************************/ + +/* Outcome of one xipfs defragmentation pass. Returned by the + * XIPFSIOC_DEFRAG ioctl. + */ + +struct xipfs_defrag_result_s +{ + size_t largest_free_run; /* Bytes in the largest contiguous free run */ + uint32_t blocks_reclaimed; /* Erase blocks coalesced into free space */ + uint32_t blocks_pinned; /* Blocks skipped because pin count > 0 */ + uint32_t extents_moved; /* Number of atomic relocations performed */ + int reason; /* One of XIPFS_DEFRAG_* */ +}; + +/* Argument to the XIPFSIOC_DEFRAG ioctl */ + +struct xipfs_defrag_arg_s +{ + uint32_t max_ms; /* 0 means "no time budget" */ + struct xipfs_defrag_result_s result; /* Filled in on return */ +}; + +/* One entry returned by the XIPFSIOC_LISTPINNED ioctl. Without this, + * "blocked by pins" is a dead end the caller cannot resolve; with it the + * application can unload an idle module and re-trigger defrag. + */ + +struct xipfs_pinned_entry_s +{ + char path[XIPFS_PATH_MAX + 1]; /* Relative to the mountpoint */ + uint32_t start_block; + uint32_t nblocks; + uint32_t pincount; +}; + +/* Argument to the XIPFSIOC_LISTPINNED ioctl */ + +struct xipfs_pinned_arg_s +{ + FAR struct xipfs_pinned_entry_s *entries; /* Caller supplied array */ + size_t nentries; /* Capacity of that array */ + size_t count; /* OUT: entries filled in */ +}; + +/* Argument to the XIPFSIOC_EXTENTINFO ioctl. Reports the physical + * placement of the file behind the file descriptor, which is what the + * contiguity invariant tests assert against. + */ + +struct xipfs_extent_info_s +{ + uint32_t start_block; + uint32_t nblocks; + uint32_t erasesize; + uint32_t size; + uint32_t pincount; + uint32_t data_start; /* First block of the data region */ + uint32_t data_nblocks; /* Length of the data region */ + uintptr_t xipaddr; /* Direct flash address, or 0 if not XIP capable */ +}; + +/* Argument to the XIPFSIOC_FAULTINJECT ioctl */ + +struct xipfs_fault_s +{ + int32_t count; /* Operations to allow before failing; negative disables */ + uint8_t mode; /* XIPFS_FAULT_CLEAN or XIPFS_FAULT_TORN */ +}; + +#endif /* __INCLUDE_NUTTX_FS_XIPFS_H */ diff --git a/include/sys/mman.h b/include/sys/mman.h index bedc91c1a1799..c2994a206aeb7 100644 --- a/include/sys/mman.h +++ b/include/sys/mman.h @@ -70,6 +70,16 @@ #define MAP_UNINITIALIZED (1 << 26) /* Bit 26: Do not clear the anonymous pages */ +/* NuttX specific. Requests a strict execute-in-place mapping: the file + * system must resolve the mapping directly onto the memory mapped media or + * fail with ENXIO. It must never fall back to copying the file into RAM, + * even when CONFIG_FS_RAMMAP is enabled for the benefit of other consumers. + * This is what an XIP module loader must use: a silent RAM copy would + * defeat the entire purpose of executing in place. + */ + +#define MAP_XIP_STRICT (1 << 27) /* Bit 27: In-place or fail */ + /* Failure return */ #define MAP_FAILED ((FAR void*)-1) diff --git a/include/sys/statfs.h b/include/sys/statfs.h index a47a7be2970b2..9545070698759 100644 --- a/include/sys/statfs.h +++ b/include/sys/statfs.h @@ -98,6 +98,7 @@ #define CROMFS_MAGIC 0x4d4f5243 #define RPMSGFS_MAGIC 0x54534f47 #define ZIPFS_MAGIC 0x504b +#define XIPFS_MAGIC 0x58495046 #define V9FS_MAGIC 0x01021997 #if defined(CONFIG_FS_LARGEFILE)