fix(tx16s): execute USB/SD interrupt-path code from RAM (USB->DFU crash on some MK1 units)

[AdsumSOK]

Problem

On some TX16S MK1 units, selecting USB joystick or mass-storage crashes the radio within seconds to minutes; with the cable plugged it then reboots into DFU. Long-standing reports: #5899, #7356, #6047, #6414, #3039, #4262 - always the same pattern: only some units, appeared around 2.11, 2.10.6 works, one user fixed it by swapping the main board, and the bootloader's own USB storage is immune.

Root cause

Instrumented builds (persistent fault capture across reboots on an affected unit) recorded three crashes:

#	active ISR	fault	stacked PC points at
1	OTG_FS	NOCP	a valid pop in USB_EPStartXfer
2	DMA2_Stream0 (ADC)	UNDEFINSTR	a valid str r3, [sp, #4] in adc_start_read
3	OTG_FS	UNDEFINSTR	a valid and.w in HAL_PCD_IRQHandler

All three faulted while executing provably valid, boundary-aligned instructions located above 0x08100000 (flash bank 2). Stack overflow was ruled out by watermarking (~480B of 22KB used); illegal FreeRTOS-from-ISR calls were ruled out with runtime counters (zero hits).

Conclusion: on marginal units (likely shortage-era flash), instruction fetches from the upper flash bank return corrupted data under heavy IRQ/DMA load (USB + SDIO + LCD/audio DMA). This explains every historical oddity: which units crash (flash margin varies per chip), which versions crash (the linker moves the hot handlers across the bank boundary as the firmware grows), and why the bootloader is immune (its code sits at the very start of bank 1).

Fix

Link the USB device stack (LL/PCD drivers, core, MSC class), the MSC storage glue and the SDIO diskio into a .fastcode output section placed in RAM and copied at startup (~14KB), so the hot interrupt paths never fetch from flash. The affected object libraries are kept out of LTO, since linker file-pattern matching cannot see through LTO partitions.

No functional change; RAM cost ~14KB (192KB part).

Validation

A/B on an affected TX16S MK1 (2022), EdgeTX 2.12.2 base, continuous mass-storage stress reads (md5sum loop over USB from the host):

• hot code in flash: hard fault after ~55 s
• hot code in RAM: 24 minutes clean, zero faults (test stopped, not the radio)

Full evidence

The complete diagnostic work is public on the fork:

• Raw crash captures (persistent fault records: PC/LR/xPSR/CFSR/HFSR, stack watermark, checkpoint trail): bench/logs/crash-avant-c5.log
• Instrumentation used to capture them (fault handlers persisting to noinit RAM + RTC backup registers, dumped to SD at next boot): branch diag/usb-crash-capture
• Test/bench history: bench/logs/bench.log

The three decoded crash records (2.12.2 base, TX16S MK1 2022)

==== crash record ====                     # crash 1
fw: pre-2.12.2-selfbuild (84b52f5e)
[bkp] fault=3 ipsr=3                       # HardFault (forced)
[bkp] pc=0810C836 lr=00000002 xpsr=21000253  # IPSR 0x53 = IRQ67 = OTG_FS
[bkp] cfsr=00080000 hfsr=40000000          # NOCP
last checkpoint: 7 (MSC read done), isr_count=7983
-> pc = valid `pop {..., pc}` in USB_EPStartXfer

==== crash record ====                     # crash 2
fw: pre-2.12.2-selfbuild (bc6b19f6)
[bkp] pc=08106B36 lr=FFFFFFF1 xpsr=81000048  # IPSR 0x48 = IRQ56 = DMA2_Stream0
[bkp] cfsr=00010000 hfsr=40000000 stack_low=1000FE20  # UNDEFINSTR, stack fine
last checkpoint: 6 (MSC read in progress), isr_count=34796
-> pc = valid `str r3, [sp, #4]` in adc_start_read

==== crash record ====                     # crash 3
fw: pre-2.12.2-selfbuild (37870f09)
[bkp] pc=0810A9F6 lr=08109F8B xpsr=81000053  # IRQ67 = OTG_FS
[bkp] cfsr=00010000 hfsr=40000000 stack_low=1000FE58  # UNDEFINSTR
last checkpoint: 7, isr_count=6953
-> pc = valid `and.w fp, fp, r2` in HAL_PCD_IRQHandler (coherent LR)

🤖 Generated with Claude Code

Summary by CodeRabbit

• Performance
  • Improved responsiveness and timing for USB, mass-storage, and SDIO interrupt handling by running critical routines from faster memory.
• Reliability
  • Updated startup behavior safely supports devices with or without the optimized code section.
  • Bootloader builds now preserve the required code layout for consistent operation.

[coderabbitai]

📝 Walkthrough

Walkthrough

Adds a RAM-executed .fastcode linker section for selected STM32 interrupt-path code, preserves section matching by adjusting bootloader LTO options, provides fallback linker symbols, and copies the section into RAM during Reset_Handler.

Changes

STM32 fastcode execution

Layer / File(s)	Summary
Fastcode linker layout and fallback symbols radio/src/boards/generic_stm32/linker/*.ld	Defines the .fastcode section in RAM with a flash load address and provides zero-valued fallback symbols when the section is absent.
Bootloader section matching radio/src/bootloader/CMakeLists.txt	Restricts LTO application so the input sections remain available for .fastcode pattern matching.
Startup fastcode copy radio/src/targets/common/arm/stm32/system_init.c	Copies the linker-defined fastcode range from flash to RAM during Reset_Handler.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Reset_Handler
  participant Flash
  participant RAM
  Reset_Handler->>Flash: Read fastcode_load source
  Flash-->>Reset_Handler: Provide fastcode bytes
  Reset_Handler->>RAM: Copy from sfastcode to efastcode
  RAM-->>Reset_Handler: Fastcode available in RAM

Loading

Suggested labels: bug/regression ↩️

Suggested reviewers: raphaelcoeffic, philmoz, 3djc, gagarinlg

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly matches the main change: moving USB/SD interrupt-path code to RAM to address TX16S MK1 crashes.
Description check	✅ Passed	The description covers the problem, root cause, fix, validation, and supporting evidence required by the template.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[ulph0]

Great work on the diagnosis. I've been tracking the same crash pattern (USB -> hard fault -> DFU reboot on TX16S MK1) independently and traced it to an STM32F429 errata (PA12 noise coupling into Bank2 flash reads — documented in ES0166 §2.4.6). Full analysis and evidence in my issue thread:

• Full analysis: Whilst powered on and inserting USB lead connected to pc, radio powers off every time 2.11.0 RC #5899 (comment) (
  rootcause03 attachment)
• Tested fix on my own MK1: same symptoms, same conclusion

Your RAM-copy approach is clever and much more mergeable than my Bank1
migration. However, I want to flag that it only covers the USB/SDIO ISR
paths. My crash captures also showed faults in the ADC ISR (DMA2_Stream0)
and in code paths that the mixer task hits while USB is active — both
live in Bank2 and can still corrupt when USB traffic is ongoing.

Combining your RAM copy (for ISRs) with a freeze gate for the main loop
and/or moving the mixer hot path to Bank1 would cover all crash vectors.

[AdsumSOK]

Update after further testing on the affected unit — and thanks @ulph0

@ulph0 — thank you, this is an excellent independent confirmation. Your errata reference (ES0206, PA12/USB-DP noise coupling into Bank2 flash reads, fixed in silicon rev 3) supplies the physical mechanism behind everything we measured on our side: valid Bank2 instructions raising UNDEFINSTR/NOCP/precise bus faults only while USB is active, on an early MK1 only. Two independent investigations, same conclusion.

Your warning about coverage was correct, and we can now back it with data. We extended the .fastcode section to the ADC and mixer-scheduler interrupt paths (one of our captured faults was in adc_start_read) and tested on the affected radio — a TX16S MK1 with a 2020-2022-era main board (original firmware from Nov 2022, silicon presumably rev < 3). Result: worse, not better — joystick mode froze hard (thread-mode hang: the watchdog is fed from an ISR, so it never fires) and mass storage crashed within seconds. Taking stm32_drivers out of LTO reshuffled the unprotected flash layout, moving other hot code into the vulnerable region. This confirms the key insight:

• Any Bank2 access during USB activity is a potential crash vector — thread-mode code included.
• ISR-only RAM placement is therefore a layout lottery: each rebuild re-rolls the dice for everything else. Our 24-minute clean validation of the current PR state was partly placement luck; your "I shuffled code around until it worked" experience is the same phenomenon.

The PR is left at the validated state (USB/SDIO interrupt paths in RAM: 24 min continuous MSC stress clean vs ~55 s crash baseline on the same unit). It is a real improvement for mass storage, but it is partial.

Proposed plan for a deterministic fix (follow-up work):

1. Code-in-Bank1 layout: place all executable code in Bank1, and demote cold, error-tolerant assets (fonts, bitmaps — a large share of the 1.6MB image) to Bank2. Guaranteed by the linker script, invariant across builds. We will first measure whether the executable portion fits (<1MB).
2. If tight, combine with @ulph0's complementary approach: minimize thread-mode activity during USB sessions (UI freeze in joystick/serial modes).
3. Validation protocol on affected hardware: both USB modes, multi-hour stress, several rebuilds with intentionally shuffled layout to prove invariance.
4. If maintainers prefer, this could be gated behind a build option (e.g. EARLY_MK1_FLASH_ERRATA) rather than applied to all X10 targets.

Guidance welcome: would you rather merge the current partial-but-validated improvement now and take the deterministic layout as a follow-up PR, or hold this one until the full fix is ready?

[AdsumSOK]

Closing this PR — further testing invalidated the approach as-is.

Additional stress testing on the affected unit showed that a rebuild of the exact same source crashes in mass-storage mode within seconds: the 24-minute clean validation was obtained on a differently-laid-out build, confirming that ISR-only RAM placement leaves the outcome to per-build layout luck (as @ulph0 warned). Merging this would give affected users a false sense of fix.

I'll come back with the deterministic approach outlined above (all executable code constrained to Bank1, cold assets in Bank2, plus UI freeze during USB if needed), validated across multiple intentionally-reshuffled builds on affected hardware. The diagnosis part of this PR (errata-driven Bank2 fetch corruption, crash captures) remains valid and is confirmed by two independent investigations.

[3djc]

Please keep up the great work you are doing here.

[gagarinlg]

Be careful with putting things, especially interrupt handlers, into RAM.
The RAm is way slower than the internal flash, so every access or code execution will be way slower. There are multiple reasons:

1. The RAM has a 16 Bit interface, the flash 32 Bit
2. The clock speed of the RAM is way slower the the clock speed of the flash
3. The bus access takes another path and CPU RAM access collides with a lot of other things. Like the frame buffer for the UI.

[AdsumSOK]

The complete fix discussed above is now up as #7557: all code executed during USB sessions is kept in flash bank 1 by the linker (overflow = link error), UI frozen during sessions, with an MPU fence for validation builds. On the affected MK1 used for the A/B here: stock crashed at 55 s, this PR ran 35 min / 504 MB sha256-verified MSC plus active joystick sessions with the fence armed — zero faults. @ulph0 your freeze-gate + bank-1 suggestion is exactly what it implements.