[#1570] Add iceoryx2-dmabuf with parallel dmabuf::Service variant by julienzarka · Pull Request #1597 · eclipse-iceoryx/iceoryx2

julienzarka · 2026-05-06T07:41:53Z

[#1570] Add iceoryx2-dmabuf with parallel dmabuf::Service variant

Closes #1570. Supersedes #1572.

Summary

Redesigned version of #1572 incorporating the maintainer feedback that DMA-BUF fd-passing "should be possible to implement as a service variant in iceoryx2."

An architecture spike (commit c4d719284) found that impl iceoryx2::service::Service for dmabuf::Service is not viable on stable Rust 1.85: payload_start_address on cal::SharedMemory is called at an infallible call-site in data_segment.rs:247, PointerOffset is a packed u64 that cannot losslessly encode RawFd + token, and default associated types (type FdConnection) require feature(associated_type_defaults) which is unstable.

dmabuf::Service therefore ships as a parallel construct that embeds Node<ipc::Service> for the control-plane (discovery, dynamic config, monitoring, static storage) and introduces two purpose-built standalone data-plane traits. The public Service trait is untouched; iceoryx2-cal gains no new modules.

What changed vs #1572

#1572 (sidecar)	This PR (parallel service)
`FdSidecarPublisher<S, Meta>`	`DmaBufServicePublisher<Meta>` (typed alias: `DmaBufPublisher<Meta>`)
`FdSidecarToken` in iceoryx2 user-header	Internal token in cal-layer wire frame; user-header is caller-owned
Two ports per service (UDS + iceoryx2)	One uniform `Service::open_or_create` entry point
`iceoryx2::port::side_channel::SideChannel` trait in core	Removed; entirely in `iceoryx2-dmabuf`
~3K LOC standalone crate	~1.5K LOC src + tests + benchmarks
iceoryx2-cal untouched	Still untouched, confirmed by spike

Architecture overview

Full design rationale lives in:

iceoryx2-dmabuf/specs/arch-dmabuf-service-variant.adoc: 8 design decisions (D1 to D8), Mermaid component diagram, post-spike pivot section (why parallel construct, not impl Service).
iceoryx2-dmabuf/specs/spec-dmabuf-service-variant.adoc: numbered requirements with acceptance criteria.

Key invariant. Forward fd channel and back ack channel share one bidirectional UnixStream. The presence of an SCM_RIGHTS ancillary message disambiguates the two frame types:

Forward (publisher to subscriber): [8B len][8B token][SCM_RIGHTS fd]
Back-ack (subscriber to publisher): [8B magic 0x4D4F5346][8B token]

No second socket, no out-of-band signalling.

The two standalone data-plane traits introduced are:

FdBackedSharedMemory (src/shm.rs): fd-backed buffer; mmap on open, munmap on drop. NOT a sub-trait of cal::SharedMemory.
FdPassingConnection (src/connection.rs): fd-passing over a Unix domain socket via SCM_RIGHTS. NOT a sub-trait of cal::ZeroCopyConnection.

Both are #[cfg(target_os = "linux")] with non_linux.rs stubs so the crate compiles on Darwin and other platforms.

Cargo features

Feature	Description	Platform
`default = ["std"]`	iceoryx2 std support	All
`dma-buf`	`DmaBufPublisher` / `DmaBufSubscriber` typed convenience	Linux only
`peercred`	`SO_PEERCRED` UID check on UDS accept	Linux only

dma-buf pulls the dma-buf 0.5 crate. Without it the core DmaBufServicePublisher / DmaBufServiceSubscriber API is still available; the typed DmaBufPublisher<Meta> convenience layer is gated.

Commit structure (15 commits ahead of `upstream/main`)

dd0c18f docs: spec + arch + plan for dmabuf service variant
c4d7192 docs: pivot to parallel dmabuf::Service per Task 0a spike findings
48c805d feat(iceoryx2-dmabuf): add ExternalFdBuffer + FdBackedSharedMemory
f031d79 feat(iceoryx2-dmabuf): add FdPassingConnection standalone trait + Linux impl
a21ec5b fix(iceoryx2-dmabuf): address review findings on connection + shm
891bd2a feat(iceoryx2-dmabuf): add dmabuf::Service parallel construct + port factory
3cc1fb5 feat(iceoryx2-dmabuf): add DmaBufPublisher/Subscriber typed convenience
8fc49e8 feat(iceoryx2-dmabuf): widen wire to v2; add token + back-channel ack
c5c736e test(iceoryx2-dmabuf): migrate fd-identity + heap-roundtrip tests
076bcc6 docs(iceoryx2-dmabuf): add service-variant examples + rewrite README
16ce875 chore(benchmarks): add iceoryx2-benchmarks-dmabuf for service-variant API
7da2d5e docs: mark sidecar specs superseded; update release notes for dmabuf::Service
1485d61 docs(iceoryx2-dmabuf): add PR-MESSAGE-design-c.md (supersedes PR #1572)
d401612 docs: update arch/spec/README/PR-MESSAGE for wire v2 (post-Task-4b)
84b0287 test(iceoryx2-dmabuf): add missing spec-required + migration tests

Each commit compiles and passes cargo clippy -p iceoryx2-dmabuf on its own. git bisect between any two adjacent commits stays green per-crate.

Test evidence

Reviewed by four parallel automated agents (code reviewer, test validator, docs validator, sidecar-vs-new comparator). All blocker findings closed before push. Final test set:

tests/shm_tests.rs drop_munmaps_and_closes_fd, payload_writeable, from_owned_fd_succeeds, ExternalFdBuffer
tests/shm_non_linux_tests.rs non_linux_connection_stub_returns_unsupported (runs on Darwin)
tests/connection_tests.rs send_recv_memfd_roundtrip, fanout_one_pub_three_sub_100_frames,
subscriber_disconnect_publisher_prunes, back_channel_release_roundtrip,
peer_uid_mismatch_rejected (peercred + root),
truncated_frame_returns_error,
protocol_drift_detected_on_unexpected_frame_kind,
size_boundary_roundtrip
tests/service_tests.rs service_open_create_idempotent, publish_receive_memfd_through_dmabuf_service,
meta_user_header_is_callee_owned, service_dropped_subscriber_recovers
tests/it_roundtrip.rs typed_publish_receive_via_memfd_wrapped_in_dmabuf
tests/it_dmabuf_identity.rs fd_identity_preserved_through_roundtrip
tests/it_dmabuf_heap.rs heap_allocation_roundtrip_with_sync_ioctl (skips when /dev/dma_heap/system absent)
src/path.rs::tests 3 unit tests (deterministic, distinct, format)
src/shm/non_linux.rs::tests from_owned_fd_returns_unsupported

All pass on Linux x86_64. Darwin compiles cleanly via non-Linux stubs; Linux-gated tests show 0-run on Darwin, which is expected.

Out of scope

Multi-planar buffers (YUV420 separate Y/UV fds)
Async transport (tokio / async-std executor integration)
sync_file GPU fence fd passing
Windows DXGI handle passing
pidfd_getfd(2) as alternative fd-passing mechanism
Allocator wrappers (DmaBufPool)

Known limitations

Pre-existing workspace clippy failure: iceoryx2-bb-loggers has mutually exclusive buffer / file features that conflict when built with --all-features. This failure pre-dates this PR. Per-crate clippy on iceoryx2-dmabuf is clean (cargo clippy -p iceoryx2-dmabuf --all-features). Same applies to iceoryx2-pal-testing watchdog under --no-default-features. Neither failure is introduced by this PR.

Migration guide for sidecar users

For anyone tracking PR #1572:

Replace FdSidecarPublisher<S, Meta> with DmaBufServicePublisher<Meta>. The S service-type generic is gone; the control-plane is always ipc::Service.
FdSidecarToken in the user-header is removed. The token now lives in the cal-layer wire frame ([8B token] field); your user-header payload is yours again with no reserved fields.
For pool-ack semantics, replace BackChannel / BufferReleased wire with subscriber.release(token) and publisher.recv_release_ack().
For external token assignment (AckLedger-style), use publish_with_token and receive_with_token.

Questions for maintainers

Crate name iceoryx2-dmabuf: keep, or rename to iceoryx2-fd-passing? The transport is generic (any RawFd); DMA-BUF is one application. Renaming now is cheaper than later.
dma-buf feature default: currently opt-in. Should it be on by default given Linux is the primary target?
Future trait surface: once feature(associated_type_defaults) stabilises, should dmabuf::Service be retrofitted to satisfy a new associated-type-default-friendly variant of the Service trait, or should the parallel-construct shape remain indefinitely?
Parallel-construct acceptability: is this shape acceptable as a long-term API, or do you prefer deferring the crate until the Service trait extension surface can accommodate it without UB stubs?

Checklist

Issue exists (#1570)
Branch prefix feat/dmabuf-service-variant
Commit prefix [#1570] or conventional commit
Eclipse copyright header on every new .rs file
Release notes added (doc/release-notes/iceoryx2-unreleased.md)
ECA signed

Refs eclipse-iceoryx#1570

Task 0a (mos:architect) proved impl crate::service::Service for dmabuf::Service is not viable: payload_start_address is called infallibly (UB on fake addr), PointerOffset cannot encode RawFd+token, and adding type FdConnection breaks 4 impl Service sites and the user-extension example. Pivot: dmabuf::Service is a parallel struct embedding ipc::Service for control-plane. FdBackedSharedMemory and FdPassingConnection are standalone traits in iceoryx2-dmabuf/src/ with no cal-layer super-bounds. Port types are concrete (no S generic). iceoryx2-cal is unchanged. Updated files: - arch-dmabuf-service-variant.adoc: new Post-spike pivot section; D1 and D5 revised; Mermaid diagram updated (Service embeds Ipc::Service via delegation); D2 revised (ExternalFdBuffer replaces NoAllocator/ShmAllocator) - spec-dmabuf-service-variant.adoc: Post-spike scope reduction note; §3-7 revised to ExternalFdBuffer; §8-12 FdBackedSharedMemory standalone (shm.rs); §13-21 FdPassingConnection standalone (connection.rs); §23-26 parallel Service struct (service.rs, NOT impl Service); §27-31 concrete port types (no S); commit shape reduced to 4-5 commits; cal acceptance criterion added - plan v1.2: Task 0a marked done with spike evidence (file:line); Task 0a Spike Result section added; Tasks 1-5 revised (cal paths removed); file structure updated (no cal entries); task→commit mapping updated to 4-5 commits Refs eclipse-iceoryx#1570

Bootstraps the iceoryx2-dmabuf crate on the new feat/dmabuf-service-variant branch with two standalone primitives (no iceoryx2-cal coupling per the Task 0a architecture spike): - ExternalFdBuffer: plain (OwnedFd, len) wrapper used as the construction handle for fd-backed SHM. - FdBackedSharedMemory trait + Linux impl: mmaps an externally-supplied fd at construction, munmaps on drop. Payload-ptr exposed for caller- synchronized writes; CPU-sync (DMA_BUF_IOCTL_SYNC) is the caller's responsibility (or via dma_buf::MappedDmaBuf). Refs eclipse-iceoryx#1570

…ux impl Standalone trait (no iceoryx2-cal coupling per Task 0a spike). Linux impl uses Unix-domain socket with SCM_RIGHTS for cross-process fd transfer. Publisher binds + accept-thread, fanout via Mutex<Vec<UnixStream>>, broken-pipe pruning. Subscriber connects + non-blocking poll/recvmsg. Optional SO_PEERCRED check via 'peercred' feature. Wire v1: [8B len LE][8B reserved][SCM_RIGHTS ancillary: 1 fd]. LinuxPublisher / LinuxSubscriber split per SRP; Linux namespace struct provides open_publisher / open_subscriber for ergonomic construction. Refs eclipse-iceoryx#1570 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

- LinuxPublisher::Drop: join accept_thread BEFORE remove_file (avoids EADDRINUSE on rapid restart, ECONNREFUSED on racing connect). - connection::Error: mark #[non_exhaustive] for forward-compat with Task 4b's wider error set. - connection::linux::recv_with_fd: replace unwrap_or-on-Result with proper Truncated error path; introduce HDR_LEN/PAYLOAD_LEN_OFFSET constants for wire offsets. - connection::linux::connected_subscriber_count: propagate LockPoisoned via Result rather than silently returning 0. - shm::non_linux::NonLinux: pub enum {} (uninhabited) so trait methods can match-on-empty rather than panic with unreachable!(). Refs eclipse-iceoryx#1570

…factory dmabuf::Service is a parallel concept (not impl iceoryx2::service::Service) per Task 0a spike findings. Composes: - Node<ipc::Service> for service discovery + lifetime - iceoryx2 PortFactory<ipc::Service, Meta, ()> for metadata channel - FdPassingConnection for fd channel (SCM_RIGHTS over UDS) Service::open_or_create::<Meta>(name) returns DmabufPortFactory<Meta>; publisher_builder() / subscriber_builder() yield the port types. Per-message correlation: publisher sends fd first then iceoryx2 sample; subscriber dequeues iceoryx2 sample then drains fd from socket. No internal token (relies on SPSC ordering of both channels). UDS path derived via DefaultHasher (sha1_smol not available). New path.rs replaces the old sha1_smol-based version; service_error.rs is a new dedicated error type for this API surface. Refs eclipse-iceoryx#1570 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…ce over dmabuf::Service DmaBufPublisher / DmaBufSubscriber are thin newtypes wrapping the service-level publisher / subscriber with &dma_buf::DmaBuf ergonomics. Borrow-and-dup via fcntl(F_DUPFD_CLOEXEC) per send so the caller's DmaBuf remains valid after publish(). Buffer length retrieved via fstat (one syscall per frame) because dma_buf::DmaBuf exposes no .size() accessor. Gated on all(target_os = "linux", feature = "dma-buf") (off by default). Re-exports dma_buf::{DmaBuf, MappedDmaBuf} for caller CPU-sync ergonomics. it_roundtrip integration test exercises the typed round-trip via memfd. Refs eclipse-iceoryx#1570 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Wire v2 framing: - forward (pub->sub): [8B len][8B token][SCM_RIGHTS fd] - back (sub->pub): [8B magic 0x4D4F5346 + 0][8B token] Both directions on the same UnixStream; disambiguation by SCM_RIGHTS ancillary presence (forward has it, ack does not). API additions: - FdPassingConnection: send_with_fd now takes token; recv_with_fd returns (fd, len, token); send_release_ack(token) and recv_release_ack() for the back-channel. - Error: new variants BadMagic, ProtocolDrift, WouldBlock. - DmaBufServicePublisher: publish_with_token + recv_release_ack. Token counter uses plain u64 (all callers take &mut self). - DmaBufServiceSubscriber: receive_with_token + release. receive() delegates to receive_with_token, drops token. - DmaBufPublisher / DmaBufSubscriber typed wrappers expose the same. dup_and_stat helper extracted to avoid duplication. Non-linux stub implements new trait methods as UnsupportedPlatform. Tests: back_channel_release_roundtrip added; existing send_with_fd calls updated to token=0. Refs eclipse-iceoryx#1570 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Ported from sidecar branch (feat/dmabuf-sidecar-git-consumable): - it_dmabuf_identity: asserts the kernel st_ino on publisher and subscriber sides matches, proving SCM_RIGHTS transfers the same fd (not a copy). - it_dmabuf_heap: exercises real /dev/dma_heap/system allocation + MappedDmaBuf::read with DMA_BUF_IOCTL_SYNC start/end pairing on cache-incoherent SoCs. Skips with a clear message when the heap device is absent. Refs eclipse-iceoryx#1570 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

End-to-end publisher/subscriber example pair using the typed DmaBufPublisher / DmaBufSubscriber API + dma-heap allocation + MappedDmaBuf::read CPU-sync ioctls. README rewritten for the post-pivot architecture with three usage levels (raw connection, service variant, typed convenience), Cargo-feature matrix, and a sidecar-to-service migration table. Refs eclipse-iceoryx#1570

… API Three benchmarks measuring the typed DmaBufPublisher / DmaBufSubscriber end-to-end: - latency: p50/p95/p99 per-frame round-trip on 4MB memfd payloads. - throughput: sustained frames/second ceiling. - fanout: 1 publisher x 3 subscribers, slowest-consumer p95. Routes through iceoryx2-dmabuf via [workspace.dependencies] so the benchmark stays insulated from path-rewrite churn. Refs eclipse-iceoryx#1570 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…:Service The two original design docs (arch-fd-sidecar.adoc, spec-dmabuf-typed-transport.adoc) gain a Status:: Superseded header pointing to Design C (arch-dmabuf-service-variant.adoc, spec-dmabuf-service-variant.adoc) — kept for historical reference but no longer the canonical design. Release notes for the unreleased iceoryx2 version updated to describe the parallel dmabuf::Service variant (post-Task 0a spike pivot). Also fix four broken rustdoc intra-doc links in iceoryx2-dmabuf that caused `cargo doc --all-features` to fail with -D warnings: - connection.rs: linux::LinuxPublisher / linux::LinuxSubscriber are cfg(target_os = "linux")-gated; replaced with backtick literals. - service.rs: crate::path::uds_path_for_service is private; removed the link brackets. - service_publisher.rs / service_subscriber.rs (3 sites): ServiceError::Connection(Error::UnsupportedPlatform) is a value expression, not a type path; split into two separate links. Refs eclipse-iceoryx#1570 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…pse-iceoryx#1572) Body of the upstream pull request landing the parallel dmabuf::Service variant per Task 0a spike findings. Includes the architecture summary, side-by-side comparison with the sidecar PR eclipse-iceoryx#1572, commit structure, test evidence, migration guide for sidecar users, and the questions-for-maintainers block. Refs eclipse-iceoryx#1570

Closes the doc-<->code drift identified by the four-agent review: - arch D3 + spec §15: pre-v2 wire 'reserved' field corrected to 'token' + back-channel ack frame format documented for both directions and the ancillary-presence disambiguation rule. - arch D1 trait snippet: send_with_fd/recv_with_fd updated with token parameter and return tuple; send_release_ack/recv_release_ack added. - arch D6 port-type snippets: publish/publish_with_token/recv_release_ack on publisher; receive/receive_with_token/release on subscriber. - README.md 'Lowest level' code block: v2 signatures + ack example. Service level block: publish_with_token + receive_with_token added. - PR-MESSAGE: commit count corrected (11 -> 12); '3' -> '4' commit grouping fixed; re-verify-before-push note added; migration table cross-checked against actual subscriber/publisher method names (release(token)/recv_release_ack() confirmed correct). - Plan: Task 4b step 6 checked as done. connection.rs:129 send_with_fd(fd, len, token), :138 recv_with_fd() -> (OwnedFd, u64, u64) service_publisher.rs:118 publish(meta, fd, len), :147 publish_with_token, :193 recv_release_ack service_subscriber.rs:116 receive, :144 receive_with_token, :197 release(token) Refs eclipse-iceoryx#1570

Closes test-coverage gaps surfaced by the four-agent review: Spec-required tests (§12, §21, §31): - shm_tests.rs::drop_munmaps_and_closes_fd - shm/non_linux.rs::tests::from_owned_fd_returns_unsupported (inline unit test) - shm_non_linux_tests.rs::non_linux_connection_stub_returns_unsupported - connection_tests.rs::peer_uid_mismatch_rejected (peercred feature, root-only) - service_tests.rs::meta_user_header_is_callee_owned Migration tests from sidecar branch: - connection_tests.rs::truncated_frame_returns_error (from error_paths) - connection_tests.rs::protocol_drift_detected_on_unexpected_frame_kind - service_tests.rs::service_dropped_subscriber_recovers (from it_service_gone) - connection_tests.rs::size_boundary_roundtrip (from prop_roundtrip) Unit tests (test pyramid base layer): - path.rs::tests::{deterministic,distinct,format} (3 tests) Refs eclipse-iceoryx#1570 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

julienzarka and others added 15 commits May 5, 2026 17:39

docs: spec + arch + plan for dmabuf service variant

dd0c18f

Refs eclipse-iceoryx#1570

Navigation Menu

Uh oh!

[#1570] Add iceoryx2-dmabuf with parallel dmabuf::Service variant - #1597

[#1570] Add iceoryx2-dmabuf with parallel dmabuf::Service variant #1597
julienzarka wants to merge 15 commits into
eclipse-iceoryx:mainfrom
julienzarka:feat/dmabuf-service-variant

julienzarka commented May 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

julienzarka commented May 6, 2026

[#1570] Add iceoryx2-dmabuf with parallel dmabuf::Service variant

Summary

What changed vs #1572

Architecture overview

Cargo features

Commit structure (15 commits ahead of upstream/main)

Test evidence

Out of scope

Known limitations

Migration guide for sidecar users

Questions for maintainers

Checklist

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Commit structure (15 commits ahead of `upstream/main`)