feat(driver-podman): add userns config

Add a `userns` option to the Podman compute driver that maps to
Podman's user namespace modes.  The mode string is split on the first
colon into the API's `nsmode` and `value` fields so parameterized
values like `auto:size=65536` and `keep-id:uid=1000,gid=1000` are
forwarded correctly.  When the mode is `auto`, the container spec
also sets `idmappings.AutoUserNs = true` as required by the API.

An allowlist validates the mode at startup: `auto` and `keep-id`
accept optional parameters; `host`, `private`, and `nomap` reject
them; everything else is an error.

Podman image volumes use overlay mounts internally and the kernel
does not support idmapped mounts on overlay (`mount_setattr` returns
EINVAL).  When userns is configured (any mode except `host`), the
driver extracts the supervisor binary from the image to a host-side
cache and bind-mounts it instead of using an image volume.

Configurable via TOML `userns = "auto"`, CLI `--userns`, or
environment variable `OPENSHELL_PODMAN_USERNS`.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>

Author: giuseppe

.agents/skills/debug-openshell-cluster/SKILL.md

@@ -216,6 +216,25 @@ Common findings:
   cannot bypass slirp4netns host-loopback isolation. Do not work around
   discovery failures by broadening the primary gateway listener to `0.0.0.0`.
 
+When `userns` is configured (e.g. `userns = "auto"` or `userns = "keep-id"`):
+
+- Supervisor delivery uses bind-mount fallback instead of image volumes because
+  overlay mounts do not support `idmapped` mounts. The supervisor binary is
+  extracted from the supervisor image and cached at
+  `$XDG_DATA_HOME/openshell/podman-supervisor/` (typically
+  `~/.local/share/openshell/podman-supervisor/`).
+- Stale cache: if the supervisor image is updated but the cached binary is not
+  refreshed, sandbox creation may fail with an ELF validation error or version
+  mismatch. Remove the cache directory and retry.
+- `auto` mode requires subuid/subgid ranges for the current user in
+  `/etc/subuid` and `/etc/subgid`. If missing, Podman returns a user-namespace
+  mapping error at container creation.
+- `private` mode requires explicit `uidmap` and `gidmap` arrays in the TOML
+  config (e.g. `uidmap = ["0:1000:1", "1:100000:65536"]`). Without both,
+  the gateway rejects the config at startup.
+- `nomap` (without hyphen) is accepted as input but canonicalized to `no-map`
+  for Podman's API.
+
 ### Step 6: Check Kubernetes Helm Gateways
 
 ```bash

architecture/compute-runtimes.md

@@ -117,7 +117,7 @@ delete, reconciliation removes the row; otherwise it can remain `Deleting`.
 | Runtime | Best fit | Sandbox boundary | Notes |
 |---|---|---|---|
 | Docker | Local development with Docker available. | Container plus nested sandbox namespace. | Uses host networking so loopback gateway endpoints work from the supervisor. |
-| Podman | Rootless or single-machine deployments. | Container plus nested sandbox namespace. | Uses the Podman REST API, OCI image volumes, and CDI GPU devices when available. |
+| Podman | Rootless or single-machine deployments. | Container plus nested sandbox namespace. | Uses the Podman REST API and CDI GPU devices when available. Delivers the supervisor via OCI image volume by default; falls back to extracting the binary to a host-side cache and bind-mounting it when `userns` is configured (overlay does not support idmapped mounts). |
 | Kubernetes | Cluster deployment through Helm. | Pod plus nested sandbox namespace. | Uses Kubernetes API objects, service accounts, secrets, PVC-backed workspace storage, and GPU resources. |
 | VM | Experimental microVM isolation. | Per-sandbox libkrun VM. | Managed endpoint-backed driver. The gateway spawns `openshell-driver-vm`, waits for its Unix socket, and then consumes it through the same remote `compute_driver.proto` path used by unmanaged endpoint drivers. The VM driver boots a cached bootstrap `rootfs.ext4`, prepares requested OCI images inside a bootstrap VM with `umoci`, attaches the prepared image disk read-only, and gives each sandbox a writable `overlay.ext4` for merged-root changes and runtime material. The driver persists each accepted launch request beside the overlay and restarts those VMs on driver startup without recreating the overlay. |
 | Extension | Out-of-tree drivers operated alongside the gateway. | Whatever boundary the driver implements. | Selected by a non-reserved custom `compute_drivers = ["<name>"]` entry with `[openshell.drivers.<name>].socket_path`, or at launch time by pairing `--drivers <name>` with `--compute-driver-socket=<path>`. Reserved built-in names such as `vm`, `docker`, `podman`, and `kubernetes` cannot be used as unmanaged socket endpoints. The gateway connects to a UDS the operator already provisioned, runs `GetCapabilities`, logs the advertised `driver_name`, and dispatches all sandbox lifecycle calls through `compute_driver.proto`. The driver process and socket lifecycle are operator-owned; the gateway does not spawn, supervise, or remove unmanaged extension drivers. The trust boundary is the socket's filesystem permissions: the operator must ensure only the gateway uid can read/write it. |
@@ -170,7 +170,7 @@ The supervisor must be available inside each sandbox workload:
 | Runtime | Delivery model |
 |---|---|
 | Docker | Bind-mounted local supervisor binary, or a binary extracted from the configured supervisor image. |
-| Podman | Read-only OCI image volume containing the supervisor binary. |
+| Podman | Read-only OCI image volume by default; host-cached bind mount when `userns` is configured. |
 | Kubernetes | Supervisor image side-loaded into the sandbox pod by image volume or init container. |
 | VM | Embedded in the guest rootfs bundle. |
 | Extension | Defined by the out-of-tree driver. |

crates/openshell-core/Cargo.toml

@@ -27,8 +27,8 @@ ipnet = "2"
 base64 = { workspace = true }
 chrono = { version = "0.4", default-features = false, features = ["clock", "std"], optional = true }
 reqwest = { workspace = true, features = ["blocking", "rustls-tls-native-roots"], optional = true }
-tar = "0.4"
-tempfile = "3"
+tar = { version = "0.4", optional = true }
+tempfile = { version = "3", optional = true }
 
 [target.'cfg(unix)'.dependencies]
 nix = { workspace = true }
@@ -38,6 +38,7 @@ default = ["telemetry"]
 ## Compile in anonymous telemetry emission support. On by default; disable with
 ## `--no-default-features` (plus any other features you need) for a build that
 ## contains no telemetry endpoint, no HTTP client, and no emission code at all.
+driver-extraction = ["dep:tar", "dep:tempfile"]
 telemetry = ["dep:reqwest", "dep:chrono"]
 
 [build-dependencies]

crates/openshell-core/src/driver_utils.rs

@@ -424,6 +424,7 @@ pub fn supervisor_image_should_refresh(image: &str) -> bool {
 // Supervisor binary extraction helpers (shared by Docker and Podman drivers)
 // ---------------------------------------------------------------------------
 
+#[cfg(feature = "driver-extraction")]
 /// Extract the payload of the first regular-file entry in a tar archive.
 ///
 /// Container archive endpoints return a single-file tar when `path` points to
@@ -454,6 +455,7 @@ pub fn extract_first_tar_entry(tar_bytes: &[u8]) -> Result<Vec<u8>, String> {
     Ok(bytes)
 }
 
+#[cfg(feature = "driver-extraction")]
 /// Atomically write `bytes` to `final_path` via a sibling temp file.
 ///
 /// Creates parent directories as needed. The temp file is synced, `chmod 755`

crates/openshell-driver-docker/Cargo.toml

@@ -11,7 +11,7 @@ license.workspace = true
 repository.workspace = true
 
 [dependencies]
-openshell-core = { path = "../openshell-core", default-features = false }
+openshell-core = { path = "../openshell-core", default-features = false, features = ["driver-extraction"] }
 
 tokio = { workspace = true }
 tonic = { workspace = true }

crates/openshell-driver-podman/Cargo.toml

@@ -15,7 +15,7 @@ name = "openshell-driver-podman"
 path = "src/main.rs"
 
 [dependencies]
-openshell-core = { path = "../openshell-core", default-features = false }
+openshell-core = { path = "../openshell-core", default-features = false, features = ["driver-extraction"] }
 
 tokio = { workspace = true }
 tonic = { workspace = true, features = ["transport"] }

crates/openshell-driver-podman/README.md

@@ -360,6 +360,7 @@ Podman resources after out-of-band container removal or label drift.
 | `OPENSHELL_SANDBOX_PROXY_AUTH_FILE` | `--sandbox-proxy-auth-file` | unset | Path to a file containing the proxy credentials as `user:pass`. Staged as a root-only Podman secret so credentials never appear in config or container metadata. Requires the insecure-auth acknowledgement below. |
 | `OPENSHELL_SANDBOX_PROXY_AUTH_ALLOW_INSECURE` | `--sandbox-proxy-auth-allow-insecure` | unset | Explicit acknowledgement (`true`) that the credential is sent as cleartext Basic auth over the plain-TCP connection to the `http://` proxy. Required when the auth file is set; rejected when it is not. |
 | `OPENSHELL_SANDBOX_PROXY_CONNECT_BY_HOSTNAME` | `--sandbox-proxy-connect-by-hostname` | unset | Send the destination hostname in CONNECT requests instead of a validated IP. Last resort for proxies whose ACLs filter on hostnames: the proxy then resolves the name itself, so sandbox SSRF/`allowed_ips` validation no longer binds the connection. |
+| `OPENSHELL_PODMAN_USERNS` | `--userns` | unset | User namespace mode for sandbox containers (e.g. `auto`). When unset, containers use the default user namespace. |
 
 Through the gateway, the same settings are the `https_proxy`, `no_proxy`,
 `proxy_auth_file`, `proxy_auth_allow_insecure`, and

crates/openshell-driver-podman/src/client.rs

@@ -513,6 +513,32 @@ impl PodmanClient {
         }
     }
 
+    /// Download a file from a container as a tar archive.
+    ///
+    /// Calls `GET /libpod/containers/{name}/archive?path={path}` and returns
+    /// the raw tar bytes. The container does not need to be running.
+    pub async fn copy_from_container(
+        &self,
+        name: &str,
+        path: &str,
+    ) -> Result<Bytes, PodmanApiError> {
+        validate_name(name)?;
+        let encoded_path = url_encode(path);
+        let (status, bytes) = self
+            .request(
+                hyper::Method::GET,
+                &format!("/libpod/containers/{name}/archive?path={encoded_path}"),
+                None,
+                API_TIMEOUT,
+            )
+            .await?;
+        if status.is_success() {
+            Ok(bytes)
+        } else {
+            Err(error_from_response(status.as_u16(), &bytes))
+        }
+    }
+
     /// Inspect a container by name or ID.
     pub async fn inspect_container(&self, name: &str) -> Result<ContainerInspect, PodmanApiError> {
         validate_name(name)?;