Skip to content

feat(k8s): decouple workspace and namespace names - #1

Draft
dhirajsb wants to merge 11 commits into
derekwaynecarr:feat/rfc-0011-phase3-namespace-per-workspacefrom
dhirajsb:codex/operator-workspace-namespace-map
Draft

feat(k8s): decouple workspace and namespace names#1
dhirajsb wants to merge 11 commits into
derekwaynecarr:feat/rfc-0011-phase3-namespace-per-workspacefrom
dhirajsb:codex/operator-workspace-namespace-map

Conversation

@dhirajsb

@dhirajsb dhirajsb commented Aug 10, 2026 •
edited
Loading

Copy link
Copy Markdown

Summary

Add an optional explicit workspace-to-namespace map for the Kubernetes driver's operator mode.

This keeps application-facing workspace names independent from platform-managed Kubernetes namespace naming conventions. When the map is configured, an unmapped workspace fails closed instead of being treated as a namespace name.

The same resolution is used for sandbox placement and Kubernetes Secret credential storage, preventing the two paths from selecting different namespaces.

Why this is needed

Kubernetes platform operators commonly enforce namespace names through organizational naming, environment, region, or ownership conventions. Those physical namespace names should not become part of an application's workspace API or require workspace renames when platform conventions change.

Label- and file-based operator modes retain their existing one-to-one workspace/namespace behavior. The explicit map is an alternative configuration source for deployments that need decoupling.

Scope

  • Add operator_workspace_namespaces to Kubernetes compute and credential-driver configuration.
  • Resolve mapped namespaces through the existing operator allowlist boundary.
  • Fail closed for unmapped workspaces.
  • Render the map through the existing Helm chart.
  • Add unit, Helm, and configuration documentation coverage.

This PR is intentionally focused on name mapping. It does not change chart topology, namespace lifecycle, or the broader tenant model.

Relationship

Validation

  • cargo fmt --all -- --check
  • cargo test -p openshell-driver-kubernetes — 184 passed
  • cargo test -p openshell-driver-kubernetes-secrets — 22 passed
  • helm lint deploy/helm/openshell
  • helm unittest deploy/helm/openshell — 81 passed
  • Structural Helm comparison against the parent branch:
    • 13 base objects / 13 head objects
    • no object identity changes
    • no field-constraint or YAML-separator findings

derekwaynecarr and others added 11 commits August 7, 2026 17:37
@derekwaynecarr
feat(k8s): add namespace-per-workspace support (RFC 0011 Phase 3)
03e4cdb 
Implement three workspace namespace modes for the Kubernetes compute
driver: shared (default, preserves current single-namespace behavior),
managed (auto-creates/deletes namespaces per workspace), and operator
(pre-provisioned namespaces with dynamic discovery via label selector
or drop-in allowlist file).

Key changes:
- WorkspaceMode enum and namespace resolution in driver config
- Managed namespace lifecycle with ServiceAccount and OpenShift SCC
  annotation propagation
- Cluster-wide sandbox CR watchers for managed/operator modes
- NamespaceValidator (Exact/Prefix/Allowlist) for SA token auth
- Workspace-aware credential secret storage
- Helm ClusterRole for multi-namespace RBAC
- Gateway config, architecture, and reference docs

Signed-off-by: Derek Carr <decarr@redhat.com>
@derekwaynecarr
test(k8s): add e2e tests for workspace namespace modes
485c65f 
Add end-to-end tests for managed and operator workspace modes
introduced in RFC 0011 Phase 3. The managed mode tests verify
namespace creation with correct labels, ServiceAccount provisioning,
sandbox CR placement, and namespace survival with remaining sandboxes.
The operator mode tests verify rejection of unlabeled and nonexistent
namespaces. The positive operator path (sandbox in labeled namespace)
is known to fail due to an RBAC gap and will be addressed separately.

Also fixes Helm 4 compatibility: move SPDX license headers inside
conditional guards in 8 chart templates to prevent empty comment-only
documents, and fix a trailing whitespace trimmer in clusterrole.yaml
that concatenated the license header with apiVersion.

Adds cleanup sweep in with-kube-gateway.sh to remove managed and
operator namespaces before Helm uninstall, and mise tasks for running
each mode independently.

Signed-off-by: Derek Carr <decarr@redhat.com>
@derekwaynecarr
feat(k8s): add operator namespace label watcher
628a4b7 
Spawn a background kube::runtime::watcher in the K8s driver that
watches namespaces matching the configured label selector and populates
the OperatorNamespaceAllowlist at runtime. The driver owns the
allowlist and exposes its Arc so the server can share the same set with
the SA token authenticator.

create_sandbox now gates pod creation on the allowlist in operator
mode — workspaces whose namespace is not yet labeled are rejected at
resource render time rather than silently proceeding. Workspace
lifecycle itself is unaffected; only sandbox (resource) creation is
gated.

Signed-off-by: Derek Carr <decarr@redhat.com>
@derekwaynecarr
fix(k8s): harden operator mode and address review findings
6a067bb 
Close the fail-open gap in operator mode when only
operator_namespace_file is configured: the allowlist is now created
unconditionally in operator mode (fail-closed from startup).

Implement the namespace file watcher using the notify crate, following
the TLS hot-reload pattern (parent-directory watch, 1s debounce,
ConfigMap symlink-swap safe). The file format is a JSON array of
namespace name strings.

Additional fixes from the 10-reviewer audit:
- Change allowlist rejection from InvalidArgument to FailedPrecondition
  so callers know the request may succeed later once the namespace is
  provisioned.
- NamespaceValidator::Allowlist now holds the OperatorNamespaceAllowlist
  newtype instead of a raw Arc<RwLock<BTreeSet>>, eliminating silent
  denial on RwLock poison.
- Verify LABEL_MANAGED_BY and LABEL_GATEWAY_ID ownership before
  deleting a managed namespace.
- Replace fixed 5s sleep in operator e2e test with a 30s poll loop.
- Add Helm validation for workspaceMode values.
- Fix Helm README type column and description for operator fields.
- Add insert/remove methods to OperatorNamespaceAllowlist; label
  watcher now uses them instead of reaching through shared().
- Reject configs with both operator_namespace_label and
  operator_namespace_file set.

Signed-off-by: Derek Carr <decarr@redhat.com>
@derekwaynecarr
feat(k8s): add workspace-level compute driver RPCs and harden RBAC
b9ab04a 
Decouple namespace lifecycle from sandbox lifecycle by adding
EnsureWorkspace/DeleteWorkspace RPCs to the ComputeDriver service.
Namespace creation now happens before credential storage and namespace
deletion happens on workspace delete, fixing credential storage in
managed workspace mode.

- Add EnsureWorkspace and DeleteWorkspace proto RPCs with
  implementations across all compute drivers (K8s managed delegates to
  ensure_namespace/delete_namespace_if_empty; others no-op)
- Wire ensure_workspace into provider create/update/refresh paths so
  the namespace exists before the credential driver writes secrets
- Wire delete_workspace into workspace deletion for cleanup
- Remove delete_namespace_if_empty from sandbox deletion path
- Scope ClusterRole secrets access to non-shared workspace modes
- Add TODO for TLS cert hot-reload in sandbox gRPC client
- Harden e2e tests with control-plane sandbox resolution assertions
- Fix docker image save --platform flag for OCI index manifests

Signed-off-by: Derek Carr <decarr@redhat.com>
@derekwaynecarr
fix(k8s): address re-review findings and add test coverage
c655db1 
- Use server-side apply for TLS secret sync (fixes second sandbox
  creation failure when TLS is enabled)
- Scope gateway-ID label selector unconditionally across all workspace
  modes (fixes operator reads/watches/deletes seeing foreign sandboxes)
- Validate operator allowlist in EnsureWorkspace and DeleteWorkspace
  RPCs (prevents credential writes to namespaces outside the allowlist)
- Extend ClusterRole secrets patch+delete to all non-shared modes with
  credential driver enabled (fixes operator credential storage RBAC)
- Validate namespace ownership on 409 conflict in ensure_namespace
  (prevents adopting unowned namespaces in managed mode)
- Replace delete_namespace_if_empty with unconditional delete_namespace
  letting Kubernetes cascade cleanup (fixes stuck terminating CRs)
- Strengthen NetworkPolicy TODO to cover both managed and operator modes
- Extract selector and ownership logic into testable free functions
- Add unit tests for gateway-ID selectors and namespace ownership
- Add Helm ClusterRole RBAC tests for operator credential driver

Signed-off-by: Derek Carr <decarr@redhat.com>
@derekwaynecarr
ci(k8s): add workspace managed and operator mode e2e to CI
11d9c21 
Wire the existing e2e:kubernetes:workspace-managed and
e2e:kubernetes:workspace-operator mise tasks into the branch-e2e
workflow so they run alongside the other core Kubernetes e2e suites.
Both are gated by run_core_e2e and included in the Core E2E result
gate.

Signed-off-by: Derek Carr <decarr@redhat.com>
@derekwaynecarr
test(k8s): add e2e tests for workspace namespace modes
dfd3306 
Add 7 new e2e tests covering workspace namespace lifecycle, TLS secret
copying, ownership conflict detection, DNS-1123 validation, operator
namespace preservation, and dynamic label watcher behavior. Fix async
sandbox deletion race condition in existing tests by polling sandbox
list instead of asserting immediately after delete.

Signed-off-by: Derek Carr <decarr@redhat.com>
@derekwaynecarr
fix(k8s): grant secrets/patch unconditionally and backfill gateway-id…
2a581bf 
… labels

Address two review findings:

1. RBAC: server-side apply (PATCH) is used for TLS secret sync in
   multi-namespace modes, but the ClusterRole only granted patch when
   the kubernetes-secrets credential driver was enabled. Grant patch
   unconditionally for non-shared modes since TLS sync always needs it;
   keep delete gated on the credential driver.

2. Upgrade safety: the new gateway-id label selector would orphan
   legacy Sandbox CRs that predate its introduction. Add a startup
   backfill in shared mode that patches any managed Sandbox CR missing
   the gateway-id label before the driver begins serving requests.

Signed-off-by: Derek Carr <decarr@redhat.com>
@derekwaynecarr
fix(k8s): address workspace namespace review findings
ddb67ae 
Signed-off-by: Derek Carr <decarr@redhat.com>
@dhirajsb
feat(k8s): map workspaces to operator namespaces
e0df6ce 
Signed-off-by: Dhiraj Bokde <dbokde@nvidia.com>
@dhirajsb
dhirajsb force-pushed the codex/operator-workspace-namespace-map branch from 91346da to e0df6ce Compare August 13, 2026 07:51
@dhirajsb dhirajsb changed the title feat(k8s): map workspaces to operator namespaces feat(k8s): decouple workspace and namespace names Aug 13, 2026
@dhirajsb

dhirajsb commented Aug 13, 2026

Copy link
Copy Markdown
Author

@derekwaynecarr please review if this helps decouple namespace from workspace name 🙏

@derekwaynecarr
derekwaynecarr force-pushed the feat/rfc-0011-phase3-namespace-per-workspace branch 3 times, most recently from bac95bf to 6699658 Compare August 14, 2026 20:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@derekwaynecarr derekwaynecarr Awaiting requested review from derekwaynecarr derekwaynecarr will be requested when the pull request is marked ready for review derekwaynecarr is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@dhirajsb @derekwaynecarr