Add HFSandboxBackend for the OpenCode harness#998
Conversation
Implements the SandboxBackend/SandboxHandle/BgJob protocol against huggingface_hub.Sandbox so OpenCode rollouts can run on HF Sandbox in addition to E2B. Runtime opencode install works (verified HF-sandbox egress); pre-baked image is a speed optimization. Includes unit tests for HFBgJob.wait() (fake sandbox, no network) and a root-based image Dockerfile mirroring the E2B template.
The transparent_proxy paths (proxy trace/log, proxy source dir) and the opencode binary probe were hardcoded to /home/user, so proxy startup failed on any root-based backend (HF Sandbox, Docker) where sandbox_home=/root. Derive them from config.sandbox_home via new opencode_runtime helpers; E2B (sandbox_home=/home/user) is unchanged.
- OpenCodeSessionFactory.create() now kills the sandbox if proxy start or agent launch fails (previously only bootstrap failures were cleaned up; a failed _start_proxy leaked the sandbox until its idle timeout). - Add opencode_env[hf] extra (huggingface_hub>=1.22) and README notes for the HFSandboxBackend + sandbox_home=/root.
huggingface_hub is already a hard core dependency (>=0.20.0); the HF backend just needs it recent enough for huggingface_hub.Sandbox, so bump the floor here instead of introducing a separate extra.
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull request overview
This PR adds a new sandbox backend for the OpenCode harness (HFSandboxBackend) built on huggingface_hub.Sandbox, and makes the harness’s transparent-proxy + opencode probe paths derive from config.sandbox_home (fixing failures on root-based sandboxes like HF/Docker). It also adds unit coverage for the HF background-job polling behavior and documents how to use the HF backend.
Changes:
- Add
envs/opencode_env/sandbox/hf.pyimplementingHFSandboxBackend/HFSandboxHandle/HFBgJobagainsthuggingface_hub.Sandbox. - Replace hardcoded
/home/userproxy/log/probe paths with helpers based onconfig.sandbox_homeand ensure sandbox teardown on setup failures. - Add HF-specific docs/tests and an optional pre-baked Docker image for faster rollouts; bump
huggingface_hubfloor for the env.
Alignment Review Notes (per skill)
- Automated Checks: Not run in this review environment (no hook execution available here).
- Open RFCs potentially relevant:
- RFC 005 (In Review): Agentic harness integration — this change appears consistent with adding a new provider-backed sandbox implementation.
- RFC 002 (In Review; proposed “Cloud Sandbox Providers” amendment): Provider-neutral sandbox capability mapping — this PR is additive and aligns with the direction of supporting multiple sandbox providers without changing the harness contract.
- Tier 2 flags: None identified from the diffs provided (no invariant violations spotted; no obvious conflicts with the above RFCs).
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/envs/test_opencode_hf_sandbox.py | Adds unit tests for HFBgJob.wait() polling semantics using fakes (no network). |
| envs/opencode_env/sandbox/hf.py | Introduces Hugging Face Sandbox backend + handle + background job polling implementation. |
| envs/opencode_env/sandbox/hf_image/Dockerfile | Adds an optional pre-baked image for HF sandbox runs (opencode + proxy deps). |
| envs/opencode_env/sandbox/init.py | Adds lazy import + user-friendly stubs for HF sandbox classes when unavailable. |
| envs/opencode_env/README.md | Documents how to switch from E2B to HF sandbox backend. |
| envs/opencode_env/pyproject.toml | Raises huggingface_hub minimum to include Sandbox. |
| envs/opencode_env/opencode_runtime.py | Adds sandbox_home-derived helpers for opencode/proxy paths used by the harness. |
| envs/opencode_env/harness.py | Uses new runtime helpers and improves sandbox teardown behavior on setup failures. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| /root/task \ | ||
| /root/workdir \ | ||
| /root/proxy | ||
| COPY interception.py /root/proxy/interception.py |
There was a problem hiding this comment.
Fixed in 764e02ea — the COPY path is relative to the envs/opencode_env build context, same as the server image.
| RUN curl -fsSL https://opencode.ai/install | bash \ | ||
| && /root/.opencode/bin/opencode --version |
There was a problem hiding this comment.
Kept intentionally: it mirrors the existing E2B template install path (sandbox/build_template.py). Pinning opencode is a separate follow-up if we want reproducible sandbox images.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 9 out of 9 changed files in this pull request and generated 3 comments.
Comments suppressed due to low confidence (1)
envs/opencode_env/sandbox/hf_image/Dockerfile:14
- Piping a remote install script directly into
bashmakes the build vulnerable to supply-chain compromise and also obscures failures in the curl/pipe stage. Download the script first (at minimum) so failures are explicit; ideally also pin a version + checksum.
RUN curl -fsSL https://opencode.ai/install | bash \
&& /root/.opencode/bin/opencode --version
| except Exception as exc: | ||
| _log.error("factory.create: bootstrap failed: %r", exc) | ||
| _log.error("factory.create: setup failed, killing sandbox: %r", exc) | ||
| sandbox.kill() | ||
| raise |
There was a problem hiding this comment.
Fixed in 4e26cb65 — the cleanup sandbox.kill() is now wrapped in try/except, so a cleanup failure can't mask the original setup exception.
| while True: | ||
| proc = next((p for p in self._sandbox.processes() if p.pid == self.pid), None) | ||
| # missing pid (reaped) or running=False both mean done |
There was a problem hiding this comment.
The repo's ruff config ignores E501 and ruff format isn't run on envs/ (existing files like e2b.py also exceed 88), so this doesn't fail CI. Left as-is for consistency.
| # Any failure past here (bootstrap/proxy/agent) must tear the sandbox down. | ||
| try: | ||
| self._bootstrap_sandbox(sandbox, oc_task) |
There was a problem hiding this comment.
Added in 4e26cb65 — tests/envs/test_opencode_factory_lifecycle.py asserts create() kills the sandbox when setup fails, and that the root cause survives if kill() also raises.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.
Comments suppressed due to low confidence (1)
envs/opencode_env/harness.py:291
- In the setup-failure path,
sandbox.kill()is called without guarding against teardown errors. Ifkill()itself raises (transient API error), it will mask the original setup exception and make debugging harder. Make sandbox teardown best-effort so the original exception always propagates.
except Exception as exc:
_log.error("factory.create: setup failed, killing sandbox: %r", exc)
sandbox.kill()
raise
| def write_text(self, path: str, content: str) -> None: | ||
| parent = str(PurePosixPath(path).parent) | ||
| if parent not in ("", "/"): | ||
| self._sbx.files.mkdir(parent) | ||
| self._sbx.files.write(path, content) |
There was a problem hiding this comment.
Fixed in 4e26cb65 — write_text now treats . (relative path) as "no parent", so files.mkdir(".") isn't attempted.
| FROM python:3.12 | ||
|
|
||
| # Proxy deps that interception.py imports. | ||
| RUN pip install --no-cache-dir "fastapi>=0.104" "uvicorn[standard]>=0.24" "httpx>=0.27" | ||
|
|
||
| RUN curl -fsSL https://opencode.ai/install | bash \ | ||
| && /root/.opencode/bin/opencode --version | ||
|
|
There was a problem hiding this comment.
Fixed in 4e26cb65 — the Dockerfile installs curl + ca-certificates before fetching the opencode installer.
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 00f5485. Configure here.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 10 out of 11 changed files in this pull request and generated 1 comment.
Comments suppressed due to low confidence (1)
envs/opencode_env/harness.py:291
sandbox.kill()inside the setup-failure handler can raise (network/provider issues) and mask the original bootstrap/proxy/agent exception, making debugging harder and potentially changing failure semantics. Consider best-effort teardown that preserves the original exception while logging any kill failure.
except Exception as exc:
_log.error("factory.create: setup failed, killing sandbox: %r", exc)
sandbox.kill()
raise
| RUN curl -fsSL https://opencode.ai/install | bash \ | ||
| && /root/.opencode/bin/opencode --version |
There was a problem hiding this comment.
Kept intentionally: it mirrors the existing E2B template (sandbox/build_template.py), which installs opencode the same way (its official installer). The pre-baked image is also optional — runtime install works without it — so it's not on the hot path.
|
Testing summary Unit tests in CI: Validated end-to-end on HF Jobs driving a real OpenCode agent:
The |
…curl, lifecycle test - create(): wrap sandbox.kill() in cleanup so it can't mask the setup exception - write_text(): treat '.' (relative path) as no parent dir to create - Dockerfile: install curl + ca-certificates before fetching the opencode installer - test: assert create() kills the sandbox on setup failure (and preserves the root cause)
Import the primitive from its defining modules instead of the top-level opencode_env package (which re-exports the client), per the server/client-separation invariant.
| try: | ||
| from ..config import OpenCodeConfig | ||
| from ..harness import OpenCodeSessionFactory | ||
| from ..task import OpenCodeTask | ||
| except ImportError: # pragma: no cover | ||
| from config import OpenCodeConfig # type: ignore | ||
| from harness import OpenCodeSessionFactory # type: ignore | ||
| from task import OpenCodeTask # type: ignore | ||
| from opencode_env.sandbox import E2BSandboxBackend |
There was a problem hiding this comment.
Good catch, fixed in 86bd0676 — E2BSandboxBackend now sits inside the same try/except, so the standalone-import fallback resolves it too.
The fallback branch switched config/harness/task to bare imports but left the backend on the package path; move it into the same try/except so the server module also imports standalone.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 13 out of 14 changed files in this pull request and generated 2 comments.
Comments suppressed due to low confidence (1)
tests/envs/test_opencode_hf_sandbox.py:28
- This test mutates the global
huggingface_hubmodule (huggingface_hub.Sandbox = ...) at import time and never restores it, which can leak into other tests when running with an older hub version. Since this PR already bumps the dependency floor tohuggingface_hub>=1.22, it’s safer to require that version in the test instead of stubbing the attribute.
import huggingface_hub # noqa: E402
if not hasattr(huggingface_hub, "Sandbox"): # pragma: no cover - only on old hub
huggingface_hub.Sandbox = type("Sandbox", (), {}) # type: ignore[attr-defined]
| def wait(self, timeout: float | None = None) -> int: | ||
| deadline = None if timeout is None else time.monotonic() + timeout | ||
| while True: | ||
| proc = next((p for p in self._sandbox.processes() if p.pid == self.pid), None) | ||
| # missing pid (reaped) or running=False both mean done | ||
| if proc is None: | ||
| return 0 | ||
| if not proc.running: | ||
| return int(proc.exit_code) if proc.exit_code is not None else 0 | ||
| if deadline is not None and time.monotonic() >= deadline: | ||
| raise TimeoutError(f"Background command did not exit within {timeout}s") | ||
| time.sleep(_WAIT_POLL_INTERVAL_S) |
There was a problem hiding this comment.
Fixed in 75f88249 — wait() now sleeps min(poll_interval, remaining) so it respects the deadline.
| # Import the primitive from its defining modules (not the top-level | ||
| # package, which re-exports the client) to keep server/client separate. | ||
| try: | ||
| from ..config import OpenCodeConfig | ||
| from ..harness import OpenCodeSessionFactory | ||
| from ..sandbox import E2BSandboxBackend | ||
| from ..task import OpenCodeTask | ||
| except ImportError: # pragma: no cover | ||
| from config import OpenCodeConfig # type: ignore | ||
| from harness import OpenCodeSessionFactory # type: ignore | ||
| from sandbox import E2BSandboxBackend # type: ignore | ||
| from task import OpenCodeTask # type: ignore |
There was a problem hiding this comment.
This mirrors the existing lazy-import pattern used for the models import a few lines above (and in every env's server module), so I'm keeping it consistent rather than diverging one block to a __package__ check. A genuine missing-dep ImportError inside ..config still surfaces, because the bare-import fallback also fails and re-raises.
…handle/backend Verified against a real cpu-basic sandbox: finished processes stay listed (running=False), and a timed-out command returns exit_code=None with timed_out=True. So a vanished pid means the sandbox was torn down (now raises instead of returning 0), and a timed-out exec now surfaces exit 124 instead of a false 0. Adds unit tests for exec/start_bg/write_text/read_text/exists/kill and backend.create.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 13 out of 14 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (1)
tests/envs/test_opencode_hf_sandbox.py:29
- This test permanently mutates the global
huggingface_hubmodule by adding aSandboxattribute at import time when running against an olderhuggingface_hub. That mutation is not restored, so it can leak into other tests in the same process and hide genuine missing-dependency issues.
Consider stubbing Sandbox only long enough to import opencode_env.sandbox.hf, then restoring the original module state.
import huggingface_hub # noqa: E402
if not hasattr(huggingface_hub, "Sandbox"): # pragma: no cover - only on old hub
huggingface_hub.Sandbox = type("Sandbox", (), {}) # type: ignore[attr-defined]
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 13 out of 14 changed files in this pull request and generated 1 comment.
Comments suppressed due to low confidence (2)
tests/envs/test_opencode_hf_sandbox.py:29
- This test file mutates the global
huggingface_hubmodule at import time by adding aSandboxattribute. Because it isn’t restored after the tests, it can leak into later tests in the same process and mask a genuinely-missing/oldhuggingface_hubdependency (or change their behavior). Prefer scoping this patch to the module with an autouse fixture that restores the original attribute after the tests complete.
import huggingface_hub # noqa: E402
if not hasattr(huggingface_hub, "Sandbox"): # pragma: no cover - only on old hub
huggingface_hub.Sandbox = type("Sandbox", (), {}) # type: ignore[attr-defined]
envs/opencode_env/harness.py:247
create()now guarantees sandbox teardown for failures that occur after provisioning (bootstrap/proxy start/agent start). The new lifecycle tests only cover_bootstrap_sandboxfailures; there’s no unit test that asserts the same teardown behavior when_start_proxy()orsession.start_agent()raises, which makes the broader teardown contract easy to regress.
# Any failure past here (bootstrap/proxy/agent) must tear the sandbox down.
try:
self._bootstrap_sandbox(sandbox, oc_task)
| # Any failure past here (bootstrap/proxy/agent) must tear the sandbox down. | ||
| try: |
There was a problem hiding this comment.
black_box is the intended default (safe minimal path; the GRPO/training path passes transparent_proxy explicitly). The misleading (default) annotation on transparent_proxy in the module docstring was removed in 880ce5e8.
…ps; guard UI stderr; tighten wait() sleep - server: only run verify + compute reward when the run is clean; dedupe the endpoint error paren - harness: set -o pipefail on the proxy-deps install - gradio: don't index an empty stderr split - hf wait(): sleep min(poll_interval, remaining) so it respects the deadline
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 14 out of 15 changed files in this pull request and generated 1 comment.
Comments suppressed due to low confidence (1)
tests/envs/test_opencode_hf_sandbox.py:29
- This test file adds a fake
huggingface_hub.Sandboxattribute at import time when running against olderhuggingface_hubversions, but it never restores the original module state. That global mutation can leak into other tests in the same process. Store whether the attribute was added and delete it inteardown_moduleto keep the tests isolated.
import huggingface_hub # noqa: E402
if not hasattr(huggingface_hub, "Sandbox"): # pragma: no cover - only on old hub
huggingface_hub.Sandbox = type("Sandbox", (), {}) # type: ignore[attr-defined]
| # A timed-out command is killed with exit_code unset; surface it as a | ||
| # non-zero exit instead of a false success. | ||
| if result.timed_out: | ||
| return ExecResult( | ||
| exit_code=124, | ||
| stdout=result.stdout, | ||
| stderr=(result.stderr or "") + f"\n[timed out after {timeout}s]", | ||
| ) | ||
| return ExecResult( | ||
| exit_code=int(result.exit_code) if result.exit_code is not None else 0, | ||
| stdout=result.stdout, | ||
| stderr=result.stderr, | ||
| ) |
There was a problem hiding this comment.
Fixed in 1c466584 — stdout/stderr are now coerced to str in both the timeout and non-timeout paths. Kept result.timed_out as a direct field access (it ships in the >=1.22 floor the package pins) rather than getattr, per the repo's no-defensive-getattr guideline.
Summary
Adds an
HFSandboxBackendso the OpenCode harness can run the agent inside aHugging Face sandbox (
huggingface_hub.Sandbox) as an alternative to E2B. Anybackend satisfying the existing
SandboxBackend/SandboxHandle/BgJobprotocolsplugs into
OpenCodeSessionFactoryunchanged.Enabling it surfaced (and this PR fixes) a latent bug: the transparent-proxy paths and
the opencode-binary probe were hardcoded to
/home/user, so proxy startup failed on anyroot-based backend. They now derive from
config.sandbox_home, which also unblocks aDocker/root backend.
What this PR does
sandbox/hf.py—HFSandboxBackend/HFSandboxHandle/HFBgJobagainsthuggingface_hub.Sandbox(create/run(..., background=)/files/processes).HFBgJob.wait()pollsprocesses()since the SDK has no blocking wait. Teardown usesSandbox.kill()(notclose(), which only drops the client and leaves the sandboxidling).
opencode_runtime.py/harness.py— derive proxy trace/log/dir, proxy source, andthe opencode binary path from
config.sandbox_homeinstead of hardcoding/home/user.E2B (
sandbox_home="/home/user") is unchanged.harness.py—OpenCodeSessionFactory.create()now tears the sandbox down on anysetup failure (proxy start / agent launch), not just bootstrap.
pyproject.toml— bump the floor tohuggingface_hub>=1.22(shipsSandbox); it isalready a hard core dependency (
>=0.20.0).tests/— unit tests forHFBgJob.wait()(fake sandbox, no network).sandbox/hf_image/Dockerfile— optional pre-baked image (opencode + proxy deps) toskip the per-rollout cold install; runtime install also works without it.
Usage:
Testing
pytest tests/envs/test_opencode_hf_sandbox.py— 10 passing, no network.(
transparent_proxy, opencode → in-sandbox proxy → HF router) installed opencode, wrotethe solution,
verify()returned reward 1.0, and the proxy trace captured 3 turns;the sandbox was torn down cleanly.
Type of Change
/home/user)Notes / follow-ups (out of scope here)
image="python:3.12"+ runtime install works.core/harness/sandbox/(ref Agentic RL with black-box harnesses in OpenEnv via LLM-call interception #940).ruff formatofenvs/opencode_env+ a latentF401insandbox/e2b.py— separate PR.HarnessRolloutWorker(Async grpo OpenEnv harness rollout trl#6420) needs avLLM reachable from the remote sandbox (returning token ids via
--return-tokens-as-token-ids);that is a networking/topology concern independent of this backend.
🤖 AI-assisted PR.
Note
Medium Risk
New remote sandbox backend and bumped huggingface_hub affect rollout provisioning and teardown; path changes are scoped to configurable sandbox_home with regression tests.
Overview
Adds
HFSandboxBackendso OpenCode rollouts can run inhuggingface_hub.Sandbox(≥1.22) instead of E2B, using the sameOpenCodeSessionFactory/ protocol surface as E2B.Path fix: proxy, opencode binary, and related in-sandbox paths are no longer fixed at
/home/user; they come fromOpenCodeConfig.sandbox_home(e.g./rootfor HF sandboxes). The harness and transparent-proxy bootstrap use the newopencode_runtimehelpers.Reliability:
OpenCodeSessionFactory.create()kills the sandbox on bootstrap, proxy, or agent startup failure (not only bootstrap). MCP rollout path skips verify/reward when setup or agent fails; setup commands get a longer timeout.Shipping:
huggingface_hub>=1.22dependency, optional pre-bakedsandbox/hf_image/Dockerfile(+ CIopencode-sandboximage), and docs/README examples forHFSandboxBackend.Reviewed by Cursor Bugbot for commit 1c46658. Bugbot is set up for automated code reviews on this repo. Configure here.