diff --git a/.agents/skills/building-templates/SKILL.md b/.agents/skills/building-templates/SKILL.md new file mode 100644 index 0000000000..19190c5e37 --- /dev/null +++ b/.agents/skills/building-templates/SKILL.md @@ -0,0 +1,36 @@ +--- +name: building-templates +description: "Build E2B sandbox templates: the CLI Dockerfile flow, the Template SDK builder, and the repo's own prepared templates. Use when creating/rebuilding templates or debugging template builds." +--- + +# Building Templates + +Two ways to define a template: + +## 1. CLI Dockerfile flow + +`e2b template create ` reads `e2b.Dockerfile` (or `Dockerfile`) in the working directory (or `-p `) and builds it into a sandbox template. Useful flags: `-c/--cmd` (start command), `--ready-cmd` (must exit 0 before the template is considered ready — doubles as a build-time smoke test), `--cpu-count`, `--memory-mb` (even number, default 1024), `--no-cache`. Needs `E2B_API_KEY`. Related: `template list`, `template delete`, `template publish/unpublish`, and `template migrate` (converts a legacy `e2b.Dockerfile` + `e2b.toml` pair to the Template SDK format). + +## 2. Template SDK (js-sdk) + +Programmatic builder: `Template().fromPythonImage('3')...` then `Template.build(template, 'name:tag', {...})` (see `packages/js-sdk/src/template/`, examples in the `Template` class jsdoc). Live examples of usage — including file-context handling and `defaultBuildLogger` — are in `packages/js-sdk/tests/template/` (run via the `template` vitest project, 180s timeout, needs `E2B_API_KEY`). + +## Repo's prepared templates (`templates/`) + +- `templates/base` — the default public template; also pushed to DockerHub. +- `templates/httpbin` — private echo-server sidecar used by the JS network-transform tests; built with `--cmd 'go-httpbin ...'` and a curl `--ready-cmd`. + +They are rebuilt via the manual `Build and push prepared templates` workflow (`.github/workflows/templates.yml`, `workflow_dispatch` with a template picker). Locally the same commands work, e.g.: + +```bash +cd templates/base && e2b template create base --memory-mb 512 +``` + +## Debugging build failures + +- Build logs stream from the API; HTTP 500s during template builds are a known transient on staging — retry before digging. +- If the build hangs at "ready", the `--ready-cmd` never exited 0 — run it inside a sandbox of the base image to check. +- The build runs remotely; local Docker is not used for `e2b template create`. + +## Devin Secrets Needed +- `E2B_API_KEY` diff --git a/.agents/skills/codegen-and-specs/SKILL.md b/.agents/skills/codegen-and-specs/SKILL.md new file mode 100644 index 0000000000..ff4cd1508c --- /dev/null +++ b/.agents/skills/codegen-and-specs/SKILL.md @@ -0,0 +1,39 @@ +--- +name: codegen-and-specs +description: "Update API specs and regenerate SDK/CLI client code in the E2B repo. Use when bumping spec refs, adding new API surface, or when the generated-files CI check fails." +--- + +# Codegen and Specs + +## Ownership: never hand-edit `spec/` + +Files under `spec/` are synced from upstream repos and any manual edit will be overwritten: + +- `spec/openapi.yml`, envd specs → `e2b-dev/infra` +- `spec/volume-api.yml` → `e2b-dev/belt` + +Copybara (`copy.bara.sky`) mirrors them in; `spec/infra-ref` and `spec/belt-ref` pin the upstream commits used by codegen. `spec/README.md` is the authoritative doc. + +## Updating generated code + +```bash +# 1. bump the pin (or let copybara have updated the spec already) +echo > spec/infra-ref # or spec/belt-ref + +# 2. refetch pinned specs + regenerate everything in Docker +make codegen +``` + +`make codegen` builds `codegen.Dockerfile` and runs `make generate` inside it, so results don't depend on local tool versions. Fetch failures only warn and fall back to the tracked copies — check the output if you expected a spec change. Belt spec fetches need a token; public infra specs fetch anonymously. + +Fetch-only helpers: `pnpm fetch:api-spec`, `pnpm fetch:envd-spec`, `pnpm fetch:volume-spec` (override the pin with `E2B_INFRA_REF=main` / `E2B_BELT_REF=main`). + +## What gets generated + +Redocly first filters the OpenAPI specs by SDK tags (internal/`x-internal` ops are stripped), then clients are generated into `packages/js-sdk`, `packages/python-sdk`, and the CLI. Commit the generated diffs together with the spec/ref change. + +## CI + +`generated_files.yml` re-runs generation on PRs touching `spec/**`, codegen Dockerfiles, `copy.bara.sky`, redocly config, fetch scripts, packages, or lockfiles, and fails if the committed generated files are stale. If it fails, run `make codegen` locally and commit the diff — don't patch generated files by hand. + +Gotcha from past sessions: a new API parameter appearing in the staging spec may not be deployed to production yet — generated code can be ahead of the production API (see `debugging-ci`). diff --git a/.agents/skills/debugging-ci/SKILL.md b/.agents/skills/debugging-ci/SKILL.md new file mode 100644 index 0000000000..3fbac8c9b6 --- /dev/null +++ b/.agents/skills/debugging-ci/SKILL.md @@ -0,0 +1,29 @@ +--- +name: debugging-ci +description: "Understand and debug GitHub Actions failures on E2B pull requests: the workflow matrix, which checks are required, and known flaky jobs. Use whenever a PR check fails." +--- + +# Debugging CI + +## The matrix (`.github/workflows/sdk_tests.yml`) + +A paths filter decides which suites run (docs-only `.md` changes trigger nothing; `spec/**`, lockfiles, `.tool-versions`, or the workflow files trigger everything): + +- **Production** JS / Python / CLI test suites — these feed the required aggregate check **`SDK Tests Status`**, which also fails if change-detection itself fails or jobs are cancelled. +- **Staging** JS / Python / CLI suites — same reusable workflows against `E2B_DOMAIN_STAGING`; staging JS runs `node-only` (Bun/Deno/Cloudflare coverage only runs on production). +- `generated_files.yml` — fails when committed generated code is stale; fix with `make codegen` (see `codegen-and-specs`). + +The JS workflow's full production matrix covers Node (Ubuntu + Windows), Bun, Deno, Cloudflare, and a Cloudflare deploy leg. + +## Known flakes and skew (from past sessions) + +- **Staging jobs flake**: template-build HTTP 500s, network-egress curl failures, other transient backend errors. Before blaming your change, check whether the same job fails on the base branch, or rerun the job. +- **Staging-before-production skew**: new API features usually reach staging first. A *production-only* failure of a test exercising a brand-new API parameter typically means the API isn't deployed to production yet, not an SDK bug. +- **Cloudflare deploy** can hit propagation / read-after-write races on freshly created preview workers — treat an isolated intermittent failure there as advisory and rerun. + +## Debug loop + +1. Get the failing job and its logs (Devin: `git_pr_checks` → `git_ci_job_logs` with the job id). +2. Reproduce locally with the same command the job runs (the reusable workflows run `pnpm run test` / `uv run pytest` in the package dir; export `E2B_DOMAIN` to mimic staging). +3. If it's a live-sandbox test, remember it needs `E2B_API_KEY`; offline mock suites are the fastest local signal. +4. Never conclude a failure is preexisting/flaky without evidence — verify against the base branch or a rerun. diff --git a/.agents/skills/releasing/SKILL.md b/.agents/skills/releasing/SKILL.md new file mode 100644 index 0000000000..d7dfd0108c --- /dev/null +++ b/.agents/skills/releasing/SKILL.md @@ -0,0 +1,30 @@ +--- +name: releasing +description: "How versioning and releases work in the E2B repo: changesets, the manual release workflow, and release candidates. Use when preparing a release, deciding whether a PR needs a changeset, or debugging the publish pipeline." +--- + +# Releasing + +## Changesets + +Versions are driven by changesets in `.changeset/`. Generate one with `pnpm changeset` at the repo root **when a PR changes the public surface** of `packages/cli` (`@e2b/cli`), `packages/js-sdk` (`e2b`), or `packages/python-sdk` (`@e2b/python-sdk`). Internal scripts, devtools, tests, docs, and skills don't need one. The python package participates in changesets via its npm-shim name `@e2b/python-sdk`. + +## Production release (`.github/workflows/release.yml`) + +Manual `workflow_dispatch`, and it hard-fails on any ref other than `main` (a feature branch carrying changesets would otherwise publish real packages). Flow: + +1. **Preflight** — parses `.tool-versions` for pinned tooling, runs `is_release.sh` / `is_release_for_package.sh` to decide which of js-sdk / python-sdk / cli have pending changesets, and builds a Slack "itinerary" (advisory only — `continue-on-error`). +2. **Tests** — the full JS / Python / CLI suites run for each package being released. +3. **Publish** (`publish_packages.yml`) — versions via changesets, publishes to npm and PyPI (needs `E2B_API_KEY`, `PYPI_TOKEN`), and pushes the version-bump commit. + +Production and candidate runs on the same ref share a concurrency group, so they serialize. + +## Release candidates (`release-candidate.yml`) + +Manual dispatch from any branch with per-package booleans, a dist-tag (`rc`/`beta`/`snapshot`), an optional preid (defaults to the branch name), and an optional skip-tests flag. Publishes prerelease versions via `publish_candidates.yml` without touching main. Use this to let users try an unmerged fix. + +## Debugging a failed release + +- Failure Slack notifications fire from the workflow; check which stage failed (preflight vs tests vs publish). +- Test-stage failures are the normal SDK suites — see `debugging-ci`. +- "Nothing to release" means no pending changesets — check `pnpm changeset status`. diff --git a/.agents/skills/setting-up-testing-environment/SKILL.md b/.agents/skills/setting-up-testing-environment/SKILL.md new file mode 100644 index 0000000000..ce00f7f573 --- /dev/null +++ b/.agents/skills/setting-up-testing-environment/SKILL.md @@ -0,0 +1,41 @@ +--- +name: setting-up-testing-environment +description: "Initialize the E2B repo for local testing: toolchain versions, dependency install, SDK/CLI builds, and credentials. Use at the start of any session that will run or test code in this repo." +--- + +# Setting Up the Testing Environment + +## Toolchain + +`.tool-versions` at the repo root is the source of truth (CI parses it): Node 22.x, pnpm, Python 3.10, uv, deno. Locally, `source ~/.nvm/nvm.sh && nvm use 24` also works for all packages (the CLI needs Node ≥ 20). + +Use pnpm for Node and uv for Python — never npm/yarn/pip. + +## Install and build + +```bash +pnpm install # repo root; installs all workspaces +pnpm --filter e2b build # js-sdk -> packages/js-sdk/dist +pnpm --filter @e2b/cli build # cli -> packages/cli/dist/index.js +cd packages/python-sdk && uv sync # python env (uv run ... after this) +``` + +## Credentials + +- `E2B_API_KEY` is required for anything that touches real sandboxes (most js-sdk tests, python `tests/sync|async`, CLI integration tests, live CLI commands). It's read from the environment; defaults may also live in `.env.local` at the repo root or `~/.e2b/config.json`. +- `E2B_DOMAIN` switches the target environment (unset = production). +- `E2B_DEBUG` switches suites to a local envd; only set it when you're running envd locally. +- Never run `e2b auth login` in headless environments — export the key instead. + +## Smoke check + +```bash +cd packages/cli && node dist/index.js sandbox list # verifies key + build +cd packages/js-sdk && npx vitest run --project connectionConfig # offline, no key +cd packages/python-sdk && uv run pytest tests/test_connection_config.py -q +``` + +Then use the package-specific skills: `testing-js-sdk`, `testing-python-sdk`, `testing-cli`, `verifying-cli-visual-output`. + +## Devin Secrets Needed +- `E2B_API_KEY` diff --git a/.agents/skills/testing-cli/SKILL.md b/.agents/skills/testing-cli/SKILL.md new file mode 100644 index 0000000000..89f59319bf --- /dev/null +++ b/.agents/skills/testing-cli/SKILL.md @@ -0,0 +1,51 @@ +--- +name: testing-cli +description: "Build, run, and test the E2B CLI (packages/cli) locally. Use when running its vitest suites or executing the CLI against real sandboxes." +--- + +# Testing the CLI + +All commands run in `packages/cli`. Use Node 24 (`nvm use 24`) and pnpm. + +## Build and run locally + +```bash +source ~/.nvm/nvm.sh && nvm use 24 +pnpm build # tsc typecheck + tsdown bundle -> dist/index.js +# or from the repo root: pnpm --filter @e2b/cli build +node dist/index.js --help +node dist/index.js sandbox list +``` + +Auth: the CLI reads `E2B_API_KEY` from the environment first, then `~/.e2b/config.json` (or `.env.local` at the repo root). Never run `e2b auth login` in headless environments — export the key instead. + +Useful non-interactive patterns: + +```bash +node dist/index.js sandbox create base -d # detach: prints sandbox ID, no attached terminal +node dist/index.js sandbox exec -- bash -lc 'pwd' # `--` stops CLI flag parsing +node dist/index.js sandbox kill +``` + +The team account usually has other live sandboxes/snapshots, so a truly empty list state may be unreachable; simulate it with a non-matching filter: `node dist/index.js sandbox list --metadata nomatch=zzz` → "No sandboxes found". + +There is no CLI flag for custom sandbox metadata — create such sandboxes via the JS SDK instead (build it with `pnpm --filter e2b build`, then `require('/packages/js-sdk')` from a small node script). + +Validate JSON output mode: `node dist/index.js --format json | node -e 'JSON.parse(require("fs").readFileSync(0,"utf8"))'`. + +## Automated tests + +```bash +pnpm run test # vitest; globalSetup runs `pnpm build` first +npx vitest run tests/utils/table.test.ts # single file +``` + +- Tests spawn the **built** CLI (`dist/index.js`) via helpers in `tests/setup.ts` (`runCli`, `runCliWithPipedStdin`) — rebuild happens automatically through globalSetup, but if you bypass vitest, run `pnpm build` yourself after editing `src/`. +- `tests/commands/**` cover command behavior; some hit the real API and need `E2B_API_KEY`. +- Unit tests import from `src/` directly (vitest aliases `e2b` to `../js-sdk/src`), so they run without building the SDK. + +For checking rendered output (tables, colors, alignment), use the `verifying-cli-visual-output` skill. + + +## Devin Secrets Needed +- `E2B_API_KEY` diff --git a/.agents/skills/testing-js-sdk/SKILL.md b/.agents/skills/testing-js-sdk/SKILL.md new file mode 100644 index 0000000000..5ee7b642f4 --- /dev/null +++ b/.agents/skills/testing-js-sdk/SKILL.md @@ -0,0 +1,44 @@ +--- +name: testing-js-sdk +description: "Run tests for the E2B JavaScript SDK (packages/js-sdk). Use when debugging its vitest suites or verifying SDK behavior against real sandboxes." +--- + +# Testing the JS SDK + +All commands run in `packages/js-sdk`. Use Node 24 (`nvm use 24`) and pnpm. + +## Test projects + +Tests are organized into vitest projects in `vitest.config.mts`: + +- `unit` — everything in `tests/**/*.test.ts` except runtimes, template, and connectionConfig. Many of these are **integration tests that create real sandboxes** and require `E2B_API_KEY`. +- `template` — template builder tests (`tests/template/**`), 180s timeout, require `E2B_API_KEY`. +- `connectionConfig` — offline config tests. +- `browser` — Playwright/chromium tests (`pnpm run playwright:install` first). + +## Running + +```bash +# everything (needs E2B_API_KEY) +pnpm run test + +# one file — fastest loop, preferred while iterating +npx vitest run tests/api/inflight.test.ts + +# one project +npx vitest run --project connectionConfig + +# alternate runtimes +pnpm run test:bun # bun: unit + connectionConfig + template +pnpm run test:deno # deno: same projects +``` + +`E2B_API_KEY` is read from the environment or from `.env` via dotenv (the repo also keeps defaults in `.env.local` at the root or `~/.e2b/config.json`). Purely offline unit tests (e.g. `tests/api/inflight.test.ts`, `tests/utils.test.ts`) run without a key. + +Fixtures in `tests/setup.ts` (`sandboxTest`/`templateTest`) create and clean up sandboxes automatically; `E2B_DEBUG` switches suites to a local envd. Fully offline suites (msw-mocked, e.g. `tests/api/`, `tests/volume/`) are the fastest signal when no key is available. + +## CI notes + +- SDK integration jobs run against both **production and staging**; staging jobs are known to flake (template-build 500s, network-egress curl errors). Before assuming your change broke CI, check whether the same job fails on the base branch or rerun the job. +- New API parameters often work on staging before production — a production-only failure of a new-feature test usually means the API isn't deployed there yet, not a code bug. + diff --git a/.agents/skills/testing-python-sdk/SKILL.md b/.agents/skills/testing-python-sdk/SKILL.md new file mode 100644 index 0000000000..e748e594c8 --- /dev/null +++ b/.agents/skills/testing-python-sdk/SKILL.md @@ -0,0 +1,36 @@ +--- +name: testing-python-sdk +description: "Run tests for the E2B Python SDK (packages/python-sdk). Use when running its pytest suites (sync or async) or verifying SDK behavior against real sandboxes." +--- + +# Testing the Python SDK + +All commands run in `packages/python-sdk`. Use uv for everything (`uv run ...`); never pip. + +## Layout + +- `tests/sync/` and `tests/async/` — integration tests against real sandboxes (need `E2B_API_KEY`). Sync and async variants must stay equivalent, except for streaming: sync has no background pump, so callbacks are passed to `wait()` and `watch_dir` is polled via `get_new_events()`. +- `tests/test_*.py` (top level) — offline unit tests (transports, codecs, config parsing, etc.). +- `tests/conftest.py` — fixtures that create/clean up sandboxes; `pytest.ini` sets `asyncio_mode=auto`, a 30s per-test timeout, and `pythonpath = tests` for shared helpers like `envd_frame_server`. + +## Running + +```bash +# full suite, 4 workers (needs E2B_API_KEY in env) +pnpm run test # == uv run pytest -n 4 --verbose -x + +# single file / test — preferred while iterating +uv run pytest tests/test_paginator.py -v +uv run pytest tests/sync/sandbox_sync/test_create.py -v -k "metadata" + +# offline-only quick check (skip integration dirs) +uv run pytest tests --ignore=tests/sync --ignore=tests/async -q +``` + +The `skip_debug` marker skips a test when `E2B_DEBUG` is set (local envd). + +## CI notes + +- Integration jobs run against both **production and staging**; staging jobs are known to flake (template-build 500s, transient backend errors). Check the base branch or rerun before blaming your change. +- New API parameters often reach staging before production — a production-only failure of a new-feature test usually means the API isn't deployed there yet. +- `tests/shared/` holds offline suites shared between sync and async (request shaping, encoding, proxy config) — the fastest signal when no key is available. diff --git a/.agents/skills/triaging-issues/SKILL.md b/.agents/skills/triaging-issues/SKILL.md new file mode 100644 index 0000000000..0b48314a15 --- /dev/null +++ b/.agents/skills/triaging-issues/SKILL.md @@ -0,0 +1,45 @@ +--- +name: triaging-issues +description: "Triage conventions for GitHub issues in e2b-dev/E2B: classification dimensions, label usage, spam handling, and the triage comment format. Use when classifying or labeling an issue." +--- + +# Triaging Issues + +New issues are normally auto-triaged by an automation; follow the same conventions when triaging manually so results stay consistent. + +## Classification dimensions + +- `type`: bug | feature | refactor | question | chore | security +- `priority`: critical | high | medium | low +- `complexity`: trivial | small | medium | large +- `affected_area`: best-guess component (e.g. "js-sdk/filesystem", "python-sdk/commands", "cli/templates", "CI pipeline", "unknown") +- `actionable`: whether there's enough information to act +- `needs_clarification`: only true when proceeding would risk building the wrong thing entirely; list the specific questions +- `summary`: one-sentence restatement (note assumptions here instead of blocking) + +Be decisive: classify with best judgment when intent is clear, even if details are thin. + +## Labels + +Apply matching **existing** repository labels (`gh label list` to see them — e.g. bug, feature, Improvement, sdk, cli, envd, Infrastructure, Build System, Code Interpreter) with `gh issue edit --add-label ...`. Never create new labels. + +## Spam + +Check the issue and each comment for spam/promotional content (off-topic self-promotion, link-farming, credit-solicitation, AI filler). Hide spam comments via the GraphQL `minimizeComment` mutation with `classifier: SPAM` (`gh api graphql`). If the issue itself is spam: `type: chore`, `priority: low`, note "spam" in the summary and that it should be closed. + +## Comment format + +Post the classification as an issue comment: + +```md +**Triage** + +- **type**: bug +- **priority**: high +- **complexity**: small +- **affected_area**: js-sdk/network transform +- **actionable**: yes +- **needs_clarification**: false +- **questions**: (only when needs_clarification is true, as a nested list) +- **summary**: One-sentence restatement of the work item. +``` diff --git a/.agents/skills/verifying-cli-visual-output/SKILL.md b/.agents/skills/verifying-cli-visual-output/SKILL.md new file mode 100644 index 0000000000..5bf0a301ef --- /dev/null +++ b/.agents/skills/verifying-cli-visual-output/SKILL.md @@ -0,0 +1,59 @@ +--- +name: verifying-cli-visual-output +description: "Verify the visual/terminal output of the E2B CLI: tables, colors, alignment, spinners, TTY vs piped behavior. Use when changing anything the CLI prints or when reviewing output formatting." +--- + +# Verifying CLI Visual Output + +The CLI's output style follows kubectl: borderless tables, uppercase headers, left-aligned columns with 3-space padding, no trailing whitespace (`packages/cli/src/utils/table.ts` → `renderTable`). Colors/bold come from `chalk` (`src/utils/format.ts`). + +## 1. Unit-test the rendering (preferred) + +Capture `console.log` lines and assert exact strings — see `tests/utils/table.test.ts` (`npx vitest run tests/utils/table.test.ts`): + +```ts +const lines: string[] = [] +vi.spyOn(console, 'log').mockImplementation((l: string) => lines.push(l)) +renderTable(rows, columns) +expect(lines).toEqual(['SANDBOX ID NAME', 'sbx-1 alpha']) +``` + +Keep row-building logic in exported pure functions (e.g. `buildTableRows` in `src/commands/sandbox/list.ts`) so formatting is testable without the API. + +To exercise the renderer directly (e.g. wide chars in a padded middle column), run a small script importing `packages/cli/src/utils/table.ts` with `npx tsx`. + +## 2. Eyeball the real output + +```bash +cd packages/cli && pnpm build +node dist/index.js sandbox list +``` + +Checks to make by eye: +- Headers uppercase; columns aligned even with wide cells (widths use `wcswidth`, so CJK/emoji count as 2 cells). +- No borders, no trailing whitespace (`node dist/index.js sandbox list | cat -A` — no spaces before `$`). +- Long values (metadata JSON) don't break alignment of preceding columns. +- Empty result sets print a sensible message, not a lone header or a crash. The team account usually has live sandboxes, so simulate the empty state with a non-matching filter: `node dist/index.js sandbox list --metadata nomatch=zzz` → "No sandboxes found". +- `template list` and `snapshot list` outputs are long; pipe through `head` when capturing. + +## 3. TTY vs piped behavior + +chalk auto-strips colors when stdout is not a TTY, so piped output must stay clean and parseable: + +```bash +node dist/index.js sandbox list | head # no ANSI escape codes expected +node dist/index.js sandbox list | grep -c $'\e' # should be 0 +FORCE_COLOR=1 node dist/index.js sandbox list # force colors while piping, to inspect them +script -qec "node dist/index.js sandbox list" /dev/null # run under a real PTY +``` + +Interactive commands (spinners, prompts via `inquirer`) need a PTY — use the `script` trick above or a tty-enabled shell; never leave them attached in CI-style runs (prefer `--detach` variants). + +## 4. Screenshot for PRs + +For user-facing output changes, run the command in a real terminal, take a screenshot, and embed it in the PR description — reviewers care about how it looks, not just the strings. + +GUI tips: in a 1024px-wide terminal, shrink the font (Konsole: Ctrl+minus x2) so wide tables don't wrap. Typing CJK/emoji via computer-use keyboard drops characters — write test scripts to a file with a file tool and run them from the terminal instead. + +## Devin Secrets Needed +- `E2B_API_KEY` diff --git a/DEV.md b/DEV.md deleted file mode 100644 index ef83fd7b27..0000000000 --- a/DEV.md +++ /dev/null @@ -1,3 +0,0 @@ -# Releasing e2b cli - -to create a changeset run `pnpm run changeset`