diff --git a/docs/network-policy/customize-network-policy.mdx b/docs/network-policy/customize-network-policy.mdx index d26cecdc901..3872a639a2d 100644 --- a/docs/network-policy/customize-network-policy.mdx +++ b/docs/network-policy/customize-network-policy.mdx @@ -148,6 +148,64 @@ Provider-composed `_provider_*` entries are excluded because OpenShell reserves Existing presets and the baseline remain in place. The preset file under `presets/` also persists across sandbox recreations. +### Custom Recipe for Raw TLS Passthrough with `tls: skip` + +OpenShell's egress proxy terminates TLS for allowed HTTPS endpoints so it can inspect traffic. +The proxy creates a new TLS connection to the upstream, including for endpoints with `access: full`. +Some Cloudflare-fronted upstreams reset that new handshake. +The host remains allowed, but clients report `ECONNRESET` or a TLS error such as `curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL`. +Protocols that require direct TLS negotiation can fail in the same way. +For example, WhatsApp's HTTP/1.1-only Noise-over-WebSocket handshake cannot use the proxy's HTTP/2 ALPN negotiation. + +Use a raw L4 CONNECT tunnel when the endpoint requires direct TLS negotiation. +Set `access: full` and `tls: skip`. +The proxy passes the encrypted bytes without modification, and the sandbox client negotiates TLS with the upstream: + +```yaml +preset: + name: cf-fronted-api + description: "Cloudflare-fronted API that resets re-originated TLS" +network_policies: + cf_fronted_api: + name: cf_fronted_api + endpoints: + - host: api.example.com + port: 443 + access: full + tls: skip + binaries: + - { path: /usr/local/bin/node } +``` + +Save the file as `nemoclaw-blueprint/policies/presets/cf-fronted-api.yaml`. +The filename without `.yaml` must match `preset.name`. +Apply the catalog preset to the running sandbox by name: + +```bash +$$nemoclaw my-assistant policy-add cf-fronted-api +``` + +The preset catalog reads `preset.name`, but `policy-add ` loads `presets/.yaml`. +If these values differ, the catalog can list a preset that `policy-add` cannot load. +After you edit the preset file, run the same command again. +`policy-add` compares the preset with the live policy and applies the changed content. + +The maintained `whatsapp` channel preset uses this structure for `web.whatsapp.com`. +Refer to `src/lib/messaging/channels/whatsapp/policy/openclaw.yaml` for an example. +It combines `tls: skip` tunnel endpoints with inspected `protocol: rest` endpoints. + + +`tls: skip` disables L7 inspection and egress-boundary credential resolution for that endpoint. +The proxy cannot filter the HTTP method, path, or body after it creates the tunnel. +Endpoint `rules` therefore cannot constrain what the agent sends through the tunnel. +The proxy also cannot replace an OpenShell credential placeholder inside the encrypted request. +This recipe does not unblock endpoints that require both raw passthrough and egress-boundary credential resolution. +The declared host, port, and `binaries` scope remain in effect. +Use `tls: skip` only for the exact hosts that require raw passthrough. +Do not use a broad wildcard. +Keep L7 inspection for every endpoint that supports it. + + ### Custom Recipe: URL-Based MCP Server Adding a Streamable HTTP MCP server URL to OpenClaw (for example `https://mcp.example.com/mcp`) can result in the sandbox proxy denying the CONNECT tunnel with `HTTP 403 Forbidden` because the target host is not in the default allowlist. @@ -183,13 +241,15 @@ network_policies: - { path: /usr/local/bin/node } ``` -Save it under `nemoclaw-blueprint/policies/presets/my-mcp.yaml` and apply it to the running sandbox: +Save it as `nemoclaw-blueprint/policies/presets/my-mcp.yaml`. +The filename without `.yaml` must match `preset.name`. +Apply it to the running sandbox by name: ```bash -$$nemoclaw my-assistant policy-add --from-file nemoclaw-blueprint/policies/presets/my-mcp.yaml +$$nemoclaw my-assistant policy-add my-mcp ``` -NemoClaw prompts for confirmation because the preset targets a custom host that is not part of the vetted allowlist, then reports `Widening sandbox egress — adding: mcp.example.com`. +NemoClaw previews the effective egress scope that the preset would open, including `mcp.example.com`, and prompts for confirmation before applying. For a publicly routed host that passes the separate SSRF checks, re-invoke the MCP tool and confirm that the CONNECT tunnel to `mcp.example.com:443` succeeds. diff --git a/docs/network-policy/integration-policy-examples.mdx b/docs/network-policy/integration-policy-examples.mdx index c67217f9ca5..9d448f2ef22 100644 --- a/docs/network-policy/integration-policy-examples.mdx +++ b/docs/network-policy/integration-policy-examples.mdx @@ -100,7 +100,12 @@ $$nemoclaw my-assistant policy-add outlook --dry-run $$nemoclaw my-assistant policy-add outlook --yes ``` -`policy-add` exits non-zero if the `outlook` preset is already applied. If you applied it in the lifecycle example above, run `$$nemoclaw my-assistant policy-remove outlook --yes` first (or skip this apply step). +When the `outlook` preset is already applied, `policy-add outlook` compares the preset with the live policy. +If they match, the command exits zero without changing the live policy. +If they differ, the command applies the preset again. +This includes changes to the preset file. +You do not need to remove the preset before you apply it again. +Removing it first temporarily deletes its entries from the live policy. Then configure the email or calendar tool credentials through the integration you run in the sandbox. Keep OAuth client secrets and refresh tokens out of policy files. diff --git a/docs/reference/commands.mdx b/docs/reference/commands.mdx index fbf96f91d6f..e0c8800b3ce 100644 --- a/docs/reference/commands.mdx +++ b/docs/reference/commands.mdx @@ -1703,7 +1703,15 @@ $$nemoclaw my-assistant policy-add pypi --yes The positional form is required in scripted workflows. Set `NEMOCLAW_NON_INTERACTIVE=1` instead of `--yes` if you want the same behavior from an environment variable. -If the preset name is unknown or already applied, the command exits non-zero with a clear error. +If the preset name is unknown, the command exits non-zero with a clear error. +If a named preset is already applied, the command compares the preset content with the live policy. +When the content matches, the command reports no changes and exits zero. +When the content differs, the command shows the normal preview and asks for confirmation before applying the preset again. +This includes changes to the preset file. +The comparison requires both the preset content and the live policy. +If either cannot be read, the command exits non-zero. +The command also exits non-zero when the name belongs to a custom preset applied with `--from-file`. +Use `--from-file` to apply that custom preset again. Built-in preset choices are scoped to the sandbox's active agent. Messaging channel presets appear only when NemoClaw has a matching channel policy for that agent; unavailable channel presets use the standard unknown-preset error before endpoint preview or confirmation. Custom preset files are tracked with the sandbox that applied them. `policy-list`, `policy-add`, and `policy-remove` compare the local registry and live gateway state using that sandbox-scoped preset metadata, so custom presets do not appear missing just because they are not part of the built-in preset catalog. diff --git a/src/lib/actions/sandbox/policy-channel-add-drift.test.ts b/src/lib/actions/sandbox/policy-channel-add-drift.test.ts new file mode 100644 index 00000000000..6f157aacf26 --- /dev/null +++ b/src/lib/actions/sandbox/policy-channel-add-drift.test.ts @@ -0,0 +1,243 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +/** + * Drift-aware `policy-add ` contract (#7323): naming an + * already-applied preset must compare the preset content against the live + * gateway policy instead of failing on the registry name alone. Users who + * edit a preset file in place (for example to add `tls: skip` endpoints) + * previously had their change silently ignored until they ran policy-remove + * followed by policy-add. + * + * - live policy still matches the preset -> successful idempotent no-op + * - preset content drifted from the live policy -> re-apply (fresh-add path, + * including the dry-run preview and interactive confirmation) + * - preset recorded but absent from the live policy -> re-apply + * - name owned by a custom (--from-file) preset -> refuse; the built-in + * content is the wrong comparison baseline and re-applying it would + * clobber the custom policy + * - preset content or live policy unreadable -> conservative failure + * because drift cannot be verified + */ + +import { afterEach, beforeEach, describe, expect, it, type MockInstance, vi } from "vitest"; + +import { CLI_NAME } from "../../cli/branding"; +import * as store from "../../credentials/store"; +import * as policies from "../../policy"; +import * as onboardSession from "../../state/onboard-session"; +import * as registry from "../../state/registry"; +import { addSandboxPolicy } from "./policy-channel"; +import * as policyContextRefresh from "./policy-context-refresh"; + +type PresetInfo = ReturnType[number]; + +class ExitError extends Error { + constructor(public readonly code: number | undefined) { + super(`process.exit(${code})`); + } +} + +const POLICY_PRESETS: PresetInfo[] = [ + { file: "pypi.yaml", name: "pypi", description: "Python Package Index access" }, +]; + +let logSpy: MockInstance; +let errSpy: MockInstance; +let promptSpy: MockInstance; +let applyPresetMock: MockInstance; +let gatewayStateMock: MockInstance; +let refreshSpy: MockInstance; + +async function captureExit(action: () => Promise): Promise { + const outcome: unknown = await action().then( + () => new Error("Expected process.exit to be called"), + (error: unknown) => error, + ); + expect(outcome).toBeInstanceOf(ExitError); + return (outcome as ExitError).code; +} + +beforeEach(() => { + delete process.env.NEMOCLAW_NON_INTERACTIVE; + + logSpy = vi.spyOn(console, "log").mockImplementation(() => undefined); + errSpy = vi.spyOn(console, "error").mockImplementation(() => undefined); + vi.spyOn(process, "exit").mockImplementation(((code?: number) => { + throw new ExitError(code); + }) as never); + + promptSpy = vi.spyOn(store, "prompt").mockResolvedValue("y"); + vi.spyOn(registry, "getSandbox").mockReturnValue({ + name: "alpha", + agent: null, + policies: ["pypi"], + }); + vi.spyOn(registry, "getCustomPolicies").mockReturnValue([]); + + vi.spyOn(onboardSession, "loadSession").mockReturnValue(null); + vi.spyOn(onboardSession, "updateSession").mockReturnValue( + undefined as unknown as onboardSession.Session, + ); + + vi.spyOn(policies, "listPresets").mockReturnValue(POLICY_PRESETS); + vi.spyOn(policies, "listCustomPresets").mockReturnValue([]); + vi.spyOn(policies, "getAppliedPresets").mockReturnValue(["pypi"]); + vi.spyOn(policies, "loadPresetForSandbox").mockImplementation( + (_sandboxName: unknown, name: unknown) => + `network_policies:\n ${String(name)}:\n host: ${String(name)}.example.com\n`, + ); + applyPresetMock = vi.spyOn(policies, "applyPreset").mockReturnValue(true); + gatewayStateMock = vi.spyOn(policies, "getPresetContentGatewayState").mockReturnValue("drift"); + vi.spyOn(policies, "getPresetEndpoints").mockReturnValue(["pypi.example.com"]); + vi.spyOn(policies, "getPresetValidationWarning").mockReturnValue(null); + + refreshSpy = vi + .spyOn(policyContextRefresh, "refreshSandboxPolicyContextFile") + .mockReturnValue({ outcome: "ok", written: true }); +}); + +afterEach(() => { + vi.restoreAllMocks(); + delete process.env.NEMOCLAW_NON_INTERACTIVE; +}); + +describe("addSandboxPolicy drift-aware named re-add", () => { + it("re-applies a named preset whose content drifted from the live policy", async () => { + gatewayStateMock.mockReturnValue("drift"); + + await addSandboxPolicy("alpha", { preset: "pypi", yes: true }); + + expect(gatewayStateMock).toHaveBeenCalledWith( + "alpha", + expect.stringContaining("pypi.example.com"), + ); + expect(logSpy).toHaveBeenCalledWith(" Preset 'pypi' no longer matches the live policy."); + expect(logSpy).toHaveBeenCalledWith( + expect.stringContaining( + "Effective egress scope that would replace the current preset policy", + ), + ); + expect(applyPresetMock).toHaveBeenCalledWith("alpha", "pypi", { suppressDisclosure: true }); + expect(refreshSpy).toHaveBeenCalledTimes(1); + expect(refreshSpy).toHaveBeenCalledWith("alpha"); + }); + + it("treats a matching named re-add as a successful no-op instead of a failure", async () => { + gatewayStateMock.mockReturnValue("match"); + + await addSandboxPolicy("alpha", { preset: "pypi", yes: true }); + + expect(logSpy).toHaveBeenCalledWith( + " Preset 'pypi' is already applied and matches the live policy; nothing to do.", + ); + expect(applyPresetMock).not.toHaveBeenCalled(); + expect(refreshSpy).not.toHaveBeenCalled(); + }); + + it("re-applies when the preset is recorded but its entries are absent from the live policy", async () => { + gatewayStateMock.mockReturnValue("absent"); + + await addSandboxPolicy("alpha", { preset: "pypi", yes: true }); + + expect(logSpy).toHaveBeenCalledWith( + " Preset 'pypi' is recorded as applied but missing from the live policy.", + ); + expect(applyPresetMock).toHaveBeenCalledWith("alpha", "pypi", { suppressDisclosure: true }); + expect(refreshSpy).toHaveBeenCalledTimes(1); + }); + + it("previews a drift re-apply without mutating on --dry-run", async () => { + gatewayStateMock.mockReturnValue("drift"); + + await addSandboxPolicy("alpha", { preset: "pypi", yes: true, dryRun: true }); + + expect(gatewayStateMock).toHaveBeenCalledTimes(1); + expect(logSpy).toHaveBeenCalledWith(" Preset 'pypi' no longer matches the live policy."); + expect(applyPresetMock).not.toHaveBeenCalled(); + expect(refreshSpy).not.toHaveBeenCalled(); + }); + + it("asks for confirmation before a drift re-apply and honors decline", async () => { + promptSpy.mockResolvedValue("n"); + gatewayStateMock.mockReturnValue("drift"); + + await addSandboxPolicy("alpha", { preset: "pypi" }); + + expect(promptSpy).toHaveBeenCalledTimes(1); + expect(logSpy).toHaveBeenCalledWith(" Preset 'pypi' no longer matches the live policy."); + expect(applyPresetMock).not.toHaveBeenCalled(); + expect(refreshSpy).not.toHaveBeenCalled(); + }); + + it("re-applies drift without prompting when NEMOCLAW_NON_INTERACTIVE=1", async () => { + process.env.NEMOCLAW_NON_INTERACTIVE = "1"; + gatewayStateMock.mockReturnValue("drift"); + + await addSandboxPolicy("alpha", { preset: "pypi" }); + + expect(promptSpy).not.toHaveBeenCalled(); + expect(applyPresetMock).toHaveBeenCalledWith("alpha", "pypi", { suppressDisclosure: true }); + expect(refreshSpy).toHaveBeenCalledTimes(1); + }); + + it("refuses a built-in re-add when the name is owned by a custom preset", async () => { + vi.spyOn(registry, "getCustomPolicies").mockReturnValue([ + { name: "pypi", content: "network_policies:\n pypi:\n host: custom.example.com\n" }, + ]); + + await expect( + captureExit(() => addSandboxPolicy("alpha", { preset: "pypi", yes: true })), + ).resolves.toBe(1); + + expect(errSpy).toHaveBeenCalledWith( + " Preset 'pypi' was applied as a custom preset (--from-file).", + ); + expect(errSpy).toHaveBeenCalledWith( + ` Edit and re-apply it with --from-file, or run '${CLI_NAME} alpha policy-remove pypi' first.`, + ); + expect(gatewayStateMock).not.toHaveBeenCalled(); + expect(applyPresetMock).not.toHaveBeenCalled(); + expect(refreshSpy).not.toHaveBeenCalled(); + }); + + it("fails without an already-applied claim when the preset content cannot be read", async () => { + vi.spyOn(policies, "loadPresetForSandbox").mockReturnValue(null); + + await expect( + captureExit(() => addSandboxPolicy("alpha", { preset: "pypi", yes: true })), + ).resolves.toBe(1); + + expect(errSpy).toHaveBeenCalledWith(" Could not read the content of preset 'pypi'."); + expect(errSpy).not.toHaveBeenCalledWith(expect.stringContaining("already applied")); + expect(gatewayStateMock).not.toHaveBeenCalled(); + expect(applyPresetMock).not.toHaveBeenCalled(); + expect(refreshSpy).not.toHaveBeenCalled(); + }); + + it("keeps the already-applied failure when the live policy cannot be read", async () => { + gatewayStateMock.mockReturnValue(null); + + await expect( + captureExit(() => addSandboxPolicy("alpha", { preset: "pypi", yes: true })), + ).resolves.toBe(1); + + expect(errSpy).toHaveBeenCalledWith(" Preset 'pypi' is already applied."); + expect(errSpy).toHaveBeenCalledWith( + " Could not read the live sandbox policy to compare (is the sandbox gateway running?).", + ); + expect(errSpy).not.toHaveBeenCalledWith(expect.stringContaining("policy-remove")); + expect(applyPresetMock).not.toHaveBeenCalled(); + expect(refreshSpy).not.toHaveBeenCalled(); + }); + + it("does not classify drift for a preset that is not applied yet", async () => { + vi.spyOn(policies, "getAppliedPresets").mockReturnValue([]); + + await addSandboxPolicy("alpha", { preset: "pypi", yes: true }); + + expect(gatewayStateMock).not.toHaveBeenCalled(); + expect(applyPresetMock).toHaveBeenCalledWith("alpha", "pypi", { suppressDisclosure: true }); + expect(refreshSpy).toHaveBeenCalledTimes(1); + }); +}); diff --git a/src/lib/actions/sandbox/policy-channel.ts b/src/lib/actions/sandbox/policy-channel.ts index c35e7b885be..254e6dfda0a 100644 --- a/src/lib/actions/sandbox/policy-channel.ts +++ b/src/lib/actions/sandbox/policy-channel.ts @@ -86,7 +86,11 @@ const YW = useColor ? "\x1b[1;33m" : ""; * `--yes`/`-y`/`--force` (or `NEMOCLAW_NON_INTERACTIVE=1`) skips the * confirmation prompt. `--from-dir` applies non-hidden files in lexicographic * order and aborts at the first failure (already-applied presets are not - * rolled back). + * rolled back). Naming an already-applied preset compares the preset content + * against the live policy: a match is a successful no-op, while drift (an + * edited preset file) re-applies the preset through the normal path (#7323). + * Names owned by a custom (--from-file) preset are refused; re-apply those + * with `--from-file`. */ export async function addSandboxPolicy( sandboxName: string, @@ -149,6 +153,7 @@ async function addSandboxPolicyUnlocked( const applied = policies.getAppliedPresets(sandboxName); let answer = null; + let reapplyState: "drift" | "absent" | null = null; if (presetArg) { const normalized = presetArg.trim().toLowerCase(); const preset = allPresets.find((item: { name: string }) => item.name === normalized); @@ -160,8 +165,55 @@ async function addSandboxPolicyUnlocked( process.exit(1); } if (applied.includes(preset.name)) { - console.error(` Preset '${preset.name}' is already applied.`); - process.exit(1); + // #7323: the registry name alone must not block a re-add. Users edit + // preset files in place (for example to add `tls: skip` endpoints), so + // compare the preset content against the live gateway policy and fall + // through to a normal re-apply when it drifted. + const customNames = registry + .getCustomPolicies(sandboxName) + .map((entry: { name: string }) => entry.name); + if (customNames.includes(preset.name)) { + // A custom preset owns this name, so the built-in content is the + // wrong comparison baseline; re-applying it would clobber the custom + // policy and double-register the name. + console.error(` Preset '${preset.name}' was applied as a custom preset (--from-file).`); + console.error( + ` Edit and re-apply it with --from-file, or run '${CLI_NAME} ${sandboxName} policy-remove ${preset.name}' first.`, + ); + process.exit(1); + } + const appliedContent = policies.loadPresetForSandbox(sandboxName, preset.name); + if (!appliedContent) { + console.error(` Could not read the content of preset '${preset.name}'.`); + process.exit(1); + } + const appliedState = policies.getPresetContentGatewayState(sandboxName, appliedContent); + if (appliedState === "match") { + // The desired state already holds: exit 0 so converging scripts can + // call policy-add idempotently, mirroring how applyPreset treats a + // byte-identical re-application as a successful no-op. + console.log( + ` Preset '${preset.name}' is already applied and matches the live policy; nothing to do.`, + ); + return; + } + if (appliedState === null) { + // Live policy unreadable: drift is unverifiable, so refuse rather + // than guess. + console.error(` Preset '${preset.name}' is already applied.`); + console.error( + " Could not read the live sandbox policy to compare (is the sandbox gateway running?).", + ); + process.exit(1); + } + // State-only notice: the downstream flow reports the dry-run, + // confirmation, and apply outcomes. + reapplyState = appliedState; + console.log( + appliedState === "drift" + ? ` Preset '${preset.name}' no longer matches the live policy.` + : ` Preset '${preset.name}' is recorded as applied but missing from the live policy.`, + ); } answer = preset.name; } else { @@ -177,7 +229,13 @@ async function addSandboxPolicyUnlocked( const presetContent = policies.loadPresetForSandbox(sandboxName, answer); if (!presetContent) return; - policies.logPresetScope(presetContent); + if (reapplyState) { + // A re-add replaces the recorded entries, so use the state-aware heading + // instead of the fresh-add "would be opened" preview. + policies.logPresetScopeForState(answer, presetContent, reapplyState); + } else { + policies.logPresetScope(presetContent); + } const presetWarning = policies.getPresetValidationWarning(answer); if (presetWarning) { diff --git a/src/lib/policy/index.ts b/src/lib/policy/index.ts index 1374e9413e7..10ebbb60d1e 100644 --- a/src/lib/policy/index.ts +++ b/src/lib/policy/index.ts @@ -13,6 +13,7 @@ import YAML from "yaml"; // Namespace access keeps resolveOpenshell spyable in focused policy tests. import * as openshellResolveModule from "../adapters/openshell/resolve"; import { loadAgent } from "../agent/defs"; +import { CLI_NAME } from "../cli/branding"; import { getMessagingPolicyKeyAliases, getMessagingPolicyPresetValidationWarnings, @@ -1541,7 +1542,12 @@ function selectFromList( return; } if (applied.includes(item.name)) { + // The picker has no live-policy context to classify drift; the named + // path (policy-add ) re-applies edited presets (#7323). process.stderr.write(`\n Preset '${item.name}' is already applied.\n`); + process.stderr.write( + ` If its preset file changed, run '${CLI_NAME} policy-add ${item.name}' to re-apply it.\n`, + ); resolve(null); return; } diff --git a/test/policies.test.ts b/test/policies.test.ts index 09724ba7fb9..fea960c35d6 100644 --- a/test/policies.test.ts +++ b/test/policies.test.ts @@ -1190,7 +1190,7 @@ exit 1 it("rejects selecting an already-applied preset", async () => { const result = await runSelectionPrompt("selectFromList", "1\n", { applied: ["npm"] }); - expect(result.stderr).toContain("Preset 'npm' is already applied."); + expect(result.stderr).toMatch(/already applied\.[\s\S]*policy-add npm'/); expect(result.selected).toBeNull(); });