Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions docs/inference/custom-endpoint-security.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,10 @@ Set `NEMOCLAW_TRUSTED_PRIVATE_INFERENCE_HOSTS` to a comma-separated list of exac
NemoClaw still resolves DNS, pins the validation connection, and rejects wildcard or suffix matches, link-local metadata, reserved destinations, and resolver failures.
This allowlist does not relax direct blueprint, `config set`, or unrelated persisted-URL validation.

After onboarding records an admitted custom endpoint, `inference set` accepts that same canonical URL for a model change without resolving it again.
The registry must record onboarding as the endpoint source, and the supplied URL must match exactly after normalization.
Legacy entries without a source, endpoints recorded by `inference set`, and different URLs still pass through the full server-side request forgery validation path.

## Use a Public Endpoint

For a public HTTP URL, NemoClaw stores the validated IP address so the downstream runtime cannot resolve the hostname again and reach another address.
Expand Down
2 changes: 2 additions & 0 deletions docs/inference/switch-models.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,8 @@ $$nemoclaw <name> shields up

For a compatible endpoint, omit `--endpoint-url` when the durable registry entry already contains the endpoint and API-family metadata.
NemoClaw reuses the recorded route and does not repoint the gateway.
You can also re-supply the same endpoint URL when the registry records that onboarding established it.
NemoClaw requires an exact canonical match and does not extend that trust to a different URL or an endpoint recorded by `inference set`.
If the route metadata is incomplete, NemoClaw stops and tells you to re-run onboarding.

For Hermes, the command also mirrors the selected model into the dashboard profile.
Expand Down
2 changes: 2 additions & 0 deletions docs/reference/commands.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -2945,6 +2945,8 @@ Use `--no-verify` only when OpenShell cannot verify the provider at switch time
When switching to `compatible-endpoint` or `compatible-anthropic-endpoint` from a different provider family, pass `--endpoint-url` with the trusted custom provider URL and, except for the Hermes case below, `--inference-api` with its API family so NemoClaw can persist a complete route identity for rebuild and shared-gateway checks.
For a Hermes `compatible-anthropic-endpoint` target, `--inference-api` may be omitted because NemoClaw deterministically selects `openai-completions`; an explicit different API family is rejected.
NemoClaw rejects loopback, link-local, private, and internal endpoint addresses, including public hostnames that resolve to a private address.
For a same-provider model change, pass `--endpoint-url` with the exact canonical endpoint URL that the target sandbox registry identifies as onboarding-established.
Missing or `inference set` provenance and every different URL remain subject to the full address validation above.
For public HTTP URLs, NemoClaw stores the validated IP address to prevent DNS rebinding.
DNS-backed HTTPS URLs are rejected because NemoClaw cannot pin the downstream peer address while preserving TLS SNI and host validation across the OpenShell runtime boundary; HTTPS IP-literal URLs remain supported.
NemoClaw accepts `http://host.openshell.internal:<port>` only with an explicit port from `1024` through `65535`; this narrow exception supports NemoClaw's sandbox-to-host inference routes and is not a general private-endpoint bypass.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -312,6 +312,7 @@ describe("runtime shared gateway route containment", () => {
provider: customRoute.provider,
model: customRoute.model,
canReuseRecordedRoute: false,
onboardEndpointUrl: null,
getSandboxes: () => [alpha, peer],
rewriteUrlWithDnsPinning,
}),
Expand Down
103 changes: 67 additions & 36 deletions src/lib/actions/inference-set-provider-alias.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -279,11 +279,10 @@ describe("runInferenceSet SSRF-block guidance — facet 2 (#6321)", () => {
});
}

it("keeps the SSRF guard AND adds an actionable hint when the sandbox is already on this provider", async () => {
// The reporter's case: a sandbox onboarded on compatible-endpoint against an
// internal Hub. `inference set --endpoint-url <internal>` still (correctly)
// trips the SSRF guard — but the message now tells the operator they can
// omit --endpoint-url to switch only the model.
it("keeps the SSRF guard when same-endpoint onboarding provenance is missing", async () => {
// Legacy registry rows have no machine-checkable endpoint source. Exact
// string equality is insufficient because inference set also persists the
// current endpoint, so the guarded path remains authoritative.
const deps = createDeps({
config: { agents: { defaults: { model: { primary: "inference/nvidia/model-a" } } } },
entry: {
Expand Down Expand Up @@ -319,10 +318,11 @@ describe("runInferenceSet SSRF-block guidance — facet 2 (#6321)", () => {
expectNoInferenceMutation(deps.calls);
});

it("keeps the SSRF guard AND guides on the anthropicCompatible provider family (#6321)", async () => {
// The reporter's exact provider family: the same-URL switch on
// compatible-anthropic-endpoint (reached via the anthropicCompatible alias)
// must still hit the guard and receive the omit-flag guidance.
it("accepts the same onboard-provenanced internal endpoint for anthropicCompatible (#6321)", async () => {
// The reporter's exact provider family now has a durable trust boundary:
// the canonical supplied URL must match the URL whose registry source is
// onboarding. DNS re-resolution is not required for that exact identity.
const guard = ssrfGuard();
const deps = createDeps({
config: { agents: { defaults: { model: { primary: "inference/anthropic/model-a" } } } },
entry: {
Expand All @@ -331,34 +331,65 @@ describe("runInferenceSet SSRF-block guidance — facet 2 (#6321)", () => {
provider: "compatible-anthropic-endpoint",
model: "anthropic/model-a",
endpointUrl: "https://inference-api.nvidia.com/v1",
endpointSource: "onboard",
credentialEnv: "COMPATIBLE_ANTHROPIC_API_KEY",
preferredInferenceApi: "anthropic-messages",
},
rewriteConfigUrlsWithDnsPinning: ssrfGuard(),
rewriteConfigUrlsWithDnsPinning: guard,
});
const attempt = runInferenceSet(
{
provider: "anthropicCompatible",
model: "anthropic/model-b",
await expect(
runInferenceSet(
{
provider: "anthropicCompatible",
model: "anthropic/model-b",
endpointUrl: "https://inference-api.nvidia.com/v1",
noVerify: true,
},
deps,
),
).resolves.toBeTruthy();
expect(guard).not.toHaveBeenCalled();
expect(deps.calls.updateSandbox.mock.calls.at(-1)).toEqual([
"alpha",
expect.objectContaining({ endpointSource: "onboard" }),
]);
});

it("accepts the same onboard-provenanced internal endpoint after canonicalization (#6321)", async () => {
const guard = ssrfGuard();
const deps = createDeps({
config: {
agents: { defaults: { model: { primary: "inference/nvidia/model-a" } } },
models: { providers: { inference: { api: "openai-completions", models: [] } } },
},
entry: {
name: "alpha",
agent: "openclaw",
provider: "compatible-endpoint",
model: "nvidia/model-a",
endpointUrl: "https://inference-api.nvidia.com/v1",
noVerify: true,
endpointSource: "onboard",
credentialEnv: "COMPATIBLE_API_KEY",
preferredInferenceApi: "openai-completions",
},
deps,
);
await expect(attempt).rejects.toThrow(/endpoint-url is not allowed:/);
await expect(attempt).rejects.toThrow(/already configured for 'compatible-anthropic-endpoint'/);
await expect(attempt).rejects.toThrow(/omit --endpoint-url/);
expectNoInferenceMutation(deps.calls);
rewriteConfigUrlsWithDnsPinning: guard,
});

await expect(
runInferenceSet(
{
provider: "compatible-endpoint",
model: "nvidia/model-b",
endpointUrl: "https://inference-api.nvidia.com/v1/",
noVerify: true,
},
deps,
),
).resolves.toBeTruthy();
expect(guard).not.toHaveBeenCalled();
});

it("re-supplying the SAME onboard-recorded internal endpoint is rejected with omit-guidance (no bypass) (#6321)", async () => {
// The recorded `entry.endpointUrl` is NOT trusted to skip the guard: this
// same `inference set` action persists endpointUrl, so a string-equality
// bypass would be self-authorizing (a value this command wrote could later
// authorize an internal-resolving switch). Re-supplying the exact recorded
// internal URL therefore still goes through the DNS-pinning SSRF guard and is
// rejected — with actionable guidance to omit --endpoint-url for a model-only
// switch on the already-established route (see the guided-path test below).
it("keeps the SSRF guard for an inference-set-authored endpoint", async () => {
const guard = ssrfGuard();
const deps = createDeps({
config: {
Expand All @@ -371,6 +402,7 @@ describe("runInferenceSet SSRF-block guidance — facet 2 (#6321)", () => {
provider: "compatible-endpoint",
model: "nvidia/model-a",
endpointUrl: "https://inference-api.nvidia.com/v1",
endpointSource: "inference-set",
credentialEnv: "COMPATIBLE_API_KEY",
preferredInferenceApi: "openai-completions",
},
Expand All @@ -381,23 +413,21 @@ describe("runInferenceSet SSRF-block guidance — facet 2 (#6321)", () => {
{
provider: "compatible-endpoint",
model: "nvidia/model-b",
// Same internal URL onboarding recorded, even a trailing-slash variant.
endpointUrl: "https://inference-api.nvidia.com/v1/",
endpointUrl: "https://inference-api.nvidia.com/v1",
noVerify: true,
},
deps,
);
await expect(attempt).rejects.toThrow(/endpoint-url is not allowed:/);
await expect(attempt).rejects.toThrow(/omit --endpoint-url/);
// The guard WAS consulted for the re-supplied URL — no string-equality bypass.
expect(guard).toHaveBeenCalled();
expectNoInferenceMutation(deps.calls);
});

it("still blocks a DIFFERENT internal endpoint even on a same-provider sandbox (no blanket exemption) (#6321)", async () => {
// Every supplied `--endpoint-url` goes through the SSRF guard (no bypass),
// so a *different* internal URL than the recorded one is blocked. Pinned as a
// regression: the fix does not hand the sandbox a way to reach arbitrary
// internal services.
// Onboarding provenance authorizes only the exact canonical endpoint it
// accompanies. A different internal URL still reaches the SSRF guard, so
// the fix cannot be used to reach arbitrary internal services.
const deps = createDeps({
config: { agents: { defaults: { model: { primary: "inference/nvidia/model-a" } } } },
entry: {
Expand All @@ -406,6 +436,7 @@ describe("runInferenceSet SSRF-block guidance — facet 2 (#6321)", () => {
provider: "compatible-endpoint",
model: "nvidia/model-a",
endpointUrl: "https://inference-api.nvidia.com/v1",
endpointSource: "onboard",
credentialEnv: "COMPATIBLE_API_KEY",
preferredInferenceApi: "openai-completions",
},
Expand Down
34 changes: 27 additions & 7 deletions src/lib/actions/inference-set-route-containment.ts
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ import { InferenceSetError } from "./inference-set-error";
*/
export type RegistryInferenceMetadata = Pick<
SandboxEntry,
"endpointUrl" | "credentialEnv" | "preferredInferenceApi" | "nimContainer"
"endpointUrl" | "endpointSource" | "credentialEnv" | "preferredInferenceApi" | "nimContainer"
>;

export interface ExplicitCustomRouteOptions {
Expand Down Expand Up @@ -192,6 +192,7 @@ function explicitCustomProviderMetadataWithoutDns(
// borrowing from an unrelated onboard session or global OpenShell provider.
return {
endpointUrl: normalizeCustomEndpointUrlWithoutDns(options.endpointUrl),
endpointSource: "inference-set",
credentialEnv: normalizeExplicitCredentialEnv(provider, options.credentialEnv),
preferredInferenceApi: normalizeExplicitInferenceApi(provider, options.inferenceApi),
nimContainer: null,
Expand All @@ -215,6 +216,7 @@ function matchingSessionMetadata(options: {
}
return {
endpointUrl: session.endpointUrl,
endpointSource: null,
credentialEnv: session.credentialEnv ?? null,
preferredInferenceApi: session.preferredInferenceApi ?? null,
nimContainer: session.nimContainer ?? null,
Expand All @@ -234,6 +236,7 @@ function registryMetadataForProviderSwitch(options: {
if (entry.provider === provider) {
return {
endpointUrl: entry.endpointUrl ?? null,
endpointSource: entry.endpointSource ?? null,
credentialEnv: entry.credentialEnv ?? null,
preferredInferenceApi: entry.preferredInferenceApi ?? null,
nimContainer: entry.nimContainer ?? null,
Expand All @@ -250,6 +253,7 @@ function registryMetadataForProviderSwitch(options: {
}
return {
endpointUrl: null,
endpointSource: null,
credentialEnv: null,
preferredInferenceApi: null,
nimContainer: null,
Expand Down Expand Up @@ -324,6 +328,7 @@ export async function finalizeInferenceSetRoute(options: {
provider: string;
model: string;
canReuseRecordedRoute: boolean;
onboardEndpointUrl: string | null;
getSandboxes: () => SandboxEntry[];
rewriteUrlWithDnsPinning: RewriteConfigUrlsWithDnsPinning;
}): Promise<{
Expand All @@ -338,15 +343,29 @@ export async function finalizeInferenceSetRoute(options: {
};
}
let endpointUrl: string;
let endpointSource: RegistryInferenceMetadata["endpointSource"];
try {
// A supplied endpoint always goes through the host DNS-pinning SSRF guard,
// even when it equals the value already recorded for this sandbox. The
// registry value is not exclusive onboarding provenance because inference
// set persists it too, so equality must never authorize a guard bypass.
endpointUrl = await normalizeCustomEndpointUrl(
const suppliedEndpoint = normalizeCustomEndpointUrlWithoutDns(
prepared.preliminaryExplicitMetadata.endpointUrl,
options.rewriteUrlWithDnsPinning,
);
const onboardEndpoint = options.onboardEndpointUrl
? normalizeCustomEndpointUrlWithoutDns(options.onboardEndpointUrl)
: null;
// The recorded URL alone is not an authority boundary because inference
// set writes it too. Bypass DNS re-resolution only when the registry also
// carries the endpoint's onboarding source and the canonical identities
// match exactly. Missing, inference-set, or mismatched provenance remains
// on the full DNS-pinning SSRF path (#6321).
if (onboardEndpoint !== null && suppliedEndpoint === onboardEndpoint) {
endpointUrl = suppliedEndpoint;
endpointSource = "onboard";
} else {
endpointUrl = await normalizeCustomEndpointUrl(
suppliedEndpoint,
options.rewriteUrlWithDnsPinning,
);
endpointSource = "inference-set";
}
} catch (error) {
// Only augment the SSRF/DNS-pinning rejection. Missing or malformed URLs
// keep their original diagnostics so the guidance cannot contradict them.
Expand All @@ -369,6 +388,7 @@ export async function finalizeInferenceSetRoute(options: {
const registryMetadata: RegistryInferenceMetadata = {
...prepared.preliminaryExplicitMetadata,
endpointUrl,
endpointSource,
};
assertGatewayRouteCompatibility({
gatewayName: prepared.gatewayName,
Expand Down
5 changes: 5 additions & 0 deletions src/lib/actions/inference-set.ts
Original file line number Diff line number Diff line change
Expand Up @@ -736,6 +736,10 @@ async function runInferenceSetWithoutHostLock(
entry.endpointUrl.trim().length > 0 &&
typeof entry.preferredInferenceApi === "string" &&
entry.preferredInferenceApi.trim().length > 0,
onboardEndpointUrl:
entry.provider === provider && entry.endpointSource === "onboard"
? (entry.endpointUrl ?? null)
: null,
getSandboxes: () => deps.listSandboxes().sandboxes,
rewriteUrlWithDnsPinning: deps.rewriteConfigUrlsWithDnsPinning,
});
Expand Down Expand Up @@ -814,6 +818,7 @@ async function runInferenceSetWithoutHostLock(
provider,
model,
endpointUrl: registryMetadata.endpointUrl ?? null,
endpointSource: registryMetadata.endpointSource ?? null,
credentialEnv: registryMetadata.credentialEnv ?? null,
preferredInferenceApi,
nimContainer: registryMetadata.nimContainer ?? null,
Expand Down
18 changes: 18 additions & 0 deletions src/lib/actions/sandbox/rebuild-gpu-opt-out.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -203,6 +203,24 @@ describe("buildRebuildRecreateOnboardOpts", () => {
expect(opts.toolDisclosure).toBe("direct");
});

it("preserves only recognized endpoint provenance across authoritative rebuild", () => {
const onboard = buildRebuildRecreateOnboardOpts({
...baseArgs,
sb: { ...dashboard, endpointSource: "onboard" },
});
const legacy = buildRebuildRecreateOnboardOpts({ ...baseArgs, sb: dashboard });
const malformed = buildRebuildRecreateOnboardOpts({
...baseArgs,
sb: { ...dashboard, endpointSource: "forged" } as typeof dashboard & {
endpointSource: never;
},
});

expect(onboard.endpointSource).toBe("onboard");
expect(legacy.endpointSource).toBeNull();
expect(malformed.endpointSource).toBeNull();
});

it("carries durable observability intent into inner onboard", () => {
const enabled = buildRebuildRecreateOnboardOpts({
...baseArgs,
Expand Down
7 changes: 7 additions & 0 deletions src/lib/actions/sandbox/rebuild-gpu-opt-out.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,10 @@
// SPDX-License-Identifier: Apache-2.0

import { loadAgent } from "../../agent/defs";
import {
type InferenceEndpointSource,
normalizeInferenceEndpointSource,
} from "../../inference/selection";
import { shouldManageDashboardForAgent } from "../../onboard/dashboard-runtime";
import {
type DcodeAutoApprovalMode,
Expand Down Expand Up @@ -38,6 +42,7 @@ export type RebuildGpuOptOutEntry = {
dcodeAutoApprovalMode?: DcodeAutoApprovalMode;
observabilityEnabled?: boolean;
policyTier?: string | null;
endpointSource?: InferenceEndpointSource | null;
};

// Modern source of truth is the persisted `sandboxGpuMode` string ("0" / "1" /
Expand Down Expand Up @@ -95,6 +100,7 @@ export type RebuildRecreateOnboardOpts = {
nonInteractive: true;
recreateSandbox: true;
authoritativeResumeConfig: true;
endpointSource?: InferenceEndpointSource | null;
acceptThirdPartySoftware: true;
agent: string | null | undefined;
fromDockerfile: string | null;
Expand Down Expand Up @@ -167,6 +173,7 @@ export function buildRebuildRecreateOnboardOpts(args: {
nonInteractive: true,
recreateSandbox: true,
authoritativeResumeConfig: true,
endpointSource: normalizeInferenceEndpointSource(args.sb?.endpointSource),
acceptThirdPartySoftware: args.usageNoticeAccepted,
agent: args.rebuildAgent,
fromDockerfile: args.storedFromDockerfile,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ const registryRoute: RegistryInferenceRoute = {
provider: target.provider,
model: target.model,
endpointUrl: "https://inference.example.test/v1",
endpointSource: null,
preferredInferenceApi: "openai-completions",
source: "registry",
};
Expand Down
1 change: 1 addition & 0 deletions src/lib/actions/sandbox/rebuild-resume-config.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -282,6 +282,7 @@ describe("prepareRebuildResumeConfig", () => {
provider: "compatible-endpoint",
model: "m",
endpointUrl: "https://registry.example.test/v1",
endpointSource: null,
preferredInferenceApi: "openai-completions",
source: "registry",
});
Expand Down
1 change: 1 addition & 0 deletions src/lib/actions/sandbox/rebuild-resume-preflight.ts
Original file line number Diff line number Diff line change
Expand Up @@ -168,6 +168,7 @@ function getRegistryInferenceRoute(
provider: registrySelection.provider,
model: registrySelection.model,
endpointUrl: rebuildEndpoint.endpointUrl,
endpointSource: registrySelection.endpointSource ?? null,
preferredInferenceApi: registrySelection.preferredInferenceApi,
source: "registry",
};
Expand Down
Loading
Loading