Skip to content
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/get-started/quickstart-hermes.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -151,7 +151,7 @@ Use these details when your first-run path needs more control.
```

Leave `CHAT_UI_URL` unset when you use SSH local port forwarding to `127.0.0.1:18789`.
Hermes API clients authenticate with the bearer token from the generated Hermes environment, not an OpenClaw dashboard URL token.
Hermes API clients authenticate with the bearer token returned by `nemohermes my-hermes gateway-token --quiet`, not an OpenClaw dashboard URL token.
</Accordion>

<Accordion title="Onboarding and Integration Details">
Expand Down
7 changes: 7 additions & 0 deletions docs/manage-sandboxes/recover-rebuild-sandboxes.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,13 @@ The session then follows the current default selected through `inference set`, w

<AgentOnly variant="hermes">
The rebuild command preserves Hermes state, registered policies, and managed MCP configuration while recreating the container.
A rebuild creates a new sandbox home and a new Hermes API bearer token.
After the rebuild succeeds, retrieve the replacement token before reconnecting API clients:

```bash
$$nemoclaw <sandbox-name> gateway-token --quiet
```

Before post-restore repairs, NemoClaw verifies that the recreated sandbox still identifies as Hermes and exits nonzero if its identity does not match the rebuild target.
After state restore, NemoClaw restores managed MCP configuration through the normal lifecycle, then re-proves or recovers gateway health and performs final MCP reconciliation.
`rebuild` exits nonzero instead of reporting success when it cannot verify final gateway health or managed MCP state.
Expand Down
13 changes: 13 additions & 0 deletions docs/reference/commands.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -1545,6 +1545,12 @@ Do not log it, share it, or commit it to version control.

Print the Hermes API bearer token for a running sandbox to stdout.
NemoClaw retrieves the sandbox's `API_SERVER_KEY`, which authenticates OpenAI-compatible clients on the forwarded API port.
During a normal sandbox lifecycle, the token is generated once for each sandbox home.
Different sandbox homes receive different tokens.
NemoClaw preserves it across a gateway restart, sandbox stop and start, and host OpenShell gateway restart.
When you rebuild or replace the sandbox, the replacement home receives a new token.
At gateway startup, NemoClaw also generates a new token when `API_SERVER_KEY` is missing or is not exactly 64 lowercase hexadecimal characters.
If an ordinary restart changes the token while the existing `API_SERVER_KEY` was present and valid, collect the before and after sandbox identity plus redacted mint logs and report it as a bug.
Capture the token and pass it in the `Authorization` header:

```bash
Expand All @@ -1553,8 +1559,13 @@ curl -fsS -H "Authorization: Bearer $TOKEN" \
http://127.0.0.1:8642/v1/models
```

<Warning>
Treat the token like a password.
Do not log it, share it, or commit it to version control.
</Warning>

The sandbox must be running for `nemohermes my-assistant gateway-token --quiet` to retrieve the token.
Use this supported command instead of reading or editing `.hermes/.env` directly.
For browser access to the dashboard, use `nemohermes my-assistant dashboard-url`.

</AgentOnly>
Expand Down Expand Up @@ -2515,6 +2526,8 @@ After restore, the command runs `openclaw doctor --fix` for cross-version struct
<AgentOnly variant="hermes">

After restore, the command restores Hermes manifest-defined state and starts the rebuilt Hermes gateway with the regenerated `/sandbox/.hermes` config.
A rebuild creates a new sandbox home and a new Hermes API bearer token.
After the rebuild succeeds, retrieve the replacement token with `nemohermes my-assistant gateway-token --quiet` before reconnecting API clients.
For an older Hermes image that predates sealed shields transitions, rebuild is the only workflow authorized to use the descriptor-safe compatibility transition.
The compatibility path verifies the strict root-owned hash and the in-tree hash, publishes fresh config inodes to revoke retained write descriptors, and restores the trusted lock posture if the transition cannot finish.
Ordinary `shields up` and `shields down` commands refuse the older protocol and direct you to rebuild.
Expand Down
85 changes: 82 additions & 3 deletions src/lib/actions/sandbox/rebuild-post-restore-phase.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ import * as rebuildMessaging from "./rebuild-messaging-phase";
import { runRebuildPostRestorePhase } from "./rebuild-post-restore-phase";
import * as sessionModels from "./reconcile-session-models";

describe("rebuild post-restore session model reconciliation (#7102)", () => {
describe("rebuild post-restore phase", () => {
let agentName: "openclaw" | "hermes";
let order: string[];

Expand Down Expand Up @@ -98,13 +98,13 @@ describe("rebuild post-restore session model reconciliation (#7102)", () => {
};
}

it("reconciles OpenClaw sessions after doctor and before later config writes", async () => {
it("reconciles OpenClaw sessions after doctor and before later config writes (#7102)", async () => {
await runRebuildPostRestorePhase(input());

expect(order).toEqual(["doctor", "reconcile", "messaging", "config-hash"]);
});

it("does not run OpenClaw session reconciliation for another agent", async () => {
it("does not run OpenClaw session reconciliation for another agent (#7102)", async () => {
agentName = "hermes";
const args = input();

Expand All @@ -114,4 +114,83 @@ describe("rebuild post-restore session model reconciliation (#7102)", () => {
expect(sessionModels.reconcileStalePinnedSessionModelsAfterRebuild).not.toHaveBeenCalled();
expect(processRecovery.executeSandboxCommand).not.toHaveBeenCalled();
});

it("points Hermes rebuilds to the replacement API token retrieval command (#7175)", async () => {
agentName = "hermes";

await runRebuildPostRestorePhase(input());

const outputLines = vi.mocked(console.log).mock.calls.flat().map(String);
const output = outputLines.join("\n");
expect(output).toContain("Hermes API bearer token changed during rebuild");
expect(output).toContain("nemoclaw alpha gateway-token --quiet");
expect(
outputLines.findIndex((line) => line.includes("API bearer token changed")),
).toBeGreaterThan(outputLines.findIndex((line) => line.includes("rebuilt successfully")));
});

it("does not print the Hermes API token notice for OpenClaw rebuilds (#7175)", async () => {
await runRebuildPostRestorePhase(input());

const output = vi.mocked(console.log).mock.calls.flat().join("\n");
expect(output).not.toContain("Hermes API bearer token");
expect(output).not.toContain("gateway-token --quiet");
});

it("does not print the Hermes API token notice when post-restore verification is incomplete (#7175)", async () => {
agentName = "hermes";
vi.mocked(rebuildHermesPostRestore.ensureHermesGatewayAfterStateRestore).mockReturnValue(
"unverified",
);
const args = input();

await runRebuildPostRestorePhase(args);

const output = vi.mocked(console.log).mock.calls.flat().join("\n");
expect(output).not.toContain("Hermes API bearer token changed during rebuild");
expect(output).not.toContain("gateway-token --quiet");
expect(args.bail).toHaveBeenCalledWith("Hermes post-restore verification failed for 'alpha'.");
});

it("still prints the Hermes API token notice when a non-fatal post-restore step is unverified (#7175)", async () => {
agentName = "hermes";
vi.mocked(messagingHostForward.ensureMessagingHostForwardAfterRebuild).mockReturnValue(false);
const args = input();

await runRebuildPostRestorePhase(args);

const output = vi.mocked(console.log).mock.calls.flat().join("\n");
expect(args.bail).not.toHaveBeenCalled();
expect(output).toContain("rebuilt but some post-restore steps were incomplete");
expect(output).toContain("Hermes API bearer token changed during rebuild");
expect(output).toContain("nemoclaw alpha gateway-token --quiet");
});

it("prints the Hermes API token notice after gateway recovery (#7175)", async () => {
agentName = "hermes";
vi.mocked(rebuildHermesPostRestore.ensureHermesGatewayAfterStateRestore).mockReturnValue(
"recovered",
);
const args = input();

await runRebuildPostRestorePhase(args);

const output = vi.mocked(console.log).mock.calls.flat().join("\n");
expect(args.bail).not.toHaveBeenCalled();
expect(output).toContain("Hermes gateway recovered after state restore");
expect(output).toContain("Hermes API bearer token changed during rebuild");
});

it("does not print the Hermes API token notice after a shields relock failure (#7175)", async () => {
agentName = "hermes";
const args = input();
args.relockShieldsIfNeeded = vi.fn(() => false);

await runRebuildPostRestorePhase(args);

const output = vi.mocked(console.log).mock.calls.flat().join("\n");
expect(output).not.toContain("Hermes API bearer token changed during rebuild");
expect(output).not.toContain("gateway-token --quiet");
expect(args.bail).toHaveBeenCalledWith("Failed to re-apply shields lockdown.");
});
});
12 changes: 12 additions & 0 deletions src/lib/actions/sandbox/rebuild-post-restore-phase.ts
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,16 @@ export function printSuccessfulRebuildSummary(
}
}

function printHermesApiTokenChangeNotice(sandboxName: string, targetAgentName: string): void {
if (targetAgentName !== "hermes") {
return;
}
console.log(` ${YW}\u26a0${R} Hermes API bearer token changed during rebuild.`);
console.log(
` Retrieve the new token with \`${CLI_NAME} ${sandboxName} gateway-token --quiet\`.`,
);
}

export function resolveRestoredPolicyRegistryState(
sandboxEntry: Pick<RebuildSandboxEntry, "policyPresetsFinalized">,
restoredBuiltinPresets: readonly string[],
Expand Down Expand Up @@ -324,5 +334,7 @@ export async function runRebuildPostRestorePhase(
bail(
`Prepared backup recovery for '${sandboxName}' completed with unverified post-restore state.`,
);
return;
}
printHermesApiTokenChangeNotice(sandboxName, targetAgentName);
}
49 changes: 49 additions & 0 deletions test/e2e/live/rebuild-hermes.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -159,6 +159,34 @@ function expectEqual(actual: string | undefined, expected: string, message: stri
}
}

async function hermesApiTokenDigest(
host: HostCliClient,
apiKey: string | undefined,
artifactName: string,
): Promise<string> {
const result = await host.command(
"bash",
[
"-lc",
[
'token="$(nemoclaw "$SANDBOX_NAME" gateway-token --quiet)"',
'case "$token" in ""|*[!0-9a-f]*) exit 2 ;; esac',
'[ "${#token}" -eq 64 ] || exit 2',
"printf '%s' \"$token\" | sha256sum | cut -d' ' -f1",
].join(" && "),
],
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Outdated
{
artifactName,
env: testEnv(apiKey, { SANDBOX_NAME }),
redactionValues: apiKey ? [apiKey] : [],
timeoutMs: OPENSHELL_TIMEOUT_MS,
},
);
expectExitZero(result, "retrieve and hash Hermes API bearer token");
expect(result.stdout.trim()).toMatch(/^[0-9a-f]{64}$/);
return result.stdout.trim();
}

async function bestEffortPrecleanHermesResources(
host: HostCliClient,
apiKey: string | undefined,
Expand Down Expand Up @@ -812,6 +840,11 @@ test(STALE_BASE_REBUILD
},
session: sessionSummary,
});
const preRebuildApiTokenDigest = await hermesApiTokenDigest(
host,
apiKey,
"phase-4-api-token-before-rebuild",
);

switch (STALE_BASE_REBUILD) {
case false:
Expand Down Expand Up @@ -846,6 +879,9 @@ test(STALE_BASE_REBUILD
onOutput: progress.onOutput,
});
expectExitZero(rebuild, "nemoclaw rebuild Hermes sandbox");
const rebuildOutput = resultText(rebuild);
expect(rebuildOutput).toContain("Hermes API bearer token changed during rebuild");
expect(rebuildOutput).toContain(`nemoclaw ${SANDBOX_NAME} gateway-token --quiet`);

const oldImageInspect = await host.command(
"docker",
Expand Down Expand Up @@ -935,6 +971,19 @@ test(STALE_BASE_REBUILD
expectExitZero(restoredEnv, "read Hermes .env after rebuild");
expect(restoredEnv.stdout).toContain(`DISCORD_BOT_TOKEN=${DISCORD_PLACEHOLDER}`);

const postRebuildApiTokenDigest = await hermesApiTokenDigest(
host,
apiKey,
"phase-7-api-token-after-rebuild",
);
const stablePostRebuildApiTokenDigest = await hermesApiTokenDigest(
host,
apiKey,
"phase-7-api-token-stability-check",
);
expect(postRebuildApiTokenDigest).not.toBe(preRebuildApiTokenDigest);
expect(stablePostRebuildApiTokenDigest).toBe(postRebuildApiTokenDigest);

const restoredConfig = await host.command(
"openshell",
["sandbox", "exec", "--name", SANDBOX_NAME, "--", "cat", "/sandbox/.hermes/config.yaml"],
Expand Down
64 changes: 64 additions & 0 deletions test/hermes-api-key-lifecycle-docs.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
// SPDX-License-Identifier: Apache-2.0

import { readFileSync } from "node:fs";
import path from "node:path";

import { describe, expect, it } from "vitest";

const COMMANDS_PATH = "docs/reference/commands.mdx";
const REBUILD_GUIDE_PATH = "docs/manage-sandboxes/recover-rebuild-sandboxes.mdx";
const QUICKSTART_PATH = "docs/get-started/quickstart-hermes.mdx";

function readDoc(relativePath: string): string {
return readFileSync(path.join(process.cwd(), relativePath), "utf8");
}

function readHermesGatewayTokenSection(): string {
const commands = readDoc(COMMANDS_PATH);
const heading = "### `$$nemoclaw <name> gateway-token`";
const sectionStart = commands.indexOf(heading);
const sectionEnd = commands.indexOf("\n### `", sectionStart + heading.length);

expect(sectionStart).toBeGreaterThanOrEqual(0);
expect(sectionEnd).toBeGreaterThan(sectionStart);

const section = commands.slice(sectionStart, sectionEnd);
const hermesStart = section.indexOf('<AgentOnly variant="hermes">');
const hermesEnd = section.indexOf("</AgentOnly>", hermesStart);

expect(hermesStart).toBeGreaterThanOrEqual(0);
expect(hermesEnd).toBeGreaterThan(hermesStart);
return section.slice(hermesStart, hermesEnd);
}

describe("Hermes API bearer token lifecycle documentation (#7175)", () => {
it("distinguishes stable restarts from key-generating replacement operations", () => {
const commands = readHermesGatewayTokenSection();

expect(commands).toContain("generated once for each sandbox home");
expect(commands).toContain("Different sandbox homes receive different tokens");
expect(commands).toContain(
"gateway restart, sandbox stop and start, and host OpenShell gateway restart",
);
expect(commands).toContain("rebuild or replace the sandbox");
expect(commands).toContain("missing or is not exactly 64 lowercase hexadecimal characters");
expect(commands).toContain("while the existing `API_SERVER_KEY` was present and valid");
Comment thread
coderabbitai[bot] marked this conversation as resolved.
});

it("points rebuild operators to supported token retrieval", () => {
const commands = readHermesGatewayTokenSection();
const rebuildGuide = readDoc(REBUILD_GUIDE_PATH);
const quickstart = readDoc(QUICKSTART_PATH);

expect(commands).toContain("nemohermes my-assistant gateway-token --quiet");
expect(rebuildGuide).toContain("$$nemoclaw <sandbox-name> gateway-token --quiet");
expect(commands).toContain("replacement home receives a new token");
expect(rebuildGuide).toContain("new Hermes API bearer token");
expect(commands).toContain("Treat the token like a password");
expect(commands).toContain("The sandbox must be running");
expect(commands).toContain("instead of reading or editing `.hermes/.env` directly");
expect(quickstart).toContain("nemohermes my-hermes gateway-token --quiet");
expect(quickstart).not.toContain("bearer token from the generated Hermes environment");
});
});
Loading
Loading