Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
9b2dd96
fix(security): scrub migration and backup credentials consistently
Ayush7614 Jul 28, 2026
186929b
fix(security): strip export-prefixed secrets in .env scrubbing
Ayush7614 Jul 28, 2026
e069379
fix(security): harden credential scrubbing fail-closed paths
Ayush7614 Jul 28, 2026
eb2b024
fix(security): fail closed on backup scrub errors
apurvvkumaria Jul 29, 2026
147c1f7
docs(security): clarify snapshot credential filtering
apurvvkumaria Jul 29, 2026
6e45761
docs(security): refine snapshot cleanup behavior
apurvvkumaria Jul 29, 2026
a596539
fix(security): sanitize every migration snapshot artifact
apurvvkumaria Jul 29, 2026
993284c
fix(security): harden migration snapshot sanitization
apurvvkumaria Jul 29, 2026
4d3c524
merge(security): sync migration snapshot sanitization
apurvvkumaria Jul 29, 2026
e366086
test(security): restore migration growth budget
apurvvkumaria Jul 29, 2026
f269b38
fix(security): fail closed without no-follow support
apurvvkumaria Jul 30, 2026
fa903fd
fix(security): fail closed without backup no-follow
apurvvkumaria Jul 30, 2026
21be527
fix(security): scrub common OAuth credential fields
apurvvkumaria Jul 29, 2026
b4e06a9
merge(security): sync migration scrubber with main
apurvvkumaria Jul 31, 2026
72fe613
merge(security): refresh migration scrubber base
apurvvkumaria Jul 31, 2026
16580e5
fix(security): pin migration sanitizer traversal
apurvvkumaria Jul 31, 2026
015fee1
test(security): cover sanitizer path races
apurvvkumaria Jul 31, 2026
9aea329
docs(security): document snapshot sanitizer boundary
apurvvkumaria Jul 31, 2026
4ee0172
merge(main): sync PR #7847 with current main
apurvvkumaria Jul 31, 2026
32c8065
fix(security): pin rebuild backup traversal
apurvvkumaria Jul 31, 2026
1575950
test(security): cover sanitizer boundary failures
apurvvkumaria Jul 31, 2026
5af445f
merge(main): sync credential scrub hardening
apurvvkumaria Jul 31, 2026
6b22a4f
merge(main): refresh credential scrub hardening
apurvvkumaria Jul 31, 2026
c4989d1
Merge branch 'main' into codex/pr7765-credential-scrub
senthilr-nv Jul 31, 2026
d59a0e3
fix(security): pin snapshot sanitizer interpreter
apurvvkumaria Jul 31, 2026
eb10756
merge: sync main into credential scrub
apurvvkumaria Jul 31, 2026
5b8dd52
test(security): keep sanitizer fixtures linear
apurvvkumaria Jul 31, 2026
5a27ad6
Merge branch 'main' into codex/pr7765-credential-scrub
senthilr-nv Aug 1, 2026
3634a20
Merge branch 'main' into codex/pr7765-credential-scrub
cv Aug 2, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ci/test-file-size-budget.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"$comment": "SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.\nSPDX-License-Identifier: Apache-2.0",
"defaultMaxLines": 1500,
"legacyMaxLines": {
"nemoclaw/src/commands/migration-state.test.ts": 1565,
"nemoclaw/src/commands/migration-state.test.ts": 1562,
"src/lib/inference/nim.test.ts": 2068,
"src/lib/onboard/preflight.test.ts": 1904,
"test/generate-openclaw-config.test.ts": 1941,
Expand Down
8 changes: 8 additions & 0 deletions docs/get-started/prerequisites.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ If you cannot add memory, configure at least 8 GB of swap to work around the iss
|------------|----------------------------------|
| Node.js | 22.19 or later |
| npm | 10 or later |
| Python | Python 3 at a trusted system location, with POSIX descriptor-relative filesystem support |
| Docker | Docker Engine, Docker Desktop, or Colima on a tested platform |
| Platform | Refer to [Platforms](#platforms) below |

Expand All @@ -40,6 +41,13 @@ When Docker is missing, the installer downloads Docker's official convenience sc
In a non-interactive run, the installer can reactivate the group through `sg docker` and continue onboarding.
If that path is unavailable or does not restore Docker access, the installer exits with `newgrp docker` guidance before it starts onboarding.

NemoClaw uses an isolated `python3` helper for descriptor-relative migration snapshot sanitization and deletion.
NemoClaw does not resolve this credential-bearing helper through the host `PATH`.
It accepts a verified executable at `/usr/bin/python3`, `/usr/local/bin/python3`, `/opt/homebrew/bin/python3`, `/opt/local/bin/python3`, or beside the Node.js executable.
The sanitizer fails closed before it reads snapshot content if no candidate passes its ownership, permission, and executable checks.
Supported Linux, macOS, and WSL environments provide the required POSIX filesystem operations.
Native Windows is not a supported execution path; use WSL.

If you choose the native Linux Ollama install path, the onboard wizard also requires `zstd` for Ollama archive extraction.
The installer also requires `strings` from `binutils` to verify the OpenShell binary before it continues with OpenShell install work.

Expand Down
8 changes: 8 additions & 0 deletions docs/manage-sandboxes/backup-restore.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,14 @@ Snapshots capture all workspace state directories defined in the agent manifest
Agent manifests can also declare durable top-level state files.
Treat snapshot directories as private local data.

Before NemoClaw marks a snapshot complete, it strips recognized credential values from copied JSON, YAML, and `.env` files.
It preserves OpenShell credential placeholders so rebuild can reattach the host-side provider.
If NemoClaw cannot sanitize a copied configuration or environment file, it omits that file from the snapshot.
If it cannot remove the unsafe file, snapshot creation returns an error.
It deletes the incomplete backup when cleanup succeeds and reports when the backup remains.
This sanitization uses an isolated `python3` helper on POSIX hosts to keep reads, replacements, and removals anchored to opened directory descriptors.
If a copied file or parent directory changes identity during the operation, snapshot creation fails closed instead of following the changed path.

<AgentOnly variant="hermes">
Hermes snapshots include `SOUL.md`, the Web Dashboard profile under `.hermes/dashboard-home/`, the SQLite database behind `.hermes/state.db`, and the default kanban board in `.hermes/kanban.db`.
The default-profile snapshot also includes cron execution history in `.hermes/runtime/cron-executions.db` and Discord replay state in `.hermes/gateway/discord_message_recovery.db`.
Expand Down
12 changes: 9 additions & 3 deletions docs/reference/host-files-and-state.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -45,14 +45,20 @@ If you see `registry.json` in older tests, notes, or discussions, treat it as le
| Path | Purpose | Safe to delete |
|---|---|---|
| `~/.nemoclaw/state/` | Operational coordination and history for lifecycle locks, shields transitions, timers, and audit events, local routing, and port-forward helpers. | No. Deleting it can disrupt an active operation and discard security or recovery context. |
| `~/.nemoclaw/snapshots/` | Full copies of host `~/.openclaw` state created by blueprint migration and rollback flows. | Only after you no longer need the corresponding rollback or restore point. The host CLI does not expose the direct runner's retention actions. |
| `~/.nemoclaw/snapshots/` | Copies of host `~/.openclaw` state and configured external roots created by blueprint migration and rollback flows. NemoClaw excludes known authentication-state files and strips recognized credential values from copied JSON, YAML, and `.env` files. | Only after you no longer need the corresponding rollback or restore point. The host CLI does not expose the direct runner's retention actions. |
| `~/.nemoclaw/rebuild-backups/` | Host-side snapshots written by `backup-all`, `snapshot create`, and rebuild flows. | Only after you no longer need rollback or restore points. |
| `~/.nemoclaw/backups/` | Workspace backups written by legacy backup helpers and some recovery flows. | Only after confirming you no longer need those workspace archives. |
| `~/.nemoclaw/mounts/` | Default local mount points created by share or mount commands. | Unmount first, then remove unused directories. |
| `~/.nemoclaw/blueprints/` | Cached blueprint inputs used by onboarding and sandbox recreation. | Avoid manual deletion unless you plan to rerun onboarding from fresh inputs. |

## Migration Snapshot Retention

Before NemoClaw retains a migration snapshot, it recursively sanitizes the copied OpenClaw state and every configured external root.
It preserves empty or comment-only YAML files and omits copied JSON, YAML, or `.env` files that it cannot sanitize.
If NemoClaw cannot remove an unsafe copied artifact, snapshot creation fails and attempts to delete the incomplete snapshot directory.
Sanitization requires `python3` on a POSIX host so every traversal and mutation can remain anchored to opened directory descriptors.
It fails closed if a copied file or parent directory changes identity during sanitization.

The direct blueprint runner accepts these action arguments for migration snapshots:

```text
Expand All @@ -67,8 +73,8 @@ An integration that invokes the direct runner can use `snapshots list` first to
`snapshots delete` accepts only one timestamped directory directly under `~/.nemoclaw/snapshots/`.
Both deletion commands are irreversible: they do not modify a running sandbox, but they remove host state that could otherwise be used for rollback or restore.

Snapshot deletion requires `python3` on a POSIX host.
Listing works on native Windows, but deletion does not; use WSL to prune or delete snapshots on Windows.
Snapshot sanitization and deletion require `python3` on a POSIX host.
Listing works on native Windows, but migration snapshot creation and deletion do not; use WSL for those operations on Windows.

## Uninstall Behavior

Expand Down
44 changes: 44 additions & 0 deletions nemoclaw/src/commands/migration-state-sanitizer-test-fixture.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
// SPDX-License-Identifier: Apache-2.0

import { isSensitiveFile, stripCredentials } from "../security/credential-filter.js";

interface TestFsEntry {
type: "file" | "dir" | "symlink";
content?: string;
}

/** Model the sanitizer's public data contract for migration-state's in-memory filesystem tests. */
export function buildMigrationStateSanitizerMock(store: Map<string, TestFsEntry>) {
function sanitizeJsonAt(filePath: string): boolean {
const entry = store.get(filePath);
if (entry?.type !== "file") return false;
try {
const parsed: unknown = JSON.parse(entry.content ?? "");
if (parsed && typeof parsed === "object" && !Array.isArray(parsed)) {
delete (parsed as Record<string, unknown>)["gateway"];
}
store.set(filePath, {
type: "file",
content: JSON.stringify(stripCredentials(parsed), null, 2),
});
return true;
} catch {
return false;
}
}

return {
sanitizeMigrationDirectory: (rootPath: string) => {
for (const [filePath, entry] of [...store.entries()]) {
if (entry.type !== "file" || !filePath.startsWith(`${rootPath}/`)) continue;
if (isSensitiveFile(filePath.split("/").at(-1) ?? "")) {
store.delete(filePath);
} else if (filePath.toLowerCase().endsWith(".json") && !sanitizeJsonAt(filePath)) {
store.delete(filePath);
}
}
},
sanitizeOpenClawConfigFile: sanitizeJsonAt,
};
}
71 changes: 34 additions & 37 deletions nemoclaw/src/commands/migration-state.test.ts
Original file line number Diff line number Diff line change
@@ -1,31 +1,23 @@
// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
// SPDX-License-Identifier: Apache-2.0

import { describe, it, expect, beforeEach, vi } from "vitest";
import { beforeEach, describe, expect, it, vi } from "vitest";
import type { PluginLogger } from "../index.js";
import { setConfigValue } from "./migration-state.js";

// ---------------------------------------------------------------------------
// fs mock — thin in-memory store keyed by absolute path
// ---------------------------------------------------------------------------

interface FsEntry {
type: "file" | "dir" | "symlink";
content?: string;
}

const store = new Map<string, FsEntry>();

function addDir(p: string): void {
store.set(p, { type: "dir" });
}

function addFile(p: string, content: string): void {
store.set(p, { type: "file", content });
}

function addSymlink(p: string): void {
store.set(p, { type: "symlink" });
type FsEntry = { type: "file" | "dir" | "symlink"; content?: string };
const { store } = vi.hoisted(() => ({ store: new Map<string, FsEntry>() }));
const descriptors = new Map<number, string>();
const addDir = (p: string): void => void store.set(p, { type: "dir" });
const addFile = (p: string, content: string): void => void store.set(p, { type: "file", content });
const addSymlink = (p: string): void => void store.set(p, { type: "symlink" });
// Keep mock failures explicit without adding control-flow branches to the test budget.
function fileAt(p: string | number): FsEntry {
const resolvedPath = typeof p === "number" ? descriptors.get(p) : p;
const entry = resolvedPath === undefined ? undefined : store.get(resolvedPath);
expect(entry?.type, `expected file at ${String(resolvedPath ?? p)}`).toBe("file");
return entry as FsEntry;
}

vi.mock("node:fs", async (importOriginal) => {
Expand All @@ -37,11 +29,15 @@ vi.mock("node:fs", async (importOriginal) => {
addDir(p);
}),
chmodSync: vi.fn(),
readFileSync: (p: string) => {
const entry = store.get(p);
if (entry?.type !== "file") throw new Error(`ENOENT: ${p}`);
return entry.content ?? "";
readFileSync: (p: string | number) => fileAt(p).content ?? "",
openSync: (p: string) => {
fileAt(p);
const fd = Math.max(99, ...descriptors.keys()) + 1;
descriptors.set(fd, p);
return fd;
},
fstatSync: (fd: number) => ({ isFile: () => fileAt(fd).type === "file" }),
closeSync: (fd: number) => descriptors.delete(fd),
writeFileSync: vi.fn((p: string, data: string) => {
store.set(p, { type: "file", content: data });
}),
Expand All @@ -60,7 +56,7 @@ vi.mock("node:fs", async (importOriginal) => {
}
}
}),
rmSync: vi.fn(),
rmSync: vi.fn((p: string) => store.delete(p)),
renameSync: vi.fn((oldPath: string, newPath: string) => {
for (const [k, v] of store) {
if (k === oldPath || k.startsWith(oldPath + "/")) {
Expand Down Expand Up @@ -97,19 +93,25 @@ vi.mock("node:fs", async (importOriginal) => {
};
});

vi.mock("../security/snapshot-sanitizer.js", async () =>
(await import("./migration-state-sanitizer-test-fixture.js")).buildMigrationStateSanitizerMock(
store,
),
);

// Mock tar to avoid real archive creation
vi.mock("tar", () => ({
create: vi.fn(async () => {}),
}));

import {
detectHostOpenClaw,
createSnapshotBundle,
cleanupSnapshotBundle,
createArchiveFromDirectory,
createSnapshotBundle,
detectHostOpenClaw,
type HostOpenClawState,
loadSnapshotManifest,
restoreSnapshotToHost,
type HostOpenClawState,
type SnapshotManifest,
} from "./migration-state.js";

Expand All @@ -125,12 +127,11 @@ function makeLogger(): PluginLogger {
describe("commands/migration-state", () => {
beforeEach(() => {
store.clear();
descriptors.clear();
vi.clearAllMocks();
});

// -------------------------------------------------------------------------
// detectHostOpenClaw
// -------------------------------------------------------------------------

describe("detectHostOpenClaw", () => {
it("returns exists=false when no state dir or config", () => {
Expand Down Expand Up @@ -448,9 +449,7 @@ describe("commands/migration-state", () => {
});
});

// -------------------------------------------------------------------------
// createSnapshotBundle
// -------------------------------------------------------------------------

describe("createSnapshotBundle", () => {
it("returns null when stateDir is missing", () => {
Expand Down Expand Up @@ -579,6 +578,8 @@ describe("commands/migration-state", () => {
const logger = makeLogger();
addDir("/home/user/.openclaw");
addFile("/home/user/.openclaw/openclaw.json", JSON.stringify({ version: 1 }));
addDir("/home/user/.openclaw/agents");
addDir("/home/user/.openclaw/agents/main");
addDir("/home/user/.openclaw/agents/main/agent");
addFile(
"/home/user/.openclaw/agents/main/agent/auth-profiles.json",
Expand Down Expand Up @@ -864,9 +865,7 @@ describe("commands/migration-state", () => {
});
});

// -------------------------------------------------------------------------
// cleanupSnapshotBundle
// -------------------------------------------------------------------------

describe("cleanupSnapshotBundle", () => {
it("removes temporary snapshot directory", async () => {
Expand Down Expand Up @@ -898,9 +897,7 @@ describe("commands/migration-state", () => {
});
});

// -------------------------------------------------------------------------
// createArchiveFromDirectory
// -------------------------------------------------------------------------

describe("createArchiveFromDirectory", () => {
it("calls tar.create with correct options", async () => {
Expand Down
Loading
Loading