Skip to content

fix(rebuild): refresh stale agent primary model on config restore (#7210) - #7216

Merged
ericksoa merged 5 commits into
mainfrom
fix/7210-reconcile-agent-primary-model-on-rebuild
Jul 20, 2026
Merged

fix(rebuild): refresh stale agent primary model on config restore (#7210)#7216
ericksoa merged 5 commits into
mainfrom
fix/7210-reconcile-agent-primary-model-on-rebuild

Conversation

@yanyunl1991

@yanyunl1991 yanyunl1991 commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Summary

Switching a managed model on a sandbox that has MCP servers completes via rebuild --yes --tool-disclosure progressive (the generic onboard recreate is refused to preserve MCP state). After that rebuild the gateway route, the vLLM backend, and status show the new model, but openclaw.jsonagents.defaults.model.primary (and openclaw agents list) stay on the OLD model. This PR re-owns the agent's primary model routing from the fresh rebuild.

Closes #7210.

Reproduction

Reproduced on our DGX Spark aarch64 test host (GB10 GPU) with managed vLLM. The reporter's Station-only models (Nemotron-3-Ultra 550B, DeepSeek-V4-Flash) don't fit a GB10, so two Spark-servable managed profiles are used — nemotron-3-nano-4bqwen3.6-27b; the mechanism (config-restore merge, MCP-forced rebuild path) is model-agnostic.

Steps:

  1. onboard managed vLLM model A (nemotron-3-nano-4b).
  2. nemoclaw <sb> mcp add deepwiki --url https://mcp.deepwiki.com/mcp --env DEEPWIKI_TOKEN.
  3. docker stop nemoclaw-vllmonboard managed vLLM model B (qwen3.6-27b), reuse the sandbox → recreate is refused because MCP servers are present → run the directed nemoclaw <sb> rebuild --yes --tool-disclosure progressive.
  4. Inspect the agent binding.

Observed on main (before fix) — route/backend new, agent binding stale:

nemoclaw <sb> status        -> Model: Qwen/Qwen3.6-27B-FP8         (NEW, correct)
curl :8000/v1/models        -> Qwen/Qwen3.6-27B-FP8               (NEW, correct)
openclaw agents list (main) -> Model: inference/...Nemotron-3-Nano-4B-FP8   (STALE)
openclaw.json agents.defaults.model.primary = inference/...Nemotron-3-Nano-4B-FP8  (STALE)

Observed on fix/... (after fix) — agent binding follows the switch:

openclaw agents list (main) -> Model: inference/Qwen/Qwen3.6-27B-FP8        (NEW)
openclaw.json agents.defaults.model.primary = inference/Qwen/Qwen3.6-27B-FP8 (NEW)
# route/backend unchanged (NEW); MCP server 'deepwiki' still present after rebuild

Analysis

The rebuild config-restore merge (src/lib/state/openclaw-config-merge.tsmergeOpenClawRestoredConfig) overlays the backup onto the freshly generated config and then re-owns individual runtime sections (gateway/proxy/diagnostics, channels, models, plugins, tools). But agents is listed in backupDurableSections and is inherited wholesale from the pre-switch backup snapshot — including agents.defaults.model.primary. So even though models.providers routing is refreshed from the registry (new model), the agent's own routing reference (agents.defaults.model.primary, and the main/default agents.list[].model) stays on the pre-switch model. Restore never reconciled that reference against the fresh rebuild. On the MCP-forced rebuild --tool-disclosure progressive path there is no subsequent onboard-recreate to rewrite the config, so the stale primary persists.

Fix

After the section merges, reconcileAgentPrimaryModel re-owns the agent's primary model routing from the fresh config: it rewrites agents.defaults.model.primary and the matching main/default agents.list[].model (agent id main wins, else the first default: true, and only when its model is a string routing reference) from the freshly generated config — and only when the fresh config carries a primary, so backups with no rebuild-owned routing are left untouched. All other agent config (durable tuning like thinkingDefault, customAgents, and intentional non-default per-agent model pins) is still inherited from the backup. A removalCondition is documented on the helper. This revives the approach proposed in the previously-closed #7063.

Tests lock the whole contract: the stale primary + main list model are re-owned to the fresh model; a non-default per-agent pin and durable tuning are left untouched; and a fresh config with no agent primary leaves the backup routing intact.

Changes

  • src/lib/state/openclaw-config-merge.ts: add agentPrimaryModelPath ownership + reconcileAgentPrimaryModel (with readAgentPrimaryModelRef/updateMainAgentListModel), called at the end of mergeOpenClawRestoredConfig.
  • src/lib/state/openclaw-config-merge.test.ts: cover the re-own contract and its boundaries.

Type of Change

  • Code change (feature, bug fix, or refactor)

Verification

  • Unit tests for the merge contract pass (openclaw-config-merge.test.ts, 24 tests)
  • npm run typecheck:cli and npm run build:cli pass
  • Reproduced end-to-end on our DGX Spark aarch64 test host (managed vLLM, MCP present, rebuild --tool-disclosure progressive): before the fix the agent primary stayed on the old model; after the fix it follows the switch, consistent with the route/backend, and the MCP server survives the rebuild.
  • No secrets, API keys, or credentials committed

AI Disclosure

  • AI-assisted — tool: Claude Code

Signed-off-by: Yanyun Liao yanyunl@nvidia.com

Summary by CodeRabbit

  • Bug Fixes
    • Fixed configuration restoration to re-assign agent model routing to the newly selected managed primary model after a rebuild.
    • Preserved durable agent settings and intentional per-agent model assignments during restore.
    • If the fresh config doesn’t provide a primary model, restoration now keeps the backed-up primary model routing unchanged.
  • Tests
    • Added coverage for primary-model routing behavior during sandbox re-creation and config merge scenarios.

)

Switching a managed model on a sandbox that has MCP servers cannot take the
generic onboard recreate path (recreate is refused to preserve MCP state), so
the switch is completed via `rebuild --yes --tool-disclosure progressive`.
After that rebuild the gateway route, the vLLM backend, and `status` all show
the new model, but `openclaw.json` -> `agents.defaults.model.primary` (and
`openclaw agents list`) still show the OLD model.

Root cause: the rebuild config-restore merge treats the whole `agents` section
as backup-durable, so `agents.defaults.model.primary` is inherited from the
pre-switch backup snapshot even though `models.providers` routing was already
refreshed from the registry. Restore never reconciled the agent's model
routing reference against the fresh rebuild.

Re-own the primary model routing from the fresh config after the merge: rewrite
`agents.defaults.model.primary` and the matching main/default `agents.list[]`
model from the freshly generated config, and only when the fresh config carries
a primary. Other agent config (durable tuning, customAgents, and intentional
non-default per-agent model pins) is still inherited from the backup. This
revives the approach proposed in the previously-closed #7063.

Fixes #7210

Signed-off-by: Yanyun Liao <yanyunl@nvidia.com>
@coderabbitai

coderabbitai Bot commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: ae2f8eb2-5047-4d5c-b097-d56e1341b16c

📥 Commits

Reviewing files that changed from the base of the PR and between 62ba561 and d8fb709.

📒 Files selected for processing (3)
  • src/lib/state/openclaw-config-merge.test.ts
  • src/lib/state/openclaw-config-merge.ts
  • src/lib/state/sandbox-recreated-openclaw-restore.test.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • src/lib/state/openclaw-config-merge.ts

📝 Walkthrough

Walkthrough

The restore merge now re-owns the agent primary model from the fresh configuration, updates the corresponding default routing entry, preserves backup-only settings and non-default pins, and retains backup routing when the fresh configuration omits a primary model.

Changes

Agent routing reconciliation

Layer / File(s) Summary
Define and implement routing reconciliation
src/lib/state/openclaw-config-merge.ts
The restore ownership contract adds agents.defaults.model.primary; reconciliation helpers update merged agent defaults and the main or eligible default list entry from the fresh configuration.
Apply and validate merged routing
src/lib/state/openclaw-config-merge.ts, src/lib/state/openclaw-config-merge.test.ts, src/lib/state/sandbox-recreated-openclaw-restore.test.ts
The merge invokes reconciliation after existing merges, with tests covering managed-model switches, preserved pins and settings, absent fresh primary models, no-main-agent fallback, and sandbox restoration.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related issues

Possibly related PRs

Suggested labels: bug-fix, integration: openclaw, area: sandbox

Suggested reviewers: jyaunches, cv, cjagwani

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately and concisely summarizes the main change: refreshing stale agent primary model routing during config restore.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/7210-reconcile-agent-primary-model-on-rebuild

Comment @coderabbitai help to get the list of available commands.

@github-code-quality

github-code-quality Bot commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage in commit 66aed04 in the fix/7210-reconcile-a... branch remains at 96%, unchanged from commit a5ccdca in the main branch.

TypeScript / code-coverage/cli

The overall coverage in commit 66aed04 in the fix/7210-reconcile-a... branch remains at 80%, unchanged from commit a5ccdca in the main branch.

Show a code coverage summary of the most impacted files.
File main a5ccdca fix/7210-reconcile-a... 66aed04 +/-
src/lib/onboard...box-prebuild.ts 88% 69% -19%
src/lib/domain/.../connect-env.ts 97% 89% -8%
src/lib/state/o...config-merge.ts 96% 95% -1%
src/lib/actions...ess-recovery.ts 78% 77% -1%
src/lib/messagi...flow-planner.ts 93% 93% 0%
src/lib/actions...eway-restart.ts 90% 93% +3%
src/lib/onboard...host-anchors.ts 90% 94% +4%
src/lib/state/m...-acquisition.ts 84% 89% +5%
src/lib/platform.ts 84% 89% +5%
src/lib/onboard...corporate-ca.ts 92% 100% +8%

Updated July 20, 2026 15:04 UTC

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/lib/state/openclaw-config-merge.ts`:
- Around line 487-489: Remove the repository-local retirement ledger comment
describing the temporary removal condition from the surrounding
configuration-merge code. Do not replace it with another migration-status or
completion note; track the retirement externally in the linked GitHub issue or
PR.
- Around line 468-470: Update the default-agent selection loop so `defaultAgent`
is assigned only when `entry.default === true` and `typeof entry.model ===
"string"`, ensuring later valid default entries remain eligible. Add a
regression test covering configuration without a `main` agent where an
invalid-model default precedes a routable default, and verify the latter
receives `primaryModelRef`.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: ef6babc5-8ffc-4526-9265-b3606feecadc

📥 Commits

Reviewing files that changed from the base of the PR and between 3c4ea5a and 62ba561.

📒 Files selected for processing (2)
  • src/lib/state/openclaw-config-merge.test.ts
  • src/lib/state/openclaw-config-merge.ts

Comment thread src/lib/state/openclaw-config-merge.ts Outdated
Comment thread src/lib/state/openclaw-config-merge.ts Outdated
@github-actions

github-actions Bot commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor — Informational

Advisor assessment: Informational / high confidence
Next action: No advisor follow-up needed.
Findings: 0 blockers · 0 warnings · 0 suggestions
Status: No actionable findings remain in the canonical review ledger.

Model lanes

  • GPT-5.6 Terra (primary): Completed · high confidence · 0 blockers · 0 warnings · 0 suggestions
  • Nemotron 3 Ultra (second opinion): Completed · high confidence · 0 blockers · 0 warnings · 0 suggestions
  • Model comparison: normalized findings match; normalized E2E selections differ; severity counts match.

Nemotron output stays in workflow artifacts and does not change the assessment above.

E2E guidance

Advisory only. E2E / PR Gate selects and runs jobs independently.

Recommended E2E: onboard-repair, onboard-resume

3 optional E2E recommendations
  • rebuild-openclaw
  • state-backup-restore
  • sandbox-rebuild

Workflow run details

This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge.

@ericksoa ericksoa left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maintainer review complete on exact head 66aed04 against current main a5ccdca. Reviewed the config-restore ownership fix and regression coverage; 29/29 focused local tests passed after the latest base refresh, both advisor lanes reported no findings, CodeRabbit has no unresolved major/critical findings, and all required exact-head CI/E2E gates are green.

@ericksoa
ericksoa merged commit 5b547cd into main Jul 20, 2026
52 checks passed
@ericksoa
ericksoa deleted the fix/7210-reconcile-agent-primary-model-on-rebuild branch July 20, 2026 16:06
@wscurran wscurran added area: cli Command line interface, flags, terminal UX, or output area: inference Inference routing, serving, model selection, or outputs bug-fix PR fixes a bug or regression integration: openclaw OpenClaw integration behavior platform: dgx-station Affects DGX Station hardware or workflows labels Jul 20, 2026
@ericksoa ericksoa mentioned this pull request Jul 20, 2026
21 tasks
ericksoa added a commit that referenced this pull request Jul 20, 2026
<!-- markdownlint-disable MD041 -->
## Summary

Add the canonical dated changelog entry for NemoClaw `v0.0.89` before
the release plan captures the tag commit.
The entry summarizes the user-visible Station preparation, inference,
recovery, policy-disclosure, and CLI-containment changes merged since
`v0.0.88`.

## Changes

- Add `docs/changelog/2026-07-20.mdx` with the exact `## v0.0.89`
release heading, parser-safe SPDX comment, summary, and detailed
bullets.
- Link each shipped theme to the most specific published OpenClaw
documentation routes.
- Keep internal E2E, advisory-registry, and review-workflow refactors
out of the user-facing release summary.

Source summary:

- #7214, #7241, #7237, #7223, #7204, #7202, #7183, and #7090 ->
`docs/changelog/2026-07-20.mdx`: Summarize qualified DGX Station
identity, package-state, PackageKit, DKMS, and reboot-handoff fixes.
- #7242, #7221, #7186, #7164, and #6874 ->
`docs/changelog/2026-07-20.mdx`: Summarize inference endpoint
provenance, provider attachment, managed vLLM cleanup and progress, and
Ollama selection guidance.
- #7225, #7216, #7192, #7136, #7096, and #6910 ->
`docs/changelog/2026-07-20.mdx`: Summarize sandbox readiness, recovery
guidance, rebuilt model routing, durable Hermes state, and gateway
restart behavior.
- #7187 -> `docs/changelog/2026-07-20.mdx`: Summarize complete effective
messaging-preset egress disclosure before policy mutation.
- #7218, #7165, and #7184 -> `docs/changelog/2026-07-20.mdx`: Summarize
structured output containment, terminal-safe route display, and Hermes
forward cleanup.

## Type of Change

- [ ] Code change (feature, bug fix, or refactor)
- [ ] Code change with doc updates
- [x] Doc only (prose changes, no code sample modifications)
- [ ] Doc only (includes code sample changes)

## Quality Gates

- [ ] Tests added or updated for changed behavior
- [ ] Existing tests cover changed behavior — justification:
- [x] Tests not applicable — justification: This PR adds release-history
prose only; the focused changelog contract test validates its required
structure and routes.
- [x] Docs updated for user-facing behavior changes
- [ ] Docs not applicable — justification:
- [ ] Sensitive paths changed (security, policy, credentials, preflight,
onboarding, inference, runner, sandbox, or messaging)
- [ ] Sensitive-path review completed or maintainer-approved waiver
recorded — reviewer/approval link/justification:
- [ ] Non-success, skipped, or missing CI check accepted by maintainer —
check name, approval link, and follow-up issue:

## Verification

- [x] PR description includes a `Signed-off-by:` line and every commit
appears as `Verified` in GitHub
- [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or
`npm run check:diff` passed when hooks were skipped or unavailable
- [x] Targeted behavior tests pass for the current change set, or tests
are marked not applicable above — `mise exec node@22.23.1 -- npx vitest
run test/changelog-docs.test.ts` (6 passed).
- [ ] Applicable broad gate passed — `npm test` for broad
runtime/test-harness changes; `npm run check` for repo-wide
validation/coverage changes — not applicable to this doc-only release
entry.
- [x] Quality Gates section completed with required justifications or
waivers
- [x] No secrets, API keys, or credentials committed
- [ ] `npm run docs` builds without warnings (doc changes only) — `mise
exec node@22.23.1 -- npm run docs` completed with 0 errors and 2
existing site-wide warnings.
- [x] Doc pages follow the [style
guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md)
(doc changes only)
- [ ] New doc pages include SPDX header and frontmatter (new pages only)
— the native changelog entry uses the required parser-safe MDX SPDX
comment and does not use frontmatter.

---
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Expanded DGX Station installation support for qualified GB300 and
OTA-upgraded environments.
* Preserved selected inference providers, endpoints, model pins, and
tuning settings during sandbox creation and rebuilds.
* Improved sandbox recovery by validating availability and stability
before restarting services.
* Added clearer policy output showing the complete effective messaging
egress configuration.

* **Bug Fixes**
* Hardened status and inference route displays by sanitizing terminal
control characters.
* Improved Hermes uninstall behavior by stopping detached dashboard
forwarding.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: cli Command line interface, flags, terminal UX, or output area: inference Inference routing, serving, model selection, or outputs bug-fix PR fixes a bug or regression integration: openclaw OpenClaw integration behavior platform: dgx-station Affects DGX Station hardware or workflows

Projects

None yet

4 participants