fix(onboard): replace rebuild recovery flag with authorization receipt - #6675
Merged
Conversation
Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughThe change replaces the authoritative rebuild recovery flag with a target-bound, time-limited provider recovery receipt. Receipts are minted after preflight, activated for matching sessions, propagated through onboarding, and validated against reservations inside provider recovery locks. ChangesProvider recovery authorization
Estimated code review effort: 4 (Complex) | ~60 minutes Sequence Diagram(s)sequenceDiagram
participant RebuildPreflight
participant RebuildProviderFlowOptions
participant HandleProviderInferenceState
participant ProviderRecoveryReceiptLedger
participant ProviderInferenceRecovery
RebuildPreflight->>RebuildProviderFlowOptions: mint and attach target-bound receipt
RebuildProviderFlowOptions->>ProviderRecoveryReceiptLedger: activate receipt for session
ProviderRecoveryReceiptLedger-->>HandleProviderInferenceState: activated receipt and ledger
HandleProviderInferenceState->>ProviderInferenceRecovery: pass recovery authorization artifacts
ProviderInferenceRecovery->>ProviderRecoveryReceiptLedger: validate receipt in mutation lock
ProviderRecoveryReceiptLedger-->>ProviderInferenceRecovery: allow or reject recovery
Suggested labels: Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
Contributor
Code Coverage OverviewLanguages: TypeScript TypeScript / code-coverage/pluginThe overall coverage remains at 96%, unchanged from the TypeScript / code-coverage/cliThe overall coverage in the Show a code coverage summary of the most impacted files.
Updated |
Contributor
E2E Advisor RecommendationRequired E2E: Dispatch hint: Full advisor summaryE2E Recommendation AdvisorBase: Required E2E
Optional E2E
New E2E recommendations
Dispatch hint
|
Contributor
E2E Target RecommendationRequired E2E targets: Dispatch required E2E targets:
Full E2E target advisor summaryE2E Target AdvisorBase: Required E2E targets
Optional E2E targets
Relevant changed files
|
Contributor
PR Review Advisor (Nemotron Ultra) — Changes requestedMerge posture: Do not merge yet Action checklist
Findings index
Review findings by urgency: 0 required fixes, 3 items to resolve/justify, 1 in-scope improvement
|
Contributor
PR Review Advisor — No blocking findingsMerge posture: No blocking advisor findings This is an automated review. Required findings need action before merge. Warnings and optional suggestions do not require a response or follow-up. A human maintainer makes the final merge decision. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/lib/onboard/rebuild-route-handoff.ts`:
- Around line 120-126: Update routesMatch to also compare
left.preferredInferenceApi with right.preferredInferenceApi, while preserving
the existing provider, model, and endpointUrl comparisons.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: c1db93c8-1866-4b38-af03-db60d14b643a
📒 Files selected for processing (13)
src/lib/actions/sandbox/rebuild-gpu-opt-out.tssrc/lib/actions/sandbox/rebuild-preflight-target-phase.tssrc/lib/onboard/authoritative-rebuild-target.test.tssrc/lib/onboard/authoritative-rebuild-target.tssrc/lib/onboard/machine/core-flow-phases.tssrc/lib/onboard/machine/handlers/provider-inference-authoritative-recovery.test.tssrc/lib/onboard/machine/handlers/provider-inference-recovery-gating.test.tssrc/lib/onboard/machine/handlers/provider-inference-recovery.tssrc/lib/onboard/machine/handlers/provider-inference.test-support.tssrc/lib/onboard/machine/handlers/provider-inference.tssrc/lib/onboard/rebuild-route-handoff.test.tssrc/lib/onboard/rebuild-route-handoff.tssrc/lib/onboard/types.ts
…InferenceApi routesMatch compared provider, model, and endpointUrl but not preferredInferenceApi, so a receipt could activate against a target on a different inference API family sharing the same provider/model/endpoint. Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/lib/onboard/machine/handlers/provider-inference.test.ts`:
- Around line 824-839: Update the recovery test around
handleProviderInferenceState to remove or disable authoritativeResumeConfig and
keep provider_selection incomplete, ensuring the activated
providerRecoveryReceipt and providerRecoveryReceiptLedger are the only
authorization path. Assert the public handler succeeds through receipt
validation and verify the legacy resume shortcut cannot execute.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: a24b7b61-7458-4174-b325-921446644cb9
📒 Files selected for processing (3)
src/lib/onboard/machine/handlers/provider-inference.test.tssrc/lib/onboard/rebuild-route-handoff.test.tssrc/lib/onboard/rebuild-route-handoff.ts
🚧 Files skipped from review as they are similar to previous changes (2)
- src/lib/onboard/rebuild-route-handoff.test.ts
- src/lib/onboard/rebuild-route-handoff.ts
cv
approved these changes
Jul 12, 2026
cv
left a comment
cv
left a comment
Collaborator
There was a problem hiding this comment.
Reviewed current PR commit e3f5d23. All 50 checks are green, including the seven selected live E2E scenarios (cloud-onboard, inference-routing, network-policy, onboard-repair, onboard-resume, state-backup-restore, and upgrade-stale-sandbox). The focused receipt/recovery suite passes 76 tests locally, CLI type-check and normal hooks passed, replay and fabricated-provenance paths are covered, all eight commits are GitHub Verified with DCO present, no review threads remain unresolved, CodeRabbit has no blocking findings, and the provenance-valid primary advisor recommends merge_as_is.
Hadar301
pushed a commit
to Hadar301/NemoClaw-OpenShift
that referenced
this pull request
Jul 12, 2026
NVIDIA#6675) <!-- markdownlint-disable MD041 --> ## Summary Replace the authoritative-rebuild incomplete-session boolean bypass with a one-shot, target-bound provider-recovery authorization receipt. The receipt is minted after locked rebuild preflight, activated against the onboard session at provider selection, and rechecked inside the gateway and sandbox mutation locks, so recovering a recorded provider is bound to the exact rebuild target and to the reservation owner. ## Related Issue Resolves NVIDIA#6666 ## Changes - Add `ProviderRecoveryReceipt` with `mintProviderRecoveryReceipt` and a single-use `createProviderRecoveryReceiptLedger` (`activate` binds one session and refuses replay, cross-sandbox, expired, malformed, or route-mismatched receipts; `validateInLock` re-binds to the live reservation owner) in `rebuild-route-handoff.ts`. - Mint the receipt after preflight validates the target in `rebuild-preflight-target-phase.ts`; carry it on `OnboardOptions` and the rebuild recreate options. - Activate the receipt against the live session and target in `rebuildProviderFlowOptions`, threading it through the core flow phases into the provider-inference handler. - Drop the `authoritativeResumeConfig` incomplete-session bypass and its temporary comment in the recovery gate; the gate now keys on the activated receipt and the in-lock recheck also runs `validateInLock`. - Add unit coverage for the receipt primitive and two behavioural tests: only the reservation owner recovers the same sandbox concurrently, and a foreign reservation introduced after selection is rejected inside the mutation lock. ## Type of Change - [x] Code change (feature, bug fix, or refactor) - [ ] Code change with doc updates - [ ] Doc only (prose changes, no code sample modifications) - [ ] Doc only (includes code sample changes) ## Quality Gates - [x] Tests added or updated for changed behavior - [ ] Existing tests cover changed behavior — justification: - [ ] Tests not applicable — justification: - [ ] Docs updated for user-facing behavior changes - [x] Docs not applicable — justification: - [x] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [ ] Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: ## Verification - [x] PR description includes the DCO sign-off declaration and every commit appears as `Verified` in GitHub - [ ] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or `npm run check:diff` passed when hooks were skipped or unavailable - [x] Targeted behavior tests pass for the current change set, or tests are marked not applicable above — command/result or justification: - [ ] Applicable broad gate passed — `npm test` for broad runtime/test-harness changes; `npm run check` for repo-wide validation/coverage changes — command/result: - [x] Quality Gates section completed with required justifications or waivers - [x] No secrets, API keys, or credentials committed - [ ] `npm run docs` builds without warnings (doc changes only) - [ ] Doc pages follow the [style guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md) (doc changes only) - [ ] New doc pages include SPDX header and frontmatter (new pages only) --- Signed-off-by: Tinson Lai <tinsonl@nvidia.com> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added one-time provider recovery receipts with an associated receipt ledger to authorize recorded-provider recovery during rebuilds. * Receipts are minted during rebuild preflight and carried through the onboarding/rebuild flow to enable secure resumption. * **Bug Fixes** * Strengthened recovery gating to block unauthorized concurrent and lock-time recovery attempts. * Added stricter validation for receipt expiry, one-shot replay, session/sandbox/gateway matching, and reservation ownership. * **Tests** * Expanded coverage for receipt minting, ledger activation, replay/session binding, and recovery recheck behavior under concurrent/foreign reservations. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: Tinson Lai <tinsonl@nvidia.com> Co-authored-by: Carlos Villela <cvillela@nvidia.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replace the authoritative-rebuild incomplete-session boolean bypass with a one-shot, target-bound provider-recovery authorization receipt. The receipt is minted after locked rebuild preflight, activated against the onboard session at provider selection, and rechecked inside the gateway and sandbox mutation locks, so recovering a recorded provider is bound to the exact rebuild target and to the reservation owner.
Related Issue
Resolves #6666
Changes
ProviderRecoveryReceiptwithmintProviderRecoveryReceiptand a single-usecreateProviderRecoveryReceiptLedger(activatebinds one session and refuses replay, cross-sandbox, expired, malformed, or route-mismatched receipts;validateInLockre-binds to the live reservation owner) inrebuild-route-handoff.ts.rebuild-preflight-target-phase.ts; carry it onOnboardOptionsand the rebuild recreate options.rebuildProviderFlowOptions, threading it through the core flow phases into the provider-inference handler.authoritativeResumeConfigincomplete-session bypass and its temporary comment in the recovery gate; the gate now keys on the activated receipt and the in-lock recheck also runsvalidateInLock.Type of Change
Quality Gates
Verification
Verifiedin GitHubpre-commit,commit-msg, andpre-pushhooks passed, ornpm run check:diffpassed when hooks were skipped or unavailablenpm testfor broad runtime/test-harness changes;npm run checkfor repo-wide validation/coverage changes — command/result:npm run docsbuilds without warnings (doc changes only)Signed-off-by: Tinson Lai tinsonl@nvidia.com
Summary by CodeRabbit
New Features
Bug Fixes
Tests