Skip to content

fix(e2e): restore CLI artifact for security posture - #8621

Merged
cv merged 1 commit into
mainfrom
codex/restore-security-posture-cli-artifact
Aug 8, 2026
Merged

fix(e2e): restore CLI artifact for security posture#8621
cv merged 1 commit into
mainfrom
codex/restore-security-posture-cli-artifact

Conversation

@apurvvkumaria

@apurvvkumaria apurvvkumaria commented Aug 8, 2026

Copy link
Copy Markdown
Collaborator

Summary

The trusted security-posture job now restores the exact candidate CLI artifact produced by generate-matrix before telemetry, installation, and live testing. This supplies nemoclaw/dist/shared/sandbox-name.cjs while retaining the existing receipt, provenance, digest, and security-posture gates.

Changes

  • Restore the pinned, content-addressed CLI artifact immediately after the no-build prepare-e2e step in security-posture.
  • Register security-posture as an exact-artifact consumer and require sandbox-name.cjs at both producer and restore contract boundaries.
  • Update workflow tests, runner-comparison ordering coverage, and the internal E2E telemetry description.
  • Root cause: prepare-e2e skipped the CLI build for security-posture, while the artifact-consumer registry also excluded that job even though generate-matrix packaged the required shared module. The former contract test encoded that exclusion, so it did not catch the missing restore. The failure is visible in current-main run 31243363889.

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with doc updates
  • Doc only (prose changes, no code sample modifications)
  • Doc only (includes code sample changes)

Quality Gates

  • Tests added or updated for changed behavior
  • Existing tests cover changed behavior — justification:
  • Tests not applicable — justification:
  • Docs updated for user-facing behavior changes
  • Docs not applicable — justification: No user-facing behavior changes; the internal test/e2e/README.md was updated to match the trusted workflow ordering.
  • Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
  • Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: Independent frozen-diff security and correctness review passed for 4c2e1db90; receipt, provenance, credential scope, failure ordering, and security-posture controls remain intact.
  • Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue:

Documentation Writer Review

  • Documentation writer subagent reviewed the completed changes
  • Result: docs-updated
  • Evidence: Updated test/e2e/README.md to document artifact restoration before telemetry and the resulting security-posture measurement window.
  • Agent: Codex Desktop

DGX Station Hardware Evidence

  • Tested on DGX Station
  • Tested commit:
  • Station profile/scenario:
  • Result:
  • Supporting evidence:

Verification

  • PR description includes a Signed-off-by: line and every commit appears as Verified in GitHub
  • Normal pre-commit, commit-msg, and pre-push hooks passed, or npm run validate:pr passed after refreshing origin/main when hooks were skipped or unavailable
  • Targeted behavior tests pass for the current change set, or tests are marked not applicable above — command/result or justification: Four focused E2E-support files passed 78 tests; the origin/main path-scoped selection passed 397 tests across 36 files.
  • Applicable broad gate passed — npm test for broad runtime/test-harness changes; npm run check for repo-wide validation/coverage changes — command/result:
  • Quality Gates section completed with required justifications or waivers
  • No secrets, API keys, or credentials committed
  • npm run docs builds without warnings (doc changes only)
  • Doc pages follow the style guide (doc changes only)
  • New doc pages include SPDX header and frontmatter (new pages only)

Signed-off-by: Apurv Kumaria akumaria@nvidia.com

Summary by CodeRabbit

  • Bug Fixes

    • Improved security-posture end-to-end workflow reliability by restoring the exact CLI artifact before telemetry and tests run.
    • Ensured required shared CLI components are included during packaging and restoration.
    • Updated workflow validation to enforce consistent artifact and telemetry initialization behavior.
  • Documentation

    • Clarified artifact restoration and telemetry initialization timing in end-to-end testing guidance.

Signed-off-by: Apurv Kumaria <akumaria@nvidia.com>
@apurvvkumaria apurvvkumaria self-assigned this Aug 8, 2026
@coderabbitai

coderabbitai Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 5aafd7e1-d4e0-4d3f-9067-7954ab4219e1

📥 Commits

Reviewing files that changed from the base of the PR and between 5722e63 and 4c2e1db.

📒 Files selected for processing (7)
  • .github/workflows/e2e.yaml
  • test/e2e/README.md
  • test/e2e/support/cli-artifact-workflow-boundary.test.ts
  • test/e2e/support/runner-comparison-workflow-boundary.test.ts
  • tools/e2e/cli-artifact-workflow-boundary.mts
  • tools/e2e/prepare-e2e-workflow-boundary.mts
  • tools/e2e/runner-comparison-workflow-boundary.mts
💤 Files with no reviewable changes (2)
  • tools/e2e/prepare-e2e-workflow-boundary.mts
  • tools/e2e/runner-comparison-workflow-boundary.mts

📝 Walkthrough

Walkthrough

The E2E workflow now restores the exact-commit CLI artifact for security-posture. Boundary validation requires sandbox-name.cjs and checks artifact consumers. Runner-comparison documentation and tests now place telemetry initialization after artifact restoration.

Changes

CLI artifact alignment

Layer / File(s) Summary
Shared artifact contract and validation
tools/e2e/cli-artifact-workflow-boundary.mts, test/e2e/support/cli-artifact-workflow-boundary.test.ts
The contract includes security-posture as a CLI artifact consumer. Producer and restore validation require sandbox-name.cjs. Tests cover missing shared modules and exact artifact restoration.
Security-posture restoration and initialization
.github/workflows/e2e.yaml, tools/e2e/prepare-e2e-workflow-boundary.mts, tools/e2e/runner-comparison-workflow-boundary.mts, test/e2e/support/runner-comparison-workflow-boundary.test.ts, test/e2e/README.md
The security-posture job restores the exact-commit artifact before telemetry initialization and tests. Boundary expectations and telemetry documentation reflect this order.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

Suggested labels: release-target, v0.0.105

Suggested reviewers: cv, ericksoa, jyaunches

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes restoring the CLI artifact for the security-posture E2E job.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
  • Create stacked PR
  • Commit on current branch
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/restore-security-posture-cli-artifact

Comment @coderabbitai help to get the list of available commands.

@github-code-quality

github-code-quality Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage in commit 4c2e1db in the codex/restore-securi... branch remains at 96%, unchanged from commit 5722e63 in the main branch.

TypeScript / code-coverage/cli

The overall coverage in commit 4c2e1db in the codex/restore-securi... branch remains at 81%, unchanged from commit c6fb802 in the main branch.

Show a code coverage summary of the most impacted files.
File main c6fb802 codex/restore-securi... 4c2e1db +/-
src/lib/core/pr...mpt-activity.ts 92% 67% -25%
src/lib/policy/...ateway-state.ts 73% 56% -17%
src/lib/policy/index.ts 62% 63% +1%
src/lib/inferen...er-lifecycle.ts 76% 77% +1%
src/lib/adapters/http/probe.ts 86% 87% +1%
src/lib/credentials/store.ts 55% 57% +2%
src/lib/inferen...tion-cluster.ts 83% 85% +2%
src/lib/policy/...scope-render.ts 94% 97% +3%
src/lib/trace.ts 90% 94% +4%
src/lib/inferen...vllm-api-key.ts 83% 90% +7%

Updated August 08, 2026 14:56 UTC

@github-actions

github-actions Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor — No blocking findings reported

Advisor assessment: No blocking advisor findings reported
Next action: No advisor follow-up needed.
Findings: 0 blockers · 0 warnings · 0 suggestions

Model lanes

  • GPT-5.6 Terra (primary): Completed · high confidence · 0 blockers · 0 warnings · 0 suggestions
  • Nemotron 3 Ultra (second opinion): Failed

Second-opinion terminology and E2E selections are advisory. Live E2E does not run automatically for pull requests.

2 semantic terminology decisions

Terminology decisions are advisory. They affect the assessment only when a separate finding identifies concrete semantic impact.

  • established — exact-commit CLI artifact at .github/workflows/e2e.yaml:5171: Keep the established term.
  • justified — shared CLI modules at test/e2e/support/cli-artifact-workflow-boundary.test.ts:931: Keep the modifier because the test title identifies the restored module requirement.

E2E guidance

Advisory only. A maintainer can dispatch the default E2E suite against this exact revision.

Recommended E2E: cloud-inference, cloud-onboard, security-posture

Workflow run details

This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge.

@cv
cv merged commit 37f5475 into main Aug 8, 2026
64 of 67 checks passed
@cv
cv deleted the codex/restore-security-posture-cli-artifact branch August 8, 2026 14:58
@wscurran wscurran added chore Build, CI, dependency, or tooling maintenance area: e2e End-to-end tests, nightly failures, or validation infrastructure area: ci CI workflows, checks, release automation, or GitHub Actions security labels Aug 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: ci CI workflows, checks, release automation, or GitHub Actions area: e2e End-to-end tests, nightly failures, or validation infrastructure chore Build, CI, dependency, or tooling maintenance security

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants