Resync the vendored omni-github result schema with upstream - #6897
Resync the vendored omni-github result schema with upstream#6897Neil4561 wants to merge 2 commits into
Conversation
$defs.customValue (and its nested array items) used oneOf listing both integer and number. Under JSON Schema draft 2020-12 integer is a subset of number, so any integral value (e.g. 4096 or 100.0) matches both branches and oneOf rejects it. The upload action enforces this schema as a gate and skips the upload on validation failure, so a producer emitting integral numeric custom values has its results silently dropped. Change both oneOf to anyOf to preserve the intended union while allowing integral numbers.
The copy under .github/actions/upload-omni-github-test-results was taken once in isaac-sim#6234 and never refreshed, so it has drifted from the source in NVIDIA-Omniverse/omni-github at docs/test-results/clients/artifact-upload/ schemas/result-json.schema.json. The upload action validates every result file against this copy and silently drops anything that fails, so a stale copy can reject payloads the service itself would accept. Replace it wholesale rather than patching individual rules. The file is now byte-identical to upstream. The drift is additive: upstream gained profileDetailString and seven optional testRow fields (error, failure_component, failure_layer, reliability_class, retry_kind, retry_reason, test_category). Nothing this repository relies on was removed. The converter's own output and the perf-smoke gate's result JSON both validate against the resynced schema unchanged. This also subsumes the oneOf -> anyOf fix in isaac-sim#6433, which upstream had already made; that PR remains the minimal standalone fix if this lands after it.
Greptile SummaryResynchronizes the vendored omni-github result schema with upstream, expanding accepted test metadata and aligning custom-value validation.
Confidence Score: 5/5The PR appears safe to merge, with no concrete repository payload path found that regresses under the synchronized schema. The local result converter does not emit the newly added metadata or array-valued custom fields, while the schema changes expand accepted metadata and restore upstream custom-value semantics. Important Files Changed
Reviews (1): Last reviewed commit: "Resync the vendored omni-github result s..." | Re-trigger Greptile |
There was a problem hiding this comment.
Isaac Lab Review Bot
Reviewed the vendored omni-github schema resync, including the added optional testRow fields, the new shared string definition, and the customValue validation changes. No actionable finding is supported by the supplied patch.
- Design and architecture: Keeping the repository’s local validation schema synchronized with the upstream service schema is consistent with the pinned-vendor design. The patch does not alter workflow wiring or producer paths.
- API: Existing required
testRowfields remain unchanged, while the seven new fields are optional and valid underadditionalProperties: false. The new$reftargets resolve correctly. The array-levelanyOfmakes custom arrays homogeneous, so this refresh is not strictly additive for previously accepted mixed-type arrays, but that behavior matches the stated upstream contract and does not establish a defect in this vendor resync. - Implementation: The added definitions, references, enums, and type constraints are internally consistent. Changing scalar
customValuefromoneOftoanyOfpermits integers that also satisfy JSON Schema’snumbertype. The supplied verification covers the repository converter output and perf-smoke payload; no changed producer emits the newly added fields.
No blocking issues. No inline issue met the actionable-evidence threshold; the assessment above records the review feedback.
Automated review; human maintainers own approval decisions.
Follows up on @fatima-anes's review of #6433: rather than patching one rule of a stale copy, this refreshes the whole vendored schema from upstream.
Why
.github/actions/upload-omni-github-test-results/result-json.schema.jsonis a pinned copy of the schema inNVIDIA-Omniverse/omni-githubatdocs/test-results/clients/artifact-upload/schemas/result-json.schema.json. Pinning is deliberate —action.ymlnotes the omni-github team recommends validating against a local copy — but ours was taken once in #6234 on 2026-06-26 and never refreshed.That matters because the upload action treats this file as a hard gate and silently drops any result that fails validation:
A stale copy therefore rejects payloads the service itself would accept, and says nothing about it.
What changed
The file is now byte-identical to upstream. The drift is entirely additive — upstream gained
$defs.profileDetailStringand seven optionaltestRowfields:error,failure_component,failure_layer,reliability_class,retry_kind,retry_reason,test_categoryNothing this repository relies on was removed upstream, and no producer here emits the new fields yet, so this is a no-op for current uploads and unblocks the new ones.
Relationship to #6433
Upstream already uses
anyOfforcustomValue, so #6433 was correcting our copy toward upstream rather than diverging from it. This branch is based on #6433 and subsumes it.Either order works. If #6433 merges first this reduces to the remaining drift; if it doesn't, #6433 can be closed as superseded. #6433 is the smaller, already-approved change, so merging it first is the lower-risk path — it is what unblocks the perf-smoke gate's dashboard rows.
Verification
cmp).test_junit_to_omni_github_results.pypasses (4 tests).num_envs: 4096,fps_mean), which is precisely what Fix omni-github result schema rejecting integral numeric custom values #6433 addresses.Follow-up, not in this PR
The perf-smoke gate carries a
_relax_custom_value_oneoftest helper that rewritesoneOftoanyOfin memory before validating, added while #6433 was pending. It should be deleted once this or #6433 lands, but it lives intools/perf_smoke_test/, which is not ondevelopyet, so it belongs with #6864.Also worth considering separately:
failure_layer(code | infrastructure | environment | upstream | downstream) andretry_kindmap closely onto distinctions the perf gate already makes — a stale CI image is anenvironmentfailure, not acodeone. Today that only exists as prose in the PR comment.