Add optional rerun debug view of placement validation - #984
Draft
xyao-nv wants to merge 12 commits into
Draft
Conversation
xyao-nv
changed the title
alexmillane
reviewed
Jul 31, 2026
alexmillane
left a comment
alexmillane
left a comment
Collaborator
There was a problem hiding this comment.
Thanks for adding this functionality! Looks pretty good.
See suggestions below!
| self.params = params or ObjectPlacerParams() | ||
| self._solver = RelationSolver(params=self.params.solver_params) | ||
| # Populated before the validators are built so a check can add its own layer to the same view. | ||
| self.params.debug_visualizer = get_or_create_placement_visualizer(self.params) |
Collaborator
There was a problem hiding this comment.
Suggestion that we only keep params in the params struct? I.e. no objects that implement run-time functionality.
Collaborator
Author
There was a problem hiding this comment.
Make sense, moved it to placer's member.
| # Per-check candidate indices that check was actually run on; expensive checks skip candidates. | ||
| evaluated_slots_by_check: dict[str, list[int]] = {} | ||
| layout_pass_verdicts_by_check: dict[str, list[bool]] = {} | ||
| # Layouts are drawn before the checks run so a check's own layer lands on top of its candidate. |
Collaborator
There was a problem hiding this comment.
"so a check's own layer lands on top of its candidate."
Not sure I understand that. What's a check's "layer"?
Collaborator
Author
There was a problem hiding this comment.
Agent's docstring is confusing af. Updated with mine.
| debug_visualize_rrd_path: str | None = None | ||
| """Path to record the debug visualization to as a Rerun ``.rrd`` file, for headless runs. | ||
|
|
||
| Enables the visualization on its own; combine with ``debug_visualize`` to both record and watch live.""" |
Collaborator
There was a problem hiding this comment.
Enables the visualization on its own? Does this mean that this also starts the live view?
Consider rewording.
| self, | ||
| positions: dict[ObjectBase, tuple[float, float, float]], | ||
| orientations: dict[ObjectBase, float], | ||
| batch_slot: int, |
Collaborator
There was a problem hiding this comment.
What is a batch_slot.
This is diffcult to detemine what it is from the variable name.
I would guess that slot is coming from rerun? This is just the index of the solution among the batch right? Suggestion to use batch_idx, solution_idx.
The slot terminology says what this variable is used for inside the function, not what it actually is. In general, prefer describing what something is, rather than what you intend to use it for.
Placement solving is sim-free, so its candidate layouts have been hard to inspect without starting Isaac Sim. Opting in via ObjectPlacerParams now streams every candidate to a Rerun viewer (or an .rrd, for headless runs). - Add PlacementRerunVisualizer: one process-wide view that draws each candidate's object boxes and the checks that accepted or rejected it, one frame per candidate on a `candidate` timeline. - Turn it on with ObjectPlacerParams.debug_visualize / debug_visualize_rrd_path; ObjectPlacer builds the view before its validators so a check can add its own layer to it. - Let the cuRobo reachability check layer on what only it knows: the robot base frame, the top-down grasps it solved, per-target reachable/unreachable verdicts, and its IK error scalars. - Keep frames aligned across checks: expensive checks only see the candidates that passed the cheap ones, so the placer tells the view which candidates the running check was handed. Signed-off-by: Xinjie Yao <xyao@nvidia.com>
- Add debug_visualize / debug_visualize_rrd_path to PlacementValidatorSpec so an env's YAML can ask for the Rerun view without editing Python. - Forward both through build_checks_for_placer_params into ObjectPlacerParams. - Turn the view on in the butter_raisin_box scene as a worked example. - Cover the default-off and forwarding paths in the graph-spec tests. Signed-off-by: Xinjie Yao <xyao@nvidia.com>
- Spawn the Rerun viewer under `setpriv --pdeathsig TERM` instead of `rr.spawn()`, so the kernel closes the window when the run exits. - Isaac Sim's `SimulationApp.close()` ends in `os._exit()`, so no `atexit` hook can do this. - Stops a stale viewer from holding port 9876, which silently made the next run's spawn a no-op. - Wait for the viewer to serve before logging, replacing the readiness wait `rr.spawn()` did. Signed-off-by: Xinjie Yao <xyao@nvidia.com>
- Take `debug_visualize: true` out of the shipped butter/raisin scene, where it spawned a viewer window on every build, and move the worked example to a test-data graph YAML. - Guard that with a test asserting no versioned env under `isaaclab_arena_environments` asks for it. - Track which candidates each check was run on, so the view no longer draws an expensive check as rejecting a layout it skipped; the summary count now derives from the same record. - Warn when the viewer port is already served, and confirm the spawned viewer is still alive once it answers, so a run cannot silently log into somebody else's window. - Fall back to killing a viewer that ignores SIGTERM instead of raising out of close(). Signed-off-by: Xinjie Yao <xyao@nvidia.com>
- Acceptance now follows the placer, which gates layouts on required_checks alone. - A failure the placer does not gate on reads as accepted, naming the advisory check. - Pull the wording out into summarize_candidate_verdict() so the rule is unit-testable. Signed-off-by: Xinjie Yao <xyao@nvidia.com>
- Show the env graph YAML and Python routes in the visualizer module, pointing at the worked example. - Say the same from the reachability check, whose layer only appears once that view is on. - Drop ACCEPTED_COLOR / REJECTED_COLOR, which nothing has ever drawn with. Signed-off-by: Xinjie Yao <xyao@nvidia.com>
Keep ObjectPlacerParams data-only: checks reach the live Rerun view through get_active_placement_visualizer() rather than a field on the params, and the per-batch layout/verdict logging moves off ObjectPlacer onto the visualizer that owns it. Rename debug_visualize_rrd_path to debug_visualize_output_path and the opaque batch "slot" to batch_index, shorten the docstrings that described code living elsewhere, and fix the enabled check name in the worked YAML example (ik_reachability -> ik_reachable), which was being silently dropped as unregistered. Signed-off-by: Xinjie Yao <xyao@nvidia.com>
Say what each piece does in plain terms: shorter constant docs, one straightforward line per function, and the remaining rationale (setpriv, the shared view, the deepcopy guard) stated directly instead of alluded to. Signed-off-by: Xinjie Yao <xyao@nvidia.com>
xyao-nv
force-pushed
the
xyao/feature/placement-rerun-debug-view
branch
from
August 3, 2026 05:59
64f02fe to
5c0022c
Compare
xyao-nv
commented
Aug 3, 2026
| def close(self) -> None: | ||
| """Flush pending data and shut down the viewer window this run spawned. Idempotent. | ||
|
|
||
| Only needed to close the window early -- a run that just exits leaves it to the viewer's |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Rerun debug view of placement validation
Detailed description
placement_validators.debug_visualize(and/ordebug_visualize_rrd_pathfor headless runs) in an env graph YAML, orObjectPlacerParams(debug_visualize=True).Example viz
In base docker (during solver validation):
https://github.com/user-attachments/assets/d73882ee-ab13-4309-aa7b-f19de10ad13c
In curobo docker (where reachability is enabled)
https://github.com/user-attachments/assets/55d99e37-6eff-4558-8444-e0a54f251f04