Enable SimReady search API in agentic env gen

[xyao-nv]

Summary

Enable SimReady search API in agentic env gen

Detailed description

• Remove NormalizeSpec from Add SimReady search to agentic environment generation. #964. Instead it sticks to SpecInference. And objects the catalog cannot cover now carry thesimready_usd_object` registry name and are searched in the SimReady library after spec inference
• Matching is on whole words, and the phrase's last word — the object itself — must appear in the asset path. To avoid substring matching, e.g. "bin" turns into cabinet USD.
• Each candidate is read before it is accepted, and only assets with exactly one rigid body are kept; multi-part props are turned down with the reason recorded in the traces.
• When nothing usable is found, the object ids are named back to the model for one further inference pass; an id already searched for is not searched again. If the replacement still cannot be spawned, the runner names the unavailable objects and shows the traces to users.
• Opt-in, off by default: --enable_simready_search on the environment generation runner, or the "SimReady search" expander in the review GUI.

simready_droid_pick_place_cans_hammer_maple_table.yaml

Other examples

When Simready ones are not valid, fall back to the closest from Simready / Arena registry.

[runner] generation traces:
  simready search found no usable asset for 'pancake'
  no SimReady asset for object 'pancake'; it cannot be spawned
  asking for a replacement of: pancake

=== ArenaEnvGraphSpec (env_name='droid_pick_bagel_into_grey_bin') ===

assets:
  droid                    role=embodiment         registry_name=droid_abs_joint_pos
  maple_table              role=background         registry_name=maple_table_robolab
  bagel                    role=object             registry_name=bagel_00_objaverse_robolab
  grey_bin                 role=object             registry_name=grey_bin_robolab

[runner] generation traces:
  simready search found no usable asset for 'disinfectant spray'
  no SimReady asset for object 'disinfectant_spray'; it cannot be spawned
  asking for a replacement of: disinfectant_spray

=== ArenaEnvGraphSpec (env_name='droid_pick_soft_scrub_into_grey_bin') ===

assets:
  droid                    role=embodiment         registry_name=droid_abs_joint_pos
  maple_table              role=background         registry_name=maple_table_robolab
  soft_scrub               role=object             registry_name=soft_scrub_ycb_robolab
  grey_bin                 role=object             registry_name=grey_bin_robolab

relations:
  is_anchor        subject=maple_table
  on               subject=soft_scrub  reference=maple_table
  on               subject=grey_bin  reference=maple_table

task: composition=TaskCompositionType.ATOMIC
  description: Pick up a soft scrub cleaner from the maple table and place it into a grey bin on the table.
  [0] kind=PickAndPlaceTask

[greptile-apps]

Greptile Summary

Adds opt-in SimReady asset discovery to agentic environment generation.

• Searches and ranks whole-word asset matches, rejecting assets that do not contain exactly one rigid body.
• Adds fallback inference for unavailable objects and reports generation traces in the CLI and review GUI.
• Registers a dynamic SimReady USD object with Physics variant-aware type and contact-sensor handling.
• Adds the optional search dependency and coverage for search, generation, USD helpers, asset construction, and GUI behavior.

Confidence Score: 4/5

The packaging incompatibility must be fixed before merging because no supported runtime can currently use the advertised SimReady search feature.

Native uv rejects the new extra alongside either IsaacLab group due to the declared dependency conflicts, and the Docker image does not install the extra, so enabled searches return no candidates in every supported installation path.

Files Needing Attention: pyproject.toml and docker/Dockerfile.isaaclab_arena

Important Files Changed

Filename	Overview
isaaclab_arena/agentic_environment_generation/environment_generation_agent.py	Orchestrates SimReady resolution and one replacement-inference pass for assets that cannot be found.
isaaclab_arena/agentic_environment_generation/simready_asset_search.py	Implements backend configuration, whole-word ranking, rigid-body validation, and synchronous search integration.
isaaclab_arena/assets/simready_object_library.py	Registers a dynamic USD-backed object and applies the SimReady Physics variant during construction.
isaaclab_arena/utils/usd/rigid_bodies.py	Adds variant-aware rigid-body inspection and caching for local or remote USD assets.
isaaclab_arena_examples/agentic_environment_generation/review_gui/generation_panel.py	Adds SimReady configuration controls, configuration-sensitive agent caching, and explicit warning feedback.
pyproject.toml	Adds the SimReady dependency but leaves it incompatible with both supported native simulator groups and absent from Docker.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[User prompt] --> B[Spec inference]
  B --> C{Catalog asset?}
  C -->|Yes| D[Validated Arena asset]
  C -->|No| E[SimReady search]
  E --> F{One rigid body?}
  F -->|Yes| G[Attach USD path and Physics variant]
  F -->|No| H[Name unavailable object]
  H --> I[One replacement inference pass]
  I --> B
  D --> J[Generated environment spec]
  G --> J

Loading

_{Reviews (1): Last reviewed commit: "fix test and uv test" | Re-trigger Greptile}

[arena-review-bot]

🤖 Isaac Lab-Arena Review Bot

Summary

This PR adds an opt-in SimReady asset search to agentic environment generation: objects the Arena catalog can't cover are left to SimReady, searched by phrase after spec inference, filtered to whole-word matches on the object noun, and accepted only if the USD holds exactly one rigid body. It's cleanly off-by-default (CLI flag + GUI expander), keeps SimReady names/URLs out of the asset classes to avoid a pre-sim pxr import, and is well tested. Boundaries and placement look sound; my only notes are about a redundant config flag and a loose return type.

Findings

🟡 simready_asset_search.py:44 — SimReadySearchConfig.enabled is set in three places (constructor default, simready_search_config_from_cli, and forced via replace(..., enabled=True)) but never read. The search never checks it and the agent gates on its own enable_simready_search, so two flags encode one decision and can silently diverge. Suggest dropping the field and keeping the agent flag as the single gate.

🔵 generation_panel.py:61 — _simready_config_from_session returns tuple[bool, object]; callers then reach into .source, .s3_url, etc. with no type-checker help. Return tuple[bool, SimReadySearchConfig] instead.

Test Coverage

Strong. New unit tests cover word-splitting/ranking, rigid-body rejection, the search fall-through, the retry/unavailable-object flow in the agent, the registered SimReadyUsdObject, the variant-aware rigid-body helpers, and the GUI severity path. The pure-USD tests build local tmp_path stages and defer pxr into the test bodies, consistent with keeping module import safe before sim init.

Verdict

Ship it (minor cleanups optional).

[qianl-nv]

Thanks for jumping on this!

have one large size comment about how the inference calls are structured. happy to discuss in more detail.

did a quick pass on the rest. will do a more detailed review after we resolve the large refactor comment.

[qianl-nv]

Thanks for restructuring it! Add some nits. In particular, suggests clean up the EnvironmentGenerationAgent.traces usage hygiene to separate error vs info logs into separate dict entry/list under trances, then can clean up some of the ac-hoc logic in UI for trace display.

Approved, feel free to merge after address the comments.