Refactor articulation actuator ownership - #6839
Draft
AntoineRichard wants to merge 33 commits into
Draft
Conversation
Add ActuatorCollection as the backend-neutral owner for actuator state and command APIs. Route legacy articulation target and gain setters through the collection with deprecation warnings. Move common articulation actuator-control forwarding into a shared helper and keep backend adapters focused on command submission, friction writes, and native actuator paths for PhysX, OVPhysX, and Newton. Add changelog fragments and focused ActuatorCollection tests.
Describe how Newton actuator authoring will reuse the shared asset cache to resolve remote neural-network checkpoints before PyTorch loads them.
Define the scoped implementation and real-task verification steps for cached remote actuator-network checkpoints.
Treat generated actuator plots as opaque image assets so Git and GitHub do not report their XML serialization as reviewable line churn. Mark them as generated so GitHub collapses the files by default.
Resolve actuator-network paths through the shared asset cache before PyTorch adds Newton metadata. This lets remote MLP and LSTM checkpoints load through the Newton actuator adapter.
Keep temporary workflow documents out of the Sphinx source tree so strict documentation builds do not report orphan warnings.
Separate actuator-model inputs from processed joint commands so the public API uses precise terminology across physics backends. Update the migration guide, tutorials, pipeline diagrams, and actuator parameter tool to match.
github-actions
Bot
added
documentation
Aggregate disjoint stateless actuator groups without coupling their logical configuration or parameter access. Keep batch staging pointer-stable, fuse implicit execution in Warp, and reuse cached gather and scatter launches. Capture graphable Newton actuators on the PhysX path with alternating state graphs, eager fallback, and protection against unsafe nested stateful capture.
AntoineRichard
commented
Aug 3, 2026
Comment on lines
+253
to
+277
| class JointCommand: | ||
| """Processed commands produced for the simulated joints.""" | ||
|
|
||
| def __init__(self, collection: ActuatorCollection) -> None: | ||
| """Initialize the joint command view. | ||
|
|
||
| Args: | ||
| collection: Owning actuator collection. | ||
| """ | ||
| self._collection = collection | ||
|
|
||
| @property | ||
| def position(self) -> ProxyArray: | ||
| """Processed position commands [m or rad, depending on joint type].""" | ||
| return self._collection._joint_pos_target_sim_ta | ||
|
|
||
| @property | ||
| def velocity(self) -> ProxyArray: | ||
| """Processed velocity commands [m/s or rad/s, depending on joint type].""" | ||
| return self._collection._joint_vel_target_sim_ta | ||
|
|
||
| @property | ||
| def effort(self) -> ProxyArray: | ||
| """Processed effort commands [N or N·m, depending on joint type].""" | ||
| return self._collection._joint_effort_target_sim_ta |
Collaborator
Author
There was a problem hiding this comment.
Why next the class definition here?
Comment on lines
+64
to
+90
| class Command: | ||
| """Commands received by the actuator models. | ||
|
|
||
| Position and velocity commands use joint-side coordinates. All command | ||
| arrays are indexed by articulation joint, not by motor shaft. | ||
| """ | ||
|
|
||
| def __init__(self, collection: ActuatorCollection) -> None: | ||
| """Initialize the command view. | ||
|
|
||
| Args: | ||
| collection: Owning actuator collection. | ||
| """ | ||
| self._collection = collection | ||
|
|
||
| @property | ||
| def position(self) -> ProxyArray: | ||
| """Desired positions [m or rad, depending on joint type].""" | ||
| return self._collection._joint_pos_target_ta | ||
|
|
||
| @property | ||
| def velocity(self) -> ProxyArray: | ||
| """Desired velocities [m/s or rad/s, depending on joint type].""" | ||
| return self._collection._joint_vel_target_ta | ||
|
|
||
| @property | ||
| def effort(self) -> ProxyArray: |
Collaborator
Author
There was a problem hiding this comment.
Why nest class definition this class?
Comment on lines
+492
to
+493
| self._soft_joint_vel_limits = wp.zeros(shape, dtype=wp.float32, device=self.device) | ||
| self._gear_ratio = wp.ones(shape, dtype=wp.float32, device=self.device) |
Collaborator
Author
There was a problem hiding this comment.
Why do we define this here?
Collaborator
Author
There was a problem hiding this comment.
Is it just for compatiblity?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR gives actuator behavior one backend-neutral runtime owner,
ActuatorCollection, and makes that ownership boundary useful for performance. It is based ondevelopafter the articulation-ordering work in #6784 merged.Previously, each backend articulation simultaneously owned simulated state, actuator groups, raw targets, processed targets, telemetry, gain resolution, actuator execution, ordering conversion, and backend submission. The refactor now separates those responsibilities:
ActuatorCollectionowns named actuator groups, input commands, processed joint commands, telemetry, resolved parameters, and model lifecycle.ActuatorControldefines the narrow backend contract.PhysxActuatorControl,NewtonActuatorControl, andOvPhysxActuatorControlown backend ordering, staging, native-actuator integration, property writes, and submission.Articulationremains responsible for simulated multibody state, topology, and lifecycle orchestration.Public API and terminology
The API distinguishes the two sides of an actuator model:
All public arrays remain expressed on the simulated joint side and indexed in articulation public joint order. “Actuator command” names the receiving component; it does not imply motor-shaft indexing.
Existing articulation target/gain methods and
ArticulationDatacommand/torque properties remain available as deprecated forwarding aliases. This PR does not remove a previously released public API.Logical groups and execution aggregation
Named actuator groups remain the configuration and access surface. Execution batches are an internal optimization and may combine disjoint stateless groups of the same exact actuator class even when their gains, limits, or gear ratios differ.
ImplicitActuatorbatches compute and publish processed commands and telemetry in one fused Warp launch.IdealPDActuatorandDCMotorbatches use preallocated pointer-stable staging, in-place Torch compute, and cached Warp gather/scatter launches.This removes redundant compute/launch work without changing configuration syntax, group lookup, per-group parameter writes, or output accessors.
Backend behavior
PhysxActuatorWrapper. Mixed implicit/explicit groups retain solver-drive commands where required.Newton actuators on PhysX CUDA graphs
For graphable Newton actuators, PhysX attempts to capture the complete native sequence—effort staging, all actuator models, and torque telemetry—and replays it automatically.
Two alternating graphs preserve state ping-pong for delayed/stateful controllers: graph A reads state A and writes B; graph B reads B and writes A. Capture failure falls back to eager execution. Stateful native actuators are rejected inside a caller-owned outer CUDA capture because Python-side state ownership cannot advance on replay; callers should let the PhysX adapter own their alternating graphs. Stateless native execution remains composable with an outer capture.
Neural actuator checkpoints
Newton MLP/LSTM checkpoints are resolved through Isaac Lab's shared
retrieve_file_path()cache before PyTorch loads and re-saves them with Newton metadata. This enables remote HTTP/Nucleus-style checkpoint paths without adding a dependency while retaining local TorchScript and dictionary checkpoint support.Documentation
actuators.command.tools/actuator_parameters.py.Performance
Franka Reach end-to-end runtime
Protocol: PhysX
Isaac-Reach-Franka, 4096 environments, seed 42, no visualizer, 100 warm-up steps, 1000 measured steps, three runs per variant. Franka has three logical implicit-actuator groups, so the final branch exercises automatic aggregation and the fused implicit Warp path.developbaseRelative to
develop, the final branch reduced mean step time by 8.35% and increased mean throughput by 9.34%. Median step time improved by 5.61%, with median throughput up 5.94%.The raw final run means were 14.0797, 14.3350, and 12.4917 ms, so variance is material. The final pointer-stability/graph pass did not resolve as an additional task-level gain over the already-aggregated PR snapshot. The performance claim is therefore the complete
develop-to-final result; automatic merging is the primary optimization, while graph replay is structurally cheaper but below this benchmark's noise floor.Go2 graph/native-path isolation
The same 4096-environment, 100-warm-up/1000-measured-step protocol was run on Go2, which contains one DC motor group and therefore does not exercise merging.
The mean favors Lab by 1.09%, while the median favors Newton by 3.68% in step time. This disagreement is run-to-run noise, not evidence of a graph speedup.
Training evidence
Long-form runs used PhysX physics, 4096 environments, RSL-RL, seed 42, and 500 learning iterations. “PhysX + Newton actuators” changes actuator execution, not the physics backend.
develop, regular PhysX actuatorsThe controlled Go2 training pair reached comparable reward and episode length. The other robots establish that diverse explicit actuator configurations complete meaningful training, not matched per-robot performance parity.
Validation
./isaaclab.sh -p -m pytest source/isaaclab/test/actuators -q— 448 passed./isaaclab.sh -d— Sphinx warning-as-error build succeeded./isaaclab.sh -f— all repository-wide hooks passed before commit and pushThe fused GPU implicit telemetry equation differs from the previous Torch expression by at most one float32 ULP (
4.768e-7) for the exercised values because of arithmetic ordering. Processed position, velocity, and feed-forward effort commands sent to PhysX remain exact.One existing reversed-public-joint-ordering test triggers a PhysX tensor-setter CUDA illegal-memory-access on this host. It reproduces at the untouched pre-optimization revision with every Isaac Lab package import pinned to that checkout. Because the fault poisons the CUDA process, the graph/state suite was run in fresh focused processes; the ordering failure is not caused by this performance pass.
Type of change
Checklist
./isaaclab.sh -fCONTRIBUTORS.md