Reinforcement Learning Policies Examples in Isaac Sim#
About#
The isaac_sim_policy_example Extension is a framework and has a set of helper functions to deploy Isaac Lab Reinforcement Learning Policies in Isaac Sim. For details for training and building the policy in Isaac Sim, visit deploying policy in Isaac Sim.
This Extension is enabled by default. If it is ever disabled, it can be re-enabled from the Extension Manager by searching for isaacsim.robot.policy.example.
To run examples below activate Windows > Examples > Robotics Examples which will open the Robotics Examples tab.
Physics Engines and Sim-to-Sim Transfer#
Isaac Sim supports PhysX and Newton physics backends. These examples select the policy, environment configuration, and actuator implementation that match the active engine. Select the physics engine before loading an example. To change engines, stop and reload the example so its robot and controller are reconstructed for the new backend.
Note
Newton support is experimental and less mature than the default PhysX backend. In these policy examples, Newton is currently supported for Unitree H1, Boston Dynamics Spot, and ANYmal C. The Franka policy example is not currently supported with Newton.
Running a policy trained with one physics engine on another is known as sim-to-sim policy transfer. Cross-engine evaluation can expose differences in dynamics, contacts, actuator behavior, and joint or link ordering. Observation and action remapping or engine-specific asset and configuration tuning may be required; successful execution with one backend does not guarantee identical behavior with another.
For backend setup and compatibility details, see Newton Physics Backend. For the cross-engine policy workflow, see Sim-to-Sim Policy Transfer in Isaac Lab.
Unitree H1 Humanoid Example#
The Unitree H1 humanoid example can be accessed by creating a empty stage.
Open the example menu using Robotics Examples > POLICY > Humanoid.
(Optional) Use the Physics Engine menu in the viewport to switch between PhysX and Newton before loading. The example automatically selects the matching policy for the active engine.
Press LOAD to open the scene.
This example uses an H1 Flat Terrain Policy trained in Isaac Lab to control the humanoid’s locomotion. Both PhysX and Newton policies are provided so you can compare locomotion behavior across physics engines.
Controls:
Forward: UP ARROW / NUM 8
Turn Left: LEFT ARROW / NUM 4
Turn Right: RIGHT ARROW / NUM 6
Boston Dynamics Spot Quadruped Example#
The Boston Dynamics Spot quadruped example can be accessed by creating a empty stage.
Open the example menu using Robotics Examples > POLICY > Quadruped.
(Optional) Use the Physics Engine menu in the viewport to switch between PhysX and Newton before loading. The example automatically selects the matching policy for the active engine.
Press LOAD to open the scene.
This example uses a Spot Flat Terrain Policy trained in Isaac Lab to control the quadruped’s locomotion. Both PhysX and Newton policies are provided so you can compare locomotion behavior across physics engines.
Controls:
Forward: UP ARROW / NUM 8
Backward: BACK ARROW / NUM 2
Move Left: LEFT ARROW / NUM 4
Move Right: RIGHT ARROW / NUM 6
Turn Left: N / NUM 7
Turn Right: M / NUM 9
Unitree Go2 Quadruped Example#
The Unitree Go2 quadruped example can be accessed by creating a empty stage.
Open the example menu using Robotics Examples > POLICY > Go2.
(Optional) Use the Physics Engine menu in the viewport to switch between PhysX and Newton before loading. The example automatically selects the matching policy for the active engine.
Press LOAD to open the scene.
This example uses a Go2 Flat Terrain Policy trained in Isaac Lab to control the quadruped’s locomotion. Both PhysX and Newton policies are provided so you can compare locomotion behavior across physics engines.
Controls:
Forward: UP ARROW / NUM 8
Backward: BACK ARROW / NUM 2
Move Left: LEFT ARROW / NUM 4
Move Right: RIGHT ARROW / NUM 6
Turn Left: N / NUM 7
Turn Right: M / NUM 9
Franka Panda Open Drawer Example#
The Franka Panda Open Drawer example can be accessed by creating a empty stage.
Open the example menu using Robotics Examples > POLICY > Franka.
Press LOAD to open the scene.
This example uses the Franka Open Drawer Policy trained in Isaac Lab to control the robot’s arm. The robot will open the drawer, hold it open until the would reset.
Policies Files#
The policies used in the examples are trained in Isaac Lab and are available here:
Name |
Policy |
Parameters |
H1 Flat Terrain Policy (PhysX) |
||
H1 Flat Terrain Policy (Newton) |
||
Spot Flat Terrain Policy (PhysX) |
||
Spot Flat Terrain Policy (Newton) |
||
ANYmal C Flat Terrain Policy (PhysX) |
||
ANYmal C Flat Terrain Policy (Newton) |
ANYmal C Flat Terrain Policy (Newton) Environment Parameters |
|
Franka Panda Open Drawer Policy |
||
Go2 Flat Terrain Policy (PhysX) |
||
Go2 Flat Terrain Policy (Newton) |
Note
The policies can also be downloaded directly from the Content Browser by right clicking the policy and selecting Download.
Warning
The example policies uses separate robots for physx and newton, depending on the physics engine selected initially. Switching the physics engine will require the robot to be respawned.
API Documentation#
See the API documentation for complete usage information.
Standalone Examples#
Note
Each standalone example accepts --device cpu or --device cuda and --engine physx or
--engine newton. They use PhysX and CUDA by default.
The locomotion examples are not interactive: each drives a scripted, closed loop of base-frame velocity commands and renders the commanded course beside the path the robot actually travels. For keyboard-driven locomotion, use the interactive examples in the Robotics Examples browser.
h1_standalone.py
This standalone example demonstrates a Unitree H1 controlled by a flat terrain policy, following a set of predetermined command sequences. It may be run via the following command:
./python.sh standalone_examples/api/isaacsim.robot.policy.examples/h1_standalone.py --num-robots <number of robot> --env-url </path/to/environment>
For example, this will spawn 5 robots on the flat grid scene below:
./python.sh standalone_examples/api/isaacsim.robot.policy.examples/h1_standalone.py --num-robots 5 --env-url /Isaac/Environments/Grid/default_environment.usd
spot_standalone.py
This standalone example demonstrates a Boston Dynamics Spot controlled by a flat terrain policy, following a set of predetermined command sequences. It may be run via the following command:
./python.sh standalone_examples/api/isaacsim.robot.policy.examples/spot_standalone.py
anymal_standalone.py
This standalone example demonstrates an ANYmal C robot that is controlled by a neural network policy. The rough terrain policy was trained in Isaac Lab and takes as input the state of the robot, the commanded base velocity, and the surrounding terrain and outputs joint position targets. The example may be run via the following command:
./python.sh standalone_examples/api/isaacsim.robot.policy.examples/anymal_standalone.py
cartpole_standalone.py
This standalone example demonstrates a cartpole balanced by a policy trained in Isaac Lab. The policy observes no command channel, so the example steps it without one and the cart keeps the pole upright indefinitely. It may be run via the following command:
./python.sh standalone_examples/api/isaacsim.robot.policy.examples/cartpole_standalone.py