Is this page helpful?

Stage 5: Deploy the policy#

Deployment must recreate the trained policy’s interface and robot dynamics: observation definitions and order, history, action order and scale, actuator behavior, initial state, and control timing.

Follow one of the two paths below, not both. Option 1: Python deployment runs simulation and policy inference in Isaac Sim and is the shorter path for validating that a trained policy reproduces its training behavior. Option 2: Isaac ROS deployment exports an AGILE LEAPP bundle and connects its inference runtime to Isaac Sim through ros2_control and ROS 2 topics, which is the path toward running the same policy on hardware.

Option 1: Python deployment#

Export the IO descriptor#

Export the IO descriptor after training, when the timestamped run directory is known. Run this command before changing the task configuration, and replace TIMESTAMP_h2_full_50000 with the directory name created by the Stage 4 training command:

uv run scripts/train.py \
    --task Velocity-H2-History-v0 \
    --num_envs 1 \
    --max_iterations 0 \
    --run_name export_io_descriptors \
    --headless \
    --viz none \
    env.export_io_descriptors=true \
    env.log_dir=logs/rsl_rl/velocity_h2_history/TIMESTAMP_h2_full_50000

The zero-iteration run initializes the environment and exports its interface without performing a training iteration. It writes IO_descriptors.yaml to logs/rsl_rl/velocity_h2_history/TIMESTAMP_h2_full_50000/io_descriptors/IO_descriptors.yaml. AGILE also creates an unused timestamped run for this invocation. For more information about the exported interface, see Isaac Lab IO descriptors.

Locate the deployment artifacts#

The Python deployment consumes the exported TorchScript policy, env.yaml, and IO_descriptors.yaml. The environment configuration supplies the robot’s initial state, actuator parameters, simulation timing, and other deployment properties. The IO descriptor supplies ordered policy observations and actions, including observation-history requirements.

Keep the artifacts in their native locations under the same training run:

logs/rsl_rl/velocity_h2_history/TIMESTAMP_h2_full_50000/
|-- exported/
|   `-- policy.pt
|-- io_descriptors/
|   `-- IO_descriptors.yaml
`-- params/
    |-- agent.yaml
    `-- env.yaml

A custom integration can reference each path separately. Keep agent.yaml with the training record; the Python policy runner does not consume it at deployment.

Run the Isaac Sim deployment#

Read Deploying policies in Isaac Sim for the complete artifact and controller workflow. Then run the H2 standalone example from the Isaac Sim root directory:

./python.sh standalone_examples/api/isaacsim.robot.policy.examples/h2_standalone.py

The example uses tested H2 USD and policy artifacts from Nucleus. Because H2 is not a bundled policy, the script creates its policy specification directly. Its command loop walks forward, backward, left, and right, and then turns in both directions.

PolicyArtifact groups the model, environment configuration, and IO descriptor. PolicySpec adds the H2 USD, PhysX engine, and custom binding. RobotPolicyRunner configures the articulation, builds observations, runs the policy at the exported rate, and applies joint targets. The core construction is:

artifact = PolicyArtifact.from_files(model_path, env_config_path, descriptor_path)
spec = PolicySpec(
    name="unitree_h2_velocity_history",
    engines={"physx": artifact},
    usd_path=robot_usd_path,
    binding=partial(derive_h2_policy_binding, artifact),
)
robot = RobotPolicyRunner(spec, prim_path="/World/H2", training_engine="physx")
robot.spawn()

robot.restart_from_default_state(command)
robot.step(step_size, command)

The runner normally derives the binding directly from IO_descriptors.yaml. H2’s descriptor also contains a zero-width RandomPositionAction that moved arms and head during training but consumed no policy output. derive_h2_policy_binding removes that term, so deployment commands only the 14 trained leg and waist joints. The policy controller maintains the five-frame observation history.

For another humanoid, replace the robot asset and policy artifacts. Add a custom binding only when the generic descriptor mapping cannot reproduce an exported term.

Option 2: Isaac ROS deployment#

Export the LEAPP bundle#

Export the trained checkpoint as an AGILE LEAPP bundle. Run the following command from the AGILE repository, and replace RUN_DIRECTORY_NAME with the Stage 4 training run:

uv run scripts/export_policy_leapp.py \
    --task Velocity-H2-History-v0 \
    --load_run RUN_DIRECTORY_NAME \
    --disable_graph_visualization

The export creates a task-named bundle under the training run:

<RUN_DIRECTORY>/leapp-bundles/Velocity-H2-History-v0/
|-- Velocity-H2-History-v0.yaml
|-- Velocity-H2-History-v0.onnx
`-- Velocity-H2-History-v0_initial_values.safetensors

Keep the bundle together. The YAML describes the policy graph and references the model and initial-state files beside it. Follow Use an exported AGILE bundle to inspect and load it.

Prepare the Isaac ROS integration#

Follow Isaac ROS getting started to create and activate an Isaac ROS workspace. For Isaac Sim deployment, use the source-build path because the Isaac ROS Deploy Isaac Sim extension is distributed as source.

Follow the package layout in Bring your own embodiment:

<robot>_description/    # URDF, xacro, and simulator assets
<robot>_ros2_control/   # Real-hardware plugin; not required for this simulation stage
<robot>_bringup/        # Controller configuration, policy groups, and launch files

For this simulation workflow, concentrate on <robot>_description and <robot>_bringup. Declare the command interfaces and joint and IMU state interfaces expected by the policy. Keep names consistent between the robot description, LEAPP metadata, controller configuration, and Isaac Sim topics.

Reuse TopicBasedSystemInterface to connect ros2_control to Isaac Sim. You do not need to write a custom SystemInterface for this path.

Prepare the simulation USD#

The Isaac ROS Deploy bake scripts prepare the USD in two stages:

robot.usd
    -> add_newton_actuators_to_robot
    -> add_ros2_bridge_to_robot
    -> robot_deploy.usd

Before baking, give every commanded joint a finite physics:maxForce value. The first stage adds the explicit actuator model and removes competing native drive gains. The second adds default joint state, IMU, clock, and joint-command ROS 2 graphs. Add bridge nodes when the policy needs signals outside this default set. Use the exact commands in the Isaac Sim section of Bring your own embodiment.

Configure and validate the controllers#

In <robot>_bringup, adapt controller_manager.yaml to declare controller plugins, joints, and gains. Adapt controller_groups.yaml to select the LEAPP bundle, map policy sources to interfaces, and define controller activation order. Use the G1 files linked from Bring your own embodiment as a structural reference, but remove G1-specific joints and thresholds.

Validate the integration in this order:

  1. Expand the robot description and confirm that joint and sensor interfaces have the expected names.

  2. Bake the deploy USD, launch it through the Isaac ROS Deploy extension, and press Play.

  3. Confirm that Isaac Sim joint and IMU state topics update before loading the policy.

  4. Launch the controller manager with the Isaac Sim hardware type and confirm that it claims the expected interfaces.

  5. Load the policy controller group, send a conservative velocity command, and compare simulated motion with the AGILE evaluation from Stage 4.

Troubleshooting by layer#

Symptom

Likely layer

Check

IO_descriptors.yaml is missing.

Python export.

Confirm that the task is manager based and both env.export_io_descriptors and env.log_dir were set.

Deployment reports an observation or action width mismatch.

Artifact interface.

Check that policy, environment YAML, and IO descriptor come from the same task and training run.

The policy moves the wrong joints.

Binding or joint order.

Compare the descriptor order, resolved articulation order, action selection, and custom binding.

The policy runs but differs from AGILE evaluation.

Plant or timing fidelity.

Compare physics step, control interval, actuator model, delay, gains, action scale, default pose, and history.

Isaac ROS Deploy receives no state.

Simulation bridge.

Check baked ROS 2 graphs, topic names, clock, joint state, and IMU publishers.

ros2_control cannot claim an interface.

Controller configuration.

Compare TopicBasedSystemInterface, controller joint names, command interfaces, and policy metadata.

LEAPP cannot load the bundle.

Bundle integrity.

Keep the YAML, ONNX, and initial-value files together and verify paths referenced by the YAML.

Completion checklist#

  • All deployment artifacts come from the same Stage 4 run.

  • The runtime reproduces actor observation definitions, ordering, scaling, noise policy, and five-frame history.

  • Action width, order, offsets, scaling, and controlled joints match training.

  • The deployed USD, initial pose, actuator behavior, physics step, control interval, and delay match training.

  • The Python runner or ROS 2 bridge loads without interface errors.

  • Conservative forward, lateral, yaw, and zero commands match the AGILE evaluation behavior.

  • Reset and ordinary disturbance behavior remain consistent with evaluation.

Return to the Humanoid workflow overview.