Is this page helpful?

Stage 2: Create a custom humanoid task in AGILE#

In this stage, connect the robot definition from Stage 1 to an AGILE environment and an RSL-RL training runner, the reinforcement learning library AGILE trains with. This wiring gives AGILE a stable task ID, but it does not yet make the MDP correct for your robot. Stage 3 examines and validates every environment component.

Install AGILE#

AGILE is a framework for developing humanoid locomotion and loco-manipulation behaviors with RL. It builds on Isaac Lab’s manager-based environments and adds humanoid tasks, training and evaluation workflows, pretrained policies, and deployment tools.

Follow Getting Started: Installation to get started with AGILE.

Play with AGILE#

Run one known policy before you modify AGILE to preview the expected result in the visualizer.

uv run scripts/eval.py --task Velocity-G1-History-v0 \
    --checkpoint agile/data/policy/velocity_g1/unitree_g1_velocity_history_state_dict.pt \
    --viz kit

See Pre-trained Policies: Usage for all available policies and supported checkpoint formats.

Understand an AGILE task#

An AGILE task is a chain of configuration choices. Each layer answers a different question:

robot configuration
        |
        |  Which USD, pose, joints, and actuators represent the robot?
        v
environment configuration
        |
        |  Which scene and MDP terms define the learning problem?
        v
Gymnasium task registration
        |
        |  Which stable task ID selects that environment?
        v
RSL-RL runner configuration
        |
        |  How does the learning algorithm collect and optimize experience?
        v
training run

Keeping these layers separate lets multiple tasks reuse one robot and lets one environment pair with different learning configurations. Use Task System: Directory Structure as the file-layout reference. For H2, the chain is implemented by:

agile/rl_env/
|-- assets/
|   `-- robots/
|       `-- unitree_h2.py
`-- tasks/
    `-- locomotion/
        `-- h2/
            |-- __init__.py
            |-- velocity_history_env_cfg.py
            `-- agents/
                `-- rsl_rl_ppo_cfg.py

Robot configuration#

Create a robot configuration under agile/rl_env/assets/robots/. Stage 1 produced the necessary source values; this step encodes them so every environment can spawn the same physical model.

Define:

  • The USD path and articulation spawn properties.

  • The default root pose, joint positions, and joint velocities.

  • Stable joint-name groups for the legs, waist, arms, head, and feet.

  • Foot links and links that indicate a fall on contact.

  • Actuator groups, position and velocity limits, gains, armature, friction, saturation effort, and delay.

The H2 file first gives important joints and links stable names:

LEG_JOINT_NAMES = [".*_hip_.*_joint", ".*_knee_joint", ".*_ankle_.*_joint"]
WAIST_JOINT_NAMES = ["waist_yaw_joint", "waist_roll_joint", "waist_pitch_joint"]
ARM_JOINT_NAMES = [".*_shoulder_.*_joint", ".*_elbow_joint", ".*_wrist_.*_joint"]
HEAD_JOINT_NAMES = ["head_pitch_joint", "head_yaw_joint"]
FEET_LINK_NAMES = ["left_ankle_pitch_link", "right_ankle_pitch_link"]
DEFAULT_PELVIS_HEIGHT = 0.915

These values connect the robot definition to the rest of the task. LEG_JOINT_NAMES contributes the 12 leg joints to the controlled set, which Stage 3 completes by adding waist_roll_joint and waist_pitch_joint for 14 controlled joints in total. FEET_LINK_NAMES identifies the bodies used by contact-based terms, and DEFAULT_PELVIS_HEIGHT supplies the standing-height reference for spawning, rewards, and terminations. Update each value from the Stage 1 USD, then verify and record the order in which the joint expressions resolve.

H2 then separates its motors by mechanical role. For readability, this excerpt names the actuator mapping h2_actuators; the AGILE source places the same mapping directly inside the articulation configuration. The leg group illustrates the parameters that connect an actuator model to the MDP:

h2_actuators = {
    "legs": DelayedDCMotorCfg(
        ...,
        joint_names_expr=[
            ".*_hip_yaw_joint",
            ".*_hip_roll_joint",
            ".*_hip_pitch_joint",
            ".*_knee_joint",
        ],
        saturation_effort=360.0,
        min_delay=MIN_DELAY_PHY_STEPS,
        max_delay=MAX_DELAY_PHY_STEPS,
    ),
    "feet": DelayedDCMotorCfg(...),
    "waist": DelayedDCMotorCfg(...),
    "arms": DelayedDCMotorCfg(...),
    "head": DelayedDCMotorCfg(...),
}

joint_names_expr assigns each joint to an actuator group. The omitted H2 fields provide per-joint effort and velocity limits, stiffness, damping, and armature. saturation_effort caps the motor model, while min_delay and max_delay randomize command latency from zero to four physics steps. H2 uses separate groups because its feet, waist, arms, and head have different limits, gains, and delay behavior. Carry the measured values from Stage 1 into the corresponding custom groups instead of applying one motor model globally.

Finally, the articulation configuration combines the asset, initial state, and actuator groups:

H2_DELAYED_DC_MOTOR = NestedRigidBodyArticulationCfg(
    spawn=sim_utils.UsdFileCfg(
        ...,
        usd_path=H2_USD_PATH,
        variants={"collision_profile": "feet_only", "hands": "none"},
        activate_contact_sensors=True,
    ),
    soft_joint_pos_limit_factor=0.9,
    init_state=ArticulationCfg.InitialStateCfg(
        pos=(0.0, 0.0, DEFAULT_PELVIS_HEIGHT + 0.1),
        joint_pos={
            ".*_hip_pitch_joint": -0.10,
            ".*_knee_joint": 0.30,
            ".*_ankle_pitch_joint": -0.20,
        },
        joint_vel={".*": 0.0},
    ),
    actuators=h2_actuators,
)

usd_path selects the asset. On the reference H2 path it is not a file you prepared: fetch_h2_usd() retrieves H2 from the AGILE Robot Menagerie, as What Stage 2 actually trains on explains. Point usd_path at your own USD when you substitute your own humanoid. The H2 variants selection requests feet-only collision and omits hands, which are variant sets the Menagerie asset defines; a USD you prepared yourself will have whatever variant sets you authored, or none. activate_contact_sensors enables the signals used later by contact terms. soft_joint_pos_limit_factor=0.9 keeps the training limit inside the hard USD limit, leaving a safety margin.

init_state is what determines the spawn and per-episode reset pose, and it overrides the pose authored in the USD. The values here are a shallower crouch than Unitree’s published home keyframe that Stage 1 authors, so reconcile the two deliberately rather than assuming your USD pose carries through. Whichever you choose, the initial root height and crouch must produce a collision-free pose before training begins.

The robot configuration describes the simulated robot that receives policy actions. If its initial pose, actuator model, or limits disagree with the USD or later deployment runtime, the policy learns against the wrong dynamics.

Environment configuration#

Complete the Gymnasium task registration and RSL-RL runner configuration below, then use Stage 3 to configure and validate the environment’s MDP components.

Gymnasium task registration#

Create the task directory and register its Gymnasium environment. The H2 task uses this registration:

import gymnasium as gym

from . import agents

gym.register(
    id="Velocity-H2-History-v0",
    entry_point="isaaclab.envs:ManagerBasedRLEnv",
    disable_env_checker=True,
    kwargs={
        "env_cfg_entry_point": f"{__name__}.velocity_history_env_cfg:H2LowerVelocityHistoryEnvCfg",
        "rsl_rl_cfg_entry_point": f"{agents.__name__}.rsl_rl_ppo_cfg:H2VelocityHistoryPpoRunnerCfg",
    },
)

The important fields have distinct responsibilities:

  • id is the public task name passed to --task. It must be unique and stable because training logs, evaluation commands, and exported bundles use it.

  • env_cfg_entry_point selects the environment configuration class. For H2, it selects H2LowerVelocityHistoryEnvCfg, which composes the scene and all MDP managers.

  • rsl_rl_cfg_entry_point selects the RSL-RL runner configuration. It controls the policy network, proximal policy optimization (PPO) algorithm, rollout length, save interval, symmetry augmentation, and experiment naming.

  • Imports in parent __init__.py files are intentional side effects. Importing the task package executes its gym.register call, making the task ID visible before a script asks Gymnasium to create it.

Follow Adding a New Task for the exact directory creation and parent-import procedure.

RSL-RL runner configuration#

The H2 runner begins with the rollout, training, checkpoint, and output settings:

@configclass
class H2VelocityPpoRunnerCfg(RslRlOnPolicyRunnerCfg):
    seed = 42
    num_steps_per_env = 24
    max_iterations = 50_000
    save_interval = 250
    experiment_name = "velocity_h2_lower"
    run_name = "velocity_h2_lower"
    empirical_normalization = False
    ...

num_steps_per_env=24 is the rollout length collected from every parallel environment before each PPO update. max_iterations bounds the run and save_interval controls raw checkpoint cadence. experiment_name chooses the log directory; run_name identifies this configuration within it. Change both for the custom humanoid so its outputs cannot be mistaken for H2 results. Keep the seed and normalization choice fixed for the first comparison.

The policy configuration sizes the actor that will be deployed and the larger critic used only during training:

policy = RslRlPpoActorCriticCfg(
    init_noise_std=1.0,
    actor_hidden_dims=[256, 256, 128],
    critic_hidden_dims=[512, 256, 128],
    activation="elu",
)

H2 uses a 256, 256, 128 actor and a 512, 256, 128 critic because the critic processes privileged observations. init_noise_std controls initial exploration, and activation selects the nonlinearity. Keep this network for the first baseline; change it only after the environment learns and a measured capacity or optimization problem motivates the change.

The PPO block defines the update behavior and the H2-specific symmetry augmentation:

algorithm = RslRlPpoAlgorithmCfg(
    ...,
    clip_param=0.2,
    learning_rate=1.0e-3,
    gamma=0.99,
    lam=0.95,
    desired_kl=0.01,
    symmetry_cfg=RslRlSymmetryCfg(
        use_data_augmentation=True,
        use_mirror_loss=False,
        data_augmentation_func=lr_mirror_H2,
    ),
)

clip_param limits each policy update, learning_rate controls its step size, and gamma and lam set the return and advantage horizons. desired_kl is the target used by the adaptive schedule in the full H2 configuration. The symmetry function is the parameter most likely to require immediate replacement: lr_mirror_H2 assumes H2’s observation and action order, so it is invalid for a custom ordering until a custom mirror mapping is verified.

The history task inherits those settings and changes its output identity:

@configclass
class H2VelocityHistoryPpoRunnerCfg(H2VelocityPpoRunnerCfg):
    experiment_name = "velocity_h2_history"
    run_name = "velocity_h2_history"
    wandb_project = "Velocity-H2-History"

Use the complete H2 class in agile/rl_env/tasks/locomotion/h2/agents/rsl_rl_ppo_cfg.py when creating the custom runner. The excerpts above show only the parameters you are likely to change; the rest should stay at the H2 baseline. The Algorithms: PPO section explains the algorithm AGILE integrates, while Stage 4 covers training-time diagnosis.

Troubleshooting by layer#

Symptom

Likely layer

Check

AGILE does not recognize the task ID.

Package registration.

Check the exact id and confirm that every parent __init__.py imports the child task package.

The task ID exists but loads the wrong class.

Gymnasium registration.

Check both entry-point module paths and class names for stale H2 references.

The robot fails while the environment initializes.

Robot configuration.

Check the USD path, articulation prim, joint expressions, actuator coverage, and default pose.

Training writes into the H2 log directory.

Runner configuration.

Replace experiment_name and run_name in the custom RSL-RL runner class.

Left-right augmentation maps unrelated joints.

Runner symmetry.

Replace or disable lr_mirror_H2 until a test proves the custom mapping preserves observations and actions.

Completion checklist#

  • AGILE is installed from its locked environment and a pretrained policy runs.

  • The custom robot configuration uses the USD, names, pose, limits, and actuator values recorded in Stage 1.

  • The custom task directory follows AGILE’s documented structure.

  • The Gymnasium task ID is unique and imports make it discoverable.

  • env_cfg_entry_point selects the custom environment class.

  • rsl_rl_cfg_entry_point selects a runner with custom experiment naming and a safe symmetry choice.

Continue with Stage 3: Configure and validate the locomotion MDP.