[isaacsim.robot_motion.controllers] controllers#

Version: 0.3.2

Overview#

The isaacsim.robot_motion.controllers extension provides portable, GPU-accelerated robot motion controllers that implement the BaseController interface from isaacsim.robot_motion.experimental.motion_generation.

Each controller reads its setpoint from a named control-point site in setpoint_state.sites and writes joint velocity and position targets into the returned RobotState. On a CUDA device the projection and kinematics are captured into a CUDA graph at construction time, keeping per-step overhead to a pair of device-to-device copies plus a graph launch.

Controllers#

DifferentialDriveController converts a control-point velocity setpoint into left and right wheel angular velocity targets using the differential drive kinematic model:

omega_L = (2 * v - omega * wheel_base) / (2 * wheel_radius)
omega_R = (2 * v + omega * wheel_base) / (2 * wheel_radius)

Linear speed v and yaw rate omega are extracted from the site by projecting the 3D linear and angular velocity vectors onto configurable forward_direction and rotation_direction unit vectors.

AckermannController converts a control-point linear-velocity setpoint into steerable-wheel angular velocities, steering-angle position targets, and (optionally) non-steerable-wheel angular velocities, using the Ackermann kinematic model. The setpoint encodes speed v and body turning angle θ as the velocity vector [v·cos θ, v·sin θ, 0]. Alternatively, direct_command=True allows passing linear_speed and turning_angle as keyword arguments directly to forward(), bypassing the velocity-vector projection step.

HolonomicController converts a control-point body twist setpoint [vx, vy, wz] into per-wheel angular velocity targets for mecanum-wheeled robots using closed-form inverse kinematics:

phi_dot_i = K_i * (M[i, :] @ [vx, vy, wz])

The kinematic matrix M (N × 3) and conversion diagonal K (N,) are precomputed at construction from the authored wheel positions, orientations, radii, and mecanum roller angles, after transforming that geometry into the command-site frame given by command_site_position and command_site_quaternion — so the wheels may be measured in whatever frame is convenient and the site moved independently of them. Wheel geometry is conveniently obtained from USD assets via HolonomicRobotUsdSetup.get_holonomic_controller_params(). Planar linear speed and yaw rate are clamped independently before the matrix multiply; per-wheel speed is clamped after.

Example#

import math

import isaacsim.robot_motion.controllers as ctrl
import isaacsim.robot_motion.experimental.motion_generation as mg
import warp as wp

# --- Differential drive ---

dd_controller = ctrl.DifferentialDriveController(
    robot_joint_space=robot.dof_names,
    left_wheel_joint="left_wheel_joint",
    right_wheel_joint="right_wheel_joint",
    wheel_radius=0.03,
    wheel_base=0.1125,
)

dd_setpoint = mg.RobotState(
    sites=mg.SpatialState.from_name(
        spatial_space=["control_point"],
        linear_velocities=(["control_point"], wp.array([[0.2, 0.0, 0.0]], dtype=wp.float32)),
        angular_velocities=(["control_point"], wp.array([[0.0, 0.0, 1.0]], dtype=wp.float32)),
    )
)

desired_state = dd_controller.forward(estimated_state, dd_setpoint, t)

# --- Ackermann ---

ack_controller = ctrl.AckermannController(
    robot_joint_space=robot.dof_names,
    left_steerable_wheel_joint="front_left_wheel",
    right_steerable_wheel_joint="front_right_wheel",
    left_steering_joint="front_left_steering",
    right_steering_joint="front_right_steering",
    steerable_wheel_radius=0.3,
    wheel_base=1.5,
    track_width=1.2,
)

v, theta = 1.0, 0.3  # speed [m/s], turning angle [rad]
ack_setpoint = mg.RobotState(
    sites=mg.SpatialState.from_name(
        spatial_space=["control_point"],
        linear_velocities=(
            ["control_point"],
            wp.array([[v * math.cos(theta), v * math.sin(theta), 0.0]], dtype=wp.float32),
        ),
    )
)

desired_state = ack_controller.forward(estimated_state, ack_setpoint, t)

# --- Holonomic (omni / mecanum) ---

holo_controller = ctrl.HolonomicController(
    robot_joint_space=robot.dof_names,
    wheel_joint_names=["axle_0_joint", "axle_1_joint", "axle_2_joint"],
    wheel_radius=wheel_radius,        # from HolonomicRobotUsdSetup
    wheel_positions=wheel_positions,  # from HolonomicRobotUsdSetup
    wheel_orientations=wheel_orientations,
    mecanum_angles=[90.0, 90.0, 90.0],  # 90° = omni; 45°/135° = standard mecanum
)

holo_setpoint = mg.RobotState(
    sites=mg.SpatialState.from_name(
        spatial_space=["control_point"],
        linear_velocities=(["control_point"], wp.array([[0.4, 0.0, 0.0]], dtype=wp.float32)),
        angular_velocities=(["control_point"], wp.array([[0.0, 0.0, 0.0]], dtype=wp.float32)),
    )
)

desired_state = holo_controller.forward(estimated_state, holo_setpoint, t)

Integration#

This extension depends on isaacsim.robot_motion.experimental.motion_generation for the BaseController interface and state types.

Enable Extension#

The extension can be enabled (if not already) in one of the following ways:

Command-line interface

Define the next entry as an application argument from a terminal.

APP_SCRIPT.(sh|bat) --enable isaacsim.robot_motion.controllers

Experience/extension configuration

Define the next entry under [dependencies] in an experience (.kit) file or an extension configuration (extension.toml) file.

[dependencies]
"isaacsim.robot_motion.controllers" = {}

Extension Manager UI

Open the Window > Extensions menu in a running application instance and search for isaacsim.robot_motion.controllers. Then, toggle the enable control button if it is not already active.

Python API#

Controllers

`AckermannController`	Single-robot Ackermann steering controller.
`DifferentialDriveController`	Single-robot differential drive controller.
`HolonomicController`	Holonomic (omni / mecanum) drive controller.

Controllers#

class AckermannController( *, robot_joint_space: list[str], left_steerable_wheel_joint: str, right_steerable_wheel_joint: str, left_steering_joint: str, right_steering_joint: str, steerable_wheel_radius: float, wheel_base: float, track_width: float, left_non_steerable_wheel_joint: str | None = None, right_non_steerable_wheel_joint: str | None = None, non_steerable_wheel_radius: float | None = None, non_steerable_track_width: float | None = None, max_linear_speed: float | None = None, max_turning_angle: float | None = None, steerable_wheels_at_rear: bool = False, direct_command: bool = False, linear_speed_kwarg: str = 'linear_speed', turning_angle_kwarg: str = 'turning_angle', control_point_name: str = 'control_point', forward_direction: list[float] | ndarray | array = (1.0, 0.0, 0.0), rotation_direction: list[float] | ndarray | array = (0.0, 0.0, 1.0), device: Device | str | None = None, )#

Bases: BaseController

Single-robot Ackermann steering controller.

Converts a control-point linear-velocity setpoint into:

Steerable-wheel angular velocities and steering-angle position targets.
Non-steerable-wheel angular velocities (if any).

The setpoint must be a real velocity vector at the control point. For a vehicle travelling at total speed v with body turning angle θ, the site’s linear_velocity should be [v·cos θ, v·sin θ, 0] in the body frame.

The controller recovers speed and angle by projection onto the body axes:

v_x = dot(linear_velocity, forward_direction)
v_y = dot(linear_velocity, lateral_direction)
linear_speed = sign(v_x) · ‖[v_x, v_y]‖
turning_angle = atan2(v_y, v_x) (forward, v_x > 0)
turning_angle = atan2(-v_y, -v_x) (reversing, v_x ≤ 0)

where lateral_direction = normalize(cross(rotation_direction, forward_direction)).

Negating both components for reversing keeps turning_angle in (-π/2, π/2). Positive turning angle steers to the left (counterclockwise from above with the default rotation_direction = [0, 0, 1]).

Both direction vectors are normalized at construction time and must be at least 89.9 degrees apart.

Parameters:

robot_joint_space – Ordered list of all joint names in the robot.
left_steerable_wheel_joint – Name of the left steerable wheel joint in robot_joint_space (velocity target).
right_steerable_wheel_joint – Name of the right steerable wheel joint in robot_joint_space (velocity target).
left_steering_joint – Name of the left steering joint in robot_joint_space (position target — the physical steering angle of the wheel).
right_steering_joint – Name of the right steering joint in robot_joint_space (position target).
steerable_wheel_radius – Radius of the steerable wheels [m].
wheel_base – Axle-to-axle distance, front to rear [m].
track_width – Lateral wheel-to-wheel distance [m].
left_non_steerable_wheel_joint – Name of the left non-steerable wheel joint in robot_joint_space (velocity target). Must be provided together with right_non_steerable_wheel_joint. Its lateral offset is derived automatically as +non_steerable_track_width / 2.
right_non_steerable_wheel_joint – Name of the right non-steerable wheel joint in robot_joint_space (velocity target). Must be provided together with left_non_steerable_wheel_joint. Its lateral offset is derived automatically as -non_steerable_track_width / 2.
non_steerable_wheel_radius – Radius of the non-steerable wheels [m]. When None, falls back to steerable_wheel_radius.
non_steerable_track_width – Lateral distance between the non-steerable wheels [m]. When None, falls back to track_width.
max_linear_speed – Forward/backward speed limit [m/s]. When None, no limit is applied.
max_turning_angle – Steering angle clamp [rad]. Must be in (0, π/2].
steerable_wheels_at_rear – Pass True for rear-wheel-steering vehicles (e.g. forklifts).
direct_command – Pass True to ignore setpoint_state in forward() and read speed and angle directly from kwargs instead.
linear_speed_kwarg – Name of the forward() kwarg that carries the signed total speed [m/s] in direct mode.
turning_angle_kwarg – Name of the forward() kwarg that carries the body turning angle [rad] in direct mode.
control_point_name – Name of the site in setpoint_state.sites from which linear_velocity is read. Unused when direct_command=True.
forward_direction – Robot forward axis in the body frame. Normalized at construction.
rotation_direction – Yaw axis in the body frame. Normalized at construction. Must be at least 89.9° from forward_direction. Used to derive lateral_direction as normalize(cross(rotation_direction, forward_direction)).
device – Warp device for internal buffers. Defaults to wp.get_device().

Raises:

ValueError – If either direction vector is zero, not 3-element, or the two directions are less than 89.9 degrees apart.
ValueError – If steerable_wheel_radius, wheel_base, or track_width is not strictly positive.
ValueError – If non_steerable_wheel_radius is provided and not strictly positive.
ValueError – If non_steerable_track_width is provided and not strictly positive.
ValueError – If max_linear_speed is provided and not strictly positive.
ValueError – If max_turning_angle is provided and not in (0, π/2].
ValueError – If exactly one of left_non_steerable_wheel_joint / right_non_steerable_wheel_joint is provided (both or neither required).
ValueError – If any two of the required joint names are identical.
ValueError – If any required joint name is not in robot_joint_space.

forward(

estimated_state: RobotState,

setpoint_state: RobotState | None,

t: float,

**kwargs: object,

) → RobotState | None#

Compute wheel velocity and steering targets.

Dispatches to _forward_direct() or _forward_site() depending on the direct_command flag set at construction.

Parameters:

estimated_state – Current estimated state of the robot. Not used by this controller but required by the BaseController interface.
setpoint_state – Desired robot state containing the control-point site. Ignored when direct_command=True.
t – Current clock time [s].
**kwargs – In direct mode, must supply the kwargs named by linear_speed_kwarg (signed total speed [m/s]) and turning_angle_kwarg (body turning angle [rad]).

Returns:

RobotState whose JointState holds velocity targets for all wheel joints (steerable and non-steerable) in the order [left_steer, right_steer, left_ns, right_ns], and position targets for the two steering joints [left_steering, right_steering]. None if required inputs are absent.

reset(

estimated_state: RobotState,

setpoint_state: RobotState | None,

t: float,

**kwargs: object,

) → bool#

Reset the controller.

In non-direct mode the stored prev_theta is zeroed so the next command starts with a clean steering history.

Parameters:

estimated_state – Current estimated state of the robot.
setpoint_state – Optional desired state of the robot.
t – Current clock time [s].
**kwargs – Unused; accepted for interface compatibility.

Returns:

Always True.

class DifferentialDriveController( *, robot_joint_space: list[str], left_wheel_joint: str, right_wheel_joint: str, wheel_radius: float, wheel_base: float, control_point_name: str = 'control_point', forward_direction: list[float] | ndarray | array = (1.0, 0.0, 0.0), rotation_direction: list[float] | ndarray | array = (0.0, 0.0, 1.0), max_linear_speed: float | None = None, max_angular_speed: float | None = None, max_wheel_speed: float | None = None, device=None, )#

Bases: BaseController

Single-robot differential drive controller.

Converts a control-point velocity setpoint into left / right wheel velocity commands using the differential drive kinematic model:

omega_L = (2 * v - omega * wheel_base) / (2 * wheel_radius)
omega_R = (2 * v + omega * wheel_base) / (2 * wheel_radius)

The scalar commands are extracted from the named site in setpoint_state.sites by projection:

v = dot(forward_direction, site.linear_velocity)
omega = dot(rotation_direction, site.angular_velocity)

The control point is interpreted as the point midway between the two front wheels, which may differ from the robot’s root (e.g. centre of mass).

Both direction vectors are normalized at construction time and must be at least 89.9 degrees apart.

Parameters:

robot_joint_space – Ordered list of all joint names in the robot.
left_wheel_joint – Name of the left wheel joint in robot_joint_space.
right_wheel_joint – Name of the right wheel joint in robot_joint_space.
wheel_radius – Wheel radius [m].
wheel_base – Lateral wheel-to-wheel distance [m].
control_point_name – Name of the site in setpoint_state.sites from which linear and angular velocities are read.
forward_direction – Robot forward axis in the body frame. Normalized at construction.
rotation_direction – Yaw axis in the body frame. Normalized at construction. Must be at least 89.9° from forward_direction.
max_linear_speed – Forward/backward speed limit [m/s]. When None, no limit is applied.
max_angular_speed – Yaw-rate limit [rad/s]. When None, no limit is applied.
max_wheel_speed – Per-wheel angular velocity limit [rad/s]. When None, no limit is applied.
device – Warp device for internal buffers. When a CUDA device is used, a CUDA graph is captured at construction.

Raises:

ValueError – If either direction vector is zero, not 3-element, or the two directions are less than 89.9 degrees apart.
ValueError – If wheel_radius or wheel_base is not strictly positive.
ValueError – If any of max_linear_speed, max_angular_speed, or max_wheel_speed is provided and not strictly positive.
ValueError – If left_wheel_joint or right_wheel_joint is not in robot_joint_space.

forward(

estimated_state: RobotState,

setpoint_state: RobotState | None,

t: float,

**kwargs: object,

) → RobotState | None#

Compute left and right wheel velocity targets from a control-point setpoint.

Parameters:

estimated_state – Current estimated state of the robot. Not used by this controller but required by the BaseController interface.
setpoint_state – Desired robot state. Must have a sites field containing the named control point with both linear and angular velocities.
t – Current clock time [s].
**kwargs – Unused; accepted for interface compatibility.

Returns:

RobotState with JointState velocity targets for the two wheel joints, or None if the control point site or its velocities are absent.

reset(

estimated_state: RobotState,

setpoint_state: RobotState | None,

t: float,

**kwargs: object,

) → bool#

Reset the controller.

This controller is stateless, so reset always succeeds immediately.

Parameters:

estimated_state – Current estimated state of the robot.
setpoint_state – Optional desired state of the robot.
t – Current clock time [s].
**kwargs – Unused; accepted for interface compatibility.

Returns:

Always True.

Bases: BaseController

Holonomic (omni / mecanum) drive controller.

Converts a planar twist setpoint at a named command site into per-wheel angular velocities using the closed-form inverse kinematics of the wheel base, and returns them as a RobotState whose joint state holds velocity targets for the wheel joints.

The command is read from the named site in setpoint_state.sites and is interpreted as the twist of the command site: linear_velocity is the velocity of the site’s origin and angular_velocity is taken about rotation_direction.

The command site is placed by command_site_position and command_site_quaternion, given in the same frame as wheel_positions and wheel_orientations. The wheel geometry is transformed into the site frame at construction, so the wheels may be measured in whatever frame is convenient (a robot root, a USD centre-of-mass prim) and the site moved independently of them. rotation_direction is then expressed in the command-site frame, so the default [0, 0, 1] means “the site frame’s own +Z”.

The kinematics are precomputed once at construction into two constant operators:

M (N×3) maps the command-site twist [v_u, v_v, w] to per-wheel no-slip-axis contact speeds u via u_i = a_iᵀ (v_c + ω × r_i).
K (diagonal, length N) converts each contact speed to a wheel joint angular velocity: φ̇_i = u_i / (r_i · cos γ_i), where γ_i is the roller offset angle (mecanum_angle_i − 90°).

On a CUDA device the full forward pass (project twist → clamp → matrix multiply) is captured into a CUDA graph at construction time. Each forward() call then copies the live setpoint into staging buffers and launches the graph with no CPU–GPU round-trips.

Parameters:

robot_joint_space – The ordered list of joint names defining the joint space of the controlled robot (for example, Articulation.dof_names).
wheel_joint_names – Names of the wheel joints, one per wheel, ordered to match wheel_positions[i] / wheel_orientations[i] / mecanum_angles[i]. Each name must be unique and present in robot_joint_space.
wheel_radius – Radius of each wheel (scalar broadcast to all wheels, or per-wheel array).
wheel_positions – Positions of each wheel, in the same frame as command_site_position.
wheel_orientations – Quaternion orientations of each wheel, in the same frame as command_site_quaternion, in [w, x, y, z] order.
mecanum_angles – Mecanum roller angle of each wheel in degrees, measured from the wheel axle (90 = omni / plain wheel, 45 or 135 = standard mecanum). This is the legacy isaacmecanumwheel:angle convention returned by HolonomicRobotUsdSetup, so it can be passed straight through. Scalar broadcast or per-wheel array.
wheel_axis – Local rotation (spin) axis of the wheel joint.
command_site_position – Position of the command site, in the same frame as wheel_positions. Defaults to the origin of that frame.
command_site_quaternion – Orientation of the command site, in the same frame as wheel_orientations, in [w, x, y, z] order. Defaults to identity, in which case the wheel frame is used as the command-site frame directly.
rotation_direction – Yaw axis, expressed in the command-site frame. Defaults to that frame’s [0, 0, 1].
max_linear_speed – Maximum planar linear speed [m/s]. None means no limit.
max_angular_speed – Maximum yaw rate [rad/s]. None means no limit.
max_wheel_speed – Maximum individual wheel angular velocity [rad/s]. None means no limit. Applied independently per wheel after the matrix multiply; this can distort the commanded direction.
control_point_name – Name of the site in setpoint_state.sites from which the twist command is read.
device – Warp device for internal buffers. If the device is a CUDA device, a CUDA graph is captured at construction time.

Raises:

ValueError – If wheel_radius, wheel_positions, or wheel_orientations is None.
ValueError – If wheel_joint_names contains duplicates, has a name not in robot_joint_space, or does not have one entry per wheel.
ValueError – If rotation_direction is zero, command_site_position does not have shape (3,), or command_site_quaternion does not have shape (4,) or is zero.
ValueError – If any wheel’s axle is parallel to rotation_direction (undefined rolling direction) or its mecanum angle is 0 or 180 degrees.
ValueError – If any provided speed limit is not strictly positive.

forward(

estimated_state: RobotState,

setpoint_state: RobotState | None,

t: float,

**kwargs: object,

) → RobotState | None#

Convert a command-site twist setpoint into per-wheel velocity targets.

Parameters:

estimated_state – Current estimated state of the robot (unused).
setpoint_state – Desired robot state containing the named command-site with both linear and angular velocities.
t – Current clock time (unused).
**kwargs – Additional keyword arguments (unused).

Returns:

RobotState whose joint state contains velocity targets for the wheel joints, or None if the command-site or its velocities are absent.

reset(

estimated_state: RobotState,

setpoint_state: RobotState | None,

t: float,

**kwargs: object,

) → bool#

Reset the controller.

The holonomic controller is stateless, so this always succeeds.

Parameters:

estimated_state – Current estimated state of the robot (unused).
setpoint_state – Desired setpoint state (unused).
t – Current clock time (unused).
**kwargs – Additional keyword arguments (unused).

Returns:

Always True.

Python API