[isaacsim.robot_motion.pink] pink#

Version: 0.1.7

Overview#

The isaacsim.robot_motion.pink extension provides access to the PINK (Python Inverse Kinematics) library within Isaac Sim’s motion generation framework. PINK formulates differential inverse kinematics as a quadratic program (QP) with weighted tasks and safety constraints, powered by the Pinocchio rigid-body dynamics library.

Concepts#

PINK Integration#

PINK solves differential inverse kinematics by composing weighted task objectives (end-effector tracking, posture regularization, velocity damping) with inequality constraints from joint limits, velocity limits, and control barrier functions. The extension wraps this into Isaac Sim’s BaseController interface for seamless integration with simulation workflows.

Robot Configuration#

Robot setup requires a URDF file describing the kinematic chain. Pinocchio loads the URDF and provides forward kinematics, Jacobians, and frame placement. An optional SRDF file can configure self-collision exclusion pairs.

Key Components#

`PinkRobot`#

The PinkRobot dataclass holds the Pinocchio model, data, and controlled joint names. Load robots using load_pink_robot for custom URDFs or load_pink_supported_robot for pre-configured robots bundled with the extension.

`PinkIKController`#

PinkIKController implements the BaseController interface using PINK’s solve_ik. On each forward() call it updates task targets from the setpoint, solves the QP to obtain a joint velocity, and integrates the configuration. The controller supports configurable frame tasks, posture regularization, arbitrary user-supplied tasks, limits, and barriers.

Integration#

The extension integrates with Isaac Sim’s experimental motion generation framework through the isaacsim.robot_motion.experimental.motion_generation dependency. The controller implements the BaseController interface, ensuring compatibility with SelectableController, CombinedController, ChainedController, and TrajectoryFollower from the broader motion generation ecosystem.

Enable Extension#

The extension can be enabled (if not already) in one of the following ways:

Command-line interface

Define the next entry as an application argument from a terminal.

APP_SCRIPT.(sh|bat) --enable isaacsim.robot_motion.pink

Experience/extension configuration

Define the next entry under [dependencies] in an experience (.kit) file or an extension configuration (extension.toml) file.

[dependencies]
"isaacsim.robot_motion.pink" = {}

Extension Manager UI

Open the Window > Extensions menu in a running application instance and search for isaacsim.robot_motion.pink. Then, toggle the enable control button if it is not already active.

Python API#

Configuration Loading

`load_pink_robot`	Load a PINK robot from a URDF file via Pinocchio.
`load_pink_supported_robot`	Load a pre-configured robot bundled with the extension.
`PinkRobot`	Robot configuration for PINK inverse kinematics.

Inverse Kinematics Controller

PinkIKController

Reactive inverse kinematics controller using PINK's differential IK solver.

Transform Utilities

`impl.utils.isaac_sim_position_quaternion_to_se3`	Convert Isaac Sim (position, quaternion) to a Pinocchio SE3 transform.
`impl.utils.se3_to_isaac_sim_position_quaternion`	Convert a Pinocchio SE3 transform to Isaac Sim (position, quaternion).
`impl.utils.map_joint_positions_to_pinocchio`	Map Isaac Sim joint positions to a Pinocchio configuration vector.
`impl.utils.map_pinocchio_velocity_to_joint_state`	Convert a Pinocchio tangent velocity to an Isaac Sim JointState with integrated positions.

Configuration Loading#

load_pink_robot( urdf_path: Path | str, package_dirs: list[str] | None = None, srdf_path: Path | str | None = None, build_collision_model: bool = False, ) → PinkRobot#

Load a PINK robot from a URDF file via Pinocchio.

Parses the URDF into a Pinocchio model and optionally builds collision geometry for self-collision avoidance barriers.

Parameters:

urdf_path – Path to the URDF file.
package_dirs – List of package directories for resolving mesh paths in the URDF. Defaults to the URDF’s parent directory.
srdf_path – Optional path to an SRDF file for collision pair exclusion.
build_collision_model – If True, build the collision geometry model from the URDF. Required for SelfCollisionBarrier support.

Returns:

PinkRobot containing the Pinocchio model and controlled joint information.

Raises:

FileNotFoundError – If the URDF file does not exist.
ValueError – If the URDF cannot be parsed by Pinocchio.

Example

robot = load_pink_robot(
    urdf_path="/path/to/franka/robot.urdf",
    build_collision_model=True,
)

load_pink_supported_robot( robot_name: str, ) → PinkRobot#

Load a pre-configured robot bundled with the extension.

Loads a robot from the extension’s robot_configurations directory. Each supported robot has a subdirectory containing at minimum a robot.urdf file and optionally an SRDF for collision pair configuration.

Parameters:: robot_name – Name of the robot (e.g., “franka”, “ur10”). Must match a subdirectory under robot_configurations/.
Returns:: PinkRobot for the specified robot.
Raises:: FileNotFoundError – If the robot name does not correspond to a bundled configuration.

Example

robot = load_pink_supported_robot("franka")

class PinkRobot(directory: ~pathlib.Path, model: ~pinocchio.pinocchio_pywrap_default.Model, data: ~pinocchio.pinocchio_pywrap_default.Data, controlled_joint_names: list[str], collision_model: ~pinocchio.pinocchio_pywrap_default.GeometryModel | None = None, collision_data: ~pinocchio.pinocchio_pywrap_default.GeometryData | None = None, q0: ~numpy.ndarray = <factory>)#

Bases: object

Robot configuration for PINK inverse kinematics.

Encapsulates a Pinocchio model and associated data needed for differential IK solving with the PINK library. The model is loaded from URDF and provides forward kinematics, Jacobian computation, and frame placement.

Parameters:

directory – Path to the robot configuration directory containing the URDF file.
model – Pinocchio rigid-body model parsed from the URDF.
data – Pinocchio model data (pre-allocated workspace for FK/Jacobians).
controlled_joint_names – Ordered list of actuated joint names controlled by the IK solver.
collision_model – Pinocchio geometry model for collision checking. None if not loaded.
collision_data – Pinocchio geometry data for collision distance queries. None if not loaded.
q0 – Neutral (home) configuration vector. Defaults to the Pinocchio model neutral pose.

collision_data: GeometryData | None = None#

collision_model: GeometryModel | None = None#

controlled_joint_names: list[str]#

data: Data#

directory: Path#

model: Model#

q0: ndarray#

Inverse Kinematics Controller#

class PinkIKController( pink_robot: PinkRobot, robot_joint_space: list[str], robot_site_space: list[str], *, tool_frame: str | None = None, position_cost: float | list[float] = 1.0, orientation_cost: float | list[float] = 1.0, posture_cost: float | None = 0.001, damping: float = 1e-12, gain: float = 1.0, lm_damping: float = 0.0, solver: str = 'osqp', extra_tasks: list | None = None, extra_limits: list | None = None, extra_barriers: list | None = None, pre_step_callback: Callable | None = None, dt: float, )#

Bases: BaseController

Reactive inverse kinematics controller using PINK’s differential IK solver.

Implements the BaseController interface by wrapping PINK’s solve_ik into a closed-loop reactive controller. On each forward() call the controller:

Updates the Pinocchio configuration from the estimated robot state.
Updates task targets from the setpoint (end-effector pose, posture, etc.).
Solves the QP to obtain a joint velocity.
Integrates the velocity and returns the result as a RobotState.

The controller manages a FrameTask for end-effector tracking and an optional PostureTask for joint regularization. Users may supply additional PINK tasks, limits, and barriers through the constructor.

Parameters:

pink_robot – Robot loaded via load_pink_robot() or load_pink_supported_robot().
robot_joint_space – Full ordered joint-space of the controlled robot in Isaac Sim.
robot_site_space – Full ordered site-space (frame names) of the controlled robot.
tool_frame – Pinocchio frame name for the end-effector. If None, the last frame in the model is used.
position_cost – Cost weight(s) for the end-effector position task, in [cost]/[m]. Scalar or 3D vector for anisotropic weighting.
orientation_cost – Cost weight(s) for the end-effector orientation task, in [cost]/[rad]. Scalar or 3D vector for anisotropic weighting.
posture_cost – Cost weight for the posture regularization task, in [cost]/[rad]. Set to 0.0 or None to disable.
damping – Tikhonov regularization added to the QP Hessian for numerical stability.
gain – Proportional gain for all managed tasks (0.0 to 1.0). A value of 1.0 corresponds to dead-beat control (full error correction per step).
lm_damping – Levenberg-Marquardt damping for the frame task.
solver – QP solver backend name (e.g. "osqp", "clarabel").
extra_tasks – Additional PINK Task instances to include in the QP objective.
extra_limits – Additional PINK Limit instances beyond the default configuration and velocity limits.
extra_barriers – PINK Barrier instances for safety constraints (e.g. SelfCollisionBarrier, PositionBarrier, BodySphericalBarrier).
pre_step_callback –
Optional callable invoked at the start of each forward() call, after the configuration has been updated from the estimated state but before solve_ik is called. Signature:
```
callback(configuration: pink.Configuration, setpoint_state: RobotState | None)
```
Use this to update targets on extra tasks that need per-step updates (e.g. RelativeFrameTask, ComTask, JointVelocityTask).
dt – Integration timestep in seconds used for solve_ik.

Example

from pink.tasks import RelativeFrameTask

relative_task = RelativeFrameTask("frame_a", "frame_b",
                                  position_cost=1.0, orientation_cost=0.5)

def update_relative_target(configuration, setpoint_state):
    relative_task.set_target_from_configuration(configuration)

controller = PinkIKController(
    pink_robot=robot,
    robot_joint_space=articulation.dof_names,
    robot_site_space=["panda_hand"],
    tool_frame="panda_hand",
    extra_tasks=[relative_task],
    pre_step_callback=update_relative_target,
    dt=1.0 / 60.0,
)

forward(

estimated_state: RobotState,

setpoint_state: RobotState | None,

t: float,

**kwargs: Any,

) → RobotState | None#

Compute desired joint positions by solving the differential IK QP.

Updates the Pinocchio configuration from estimated_state, sets task targets from setpoint_state, solves the QP, and integrates the resulting velocity.

Parameters:

estimated_state – Current estimated robot state (joint positions required).
setpoint_state – Desired setpoint containing target site poses and/or joint posture targets. The tool frame must match the frame configured at init.
t – Current simulation clock time (unused by the stateless QP, but required by the BaseController interface).
**kwargs – Additional arguments (unused).

Returns:

RobotState containing desired joint positions and velocities for the controlled joints, or None if the controller is not yet initialized.

get_frame_task() → FrameTask#

Get the end-effector FrameTask for external configuration.

Returns:: The PINK FrameTask controlling end-effector tracking.

get_posture_task() → PostureTask | None#

Get the PostureTask if configured, for external target updates.

Returns:: The PINK PostureTask, or None if posture regularization is disabled.

reset(

estimated_state: RobotState,

setpoint_state: RobotState | None,

t: float,

**kwargs: Any,

) → bool#

Initialize the controller from the current robot state.

Creates the PINK Configuration from estimated_state joint positions, sets the posture task target to the current configuration, and initializes the frame task target from the current end-effector pose.

Parameters:

estimated_state – Current estimated robot state (joint positions required).
setpoint_state – Initial setpoint (currently unused during reset).
t – Current simulation clock time.
**kwargs – Additional arguments (unused).

Returns:

True if reset succeeded, False if joint positions could not be extracted.

Transform Utilities#

The transform utilities convert between Isaac Sim’s (position, quaternion) representation and Pinocchio’s SE3 transforms.

isaac_sim_position_quaternion_to_se3( position: ndarray | array | list[float], quaternion: ndarray | array | list[float], ) → SE3#

Convert Isaac Sim (position, quaternion) to a Pinocchio SE3 transform.

Parameters:

position – Translation [x, y, z].
quaternion – Orientation as quaternion [w, x, y, z] (Isaac Sim convention).

Returns:

Pinocchio SE3 rigid-body transform.

Raises:

ValueError – If position is not size 3 or quaternion is not size 4.

se3_to_isaac_sim_position_quaternion( transform: SE3, ) → tuple[ndarray, ndarray]#

Convert a Pinocchio SE3 transform to Isaac Sim (position, quaternion).

Parameters:: transform – Pinocchio SE3 rigid-body transform.
Returns:: Tuple of (position, quaternion) where position is shape (3,) and quaternion is shape (4,) in (w, x, y, z) format.

map_joint_positions_to_pinocchio( joint_names: list[str], joint_positions: ndarray, model: Model, q_current: ndarray | None = None, ) → ndarray#

Map Isaac Sim joint positions to a Pinocchio configuration vector.

Builds a full Pinocchio configuration vector by placing the provided joint values at their correct indices in the model. Joints not in joint_names retain values from q_current (or the model neutral pose if not given).

Parameters:

joint_names – Ordered joint names matching joint_positions.
joint_positions – Joint position values corresponding to joint_names.
model – Pinocchio model providing joint index mapping.
q_current – Base configuration to fill unspecified joints. Defaults to model neutral.

Returns:

Full Pinocchio configuration vector of size model.nq.

Raises:

ValueError – If an input vector has the wrong size, or a named joint has more than one degree of freedom.

map_pinocchio_velocity_to_joint_state( velocity: ndarray, model: Model, controlled_joint_names: list[str], robot_joint_space: list[str], dt: float, q_current: ndarray, current_joint_positions: ndarray, ) → JointState#

Convert a Pinocchio tangent velocity to an Isaac Sim JointState with integrated positions.

Integrates the velocity over dt to produce target positions and packages both positions and velocities into a JointState for the motion generation API.

Parameters:

velocity – Tangent-space velocity vector of size model.nv.
model – Pinocchio model.
controlled_joint_names – Names of joints controlled by the IK solver.
robot_joint_space – Full ordered joint-space of the robot in Isaac Sim.
dt – Integration timestep in seconds.
q_current – Current configuration vector (pre-integration).
current_joint_positions – Current Isaac Sim positions for controlled_joint_names. These preserve the unwrapped angle of continuous joints.

Returns:

JointState containing integrated target positions and velocities for controlled joints.

Raises:

ValueError – If an input vector has the wrong size, or a named joint has more than one degree of freedom.

[isaacsim.robot_motion.pink] pink#

Overview#

Concepts#

PINK Integration#

Robot Configuration#

Key Components#

PinkRobot#

PinkIKController#

Integration#

Enable Extension#

Python API#

Configuration Loading#

Inverse Kinematics Controller#

Transform Utilities#

`PinkRobot`#

`PinkIKController`#