cuMotion World Interface Tutorial#

Tutorial

This tutorial demonstrates how to use the CumotionWorldInterface class in the cuMotion integration to manage world state, obstacles, and robot base transforms for motion planning and control.

By the end of this tutorial, you’ll understand:

How to set up a CumotionWorldInterface
How to discover obstacles using SceneQuery
How to configure obstacle representations and safety tolerances
How to synchronize obstacle transforms

Prerequisites

Review the Scene Interaction tutorial to understand SceneQuery, WorldBinding, ObstacleStrategy, and the WorldInterface interface.
Know how to add collider presets to objects in Isaac Sim. You can add a collider preset by selecting an object in the stage tree, going to the Property tab, clicking the + Add button, and selecting Physics > Colliders Preset (or Physics > Rigid Body with Colliders Preset for dynamic objects).

To follow along with the tutorial, run your Isaac Sim instance. Open Window > Examples > Robotics Examples, then select Motion Generation > cuMotion > World Interface. Within the isaacsim.robot_motion.cumotion.examples extension, there is a fully functional example of the CumotionWorldInterface being used to manage world state, discover obstacles, and synchronize obstacle transforms.

Key Concepts#

The CumotionWorldInterface is the bridge between Isaac Sim’s USD scene and cuMotion’s collision world. It works together with the Motion Generation API classes to provide a complete world state management:

WorldInterface Implementation: CumotionWorldInterface implements the WorldInterface interface from the Motion Generation API, enabling use with WorldBinding and other Motion Generation API components
cuMotion World Management: Manages obstacles in cuMotion’s internal world representation, providing a world_view for collision queries
Debug Visualization: Optional visualization of collision geometries for debugging

The CumotionWorldInterface works with Motion Generation API classes:

SceneQuery discovers objects in the USD scene
ObstacleStrategy configures how obstacle geometries are approximated
WorldBinding initializes obstacle geometry from USD and synchronizes obstacle transforms to cuMotion using the world interface

Searching for Obstacles#

The first step is to create a SceneQuery and use it to discover objects in the USD scene:

    # Create scene query to discover obstacles
    scene_query = SceneQuery()

    # Find all objects in a bounding box
    objects = scene_query.get_prims_in_aabb(
        search_box_origin=[0.0, 0.0, 0.0],
        search_box_minimum=[-100.0, -100.0, -100.0],
        search_box_maximum=[100.0, 100.0, 100.0],
        tracked_api=TrackableApi.PHYSICS_COLLISION,
    )

    print("Discovered objects:", objects)

The SceneQuery searches for prims in the specified axis-aligned bounding box that have the specified collision API applied.

Configuring Obstacle Representations#

Obstacle geometries need to be approximated for collision checking. The ObstacleStrategy manages how different geometry types are represented:

In the case of cuMotion, there are not natively supported obstacle types for the shapes Cone or Cylinder, so we use the OBB representation for both, which CumotionWorldInterface maps to the native cumotion.CUBOID type.
For Mesh objects, we use the OBB representation for faster collision checking, but you can also use the TRIANGULATED_MESH representation for more accurate collision checking.
The safety tolerance adds extra padding around obstacles to ensure safe clearance during planning.

Note

Since ObstacleRepresentation is a StrEnum, you can use either the enum value (e.g., ObstacleRepresentation.OBB) or the string directly (e.g., "obb") when creating ObstacleConfiguration objects. See Scene Interaction for details on how to configure obstacle representations.

    # Set up obstacle strategy
    obstacle_strategy = ObstacleStrategy()

    # Set default safety tolerance for all obstacles
    obstacle_strategy.set_default_safety_tolerance(0.06)

    # Configure specific geometry types
    obstacle_strategy.set_default_configuration(
        Mesh, ObstacleConfiguration(representation="obb", safety_tolerance=0.01)
    )
    obstacle_strategy.set_default_configuration(
        Cone, ObstacleConfiguration(representation="obb", safety_tolerance=0.01)
    )
    obstacle_strategy.set_default_configuration(
        Cylinder, ObstacleConfiguration(representation="obb", safety_tolerance=0.01)
    )

Creating the World Interface and World Binding#

The WorldBinding can connect a CumotionWorldInterface with the USD scene. This tutorial injects a KitCumotionDebugVisualizer into the world interface, which will:

Create debug visualizations of the obstacles existing in the cumotion.World
Those objects will be colored in red to indicate that they are enabled for collision checking (no-go region).
Those objects will be colored in green to indicate that they are disabled for collision checking (go region).

    # Create world interface with optional debug visualizations
    world_interface = CumotionWorldInterface(debug_visualizer=KitCumotionDebugVisualizer())

    # Create world binding
    world_binding = WorldBinding(
        world_interface=world_interface,
        obstacle_strategy=obstacle_strategy,
        tracked_prims=objects,
        tracked_collision_api=TrackableApi.PHYSICS_COLLISION,
    )

    # Initialize the world binding (populates obstacles into cuMotion)
    world_binding.initialize()

The WorldBinding automatically extracts collision geometry from the tracked prims, and passes the appropriate data based on the obstacle strategy to the CumotionWorldInterface to be converted to cuMotion obstacle representations. Note that the CumotionWorldInterface can also be used without (or partially without) a WorldBinding. For example, you may:

Use WorldBinding to initially populate the scene, then manually call CumotionWorldInterface.update_obstacle_transforms() when transforms come from perception algorithms rather than the USD scene.
Directly populate CumotionWorldInterface using Isaac Sim Core API objects (Sphere, Cube) for simple scenes.

Synchronizing the World Binding#

Synchronize the world binding each frame to track moving obstacles, as covered in the Scene Interaction tutorial.

Synchronize transforms Update the positions and orientations of tracked obstacles:

    # Update the positions and orientations of tracked obstacles.
    world_binding.synchronize_transforms()

The binding reads obstacle shape properties, local scales, and collision-enabled state during initialization. It does not synchronize changes to those properties at runtime. Rebuild the binding when the obstacle configuration changes.

Note

The synchronization method described above is only used for collision objects that you are tracking. The robot base position is not automatically synchronized, and robot base pose is not generally part of the WorldInterface interface. The CumotionWorldInterface provides an additional function to update the robot base pose, which is covered in the section that follows.

Updating Robot Base Transforms#

The world interface needs to know the robot base transform to convert between world coordinates and robot base frame coordinates:

    # Update world interface with robot base transform
    world_binding.get_world_interface().update_world_to_robot_root_transforms(articulation.get_world_poses())

This should be called whenever the robot base moves, and will update the transforms of all obstacles in the cumotion.World to be relative to the robot base frame.

Exploring the Tutorial#

Note

To experiment with this tutorial interactively, see the scenario.py file in the isaacsim.robot_motion.cumotion.examples extension at isaacsim/robot_motion/cumotion/examples/world_interface/scenario.py.

This tutorial provides an interactive environment for experimenting with the CumotionWorldInterface. Here are some ways to explore the tutorial and learn about the CumotionWorldInterface:

Basic usage: move a single obstacle around the scene and see how the world interface tracks it.

In the example video below, we simply:

Run the example
Move around the cube obstacle (Shift + drag)
Run the example to synchronize the USD object’s transform with the internal cuMotion obstacle, represented as a red OBB.

../_images/isim_6.1_full_tut_viewport_moving_single_obstacle.webp — Moving a single obstacle around the scene.#

Adding obstacles

In the example video below, we:

Create a Sphere
Add collider Rigid Body with Colliders preset to the Sphere (so it is a physics collision and a dynamic object)
Reset the example (which re-generates the CumotionWorldInterface and its managed cumotion.World)
The Sphere is automatically found by the SceneQuery and added to the cumotion.World through the WorldBinding and CumotionWorldInterface.

../_images/isim_6.1_full_tut_viewport_adding_ball_obstacle.webp — Adding a ball obstacle to the scene.#

Changing obstacle representations

To learn further, try changing some settings in the ObstacleStrategy and see how the CumotionWorldInterface and cumotion.World react. For example, you can add a Mesh obstacle and represent it as TRIANGULATED_MESH instead of OBB to compare collision accuracy. In the interactive example’s scenario.py, configure the default representation for Mesh as follows:

# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

from isaacsim.core.experimental.objects import Mesh
from isaacsim.robot_motion.experimental.motion_generation import ObstacleConfiguration, ObstacleStrategy

obstacle_strategy = ObstacleStrategy()
obstacle_strategy.set_default_configuration(
    Mesh,
    ObstacleConfiguration(
        representation="triangulated_mesh",
        safety_tolerance=0.1,
    ),
)

Reload the example and select RESET to rebuild the WorldBinding. This setting applies only to Mesh prims; the Cone and Cylinder defaults remain OBB. The following two videos show meshes represented as OBB and TRIANGULATED_MESH, respectively.

../_images/isim_6.1_full_tut_viewport_mesh_representation_obb.webp — Representing a mesh as `OBB`#

../_images/isim_6.1_full_tut_viewport_mesh_representation_triangulated_mesh.webp — Representing a mesh as `TRIANGULATED_MESH`#

Summary#

This tutorial demonstrated:

World Interface Setup: Creating a CumotionWorldInterface (often using WorldBinding for centralized world state management)
Obstacle Discovery: Using SceneQuery to discover objects in the USD scene
Obstacle Configuration: Configuring obstacle representations and safety tolerances with ObstacleStrategy
World Synchronization: Synchronizing tracked obstacle transforms and rebuilding the binding when obstacle properties change
Debug Visualizations: Enabling visualizations to understand obstacle representations

The cuMotion world interface provides a centralized, efficient way to manage world state for all motion planning and control algorithms.

Next Steps#

RMPflow tutorial - Using the world interface with reactive control
Graph Planner tutorial - Collision-free path planning with obstacles
Trajectory Generator tutorial - Smooth path execution (collision-unaware)
Trajectory Optimizer tutorial - Optimization-based planning with world awareness

Back to Gallery View