Hello Robot

Learning Objectives

This tutorial details how to add and move a mobile robot in NVIDIA Isaac Sim in an extension application. After this tutorial, you will understand how to add a robot to the simulation and apply actions to its wheels using Python.

10-15 Minute Tutorial

Getting Started

Prerequisites

• Review Hello World prior to beginning this tutorial.

Begin with the source code of the Hello World example developed in the previous tutorial: Hello World.

Adding a Robot

Begin by adding a NVIDIA Jetbot to the scene. You can do so by accessing the library of NVIDIA Isaac Sim robots, sensors, and environments located on a Omniverse Nucleus Server using Python, as well as navigate through it using the Content window, under Isaac Sim > robots > NVIDIA > jetbot.usd.

Note

The server shown in these steps has been connected to in Workstation Setup. Follow these steps first before proceeding.

1. Add the assets by simply dragging them to the stage window or the viewport.

2. Try to do the same thing through Python in the Hello World example.

3. Create a new stage: File > new > Don’t Save

4. Open the hello_world.py file by clicking the Open Source Code button in the Hello World window.

Import packages:

import carb
import isaacsim.core.experimental.utils.stage as stage_utils
from isaacsim.core.experimental.prims import Articulation
from isaacsim.examples.base.base_sample_experimental import BaseSample
from isaacsim.storage.native import get_assets_root_path

Setup scene:

        # Add the Jetbot robot to the stage
        asset_path = assets_root_path + "/Isaac/Robots/NVIDIA/Jetbot/jetbot.usd"
        stage_utils.add_reference_to_stage(usd_path=asset_path, path="/World/Fancy_Robot")

Articulate the robot:

        # Wrap the Jetbot with the Articulation class for control
        self._jetbot = Articulation("/World/Fancy_Robot")

Complete code:

# -- Begin importing Isaac packages -- #
import carb
import isaacsim.core.experimental.utils.stage as stage_utils
from isaacsim.core.experimental.prims import Articulation
from isaacsim.examples.base.base_sample_experimental import BaseSample
from isaacsim.storage.native import get_assets_root_path

# -- End of importing Isaac packages -- #


class HelloWorld(BaseSample):
    def __init__(self) -> None:
        super().__init__()

    def setup_scene(self):
        # Add ground plane
        ground_plane = stage_utils.add_reference_to_stage(
            usd_path=get_assets_root_path() + "/Isaac/Environments/Grid/default_environment.usd",
            path="/World/ground",
        )

        # Get the assets root path from the Nucleus server
        assets_root_path = get_assets_root_path()
        if assets_root_path is None:
            carb.log_error("Could not find nucleus server with /Isaac folder")
            return

        # -- Begin adding Jetbot -- #
        # Add the Jetbot robot to the stage
        asset_path = assets_root_path + "/Isaac/Robots/NVIDIA/Jetbot/jetbot.usd"
        stage_utils.add_reference_to_stage(usd_path=asset_path, path="/World/Fancy_Robot")
        # -- End of adding Jetbot -- #

    async def setup_post_load(self):
        # -- Begin articulation -- #
        # Wrap the Jetbot with the Articulation class for control
        self._jetbot = Articulation("/World/Fancy_Robot")
        # -- End of articulation -- #

        # Print info about the Jetbot
        print("Number of DOFs: " + str(self._jetbot.num_dofs))
        print("DOF names: " + str(self._jetbot.dof_names))
        print("Joint Positions: " + str(self._jetbot.get_dof_positions().numpy()))

Click the LOAD button to load the scene and see the Jetbot appear. Although it is being simulated, it is not moving. The next section walks through how to make the robot move.

Move the Robot

In NVIDIA Isaac Sim, Robots are constructed of physically accurate articulated joints. Applying actions to these articulations make them move.

Next, apply random velocities to the Jetbot’s wheel joints to get it moving.

Importing SimulationManager:

from isaacsim.core.simulation_manager import SimulationManager

Registering callbacks:

        # Register a physics callback to send actions every physics step
        from isaacsim.core.simulation_manager.impl.isaac_events import IsaacEvents

        self._physics_callback_id = SimulationManager.register_callback(
            self.send_robot_actions, IsaacEvents.POST_PHYSICS_STEP
        )

Define command function:

    def send_robot_actions(self, dt, context):
        # Apply random velocity targets to the wheel joints
        # Jetbot has 2 DOFs: left_wheel_joint and right_wheel_joint
        random_velocities = 5 * np.random.rand(1, 2)  # Shape: (1, num_dofs)
        self._jetbot.set_dof_velocity_targets(random_velocities)

Complete code:

import carb
import isaacsim.core.experimental.utils.stage as stage_utils
import numpy as np
from isaacsim.core.experimental.prims import Articulation

# -- Begin importing SimulationManager -- #
from isaacsim.core.simulation_manager import SimulationManager

# -- End of importing SimulationManager -- #
from isaacsim.examples.base.base_sample_experimental import BaseSample
from isaacsim.storage.native import get_assets_root_path


class HelloWorld(BaseSample):
    def __init__(self) -> None:
        super().__init__()
        self._physics_callback_id = None

    def setup_scene(self):
        # Add ground plane
        ground_plane = stage_utils.add_reference_to_stage(
            usd_path=get_assets_root_path() + "/Isaac/Environments/Grid/default_environment.usd",
            path="/World/ground",
        )

        # Get the assets root path from the Nucleus server
        assets_root_path = get_assets_root_path()
        if assets_root_path is None:
            carb.log_error("Could not find nucleus server with /Isaac folder")
            return

        # Add the Jetbot robot to the stage
        asset_path = assets_root_path + "/Isaac/Robots/NVIDIA/Jetbot/jetbot.usd"
        stage_utils.add_reference_to_stage(usd_path=asset_path, path="/World/Fancy_Robot")

    async def setup_post_load(self):
        # Wrap the Jetbot with the Articulation class for control
        self._jetbot = Articulation("/World/Fancy_Robot")

        # -- Begin registering callback -- #
        # Register a physics callback to send actions every physics step
        from isaacsim.core.simulation_manager.impl.isaac_events import IsaacEvents

        self._physics_callback_id = SimulationManager.register_callback(
            self.send_robot_actions, IsaacEvents.POST_PHYSICS_STEP
        )
        # -- End of registering callback -- #

    # -- Begin sending actions -- #
    def send_robot_actions(self, dt, context):
        # Apply random velocity targets to the wheel joints
        # Jetbot has 2 DOFs: left_wheel_joint and right_wheel_joint
        random_velocities = 5 * np.random.rand(1, 2)  # Shape: (1, num_dofs)
        self._jetbot.set_dof_velocity_targets(random_velocities)

    # -- End of sending actions -- #

    def physics_cleanup(self):
        # Clean up callback when the extension is unloaded
        if self._physics_callback_id is not None:
            SimulationManager.deregister_callback(self._physics_callback_id)
            self._physics_callback_id = None

Click the LOAD button to load the scene and watch the Jetbot move with random velocities.

Note

Pressing STOP, then PLAY in this workflow might not reset the world properly. Use the RESET button instead.

Extra Practice

This example applies random velocities to the Jetbot articulation controller. Try the following exercises:

1. Make the Jetbot move backwards (hint: use negative velocities).

2. Make the Jetbot turn right (hint: apply different velocities to each wheel).

3. Make the Jetbot stop after 5 seconds (hint: track elapsed time in the callback).

Controlling Specific Joints

You can also control specific joints by their names or indices. Here’s how to get the wheel joint indices and apply velocities only to specific joints:

Getting wheel indices:

        # Print available DOF names
        print("Available DOFs:", self._jetbot.dof_names)

        # Get indices for specific wheel joints
        self._wheel_indices = self._jetbot.get_dof_indices(["left_wheel_joint", "right_wheel_joint"]).numpy()
        print("Wheel indices:", self._wheel_indices)

Setting wheel velocities using indices:

        # Apply velocity targets to specific DOF indices
        wheel_velocities = np.array([[10.0, 10.0]])  # Both wheels same speed = forward
        self._jetbot.set_dof_velocity_targets(wheel_velocities, dof_indices=self._wheel_indices)

Complete code:

import carb
import isaacsim.core.experimental.utils.stage as stage_utils
import numpy as np
from isaacsim.core.experimental.prims import Articulation
from isaacsim.core.simulation_manager import SimulationManager
from isaacsim.examples.base.base_sample_experimental import BaseSample
from isaacsim.storage.native import get_assets_root_path


class HelloWorld(BaseSample):
    def __init__(self) -> None:
        super().__init__()
        self._physics_callback_id = None

    def setup_scene(self):
        # Add ground plane
        ground_plane = stage_utils.add_reference_to_stage(
            usd_path=get_assets_root_path() + "/Isaac/Environments/Grid/default_environment.usd",
            path="/World/ground",
        )

        # Add the Jetbot robot to the stage
        assets_root_path = get_assets_root_path()
        asset_path = assets_root_path + "/Isaac/Robots/NVIDIA/Jetbot/jetbot.usd"
        stage_utils.add_reference_to_stage(usd_path=asset_path, path="/World/Fancy_Robot")

    async def setup_post_load(self):
        # Wrap the Jetbot with the Articulation class
        self._jetbot = Articulation("/World/Fancy_Robot")

        # -- Begin getting indices -- #
        # Print available DOF names
        print("Available DOFs:", self._jetbot.dof_names)

        # Get indices for specific wheel joints
        self._wheel_indices = self._jetbot.get_dof_indices(["left_wheel_joint", "right_wheel_joint"]).numpy()
        print("Wheel indices:", self._wheel_indices)
        # -- End of getting indices -- #

        # Register physics callback
        from isaacsim.core.simulation_manager.impl.isaac_events import IsaacEvents

        self._physics_callback_id = SimulationManager.register_callback(
            self.send_robot_actions, IsaacEvents.POST_PHYSICS_STEP
        )

    def send_robot_actions(self, dt, context):
        # -- Begin setting wheel velocity -- #
        # Apply velocity targets to specific DOF indices
        wheel_velocities = np.array([[10.0, 10.0]])  # Both wheels same speed = forward
        self._jetbot.set_dof_velocity_targets(wheel_velocities, dof_indices=self._wheel_indices)
        # -- End of setting wheel velocity -- #

    def physics_cleanup(self):
        if self._physics_callback_id is not None:
            SimulationManager.deregister_callback(self._physics_callback_id)
            self._physics_callback_id = None

Summary

This tutorial covered the following topics:

1. Adding NVIDIA Isaac Sim library components from a Nucleus Server

2. Adding a robot to the stage using stage_utils.add_reference_to_stage()

3. Wrapping a robot with the Articulation class for control

4. Using set_dof_velocity_targets() to apply velocity control

5. Registering physics callbacks with SimulationManager

6. Controlling specific joints by name or index

Next Steps

Continue on to the next tutorial in the Essential Tutorials series, Adding a Manipulator Robot, to learn how to add a manipulator robot to the simulation.

Further Learning

Nucleus Server

• For an overview of how to best leverage a Nucleus Server, see the Nucleus Overview in NVIDIA Omniverse tutorial.

Robot Specific Extensions

• NVIDIA Isaac Sim provides several robot extensions, including isaacsim.robot.experimental.wheeled_robots, and isaacsim.robot.experimental.manipulators.examples. To learn more, check out the standalone examples located at standalone_examples/api/isaacsim.robot.experimental.manipulators/franka and standalone_examples/api/isaacsim.robot.experimental.manipulators/universal_robots/.