Quickstart with a Robot

Please complete the previous Quickstart with Isaac Sim before starting this one.

Tutorial

GUI

Add a robot to Stage

Start with a new stage (File > New Stage). Add robot to the scene by going to the top Menu Bar and Click Create > Robot > Franka Emika Panda Arm.

Examine the robot

Use Physics Inspector to examine the robot’s joint properties. Go to Tools > Physics > Physics Inspector. In the newly appeared window on the left, select Franka to inspect. The window will populate the joint information, such as the upper and lower limits as well as its default position by default. Click on the hamburger icon on the top right to see more options, such as the joint stiffness and damping.

If you make any changes to these values, you may see the robot move on the Stage corresponding to the change. A green check mark will appear if you want to commit the changes to be the new default values for the robot.

Control the Robot

The GUI-based robot controllers are inside the Omniverse visual programming tool, OmniGraphs. There are more involved tutorials about OmniGraph in the Omnigraph section. For the purpose of this tutorial, we will generate the graph using a shortcut tool, and then examine the graph in the OmniGraph editor.

1. Open the graph generator by going to Tools -> Robotics -> Omnigraph Controllers -> Joint Position.

2. In the newly appeared “Articulation Position Controller Inputs” popup window, click Add for the “Robot Prim” field, select Franka as the Target.

3. Click OK to generate the graph.

To move the robot, you need to change the values in the “JointCommandArray” node inside the Position_Controller graph. You can do this by either selecting the node on the Stage tree, or selecting the node in the graph editor. Both will open up lead to the Properties panel showing the joint command values. Press Play first to start the simulation, then type or slide the values with name starting with “input*” to see the robot move. Those inputs corresponding to each joints on the robot, starting with the base joint.

To visualize the generated graph, open an graph editor window: Window -> Graph Editors -> Action Graph. The editor window should open below the Viewport. Select Edit Action Graph in the middle of the graph editor window, and select the only existing graph on the list.

Extension

Add a robot to Stage

Start with a new Stage (File -> New). To add a robot to the scene, copy-paste the following code snippet into the Script Editor and run it.

import carb
from isaacsim.core.prims import Articulation
from isaacsim.core.utils.stage import add_reference_to_stage
from isaacsim.storage.native import get_assets_root_path
import numpy as np

assets_root_path = get_assets_root_path()
if assets_root_path is None:
    carb.log_error("Could not find Isaac Sim assets folder")
usd_path = assets_root_path + "/Isaac/Robots/Franka/franka.usd"
prim_path = "/World/Arm"

add_reference_to_stage(usd_path=usd_path, prim_path=prim_path)
arm_handle = Articulation(prim_paths_expr=prim_path, name="Arm")
arm_handle.set_world_poses(positions=np.array([[0, -1, 0]]))

Examine the robot

Isaac Sim Core API has many function calls to retrieve information about the robot. Here are some examples for finding the number of joints and the joint names, various joint properties, and joint states.

Open a new tab in the Script Editor, copy-paste the following code snippet. This can only be run after the previous adding robot step, where arm_handle has already been established. Press Play before running the snippet. Physics must be running for these commands to work.

# Get the number of joints
num_joints = arm_handle.num_joints
print("Number of joints: ", num_joints)

# Get joint names
joint_names = arm_handle.joint_names
print("Joint names: ", joint_names)

# Get joint limits
joint_limits = arm_handle.get_dof_limits()
print("Joint limits: ", joint_limits)

# Get joint positions
joint_positions = arm_handle.get_joint_positions()
print("Joint positions: ", joint_positions)

Notice when you pressed “Run”, it only prints the state once, even if the simulation is running. You’d have to keep pressing “Run” if you want to see more recent states. If you want to see the information printed at every physics step, you would need to insert these commands into a physics callback that runs at each physics step. We will go more in depth on how time stepping works in the next section Workflows.

To insert the commands into a physics callback, run the following snippet in a separate tab in the Script Editor.

import asyncio
from isaacsim.core.api.simulation_context import SimulationContext

async def test():
    def print_state(dt):
        joint_positions = arm_handle.get_joint_positions()
        print("Joint positions: ", joint_positions)
    simulation_context = SimulationContext()
    await simulation_context.initialize_simulation_context_async()
    await simulation_context.reset_async()
    simulation_context.add_physics_callback("printing_state", print_state)

asyncio.ensure_future(test())

Start the simulation by pressing play, then run the snippet. You should see the information printed at every physics step into the terminal.

If printing at every physics step is no longer necessary, you can remove the physics callback by running the following snippet.

simulation_context = SimulationContext()
simulation_context.remove_physics_callback("printing_state")

Control the Robot

There are many ways to control the robot in Isaac Sim. The lowest level is sending direct joint commands to set position, velocity, and efforts. Here is an example of how to control the robot using the Articulation API at the joint level.

Open a new tab in the Script Editor, copy-paste the following code snippet. This can only be run after the previous adding robot step, where arm_handle has already been established. Press Play before running the snippet. Physics must be running for these commands to work. We will provide two positions for you to toggle between. If you’ve added the print state snippet above to each physics step, you should be able to see the printed joints numbers change as the robot moves.

# Set joint position randomly
arm_handle.set_joint_positions([[-1.5, 0.0, 0.0, -1.5, 0.0, 1.5, 0.5, 0.04, 0.04]])

# Set all joints to 0
arm_handle.set_joint_positions([[0.0, 0.0, 0.0 , 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]])

Similar to the get_joint_positions function above, the set_joint_positions here is only get executed once when you pressed “Run”. If you wish to send commands at every physics step, you would need to insert these commands into a physics callback that runs at each physics step.

Standalone Python

Recall that the script that runs this tutorial is located in standalone_examples/tutorials/getting_started_robot.py. To run the script, open a terminal, navigate to the root of the Isaac Sim installation, and run the following command:

Linux

./python.sh standalone_examples/tutorials/getting_started_robot.py

Windows

python.bat standalone_examples\tutorials\getting_started_robot.py

Code Explained

Line 14 to 50 in getting_started_robot.py script sets up the scene and adds robots to the stage. The script starts by import necessary modules, add the ground plane, set the camera angle, and add two robots — one arm, one mobile —- to the scene, using the same APIs that’s used in the Extension workflow.

The notable differences between the Extension workflow Python and Standalone Python are:

Starting the Simulator at the top

    from isaacsim import SimulationApp
    simulation_app = SimulationApp({"headless": False})  # start the simulation app, with GUI open

Using a “World” object

    my_world = World(stage_units_in_meters=1.0)

We will explain more about the World object in the Core API Overview. For now, think of it as the object that controls everything about this virtual world, such as physics and rendering stepping, and holding object handles.

Stepping the Simulation explicitly

At the bottom of the script, there is a loop, and a stepping function my_world.step() is called every iteration. Inside this stepping function, it move forward a fixed number of rendering and physics calculation.

The script will run for 4 cycles, and at each cycle, the arm and the car will move or stop moving. The car’s joint positions will be printed at every physics step in the last cycle.

    for i in range(4):
        print("running cycle: ", i)
        if i == 1 or i == 3:
            print("moving")
            # move the arm
            arm.set_joint_positions([[-1.5, 0.0, 0.0, -1.5, 0.0, 1.5, 0.5, 0.04, 0.04]])
            # move the car
            car.set_joint_velocities([[1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0]])
        if i == 2:
            print("stopping")
            # reset the arm
            arm.set_joint_positions([[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]])
            # stop the car
            car.set_joint_velocities([[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]])

        for j in range(100):
            # step the simulation, both rendering and physics
            my_world.step(render=True)
            # print the joint positions of the car at every physics step
            if i == 3:
                car_joint_positions = car.get_joint_positions()
                print("car joint positions:", car_joint_positions)

Notice the get_joint_positions and set_joint_positions functions are the same as the ones that are used in the Extension workflow, but since the stepping is explicit, the commands simply exist as part of the loop and gets executed every physics step by default. This is the main difference between the Extension and Standalone Python workflows. Go to the next section Workflows for more details.