Authoring a Camera Sensor USD Asset#

Tutorial

This tutorial demonstrates how to author a reusable camera sensor asset in USD for Isaac Sim. The asset can be added to robots, environments, and other scenes in downstream workflows, and is compatible with the sensor authoring APIs in Isaac Sim.

The resulting USD asset will include a Camera prim, a RenderProduct prim, and RenderVar prims. It will define OpenCV fisheye lens distortion, sensor tick rate, supported resolutions and outputs, and a USD variant set for switching tick rate based on the selected resolution.

Learning Objectives#

This tutorial details how to:

Create a camera, render product, and render var prim.
Author the camera, render product, and render var relationships.
Apply and configure the OpenCV fisheye lens distortion API schema.
Author the sensor tick rate.
Store multiple sensor configurations in a variant set.
Save the asset, validate its structure, and verify that it is preserved.

Prerequisites#

Review the introduction to camera sensors.
Familiarity with the Script Editor for running Python APIs inside the GUI.

Overview#

This tutorial is organized as a single workflow with three sequential phases:

Set Up the Camera and Test Environment: Create the sensor root Xform, add the camera, load a test environment, configure the camera’s properties, and apply lens distortion in the GUI.
Author the Sensor in USD: Use the Script Editor to create the render product, render var, relationships, sensor tick rate, and variants.
Validate and Save the Asset: Save the asset, run the RGB sensor validator, and verify the result in a new scene.

Set Up the Camera and Test Environment#

Create the Sensor Root `Xform`#

Start from a new, empty stage (File > New), then author a single root Xform (a transform prim) to contain the whole sensor. Grouping the sensor under one Xform and making it the stage’s default prim gives the finished asset a clean entry point when it is referenced into other stages.

Right-click inside the Stage panel (Window > Stage if it is not open) and select Create > Xform, or click Create > Xform from the top Menu Bar. A new Xform is added to the stage, by default under the /World prim.
Rename the new Xform to CameraSensor: double-click its name in the Stage panel (or right-click it and select Rename), then type CameraSensor.
Make the Xform a top-level prim so it is the asset’s root. If it was created under /World, select it, open the Edit menu, and click Unparent. Confirm that its path in the Property panel is now /CameraSensor rather than /World/CameraSensor.
Right-click the /CameraSensor prim and select Set as Default Prim.

All of the sensor prims you create in the following steps are placed under this Xform.

../_images/isim_6.1_full_tut_gui_camera_sensor_root_default_prim.png — Stage tree showing the top-level `/CameraSensor` `Xform` set as the stage’s default prim.#

Create a Camera#

Add a camera as a child of the sensor root Xform. This follows the same workflow as Tutorial 4: Add Camera and Sensors to a Robot.

In the Stage panel, select the /CameraSensor Xform so the new camera is created underneath it.
From the Menu Bar, select Create > Camera. A camera appears in the Stage panel, and a grey wireframe representing the camera’s view appears in the viewport.
Rename the camera to CameraRGB. This will help us distinguish between the camera you are authoring and the one loaded from the warehouse scene.

You will aim the camera once the warehouse scene is loaded in the next step.

Load the Test Environment#

This tutorial uses a warehouse USD as a temporary test environment that provides visible geometry for aiming and verifying the camera. It is not part of the final sensor asset and is removed before saving.

Open Content Browser if it’s not already open (Window > Browsers > Content).
In the Stage panel, select /World or the top-level environment prim where you want to load the warehouse.
In the Content Browser, open Isaac Sim > Environments > Simple_Warehouse, then drag warehouse.usd into the viewport to add it to the current stage. After it loads, verify that the warehouse is visible in the viewport.
Select the /CameraSensor root Xform and set its transform to point the camera toward the warehouse. Authoring the pose on the root (not the camera) keeps the camera at the asset’s origin, so downstream scenes can place the whole sensor by transforming just the root. For reference, the example images in this tutorial use the following values on the /CameraSensor Xform:
- Translate: X = 0.0, Y = -2.72255, Z = 1.98124
- Orient: X = 90.0, Y = 60.127, Z = 0.0
- Scale: X = 1.0, Y = 1.0, Z = 1.0
Switch the viewport to look through the camera by clicking Perspective at the upper-left of the viewport and selecting Cameras > CameraRGB. Confirm that the camera sees the scene.

Note

The camera icon is hidden by default in the viewport. To see the camera icon, go to the eye menu on the top edge of the viewport, and select Show By Type > Cameras.

../_images/isim_6.1_full_tut_viewport_camera_sensor_warehouse_view.png — Looking through `/CameraSensor/CameraRGB` at the warehouse test environment using the example transform values.#

Configure Camera Properties#

Select the camera prim in the Stage tree. In the Property panel, configure the camera attributes needed for the sensor. For this tutorial you can use values such as:

Focal Length: 24. This controls the field of view; smaller values give a wider view.
Horizontal Aperture: 20.955 and Vertical Aperture: 15.2908. The sensor size and focal length set the field of view.
Clipping Range: (0.1, 100000). These are the near and far distances the camera renders.
Projection: perspective.

For this simple demonstration, the values do not match a calibrated physical camera, and were selected to produce clear camera output.

../_images/isim_6.1_full_tut_gui_camera_sensor_camera_properties.png — The **Property** panel showing the configured camera lens, aperture, projection, and clipping values.#

Note

On Camera prims, focal length and aperture are defined in tenths of stage units rather than stage units. For full detail on these optical parameters, see Camera Sensors.

Add Lens Distortion#

Omniverse cameras support modeling several physical lens distortion. See Calibration and Camera Lens Distortion Models for more information about supported lens distortion models. In this example, you will apply an OpenCV fisheye lens distortion model.

In the Stage window, select the /CameraSensor/CameraRGB prim, then in the Property panel click Add > Edit API Schema and select OmniLensDistortionOpenCvFisheyeAPI. See How To Add Schemata to Cameras for the detailed UI workflow. If Edit API Schema does not appear as an option, open the Extensions window (Window > Extensions), search for omni.kit.widget.schema_api (Property Window Edit API Schema), and enable it.

Once the schema is applied, its attributes appear in the Property panel under the Extra Properties section. Scroll down to that section to find the distortion attributes (fx, fy, cx, cy, and k1–k4) and edit them there.

For demonstration purposes, set the distortion attributes to values slightly offset from their defaults so the fisheye distortion is applied, for example:

fx: 905 and fy: 805. These are the focal lengths in pixels.
cx: 1030 and cy: 515. This is the principal point in pixels.
k1: 0.00247 (leave k2–k4 at their default 0). These are the radial distortion coefficients.

For calibrated sensors, use the values from the camera calibration process. Confirm that the distortion appears in the viewport or rendered camera output.

../_images/isim_6.1_full_tut_viewport_camera_sensor_fisheye_view.png — Camera output after applying the OpenCV fisheye lens distortion.#

Complete the Sensor Authoring#

Three prim types work together to define the sensor and its output, each with a distinct role:

Camera prim: Configures the physical camera being modeled, including its intrinsics (focal length, aperture, and lens distortion), extrinsics (pose in the scene), and tick rate.
RenderProduct prim: Controls how the RTX renderer renders that camera, including the output resolution, and references the Camera it draws from.
RenderVar prim: Specifies which outputs the RTX renderer produces for the sensor, such as color, depth, or surface normals.

You already created the /CameraSensor root Xform and the CameraRGB Camera prim in the GUI, so the snippets below build on the current stage rather than recreating them.

All code examples come from the complete, runnable file camera_sensor_usd_tutorial.py. The full script is provided here as a reference: it contains the complete authoring workflow described in this tutorial and can be pasted into the Script Editor and run in one shot.

camera_sensor_usd_tutorial.py

# <start-apply-sensor-api-snippet>
import isaacsim.core.experimental.utils.prim as prim_utils
import isaacsim.core.experimental.utils.stage as stage_utils
from isaacsim.core.experimental.objects import Camera
from pxr import Gf, UsdRender

# Reuse the GUI-authored root, or create it when running the complete script on an empty stage.
sensor_root_path = "/CameraSensor"
root_prim = stage_utils.define_prim(sensor_root_path, type_name="Xform")
stage_utils.get_current_stage(backend="usd").SetDefaultPrim(root_prim)
camera_path = sensor_root_path + "/CameraRGB"
# Preserve the GUI-authored transform when wrapping the camera.
camera = Camera(camera_path, reset_xform_op_properties=False)
camera_prim = camera.prims[0]
# Variants set the tick rate later; the default is 0 Hz (every frame).
# `OmniSensorAPI` has no typed schema for `prim_utils.ensure_api`.
if not prim_utils.has_api(camera_prim, "OmniSensorAPI"):
    camera_prim.ApplyAPI("OmniSensorAPI")
# <end-apply-sensor-api-snippet>

# <start-create-render-product-snippet>
# Variants set the resolution later.
render_product_path = sensor_root_path + "/RenderProduct"
render_product_prim = stage_utils.define_prim(render_product_path, type_name="RenderProduct")
# Render product relationships have no experimental wrapper.
render_product = UsdRender.Product(render_product_prim)
render_product.CreateCameraRel().SetTargets([camera_path])
# <end-create-render-product-snippet>

# <start-add-render-var-snippet>
# `sourceName` selects the output type.
render_var_path = render_product_path + "/RenderVar"
render_var_prim = stage_utils.define_prim(render_var_path, type_name="RenderVar")
source_name_attr = prim_utils.create_prim_attribute(render_var_prim, name="sourceName", type_name="string")
source_name_attr.Set("LdrColor")
# Render product relationships have no experimental wrapper.
render_product.CreateOrderedVarsRel().AddTarget(render_var_path)
# <end-add-render-var-snippet>

# <start-author-variants-snippet>
resolution_attr = prim_utils.create_prim_attribute(render_product_prim, name="resolution", type_name="int2")
tick_rate_attr = prim_utils.create_prim_attribute(camera_prim, name="omni:sensor:tickRate", type_name="float")
# Put variants on the root so they can configure both descendants.
# Experimental APIs cannot create variants or enter edit contexts.
variant_set = root_prim.GetVariantSets().AddVariantSet("SensorConfig")
configs = {
    "HighResolution_60Hz": (Gf.Vec2i(1920, 1080), 60.0),
    "HighSpeed_120Hz": (Gf.Vec2i(1280, 720), 120.0),
}
for variant_name, (resolution, tick_rate) in configs.items():
    variant_set.AddVariant(variant_name)
    prim_utils.set_prim_variants(root_prim, variants=[("SensorConfig", variant_name)])
    # Author values in the selected variant.
    with variant_set.GetVariantEditContext():
        resolution_attr.Set(resolution)
        tick_rate_attr.Set(tick_rate)
# Select the default variant.
prim_utils.set_prim_variants(root_prim, variants=[("SensorConfig", "HighResolution_60Hz")])
# <end-author-variants-snippet>

# <start-visualize-snippet>
import omni.kit.viewport.utility

viewport = omni.kit.viewport.utility.get_active_viewport()
if viewport is None:
    raise RuntimeError("No active viewport is available")

viewport_render_product_path = viewport.render_product_path
viewport.render_product_path = render_product_path
# <end-visualize-snippet>

# <start-restore-viewport-snippet>
viewport.render_product_path = viewport_render_product_path
# <end-restore-viewport-snippet>

Open the Script Editor from the top Menu Bar by clicking Window > Script Editor, then paste each snippet and click Run.

Apply the Sensor API#

Apply the OmniSensorAPI schema to the camera so it behaves as a sensor with a tick rate, the rate in Hz at which the renderer renders it. This snippet only applies the schema; the variant set in the next step authors the tick rate values. On its own, the tick rate stays at the schema default of 0, the autotrigger mode that renders every frame. For more detail, see Multi-Tick Rendering.

import isaacsim.core.experimental.utils.prim as prim_utils
import isaacsim.core.experimental.utils.stage as stage_utils
from isaacsim.core.experimental.objects import Camera
from pxr import Gf, UsdRender

# Reuse the GUI-authored root, or create it when running the complete script on an empty stage.
sensor_root_path = "/CameraSensor"
root_prim = stage_utils.define_prim(sensor_root_path, type_name="Xform")
stage_utils.get_current_stage(backend="usd").SetDefaultPrim(root_prim)
camera_path = sensor_root_path + "/CameraRGB"
# Preserve the GUI-authored transform when wrapping the camera.
camera = Camera(camera_path, reset_xform_op_properties=False)
camera_prim = camera.prims[0]
# Variants set the tick rate later; the default is 0 Hz (every frame).
# `OmniSensorAPI` has no typed schema for `prim_utils.ensure_api`.
if not prim_utils.has_api(camera_prim, "OmniSensorAPI"):
    camera_prim.ApplyAPI("OmniSensorAPI")

Confirm that the OmniSensorAPI schema is applied to the camera prim.

../_images/isim_6.1_full_tut_gui_camera_sensor_omnisensorapi_tickrate.png — The **Property** panel showing `OmniSensorAPI` applied with the `omni:sensor:tickRate` attribute available.#

Create a Render Product Prim#

Create a render product prim and connect it to the existing camera through the camera relationship. The render product defines the rendered output associated with the camera. Its resolution is authored later by the variant set, so it is not set here.

# Variants set the resolution later.
render_product_path = sensor_root_path + "/RenderProduct"
render_product_prim = stage_utils.define_prim(render_product_path, type_name="RenderProduct")
# Render product relationships have no experimental wrapper.
render_product = UsdRender.Product(render_product_prim)
render_product.CreateCameraRel().SetTargets([camera_path])

After running the script, confirm that the render product prim appears in the Stage tree at /CameraSensor/RenderProduct and that its camera relationship targets /CameraSensor/CameraRGB.

Create a Render Var Prim and Connect It#

Create a render var prim and connect it to the render product. The render var defines the type of output produced by the render product, such as color, depth, or normals.

In this example, the render var is created as a child of the render product and connected through the render product’s orderedVars relationship.

# `sourceName` selects the output type.
render_var_path = render_product_path + "/RenderVar"
render_var_prim = stage_utils.define_prim(render_var_path, type_name="RenderVar")
source_name_attr = prim_utils.create_prim_attribute(render_var_prim, name="sourceName", type_name="string")
source_name_attr.Set("LdrColor")
# Render product relationships have no experimental wrapper.
render_product.CreateOrderedVarsRel().AddTarget(render_var_path)

After running the script, select the render product prim and verify that it references the camera and the render var.

../_images/isim_6.1_full_tut_gui_camera_sensor_render_var.png — The **Property** panel for `/CameraSensor/RenderProduct` showing the `camera` relationship targeting `/CameraSensor/CameraRGB` and the `orderedVars` relationship targeting `/CameraSensor/RenderProduct/RenderVar`.#

Note

The resolution is stored on the render product’s resolution attribute as an integer pair of width and height. It is authored by the variant set below (1920 x 1080 or 1280 x 720); until then the render product uses USD’s default resolution.

Point the Viewport to the Authored Render Product#

Before adding the optional variant set, redirect the viewport to /CameraSensor/RenderProduct so you can see the sensor output as you configure it.

First, enable the viewport’s resolution overlay:

Click the Eye symbol (Show Settings) in the viewport toolbar.
Select Heads Up Display > Resolution.

The overlay shows the current viewport resolution (typically 1280×720). This reflects the native resolution Isaac Sim uses for interactive performance, not the resolution authored on /CameraSensor/RenderProduct.

To understand why they differ, open the Layer window (Window > Layers). Under Root Layer, expand the /Render/OmniverseKit/HydraTextures scope and locate the omni_kit_widget_viewport prim. This is a hidden, autogenerated RenderProduct that the viewport uses internally. When you select a camera from the viewport’s Cameras menu, Isaac Sim redirects this hidden render product to that camera and renders at 1280×720 for performance. You can modify this prim, but because it is autogenerated it cannot be shipped as part of a sensor USD asset.

To visualize the actual output of /CameraSensor/RenderProduct, run the following snippet in the Script Editor to redirect the active viewport to use the render product you just created:

import omni.kit.viewport.utility

viewport = omni.kit.viewport.utility.get_active_viewport()
if viewport is None:
    raise RuntimeError("No active viewport is available")

viewport_render_product_path = viewport.render_product_path
viewport.render_product_path = render_product_path

After running the snippet, the Resolution overlay in the viewport will change from the native 1280×720 to match whatever resolution is authored on /CameraSensor/RenderProduct. The viewport is now rendering through the authored render product rather than the hidden autogenerated one.

Note

When you redirect the viewport to /CameraSensor/RenderProduct using the snippet above, the Resolution overlay may show an unexpected value. This is a known Kit behavior when switching the active render product at runtime and will be addressed in a future update. The authored render product and its relationships are correct; this only affects the viewport display while using this tutorial’s visualization snippet.

The snippet saves the viewport’s original render product path in viewport_render_product_path. Before you delete the warehouse or switch stages, run the following snippet to restore the viewport to its original render product:

viewport.render_product_path = viewport_render_product_path

Add a Sensor Configuration Variant Set (Optional)#

Variants can be used to store multiple sensor configurations in a single USD asset. The following snippet creates a SensorConfig variant set on the sensor root Xform with two variants that switch between a high-resolution mode (HighResolution_60Hz, 1920 x 1080 at 60 Hz) and a high-speed mode (HighSpeed_120Hz, 1280 x 720 at 120 Hz). Each variant authors both the render product resolution and the camera tick rate, so switching the variant reconfigures the whole sensor at once.

For background on variant sets, see the OpenUSD Variant Set documentation. This SensorConfig setup is one example of a useful variant set. Developers can vary any attributes on the Camera prim and RenderProduct prim to match their product specifications.

The variant set is authored on the root Xform, not the camera, because a variant can only author opinions on the prim it lives on and that prim’s descendants. Both the camera and the render product are descendants of the root. The earlier steps left the resolution and tick rate unset, so these per-variant opinions are the only ones and take effect directly.

resolution_attr = prim_utils.create_prim_attribute(render_product_prim, name="resolution", type_name="int2")
tick_rate_attr = prim_utils.create_prim_attribute(camera_prim, name="omni:sensor:tickRate", type_name="float")
# Put variants on the root so they can configure both descendants.
# Experimental APIs cannot create variants or enter edit contexts.
variant_set = root_prim.GetVariantSets().AddVariantSet("SensorConfig")
configs = {
    "HighResolution_60Hz": (Gf.Vec2i(1920, 1080), 60.0),
    "HighSpeed_120Hz": (Gf.Vec2i(1280, 720), 120.0),
}
for variant_name, (resolution, tick_rate) in configs.items():
    variant_set.AddVariant(variant_name)
    prim_utils.set_prim_variants(root_prim, variants=[("SensorConfig", variant_name)])
    # Author values in the selected variant.
    with variant_set.GetVariantEditContext():
        resolution_attr.Set(resolution)
        tick_rate_attr.Set(tick_rate)
# Select the default variant.
prim_utils.set_prim_variants(root_prim, variants=[("SensorConfig", "HighResolution_60Hz")])

To switch between variants after running the script:

Select /CameraSensor in the Stage tree.
In the Property panel, scroll to the top of the panel. Variant sets appear as dropdowns above the prim attributes, in a section labeled Variants.
Use the SensorConfig dropdown to switch between HighResolution_60Hz and HighSpeed_120Hz.
Verify that selecting HighResolution_60Hz sets the resolution to 1920 x 1080 and the tick rate to 60, and that selecting HighSpeed_120Hz sets the resolution to 1280 x 720 and the tick rate to 120.

../_images/isim_6.1_full_tut_gui_camera_sensor_variant_set.png — The **Property** panel showing the **SensorConfig** variant set with `HighResolution_60Hz` selected.#

Validate and Save the Asset#

Save the USD#

Before saving, remove the warehouse so the asset contains only the sensor. In the Stage tree, select the warehouse root prim (the prim that was added when you loaded warehouse.usd) and delete it. Because /CameraSensor is the stage’s default prim, the remaining sensor hierarchy stays intact and becomes the asset’s entry point.

Also zero out any transforms on the /CameraSensor root Xform and the CameraRGB prim beneath it. The pose you set on the root to aim the sensor at the test environment was only for verification; resetting translation and rotation to 0 and scale to 1 in the Property panel gives the saved asset a clean origin that downstream scenes can position themselves.

Then save the stage as a USD file. From the top Menu Bar, click File > Save As, then choose a file name and location, for example /path/to/camera_sensor.usd.

Validate the Saved USD#

Use the Asset Validator window to validate the saved file with RGBSensorUsdRule:

From the Menu Bar, select Window > Asset Validator.
In the Asset Validator window, select Prims, then use the dropdown to its right to select /CameraSensor.
Click Disable All, search for RGBSensorUsdRule, and enable that rule.
Click Analyze and confirm that the Results panel reports no issues.

Note

If Asset Validator is not available in the Window menu, open Window > Extensions, search for omni.asset_validator.ui, and enable it.

RGBSensorUsdRule checks the default Xform, Camera and RenderProduct counts, OmniSensorAPI and tick rate, positive resolution, camera target, and the LdrColor RenderVar connection. It does not validate camera optics or lens-distortion calibration; verify those settings in the Property panel and rendered output.

Reference the Asset into a New Scene#

To confirm the saved asset is reusable, bring it into a fresh scene the way a downstream workflow would, then check that it reproduces the camera output you verified earlier.

Create a new stage (File > New). Add the warehouse test environment again: in the Content Browser, open Isaac Sim > Environments > Simple_Warehouse and drag warehouse.usd into the viewport.
Drag the saved camera_sensor.usd into the viewport (or right-click it and choose Add as Reference) to reference the sensor asset into the stage.
In the Stage tree, expand the referenced sensor prim and confirm it contains only the CameraRGB, RenderProduct, and RenderVar prims, with no warehouse geometry.
Select the referenced sensor root Xform and set its transform to the same example values you used earlier. This re-applies the pose you zeroed out before saving, so the camera points at the warehouse again.
Switch the viewport to look through the camera (Perspective > Cameras > CameraRGB) and confirm the view matches what you saw before saving, including the fisheye distortion if you configured it.

Troubleshooting#

Render product does not use the camera. Verify that the render product’s camera relationship points to /CameraSensor/CameraRGB and that the camera prim exists.

Render var output is missing. Verify that the render product’s orderedVars relationship targets the render var prim, and that the selected render var sourceName is supported by the current renderer and Isaac Sim version.

Lens distortion is not visible. Verify that the OmniLensDistortionOpenCvFisheyeAPI schema was applied to the camera, that the distortion attributes were authored, that the omni:lensdistortion:model attribute is set to opencvFisheye, and that the distortion values are large enough to produce a visible effect.

Variants do not update values. Verify that the variant set was authored on the /CameraSensor root Xform, since a variant can only affect the prim it is defined on and that prim’s descendants. Confirm that the resolution and tick-rate values were authored inside the variant edit context, the intended variant is selected, and the stage was saved after the variants were created. Also verify that there are no direct opinions on those attributes overriding the variant values, since direct opinions are stronger than variant opinions.

Saved asset still contains the warehouse. Delete the warehouse root prim before saving, and confirm that /CameraSensor is set as the stage’s default prim so the referenced asset contains only the sensor.

Summary#

In this tutorial, you authored a reusable camera sensor USD asset that includes a camera, a render product, a render var, the render relationships, and OpenCV fisheye lens distortion settings. If you completed the optional steps, the asset also includes the sensor tick rate and a variant set for switching between sensor configurations. You can now use this USD asset in other Isaac Sim workflows or as a starting point for building more advanced camera sensor assets.

Next Steps#

Learn more about camera modeling in Camera Sensors.
Browse the ready-made camera and depth sensor assets that ship with Isaac Sim.

Back to Gallery View

Authoring a Camera Sensor USD Asset#

Learning Objectives#

Prerequisites#

Overview#

Set Up the Camera and Test Environment#

Create the Sensor Root Xform#

Create a Camera#

Load the Test Environment#

Configure Camera Properties#

Add Lens Distortion#

Complete the Sensor Authoring#

Apply the Sensor API#

Create a Render Product Prim#

Create a Render Var Prim and Connect It#

Point the Viewport to the Authored Render Product#

Add a Sensor Configuration Variant Set (Optional)#

Validate and Save the Asset#

Save the USD#

Validate the Saved USD#

Reference the Asset into a New Scene#

Troubleshooting#

Summary#

Next Steps#

Create the Sensor Root `Xform`#