ZeroMQ Bridge

Tool Reference

This tutorial walks through an example of bridging NVIDIA Isaac Sim to an external Python process using ZeroMQ (ZMQ). It is an alternative to the ROS 2 bridge for cases where you want a lightweight, dependency-free channel to stream simulation data out of Isaac Sim and send commands back in — without running a ROS graph.

The example drives a Nova Carter robot around the Simple Warehouse: NVIDIA Isaac Sim streams the robot’s front camera to a standalone viewer window, and the viewer teleoperates the robot with the keyboard.

Note

The publish/subscribe nodes shown here are an illustrative example, not a comprehensive bridge. They cover exactly what this example needs — a camera image, a clock, the robot’s pose, and wheel-velocity commands. Treat them as a pattern you can extend with your own message types (see Extending the Bridge), rather than a complete, general-purpose transport.

Learning Objectives

In this tutorial, you learn how to:

• Run the ZMQ bridge example scene: stream the Nova Carter robot’s camera to a standalone window, teleoperate the robot with the keyboard over ZMQ, and read its live clock and world pose in the viewer.

• Understand the publish (send) and subscribe (receive) OmniGraph graphs that make up the bridge.

• Extend the bridge with a new message type.

How the ZMQ Bridge Works

All of the ZMQ functionality is packaged in a single extension, isaacsim.zmq.bridge. Enable it to run the example scene or to build your own ZMQ graph — it pulls in the ZeroMQ socket wrappers and protobuf schemas (Clock, Image, Pose, …) along with the OmniGraph publish/subscribe nodes and the ZMQCameraHelper node. See its API documentation.

Every message is a two-frame ZeroMQ multipart message, [topic, payload], where topic is a short routing string and payload is a serialized protobuf. Each publish node owns a non-blocking PUSH socket and each subscribe node owns a SUB socket, so there is one socket per node. By default the example uses two ports:

• 5561 — simulation → external process (Isaac Sim publishes; the client PULLs).

• 5557 — external process → simulation (the client PUBs; Isaac Sim SUBscribes).

The topics this example exchanges:

Topic	Direction	Payload
image	Isaac → client	Front-camera color image
clock	Isaac → client	Simulation + system time
pose	Isaac → client	Robot world pose (position + orientation)
joint_command	client → Isaac	Drive-wheel velocity targets

Note

Additional message types (for example joint_states and update_prim_attribute) also ship with isaacsim.zmq.nodes. Adding a new type is a matter of writing a .proto schema and a matching publish/subscribe node — see Extending the Bridge.

Running the Example

1. Enable the isaacsim.zmq.bridge extension from the Window > Extensions manager (search for isaacsim.zmq.bridge).

2. Open the example scene by going to the Isaac Sim Content browser and clicking Isaac Sim > Samples > ZeroMQ > carter_warehouse_zmq.usd. The viewport opens on a chase camera behind the robot.

3. Press Play.

4. Install the standalone tool’s viewer dependencies (ovui + glfw) into the build’s Python. This is a one-time step. Run these commands from the root of the build (the directory containing python.sh):

   ./python.sh -m pip install -r tools/zmq_bridge/requirements.txt
   

5. From the root of the build, launch the standalone server with Isaac Sim’s Python:

   ./python.sh tools/zmq_bridge/zmq_server.py
   

   A window opens showing the robot’s front-camera color stream. To receive and display without sending any commands, pass --subscribe_only 1:

   ./python.sh tools/zmq_bridge/zmq_server.py --subscribe_only 1
   

   Note

   The protobuf stubs are located automatically from the Isaac Sim install. If the launcher reports it could not find them, set ISAAC_ZMQ_PROTO_DIR to the directory containing image_pb2.py before launching:

   export ISAAC_ZMQ_PROTO_DIR=<isaac_sim_root>/exts/isaacsim.zmq.protos/isaacsim/zmq/protos
   

6. Click the client window to focus it, then drive:

   Input	Action
   W / S	Drive forward / back
   A / D	Turn left / right

   The status bar below the image shows the simulation clock, the robot’s world position, and its heading (yaw).

The standalone client streams the robot’s front camera and teleoperates it over ZMQ; the status bar below the image reports the simulation clock, the robot’s world position, heading, and wheel velocities.

Graph Explained

The scene contains two OmniGraph graphs. The send graph publishes data out of Isaac Sim:

• On Playback Tick: ticks the graph every simulation frame.

• Isaac Create Render Product: creates a render product from the robot’s front camera.

• ZMQ Camera Helper: attaches the color-image writer to the render product and publishes it on the image topic.

• ZMQ Publish Clock: publishes the simulation and system time on the clock topic.

• Isaac Read World Pose → ZMQ Publish Pose: reads the robot chassis world transform and publishes it on the pose topic.

Note

CUDA-IPC is not available on DGX Spark, so the ZMQ Camera Helper node’s useIpc input must not be turned on there. With it on, the client cannot open the published handle and the image area stays blank.

The send graph: On Playback Tick drives the camera render-product and helper chain along with the clock and pose publishers.

The receive graph applies commands coming back from the client:

• ZMQ Subscribe Joint Command: receives wheel-velocity targets on the joint_command topic.

• Isaac Articulation Controller: applies those velocities to the drive-wheel joints. Its robotPath input must point at the articulation root (the chassis_link prim), not the top-level robot prim.

The receive graph: ZMQ Subscribe Joint Command feeds wheel-velocity targets to the Articulation Controller.

Extending the Bridge

Extending the bridge means adding a protobuf schema and an OmniGraph node, so you need a source checkout of Isaac Sim from GitHub that you can rebuild. To stream a new message type, follow the same pattern the example nodes use:

1. Add a .proto schema under source/extensions/isaacsim.zmq.protos/proto/, then register it in the build: open source/extensions/isaacsim.zmq.protos/premake5.lua and add the path to your new file to the files { ... } list that feeds protoc (alongside clock.proto, image.proto, and the others). This is what generates the C++ and Python bindings.

2. Add a publish node (for data leaving Isaac Sim) or a subscribe node (for commands coming in) under source/extensions/isaacsim.zmq.nodes/nodes/: create an .ogn file describing the node’s inputs and outputs and a matching .cpp file implementing its compute, modeled on an existing pair such as OgnZMQPublishPose.{ogn,cpp} or OgnZMQSubscribeJointCommand.{ogn,cpp}.

3. Build the repo (./build.sh) so protoc regenerates the bindings and your new node compiles into the extension.

4. Wire the new node into the send or receive graph, and parse the new topic in your external client.

Because each node owns its own socket and filters by topic, new message types coexist on the same ports without interfering with the existing ones.

Further Learning

• Built-In Nodes for Data in and Out — built-in OmniGraph nodes for moving data into and out of NVIDIA Isaac Sim, useful reference when wiring a custom bridge graph.

• ROS 2 — the ROS 2 bridge, an alternative approach for streaming data to and from external processes.