Derive sensor readings from physics state#
Turn state reported by the physics engine into two kinds of sensor readings. This tutorial derives inertial measurement unit (IMU) output from a rigid body’s pose and velocities, then reports position, velocity, and measured effort for every degree of freedom (DOF) in an articulation.
Learning objectives#
After completing this tutorial, you can:
Read rigid-body and articulation state through
isaacsim.physics.entities.Derive orientation, angular velocity, and specific force in a mounted sensor frame.
Distinguish measured joint effort from commanded actuation.
Stamp readings with the simulated clock maintained by
PhysicsManager.Keep sensor calculations independent of a stage and physics engine so unit tests can use synthetic states.
Prerequisites#
Complete Install and run.
Run the commands in this tutorial from the repository root.
Allow the Joint State Sensor example to resolve the SimpleArticulation asset, or set
ISAACSIM_ASSET_ROOTto another asset root.
Examples used#
Example |
Role in this tutorial |
|---|---|
Derives IMU readings from rigid-body state over time. |
|
Reads articulation position, velocity, and effort. |
Each example page lists its source directory, requirements, and runner commands.
Separate state acquisition from sensor interpretation#
Both examples use the same boundary:
PhysicsManageradvances OvPhysX by a fixed time step.A physics entity reads measured state from the active simulation.
A sensor class converts that state into a reading without accessing the stage or engine.
The reading uses simulation time rather than a count of completed steps.
The separation makes the calculations testable. A unit test can supply recorded or synthetic body state without starting a simulation, while the runnable example verifies that the same code consumes live physics state.
Derive an IMU reading#
Read measured body state#
RigidBodyEntity exposes the body pose and world-frame linear and angular velocities after each physics step:
def _read_body_state(body: RigidBodyEntity) -> BodyState:
"""Read the world-frame state of a single rigid body.
Args:
body: Rigid body entity wrapping exactly one prim.
Returns:
The body state, orientation included unreordered.
"""
# The entity hands back the engine's xyzw ordering, which is the sensor's, so nothing moves.
_, orientations = body.get_world_poses()
linear_velocities, angular_velocities = body.get_velocities()
return BodyState(
orientation=orientations[0].list(),
linear_velocity=linear_velocities[0].list(),
angular_velocity=angular_velocities[0].list(),
)
This is the state the physics engine already computed. The sensor derives a value from it; it does not make a second measurement of the simulated body.
Transform the sample to the sensor frame#
For each sample, rotate the mount offset into the world and add the transport term \(\boldsymbol{\omega} \mathbin{\times} \mathbf{r}\). An offset point on a rotating body has this additional linear velocity:
def sample_count(self) -> int:
"""Number of samples currently buffered."""
return len(self._samples)
def reset(self) -> None:
"""Discard every buffered sample.
A caller resets the sensor when the body state jumps and a stale sample would contaminate the finite difference.
"""
self._samples.clear()
def sample(self, time: float, state: BodyState) -> None:
"""Derive one sample from a body state and buffer it.
The sample may have any timestamp because `read()` skips differences over a zero or negative interval.
Args:
time: Simulation time of the state, in seconds. Must increase for the difference to
mean anything.
state: World-frame state of the body the sensor is mounted on.
"""
body_orientation = wp.normalize(state.orientation)
# The mount offset in the world, which is the lever arm the transport term needs.
lever_arm = wp.quat_rotate(body_orientation, self._mount_translation)
# wp.mul applies the right operand first, so this is the body rotation after the mount.
orientation = wp.normalize(wp.mul(body_orientation, self._mount_orientation))
# An offset point picks up the transport term w x r; angular velocity is the same everywhere.
world_linear_velocity = state.linear_velocity + wp.cross(state.angular_velocity, lever_arm)
self._samples.append(
_Sample(
time=time,
orientation=orientation,
linear_velocity_world=world_linear_velocity,
angular_velocity=wp.quat_rotate_inv(orientation, state.angular_velocity),
)
)
The sensor buffers two windows of state. It averages orientation and angular velocity, differences world-frame velocity across the windows, subtracts gravity to obtain specific force, and only then rotates the result into the sensor frame:
def read(self) -> ImuReading | None:
"""Produce a filtered reading from the buffered samples.
Returns:
The reading, or ``None`` until ``2 * filter_width`` samples have been buffered.
"""
if len(self._samples) < 2 * self._filter_width:
return None
# Newest first, so an index and that index one window later are a difference apart.
samples = list(reversed(self._samples))
newest = samples[0]
window = samples[: self._filter_width]
angular_velocity = sum((entry.angular_velocity for entry in window), wp.vec3d(ZERO_VECTOR))
angular_velocity = angular_velocity * wp.float64(1.0 / len(window))
# Difference the world-frame velocity, never the sensor-frame one: differentiating in the
# rotating frame subtracts the transport term back out.
linear_acceleration = wp.vec3d(ZERO_VECTOR)
usable = 0
for index in range(self._filter_width):
newer, older = samples[index], samples[index + self._filter_width]
interval = newer.time - older.time
if interval > 0.0:
difference = newer.linear_velocity_world - older.linear_velocity_world
linear_acceleration = linear_acceleration + difference * wp.float64(1.0 / interval)
usable += 1
# Average over the usable differences only, or a repeated timestamp biases this toward zero.
if usable > 0:
linear_acceleration = linear_acceleration * wp.float64(1.0 / usable)
# An accelerometer measures specific force, a - g, still in the world frame; only the
# reported result is rotated into the sensor frame.
linear_acceleration = linear_acceleration - self._gravity
return ImuReading(
time=newest.time,
# Newest first, so the newest sample sets the sign convention.
orientation=average_quaternions([entry.orientation for entry in window]),
angular_velocity=angular_velocity,
linear_acceleration=wp.quat_rotate_inv(newest.orientation, linear_acceleration),
)
During free fall, specific force is near zero even though coordinate acceleration is gravity. At rest on the ground, the accelerometer reports approximately one gravity along sensor Z. This is the quantity carried by an IMU linear acceleration field.
Sample at simulation time#
Read state after each step and stamp it with the time that PhysicsManager advanced to:
for step in range(STEP_COUNT + 1):
manager.step()
# The buffers hold the state after the step, so the manager's clock stamps it.
sensor.sample(manager.get_simulated_time(), _read_body_state(body))
if step == FREE_FALL_STEP:
free_fall_reading = sensor.read()
The first reading is unavailable until the sensor has buffered 2 * filter_width samples. Reset the buffer after
teleporting the body or changing time discontinuously so an old state does not contaminate the finite difference.
Derive a joint-state reading#
Open and identify the articulation#
The Joint State Sensor example opens the SimpleArticulation asset as its simulation stage. The asset contains one revolute DOF and one prismatic DOF, along with its physics scene:
ovstage_stage = Stage("ovstage").open_stage(ROBOT_USD, make_default=False)
# Everything goes through the manager. The module-level `initialize`, `simulate` and
# `close` do the same job but are on their way out of the API.
manager = PhysicsManager.get_instance()
if not manager.switch_physics_engine(ENGINE):
raise RuntimeError("OvPhysX physics engine is unavailable.")
# The step size is configured once here rather than passed to every step.
manager.setup(dt=TIME_STEP)
if not manager.initialize(ovstage_stage.get_stage_ptr(), ovstage_stage.get_stage_id()):
raise RuntimeError("Physics initialization failed.")
robot = ArticulationEntity(ENGINE, ROBOT_PATH)
A joint can have more than one free axis, so physics APIs and readings count DOFs rather than joint prims. This asset has one DOF per joint, making the two counts equal only for this example.
Label each DOF#
The articulation reports DOF names in the same order as its state arrays. SimpleArticulation includes its joint type in each name, so this example can classify the axes without a separate lookup:
def _dof_type_from_name(name: str) -> DofType:
"""Read a DOF's kind off its joint name, which only this robot's naming makes possible.
Args:
name: DOF name, as the articulation reports it.
Returns:
The DOF's kind.
Raises:
ValueError: If the name matches neither joint kind.
"""
for keyword, dof_type in (("Revolute", DofType.ROTATION), ("Prismatic", DofType.TRANSLATION)):
if keyword in name:
return dof_type
raise ValueError(f"DOF {name!r} names neither a revolute nor a prismatic joint.")
This naming shortcut is asset-specific. For another robot, obtain the rotational or translational type from that asset’s metadata rather than inferring it from arbitrary joint names.
Read measured state and effort#
Read one position, velocity, and projected joint force for each DOF in articulation order:
def _read_articulation_state(robot: ArticulationEntity) -> ArticulationState:
"""Read the raw DOF state of a single articulation.
Args:
robot: Articulation entity wrapping exactly one prim.
Returns:
The state, in stage units and articulation order, exactly as the engine reports it.
"""
return ArticulationState(
positions=robot.get_dof_positions()[0],
velocities=robot.get_dof_velocities()[0],
# Not `get_dof_efforts()`, the actuation force a drive commands. The shipped sensor's
# `efforts` is the measured joint force on the DOF axis, which the Newton backend does not
# report at all: a caller there passes `efforts=None`.
efforts=robot.get_dof_projected_joint_forces()[0],
)
get_dof_efforts() returns actuation requested by a drive. get_dof_projected_joint_forces() returns the force
that the joint constraint carries along each DOF axis. The latter is the measured effort reported by this sensor.
The sensor follows the library convention and substitutes zeros when a backend does not expose measured joint forces;
callers must track backend support separately.
Drive and observe the joints#
The asset starts with position drives. Set stiffness to zero, retain damping, and apply velocity targets before reading the moving and stopped states:
robot.set_dof_gains(
stiffnesses=wp.array([[0.0] * len(dofs)], dtype=wp.float32, device="cpu"),
dampings=wp.array([[DRIVE_DAMPING] * len(dofs)], dtype=wp.float32, device="cpu"),
)
robot.set_dof_velocity_targets(wp.array([targets], dtype=wp.float32, device="cpu"))
Rotational position, velocity, and effort use radians, radians per second, and newton-meters. Translational values use stage distance units, stage distance units per second, and newtons. The example stage uses meters, so its reported linear values are already in SI units.
Run the examples#
Run the falling-body IMU workflow:
source/examples/example.sh run physics_sensors.imu_sensor
.\source\examples\example.bat run physics_sensors.imu_sensor
The output compares one free-fall reading with one resting reading.
Run the articulation workflow:
source/examples/example.sh run physics_sensors.joint_state_sensor
.\source\examples\example.bat run physics_sensors.joint_state_sensor
The output compares the moving articulation with the state after its slider reaches the joint limit.
Verify the workflows#
Run the tests for both examples:
source/examples/example.sh test physics_sensors.imu_sensor
source/examples/example.sh test physics_sensors.joint_state_sensor
.\source\examples\example.bat test physics_sensors.imu_sensor
.\source\examples\example.bat test physics_sensors.joint_state_sensor
The runner executes each simulation and its unit tests. The unit tests exercise validation and sensor calculations with synthetic states, including quaternion sign changes, transport acceleration, missing efforts, and misaligned DOF arrays. The simulation runs check the reported free-fall, resting, moving, and stopped readings.
Tradeoffs#
Physics-derived readings provide deterministic ground truth, but they do not automatically model hardware bias, noise, quantization, latency, or saturation. Add those effects after the ideal state conversion when your task needs sensor realism.
The IMU’s finite-difference calculation amplifies noisy velocities, while a
wider filter reduces noise at the cost of lag. Change
INITIAL_ANGULAR_VELOCITY to rotate around X or Y so the offset mount
produces a nonzero transport term. For the joint-state workflow, increase
SPIN_RATE and observe how the slider’s measured constraint force changes.
Rerun the corresponding tests after each change.
Next steps#
Use the same acquisition boundary when validating the measured motion in Control differential-drive and holonomic-drive robots or Generate manipulator motion from trajectories and targets.