Tutorial 15: Validate System Identification with Sim-to-Sim Recovery

Tutorial

This tutorial runs a complete known-truth recovery experiment using the double-pendulum sample assets under IsaacSim/Samples/SystemIdentification/. Those assets supply the USDA, seed trajectory, mapping, RunSpec, and reference reports; the extension itself is unchanged. The existing sim-to-sim tool generates synthetic telemetry from deliberately perturbed parameters, runs the production preflight and optimizer from nominal values, and scores the recovered parameters against the injected truth.

Known-truth recovery is stronger than checking trajectory RMSE alone. It fails when a parameter is mapped to the wrong joint, applied with the wrong scale, or hidden by parameter coupling even when the fitted trajectory looks plausible.

For the extension workflow and backend reference, see System Identification Extension.

About 10 minutes after first-time shader and Newton-kernel compilation

What this test proves

The runnable command performs three phases in one headless NVIDIA Isaac Sim process:

1. Generate: import the supplied USDA, apply a deterministic truth vector, replay two multisine chunks, add seeded sensor noise, and write canonical CSV telemetry.

2. Identify: start from nominal parameter scales and run the normal RunSpec preflight, preparation, and differentiable Newton solve.

3. Score: compare the selected result with the injected truth and write independent per-family recovery gates.

The Samples folder also preserves a separate Newton-to-PhysX reference report. That report is reviewed later as supporting evidence; reproducing its target-runtime truth rollout is intentionally outside this tutorial.

A matched-solver pass validates the configured identification plumbing. It does not prove that the simulator matches a physical robot because generation and identification share the same model family. Use recorded validation data and deployment-representative motion after this test.

Prerequisites

• A built NVIDIA Isaac Sim source checkout that contains the System Identification headless tools.

• isaacsim.robot_setup.sysid.ui enabled. Open Window > Extensions, search for system identification, and enable isaacsim.robot_setup.sysid.ui. The interactive extension automatically enables the isaacsim.robot_setup.sysid backend.

Locate the double-pendulum sample assets in the Content Browser at IsaacSim/Samples/SystemIdentification/. The folder contains:

• IsaacSim/Samples/SystemIdentification/double_pendulum.usda, a valid two-link articulation with two revolute joints;

• IsaacSim/Samples/SystemIdentification/double_pendulum_seed.csv, a deterministic 100 Hz multisine trajectory;

• IsaacSim/Samples/SystemIdentification/double_pendulum_column_map.json;

• IsaacSim/Samples/SystemIdentification/double_pendulum_base_spec.json with separate 200-sample training and validation chunks;

• IsaacSim/Samples/SystemIdentification/reference_simsim_report.json and IsaacSim/Samples/SystemIdentification/reference_physx_transfer_report.json for comparison.

Copy those files into a local working directory before you run headless commands so relative RunSpec paths resolve and writable outputs stay on disk. Do not modify the Samples originals.

The differentiable Newton rollout is a headless backend. The extension UI can prepare and export a RunSpec, but the Featherstone gradient solve in this tutorial runs from the command line. CUDA graph capture is disabled in the supplied RunSpec so the tutorial’s matched-Featherstone reference remains uncaptured. This choice does not require disabling capture for a separate Newton MuJoCo population solve.

Step 1: Inspect the example design

The example identifies four scale parameters from nominal value 1.0:

Parameter	Initial	Injected truth	Bounds
Joint 1 stiffness	1.0	0.85	0.5 to 1.5
Joint 2 damping	1.0	1.20	0.5 to 1.5
Joint 1 friction	1.0	1.40	0.5 to 1.8
Global link mass	1.0	1.08	0.8 to 1.3

The command trajectory combines 0.7 Hz, 1.9 Hz, and 3.1 Hz sinusoids with different amplitudes and signs on the two joints. This creates acceleration, velocity reversal, and coupled motion without driving the small articulation into its joint limits.

For your own mechanism, choose frequencies and amplitudes from the safe operating envelope rather than copying these numbers. The MathWorks input-signal design reference covers multisine, PRBS, and band-limited excitation. Confirm joint limits, effort limits, collision clearance, temperature, and emergency-stop behavior before applying an excitation to hardware.

Step 2: Run the supplied recovery

Copy the Samples files into a local working directory, make that directory current so the RunSpec’s relative paths resolve, then run the existing tool:

$IsaacSimRoot = "C:\isaac-sim"
Push-Location C:\sysid_tutorial

& "$IsaacSimRoot\_build\windows-x86_64\release\python.bat" `
    "$IsaacSimRoot\source\extensions\isaacsim.robot_setup.sysid\tools\headless_sysid_simsim.py" `
    --run-spec double_pendulum_base_spec.json `
    --output-dir results\local_matched_run `
    --generator-substeps 8 `
    --noise-position 0.0002 `
    --noise-velocity 0.002 `
    --noise-torque 0.005 `
    --seed 11

The --run-spec argument points at the local copy of IsaacSim/Samples/SystemIdentification/double_pendulum_base_spec.json. Keep --output-dir under the local working directory. Do not write results into the Samples folder.

The explicit arguments and supplied RunSpec reproduce the reference settings:

• matched differentiable Featherstone generation and identification;

• 8 substeps at a 0.01 s telemetry interval;

• 80 Adam iterations at learning rate 0.03;

• position, velocity, and torque noise standard deviations of 0.0002 rad, 0.002 rad/s, and 0.005 N m;

• random seed 11.

The first launch can spend most of its time loading extensions and compiling kernels. During optimization, accepted means that iteration established a new best cost. A sequence of rejected lines is not an error: Adam continues updating and can improve again after an overshoot.

Step 3: Inspect the supplied inputs

Inspect these Samples inputs before running (after copying them into your local working directory):

Artifact	Interpretation
IsaacSim/Samples/SystemIdentification/double_pendulum_base_spec.json	Complete RunSpec pointing at the sibling USDA and seed trajectory.
IsaacSim/Samples/SystemIdentification/double_pendulum.usda	Self-contained two-link articulation used by both generator and identifier.
IsaacSim/Samples/SystemIdentification/double_pendulum_seed.csv	Two-joint multisine commands and the initial position/velocity source used by the generator.
IsaacSim/Samples/SystemIdentification/double_pendulum_column_map.json	Canonical column mapping and explicit joint_0, joint_1 order.

These files make the example inspectable and editable. Keep the Samples originals and reference reports unchanged and write experiments to a new local output directory.

Step 4: Read the recovery artifacts

The local output directory contains:

Artifact	Interpretation
simsim_telemetry.csv	Generated position, velocity, command, and link-side torque samples.
simsim_column_map.json	Mapping used by the production ingestion path.
simsim_identify_spec.json	Derived RunSpec whose chunk bounds match the generated samples and whose recording-only command alignment is zero.
simsim_manifest.json	Truth values, generator bridge, substeps, noise, seed, and generated chunk windows.
simsim_report.json	Recovered values, absolute and relative errors, gates, final cost, and overall verdict.

The Samples file IsaacSim/Samples/SystemIdentification/reference_simsim_report.json preserves the published run independently of your local output.

The reference run produced:

Parameter	Truth	Recovered	Relative error	Gate
Joint 1 stiffness	0.85	0.849612	0.046%	Pass
Joint 2 damping	1.20	1.198056	0.162%	Pass
Joint 1 friction	1.40	1.375371	1.76%	Pass
Global link mass	1.08	1.079800	0.019%	Pass

All four parameters pass their independent recovery gates. The best weighted rollout cost falls from 16.74314 to 0.00148088 over 80 Adam iterations.

Do not replace the parameter gates with a final-cost threshold. Cost units change with weights, sample counts, and trajectory duration; known-truth error answers whether the configured parameters were actually recovered.

Step 5: Understand the diagnostic messages

The pre-solve check can label drive gains unknown and friction weak even though this matched known-truth run recovers them. The analytical identifiability check uses local torque regressors; the gradient optimizer also uses multi-step position and velocity response. Treat the two reports as complementary:

• the recovery gate is direct evidence for this injected truth and trajectory;

• the identifiability verdict warns how confidently the same parameter can be separated in a different or noisier experiment;

• a warning must not be rewritten as identifiable merely because a matched benchmark passed.

The generated response also appears delayed relative to the command because the closed-loop plant cannot move instantaneously. Do not apply the recording-oriented command-alignment suggestion to this synthetic CSV. The tool intentionally keeps command_alignment_seconds = 0 because generator and identifier already share the canonical command clock.

Step 6: Review the Newton-to-PhysX reference artifact

The Samples file IsaacSim/Samples/SystemIdentification/reference_physx_transfer_report.json records a separate experiment that created three PhysX rollouts over the 200-sample validation chunk:

1. the injected truth vector, which is the target-runtime reference;

2. the authored nominal vector;

3. the Newton-recovered vector.

Comparing the latter two against the PhysX truth rollout isolates parameter transfer from simulator trajectory parity. Newton and PhysX are not required to produce identical trajectories. They are required to interpret the transferred scale parameters consistently enough that the recovered PhysX replay moves toward the known PhysX plant.

The reference run produced:

Signal	Nominal PhysX RMSE	Transferred PhysX RMSE	Reduction
Position	0.0214477 rad	0.0000647 rad	99.70%
Velocity	0.0956012 rad/s	0.0002724 rad/s	99.72%
Torque	0.0558424 N m	0.00021895 N m	99.61%

The Newton-recovered vector reduced all three held-out PhysX errors by more than 99.6% relative to the authored nominal vector. The report sets passed = true only when every reported held-out RMSE is lower after transfer. The artifact is supplied for review and is not regenerated by the tutorial command.

This is a known-truth parameter-transfer check, not a solver-parity measurement and not evidence that the same improvement will occur on hardware. Reproducing it requires independently generating and preserving the target-runtime truth rollout. Keeping that specialized experiment outside the public tutorial avoids adding a documentation-only execution mode to the extension.

For a cross-solver identification diagnostic that uses the existing tool, generate the telemetry with Newton MuJoCo and keep differentiable Featherstone as the identifier:

& "$IsaacSimRoot\_build\windows-x86_64\release\python.bat" `
    "$IsaacSimRoot\source\extensions\isaacsim.robot_setup.sysid\tools\headless_sysid_simsim.py" `
    --run-spec double_pendulum_base_spec.json `
    --output-dir results\local_cross_solver_run `
    --generation-bridge mujoco `
    --noise-position 0.0002 `
    --noise-velocity 0.002 `
    --noise-torque 0.005 `
    --seed 11

The MuJoCo run is intentionally a separate question and is not expected to match the tight matched-Featherstone recovery errors.

Step 7: Place sim-to-sim in the acceptance workflow

Use the checks in this order:

1. Run the supplied matched-solver recovery after mapping or parameter-application changes.

2. Recreate known-truth recovery on the intended robot and parameter phase.

3. Review the preserved known-truth Newton-to-PhysX transfer report.

4. Run cross-solver identification when model-family transfer matters.

5. Fit recorded data and require improvement on held-out chunks.

6. Run reload/writeback checks and, when CUDA graph capture is used, compare captured and uncaptured results with declared numerical tolerances.

7. Validate the saved USD on deployment-representative motions and contacts.

Newton MuJoCo resets its rollout-local warm-start, actuator-activation, applied-force, and control buffers before restoring the initial joint state on both paths. Capture can therefore remain enabled for repeated, fixed-shape optimization. Small floating-point differences can remain on the GPU, so do not require bitwise equality. Disable capture when the captured-versus-uncaptured difference exceeds the acceptance tolerance for the same RunSpec.

Troubleshooting

The USDA does not open

Run once with the supplied RunSpec before editing any Samples copies. The supplied stage has no external asset dependencies. If a copied RunSpec fails, confirm that stage.input_path still points at double_pendulum.usda as a sibling of the RunSpec in your local working directory, matching IsaacSim/Samples/SystemIdentification/double_pendulum.usda.

Recovery reaches a bound

Confirm that the USD link transforms agree with the joint frames. An inconsistent initial assembly can produce saturated efforts while remaining syntactically valid USD. Then check parameter coupling, reduce the active set, or redesign the excitation before increasing iterations.

The reference PhysX transfer reports a failure

First confirm that IsaacSim/Samples/SystemIdentification/reference_physx_transfer_report.json and its checksum are unchanged and that it uses reference = "isaac_sim_physx_known_truth". Comparing a PhysX rollout directly with a Newton-generated trajectory measures solver parity and can hide an otherwise valid parameter transfer. If reproducing the experiment independently, confirm timestep, joint order, actuator ownership, effort clamps, parameter bounds, and that all transferred families are supported by both bridges. Do not loosen the recovery gates solely to make the transfer report pass.

The report says confidence_verdict: unknown

Recovery gates and post-solve confidence answer different questions. The example disables optional confidence analysis to keep the walkthrough focused and repeatable. The known-truth pass/fail verdict remains valid.

Summary

This tutorial covered:

1. Running a portable double-pendulum known-truth recovery with the existing sim-to-sim tool and Samples assets under IsaacSim/Samples/SystemIdentification/.

2. Inspecting the valid supplied USDA, RunSpec, trajectory, and mapping.

3. Interpreting independent parameter gates and optimizer convergence.

4. Separating analytical identifiability warnings from direct recovery evidence.

5. Reviewing preserved Newton-to-PhysX known-truth parameter-transfer evidence.

6. Keeping parameter transfer distinct from solver parity.

7. Using cross-solver, recorded-data, parity, writeback, and task checks as a layered acceptance workflow.

Back to Gallery View