Is this page helpful?

Benchmark Services Migration#

Isaac Sim 6.0 changes benchmark recorder selection. Benchmark scripts that track any of the following should migrate to explicit recorder lists and warmup phases:

  • Startup

  • Scene loading

  • Render time

  • Physics time

  • GPU frame time

  • Workflow key performance indicators (KPIs)

Recorder Selection#

BaseIsaacBenchmark no longer accepts gpu_frametime=True. Build a recorder list from DEFAULT_RECORDERS and append "gpu_frametime" when the benchmark needs GPU frame metrics.

# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

from isaacsim.benchmark.services import DEFAULT_RECORDERS, BaseIsaacBenchmark

enable_gpu_frametime = True

recorders = list(DEFAULT_RECORDERS)
if enable_gpu_frametime:
    recorders.append("gpu_frametime")

benchmark = BaseIsaacBenchmark(
    benchmark_name="my_workflow",
    workflow_metadata={"metadata": [{"name": "scene", "data": "warehouse.usd"}]},
    recorders=recorders,
)

benchmark.set_phase("loading", start_recording_frametime=False, start_recording_runtime=True)
# Load the stage or initialize the workflow here.
benchmark.store_measurements()

benchmark.set_phase("benchmark", warmup_frames=15)
# Step or render the workflow under test here.
benchmark.store_measurements()
benchmark.stop()

Use the recorder registry for custom collectors. Import MeasurementDataRecorderRegistry from isaacsim.benchmark.services.datarecorders, not from isaacsim.benchmark.services. Do not depend on removed collector classes or private attributes in BaseIsaacBenchmark.

Warmup and Phase Validation#

6.0 benchmark services support warmup frames on benchmark phases. Add warmup when the workflow measures any of the following:

  • Steady-state rendering

  • Physics

  • Sensors

  • ROS

  • Synthetic data generation

Migration actions:

  • Call store_measurements() before starting a new phase.

  • Use set_phase(..., warmup_frames=N) for steady-state phases that should exclude renderer, shader, asset-load, ROS graph, or sensor startup costs.

  • Use start_recording_frametime=False on loading phases when loading time is the metric and frame time would add noise.

  • Validate output KPI names after changing recorder lists. Dashboards and CI parsers can silently miss renamed or removed measurements.

Removed Examples Extension#

Isaac Sim 6.0 removes isaacsim.benchmark.examples with no drop-in replacement extension. Use the maintained standalone benchmark scripts in the source tree as implementation references, then keep project benchmarks under project ownership.

Validation Checklist#

Verify the following before merging:

  • No benchmark constructs BaseIsaacBenchmark(..., gpu_frametime=True).

  • Required recorder names are passed through recorders=[...].

  • Custom recorders are registered through MeasurementDataRecorderRegistry imported from isaacsim.benchmark.services.datarecorders.

  • Each phase calls store_measurements() before the benchmark stops or a new phase starts.

  • KPI output contains every metric consumed by dashboards, release gates, or performance comparisons.

  • Delete removed benchmark example extension dependencies from app and extension manifests.