Replicator Composer Manual

Note

Replicator Composer is deprecated. Please view the documentation for Replicator YAML.

Replicator Composer is a tool for creating parameterizable offline datasets in Isaac Sim.

This page explains how to write the input parameter file.

• The Replicator Composer Parameter List provides the list of input parameters.

• The Replicator Composer Tutorial explains how to run Replicator Composer.

Sections

• Distributions - Documentation for Distribution types. Parameters set to distributions enable random datasets.

• Profiles - How to incorporate re-usable parameter sets.

• Groups - How to add objects and lights to a scene using a parameter group.

• Asset Lists - How to create an Asset List file.

• Paths - Various rules for resolving paths in Replicator Composer.

• Units - Units of parameters.

• Optimizing Dataset - How to speed up dataset generation and reduce dataset size.

Parameterization

Each parameter has one value type: number, string, bool, or tuple.

A parameter can be set to one of the following:

• A deterministic value: a primitive like 5, False, or (255, 0, 0)

• A stochastic value: a Distribution

A stochastic value is defined as a Distribution which, when sampled, outputs a primitive value. Distributions are sampled during runtime.

Note, all parameter values are written in Python syntax.

Distributions

Note, all distributions are using omni.replicator.core distributions, please see the docs there.

Uniform Distribution

Constructor(s)

Uniform(min_val: float, max_val: float)

Behavior

Returns a float between min_val and max_val.

Example(s)

obj_scale: Uniform(0.5, 1.5)
obj_color: Uniform((0, 0, 0), (1, 1, 1))

Normal Distribution

Constructor(s)

Normal(mean: float, var: float)

Behavior

Returns Gaussian (centered at mean with standard deviation std).

Example(s)

obj_color: Normal((0.5, 0.5, 0.5), (0.1, 0.1, 0.1))
obj_vel: Normal((0, 0, 0), (100, 100, 100))
camera_coord: Normal((0, 0, 0), (500, 500, 500))

Range Distribution

Note, Range Distribution is using omni.replicator.core.distributions.uniform. But for backward compatibility, we support the Range syntax.

Constructor(s)

Range(min_val: int, max_val: int)

Behavior

Returns an integer between min_val and max_val.

Example(s)

obj_count: Range(0, 10)

Choice Distribution

Constructor(s)

Choice(choices: List[Any], p(optional): List[float], with_replacements(optional): bool=True)

Behavior

Returns an element from an element list, provided by an input_list.

Elements are uniformly sampled or weighted by weights, where weights will be normalized.

If with_replacements==True, allow re-sampling the same element. If with_replacements==False, each element can only be sampled once.

Example(s)

obj_color: Choice([(1, 0, 0), (0, 1, 0), (0, 0, 1)])
obj_model: Choice(["assets/models/shapes.txt"])
obj_model: Choice(["assets/models/warehouse.txt", "assets/models/hospital.txt", "/Isaac/Props/Forklift/forklift.usd"])

Walk Distribution

Constructor(s)

Walk(input_list: List[Any], ordered=True: bool)
Walk(file: str, ordered=True: bool)

Behavior

Identical to a Choice, except elements are sampled w/o replacement until the input_list is empty. Once empty, the list is re-populated.

• If ordered is True: elements in the list are sampled by walking from start to end.

• If ordered is False: elements in the list are sampled randomly with removal.

Example(s)

obj_color: Walk("assets/colors/macbeth_chart.txt")
camera_coord: Walk("assets/coords/camera_trajectory.txt")

Profiles

In the Input Parameter file, the profiles key provides a list of parameter files that each supply a parameter set.

They can be used to create re-usable parameter sets that can be pointed to by different input parameter files.

• The first (top) parameter file will override the parameters of the second parameter file, and so on.

• The profile file parameters/profiles/default.yaml is automatically set as the lowest parameter set (which provides all default parameter values). Note, parameters/profiles/default.yaml is not suggested to be changed.

• The input parameter file is automatically set as the highest parameter set.

• Note, profiles defined within a profile parameter file will be ignored.

• Profiles can define groups to create inheritable groups (see Group Inheritance).

The following input.yaml contains three profiles:

...

profiles:
   - parameters/profiles/hawk_camera.yaml       # provides camera parameters tuned to the Hawk camera
   - parameters/profiles/obj_classes.yaml       # provides useful inheritable object parameter groups
   - parameters/profiles/mtrepte_default.yaml   # default parameter set tuned to mtrepte's preferences

Which creates the following parameter set stack:

1. input.yaml
2. hawk_camera.yaml
3. obj_classes.yaml
4. mtrepte_default.yaml
5. default.yaml

Groups

A group is a set of object and/or light parameters (parameters prefixed with obj_* or light_*). A group is nested with a unique group name. Group names cannot be prefixed with obj_ or light_.

For each scene, Replicator Composer will loop through each defined group to spawn in objects and lights.

Therefore, in order to spawn in any objects or lights, there must be at least one group with a non-zero obj_count or light_count.

For instance, the 4 groups defined in the following input.yaml.

# flying cube
group_a:
    obj_model: /Isaac/Props/Shapes/cube.usd
    obj_count: 1

# dropped warehouse objects
group_b:
    obj_model: Choice(["assets/models/warehouse.txt"])
    obj_count: Range(0, 20)
    obj_physics: True

# colorful flying lights
group_c:
    light_color: Uniform((0, 0, 0), (1, 1, 1))
    light_count: Range(0, 5)

# warm ceiling lights
group_d:
    light_temp: 3000
    light_temp_enabled: True
    light_count: 4
    light_coord_camera_relative: False
...

Group Inheritance

Similar to polymorphism in object-oriented programming, groups can be defined in terms of a hierarchical inherited group tree.

This is helpful for creating inheritable group parameter sets.

For instance, the following profile.yaml

random_objects:
    obj_model: /Isaac/Props/Shapes/cube.usd

    obj_count: 0  # prevents objects of this group from spawning

    obj_rot: Uniform((0, 0, 0), (360, 360, 360))
    obj_size: Uniform(100, 200)

    obj_reflectance: Uniform(0, 1)
    obj_metallicness: Uniform(0, 1)

and input.yaml

random_flying_shapes:
    obj_model: Choice(["assets/models/shapes.txt"])
    obj_count: Range(0, 10)

    inherit: random_objects

random_falling_warehouse_objects:
    obj_model: Choice(["assets/models/warehouse.txt"])
    obj_count: 5

    obj_physics: True

    inherit: random_objects

...

profiles:
   - parameters/profiles/profile.yaml

Creates this object group polymorphism:

Asset Lists

Parameters set to Choice distributions can point to TXT files called Asset Lists, where each line provides an element in the element list of a Choice.

These lists can contain colors, object model paths, object textures paths, camera coordinates, etc.

For instance, the following input.yaml

random_shapes:
    obj_model: Choice(["assets/models/shapes.txt"])
    obj_color: Choice(["assets/colors/macbeth_chart.txt", (1, 0, 0)])
    obj_count: Range(0, 20)

with models/shapes.txt

/Isaac/Props/Shapes/cube.usd
/Isaac/Props/Shapes/sphere.usd
/Isaac/Props/Shapes/cylinder.usd
...

with colors/macbeth_chart.txt

(0.06, 0.55, 0.87)
(0.48, 0.8, 0.87)
(0.6, 0.33, 0.62)
(0.54, 0.51, 0.84)
(0.68, 0.56, 0.42)
...

Paths

There are four ways paths in the input parameterization and command line arguments are resolved.

Nucleus Server Paths

If a path is the primitive value of a parameter satisfying *_model or *_texture or *_material, the path is resolved as nucleus_server + path.

The path must be prefixed with /.

For instance, for the following input.yaml file

forklift_texture_randomized:
  obj_model: /Isaac/Props/Forklifts/forklift.usd
  obj_texture: Choice(["assets/textures/photos.txt"])
  obj_count: 1


...

nucleus_server: localhost

With this textures/photos.txt file

/Users/mtrepte/photos/gravel.png
/Users/mtrepte/photos/basket_weave.png

The paths resolve as follows

omniverse://localhost/Isaac/Props/Forklifts/forklift.usd
omniverse://localhost/Users/mtrepte/photos/gravel.png
omniverse://localhost/Users/mtrepte/photos/basket_weave.png

Note, if nucleus fails to find the asset on nucleus server, it will treat the path as an absolute path and tries to find the asset locally.

Mounted Paths

Else if the path is prefixed by *, the path is resolved as <mount> + <path>. The <mount> is provided from the command line argument --mount.

For instance, for the following input.yaml

forklifts:
    obj_model: Choice(["*/assets/models/forklifts.txt"])
    obj_count: Range(0, 5)

With the cmd

$ ./python.sh tools/composer/src/main.py \
--input */parameters/forklift_in_warehouse.yaml \
--mount /home/mtrepte/replicator-workspace

Then paths resolve as follows

/home/mtrepte/replicator-workspace/parameters/forklift_in_warehouse.yaml
/home/mtrepte/replicator-workspace/assets/models/forklifts.txt

Absolute Paths

Else if the path is prefixed by /, the path is resolved as path (absolute path).

For instance, for the following input.yaml file

forklifts:
    obj_model: Choice("/home/mtrepte/replicator-workspace/assets/models/forklifts.txt")
    obj_count: Range(0, 5)

The paths resolve as follows

/home/mtrepte/replicator-workspace/assets/models/forklifts.txt

Relative Paths

Else the path is relative to the Replicator Composer folder in the Isaac Sim source. The path is resolved as <package_path>/tools/composer + <path>

For instance, for this input.yaml

texture_randomized_objects:
     obj_model: Choice("assets/models/office.txt")
     obj_texture: Choice("assets/textures/synthetic.txt)
     obj_count: Range(0, 5)

Then the paths resolve as follows

<package_path>/tools/composer/assets/models/office.txt
<package_path>/tools/composer/assets/textures/synthetic.txt

Units

The units of parameters that refer to physical lengths are in scene units (determined by scene_meters_per_unit, which defaults to 1, which means 1 meter contains 1 scene unit), except for focal_length [mm] and horiz/vert}_aperture [mm].

Depth maps are in scene_meters_per_unit.

All rotation is in degrees.

Optimizing Dataset

Ways to speed-up generation and reduce storage footprint.

Speed up Generation

Loading many objects and textures slows generation. Reducing obj_count and material / texture complexity can help.

Also, setting obj_physics to True slows generation and reducing physics_simulate_time can help.

Reduce Dataset Size

Setting groundtruth_stereo to False and / or groundtruth_visuals to False can help. Also, disable any output types that are not needed.