RTX Sensor Materials

There are 21 sensor materials, and more can not be added at this time. Their properties are stored in json files by the same name, located in the ./data/material_files/ folder.

Index	Sensor Material Type
0	Default
1	AsphaltStandard
2	AsphaltWeathered
3	VegetationGrass
4	WaterStandard
5	GlassStandard
6	FiberGlassStandard
7	MetalAlloy
8	MetalAluminum
9	MetalAluminumOxidized
10	PlasticStandard
11	RetroMarkings
12	RetroSign
13	RubberStandard
14	SoilClay
15	ConcreteRough
16	ConcreteSmooth
17	OakTreeBark
18	FabricStandard
19	PlexiGlassStandard
20	MetalSilver
31	INVALID

How to use sensor materials

To use sensor materials, Isaac Sim must know how to map indices (seen in the Material ID return) to the sensor material type in the table above. This is done by setting the following carb settings in ./exts/omni.isaac.sensor/config/extension.toml [settings] section.

rtx.materialDb.rtSensorNameToIdMap="DefaultMaterial:0;AsphaltStandardMaterial:1;AsphaltWeatheredMaterial:2;VegetationGrassMaterial:3;WaterStandardMaterial:4;GlassStandardMaterial:5;FiberGlassStandardMaterial:6;MetalAlloyMaterial:7;MetalAluminumMaterial:8;MetalAluminumOxidizedMaterial:9;PlasticStandardMaterial:10;RetroMarkingsMaterial:11;RetroSignMaterial:12;RubberStandardMaterial:13;SoilClayMaterial:14;ConcreteRoughMaterial:15;ConcreteSmoothMaterial:16;OakTreeBarkMaterial:17;FabricStandardMaterial:18;PlexiGlassStandardMaterial:19;MetalSilverMaterial:20"

With the rtx.materialDb.rtSensorNameToIdMap set, you now need to edit a csv file to map exact material name tokens to sensor material types. This csv file is located at ./kit/rendering-data/RtxSensorMaterialMap.csv.

The RtxSensorMaterialMap.csv file contains a material prim partial names to sensor material type pairs. The ones that come with Isaac Sim by default can be deleted as they may clash with names you wish to set. There is only one csv file. It controls the material mapping for all of the content. It is read at Isaac Sim startup and any changes to it require a restart to show up.

As an example, consider this scene:

The /Root/SM_floor29/SM_floor02/SM_floor02 prim, has a material prim assigned to it whose path is /Root/SM_floor29/Looks/MI_Floor_02b. If you want to add an entry to the csv file so that the SM_floor02 prim looks like rough concrete to the rtx sensors, you would add the entry:

mi_floor_02b,ConcreteRoughMaterial

Note that in the csv mapping file, the first token after the first appearance of /Looks/ in the material prim name attached to the mesh is used, and it must always be lowercase in the csv file, no matter what the case is on the stage. Also note how the word Material is concatenated onto the sensor material type from the table above.

Debugging

The carb parameter:

[settings]
rtx.materialDb.rtSensorMaterialLogs=true

can help. If set to true it will output a list of all the materials in the scene that are NOT mapped to a sensor material. This list outputs to the terminal and the log at Isaac Sim startup.