Replicator generates labeled synthetic datasets from simulated scenes. The application defines what changes between captures, which cameras render the scene, which labels to compute and how to write the records.

How does a camera view become RGB and label files?

How the data moves
  1. InputConfigure the viewcamera prim + resolution

    A render product represents the image to sample.

  2. ProcessRead channelsannotators → RGB · depth · label IDs

    Each annotator returns a different view of the same frame.

  3. OutputSave matching outputswriter or Python → frame files

    Keep the frame index and label lookup together.

Render products, annotators, and writers

The advanced lab pairs /World/SensorCamera with a 960 × 540 resolution to create a render product. RGB, camera-plane depth, and semantic annotators read different information from that same frame. You can delegate persistence to a writer or, as in this lab, save the results with Python as PNG, NumPy, and JSON files.

Match scene state to each output

In folder 03/, rgb.png stores color, depth_m.npy stores per-pixel depth in meters, and semantic_ids.npy stores class IDs. Use semantic_labels.json to identify which IDs belong to the cube. When varying the camera, light, or object pose, retain the configuration alongside the frame index. It gives you a way to investigate why the visible area changed. Advanced 12-frame lab · Official Replicator examples

Architecture

Component Responsibility and data boundary
USD stage and semantic labels Objects, materials and cameras live on the stage. Semantic labels map objects to the classes used by annotators. Physics can settle scattered or dropped assets before capture.
Randomizers and orchestrator The tutorial mixes direct pose/scatter updates with event-triggered graph randomizers. Orchestrator stepping controls when the renderer captures the modified scene.
Render product and annotator A render product connects a camera to the capture pipeline. Annotators obtain data such as RGB, boxes or segmentation from render products and scene metadata.
Writer The writer attaches to render products and serializes selected streams in the chosen format. Multiple cameras can produce synchronized views for each capture step.

Guide diagram based on the official references below.

Mermaid
flowchart LR
    stage["USD scene and semantic labels"]
    rand["Pose, light and material randomizers"]
    render["Camera render products"]
    anno["RGB and annotation streams"]
    writer["Dataset writer"]
    qa["Visual overlays and dataset checks"]
    rand --> stage
    stage --> render
    render --> anno
    stage --> anno
    anno --> writer
    writer --> qa

Official figures and examples

The scene-based tutorial shows the warehouse capture workflow and its rendered data.

The scene-based tutorial shows the warehouse capture workflow and its rendered data. Original figure and context

The palletizing example shows data captured for the randomized UR10 scene.

The palletizing example shows data captured for the randomized UR10 scene. Original figure and context

Walkthrough

  1. Follow the scene-based warehouse tutorial with a forklift, pallet, boxes and several cameras. Assign semantic classes and place objects within task-relevant camera bounds.
  2. Seed the randomizer and vary pose, lighting and materials. Advance physics for newly placed objects, trigger a capture, then write RGB and box labels.
  3. Overlay labels on a small sample before generating a larger dataset. Split by scene or configuration to reduce correlation between training and evaluation.

Validation

  • Check missing labels, fully occluded targets, camera coverage and writer output format. The warehouse example’s KITTI annotations may require conversion for the chosen trainer.
  • Record random seeds and capture timing. Compare a synthetic validation set and real images when measuring a detector’s deployment behavior.

License and scope

Replicator is distributed as an Omniverse/Isaac Sim component. Check extension, renderer, asset and output terms for the exact distribution.

Isaac Sim · SimReady · CDF Simulation · TAO Toolkit

Official references