Physics you can see

Colliders change where objects stop. Friction changes how far they slide, and restitution changes how high they bounce.

  • Isaac Sim
  • PhysX
  • OpenUSD
  • SimReady
Physics you can see
Recorded runExample run and recording

How it works

How the data moves
  1. InputScene and collision shapes.usda · rigid body · collider · material

    The rigid body can move. The collider tells the physics engine where surfaces touch.

  2. ProcessStep the simulationgravity → contact → next pose

    Gravity pulls the object down. Friction and bounce settings affect what happens at contact.

  3. OutputWatch and measureviewport · capture.json · pose arrays

    Look for where the object stops or how far it slides. Code can compare the saved positions.

Visible geometry and contact geometry

A mesh describes what the renderer draws. A collider is the geometry the physics engine uses to calculate distances and contacts. A simple box collider can save work, but its position and dimensions have to match the object closely enough for the task. A rigid body has state including position, orientation, and velocity. At each step, the engine advances motion under gravity and resolves contacts when shapes meet. Friction affects sliding along the surface, while restitution affects how the body bounces after impact. Isaac Sim physics components

Where does a collider offset show up?

Collider setting Value in the recorded run What to look for
Aligned with the visible table aligned ≈ 0.80 m The probe meets the visible tabletop.
Collider raised by 12 cm raised ≈ 0.92 m The probe appears to stop in midair.
Table collider disabled disabled ≈ 0 m The probe passes the table and reaches the floor.

These values are the final world Z coordinates at each probe’s bottom face. For friction, compare distance traveled before stopping. For restitution, compare peak center height after the first contact. The separate PhysX/Newton experiment runs for 240 steps at 120 Hz, or two simulated seconds. A similar final height can still accompany different horizontal positions. Shared initial conditions

First time? Get the lab files

Open a terminal on the Isaac Sim machine for GPU labs, or on your own computer for CPU-only steps. Press Enter after each line. You can copy a multi-line command block as a whole.

Shell
cd "$HOME"
git clone https://github.com/james-joobs/awesome-nvidia-physical-ai.git
cd awesome-nvidia-physical-ai
pwd

This example clones into your home folder. If you already cloned the repo, skip git clone and enter your project folder. In each new terminal, check that pwd ends in awesome-nvidia-physical-ai so commands can find the labs/ files.

Run it

Before you start · Use Isaac Sim 6.1 and keep the files in the scene folder together. See how to open a USD scene if you are new to the GUI.

1. Compare collision shapes in the GUI

  1. In Isaac Sim, choose File → Open and open labs/physics/scenes/simready-live.usda. Keep simready-before.usda and simready-after.usda next to it, as the scene references both.
  2. Choose FilmCamera from the viewport’s camera menu and press Play. Look for whether the two objects stop at the same height.
  3. Press Stop to reset. Open collider-live.usda to compare an aligned, raised, and disabled collider.

2. Write the measurements to disk

To record on Linux X11, activate the Isaac Lab Python environment and enter this repository in the terminal. Check that command -v ffmpeg returns a path. Paste the following block. The --takes simready offset option selects the two scenes you just viewed.

Shell
python labs/physics/capture_gui.py labs/physics/scenes ./runs/physics "$(command -v ffmpeg)" \
  --plan labs/physics/takes.json --takes simready offset --viz kit

To compare the two solvers numerically, run the PhysX and Newton scripts in their respective Python environments. Both accept an output folder, read the same scene.json, and write position arrays. SimReady validation · PhysX · Newton XPBD

Check the result

The SimReady probes ended 14 cm apart. The aligned, raised and disabled colliders left the test object's bottom at about 0.80, 0.92 and 0 m. As friction increased, travel fell from 57.0 to 25.5 to 11.4 cm. Increasing restitution raised the first rebound. Compare your capture.json and per-take physics samples with the recorded measurements.

Example run and recording

Source code

takes.jsonView files on GitHub
JSON
[
  ["simready", "simready-live", "FilmCamera", 0, 1, 7.4],
  ["offset", "collider-live", "FilmCamera", 0, 1, 7],
  ["friction", "friction-live", "FilmCamera", 0, 1, 6],
  ["bounce", "bounce-live", "FilmCamera", 0, 1, 7],
  ["solver-replay", "physics-solvers", "FilmCamera", 0, 1, 7]
]
capture_gui.pyView files on GitHub
Python
"""Play the USD takes in the Isaac Sim GUI and capture its actual window."""

import argparse
import json
import os
from importlib.metadata import version
from pathlib import Path
import subprocess
import time
import traceback

from isaaclab.app import AppLauncher

TAKES = [
    ("01-alarm-wide", "runaway-alarm", "FilmCamera", 0.0, 1.0, 9.0),
    ("02-alarm-fall", "runaway-alarm", "FallCamera", 1.90, .78, 5.5),
    ("03-drawer-wide", "snack-guard", "FilmCamera", 0.0, 1.0, 7.0),
    ("04-drawer-close", "snack-guard", "DrawerCamera", .65, .8, 5.0),
    ("05-desk-cleanup", "desk-cleanup", "FilmCamera", 0.0, .90, 7.0),
    ("06-collider-offset", "collider-offset", "FilmCamera", 0.0, 1.0, 7.0),
    ("07-physx-newton", "physics-solvers", "FilmCamera", 0.0, 1.0, 7.0),
    ("08-collider-live", "collider-live", "FilmCamera", 0.0, 1.0, 7.0),
]


def window_id():
    deadline = time.monotonic() + 12
    while time.monotonic() < deadline:
        tree = subprocess.check_output(["xwininfo", "-root", "-tree"], text=True)
        for row in tree.splitlines():
            if '"Isaac Lab ' in row and 'mutter-x11-frames' not in row:
                return str(int(row.split()[0], 16))
        app.update()
    raise RuntimeError("Could not find Isaac Sim's native Kit window")


def pump(seconds):
    end = time.monotonic() + seconds
    while time.monotonic() < end:
        app.update()


def summarize_live(stage_name, samples):
    if stage_name == "simready-live":
        positions = [xyz[2] for xyz in samples[-1]["positions_m"]]
        assert abs(positions[0] - .844) < .016 and abs(positions[1] - .984) < .016, positions
        return {"probe_center_z_m": positions, "height_difference_m": positions[1] - positions[0]}
    if stage_name == "friction-live":
        initials = [-.80] * 3
        travel = [samples[-1]["positions_m"][i][0] - initials[i] for i in range(3)]
        assert travel[0] > travel[1] + .10 > travel[2] + .17 and travel[2] > .035, travel
        return {"coefficients": [.16, .35, .75], "distances_m": travel}
    peak = []
    for index in range(3):
        heights = [sample["positions_m"][index][2] for sample in samples]
        first = next(i for i, z in enumerate(heights) if z < .945)
        peak.append(max(heights[first + 2:]))
    assert peak[1] > peak[0] + .09 and peak[2] > peak[1] + .18, peak
    return {"coefficient_of_restitution": [0, .55, .85], "peak_center_z_m_after_initial_contact": peak}


def capture(name, stage_name, camera, begin, speed, duration, window, context, timeline, args):
    from omni.kit.viewport.utility import get_active_viewport

    desired = str(args.scenes.resolve() / f"{stage_name}.usda")
    stage = context.get_stage()
    if stage and stage.GetRootLayer().realPath == desired and stage_name.endswith("-live"):
        timeline.stop()
        timeline.commit()
        context.close_stage()
        app.update()
        stage = None
    if not stage or stage.GetRootLayer().realPath != desired:
        assert context.open_stage(desired), desired
    camera_path = f"/World/{camera}"
    deadline = time.monotonic() + 15
    while time.monotonic() < deadline:
        app.update()
        stage = context.get_stage()
        if stage and stage.GetRootLayer().realPath == desired and stage.GetPrimAtPath(camera_path):
            break
    else:
        raise RuntimeError(f"Camera {camera_path} never loaded from {desired}")
    get_active_viewport().camera_path = camera_path
    timeline.pause()
    timeline.set_current_time(begin)
    timeline.commit()
    pump(1.30)
    last = stage.GetEndTimeCode() / stage.GetTimeCodesPerSecond()
    live = stage_name.endswith("-live")
    started_physics = False
    live_samples = []
    inspector = None
    if live:
        from omni.physx import get_physx_interface
        inspector = get_physx_interface()
    print(f"TAKE_START {name} camera={camera_path} duration={duration:.2f}s", flush=True)
    output = args.output / f"{name}.mp4"
    with (args.output / f"{name}.capture.log").open("w") as log:
        recorder = subprocess.Popen([
            str(args.ffmpeg), "-nostdin", "-hide_banner", "-loglevel", "warning", "-y",
            "-f", "x11grab", "-draw_mouse", "0", "-framerate", "30",
            "-window_id", window, "-i", f"{os.environ['DISPLAY']}.0", "-t", str(duration),
            "-vf", "scale=trunc(iw/2)*2:trunc(ih/2)*2", "-c:v", "libx264", "-preset", "ultrafast",
            "-crf", "17", "-pix_fmt", "yuv420p", "-movflags", "+faststart", str(output),
        ], stdout=log, stderr=subprocess.STDOUT, start_new_session=True)
        started = time.monotonic()
        updates = 0
        while recorder.poll() is None:
            wall = time.monotonic() - started
            if wall > duration + 12:
                recorder.terminate()
                recorder.wait(timeout=8)
                raise RuntimeError(f"Screen recorder timed out during {name}")
            if live:
                if not started_physics and wall >= .72:
                    timeline.set_looping(False)
                    timeline.play()
                    timeline.commit()
                    started_physics = True
            else:
                offset = max(0, wall - .48)
                timeline.set_current_time(min(last, begin + offset * speed))
                timeline.commit()
            app.update()
            updates += 1
            if inspector and started_physics and stage_name != "collider-live" and updates % 2 == 0:
                count = 2 if stage_name == "simready-live" else 3
                positions = [inspector.get_rigidbody_transformation(f"/World/Trial_{i}/Body") for i in range(count)]
                if all(value["ret_val"] for value in positions):
                    live_samples.append({"time_s": timeline.get_current_time(),
                                         "positions_m": [list(value["position"]) for value in positions]})
        assert recorder.returncode == 0 and output.stat().st_size > 50_000, name
    actual = timeline.get_current_time()
    live_results = None
    if stage_name == "collider-live":
        live_results = {}
        for case, expected in (("aligned", .80), ("raised", .92), ("disabled", 0)):
            pose = inspector.get_rigidbody_transformation(f"/World/ColliderCase_{case}/Part")
            assert pose["ret_val"], case
            bottom = float(pose["position"][2]) - .06
            assert abs(bottom - expected) < .02, (case, bottom, expected)
            live_results[case] = bottom
    elif live:
        assert len(live_samples) > 30
        live_results = summarize_live(stage_name, live_samples)
        (args.output / f"{name}.physics.json").write_text(json.dumps({
            "scene": stage_name, "summary": live_results, "samples": live_samples,
        }, indent=2) + "\n")
    if live:
        timeline.pause()
        timeline.commit()
        print(f"LIVE_PHYSX {live_results}", flush=True)
    print(f"TAKE_DONE {name} timeline_end={actual:.3f} updates={updates}", flush=True)
    return {"file": output.name, "scene": stage_name, "camera": camera_path,
            "playback_speed": speed, "playback_start_s": begin, "playback_end_s": actual, "app_updates": updates,
            "live_physx_final_bottom_m": live_results if stage_name == "collider-live" else None,
            "live_physx": live_results if stage_name != "collider-live" else None}


parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("scenes", type=Path)
parser.add_argument("output", type=Path)
parser.add_argument("ffmpeg", type=Path)
parser.add_argument("--takes", nargs="+")
parser.add_argument("--plan", type=Path, help="JSON take list for a different set of scenes")
AppLauncher.add_app_launcher_args(parser)
args = parser.parse_args()
args.enable_cameras = True
args.output.mkdir(parents=True, exist_ok=True)
launcher = AppLauncher(args)
app = launcher.app
failed = False
try:
    import carb
    import omni.timeline
    import omni.usd

    settings = carb.settings.get_settings()
    settings.set_bool("/rtx/hydra/readTransformsFromFabricInRenderDelegate", False)
    settings.set_bool("/persistent/app/viewport/Viewport/Viewport0/scene/cameras/visible", False)
    context = omni.usd.get_context()
    timeline = omni.timeline.get_timeline_interface()
    window = window_id()
    plan = json.loads(args.plan.read_text()) if args.plan else TAKES
    selected = [take for take in plan if not args.takes or take[0] in args.takes]
    assert selected and (not args.takes or len(selected) == len(set(args.takes)))
    metadata = []
    for take in selected:
        metadata.append(capture(*take, window, context, timeline, args))
    (args.output / "capture.json").write_text(json.dumps({
        "application": "Isaac Sim GUI (launched by Isaac Lab)", "isaacsim_version": version("isaacsim"),
        "recording": "X11 capture of the native application window", "takes": metadata,
    }, indent=2) + "\n")
except Exception:
    failed = True
    traceback.print_exc()
finally:
    if failed:
        # SimulationApp's fast shutdown can otherwise hide an exception's exit code.
        (args.output / "capture-failed").touch()
    app.close()
scene.jsonView files on GitHub
JSON
{
  "gravity": 9.81,
  "step_hz": 120,
  "frames": 240,
  "box_half_extent": 0.086,
  "box_density": 450.0,
  "friction": 0.55,
  "table_pos": [0.0, 0.0, 0.78],
  "table_half_extents": [0.60, 0.30, 0.02],
  "floor_pos": [0.0, 0.0, -0.025],
  "floor_half_extents": [4.0, 4.0, 0.025],
  "boxes": [
    [-0.13, 0.0, 0.886], [-0.077, 0.0, 1.063],
    [-0.021, -0.004, 1.240], [0.039, 0.0, 1.417],
    [0.096, 0.005, 1.594], [0.149, 0.01, 1.771]
  ]
}
run_physx.pyView files on GitHub
Python
import argparse
import json
from pathlib import Path

from isaacsim import SimulationApp

parser = argparse.ArgumentParser(description="Run the shared box-stack scene using PhysX.")
parser.add_argument("output", type=Path)
output = parser.parse_args().output
output.mkdir(parents=True, exist_ok=True)
app = SimulationApp({"headless": True, "fast_shutdown": True})
import carb
import numpy as np
from omni.physx import get_physx_interface, get_physx_simulation_interface
from pxr import Gf, Sdf, Usd, UsdGeom, UsdPhysics, UsdShade, UsdUtils

spec = json.loads(Path(__file__).with_name("scene.json").read_text())
carb.settings.get_settings().set_bool("/physics/updateToUsd", True)
stage = Usd.Stage.CreateInMemory()
UsdGeom.SetStageMetersPerUnit(stage, 1.0)
UsdGeom.SetStageUpAxis(stage, UsdGeom.Tokens.z)
UsdGeom.Xform.Define(stage, "/World")
scene = UsdPhysics.Scene.Define(stage, "/World/Physics")
scene.CreateGravityMagnitudeAttr(spec["gravity"])
scene.CreateGravityDirectionAttr(Gf.Vec3f(0, 0, -1))
material = UsdShade.Material.Define(stage, "/World/Contact")
material_api = UsdPhysics.MaterialAPI.Apply(material.GetPrim())
material_api.CreateStaticFrictionAttr(spec["friction"])
material_api.CreateDynamicFrictionAttr(spec["friction"])
material_api.CreateRestitutionAttr(0.0)

def box(name, pos, half_extents, dynamic=False):
    path = f"/World/{name}"
    body = UsdGeom.Xform.Define(stage, path)
    body.AddTranslateOp().Set(Gf.Vec3d(*pos))
    collider = UsdGeom.Cube.Define(stage, f"{path}/Shape")
    collider.CreateSizeAttr(2.0)
    collider.AddScaleOp().Set(Gf.Vec3f(*half_extents))
    UsdPhysics.CollisionAPI.Apply(collider.GetPrim())
    UsdShade.MaterialBindingAPI.Apply(collider.GetPrim()).Bind(material, materialPurpose="physics")
    if dynamic:
        UsdPhysics.RigidBodyAPI.Apply(body.GetPrim())
        UsdPhysics.MassAPI.Apply(body.GetPrim()).CreateDensityAttr(spec["box_density"])
    return path

for part in ("table", "floor"):
    box(part, spec[f"{part}_pos"], spec[f"{part}_half_extents"])
half = spec["box_half_extent"]
paths = [box(f"Box{i}", pos, [half] * 3, True) for i, pos in enumerate(spec["boxes"])]
cache = UsdUtils.StageCache.Get()
stage_id = cache.Insert(stage).ToLongInt()
direct = get_physx_simulation_interface()
query = get_physx_interface()
direct.attach_stage(stage_id)
try:
    poses = [[[float(x) for x in pos] + [0.0, 0.0, 0.0, 1.0] for pos in spec["boxes"]]]
    for frame in range(spec["frames"]):
        dt = 1 / spec["step_hz"]
        direct.simulate(dt, frame * dt)
        direct.fetch_results()
        current = []
        for path in paths:
            pose = query.get_rigidbody_transformation(path)
            if not pose["ret_val"]:
                raise RuntimeError(f"PhysX returned no pose for {path}")
            current.append(list(pose["position"]) + list(pose["rotation"]))
        poses.append(current)
        if frame in (0, 119, 239):
            print(f"PHYSX | step {frame + 1} | top-body z = {current[-1][2]:.4f} m", flush=True)
finally:
    direct.detach_stage()
    cache.Erase(stage)
poses = np.array(poses, dtype=np.float32)
assert np.isfinite(poses).all()
assert np.max(np.abs(poses[-1, :, :3] - poses[0, :, :3])) > 0.1
np.save(output / "physx-poses.npy", poses)
print(f"PHYSX_DONE Isaac Sim 6.2 | {poses.shape}", flush=True)
app.close()
run_newton.pyView files on GitHub
Python
import argparse
import json
from pathlib import Path

import newton
import numpy as np
import warp as wp

root = Path(__file__).parent
parser = argparse.ArgumentParser(description="Run the shared box-stack scene using Newton XPBD.")
parser.add_argument("output", type=Path)
output = parser.parse_args().output
output.mkdir(parents=True, exist_ok=True)
spec = json.loads((root / "scene.json").read_text())
builder = newton.ModelBuilder(gravity=(0.0, 0.0, -spec["gravity"]))
shape = newton.ModelBuilder.ShapeConfig(density=spec["box_density"], mu=spec["friction"], restitution=0.0)

def body_transform(pos):
    return wp.transform(wp.vec3(*pos), wp.quat_identity())

for part in ("table", "floor"):
    x, y, z = spec[f"{part}_half_extents"]
    builder.add_shape_box(-1, xform=body_transform(spec[f"{part}_pos"]), hx=x, hy=y, hz=z, cfg=shape)
half = spec["box_half_extent"]
for pos in spec["boxes"]:
    body = builder.add_body(xform=body_transform(pos))
    builder.add_shape_box(body, hx=half, hy=half, hz=half, cfg=shape)

model = builder.finalize(device="cuda:0")
solver = newton.solvers.SolverXPBD(model, iterations=8)
pipeline = newton.CollisionPipeline(model)
contacts = pipeline.contacts()
state = model.state()
next_state = model.state()
control = model.control()
newton.eval_fk(model, model.joint_q, model.joint_qd, state)
poses = [state.body_q.numpy().copy()]
for frame in range(spec["frames"]):
    state.clear_forces()
    pipeline.collide(state, contacts)
    solver.step(state, next_state, control, contacts, 1 / spec["step_hz"])
    state, next_state = next_state, state
    poses.append(state.body_q.numpy())
    if frame in (0, 119, 239):
        print(f"NEWTON XPBD | step {frame + 1} | top-body z = {poses[-1][-1][2]:.4f} m", flush=True)
poses = np.asarray(poses)
assert np.isfinite(poses).all()
assert np.max(np.abs(poses[0, :, :3] - np.asarray(spec["boxes"]))) < 1e-5
assert np.max(np.abs(poses[-1, :, :3] - poses[0, :, :3])) > 0.1
np.save(output / "newton-poses.npy", poses)
print(f"NEWTON_DONE {newton.__version__} | {poses.shape} | {model.device}", flush=True)