How it works
- InputScene and collision shapes
.usda · rigid body · collider · materialThe rigid body can move. The collider tells the physics engine where surfaces touch.
- ProcessStep the simulation
gravity → contact → next poseGravity pulls the object down. Friction and bounce settings affect what happens at contact.
- OutputWatch and measure
viewport · capture.json · pose arraysLook for where the object stops or how far it slides. Code can compare the saved positions.
Visible geometry and contact geometry
A mesh describes what the renderer draws. A collider is the geometry the physics engine uses to calculate distances and contacts. A simple box collider can save work, but its position and dimensions have to match the object closely enough for the task. A rigid body has state including position, orientation, and velocity. At each step, the engine advances motion under gravity and resolves contacts when shapes meet. Friction affects sliding along the surface, while restitution affects how the body bounces after impact. Isaac Sim physics components
Where does a collider offset show up?
| Collider setting | Value in the recorded run | What to look for |
|---|---|---|
| Aligned with the visible table | aligned ≈ 0.80 m |
The probe meets the visible tabletop. |
| Collider raised by 12 cm | raised ≈ 0.92 m |
The probe appears to stop in midair. |
| Table collider disabled | disabled ≈ 0 m |
The probe passes the table and reaches the floor. |
These values are the final world Z coordinates at each probe’s bottom face. For friction, compare distance traveled before stopping. For restitution, compare peak center height after the first contact. The separate PhysX/Newton experiment runs for 240 steps at 120 Hz, or two simulated seconds. A similar final height can still accompany different horizontal positions. Shared initial conditions
First time? Get the lab files
Open a terminal on the Isaac Sim machine for GPU labs, or on your own computer for CPU-only steps. Press Enter after each line. You can copy a multi-line command block as a whole.
cd "$HOME"
git clone https://github.com/james-joobs/awesome-nvidia-physical-ai.git
cd awesome-nvidia-physical-ai
pwd
This example clones into your home folder. If you already cloned the repo, skip git clone and enter your project folder. In each new terminal, check that pwd ends in awesome-nvidia-physical-ai so commands can find the labs/ files.
Run it
Before you start · Use Isaac Sim 6.1 and keep the files in the scene folder together. See how to open a USD scene if you are new to the GUI.
1. Compare collision shapes in the GUI
- In Isaac Sim, choose File → Open and open
labs/physics/scenes/simready-live.usda. Keepsimready-before.usdaandsimready-after.usdanext to it, as the scene references both. - Choose FilmCamera from the viewport’s camera menu and press Play. Look for whether the two objects stop at the same height.
- Press Stop to reset. Open
collider-live.usdato compare an aligned, raised, and disabled collider.
2. Write the measurements to disk
To record on Linux X11, activate the Isaac Lab Python environment and enter this repository in the terminal. Check that command -v ffmpeg returns a path. Paste the following block. The --takes simready offset option selects the two scenes you just viewed.
python labs/physics/capture_gui.py labs/physics/scenes ./runs/physics "$(command -v ffmpeg)" \
--plan labs/physics/takes.json --takes simready offset --viz kit
To compare the two solvers numerically, run the PhysX and Newton scripts in their respective Python environments. Both accept an output folder, read the same scene.json, and write position arrays. SimReady validation · PhysX · Newton XPBD
Check the result
The SimReady probes ended 14 cm apart. The aligned, raised and disabled colliders left the test object's bottom at about 0.80, 0.92 and 0 m. As friction increased, travel fell from 57.0 to 25.5 to 11.4 cm. Increasing restitution raised the first rebound. Compare your capture.json and per-take physics samples with the recorded measurements.
Example run and recording
Source code
takes.json
View files on GitHub[
["simready", "simready-live", "FilmCamera", 0, 1, 7.4],
["offset", "collider-live", "FilmCamera", 0, 1, 7],
["friction", "friction-live", "FilmCamera", 0, 1, 6],
["bounce", "bounce-live", "FilmCamera", 0, 1, 7],
["solver-replay", "physics-solvers", "FilmCamera", 0, 1, 7]
]
capture_gui.py
View files on GitHub"""Play the USD takes in the Isaac Sim GUI and capture its actual window."""
import argparse
import json
import os
from importlib.metadata import version
from pathlib import Path
import subprocess
import time
import traceback
from isaaclab.app import AppLauncher
TAKES = [
("01-alarm-wide", "runaway-alarm", "FilmCamera", 0.0, 1.0, 9.0),
("02-alarm-fall", "runaway-alarm", "FallCamera", 1.90, .78, 5.5),
("03-drawer-wide", "snack-guard", "FilmCamera", 0.0, 1.0, 7.0),
("04-drawer-close", "snack-guard", "DrawerCamera", .65, .8, 5.0),
("05-desk-cleanup", "desk-cleanup", "FilmCamera", 0.0, .90, 7.0),
("06-collider-offset", "collider-offset", "FilmCamera", 0.0, 1.0, 7.0),
("07-physx-newton", "physics-solvers", "FilmCamera", 0.0, 1.0, 7.0),
("08-collider-live", "collider-live", "FilmCamera", 0.0, 1.0, 7.0),
]
def window_id():
deadline = time.monotonic() + 12
while time.monotonic() < deadline:
tree = subprocess.check_output(["xwininfo", "-root", "-tree"], text=True)
for row in tree.splitlines():
if '"Isaac Lab ' in row and 'mutter-x11-frames' not in row:
return str(int(row.split()[0], 16))
app.update()
raise RuntimeError("Could not find Isaac Sim's native Kit window")
def pump(seconds):
end = time.monotonic() + seconds
while time.monotonic() < end:
app.update()
def summarize_live(stage_name, samples):
if stage_name == "simready-live":
positions = [xyz[2] for xyz in samples[-1]["positions_m"]]
assert abs(positions[0] - .844) < .016 and abs(positions[1] - .984) < .016, positions
return {"probe_center_z_m": positions, "height_difference_m": positions[1] - positions[0]}
if stage_name == "friction-live":
initials = [-.80] * 3
travel = [samples[-1]["positions_m"][i][0] - initials[i] for i in range(3)]
assert travel[0] > travel[1] + .10 > travel[2] + .17 and travel[2] > .035, travel
return {"coefficients": [.16, .35, .75], "distances_m": travel}
peak = []
for index in range(3):
heights = [sample["positions_m"][index][2] for sample in samples]
first = next(i for i, z in enumerate(heights) if z < .945)
peak.append(max(heights[first + 2:]))
assert peak[1] > peak[0] + .09 and peak[2] > peak[1] + .18, peak
return {"coefficient_of_restitution": [0, .55, .85], "peak_center_z_m_after_initial_contact": peak}
def capture(name, stage_name, camera, begin, speed, duration, window, context, timeline, args):
from omni.kit.viewport.utility import get_active_viewport
desired = str(args.scenes.resolve() / f"{stage_name}.usda")
stage = context.get_stage()
if stage and stage.GetRootLayer().realPath == desired and stage_name.endswith("-live"):
timeline.stop()
timeline.commit()
context.close_stage()
app.update()
stage = None
if not stage or stage.GetRootLayer().realPath != desired:
assert context.open_stage(desired), desired
camera_path = f"/World/{camera}"
deadline = time.monotonic() + 15
while time.monotonic() < deadline:
app.update()
stage = context.get_stage()
if stage and stage.GetRootLayer().realPath == desired and stage.GetPrimAtPath(camera_path):
break
else:
raise RuntimeError(f"Camera {camera_path} never loaded from {desired}")
get_active_viewport().camera_path = camera_path
timeline.pause()
timeline.set_current_time(begin)
timeline.commit()
pump(1.30)
last = stage.GetEndTimeCode() / stage.GetTimeCodesPerSecond()
live = stage_name.endswith("-live")
started_physics = False
live_samples = []
inspector = None
if live:
from omni.physx import get_physx_interface
inspector = get_physx_interface()
print(f"TAKE_START {name} camera={camera_path} duration={duration:.2f}s", flush=True)
output = args.output / f"{name}.mp4"
with (args.output / f"{name}.capture.log").open("w") as log:
recorder = subprocess.Popen([
str(args.ffmpeg), "-nostdin", "-hide_banner", "-loglevel", "warning", "-y",
"-f", "x11grab", "-draw_mouse", "0", "-framerate", "30",
"-window_id", window, "-i", f"{os.environ['DISPLAY']}.0", "-t", str(duration),
"-vf", "scale=trunc(iw/2)*2:trunc(ih/2)*2", "-c:v", "libx264", "-preset", "ultrafast",
"-crf", "17", "-pix_fmt", "yuv420p", "-movflags", "+faststart", str(output),
], stdout=log, stderr=subprocess.STDOUT, start_new_session=True)
started = time.monotonic()
updates = 0
while recorder.poll() is None:
wall = time.monotonic() - started
if wall > duration + 12:
recorder.terminate()
recorder.wait(timeout=8)
raise RuntimeError(f"Screen recorder timed out during {name}")
if live:
if not started_physics and wall >= .72:
timeline.set_looping(False)
timeline.play()
timeline.commit()
started_physics = True
else:
offset = max(0, wall - .48)
timeline.set_current_time(min(last, begin + offset * speed))
timeline.commit()
app.update()
updates += 1
if inspector and started_physics and stage_name != "collider-live" and updates % 2 == 0:
count = 2 if stage_name == "simready-live" else 3
positions = [inspector.get_rigidbody_transformation(f"/World/Trial_{i}/Body") for i in range(count)]
if all(value["ret_val"] for value in positions):
live_samples.append({"time_s": timeline.get_current_time(),
"positions_m": [list(value["position"]) for value in positions]})
assert recorder.returncode == 0 and output.stat().st_size > 50_000, name
actual = timeline.get_current_time()
live_results = None
if stage_name == "collider-live":
live_results = {}
for case, expected in (("aligned", .80), ("raised", .92), ("disabled", 0)):
pose = inspector.get_rigidbody_transformation(f"/World/ColliderCase_{case}/Part")
assert pose["ret_val"], case
bottom = float(pose["position"][2]) - .06
assert abs(bottom - expected) < .02, (case, bottom, expected)
live_results[case] = bottom
elif live:
assert len(live_samples) > 30
live_results = summarize_live(stage_name, live_samples)
(args.output / f"{name}.physics.json").write_text(json.dumps({
"scene": stage_name, "summary": live_results, "samples": live_samples,
}, indent=2) + "\n")
if live:
timeline.pause()
timeline.commit()
print(f"LIVE_PHYSX {live_results}", flush=True)
print(f"TAKE_DONE {name} timeline_end={actual:.3f} updates={updates}", flush=True)
return {"file": output.name, "scene": stage_name, "camera": camera_path,
"playback_speed": speed, "playback_start_s": begin, "playback_end_s": actual, "app_updates": updates,
"live_physx_final_bottom_m": live_results if stage_name == "collider-live" else None,
"live_physx": live_results if stage_name != "collider-live" else None}
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("scenes", type=Path)
parser.add_argument("output", type=Path)
parser.add_argument("ffmpeg", type=Path)
parser.add_argument("--takes", nargs="+")
parser.add_argument("--plan", type=Path, help="JSON take list for a different set of scenes")
AppLauncher.add_app_launcher_args(parser)
args = parser.parse_args()
args.enable_cameras = True
args.output.mkdir(parents=True, exist_ok=True)
launcher = AppLauncher(args)
app = launcher.app
failed = False
try:
import carb
import omni.timeline
import omni.usd
settings = carb.settings.get_settings()
settings.set_bool("/rtx/hydra/readTransformsFromFabricInRenderDelegate", False)
settings.set_bool("/persistent/app/viewport/Viewport/Viewport0/scene/cameras/visible", False)
context = omni.usd.get_context()
timeline = omni.timeline.get_timeline_interface()
window = window_id()
plan = json.loads(args.plan.read_text()) if args.plan else TAKES
selected = [take for take in plan if not args.takes or take[0] in args.takes]
assert selected and (not args.takes or len(selected) == len(set(args.takes)))
metadata = []
for take in selected:
metadata.append(capture(*take, window, context, timeline, args))
(args.output / "capture.json").write_text(json.dumps({
"application": "Isaac Sim GUI (launched by Isaac Lab)", "isaacsim_version": version("isaacsim"),
"recording": "X11 capture of the native application window", "takes": metadata,
}, indent=2) + "\n")
except Exception:
failed = True
traceback.print_exc()
finally:
if failed:
# SimulationApp's fast shutdown can otherwise hide an exception's exit code.
(args.output / "capture-failed").touch()
app.close()
scene.json
View files on GitHub{
"gravity": 9.81,
"step_hz": 120,
"frames": 240,
"box_half_extent": 0.086,
"box_density": 450.0,
"friction": 0.55,
"table_pos": [0.0, 0.0, 0.78],
"table_half_extents": [0.60, 0.30, 0.02],
"floor_pos": [0.0, 0.0, -0.025],
"floor_half_extents": [4.0, 4.0, 0.025],
"boxes": [
[-0.13, 0.0, 0.886], [-0.077, 0.0, 1.063],
[-0.021, -0.004, 1.240], [0.039, 0.0, 1.417],
[0.096, 0.005, 1.594], [0.149, 0.01, 1.771]
]
}
run_physx.py
View files on GitHubimport argparse
import json
from pathlib import Path
from isaacsim import SimulationApp
parser = argparse.ArgumentParser(description="Run the shared box-stack scene using PhysX.")
parser.add_argument("output", type=Path)
output = parser.parse_args().output
output.mkdir(parents=True, exist_ok=True)
app = SimulationApp({"headless": True, "fast_shutdown": True})
import carb
import numpy as np
from omni.physx import get_physx_interface, get_physx_simulation_interface
from pxr import Gf, Sdf, Usd, UsdGeom, UsdPhysics, UsdShade, UsdUtils
spec = json.loads(Path(__file__).with_name("scene.json").read_text())
carb.settings.get_settings().set_bool("/physics/updateToUsd", True)
stage = Usd.Stage.CreateInMemory()
UsdGeom.SetStageMetersPerUnit(stage, 1.0)
UsdGeom.SetStageUpAxis(stage, UsdGeom.Tokens.z)
UsdGeom.Xform.Define(stage, "/World")
scene = UsdPhysics.Scene.Define(stage, "/World/Physics")
scene.CreateGravityMagnitudeAttr(spec["gravity"])
scene.CreateGravityDirectionAttr(Gf.Vec3f(0, 0, -1))
material = UsdShade.Material.Define(stage, "/World/Contact")
material_api = UsdPhysics.MaterialAPI.Apply(material.GetPrim())
material_api.CreateStaticFrictionAttr(spec["friction"])
material_api.CreateDynamicFrictionAttr(spec["friction"])
material_api.CreateRestitutionAttr(0.0)
def box(name, pos, half_extents, dynamic=False):
path = f"/World/{name}"
body = UsdGeom.Xform.Define(stage, path)
body.AddTranslateOp().Set(Gf.Vec3d(*pos))
collider = UsdGeom.Cube.Define(stage, f"{path}/Shape")
collider.CreateSizeAttr(2.0)
collider.AddScaleOp().Set(Gf.Vec3f(*half_extents))
UsdPhysics.CollisionAPI.Apply(collider.GetPrim())
UsdShade.MaterialBindingAPI.Apply(collider.GetPrim()).Bind(material, materialPurpose="physics")
if dynamic:
UsdPhysics.RigidBodyAPI.Apply(body.GetPrim())
UsdPhysics.MassAPI.Apply(body.GetPrim()).CreateDensityAttr(spec["box_density"])
return path
for part in ("table", "floor"):
box(part, spec[f"{part}_pos"], spec[f"{part}_half_extents"])
half = spec["box_half_extent"]
paths = [box(f"Box{i}", pos, [half] * 3, True) for i, pos in enumerate(spec["boxes"])]
cache = UsdUtils.StageCache.Get()
stage_id = cache.Insert(stage).ToLongInt()
direct = get_physx_simulation_interface()
query = get_physx_interface()
direct.attach_stage(stage_id)
try:
poses = [[[float(x) for x in pos] + [0.0, 0.0, 0.0, 1.0] for pos in spec["boxes"]]]
for frame in range(spec["frames"]):
dt = 1 / spec["step_hz"]
direct.simulate(dt, frame * dt)
direct.fetch_results()
current = []
for path in paths:
pose = query.get_rigidbody_transformation(path)
if not pose["ret_val"]:
raise RuntimeError(f"PhysX returned no pose for {path}")
current.append(list(pose["position"]) + list(pose["rotation"]))
poses.append(current)
if frame in (0, 119, 239):
print(f"PHYSX | step {frame + 1} | top-body z = {current[-1][2]:.4f} m", flush=True)
finally:
direct.detach_stage()
cache.Erase(stage)
poses = np.array(poses, dtype=np.float32)
assert np.isfinite(poses).all()
assert np.max(np.abs(poses[-1, :, :3] - poses[0, :, :3])) > 0.1
np.save(output / "physx-poses.npy", poses)
print(f"PHYSX_DONE Isaac Sim 6.2 | {poses.shape}", flush=True)
app.close()
run_newton.py
View files on GitHubimport argparse
import json
from pathlib import Path
import newton
import numpy as np
import warp as wp
root = Path(__file__).parent
parser = argparse.ArgumentParser(description="Run the shared box-stack scene using Newton XPBD.")
parser.add_argument("output", type=Path)
output = parser.parse_args().output
output.mkdir(parents=True, exist_ok=True)
spec = json.loads((root / "scene.json").read_text())
builder = newton.ModelBuilder(gravity=(0.0, 0.0, -spec["gravity"]))
shape = newton.ModelBuilder.ShapeConfig(density=spec["box_density"], mu=spec["friction"], restitution=0.0)
def body_transform(pos):
return wp.transform(wp.vec3(*pos), wp.quat_identity())
for part in ("table", "floor"):
x, y, z = spec[f"{part}_half_extents"]
builder.add_shape_box(-1, xform=body_transform(spec[f"{part}_pos"]), hx=x, hy=y, hz=z, cfg=shape)
half = spec["box_half_extent"]
for pos in spec["boxes"]:
body = builder.add_body(xform=body_transform(pos))
builder.add_shape_box(body, hx=half, hy=half, hz=half, cfg=shape)
model = builder.finalize(device="cuda:0")
solver = newton.solvers.SolverXPBD(model, iterations=8)
pipeline = newton.CollisionPipeline(model)
contacts = pipeline.contacts()
state = model.state()
next_state = model.state()
control = model.control()
newton.eval_fk(model, model.joint_q, model.joint_qd, state)
poses = [state.body_q.numpy().copy()]
for frame in range(spec["frames"]):
state.clear_forces()
pipeline.collide(state, contacts)
solver.step(state, next_state, control, contacts, 1 / spec["step_hz"])
state, next_state = next_state, state
poses.append(state.body_q.numpy())
if frame in (0, 119, 239):
print(f"NEWTON XPBD | step {frame + 1} | top-body z = {poses[-1][-1][2]:.4f} m", flush=True)
poses = np.asarray(poses)
assert np.isfinite(poses).all()
assert np.max(np.abs(poses[0, :, :3] - np.asarray(spec["boxes"]))) < 1e-5
assert np.max(np.abs(poses[-1, :, :3] - poses[0, :, :3])) > 0.1
np.save(output / "newton-poses.npy", poses)
print(f"NEWTON_DONE {newton.__version__} | {poses.shape} | {model.device}", flush=True)
