fVDB Reality Capture

Reconstruct a scene from calibrated photos, render another viewpoint, and compare the available exports.

Reconstruct a 3D Gaussian scene from photos and camera poses, render a new viewpoint, and choose an export for the next step.

How do captured photos produce a new viewpoint?

How the data moves
  1. InputLoad a calibrated captureCOLMAP images + camera poses + sparse points

    Each image is associated with a viewpoint in the scene.

  2. ProcessFit a Gaussian scenerender → compare with source → update

    Optimize splat parameters against the source images.

  3. OutputRender and exportreconstructed_model.ply · USD · triangle mesh

    Choose an export based on the downstream use.

From the capture to Gaussians

The official safety_park example uses a COLMAP capture. Each image has camera intrinsics and a pose, and the capture includes sparse 3D points found by matching features across images. fVDB Reality Capture loads the capture into an SfmScene. It initializes Gaussians at the sparse points and repeatedly adjusts their position, size, rotation, opacity, and appearance to make the rendered images at known poses match the source photos.

Read the input and output in the official run

Stage Recorded safety_park values What they mean
Load COLMAP output 124 images and 199,702 sparse points The points are matched 3D locations across views.
Clean and resize 124 images and 168,661 points The log shows 000079.jpg going from 3,840 × 2,160 to 960 × 540. The pipeline also filters points.
Initialize → optimize 168,661 initial Gaussians → 1,552,032 on the final progress line The optimizer can add or remove Gaussians. Their count describes the radiance field.

Sparse points projected into camera images in the official safety_park tutorial

The official run demonstrates loading and reconstruction. To measure novel-view quality, hold out camera views during fitting and compare renders against those photos. Official tutorial, source data, and run log

Choose an export for the next step

File Content Use
reconstructed_model.ply or Gaussian USD Splat positions, scales, orientations, and appearance Render a novel view or preview the scene.
reconstructed_mesh.ply Vertices and faces describing a triangle surface Inspect or edit surfaces, and use the geometry as a starting point for a collider.

The two PLY exports store different attributes. For simulated contact, check the mesh scale and surface before authoring a collision shape. OpenUSD · SimReady · NuRec for driving sensor reconstruction

Official sources and license

fVDB Reality Capture repository · Apache-2.0 code license. Check the terms of the sample data and dependencies separately.