Choose and evaluate a robot foundation model

Match the checkpoint to your robot or benchmark, then verify one rollout.

Start with the robot and task you can evaluate. A checkpoint for a specific camera setup, gripper, or benchmark gives you a reproducible baseline before you collect data or fine-tune a generalist model.

What you have Start with First result to verify
SO-100 or SO-101 demonstrations SmolVLA in LeRobot or MolmoAct 2 with the matching embodiment An action chunk with the expected joint order and gripper convention
A LIBERO simulation benchmark π₀.₅-LIBERO or Cosmos Policy LIBERO A rollout on the benchmark task used by that checkpoint
A DROID observation sample GR00T N1.7, π₀.₅-DROID, or Cosmos 3 Edge Policy DROID A correctly decoded trajectory for the sample's observation schema
An Isaac Lab manipulation task Isaac Lab-Arena with a supported policy interface Success rate across fixed scene variants and repeatable resets

From observations to evaluated actions

How the data moves
  1. InputObserve and specify the taskcamera views + task + robot state

    Match these inputs to the checkpoint and robot.

  2. ProcessPredict commandsVLA → action sequence

    Translate each action to the controller’s joint order, units, and coordinates.

  3. OutputScore the episodetask success + event log

    Check what actually happened after the robot applied the actions.

  1. Match the inputs. Check camera names and order, image preprocessing, task text, robot state fields, and normalization against the checkpoint and its example.
  2. Decode actions before closing the loop. Check units, coordinate frame, absolute or relative targets, joint order, and gripper convention. Record how many actions a prediction returns and how many the controller executes before requesting the next observation.
  3. Run the official example and keep a baseline. Record checkpoint revision, embodiment, action rate, inference latency, success criteria, and reset conditions. Then vary one task condition, such as object pose or lighting, while keeping the evaluation procedure fixed.
  4. Compare access and terms. Read the license for the code and the exact model card for the checkpoint, along with its backbone and data terms when applicable. A public research page, a hosted API, and downloadable weights offer different ways to reproduce a result.

Can a different robot act on the same instruction?

A VLA typically takes camera observations, an instruction, and any robot state the checkpoint expects. An action chunk covers several future steps. An illustrative [8, 7] array contains seven values per step for eight steps. The configuration defines the order, units, normalization, and whether each command is an absolute target or a delta.

GR00T N1.6-DROID follows the DROID interface. The π family also uses checkpoints and observation adapters matched to the task. Two arrays with the same shape can command different motions under different conventions. Check the observation keys and shapes on one saved sample, then verify task success in simulation. GR00T data configuration · Physical Intelligence openpi

Follow a GR00T N1.7 simulation workflow

The public, Apache-2.0 reference workflow records Isaac Lab demonstrations, converts them to LeRobot data, and fine-tunes a policy. G1 walks to an apple and puts it on a plate. H2 Plus passes the apple between hands before placing it. The G1 model and dataset require Hugging Face access. The H2 Plus artifacts are downloadable, and the workflow README reports that learned-policy evaluation for H2 Plus still awaits validation. Official workflow · G1 model card · H2 Plus model card

Cosmos Policy

The original Cosmos Policy post-trains Cosmos-Predict2 for visuomotor control. Its ALOHA planning path uses future observations and value estimates to compare candidate actions. The LIBERO checkpoint uses the NVIDIA One-Way Noncommercial License. The source code has its own Apache-2.0 license.

Cosmos 3 Edge Policy DROID and Nano Policy DROID are newer DROID action-policy checkpoints under OpenMDW-1.1. The Cosmos 3 interactive simulation checkpoints generate observations from actions or simulation controls. For example, Sim-Bimanual takes robot or camera actions and predicts video that a policy can consume as its next observation.

Other VLAs

  • OpenVLA provides a 7B manipulation baseline with inference and fine-tuning code. The model card lists MIT and the repository also identifies inherited Llama 2 terms. Model card · Upstream terms
  • MolmoAct 2 includes base and embodiment-specific checkpoints and supports depth-aware manipulation. Use the checkpoint paired with your robot or evaluation environment. Model card
  • X-VLA conditions on embodiment-specific soft prompts. Verify the supported observations and action mode in the implementation you run. Base model card · LeRobot integration

World action models

A video-action model uses features learned from video to predict how a scene may change and to produce robot commands. In the published FLUX-mimic design, a task caption and recent camera frames condition the video backbone. The action expert reads intermediate video features and the robot state to predict a trajectory. The training objective covers video and actions. A single denoising step can provide the features for control without rendering a full video. Industrial manipulation report and architecture

Name What is available as of November 17, 2026
FLUX 3 Black Forest Labs’ multimodal backbone for image, video, and audio. It underpins FLUX-mimic-x, FLUX-mimic, and FLUX 3 Action. FLUX 3 × mimic announcement
FLUX-mimic-x mimic robotics previewed a 7B model based on FLUX 3 and its updated mimic-video recipe on November 16 UTC. The team reports first place among real-time “System 1” models on RoboLab. The public leaderboard snapshot did not list the exact model under that name when checked. Public code, weights, and terms for this preview remain unverified. mimic preview · NVIDIA RoboLab leaderboard
FLUX-mimic Earlier industrial research model from mimic robotics and Black Forest Labs. The BMW Group study documents joint video-action training and reports on four tasks. Public code and weights for this FLUX-based model remain unverified. Study · Project and demos
FLUX 3 Action Black Forest Labs’ separate 7B release. Its source is Apache-2.0. The SO-101 and DROID weights require Hugging Face access and follow the FLUX Kommunity License v1.0. BFL credits NVIDIA for Jetson integration and notes that Isaac Lab and RoboLab underpin much of its evaluation. Release and technical report · Code · SO-101 · DROID
mimic-video Earlier research implementation with Apache-2.0 code built on NVIDIA Cosmos Predict2. The repository documents how to train an action decoder from intermediate features of a frozen video backbone. Check terms separately for each linked checkpoint. Paper · Code

NVIDIA policies and comparable releases

  • Cosmos3-Edge-Policy-DROID (4B) and Cosmos3-Nano-Policy-DROID (16B) are NVIDIA checkpoints for DROID manipulation under OpenMDW-1.1. The official cookbook covers serving a policy and connecting the RoboLab client. Edge · Nano · Policy-serving cookbook
  • GR00T 2 / Turbo is NVIDIA’s Cosmos-based WAM family announced November 9. The research page still lists December 2026 for code availability. Downloadable weights and their terms remain unverified. Research and demos · GR00T guide
  • DreamZero-DROID is NVIDIA GEAR’s video-and-action research baseline. Reference code uses Apache-2.0 and the downloadable 14B checkpoint is labeled CC BY-NC 4.0. Project · Code · Checkpoint and terms
  • Praxis-1 is Runway’s 27.3B WAM. It predicts video and actions jointly. Code and base, DROID, and YAM weights use Apache-2.0, with an access step for Hugging Face weights. Its November 11 release has a LeRobot integration and reports tested targets including RTX 5090 and Jetson Thor. Release · Checkpoint collection

Read an evaluation like a rollout

Camera frames + task text + robot state → policy → action chunk → simulator step → next observation. For example, in a pick-and-place task, the policy receives camera observations and the current pose, then predicts a series of motions toward the object and the container. The evaluation counts where the object ends up after the simulator executes commands and returns observations. The policy’s predicted video is a separate output.

The November 17 RoboLab snapshot shows 555/1,200 successful episodes for Praxis-1, 515/1,200 for FLUX 3 Action, 441/1,200 for Cosmos3-Nano-Policy, and 275/1,200 for Cosmos3-Edge-Policy. Use the policy revision, instruction setting, episode budget, and hardware when comparing runs. These are published simulation results. RoboLab-120 leaderboard · Policy client interfaces · Try scoring a recorded GR00T trial

For new RoboLab evaluations, v0.3.3 adds seeded object poses and lighting presets. Record the benchmark version, seed, and lighting with the score so another run can use the same conditions. v0.3.3 release notes

Recent world action research

Selected additions checked November 21, 2026. Availability is scoped to papers, code, weights, and datasets individually. These additions are source reviews, not execution-validated labs in this repository.

Research What it changes in the pipeline What you can inspect or reuse
Grana (2026-11-18) Multi-view RGB and task/subtask descriptions → local progress → composed whole-task progress. Regressions can reduce the reward while completed subtasks retain credit; supports RL and MPC scoring Paper (CC BY-SA 4.0) · Project · Repository (Apache-2.0 original code). Backbones and third-party components have separate terms; trained Grana weights and training-data distribution unconfirmed
FLORA (2026-10-22) Demonstration video → TempSeg intervals → Captioner timestamped instructions; Rewarder supplies outcome/process labels. These annotate data for curation, learning, and evaluation rather than controlling a robot Paper (CC BY 4.0) · Project · Repository · TempSeg · Captioner · Rewarder are public. Inference code: NVIDIA License §3.3; all three weight cards declare NVIDIA One-Way Noncommercial terms. Non-commercial research/evaluation only; no training code
WorldTrace (2026-11-11 revision) Past-frame KV cache → retained observations and compressed summaries → reconstructed views on scene revisits. Virtual positions keep memory addressable without retraining the video generator Paper (CC BY 4.0) · Official project and demos. Official-page code/weight distribution links and artifact licenses unconfirmed
LiDARVerse (2026-10-13) Maps/observed LiDAR/accumulated geometry → shared latents → LiDAR sweeps for generation, forecasting, or resimulation. Released PAI checkpoints are range-only and do not cover every sensor setting in the paper Paper (arXiv distribution terms) · Repository (Apache-2.0) · Public weights. Weight card lists OpenMDW-1.1 in metadata but NVIDIA Open Model License in its body; terms need clarification. Example data requires gated access and separate terms
USDCraft / USDCraft-10K (2026-10-08; dataset checked 2026-11-21) Text/images or mesh-plus-photo → LLM-authored asset program → render/geometry checks → articulated simulation asset. The 10K dataset provides already-generated USD assets to inspect before running generation Paper (CC BY 4.0) · Official project · Repository (Apache-2.0) · Dataset (Apache-2.0 declaration). External models, APIs, and input-image terms remain separate
BirdDreams: Joint Perspective and BEV Video Generation for a Plausible, Reactive Driving World Model Autonomous driving and scene reconstruction Public preprint · CC BY 4.0 · Project page · Code availability unverified · Weight availability unverified · Official source
PGX: Physically Grounded Execution for Long-Horizon Humanoid Loco-Manipulation Robot learning and control Public preprint · arXiv distribution terms · Code availability unverified · Weight availability unverified · Official source
Vision-Language-Action Models: A Life-Cycle Survey of Design, Training and Deployment Vision-language-action policies Public preprint · arXiv distribution terms · Code availability unverified · Weight availability unverified · Official source
PhyDEX: A Benchmark for Physical Reasoning in Vision-Language-Action Models via Dynamic Extrinsic Dexterity Vision-language-action policies Public preprint · CC BY 4.0 · Code availability unverified · Weight availability unverified · Official source
Composable Driving Style for End-to-End Autonomous Driving Autonomous driving and scene reconstruction Public preprint · CC BY 4.0 · Code availability unverified · Weight availability unverified · Official source
ThinkWAM (November 16) Proposes a chunk of actions, uses it to update predicted future latents, and feeds those latents into a second pass that refines the actions before execution. Public original code under MIT, with a bundled SIGReg component under CC BY-NC 4.0. The downloadable checkpoints use Apache-2.0 and have a dedicated evaluation profile. Paper · Code and third-party terms · Checkpoint card
VLALagBench (November 16 revision) Injects inference delays while moving-target tasks continue in simulation. It compares synchronous and asynchronous execution in LIBERO and includes an Isaac Sim cross-check. Public benchmark code and results, with CC BY-NC 4.0 for original code, data, and docs. Model checkpoints and simulator assets have their own terms. Paper · Repository and license
Ego2Sim2Humanoid (November 14) Converts egocentric RGB-D recordings into synchronized simulated robot views and executable whole-body trajectories. The paper reports fine-tuning GR00T N1.7 for Unitree G1 locomotion and manipulation with the converted data. Paper and evaluation figures. Code, dataset, and tuned checkpoint availability unverified. Paper
FOVI-Dex (November 12) NVIDIA and collaborators concentrate camera detail around the gaze target while keeping peripheral context. This makes large batches of rendered observations cheaper for dexterous grasping reinforcement learning. NVIDIA publication and evaluation videos are public. Code, weights, and their terms remain unverified. NVIDIA Research · Project
SWAM (November 11 revision) Uses language–video supervision where only video is available and adds action supervision for human or robot demonstrations that contain usable actions. The two branches exchange features through attention. Public Apache-2.0 code and a downloadable pretraining subset with source-specific terms. A downloadable policy checkpoint and its terms remain unverified. Paper · Code · Dataset terms
WISE (November 2) Samples candidate video-action rollouts from a frozen WAM and scores action validity, consistency with imagined motion, and task progress. With four candidates, the paper reports a gain from 6% to 24% in its Cosmos 3 Edge RoboLab subset and about 3.4× episode runtime. Preprint and evaluation protocol. Code availability unverified. Paper
DemoStream (CoRL 2026 workshop) NVIDIA and collaborators use batched task-and-motion optimization to produce successful simulated demonstrations for policy learning. Research page and paper. Code and demonstration dataset availability unverified. NVIDIA Research · Paper
FlashRT-PAI (October 2026) Optimizes end-to-end inference with fused operations and runtime scheduling. The external project covers GR00T N1.7 and Cosmos3-Edge in addition to other policies. Apache-2.0 code. Verify the supported model, GPU, precision, and runtime before comparing latency. Paper · FlashRT repository
ARC (October 8) An NVIDIA coauthored study adds action-grounded reasoning labels to DROID demonstrations and uses them to adapt π₀.₅ and Cosmos3-Nano-Policy. The project lists code, model, dataset, and RoboLab-Reasoning-50 as forthcoming. Project and status · Paper

GR00T model family · Physical Intelligence π · LeRobot · Isaac Lab · Alpamayo driving VLA

Model comparison

Scroll the table sideways to see checkpoint terms and official sources.

Additional Cosmos 3 checkpoints and research entries below were checked on November 21, 2026. Public weights, source-code licenses, and dataset rights are separate checks.

Robot foundation models (RFMs) are pretrained across tasks or robots and adapted to particular embodiments. Vision-language-action models (VLAs) turn camera observations and instructions, often with robot state, into actions. A world action model can predict future observations jointly with actions. Start with your robot or benchmark, then check the available checkpoint and its terms. Model selection and evaluation

The world action research archive covers demonstration generation, policy post-training, rollout selection, and inference on the robot. FLUX and NVIDIA WAM availability was checked on November 17, 2026.

Model Developer and role Checkpoint availability and terms Official sources
Cosmos 3 Super (64B; base / FP8) NVIDIA · Larger base video/world-model checkpoints Downloadable · OpenMDW-1.1 (base) · FP8 terms Base · FP8
Cosmos3-Nano-Sim-Auto NVIDIA · Causal driving sensor simulation from scene controls, generating camera video and LiDAR observations for closed-loop workflows Downloadable · OpenMDW-1.1 Model card · Guide
Cosmos3-Nano-Transfer-Auto NVIDIA · Batch SDG from structured scene controls and text; varies lighting/weather while generating up to 11 synchronized camera views and LiDAR Downloadable · OpenMDW-1.1 Model card · Guide
Cosmos3-Nano-Sim-Bimanual NVIDIA · Initial image plus bimanual actions/camera trajectories → future video; an observation simulator rather than an action policy Downloadable · OpenMDW-1.1 Model card · Guide
nvidia/Cosmos3-Edge-FP8 Physical AI and world models Downloadable · OpenMDW-1.1 Official source
nvidia/Cosmos3-Edge Physical AI and world models Downloadable · OpenMDW-1.1 Official source
Cosmos 3 Nano (16B; base / FP8) NVIDIA · Base video/world-model checkpoints used to derive simulation and data-generation variants Downloadable · OpenMDW-1.1 (base) · FP8 terms Base · FP8
GR00T N1.7 NVIDIA · Humanoid and robot-arm VLA Downloadable 3B checkpoint · NVIDIA Open Model License Model card · Guide
GR00T N1.7 E2E G1 NVIDIA · simulated G1 policy for locomotion and apple pick-and-place Hugging Face access required · NVIDIA Open Model License Model card · Dataset
GR00T N1.7 E2E H2 Plus NVIDIA · simulated H2 Plus policy for bimanual apple handover and placement Downloadable · NVIDIA Open Model License Model card · Dataset
GR00T 2 / Turbo NVIDIA · Cosmos-based world action model (WAM) for manipulation Announced November 9 · code planned for December 2026 · public checkpoints and weight terms unverified Research page · Guide
GR00T-H N1.7 NVIDIA · Healthcare robotics research VLA Downloadable · NVIDIA Open Model License · model card excludes clinical use Model card · Guide
π₀ / π₀-FAST / π₀.₅ Physical Intelligence · Generalist robot VLAs for local inference and fine-tuning Downloadable through openpi · check checkpoint and backbone terms · openpi code is Apache-2.0 Official checkpoints · Guide
π₀.₆ / π*₀.₆ / π₀.₇ Physical Intelligence · Learning from experience and steering robot behavior Research reports · downloadable weights unverified · hosted Pi API beta has a waitlist and unspecified model IDs π*₀.₆ · π₀.₇ · Pi API
Cosmos 3 Policy DROID NVIDIA · Robot action policies for DROID manipulation · Edge 4B and Nano 16B Downloadable · OpenMDW-1.1 Edge card · Nano card
Cosmos Policy (Predict2) NVIDIA · Video-world-model-derived robot policy and optional planning Downloadable · NVIDIA One-Way Noncommercial License for the listed checkpoint LIBERO model card and terms · Collection
SmolVLA Hugging Face · Compact LeRobot VLA for setups such as SO-100 and SO-101 Downloadable base checkpoint · Apache-2.0 Model card · LeRobot guide
OpenVLA OpenVLA research team · 7B manipulation VLA and fine-tuning baseline Downloadable · model card lists MIT · maintainers also specify inherited Llama 2 Community License terms Model card · Upstream terms
MolmoAct 2 Ai2 · Action reasoning with depth-aware manipulation Downloadable base and task checkpoints · Apache-2.0 per Ai2 Model card · Project
X-VLA 2toINF · Soft-prompted VLA for multiple robot embodiments Downloadable base checkpoint · Apache-2.0 on the model card Model card · LeRobot guide
Praxis-1 Runway · World action model that predicts video and robot actions jointly Released November 11 · Apache-2.0 weights · Hugging Face access required Base · DROID · YAM · Announcement
FLUX 3 Action Black Forest Labs · 7B world action model with SO-101 and DROID variants Hugging Face access required · FLUX Kommunity License v1.0 SO-101 · DROID · Project
FLUX-mimic-x mimic robotics · 7B real-time video-action preview on the FLUX 3 backbone from Black Forest Labs Announced November 16 UTC · public code, checkpoints, and terms unverified Developer preview · Guide
FLUX-mimic mimic robotics and Black Forest Labs · industrial manipulation video-action research Research reports and demos · public code, checkpoints, and terms unverified Industrial study · Project
DreamZero-DROID NVIDIA GEAR · 14B checkpoint for joint prediction of video and robot actions Downloadable · CC BY-NC 4.0 per model card Research project · Model card
ThinkWAM ThinkWAM authors · inference checkpoints for LIBERO, RoboTwin 2.0, and RoboCasa24 Downloadable · Apache-2.0 weights Model card · Research archive
Gemini Robotics 2.5 Google DeepMind · VLA for dexterous, long-horizon manipulation Announced November 10 · partner rollout · public checkpoints and weight terms unverified Official announcement

For vehicle trajectories and driving reasoning, see the Alpamayo guide and AlpaSim evaluation walkthrough.