Start with the robot and task you can evaluate. A checkpoint for a specific camera setup, gripper, or benchmark gives you a reproducible baseline before you collect data or fine-tune a generalist model.
| What you have | Start with | First result to verify |
|---|---|---|
| SO-100 or SO-101 demonstrations | SmolVLA in LeRobot or MolmoAct 2 with the matching embodiment | An action chunk with the expected joint order and gripper convention |
| A LIBERO simulation benchmark | π₀.₅-LIBERO or Cosmos Policy LIBERO | A rollout on the benchmark task used by that checkpoint |
| A DROID observation sample | GR00T N1.7, π₀.₅-DROID, or Cosmos 3 Edge Policy DROID | A correctly decoded trajectory for the sample's observation schema |
| An Isaac Lab manipulation task | Isaac Lab-Arena with a supported policy interface | Success rate across fixed scene variants and repeatable resets |
From observations to evaluated actions
- InputObserve and specify the task
camera views + task + robot stateMatch these inputs to the checkpoint and robot.
- ProcessPredict commands
VLA → action sequenceTranslate each action to the controller’s joint order, units, and coordinates.
- OutputScore the episode
task success + event logCheck what actually happened after the robot applied the actions.
- Match the inputs. Check camera names and order, image preprocessing, task text, robot state fields, and normalization against the checkpoint and its example.
- Decode actions before closing the loop. Check units, coordinate frame, absolute or relative targets, joint order, and gripper convention. Record how many actions a prediction returns and how many the controller executes before requesting the next observation.
- Run the official example and keep a baseline. Record checkpoint revision, embodiment, action rate, inference latency, success criteria, and reset conditions. Then vary one task condition, such as object pose or lighting, while keeping the evaluation procedure fixed.
- Compare access and terms. Read the license for the code and the exact model card for the checkpoint, along with its backbone and data terms when applicable. A public research page, a hosted API, and downloadable weights offer different ways to reproduce a result.
Can a different robot act on the same instruction?
A VLA typically takes camera observations, an instruction, and any robot state the checkpoint expects. An action chunk covers several future steps. An illustrative [8, 7] array contains seven values per step for eight steps. The configuration defines the order, units, normalization, and whether each command is an absolute target or a delta.
GR00T N1.6-DROID follows the DROID interface. The π family also uses checkpoints and observation adapters matched to the task. Two arrays with the same shape can command different motions under different conventions. Check the observation keys and shapes on one saved sample, then verify task success in simulation. GR00T data configuration · Physical Intelligence openpi
Follow a GR00T N1.7 simulation workflow
The public, Apache-2.0 reference workflow records Isaac Lab demonstrations, converts them to LeRobot data, and fine-tunes a policy. G1 walks to an apple and puts it on a plate. H2 Plus passes the apple between hands before placing it. The G1 model and dataset require Hugging Face access. The H2 Plus artifacts are downloadable, and the workflow README reports that learned-policy evaluation for H2 Plus still awaits validation. Official workflow · G1 model card · H2 Plus model card
Cosmos Policy
The original Cosmos Policy post-trains Cosmos-Predict2 for visuomotor control. Its ALOHA planning path uses future observations and value estimates to compare candidate actions. The LIBERO checkpoint uses the NVIDIA One-Way Noncommercial License. The source code has its own Apache-2.0 license.
Cosmos 3 Edge Policy DROID and Nano Policy DROID are newer DROID action-policy checkpoints under OpenMDW-1.1. The Cosmos 3 interactive simulation checkpoints generate observations from actions or simulation controls. For example, Sim-Bimanual takes robot or camera actions and predicts video that a policy can consume as its next observation.
Other VLAs
- OpenVLA provides a 7B manipulation baseline with inference and fine-tuning code. The model card lists MIT and the repository also identifies inherited Llama 2 terms. Model card · Upstream terms
- MolmoAct 2 includes base and embodiment-specific checkpoints and supports depth-aware manipulation. Use the checkpoint paired with your robot or evaluation environment. Model card
- X-VLA conditions on embodiment-specific soft prompts. Verify the supported observations and action mode in the implementation you run. Base model card · LeRobot integration
World action models
A video-action model uses features learned from video to predict how a scene may change and to produce robot commands. In the published FLUX-mimic design, a task caption and recent camera frames condition the video backbone. The action expert reads intermediate video features and the robot state to predict a trajectory. The training objective covers video and actions. A single denoising step can provide the features for control without rendering a full video. Industrial manipulation report and architecture
| Name | What is available as of November 17, 2026 |
|---|---|
| FLUX 3 | Black Forest Labs’ multimodal backbone for image, video, and audio. It underpins FLUX-mimic-x, FLUX-mimic, and FLUX 3 Action. FLUX 3 × mimic announcement |
| FLUX-mimic-x | mimic robotics previewed a 7B model based on FLUX 3 and its updated mimic-video recipe on November 16 UTC. The team reports first place among real-time “System 1” models on RoboLab. The public leaderboard snapshot did not list the exact model under that name when checked. Public code, weights, and terms for this preview remain unverified. mimic preview · NVIDIA RoboLab leaderboard |
| FLUX-mimic | Earlier industrial research model from mimic robotics and Black Forest Labs. The BMW Group study documents joint video-action training and reports on four tasks. Public code and weights for this FLUX-based model remain unverified. Study · Project and demos |
| FLUX 3 Action | Black Forest Labs’ separate 7B release. Its source is Apache-2.0. The SO-101 and DROID weights require Hugging Face access and follow the FLUX Kommunity License v1.0. BFL credits NVIDIA for Jetson integration and notes that Isaac Lab and RoboLab underpin much of its evaluation. Release and technical report · Code · SO-101 · DROID |
| mimic-video | Earlier research implementation with Apache-2.0 code built on NVIDIA Cosmos Predict2. The repository documents how to train an action decoder from intermediate features of a frozen video backbone. Check terms separately for each linked checkpoint. Paper · Code |
NVIDIA policies and comparable releases
- Cosmos3-Edge-Policy-DROID (4B) and Cosmos3-Nano-Policy-DROID (16B) are NVIDIA checkpoints for DROID manipulation under OpenMDW-1.1. The official cookbook covers serving a policy and connecting the RoboLab client. Edge · Nano · Policy-serving cookbook
- GR00T 2 / Turbo is NVIDIA’s Cosmos-based WAM family announced November 9. The research page still lists December 2026 for code availability. Downloadable weights and their terms remain unverified. Research and demos · GR00T guide
- DreamZero-DROID is NVIDIA GEAR’s video-and-action research baseline. Reference code uses Apache-2.0 and the downloadable 14B checkpoint is labeled CC BY-NC 4.0. Project · Code · Checkpoint and terms
- Praxis-1 is Runway’s 27.3B WAM. It predicts video and actions jointly. Code and base, DROID, and YAM weights use Apache-2.0, with an access step for Hugging Face weights. Its November 11 release has a LeRobot integration and reports tested targets including RTX 5090 and Jetson Thor. Release · Checkpoint collection
Read an evaluation like a rollout
Camera frames + task text + robot state → policy → action chunk → simulator step → next observation. For example, in a pick-and-place task, the policy receives camera observations and the current pose, then predicts a series of motions toward the object and the container. The evaluation counts where the object ends up after the simulator executes commands and returns observations. The policy’s predicted video is a separate output.
The November 17 RoboLab snapshot shows 555/1,200 successful episodes for Praxis-1, 515/1,200 for FLUX 3 Action, 441/1,200 for Cosmos3-Nano-Policy, and 275/1,200 for Cosmos3-Edge-Policy. Use the policy revision, instruction setting, episode budget, and hardware when comparing runs. These are published simulation results. RoboLab-120 leaderboard · Policy client interfaces · Try scoring a recorded GR00T trial
For new RoboLab evaluations, v0.3.3 adds seeded object poses and lighting presets. Record the benchmark version, seed, and lighting with the score so another run can use the same conditions. v0.3.3 release notes
Recent world action research
Selected additions checked November 21, 2026. Availability is scoped to papers, code, weights, and datasets individually. These additions are source reviews, not execution-validated labs in this repository.
| Research | What it changes in the pipeline | What you can inspect or reuse |
|---|---|---|
| Grana (2026-11-18) | Multi-view RGB and task/subtask descriptions → local progress → composed whole-task progress. Regressions can reduce the reward while completed subtasks retain credit; supports RL and MPC scoring | Paper (CC BY-SA 4.0) · Project · Repository (Apache-2.0 original code). Backbones and third-party components have separate terms; trained Grana weights and training-data distribution unconfirmed |
| FLORA (2026-10-22) | Demonstration video → TempSeg intervals → Captioner timestamped instructions; Rewarder supplies outcome/process labels. These annotate data for curation, learning, and evaluation rather than controlling a robot | Paper (CC BY 4.0) · Project · Repository · TempSeg · Captioner · Rewarder are public. Inference code: NVIDIA License §3.3; all three weight cards declare NVIDIA One-Way Noncommercial terms. Non-commercial research/evaluation only; no training code |
| WorldTrace (2026-11-11 revision) | Past-frame KV cache → retained observations and compressed summaries → reconstructed views on scene revisits. Virtual positions keep memory addressable without retraining the video generator | Paper (CC BY 4.0) · Official project and demos. Official-page code/weight distribution links and artifact licenses unconfirmed |
| LiDARVerse (2026-10-13) | Maps/observed LiDAR/accumulated geometry → shared latents → LiDAR sweeps for generation, forecasting, or resimulation. Released PAI checkpoints are range-only and do not cover every sensor setting in the paper | Paper (arXiv distribution terms) · Repository (Apache-2.0) · Public weights. Weight card lists OpenMDW-1.1 in metadata but NVIDIA Open Model License in its body; terms need clarification. Example data requires gated access and separate terms |
| USDCraft / USDCraft-10K (2026-10-08; dataset checked 2026-11-21) | Text/images or mesh-plus-photo → LLM-authored asset program → render/geometry checks → articulated simulation asset. The 10K dataset provides already-generated USD assets to inspect before running generation | Paper (CC BY 4.0) · Official project · Repository (Apache-2.0) · Dataset (Apache-2.0 declaration). External models, APIs, and input-image terms remain separate |
| BirdDreams: Joint Perspective and BEV Video Generation for a Plausible, Reactive Driving World Model | Autonomous driving and scene reconstruction | Public preprint · CC BY 4.0 · Project page · Code availability unverified · Weight availability unverified · Official source |
| PGX: Physically Grounded Execution for Long-Horizon Humanoid Loco-Manipulation | Robot learning and control | Public preprint · arXiv distribution terms · Code availability unverified · Weight availability unverified · Official source |
| Vision-Language-Action Models: A Life-Cycle Survey of Design, Training and Deployment | Vision-language-action policies | Public preprint · arXiv distribution terms · Code availability unverified · Weight availability unverified · Official source |
| PhyDEX: A Benchmark for Physical Reasoning in Vision-Language-Action Models via Dynamic Extrinsic Dexterity | Vision-language-action policies | Public preprint · CC BY 4.0 · Code availability unverified · Weight availability unverified · Official source |
| Composable Driving Style for End-to-End Autonomous Driving | Autonomous driving and scene reconstruction | Public preprint · CC BY 4.0 · Code availability unverified · Weight availability unverified · Official source |
| ThinkWAM (November 16) | Proposes a chunk of actions, uses it to update predicted future latents, and feeds those latents into a second pass that refines the actions before execution. | Public original code under MIT, with a bundled SIGReg component under CC BY-NC 4.0. The downloadable checkpoints use Apache-2.0 and have a dedicated evaluation profile. Paper · Code and third-party terms · Checkpoint card |
| VLALagBench (November 16 revision) | Injects inference delays while moving-target tasks continue in simulation. It compares synchronous and asynchronous execution in LIBERO and includes an Isaac Sim cross-check. | Public benchmark code and results, with CC BY-NC 4.0 for original code, data, and docs. Model checkpoints and simulator assets have their own terms. Paper · Repository and license |
| Ego2Sim2Humanoid (November 14) | Converts egocentric RGB-D recordings into synchronized simulated robot views and executable whole-body trajectories. The paper reports fine-tuning GR00T N1.7 for Unitree G1 locomotion and manipulation with the converted data. | Paper and evaluation figures. Code, dataset, and tuned checkpoint availability unverified. Paper |
| FOVI-Dex (November 12) | NVIDIA and collaborators concentrate camera detail around the gaze target while keeping peripheral context. This makes large batches of rendered observations cheaper for dexterous grasping reinforcement learning. | NVIDIA publication and evaluation videos are public. Code, weights, and their terms remain unverified. NVIDIA Research · Project |
| SWAM (November 11 revision) | Uses language–video supervision where only video is available and adds action supervision for human or robot demonstrations that contain usable actions. The two branches exchange features through attention. | Public Apache-2.0 code and a downloadable pretraining subset with source-specific terms. A downloadable policy checkpoint and its terms remain unverified. Paper · Code · Dataset terms |
| WISE (November 2) | Samples candidate video-action rollouts from a frozen WAM and scores action validity, consistency with imagined motion, and task progress. With four candidates, the paper reports a gain from 6% to 24% in its Cosmos 3 Edge RoboLab subset and about 3.4× episode runtime. | Preprint and evaluation protocol. Code availability unverified. Paper |
| DemoStream (CoRL 2026 workshop) | NVIDIA and collaborators use batched task-and-motion optimization to produce successful simulated demonstrations for policy learning. | Research page and paper. Code and demonstration dataset availability unverified. NVIDIA Research · Paper |
| FlashRT-PAI (October 2026) | Optimizes end-to-end inference with fused operations and runtime scheduling. The external project covers GR00T N1.7 and Cosmos3-Edge in addition to other policies. | Apache-2.0 code. Verify the supported model, GPU, precision, and runtime before comparing latency. Paper · FlashRT repository |
| ARC (October 8) | An NVIDIA coauthored study adds action-grounded reasoning labels to DROID demonstrations and uses them to adapt π₀.₅ and Cosmos3-Nano-Policy. | The project lists code, model, dataset, and RoboLab-Reasoning-50 as forthcoming. Project and status · Paper |
GR00T model family · Physical Intelligence π · LeRobot · Isaac Lab · Alpamayo driving VLA
Model comparison
Scroll the table sideways to see checkpoint terms and official sources.
Additional Cosmos 3 checkpoints and research entries below were checked on November 21, 2026. Public weights, source-code licenses, and dataset rights are separate checks.
Robot foundation models (RFMs) are pretrained across tasks or robots and adapted to particular embodiments. Vision-language-action models (VLAs) turn camera observations and instructions, often with robot state, into actions. A world action model can predict future observations jointly with actions. Start with your robot or benchmark, then check the available checkpoint and its terms. Model selection and evaluation
The world action research archive covers demonstration generation, policy post-training, rollout selection, and inference on the robot. FLUX and NVIDIA WAM availability was checked on November 17, 2026.
| Model | Developer and role | Checkpoint availability and terms | Official sources |
|---|---|---|---|
| Cosmos 3 Super (64B; base / FP8) | NVIDIA · Larger base video/world-model checkpoints | Downloadable · OpenMDW-1.1 (base) · FP8 terms | Base · FP8 |
| Cosmos3-Nano-Sim-Auto | NVIDIA · Causal driving sensor simulation from scene controls, generating camera video and LiDAR observations for closed-loop workflows | Downloadable · OpenMDW-1.1 | Model card · Guide |
| Cosmos3-Nano-Transfer-Auto | NVIDIA · Batch SDG from structured scene controls and text; varies lighting/weather while generating up to 11 synchronized camera views and LiDAR | Downloadable · OpenMDW-1.1 | Model card · Guide |
| Cosmos3-Nano-Sim-Bimanual | NVIDIA · Initial image plus bimanual actions/camera trajectories → future video; an observation simulator rather than an action policy | Downloadable · OpenMDW-1.1 | Model card · Guide |
| nvidia/Cosmos3-Edge-FP8 | Physical AI and world models | Downloadable · OpenMDW-1.1 | Official source |
| nvidia/Cosmos3-Edge | Physical AI and world models | Downloadable · OpenMDW-1.1 | Official source |
| Cosmos 3 Nano (16B; base / FP8) | NVIDIA · Base video/world-model checkpoints used to derive simulation and data-generation variants | Downloadable · OpenMDW-1.1 (base) · FP8 terms | Base · FP8 |
| GR00T N1.7 | NVIDIA · Humanoid and robot-arm VLA | Downloadable 3B checkpoint · NVIDIA Open Model License | Model card · Guide |
| GR00T N1.7 E2E G1 | NVIDIA · simulated G1 policy for locomotion and apple pick-and-place | Hugging Face access required · NVIDIA Open Model License | Model card · Dataset |
| GR00T N1.7 E2E H2 Plus | NVIDIA · simulated H2 Plus policy for bimanual apple handover and placement | Downloadable · NVIDIA Open Model License | Model card · Dataset |
| GR00T 2 / Turbo | NVIDIA · Cosmos-based world action model (WAM) for manipulation | Announced November 9 · code planned for December 2026 · public checkpoints and weight terms unverified | Research page · Guide |
| GR00T-H N1.7 | NVIDIA · Healthcare robotics research VLA | Downloadable · NVIDIA Open Model License · model card excludes clinical use | Model card · Guide |
| π₀ / π₀-FAST / π₀.₅ | Physical Intelligence · Generalist robot VLAs for local inference and fine-tuning | Downloadable through openpi · check checkpoint and backbone terms · openpi code is Apache-2.0 | Official checkpoints · Guide |
| π₀.₆ / π*₀.₆ / π₀.₇ | Physical Intelligence · Learning from experience and steering robot behavior | Research reports · downloadable weights unverified · hosted Pi API beta has a waitlist and unspecified model IDs | π*₀.₆ · π₀.₇ · Pi API |
| Cosmos 3 Policy DROID | NVIDIA · Robot action policies for DROID manipulation · Edge 4B and Nano 16B | Downloadable · OpenMDW-1.1 | Edge card · Nano card |
| Cosmos Policy (Predict2) | NVIDIA · Video-world-model-derived robot policy and optional planning | Downloadable · NVIDIA One-Way Noncommercial License for the listed checkpoint | LIBERO model card and terms · Collection |
| SmolVLA | Hugging Face · Compact LeRobot VLA for setups such as SO-100 and SO-101 | Downloadable base checkpoint · Apache-2.0 | Model card · LeRobot guide |
| OpenVLA | OpenVLA research team · 7B manipulation VLA and fine-tuning baseline | Downloadable · model card lists MIT · maintainers also specify inherited Llama 2 Community License terms | Model card · Upstream terms |
| MolmoAct 2 | Ai2 · Action reasoning with depth-aware manipulation | Downloadable base and task checkpoints · Apache-2.0 per Ai2 | Model card · Project |
| X-VLA | 2toINF · Soft-prompted VLA for multiple robot embodiments | Downloadable base checkpoint · Apache-2.0 on the model card | Model card · LeRobot guide |
| Praxis-1 | Runway · World action model that predicts video and robot actions jointly | Released November 11 · Apache-2.0 weights · Hugging Face access required | Base · DROID · YAM · Announcement |
| FLUX 3 Action | Black Forest Labs · 7B world action model with SO-101 and DROID variants | Hugging Face access required · FLUX Kommunity License v1.0 | SO-101 · DROID · Project |
| FLUX-mimic-x | mimic robotics · 7B real-time video-action preview on the FLUX 3 backbone from Black Forest Labs | Announced November 16 UTC · public code, checkpoints, and terms unverified | Developer preview · Guide |
| FLUX-mimic | mimic robotics and Black Forest Labs · industrial manipulation video-action research | Research reports and demos · public code, checkpoints, and terms unverified | Industrial study · Project |
| DreamZero-DROID | NVIDIA GEAR · 14B checkpoint for joint prediction of video and robot actions | Downloadable · CC BY-NC 4.0 per model card | Research project · Model card |
| ThinkWAM | ThinkWAM authors · inference checkpoints for LIBERO, RoboTwin 2.0, and RoboCasa24 | Downloadable · Apache-2.0 weights | Model card · Research archive |
| Gemini Robotics 2.5 | Google DeepMind · VLA for dexterous, long-horizon manipulation | Announced November 10 · partner rollout · public checkpoints and weight terms unverified | Official announcement |
For vehicle trajectories and driving reasoning, see the Alpamayo guide and AlpaSim evaluation walkthrough.