Alpamayo is a family of reasoning and action models for autonomous driving. The R1 research architecture is a useful concrete example. Check the model card before carrying that architecture or input contract over to another release.

Follow the multiview and ego-motion inputs into the VLM backbone, then distinguish the reasoning tokens from the trajectory decoder’s waypoints. Official figure source

This training-data diagram shows how driving clips become labeled examples linked to specific decisions. It is separate from inference. Official figure source
How do camera views and ego motion become a driving trajectory?
- InputPrepare the observation
clip_id + t0_us → cameras and ego historySelect a common instant for the visual and motion inputs.
- ProcessReason and predict motion
vision-language context → action expertThe action expert samples future positions and rotations.
- OutputInspect predictions
reasoning + 64 waypoints + minADECompare trajectory geometry with the recorded future and review the reasoning.
Read the anchor time from a real validation manifest
These two fields come from the first entry in NVIDIA’s Alpamayo 2 Super public validation manifest.
{
"clip_id": "003e17f1-7c44-4d82-9f32-d4cd82f6a8e8",
"t0_us": 2000000
}
The ID identifies a PhysicalAI-AV clip. The anchor t0_us is 2 seconds from the start of that clip. The loader aligns images and ego position and rotation to that time. The trajectory profile uses six cameras and four frames per camera. At 10 Hz, the requested image times here are 1.7, 1.8, 1.9, and 2.0 seconds. The loader also retains the timestamps returned by decoding. Pinned validation manifest · Data loader
Read input arrays and predicted outputs
| Data | Official trajectory example | Meaning |
|---|---|---|
image_frames |
[6, 4, 3, H, W] |
Selected cameras, frames, RGB channels, height, and width. |
ego_history_xyz |
[1, 1, 16, 3] |
Sixteen ego-frame XYZ positions up to the anchor. Rotation is a separate array. |
| One predicted trajectory | Positions [64, 3] and rotations [64, 3, 3] |
Waypoints from 0.1 to 6.4 seconds after the anchor. |
The vision-language backbone encodes the observations and the action expert samples a conditioned future trajectory. The official example reports reasoning text and a trajectory. For minADE, it computes mean XY error against the recorded future for each candidate, then takes the smallest mean. Follow the AlpaSim lab to inspect collisions or off-road events in closed loop. The numbers above come from NVIDIA’s manifest and documented data contract. 2 Super model card · Official inference script
Released checkpoints
| Model | What it adds | Model and inference code |
|---|---|---|
| Alpamayo 2 Edge · Nov 10, 2026 | Compact adaptive reasoning VLA for DRIVE AGX Thor. The Cosmos 3 Edge backbone takes surround-camera observations, ego-motion history, and navigation text. The model generates future trajectories and uses reasoning when needed. The card lists a 2.45B backbone and a 0.2B action expert. | Model card · GitHub |
| Alpamayo 2 Super · Aug 4, 2026 | 34B-class driving foundation model with a 32B Cosmos 3 Super Reasoner backbone and a diffusion action expert. Supports trajectory prediction, causal reasoning, visual question answering, 2D grounding, and auto-labeling. | Model card · GitHub |
| Alpamayo 1.5 · Mar 19, 2026 | 10B-class VLA based on Cosmos-Reason2 with navigation guidance, flexible camera counts, and visual question answering. | Model card · GitHub |
| Alpamayo 1 (R1) | Original 10B-class model pairing Chain-of-Causation reasoning with trajectories. The figures above show this version. | Model card · GitHub |
How it works
R1 conditions its reasoning and trajectory predictions on observations such as camera frames and ego-motion history. A vision-language backbone produces driving reasoning and a trajectory generation component produces continuous waypoints. A simulator driver can consume the resulting trajectory to advance the ego vehicle. Reasoning text and motion are separate outputs to inspect.
A practical workflow
Run an Alpamayo driver in AlpaSim on a scenario with a partly occluded obstacle or a complex merge. The upstream AlpaSim changelog documents driver=alpamayo2 for 2 Super. Edge has its own inference repository, and NVIDIA documents its closed-loop reinforcement-learning post-training in AlpaGym. Review reasoning, trajectory, and rollout metrics against a baseline on the same scene and route. AlpaSim changelog · Edge technical overview
What to verify
Check the checkpoint’s accepted cameras, context and coordinate frames, required memory and gated access. Evaluate trajectory quality and behavior independently of whether the explanation sounds plausible.
License and availability
Edge and Super weights are publicly downloadable on Hugging Face under OpenMDW-1.1. Their inference repositories use Apache-2.0. The Alpamayo 1 and 1.5 cards also link OpenMDW-1.1 and describe non-commercial use with commercial licensing on request. Check the intended-use wording and license of the exact checkpoint and any datasets it requires. Edge weight license · Super weight license · Edge code license · Super code license
Official references
- Architecture and workflow documentation
- R1 model card
- AlpaSim driver tutorial
- Alpamayo recipes for tuning and deployment
- AlpaJudge model card and source for evaluating driving reasoning and supplying training rewards