Cosmos3-Nano-Sim-Auto: NVIDIA's 16B world model that simulates driving in real time, on-premise

A world model is a neural simulator: a model that has learned how the world behaves and can generate what would happen if you took this maneuver, in this scenario, under these weather conditions. It does not answer questions — it produces scenarios. And NVIDIA is bringing them out of the cloud: Cosmos3-Nano-Sim-Auto is a 16-billion-parameter world model for real-time driving simulation, published on Hugging Face and GitHub under the OpenMDW 1.1 license. Here we look at what it generates, how the closed loop works, and what "on-premise" really means when we are talking about a 16B simulator.
What Cosmos3-Nano-Sim-Auto is
The checkpoint published by NVIDIA is the causal-distilled variant of the Cosmos3 family. The chain has two stages: Cosmos3-Nano does post-training on multi-sensor driving data, including LiDAR clouds, and becomes Cosmos3-Nano-Transfer-Auto; then a causal distillation turns it into a few-step model that generates video in small chunks of latent frames conditioned on the history in cache and on the controls received. The architecture is a Mixture-of-Transformers (MoT) with 16B trainable parameters, developed with the Cosmos Framework, open source on GitHub.
The difference with an image model or an LLM is not the size, it is the nature of the output: it does not generate a finished image, it generates a world that reacts. Every chunk depends on the driving commands received; the next chunk depends on the previous one. Spatial and temporal coherence between the cameras and the LiDAR is built into the model, not added in post-production.
How the closed loop works
The loop the model closes is this: a driving policy (or a human steering wheel) produces commands; a WSM renderer (World Scenario Map) converts them into scene controls — MP4 video of the scene structure, with lane markings and object cuboids, one per view; the model generates the observation; the policy reads it and commands again. At every step the world updates in response to the input.
The observation it generates is rich:
- Up to 11 camera views — the reference rig is Hyperion 8.1: front-wide 120°, front-tele 30°, cross-left and cross-right 120°, rear-left and rear-right 70°, rear-tele 30°, plus four 200° fisheyes
- 480p or 720p, 10–30 fps on all selected views, spatially and temporally synchronized
- Aligned LiDAR — a 3D point cloud from a 128-beam sensor, converted from the range maps the model generates
The inputs it accepts, per the model card:
- Textual prompt: weather, lighting, time of day; it also carries the camera configuration
- WSM controls: the synchronized structure videos, view by view
- Initial RGB (optional): the first frame of each camera
- LiDAR maps: sequences of range maps derived from the HD map, required if you ask for LiDAR output
What "16B" really weighs
16B is a number that for an LLM you would say "that fits even on 24 GB of VRAM". For a world model the count is different, because the weights are only the fixed part: the rest is the generation chunks that run in parallel across 11 views plus a sensor. The math:
| Item | Calculation | Result |
|---|---|---|
| Weights in FP16 | 16 × 10⁹ params × 2 bytes | ≈ 32 GB |
| Weights in FP8 | 16 × 10⁹ params × 1 byte | ≈ 16 GB |
| Generation buffers | 11 video streams + LiDAR, chunk by chunk | not declared |
| Temporal history in cache | previous chunks, for world coherence | not declared |
The 32 GB of FP16 weights would theoretically fit on a 32 GB GPU. But this model's job is not to read the weights: it is to generate 11 video streams and a LiDAR, frame after frame, at 10–30 fps. That is why NVIDIA declares as reference hardware the Hopper series (H100, H20, H200) and Blackwell (B200, B300, GB200, GB300), on Linux. It is not a prohibition: it is the tested and guaranteed perimeter.
Why run it on-premise
A simulator that runs on-premise has four concrete advantages:
- Closed-loop latency. The policy must read the observation and command again at every step: every round trip to a remote service is a delay that, in the real world, is measured in meters.
- The data stays where it is. Driving video, maps, fleet trajectories: they do not cross a provider, they stay on site.
- Availability. The simulator is there when the hardware is there: no queue, no quota, no network dependency. It is the same local-AI logic, applied to physics.
- The OpenMDW 1.1 license. Ready for commercial and non-commercial use: you put the simulator into production without negotiating a rendering license.
On costs, the reasoning is the one you do for any GPU you own: the capital is already spent, and the margin is electricity. A 700 W GPU under load (the TDP of an H100 SXM) in steady state consumes 0.7 kWh per hour; at 8 hours per day that is 5.6 kWh — at €0.30/kWh, roughly €1.70 per day, a handful of euros per month. The right comparison is not with cloud rendering: it is with the fact that the scenarios, before, did not exist. Now they are generated when needed, in unlimited quantity.
And the limits?
Honestly, because here transparency helps you decide whether it makes sense for you:
- It is not a consumer-workstation model. The declared hardware is datacenter: Hopper or Blackwell. If your GPU has 16–24 GB, for today this is not your model. It is the first 16B world model that is realistic to deploy — not the first one that runs on the desktop.
- Linux only. The other platforms have not been tested.
- One measured LiDAR sweep as prefix. The current causal path accepts a single real LiDAR pass at the start; longer prefixes are not supported.
- 480p/720p, 10–30 fps. It is simulation, not photorealism: for evaluating a driving policy it is more than enough.
- The rig is the declared one. From 1 to 11 views on the Hyperion rig; for custom configurations you need a post-training on the Transfer-Auto variant.
Sources
- Model card nvidia/Cosmos3-Nano-Sim-Auto
- NVIDIA/cosmos, the Cosmos Framework code on GitHub
- Technical report — Cosmos Simulation
In summary
Cosmos3-Nano-Sim-Auto is a closed-loop driving simulator: 16B parameters in a MoT architecture, generating up to 11 cameras and a synchronized LiDAR cloud at 10–30 fps in 480p/720p, conditioned by a textual prompt and the WSM controls. "On-premise" here is not an aesthetic choice: it is the loop latency, it is the data that does not cross the network, it is the OpenMDW 1.1 license for commercial use. The required hardware is datacenter, and the limits are declared with clarity in the model card. The missing piece in the Cosmos3 family: the DROID policy for on-edge robotics was already there — now there is also the world to test it in.
Article generated with AI
In these reading paths
Enjoyed this article?
Subscribe to the newsletter for new tutorials and guides.

