NVIDIA adds Cosmos 3 checkpoints for driving and robot simulation

What changed

In a November 5 article, NVIDIA describes four Cosmos 3 Nano checkpoints. Transfer-Auto generates multi-view driving data, while Sim-Auto streams synchronized camera and LiDAR observations. Sim-Transfer renders video using simulator controls, and Sim-Bimanual predicts observations from robot or camera actions.

Source: NVIDIA: Cosmos 3 models for interactive world simulation

Our analysis

Our analysis: choose the checkpoint around the information your evaluation already has. A physics simulator supplies the state for Sim-Transfer; Sim-Bimanual instead predicts the next observation from the scene and actions. Test generated sequences on held-out scenarios before using them to evaluate a policy.

Keep in mind

NVIDIA says closed-loop Sim-Auto needs a simulation runtime and control renderer. The examples are simulations, not evidence of safe real-world driving or robot operation. We have not run the models; the weights use the OpenMDW1.1 licence and its terms apply.

Source: NVIDIA: Cosmos 3 models for interactive world simulation