I organized a usage map for the NVIDIA Cosmos 3 family

Introduction

Hello, I'm Morishige from Classmethod's Manufacturing Business Technology Division.

In Running NVIDIA Cosmos 3 on DGX Spark, I covered NVIDIA Cosmos 3 as an omnimodel that integrates both the Reasoner Tower and Generator Tower into a single model. In this article, which serves as a summary of the series, I'll organize how to use Cosmos 3 in practice from the perspective of "how to differentiate usage in the field," while also running actual Japanese visual reasoning benchmarks with the understanding-focused Reasoner Tower operating as a VLM.

https://dev.classmethod.jp/articles/dgx-spark-cosmos3-omni-world-model-policy/

Cosmos 3 is centered around two omnimodels called Nano and Super, along with generation-specialized derivative models and robot control models. Furthermore, given that the existing Cosmos Reason 2 is widely used as the VLM component of VSS (NVIDIA Video Search and Summarization), the practical decision axis becomes "how far to continue using the existing Reason 2, and from where to switch to Cosmos 3."

The Cosmos 3 Family and Cosmos Reason 2

First, let me lay out the models released for Cosmos 3 alongside the existing Cosmos Reason 2 on a single map.

The key point to understand here is that in Cosmos 3, the understanding-focused Reasoner Tower and the generation-focused Generator Tower are integrated into a single omnimodel. In the previous generation, models were separated by purpose — Cosmos Reason for video understanding, Cosmos Predict for video generation — but Cosmos 3 consolidates these into the omnimodel called Nano / Super. When you only want to use understanding, you can extract just the Reasoner Tower at inference time and run it as a VLM.

As a quick reference table for features and scale, the organization would look something like this.

Nano fits comfortably within the DGX Spark™ 128 GB unified memory, while the Super series — both the omnimodel and generation-specialized derivatives — is cramped on a single DGX Spark. If you plan to run Super in production, it's safer to assume you'll need a Brev H100 or DGX Station-class environment. Cosmos 3 Edge, coming soon at a small 4B scale, looks like it will be an option aimed at edge GPUs like Jetson.

Combinations That Can Run on DGX Spark

Let's think about what can be co-resident and in what combinations within the DGX Spark budget of 128 GB unified memory.

Based on my hands-on experience running it, extracting only the Reasoner Tower of Cosmos 3 Nano loads it at about 17 GB, so placing it alongside Cosmos Reason 2 (~17 GB) keeps the total to around 35 GB. Serving the Reasoner Tower is done by overriding the architecture at startup like vllm serve nvidia/Cosmos3-Nano --hf-overrides '{"architectures": ["Cosmos3ReasonerForConditionalGeneration"]}', and by constraining --gpu-memory-utilization, they can co-reside on one machine even including the KV cache. When running the full omnimodel including generation, ~30 GB is sufficient for a single DGX Spark.

Conversely, Cosmos 3 Super — both the omnimodel and generation-specialized derivatives — almost exhausts the entire DGX Spark budget on its own, so if you seriously want to run it, it's assumed you'll delegate it to a separate node. As a decision criterion when choosing DGX Spark, the conclusion is that "a single-node environment where Nano is the main player and can be switched with Cosmos Reason 2" fits best. Building in a design from the start that offloads the Super series to a separate environment will save trouble later.

Benchmarking Cosmos 3 Nano's Reasoner

Before discussing usage differentiation, let me take actual measurements of how capable the understanding-focused Reasoner Tower is. I served the Cosmos 3 Nano Reasoner Tower on a GB10 DGX Spark and ran the Japanese visual reasoning benchmarks Heron-Bench and JMMMU, as well as robot trajectory CoT (Chain-of-Thought) and Embodied Reasoning. The comparison target is Cosmos Reason 2, the VSS default.

First, here are the Heron-Bench results (a benchmark where an LLM scores free-form responses). I've lined them up by question category and image category.

The average score was 2.777 for Cosmos 3 Nano, nearly identical to Cosmos Reason 2. Neither model dominated the other across question categories (conversation / detail / complex) or image categories (anime / art / culture, etc., 7 types); rather, the results were closely contested with small fluctuations by category. In terms of Japanese free-form responses, both appear to be at the same level.

Next are the JMMMU results (multiple-choice questions across 28 fields, scored by exact match).

The overall exact match for Cosmos 3 Nano was 0.498, slightly edging out Cosmos Reason 2. Looking at individual fields, both models are strong in areas like World History, while there are also fields like Music and Mechanical Engineering where both struggle — their tendencies for strengths and weaknesses are quite similar. The conclusion visible from actual measurements is that even in multiple-choice, both models' accuracy is nearly neck and neck.

I also confirmed structured output. For the robot trajectory CoT task (having the model output grasp and placement waypoints as a point sequence from images), the ratio of valid JSON returned for 8 inputs was 0.62, with an average inference time of about 8.3 seconds per image. If trajectory generation comes in under 10 seconds, there's realistic potential for incorporating it into small arms like SO-ARM101 or Reachy Mini for demo/interaction purposes.

In the Embodied Reasoning probe, I looked at whether reasoning could be drawn out inside <think>...</think> tags by varying prompt patterns. With plain prompts, thinking tags barely appeared, but when using phrasing that explicitly requested CoT or specified safety considerations, all 4 cases stably produced substantive reasoning inside <think> tags.

In summary, the Reasoner Tower's accuracy is on par with Cosmos Reason 2, and the decision axis for switching is not "difference in accuracy" but "properties unique to Cosmos 3" — that's the impression that emerges from these measurements.

Scenarios Where You Should Continue Using Cosmos Reason 2

First, "cases where you should continue using Cosmos Reason 2." As seen in the previous section, since Cosmos 3 Nano's Reasoner Tower and Cosmos Reason 2 are at nearly the same accuracy level on benchmarks, there are several scenarios where there's no need to switch.

The top case is when it's already integrated into an existing VSS pipeline. In VSS 3.1.0 EA, cosmos-reason2-8b is bundled as the standard hw env for DGX Spark, and compose files built on the premise of Cosmos Reason 2 are provided for all of VLM-as-Verifier / Event Reviewer / Alert Bridge. As I also mentioned in the previous VSS 3.1.0 EA verification article, the proper approach for replacing this with a Cosmos 3 series model is to wait for NVIDIA's roadmap to mature.

Next, cases where it's running on edge environments like Jetson Orin Nano / AGX Thor. Cosmos Reason 2 has a track record with ARM64 and quantization (FP8 / NVFP4), and edge deployment recipes are established. The Cosmos 3 series is currently centered on BF16, and the lightweight edge version Cosmos 3 Edge is not yet available. If the premise is running on Jetson, Cosmos Reason 2 remains the solid choice for the time being.

Third, use cases that aim to maximize the number of streams, like parallel multi-camera streaming surveillance. VSS Blueprint benchmarks also show that 14 parallel streams can be handled with Cosmos Reason 2 using a combination of DGX Spark and AGX Thor — this is a scenario where you choose based on proven track record for high-density surveillance use cases.

Fourth, event detection that completes with structured JSON, such as PPE detection, anomaly detection, and action classification. Cosmos 3 Nano also returns the same JSON structure with the same prompts, so "accuracy doesn't change if you switch" — but conversely, this also means "there's little to gain by switching." If the existing pipeline is working, there's not much reason to bother swapping out the model.

Overall, Cosmos Reason 2 retains its strengths for the foreseeable future in the position of lightweight, low-latency, and proven track record. Until momentum comes from NVIDIA to switch the standard VSS set to Cosmos 3 series, this is my personal assessment of the optimal solution for stable operation.

Scenarios Where It's Worth Switching to Cosmos 3 Nano

Conversely, there are also clear scenarios where it's worth switching from Cosmos Reason 2 to Cosmos 3 Nano. Since accuracy itself is on par within margin of error, the decision axis is not "difference in accuracy" but "properties unique to Cosmos 3 Nano."

The first is scenarios requiring causal reasoning or CoT. When you want to have the model explain step-by-step "why did this anomaly occur" or "what should be done next," the Reasoner Tower's reasoning chain comes into play. As seen in the Embodied Reasoning section above, by specifying concrete risk perspectives and procedures on the prompt side, substantive reasoning can be drawn out inside <think> tags.

The second is robotics trajectory planning. The trajectory generation of around 8 seconds measured in Robot CoT is a speed that's usable when building an interactive flow of "image input → trajectory generation → bridging to ACT / GR00T" with small arms like SO-ARM101 or Reachy Mini.

The third is simultaneous multi-modal reasoning. Cosmos 3 Nano supports not only a 256K long context but also, as an omnimodel, can handle images, video, audio, and actions all within a single model. It comes into play for inputs that need to be processed across modalities, such as long-duration report generation that spans multiple images, videos, and text, or a full-day video summary of a manufacturing line.

The fourth is scenarios requiring physical reasoning derived from a world foundation model. Since Cosmos 3 Nano's Reasoner Tower is connected to the Generator Tower through shared latent representations, the physical learning from the generation side indirectly seeps in. This fits the niche requirement of "I don't need generation, but I do want physical intuition."

In terms of implementation practicality, since the JSON output structure is compatible with Cosmos Reason 2, as long as you work out VLM-as-Verifier Sidecar compatibility and production stability, replacing the existing Cosmos Reason 2 pipeline with Cosmos 3 Nano should require minimal modifications.

Scenarios for Deploying Cosmos 3 Nano as an Omnimodel

So far, we've been talking about using the Reasoner Tower as a VLM, but Cosmos 3 Nano is an omnimodel that has both understanding and generation co-resident in a single model. It seems to show its true value in use cases where understanding and generation need to operate simultaneously.

The most representative is an end-to-end workflow of observation → planning → generation → control. The Policy Model seen in Running NVIDIA Cosmos 3 on DGX Spark is exactly this — from observation video and natural language task instructions, it simultaneously outputs predicted video and robot action sequences. Considering that it passed the official golden standard (MSE 0.05) with MSE 0.013 in article 1, there's a tangible sense that it's reaching the practical range as a VLA (Vision-Language-Action) backbone for lightweight robot arms. Incorporating it into small robots like Reachy Mini or SO-ARM101 is a direction I'd like to continue covering in future robotics series articles.

Synthetic data generation is also a use case unique to omnimodels. Since you can visualize as video "what would happen if this state were left alone" or "how would an anomaly event look as it progresses" from observed footage, this seems effective for automated generation of compliance education materials and safety training materials in manufacturing. A combination of passing anomaly events extracted by VSS Event Reviewer to Cosmos 3 Nano's generation mode is starting to look realistic.

Sim2Real robotics data augmentation is also a scenario that leverages the generation side. If a workflow can be established to mass-produce synthetic episodes in LeRobot v3 format with Cosmos 3 and feed them into policy training, the bottleneck of data collection through real-machine teleoperation could be significantly alleviated. When running on DGX Spark, Cosmos 3 Nano is the practical solution, while Super requires setting up a separate GPU environment in terms of both parameter scale and memory requirements.

Scenarios Where Generator Functionality Takes Center Stage

There are also scenarios where you want to put weight on the generation side. Video, image, audio, and action generation were split into separate models until the previous generation — Cosmos Predict 2.5, Cosmos Transfer, and so on. Cosmos 3 integrates these into an omnimodel and further provides generation-specialized derivatives such as Super-Image2Video specialized for image → video and Super-Text2Image specialized for text → image. The use cases that previously belonged to the Cosmos Predict 2.5 series look set to largely be covered by Cosmos 3 going forward.

Particularly in Sim2Real data augmentation, the part that used a two-stage pipeline of video generation with Cosmos Predict 2.5 → action inference with policy can be consolidated into a single inference with Cosmos 3 Nano. When I previously ran Predict 2.5, generating a 1280×704 video with the 2B model at 36 steps took about 30 minutes. In contrast, the Cosmos 3 Nano Policy Model measured in article 1 is smaller in output scale (640×480 × 17 frames), but outputs video and actions simultaneously in 21 seconds — that's the difference. A direct comparison isn't possible since the output scales differ, but the overall feel of the workflow is nearly a different experience.

However, generating ultra-high-resolution long-form video (movie-grade use cases like 4K × 30 seconds) falls outside the scope of Cosmos 3. This is a domain to leave to cloud-based dedicated video generation models like Veo or Wan, so if you go in thinking you can "create anything" with Cosmos 3, you'll end up with a misalignment of use cases. Cosmos 3 is designed as a backbone for Physical AI, so it's suited for operation at the resolution and frame count appropriate for robotics, autonomous driving, and industrial simulation.

Summary

I organized the usage differentiation map for NVIDIA Cosmos 3, incorporating actual measurements of the Reasoner Tower. The flow of the organization looked like this.

First, Cosmos 3 is structured around Nano / Super omnimodels, with generation-specialized derivatives and robot control models, and Cosmos 3 Edge will be joining soon. On DGX Spark, Nano is the center of practical operation, while the Super series assumes offloading to a separate environment. Next, the Japanese visual reasoning of the Reasoner Tower competes with Cosmos Reason 2 at Heron-Bench 2.777 and JMMMU 0.498, and the decision axis for switching is not accuracy but "properties unique to Cosmos 3." Specifically, scenarios where causal reasoning, CoT, multi-modal input unique to omnimodel, and physical reasoning come into play are the switching points. Furthermore, in robotics requiring an end-to-end flow of observation → planning → generation → control, and in synthetic data generation, value emerges from deploying Cosmos 3 Nano as an omnimodel.

Given a single-node environment with DGX Spark's 128 GB unified memory, I feel that the realistic solution at this stage is to differentiate by scenario — whether to continue with Cosmos Reason 2 or switch to Cosmos 3 Nano — and run them in co-resident and switchable operation. An approach that keeps Reason 2 as the main axis while sequentially incorporating new use cases where the Reasoner Tower's reasoning chain and Policy Model are effective seems to be the safe bet.

Reference Links

• NVIDIA Cosmos Platform Overview
• NVIDIA Cosmos 3 Official Announcement (NVIDIA Blog)
• Cosmos 3 License (OpenMDW 1.1 / Linux Foundation)
• Running NVIDIA Cosmos 3 on DGX Spark
• Running the Cosmos World Foundation Model on DGX Spark (Introduction to Predict 2.5 + Reason 2)
• Trying Structured Image and Video Analysis with Cosmos-Reason2 on DGX Spark
• Fine-tuning Cosmos-Reason2-8B on DGX Spark for PPE Detection
• Investigating the Current State of Manufacturing VSS Seen at VSS 3.1.0 EA and Hannover Messe
• Thinking About Where NVIDIA VSS + AI Agents + Skills Can Be Applied in Everyday Workplaces