I tried running NVIDIA Nemotron 3 Nano Omni on DGX Spark

Introduction

Hello, I'm Morishige from Classmethod's Manufacturing Business Technology Department.

NVIDIA released Nemotron 3 Nano Omni on April 28, 2026. It is an Omni model capable of handling four types of input — text, images, audio, and video — within a single model, featuring a 30B-A3B Mamba+Transformer hybrid MoE architecture. NVIDIA's official blog highlights "7.4× throughput for multi-document and 9.2× for video compared to other open Omni models" and "20% improvement in multimodal intelligence," and the release was accompanied by immediate support across Hugging Face, OpenRouter, build.nvidia.com, Ollama 0.22, and SageMaker JumpStart all at once.

https://developer.nvidia.com/blog/nvidia-nemotron-3-nano-omni-powers-multimodal-agent-reasoning-in-a-single-efficient-open-model/

Since DGX Spark and DGX Station are explicitly listed as officially supported GPUs in the vLLM official blog, I immediately tried running it on my DGX Spark. Three quantization variants — NVFP4 (~21 GB), FP8 (~33 GB), and BF16 (~62 GB) — were released simultaneously, so I compared startup, latency, and output quality for each while testing all four modalities: text, image, audio, and video.

Let me first share my hands-on impressions. The NVFP4 variant (~21 GB) handles all four modalities on DGX Spark with short response times: image (PPE detection in 2 seconds), audio (transcribing "Mary had a little lamb" in 0.85 seconds), and video (understanding a 15-second conveyor situation in 0.64 seconds) all worked smoothly. I also sent the same PPE image to Gemma 4 31B IT NVFP4 (vLLM 0.20.0) with the same quantization and backend, which returned in 9.78 seconds. The fact that even with the same NVFP4, the difference is 0.67 seconds vs. 9.78 seconds is the key takeaway of this quick report for experiencing the performance characteristics of the Omni model.

Overview of Nemotron 3 Nano Omni

Model Family

All three variants accept text + image + audio + video as input, and output text. The context length is 256K. The architecture uses a Mamba2 + Transformer hybrid, integrating CRADIO v4-H as the image encoder and Parakeet as the audio encoder. The encoder sections are kept in BF16 without quantization, so input quality is not degraded.

Key Architecture Points

In the hybrid MoE, only 3B parameters out of the total 30B are active during inference, achieving high throughput while maintaining expressive power. For video, Conv3D + EVS (Efficient Video Sampling) is used to drop redundant frames, with --video-pruning-rate 0.5 as the default recommendation. On the generation side, Reasoning mode is ON by default, and passing --reasoning-parser nemotron_v3 allows separating reasoning content from the final answer via <think>...</think>. Tool calling is configured by specifying --tool-call-parser qwen3_coder, and responses come back in a Qwen3-compatible format.

License

NVIDIA Nemotron Open Model License. Commercial use is permitted, and weights, datasets, and training techniques are openly published.

https://huggingface.co/nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16

Deployment Options

As of release day, the following options are already available. This article focuses on DGX Spark + vLLM 0.20+, but here is an overview of all the options.

On the SageMaker JumpStart side, the model ID is huggingface-vlm-nvidia-nemotron3-nano-omni-30ba3b-reasoning-fp8, with recommended instances being ml.p4d.24xlarge / ml.p5.48xlarge. It can be launched with just a few lines using JumpStartModel(...).deploy(accept_eula=True). The AWS Machine Learning Blog article is a good reference for details. For cases where you want to keep data governance entirely in-house or handle data that cannot be sent outside for licensing reasons, the local DGX Spark route is a good fit; for cases where you need peak scaling or multi-tenant operation, the JumpStart route seems more suitable.

Test Environment

Setup Procedure on DGX Spark

Installing vLLM 0.20.0

The vLLM official blog explicitly states vllm[audio]==0.20.0. Without the audio extras, a ValidationError will occur when streaming audio to /v1/chat/completions, so be sure to specify it including the extras.

mkdir -p ~/works/nemotron3-omni && cd ~/works/nemotron3-omni
uv venv --python 3.12
source .venv/bin/activate
uv pip install "vllm[audio]==0.20.0"

Starting NVFP4

NVFP4 at ~21 GB fits comfortably within DGX Spark's 128 GB UMA. The Hugging Face model card recommends specifying --moe-backend flashinfer_cutlass for NVFP4.

vllm serve nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-NVFP4 \
  --served-model-name nemotron-omni \
  --max-model-len 32768 \
  --gpu-memory-utilization 0.5 \
  --media-io-kwargs '{"video":{"num_frames":256,"fps":2}}' \
  --video-pruning-rate 0.5 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --reasoning-parser nemotron_v3 \
  --moe-backend flashinfer_cutlass \
  --trust-remote-code

Switching to FP8 and Starting

Remove --moe-backend flashinfer_cutlass since it is NVFP4-specific, and change the model ID and memory settings.

# FP8 (balanced, ~33 GB)
vllm serve nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-FP8 \
  --served-model-name nemotron-omni \
  --gpu-memory-utilization 0.6 \
  --max-model-len 32768 \
  --media-io-kwargs '{"video":{"num_frames":256,"fps":2}}' \
  --video-pruning-rate 0.5 \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder \
  --reasoning-parser nemotron_v3 \
  --trust-remote-code

BF16 Is Out of Scope This Time

The BF16 variant has a model body of ~62 GB, plus the image encoder (CRADIO v4-H, kept in BF16) and audio encoder (Parakeet, kept in BF16). On top of that, KV cache is also required (around 10 GB even with fp8_e4m3 at max_model_len 32768), and considering the remaining capacity of DGX Spark's 128 GB UMA after accounting for OS and cache, it is difficult to reach practical operating range even with --cpu-offload-gb 16.

Quantization Variant Comparison Benchmark

I ran the same prompt ("Explain in 3 short paragraphs how a hybrid Mamba+Transformer mixture-of-experts architecture differs from a dense Transformer LLM, focusing on inference efficiency.") for 5 rounds each, aligning latency and output throughput. Reasoning mode was kept ON by default.

NVFP4 is a 4-bit format that runs natively on DGX Spark's Blackwell GB10 (sm_121), operating with the smallest memory footprint while maintaining output quality. The numbers show approximately 16% lower latency and about 12% higher throughput compared to FP8. Since the encoder sections are kept in BF16, the input quality for images, audio, and video is not degraded.

Multimodal Operation Verification

I sent all four modalities sequentially to the NVFP4 model via the OpenAI-compatible /v1/chat/completions endpoint, mixing image_url, input_audio, and video_url in the messages[].content array. Since Reasoning mode tends to produce longer responses, for short-answer tasks like numeric comparisons or transcription, it was practical to pass chat_template_kwargs.enable_thinking=false.

For example, to send an image in base64, the curl command looks like this:

IMG=$(base64 -w0 ppe-sample.jpg)
curl -s http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d "{
    \"model\": \"nemotron-omni\",
    \"messages\": [{\"role\": \"user\", \"content\": [
      {\"type\": \"image_url\", \"image_url\": {\"url\": \"data:image/jpeg;base64,$IMG\"}},
      {\"type\": \"text\", \"text\": \"List the personal protective equipment visible in this image.\"}
    ]}],
    \"max_tokens\": 800,
    \"chat_template_kwargs\": {\"enable_thinking\": false}
  }"

Simply changing type to input_audio (specifying format as wav etc.) or video_url (data:video/mp4;base64,...) enables audio and video as well.

Text

Briefly describe what makes a multimodal omni model different from a text-only LLM.

Response in 4.87 seconds (277 tokens). Here is an excerpt from the beginning:

A multimodal omni model differs from a text-only LLM in several key ways:

1. Multimodal Input/Output: Unlike text-only LLMs that only process and generate text,
   multimodal omni models can handle various data types, such as images, audio, video, and text.
2. Contextual Understanding: Multimodal omni models can better understand context by integrating
   information from different modalities. ...

Image (PPE Detection)

I submitted the following image, which was also used in a previous article on Cosmos-Reason2 structured reasoning.

Response in 2.04 seconds (108 tokens).

Based on the image provided, the following personal protective equipment (PPE) is visible:

- Hard hats: Both individuals are wearing hard hats. The woman is wearing a white hard hat, and the man is wearing a yellow one.
- High-visibility vest: The woman is wearing a bright, lime-green high-visibility vest over her shirt.
- Safety glasses: The woman is wearing safety glasses, which are visible under her hard hat. The man is also wearing safety glasses, though they are less distinct.

The response described colors and wearing conditions in detail.

Audio (English, approx. 16 seconds)

I converted the vLLM official asset (mary_had_lamb.ogg) to 16 kHz mono WAV and submitted it.

Response in 0.85 seconds (43 tokens).

A man speaks in the original phonograph a little piece of practical poetry, Mary had a little lamb,
its fleece was white as snow, and everywhere that Mary went, the lamb was sure to go.

The response simultaneously transcribed and summarized the content, including context about it being a historical phonograph recording.

Video (Conveyor, 15 seconds)

I submitted a video of cardboard boxes on a conveyor line, the same one used in a previous article on VSS Event Reviewer verification.

Response in 0.64 seconds (15 tokens).

A cardboard box is moving on a conveyor belt that is curved.

The response came back concisely and within the bounds of the short two-sentence video summary instruction.

Comparison with Gemma 4 31B

While there is a fundamental difference in model architecture, I wanted to see "how does the performance and latency of Omni compare when placed alongside a different VLM running with the same backend and same quantization on DGX Spark?" So I sent the same PPE sample image with the same prompt (an instruction to return ppe_items and overall_compliance as JSON) to both Nemotron 3 Nano Omni (NVFP4) and Gemma 4 31B IT NVFP4.

For launching Gemma 4 31B IT NVFP4 with vLLM, I passed --reasoning-parser gemma4 --tool-call-parser gemma4 --max-num-batched-tokens 4096. Since the image encoder produces a large number of tokens, raising max_num_batched_tokens from 2048 to 4096 is necessary to avoid being rejected during initialization (see the vLLM official Gemma 4 recipe).

Speed and Size

There was approximately a 15× difference in actual inference time. Since both use NVFP4 + vLLM under the same conditions, this difference is almost entirely due to the model architecture itself. Omni uses a MoE (Mixture-of-Experts) structure where, despite holding 30B parameters, only 3B worth of parameters are actually used per token generation. Gemma 4 31B, on the other hand, is Dense, meaning all 31B parameters are traversed for every token. The "number of active parameters per token generation" is literally 10× different, so a latency gap of around 15× is a natural result. The effect of MoE's goal of "large model capacity, low inference cost" is clearly visible in these benchmark numbers.

Output

Both returned JSON, but with slight differences in detail and formatting.

// Nemotron 3 Nano Omni (NVFP4, 29 tokens, 0.67 seconds)
{ "ppe_items": ["hard hat", "safety vest"], "overall_compliance": "compliant" }

// Gemma 4 31B IT NVFP4 (57 tokens, 9.78 seconds)
{
  "ppe_items": ["hard hat (white)", "hard hat (yellow)", "high-visibility safety vest"],
  "overall_compliance": "non_compliant"
}

Gemma 4 returned the result wrapped in a Markdown code fence, listing helmets broken down by color. Omni returned just the JSON as instructed.

The difference in judgment between compliant and non_compliant comes from how each model interpreted the image. Omni saw "the PPE that is visible is present → compliant," while Gemma 4 determined "non_compliant including equipment not visible in the image such as safety glasses." Both are reasonable judgments, and when incorporating into a workflow, this is the kind of difference where "what counts as compliant" needs to be explicitly stated in the prompt, or the judgment will vary between VLMs.

Usage Guidelines

Handling in Japanese Environments

According to the Hugging Face model card, Nemotron 3 Nano Omni was trained only in English, and no Japanese-derived version (like the past nvidia/Nemotron-Nano-9B-v2-Japanese) has been released this time. Japanese tasks are not included in the benchmark tables either, so for Japanese-specific scores, the past Gemma 4 benchmark and Nemotron 9B-v2-Japanese hands-on are better references (specific values are listed in the table below).

However, when I tried submitting Japanese prompts, the model returned responses in Japanese without issue. I ran two queries with NVFP4 — one text, one image — in Japanese (enable_thinking=false).

Text (25.08 seconds, 279 tokens)

In response to the prompt "Please briefly explain what makes a multimodal integrated model like Nemotron 3 Nano Omni different from a text-only LLM." (excerpt from the beginning):

A multimodal integrated model like Nemotron 3 Nano Omni differs from a text-only LLM in the following ways:

1. Processing diverse input modes: Can understand and generate non-text inputs (images, audio, video, etc.).
2. Integrated reasoning: Can process text and other modalities simultaneously, enabling context-appropriate responses and generation (e.g., image description + text generation).
3. Richness of context: Incorporating visual and auditory information enables richer and more accurate contextual understanding.
4. Expanded application areas: Executes tasks impossible for text LLMs, such as visual question answering, video description, and speech recognition integration.

In brief: The main difference is that text LLMs operate in a world of "words only," while multimodal models operate in a world of "words + visual and auditory information."

Proper nouns remained in English, while the explanations were returned in structured Japanese.

Image (PPE Detection, 1.17 seconds, 31 tokens)

When I sent the same PPE sample image with the prompt "Please list the personal protective equipment (PPE) visible in this image in bullet points in Japanese," the result was:

- White safety helmet
- Green safety vest
- Yellow safety helmet

The items were listed in Japanese including colors. Compared to the English response ("hard hat / safety vest"), the level of detail doesn't seem to drop significantly.

While this is a model trained for English benchmarks and has no officially measured Japanese performance scores, the fact that simply switching the prompt to Japanese yields Japanese responses is a reassuring finding when reusing English-oriented tutorials for Japanese use cases.

Reference: Running JCommonsenseQA Scores

Relying only on the feel of "the model returned Japanese even though it's an English-only model" is too subjective, so I ran the same 3-shot prompts used in the past Gemma 4 series to test all 1,119 validation questions from JCommonsenseQA v1.1 on both Omni and Gemma 4 31B IT NVFP4. Both used vLLM 0.20.0 + NVFP4 as in this article.

It was confirmed that Gemma 4 31B IT maintains nearly the same accuracy even with NVFP4, dropping only marginally from 97.9% to 97.77%. Despite being trained only in English with relatively small active parameters (Active 3B), Omni achieved 88%, demonstrating a respectable level of capability for Japanese commonsense reasoning. While it doesn't reach Nemotron-Nano-9B-v2-Japanese (91.2%), it was honestly surprising to see Omni — which handles text, image, audio, and video all-in-one — perform this well. A full evaluation for Japanese multimodal tasks (Heron-Bench / JMMMU, etc.) is something I'd like to compile in a separate article.

Pitfalls

--media-io-kwargs Values Differ Across Sources

The vLLM official blog uses num_frames=512, fps=1, while the Hugging Face model card uses num_frames=256, fps=2 — the recommended values differ between sources. Since the number of frames ingested per video × the sampling fps directly affects VRAM usage and inference latency, it is best to decide on a fixed value early on based on your hardware situation. This time, I adopted the HF card's 256 / fps=2, and was able to run the 15-second conveyor video without putting pressure on VRAM.

Insufficient max_tokens in Reasoning Mode Results in Empty Content

Nemotron 3 series generates reasoning content within <think>...</think> before producing the final answer. However, when I set max_tokens=300 and asked for a video description (2 sentences), the reasoning consumed all 300 tokens and the content came back empty (null). Passing chat_template_kwargs.enable_thinking=false returned A cardboard box is moving on a conveyor belt that is curved. in just 13 tokens. The practical approach is: disable thinking for short-answer verification and transcription tasks, and when you want to leverage reasoning capability, enable thinking and allocate around 1500 max_tokens.

Summary

I confirmed that all four modalities work on DGX Spark on release day. With NVFP4 fitting within ~21 GB, it appears to offer the best balance of startup, latency, and throughput among the three quantization variants for everyday use. The 0.67 seconds vs. 9.78 seconds gap compared to Gemma 4 31B IT under the same NVFP4 + vLLM conditions clearly visualizes in benchmark numbers the MoE characteristic of maintaining 30B-class model capacity while keeping inference cost to Active 3B. With JCommonsenseQA showing Omni at 88% and Gemma 4 31B IT at 97.77% with NVFP4 (only a 0.13 point drop from BF16), the accuracy degradation from quantization is nearly negligible, and the fact that Japanese commonsense reasoning is already passing at a reasonable level was a pleasant discovery.

Reference information:

• NVIDIA Developer Blog: Nemotron 3 Nano Omni
• vLLM official blog: Nemotron Omni
• Hugging Face: Nemotron-3-Nano-Omni-30B-A3B-Reasoning-NVFP4
• AWS Machine Learning Blog: Nemotron 3 Nano Omni on SageMaker JumpStart