Tried Running the Image-Capable 120B MoE "Nemotron 3.5 Super VL 120B-A12B" on DGX Spark

Introduction

Hello, I'm Morishige from Classmethod's Manufacturing Business Technology Department.

NVIDIA released Nemotron 3.5 Super VL 120B-A12B on October 15. It is the largest model in the Nemotron 3.5 family, capable of reading both images and video. The announcement blog states it "runs on DGX Spark, DGX Station, and RTX systems," and the model card includes a vLLM recipe for a single DGX Spark.

https://blogs.nvidia.com/blog/nemotron-3-5-super-vl/

Bolt 74B-A9B, released on the same day, was benchmarked on DGX Spark's llama.cpp the day after its release (this article is dated 2026-10-16). Super VL, at 120B, has NVFP4 weights alone weighing 70 GiB. I was immediately curious whether it would fit in a 128GB DGX Spark, and if so, whether it would actually start up with the default 1M token context intact.

https://dev.classmethod.jp/articles/dgx-spark-nemotron-3-5-bolt-74b-a9b-first-touch/

To state the conclusion up front: the model card recipe ran as written, and it started up with the default 1M context in 11 minutes. With the external draft LiLiCorr it achieved 38.07 tok/s single-stream, and 19.06 tok/s without it. Japanese and tool-calling were best with thinking strength set to medium, achieving 9/9 on Japanese and passing all 6 agent fitness probes—a result no other model measured on the same tasks had reached. On the other hand, it could not read vertically written Japanese.

This article presents the results of running Nemotron 3.5 Super VL 120B-A12B NVFP4 on a DGX Spark with vLLM, two days after its release. I will cover whether the official recipe runs as-is, how much effect LiLiCorr has, and measurements of Japanese, code, tools, and images using the same tasks as for Bolt and Lightning.

What is Nemotron 3.5 Super VL 120B-A12B

First, let me establish the facts from primary sources. The numbers are taken from the model card, config.json, and the Hugging Face file listing.

Because Bolt 74B-A9B is a child model extracted from this model via Star Elastic, the layer count and hybrid skeleton are the same. The differences lie in the number of experts and quantization: Super VL holds 512 experts in NVFP4, while Bolt prunes experts per layer and compresses down to 2 bits. The active parameters used per token are 12B for Super VL, 9B for Bolt, and 3B for Lightning, which has a different skeleton.

Thinking comes in 3 levels. The default on, on (medium) which the card notes may reduce accuracy in exchange for fewer tokens, and off. Medium is enabled simply by passing medium_effort: true in chat_template_kwargs, the same mechanism as Bolt. Since thinking and the main response share the same max_tokens, in this article I use the term budget to refer to the output token limit.

The official benchmarks are presented as a comparison table between BF16 and NVFP4. I reproduce the card's numbers directly.

With NVFP4, code and Q&A scores barely drop. The declines appear in multilingual MMLU-ProX and in the long-running browser and repository tasks of WebVoyager and DeepSWE. Whether this gap shows up in Japanese tasks and long agent tasks is something I will examine hands-on.

Getting It Running

The Quick Start in the model card explicitly states "this vLLM configuration is for DGX Spark (GB10) only," with a startup command included. The image is vllm/vllm-openai:nightly-3f1bcfd82bb2e93fcd364fc7a5491718453a77e2, and the arm64 build of the same commit was pushed to Docker Hub on October 15. The card also lists the v0.32.0 tag, but as of October 17 it was not yet on Docker Hub.

After downloading the weights and draft with hf download, I passed the recipe's arguments verbatim.

docker run -d --name supervl --gpus all --ipc=host \
  -p 127.0.0.1:8000:8000 \
  -v "$MODEL_DIR":/model:ro -v "$DRAFT_DIR":/draft:ro \
  vllm/vllm-openai:nightly-3f1bcfd82bb2e93fcd364fc7a5491718453a77e2 /model \
  --served-model-name nemotron-3.5-super-vl \
  --max-num-seqs 1 \
  --trust-remote-code \
  --gpu-memory-utilization 0.75 \
  --max-num-batched-tokens 8192 \
  --speculative_config.model /draft \
  --speculative_config.num_speculative_tokens 7 \
  --speculative_config.method dflash \
  --mamba-backend flashinfer \
  --mamba-ssm-cache-dtype float16 \
  --enable-mamba-cache-stochastic-rounding \
  --mamba-cache-philox-rounds 5 \
  --enable-mamba-shared-prefix-checkpoint \
  --video-pruning-rate 0.5 \
  --media-io-kwargs '{"video": {"fps": 2, "num_frames": 256}}' \
  --block-size 64 --prefix-match-unit 64 \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder \
  --reasoning-parser nemotron_v3

This recipe has no --max-model-len. That means the context length defaults to the 1,048,576 from config.json. Whether a 120B model with a 1M slot would actually start on a 128GB machine was the main thing I wanted to see.

The result: it started without any changes.

The startup log shows Maximum concurrency for 1,048,576 tokens per request: 1.15x. With only 15% headroom for a 1M slot, the value --gpu-memory-utilization 0.75 looks like it was carefully tuned to hit exactly this point. On the forums there is a report of someone who added more simultaneous images, causing insufficient KV allocation and a startup rejection (basbunarhasan, 2026-10-17). If you intend to use the full 1M context, it is safer not to add anything beyond the recipe. An NVIDIA staff member in the same thread also wrote: "this recipe assumes nothing else is running during inference; if something else is co-located, lower it to 0.7" (eugr_nv, 2026-10-16).

There is one number discrepancy in the card. The deployment summary table reads "1× DGX Spark / vLLM / 1M context / MTP with 5 draft tokens." The Quick Start command, however, uses LiLiCorr DFlash with 7 tokens. Since the card's descriptive text says "LiLiCorr is recommended on DGX Spark in all scenarios," I treated the Quick Start command as authoritative for my measurements. I did not test the built-in MTP this time.

There are three lines in the startup log worth verifying: Resolved architecture: LiLiCorrDraftModel confirming the draft was loaded, SpeculativeConfig(method='dflash', model='/draft', num_spec_tokens=7) for the speculative configuration, and the KV cache line above. Additionally, checking that /metrics' vllm:spec_decode_num_drafts_total increments with each request helps prevent the pitfall of thinking speculation is active when it is not.

Measurement Conditions

The speed section was measured with a single-sequence startup following the recipe verbatim. All other sections used a startup configuration aligned with the other models: --max-model-len 131072 --max-num-seqs 8 --max-num-batched-tokens 8192 --kv-cache-dtype fp8_e4m3, --gpu-memory-utilization 0.75, no speculation, same image. The value 131,072 was aligned to the max_seq_len in the model card's TensorRT-LLM Spark configuration. Startup took 607 seconds, and the KV cache held 2,783,188 tokens.

Sampling used the recommended values of temperature 1.0 / top_p 0.95, with a base budget of 8,192 output tokens; tasks that hit the limit have an additional row at 16,384. Image tasks were also measured at the text recommended values to match conditions with Bolt (the card's image recommendation is temperature 0.6).

Comparison targets are Bolt 74B-A9B 24GB (2-bit, llama.cpp) and Lightning 30B-A3B (NVFP4, vLLM) from the same Nemotron 3.5 family. Since Bolt was measured with llama.cpp, speed is not compared—only quality judgments are placed side by side. Thinking strength is high for both, budget also 8,192. Lightning's tool-calling and agent fitness were measured with a budget of 4,096, so those columns carry a note.

During measurement, I recorded GPU temperature, power, throttle reason from nvidia-smi, and the hottest zone in /sys/class/thermal every 5 seconds.

Speed

First, using the single-sequence startup with the verbatim recipe, I examine how well LiLiCorr works. Median of 3 runs with forced output of 256 tokens at temperature 0.

2.0×. The acceptance rate (the fraction of draft candidates accepted by the main model) can be read from /metrics deltas; for this short fixed text it was 43%, advancing an average of 3.0 tokens per verification step. Compared to Bolt's LiLiCorr acceptance rate of 35% and 1.56× speedup, Super VL shows higher acceptance rate and higher speedup.

I also measured behavior when feeding long inputs with the same startup. I constructed needle-test prompts at 16K, 32K, and 64K tokens, and had the model write 512 forced tokens with thinking off.

Prefill is stable at 2,100–2,200 tok/s, so reading 64K tokens takes 30 seconds before the first response. Decode was over 90 tok/s for 16K and 32K, faster than the 38 tok/s on short text. This is because the draft acceptance rate rose to 83–85%; needle tasks tend toward formulaic output, which is easy for the draft to predict. Please do not carry these decode numbers over as expectations for Japanese conversation.

A note on thermals. On the NVIDIA forums there is a report of the power cutting out when feeding inputs of 32K or longer with the same recipe (ronspie314, 2026-10-15). A follow-up noted that when the GPU was at 81–85 °C, the hottest zone in /sys/class/thermal showed 93–96 °C (basbunarhasan, 2026-10-17). On my own unit, through the entire 64K sequence, the GPU peaked at 77 °C, the zone peaked at 83.3 °C, and maximum power was 82 W, with no throttling events at all. Since I did not feed inputs longer than 64K, anyone planning to run long inputs continuously would do well to monitor zone temperature in addition to GPU temperature.

Concurrent execution was measured with the 8-sequence startup aligned to the other models, without speculation.

Single-stream is 19.18 tok/s, essentially the same as the 19.06 measured with the 1M startup without speculation. The fact that single-stream speed is unchanged whether the context window is 1M or 131K is worth noting. At 8 concurrent requests, total throughput is 79 tok/s with 10 tok/s per stream. Lightning achieved 79.59 tok/s single-stream and 331.97 tok/s total at 8 concurrent under the same conditions. The difference between active 3B and 12B comes through directly.

Japanese

I ran 10 Japanese tasks. Budget 8,192, thinking high. Scoring uses mechanically verifiable criteria such as character count, presence of required terms, and whether JSON parses successfully. The Bolt column carries the same values as the October 16 article.

The open-ended response cannot be scored mechanically and is excluded from the total.

The high score of 8/9 matches Bolt, and the failed task is the same: summary with proper nouns. After 11,432 characters of thinking, the 8,192 budget was exhausted with no body output. I also measured a row with the budget extended to 16,384, but this time 23,447 characters of thinking again exhausted the budget. Lightning and Bolt fail this task in the same way; Nemotron 3.5's high thinking seems to have a tendency to not stop on this particular task. The other 9 tasks showed no change in score at 16,384.

The picture changes when varying thinking strength.

9/9 with medium. The summary with proper nouns task that exhausted the budget on high was handled at medium with 6,360 characters of thinking, producing a 159-character summary retaining all 6 terms. Token consumption was 60% of high, time was 7 minutes, and the score was better. Bolt's medium achieved 8/9, retaining only 4 of 6 proper nouns, so Super VL's 12B made the difference here.

Disabling thinking lets all 10 tasks finish in 56 seconds, but the 50-character constraint is violated at 53 characters and one proper noun is dropped. The same pattern appeared with Bolt and Lightning—thinking is being used to follow instructions. Reading the open-ended responses by eye: medium and high both return two pros and two cons of remote work, organized with headings and bullet points; off returns the same content more briefly. All of them read as natural Japanese, a step above the "not broken" impression I had from the 2-bit Bolt—these read as genuinely normal prose.

Code Generation

Two parts: writing 5 programs from scratch given Japanese specifications, and fixing bugs in a mini-repository. Scoring is done by running pytest ourselves; the model's self-assessment is not used.

Writing from Specifications

All 5 problems, 49 tests passing. The result matches Bolt. parse_duration (converting strings like "1h30m" to seconds), which Lightning failed due to budget overflow, was solved in 150 seconds. Super VL and Bolt were not differentiated here.

Fixing Existing Bugs

A mini-repository where 5 of 12 tests are failing is handed over, and the model must fix it using only file read/write tools.

All three bring all 12 tests to passing. Super VL fixes everything in 6 turns, with 0 malformed calls (unparseable output). The 99.7 seconds reflects the single-stream 19 tok/s speed directly—about twice Bolt's time. The amount of progress per turn is similar to Bolt, and about half of Lightning's 13 turns.

Fitness as a Tool

I examine whether it can serve as a component in an agent. The tool-calling format is the same XML as Bolt and Lightning, and the vLLM side uses --tool-call-parser qwen3_coder to convert it to OpenAI-compatible JSON.

All 5 basic call tests passed. The task of finding bugs in a pseudo-filesystem also solved in 3 turns with 6 tool calls and 0 malformed calls. Both tool calls and thinking content appear properly in streaming chunks.

required stops at 1 call; two queries are batched into one message

The behavior most prominent in Lightning—returning the same call 159 times with tool_choice: required—did not appear. It accepts the constraint and stops at 1 call, same as Bolt.

For a query like "tell me the weather in Tokyo and Osaka," it returned both calls bundled in a single message. Lightning split them into 2 turns of 1 call each, so this bundled pattern appears to have been inherited from Super VL through to Bolt. One observation is not enough to characterize this as a fixed behavior, but in environments where tool round-trips are costly, this tendency is advantageous.

Agent Fitness Probes

I examine whether the model can be hosted on harnesses like opencode or Hermes Agent using 6 probes. To be clear up front: what is being measured here is the capabilities those harnesses depend on, not the results of running the harnesses themselves.

All 6 probes passed with medium. Per-harness judgment: all 3 harnesses—opencode, Hermes Agent, and OpenClaw—rated "usable." Lightning, Bolt, and Qwen3.8-27B measured on the same probes all rated "difficult" for all 3 harnesses under every condition, so this is the first time this rating has been achieved.

High at 8,192 failed only 2 of the tool selection questions. Not choosing a cron-creation tool for "schedule this to run every morning at 9" is the same miss as Bolt and Lightning. But with the budget extended to 16,384, it passed 6/6. With 40 tool definitions loaded, thinking becomes long, and at 8,192 the budget runs out before the answer can be written. Medium passes at 8,192 by the same logic: shortening the thinking leaves room in the budget to reach the answer.

Disabling thinking causes the "place a one-line summary at the end" rule in the long system prompt to be dropped, and the daily report format fluctuates in line count across 5 tries. This is the same breakdown seen with Bolt and Lightning. In the Nemotron 3.5 family, hosting on a harness with thinking disabled is not recommended.

Images

Super VL can also read images, so I ran the same 3 probes as Bolt, plus a Japanese document image task.

All 3 passed, with 0 hallucinations (naming things not in the image). The result matches Bolt; again no differentiation between Super VL and Bolt here.

For Japanese document images, I took 2 horizontal-text and 2 vertical-text pages from the synthetic OCR dataset JSSODa (CC-BY 4.0) and scored with Character Error Rate (CER). Budget 4,096, thinking high.

The 2 horizontal-text pages were transcribed without a single error. For the 2 vertical-text pages, roughly 8,000 characters of thinking exhausted the 4,096 budget before any body output appeared. To distinguish a budget issue from a capability issue, I re-measured the same 2 pages with thinking off and with budget 16,384.

Body text did appear in both cases, but it was not readable. Looking at the output, instead of reading each column top to bottom, it was picking up characters at the same height across columns. The result was text like picking one character correctly from each column but assembling them in the wrong order. Quadrupling the budget produced the same breakdown—just a longer wait, up to nearly 10 minutes. Horizontal Japanese text is readable; vertical text is read horizontally across columns. This is not a budget issue; it is about how this model reads. Anyone working with vertically written documents should treat this as a given.

Video input, which the model card mentions, was not tested this time.

How Super VL and Bolt Differ on the Same Tasks

I rearrange the scores so far across all three Nemotron 3.5 models. Since Bolt used llama.cpp and Lightning and Super VL used vLLM, speed is omitted and only quality judgments are compared.

Looking only at high-condition scores, Super VL and Bolt are almost identical: Japanese 8/9, code 5/5, tools 5/5, images 3/3. The fact that Bolt, compressed to 2 bits, matches Super VL this closely is the effect of QAD, which I described in the Bolt article, seen now from the Super VL side.

The difference appears with medium thinking and wider budgets. Super VL rises to Japanese 9/9 and agent fitness 6/6 at medium; Bolt stays at 8/9 and 4/6 regardless. The 2 tool-selection misses can be resolved in Super VL by providing a budget of 16,384; Bolt missed the same 2 questions at all three thinking levels (I did not measure Bolt at 16,384). My reading is that the gap between active 12B and 9B is invisible on easy tasks, and shows up on tasks involving selection from many options or summarization while preserving proper nouns.

The trade-off is that Super VL is slow. Single-stream is 38 tok/s with speculation, 19 tok/s without. It does not reach Bolt's 61.61 tok/s on llama.cpp or Lightning's 79.59 tok/s. Resident memory is also close to 100 GiB, which means dedicating the entire DGX Spark to this model. If you want a single smart executor, Super VL; if you prioritize speed and co-location, Bolt; if you need to share with several users, Lightning. Measuring Super VL reaffirmed my view that even within Nemotron 3.5, the right choice depends on where you are deploying it.

Summary

Nemotron 3.5 Super VL 120B-A12B NVFP4 ran with the DGX Spark recipe from the model card exactly as written, starting up with the default 1M context in 11 minutes. LiLiCorr speculative decoding delivers 38.07 tok/s single-stream, and 19.06 tok/s without it. For inputs from 16K to 64K tokens, prefill runs at 2,100–2,200 tok/s, and time to first token for 64K input is 30 seconds.

Quality was best with thinking set to medium: Japanese 9/9, code 5 problems 49 tests all passing, tool calling 5/5, all 6 agent fitness probes passed, 3 harnesses rated "usable." This is a result that no other model measured on the same tasks had reached, and the active 12B advantage shows up on tasks with many choices. The default high exhausts its budget on some tasks within 8,192 tokens, so I recommend medium for both conversational use and harness deployment.

Things it could not do are also worth noting. Vertical Japanese text was read horizontally across columns, and this did not improve with more budget. Built-in MTP and video input were not tested.

Next is actually using that 1M context window. There are already forum reports of successfully retrieving a keyword from a 1M-token context, so I would like to measure TTFT when feeding documents of several hundred thousand tokens while keeping an eye on zone temperatures.

Reference Links

• NVIDIA Announcement Blog (Super VL and Bolt) (2026-10-15)
• NVIDIA Developer Blog Technical Overview — Optimizations for DGX Spark and the role of LiLiCorr
• nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4 — Model card. DGX Spark recipes, official benchmarks, and deployment destination list are here
• nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4-LiLiCorr — Draft for DFlash speculative decoding
• nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16 — Base model before quantization
• vLLM Support Blog (2026-10-15)
• NVIDIA Developer Forums: Speed and thermal reports on a single DGX Spark (ronspie314, from 2026-10-15)
• NVIDIA Developer Forums: vLLM configuration for Spark and LiLiCorr thread (notmy.reward438, from 2026-10-15. NVIDIA staff co-residency note is here)
• LiLiCorr (arXiv 2608.20530)
• Unsloth Nemotron 3.5 Super VL Guide — GGUF versions (UD-Q3_K_XL 65.8 GB, etc.) and llama.cpp launch examples
• OpenMDW-1.1 License
• Article measuring Nemotron 3.5 Bolt 74B-A9B on DGX Spark — Bolt figures in the comparison table were measured here
• Article measuring Nemotron 3.5 Lightning on DGX Spark — Lightning figures in the comparison table were measured here