
I tried NVIDIA Nemotron 3.5 Lightning 30B-A3B-NVFP4
This page has been translated by machine translation. View original
Introduction
Hello, I'm Mori Shigeru from Classmethod's Manufacturing Business Technology Division.
NVIDIA has released Nemotron 3.5 Lightning. It's a 30B MoE with 3B active parameters per token and a 1M token context.
Up until now, I've been following the Nemotron family from Nano to Super and Ultra on DGX Spark. The previous Ultra was 550B, and even in NVFP4 the weights were 335GB, which couldn't even fit in the 128GB DGX Spark.
This time, Lightning is the opposite. The model card includes a dedicated launch recipe for "1x DGX Spark (GB10)", and a draft specifically tuned for DGX Spark for speculative decoding was released at the same time. The vendor has named the machine at hand and written "run it this way." This is something I had to try.
To state the conclusion upfront, the speed was as advertised. With speculative decoding loaded, 115.75 tok/s standalone, and a combined 421.85 tok/s for 8 concurrent requests. That's 27% faster standalone than the previous generation with the same architecture, and the KV cache pool is 24 times larger than Gemma 4's. However, this speed comes with prerequisites for usage. This model thinks a lot, and if you're stingy with the token budget, the thinking alone will exhaust the budget and the main output won't appear. Opening it up to NVIDIA's suggested value (16,000) brings out its full strength. "Fast, but a model with prerequisites for usage" is my honest impression.
There's also another prerequisite I want to establish before measuring. NVIDIA is not releasing this model as a "one-machine-does-everything model." In developer-facing documentation, roles are clearly divided.
Frontier reasoning models such as Nemotron 3 Ultra handle orchestration and complex planning, while smaller, more efficient models handle the high-volume execution layer.
What Lightning is expected to handle is high-frequency execution such as tool calls, result validation, sub-agent delegation, and formatting. Planning goes to upper-tier models, execution goes to Lightning. The role that allocates this is NeMo Switchyard, announced at the same time. Keeping this premise in mind changes how the measured results look quite a bit.
In this article, I'll introduce the results of running Nemotron 3.5 Lightning on DGX Spark on the day of its release, measuring speed, Japanese, code generation, and tool calling under the same conditions.
What Nemotron 3.5 Lightning Is
First, let me establish what it is from primary sources. Numbers are taken from the model card and config.json.
| Item | Value |
|---|---|
| Parameters | 30B (3B active per token) |
| Architecture | Hybrid of Mamba-2 + MoE + Attention |
| Layer structure | 52 layers. Routed experts 128 + shared 1, top-6 |
| Context | 1M tokens |
| Vocabulary | 131,072 |
| Training | Over 20T tokens (pre-training cutoff September 2025, post-training May 2026) |
| License | OpenMDW-1.1 (commercial use permitted) |
| Recommended sampling | temperature 1.0 / top_p 0.95 |
Looking at the layer arrangement via layers_block_type in config.json, mamba and moe alternate, with attention inserted in only 6 places. Out of 52 layers, only 6 are attention layers. This ratio matters later.
When I opened the previous generation's config.json to choose a comparison target, the relationship was closer than I expected.
| Item | Nemotron 3 Nano 30B-A3B | Nemotron 3.5 Lightning 30B-A3B |
|---|---|---|
| Architecture | NemotronHForCausalLM |
NemotronHForCausalLM |
| Number of layers | 52 | 52 |
| Hidden dimension | 2,688 | 2,688 |
| Routed experts | 128 (top-6) | 128 (top-6) |
| Vocabulary | 131,072 | 131,072 |
| Context | 262,144 | 1,048,576 |
The skeleton is unchanged, with only the context being 4 times larger. It's a perfect comparison target for seeing where the generational difference appears. In this article, I'll compare these two side by side under the same vLLM, same tasks, and same conditions.
The available checkpoints are not just one.
| Repository | Contents |
|---|---|
...-BF16 |
Main model (BF16) |
...-NVFP4 |
Main model (NVFP4 quantized). This is what I measured |
...-Base-BF16 |
Base model before post-training |
...-NVFP4-DFlash |
Draft for speculative decoding (DFlash method) |
...-NVFP4-DSpark |
Draft for speculative decoding (DSpark method) |
The last two are drafts, not the main model. The DSpark one has 967M parameters, and the model card states it's "for DGX Spark and low-concurrency data center GPUs."
The model card provides a comparison table with BF16 for quantization degradation.
| Benchmark | BF16 | NVFP4 |
|---|---|---|
| MMLU Pro | 81.94 | 81.62 |
| GPQA Diamond | 75.44 | 75.57 |
| SWE-bench Verified | 51.56 | 52.80 |
| Terminal-Bench 2.1 | 24.58 | 23.46 |
| PinchBench | 85.37 | 83.43 |
| AA-LCR (long context) | 52.00 | 49.19 |
Other than AA-LCR for long-context reading comprehension dropping by 2.8 points, the rest are within margin of error, and SWE-bench even scores higher for NVFP4. As someone who regularly uses the quantized version, these numbers are reassuring.
Getting It Running
It ran surprisingly easily.
Until now, when running a new model on DGX Spark, the standard approach was to either build an arm64 image yourself or find one ported for GB10 by the community. This time, the model card specifies vllm/vllm-openai:v0.27.1, and an arm64 version is available in the official image. There's also a tag v0.27.1-aarch64, which was pushed on the same day the model was released.
The launch command can also be used almost verbatim from the "1x DGX Spark (GB10)" block in the model card.
vllm serve --model $MODEL_CKPT \
--moe-backend marlin \
--kv-cache-dtype fp8 \
--max-model-len 1048576 \
--enable-prefix-caching \
--speculative_config.num_speculative_tokens 3 \
--mamba-backend flashinfer \
--mamba-cache-mode align \
--reasoning-parser nemotron_v3 \
--speculative_config.method dspark \
--tool-call-parser qwen3_coder \
--enable-auto-tool-choice
--moe-backend marlin and --mamba-backend flashinfer are the GB10-specific specifications. For this article, I lowered --max-model-len to 65536 to match conditions with other models, and separated speculative decoding as a separate condition. --kv-cache-dtype was left to the engine (looking at the startup log, fp8_e4m3 was selected even without specifying it — this seems to be determined by the model's quantization settings).
The Draft Needs to Be Explicitly Named
There's one point that doesn't work as-is. Running the above command stops with this error:
ValueError: Model config must specify `dflash_config.mask_token_id`, `mask_token_id`,
`dspark_noise_token_id`, `pard_token`, or `ptd_token_id` for parallel drafting.
This is because the speculative decoding method (dspark) was specified without specifying which draft to use. Without a specification, vLLM selects the MTP head bundled with the main model as the draft, but it stops because it doesn't have the dedicated token IDs required by the DSpark method.
Naming the draft explicitly works:
--speculative_config.method dspark \
--speculative_config.model nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4-DSpark \
--speculative_config.num_speculative_tokens 3
All speculative decoding numbers in this article were measured using this approach.
Here are the measured values from download to startup:
| Item | Value |
|---|---|
| Actual weight size | 21.58 GB (NVFP4) |
| Draft (DSpark) | 1.35 GB |
| Weight loading | 144.1 seconds (17.62 GiB) |
| Engine initialization | 92.2 seconds (including 7.1 seconds for compilation) |
| Total startup | approximately 277 seconds |
When I started Muse Glimmer with vLLM it took 371 seconds, so this is about 1.3x faster to start.
The KV Cache Is Orders of Magnitude Larger
What caught my eye in the startup log was this:
GPU KV cache size: 18,448,384 tokens
Here it is alongside other models measured on the same DGX Spark with the same --max-model-len 65536 / --gpu-memory-utilization 0.85:
| Model | KV pool (tokens) | Lightning ratio |
|---|---|---|
| Nemotron 3.5 Lightning 30B-A3B | 18,448,384 | 1.00 |
| Nemotron 3 Nano 30B-A3B | 18,298,197 | 0.99 |
| Muse Glimmer 30B | 4,819,435 | 0.26 |
| Qwen3.6-27B | 2,140,842 | 0.12 |
| Gemma4-31B | 760,685 | 0.04 |
24 times Gemma4, 3.8 times Muse Glimmer. The reason is clear from config.json. Of the 52 layers, only 6 are attention layers, and the rest are Mamba-2 and MoE. Since only attention layers hold KV cache, the layer ratio directly translates to pool size.
There are other models that claim 1M token context, but whether there's enough memory to actually fit 1M is a different question. With this pool, you can mathematically hold more than 17 simultaneous 1M-token contexts. This shows that the advertised 1M is realistic, at least in terms of capacity.
Note that Lightning isn't special in this regard. The previous-generation Nemotron 3 Nano 30B-A3B with the same architecture also had nearly the same 18,298,197 tokens. This is a property of the Mamba hybrid design.
Measurement Conditions
Unless otherwise noted, all figures below were measured with the same vLLM (v0.27.1-aarch64), same max length (65,536), and same number of slots (8). Only the SGLang comparison used a dedicated image (dev-nemotron3-5-lightning), with max length and slot count aligned to the vLLM side. Since both models are loaded on separate DGX Sparks, speed comparisons are contained within the same machine.
Sampling uses each model's official recommended values. Lightning uses temperature 1.0 / top_p 0.95, and Nano uses temperature 1.0 / top_p 1.0. Only speed measurements use temperature 0 as a fixed condition for direct comparison, but please note that the Japanese and code comparisons carry the difference in recommended values.
The token budget default is 16,000. The sample code in NVIDIA's model card uses this value, and the model is designed with this premise. For tables comparing models side by side, I used values measured at 4,096 to match previous articles. This model has long thinking, so its true capability doesn't show with a narrow budget (more on that later).
Speed
I measured while varying concurrency with forced length 256 tokens at temperature 0.
| Concurrency | tok/s per request | Total tok/s | TTFT (seconds) |
|---|---|---|---|
| 1 | 79.59 | 79.59 | 0.071 |
| 2 | 65.49 | 120.58 | 0.331 |
| 4 | 55.07 | 209.23 | 0.239 |
| 8 | 43.00 | 331.97 | 0.212 |
Here it is alongside other models measured on the same DGX Spark with the same harness:
| Model | C=1 tok/s | C=8 total tok/s |
|---|---|---|
| Nemotron 3.5 Lightning 30B-A3B | 79.59 | 331.97 |
| Qwen3.6-27B (NVFP4) | 12.41 | 84.75 |
| Muse Glimmer 30B (NVFP4) | 11.67 | 88.61 |
| Gemma4-31B (NVFP4) | 6.84 | 53.56 |
6.8x Muse Glimmer, 11.6x Gemma4.
NVIDIA describes this model as "up to 4x output speed compared to models of equivalent size." The measured results at hand exceeded that claim. However, this is not a fair comparison. The bottom three are dense models that pass through all 30B weights to generate each token. Lightning is MoE, so only 3B worth actually runs. Even if it's "30B class," if only one-tenth is actually running, it's naturally going to be faster.
In fact, Qwen3.6-35B-A3B, another MoE with Active 3B, measured 76.7 tok/s without speculative decoding under different conditions in a previous test. This is roughly on par with Lightning's 79.59 tok/s here (please treat this as a reference value since the vLLM version and KV settings differ). Honestly, most of the speed comes from the "MoE with Active 3B" design, not any Lightning-specific magic.
That said, for those using DGX Spark, "which is fastest among models that use similar memory" is a practical question. In that sense, this difference directly translates to a perceptible one.
Interesting is the TTFT (time to first token): 0.071 seconds at C=1. This is 5.7x faster than Muse Glimmer's 0.406 seconds, making the "response received" feeling in conversation completely different. Moreover, even scaling up to 8 concurrent requests keeps it at 0.212 seconds — you're not waiting longer with 8 requests than with 1.
The 43.0 tok/s per request at 8 concurrent is faster than other models' standalone speed (6.8–12.4 tok/s). When sharing one machine among a team, this difference becomes even more impactful.
Adding DSpark Speculative Decoding
This is where things get interesting. Along with the Lightning main model, NVIDIA released a draft for speculative decoding tuned for DGX Spark. Since it's included from the start in the official GB10 recipe, this loaded configuration is the intended setup.
The draft is a small 967M model that pre-generates candidates, which the main model then verifies. If the predictions are correct, multiple tokens advance in one computation, making it faster.
| Concurrency | Without speculation | With DSpark | Multiplier |
|---|---|---|---|
| 1 | 79.59 | 115.75 | 1.45 |
| 2 | 65.49 | 114.15 | 1.74 |
| 4 | 55.07 | 74.52 | 1.35 |
| 8 | 43.00 | 58.74 | 1.37 |
| 8 (total) | 331.97 | 421.85 | 1.27 |
Reached 115.75 tok/s standalone and a combined 421.85 tok/s for 8 concurrent requests. The memory increases from 17.62 GiB to 19.14 GiB for the draft, but this is negligible given 128GB of unified memory.
I also confirmed that quality hasn't degraded. Running the Japanese 10 tasks again with speculative decoding, all 10 results matched the judgments without speculative decoding. I also re-ran with the model's intended budget of 16,000, and Japanese (8/9), code generation (5 problems fully solved, all 49 tests passing), and all 6 agent fitness probes matched the non-speculative judgments. Speculative decoding only "outputs faster" without changing the output — exactly as the theory says.
Note that the draft's claimed performance on SPEED-Bench is an acceptance length of 3.75 (7-step lookahead). At acceptance length 3.75, theoretically up to 3.75x speedup is possible, but the measured value at hand was 1.45x. The following warning appeared at startup:
max_num_scheduled_tokens is set to 2032 based on the speculative decoding settings.
This may lead to suboptimal performance. Consider increasing max_num_batched_tokens
to accommodate the additional draft token slots, or decrease num_speculative_tokens
or max_num_seqs.
This says that enabling speculative decoding limits the number of tokens that can be handled in one scheduling pass.
3 Lookahead Steps Seems Best
The model card specifies 3 for num_speculative_tokens, but the draft's config.json states block_size as 8. Curious whether more steps would be faster, I also measured with 8.
| Setting | C=1 speed | Japanese (automated judgment) |
|---|---|---|
| Lookahead 3 (model card value) | 115.75 | 8/9 |
Lookahead 8 (matching block_size) |
93.54 | 7/9 |
3 is faster. Setting to 8 drops standalone from 115.75 to 93.54 tok/s. The more lookahead steps, the greater the waste when predictions miss, so 3 seems to be the sweet spot for this model and hardware. It seems best to simply use the model card's recommended value.
DSpark Also Works with SGLang
DSpark isn't exclusive to vLLM. The SGLang section of the model card only shows an EAGLE configuration, but looking at the official cookbook, there's a DSpark startup command, and a community recipe (MiaAI-Lab) wrapping it with DGX Spark-specific parameters was also published on the day of release. The SGLang side uses a dedicated image lmsysorg/sglang:dev-nemotron3-5-lightning, with the arm64 version also pushed on the same day as the model.
The speculative configuration requires just 3 flags. Unlike vLLM, the draft specification is a separate mandatory flag, so the earlier problem of forgetting to specify the draft and crashing is structurally prevented.
--speculative-algorithm DSPARK \
--speculative-draft-model-path nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4-DSpark \
--speculative-dspark-block-size 3
I compared vLLM and SGLang side by side on the same machine under the same conditions (max length 65,536, 8 slots).
| Concurrency | vLLM + DSpark | SGLang + DSpark |
|---|---|---|
| 1 | 115.75 | 113.89 |
| 2 | 114.15 | 90.30 |
| 4 | 74.52 | 70.49 |
| 8 (total) | 421.85 | 377.63 |
Standalone performance is nearly identical (1.6% difference). The DSpark effect is the same regardless of which stack is used. vLLM is about 10% ahead on the parallel side. Spot-checking Japanese tasks also showed the same judgments as vLLM, except for one long-thinking problem that consumed the 4,096 budget.
The difference appeared in memory design. Under the same conditions, the KV cache pool is approximately 18.45 million tokens for vLLM versus approximately 5.44 million tokens for SGLang — a 3.4x gap. SGLang separately allocates KV cache for the draft in bf16, which reduces the main model's pool. Even with the recipe's default 1M context setting, it was approximately 5.05 million tokens with 48 simultaneous requests. This matters if you want to utilize 1M.
What was valuable about measuring with SGLang was the acceptance length. SGLang outputs acceptance length and acceptance rate in the log every batch, allowing direct verification of "how often the draft is right" — something not visible in vLLM. Over 60 samples during Japanese task generation, the median acceptance length was 3.27 (the upper limit for 3-step lookahead is 4.0), and the median acceptance rate was 0.76. The draft itself hits well enough. This serves as circumstantial evidence that the main reason the local multiplier is more modest than claimed is the scheduling limitation mentioned earlier, not the draft's accuracy.
Japanese
I submitted 10 Japanese tasks. The budget is 16,000 to match NVIDIA's recommendation. Judgment is automated, scored on mechanically verifiable conditions such as character count, presence of required terms, and whether JSON parses correctly.
| Task | Lightning | Tokens consumed | Thinking length | Nemotron 3 Nano |
|---|---|---|---|---|
| Answer within 50 characters | ○ | 1,402 | 3,790 | ○ |
| Free-form writing | ○ | 704 | 1,319 | ○ |
| Code generation | ○ | 523 | 1,785 | ○ |
| Numerical reasoning | ○ | 1,777 | 5,043 | ○ |
| Vocabulary constraints | ○ | 3,249 | 5,815 | ○ |
| JSON formatting | ○ | 444 | 1,139 | ○ |
| Summarization with proper nouns | × | 12,112 | 20,678 | × |
| Transcribing a table | ○ (5/5) | 5,286 | 12,411 | ○ (5/5) |
| Converting to polite speech | ○ (5/5) | 1,125 | 2,260 | ○ (5/5) |
| Extraction from long text | ○ (4/4) | 1,571 | 4,110 | ○ (4/4) |
| Automated total | 8/9 | 8/9 |
Free-form writing cannot be scored automatically and is excluded from the total. The Nano column was measured at a budget of 4,096 for side-by-side comparison (thinking is short there, so no cutoff occurs at that budget).
8 out of 9 mechanically scorable problems were correct. The failed one was the task of summarizing while retaining proper nouns — after thinking for 20,678 characters, it failed to retain a single one of the 6 proper nouns. The budget still had room (12,112 tokens), so it's not a matter of insufficient space — this task is simply a weakness.
This is where the view changed by varying the budget. Measuring at the side-by-side 4,096, the same task fails with "thinking consumes the entire budget, 0 characters of main output." The total is 8/9 either way, but the content is completely different. Measuring with a narrow budget makes it impossible to distinguish between being bad at the task and running out of space.
What clearly stood out was the amount of deliberation before reaching an answer. Lightning thinks 3–6 times more on the same problem compared to Nano. On a problem requiring a 50-character answer, Nano uses 385 tokens while Lightning uses 1,402; for table transcription, it's 854 versus 5,286.
However, since Lightning generates 1.3x faster overall, the actual wait time doesn't widen as much as the token count ratio suggests. It thinks more and consumes more tokens, but generates fast enough that you don't wait proportionally longer — that seems to be this model's character.
Code Generation
Two parts: 5 problems written from scratch from Japanese specifications, and a task of fixing a mini repository with bugs. Scoring is done by running pytest ourselves, not the model's self-report.
Writing from Scratch
| Problem | Lightning | Thinking length | Nemotron 3 Nano | Thinking length |
|---|---|---|---|---|
| slugify | ○ 11/11 | 3,702 | × 9/11 | 1,861 |
| parse_duration | ○ 16/16 | 17,582 | × 0/16 | 4,226 |
| merge_intervals | ○ 8/8 | 4,593 | ○ 8/8 | 331 |
| group_by_month | ○ 6/6 | 3,964 | ○ 6/6 | 1,236 |
| top_k_frequent | ○ 8/8 | 5,535 | ○ 8/8 | 1,355 |
| Total | 5/5 | 3/5 |
All problems correct, all 49 tests passed. The previous generation scored 3/5, so this is where the generational difference clearly shows.
Notable is parse_duration (parsing strings like "1h30m" into seconds), which was solved after thinking for 17,582 characters. When measured at the side-by-side 4,096, this was the only problem where thinking reached 13,091 characters and hit the budget limit, producing zero lines of code. It was simply a problem that required over 10,000 characters of thinking, and the narrow budget couldn't reach it.
Nano produced an answer in 1,438 tokens but all 16 tests failed. One ran out of thinking space, the other answered before thinking enough — a strikingly contrasting mode of failure.
The difference in thinking length is also stark. While Nano answers in 331–4,226 characters, Lightning uses 3,702–17,582 characters. Even with the same skeleton, the successor is designed to think more — and the numbers show it.
Fixing Existing Bugs
A task of passing a mini repository with 5 failing tests out of 12, and fixing it using only file read/write tools.
| Model | Fixed | Turns | Tool calls |
|---|---|---|---|
| Nemotron 3.5 Lightning | 5/5 | 13 | 12 |
| Nemotron 3 Nano 30B-A3B | 5/5 | 13 | 13 |
This was a tie. Both bring all 12 tests to passing. On tasks involving using tools to make fixes, both generations perform reliably.
Fitness as a Tool
I examine whether it can serve as an agent component. Reading the chat template, the tool call format is XML rather than JSON.
<tool_call>
<function=get_weather>
<parameter=city>
Tokyo
</parameter>
</function>
</tool_call>
This is the same format as Qwen3-Coder, and --tool-call-parser qwen3_coder is specified on the vLLM side.
All 5 basic call tasks passed.
| Task | Result | Content |
|---|---|---|
| Simple call | ○ | {'city': 'Paris'} |
| Selection from multiple candidates | ○ | Selects calculation tool with {'expression': '1847 * 362'} |
| Nested arguments | ○ | Correctly fills title / priority / assignee |
| Japanese argument | ○ | {'keyword': 'ほうじ茶ラテ'} |
| Question not requiring a tool | ○ | Answers without calling |
A task of passing a pseudo filesystem to find bugs was also solved in 6 turns with 5 tool calls. Zero broken calls (unparseable output). In streaming mode, both tool calls and thinking arrive in chunks.
Setting required Causes an Infinite Loop
There was one clear problem. Sending tool_choice: required to force a tool call results in the same tool call being returned 159 times.
| Metric | Value |
|---|---|
| Items returned | 159 (all get_weather) |
| finish_reason | length (4096 budget exhausted) |
| Time elapsed | 53.76 seconds |
| Thinking length | 474 characters |
The instruction itself is followed (tools are called). However, instead of stopping after one call, it keeps emitting the same call until the token budget runs out. Since it keeps outputting with almost no thinking (474 characters), it appears to have lost track of when to stop generating.
I've seen this behavior in other models too. Muse Glimmer and Gemma4-31B don't error on required but return 0 results. The direction is opposite, but both represent "the client's contract silently breaking." When writing clients that use required, there's no choice but to test each model on real hardware.
To check whether this runaway behavior is Lightning-specific, I measured Nemotron 3 Nano 30B-A3B, the previous generation with the same architecture, using the same vLLM, same parser, and same task. The result was 150 items with finish_reason: length — the same failure to stop. It seems to be a problem with the combination of Nemotron-series models and the qwen3_coder parser, not model-specific.
How Multiple Tools Are Called Changed Between Generations
There was also another point of divergence from the previous generation. For a question like "Tell me the weather in Tokyo and Osaka" that requires querying two locations, the calling pattern was reversed.
| Model | Calling pattern | Turns |
|---|---|---|
| Nemotron 3 Nano 30B-A3B | 2 calls in one message | 1 |
| Nemotron 3.5 Lightning | 1 call at a time, split over 2 turns | 2 |
Both ultimately reach both cities, so it's not a correctness issue. However, in environments where tool calls take time, the previous generation that can batch them together might be advantageous in some situations. When I measured Muse Glimmer previously, it sometimes called one at a time and sometimes batched 4 together depending on the situation, so it's best not to characterize a model's nature from a single observation.
Suitability as Agent Components
We examine suitability for loading into harnesses like opencode or Hermes Agent using six probes. Note upfront that what we are measuring here are the capabilities that each harness depends on, not the results of running the harness itself.
| Probe | Lightning | Nemotron 3 Nano |
|---|---|---|
| Correctly selecting from 40 tools | 5/6 | 5/6 |
| Rule compliance with long system prompt | ○ | ○ |
| Fixed format reproduction (5 times) | Undecidable | ○ 5/5 stable |
| Response language locking | ○ (3/3) | ○ |
| Suppression of excessive tool calls | ○ (2/2) | ○ |
| Multi-turn exchanges | ○ | ○ |
For tool selection, we load 40 tool definitions and issue 6 requests, checking whether the expected tool is selected. Both generations missed the same one question. For "Set this to run every morning at 9," it calls the directory listing tool instead of the cron creation tool. Sharing the same skeleton, they share the same weakness.
Where things stumbled was fixed format reproduction. The task involves having the model write a daily report template 5 times to see if it produces the same structure each time, but all 5 attempts took 53.2 seconds with 0 characters of body text. Here again, thinking is consuming the entire budget.
We tried two conditions here. Thinking that the thinking process was interfering, we tried disabling enable_thinking, and conversely, we tried expanding to the budget of 16000 used in NVIDIA's own sample code.
| Condition | Fixed format | Long system prompt compliance | Time per attempt |
|---|---|---|---|
| thinking ON / budget 4096 | 0/5 (budget exhausted) | ○ | 53.2 sec |
| thinking ON / budget 16000 | 10/15 (5 times × 3 sets) | ○ | 34–207 sec |
| thinking OFF / budget 4096 | 1/5 (structure unstable) | × | 1–2 sec |
| Previous gen Nano (ON / 4096) | 5/5 | ○ | 12–40 sec |
Expanding the budget to NVIDIA's intended level improved things from 0/5 to 3–4/5. It was indeed a budget issue. However, this does not reach the level of "writing in the same form every time." Running 3 sets of 5 attempts each yielded 3/5 · 3/5 · 4/5, totaling 10/15. In every set, the structure broke down 1–2 times. Time required also ranged from 34–207 seconds, a 6-fold spread.
With thinking disabled, responses came back in 1–2 seconds, but the structure was unstable and long system prompt rules were no longer followed. Ironically, it seems thinking was being used to comply with instructions.
For applications that continuously produce fixed-format documents unattended, the previous-generation Nano, which was stable all 5 times under the same conditions, is still more reliably trustworthy.
Recalling the premise from the beginning changes how this looks. NVIDIA's intended role for Lightning was execution work: calling tools, verifying results, and passing them along. Measured by that standard, tool calls 5/5, pseudo-filesystem exploration, and bug fixes 5/5 show it properly handles execution-layer work. What it dropped was "continuously writing a daily report template in the same structure every time" — a task closer to paperwork.
NVIDIA also states they "trained it for harnesses like OpenClaw and Hermes Agent." Since what we are measuring here are probes of the capabilities each harness requires rather than the results of actually running the harness, behavior when actually loaded needs to be verified separately. That remains homework for a follow-up.
The Next Battlefield for the 30B Class Looks Like Specialized Agents
Everything discussed so far can be distilled to a single point: viewing models as "sources of specific capabilities." Extending this view, the models you line up are unlikely to remain a single vertical column ranked by intelligence alone. Right around the time this article was being written, Fastino, in collaboration with NVIDIA, released models with specialized training of Nemotron 3.5 Lightning for finance and healthcare respectively, under Apache 2.0. The base is a lightweight MoE (Mixture of Experts) model with 30B parameters and 3B active, but Fastino's announcement claims the financial benchmark FinQA moved from 15.9% to 59.2%. It is a real example of a lightweight model matching large models in a narrowed domain. This is a use case only possible with open models where weights are publicly available, and this approach will certainly increase in industries with strong privacy and regulatory requirements.
What becomes interesting then is the idea of treating a business domain or a company itself as one large model. Specialized lightweight models serve as experts within it, and the routing role corresponds to the gating that decides which expert to call. It is the MoE structure being externalized from inside a single model to the architecture of the entire system. Looking at the combination of Nemotron 3.5 Lightning as a "lightweight execution handler" and Switchyard as a "supervisor routing work," NVIDIA may have already moved beyond the competition of "choosing the single strongest model" and into agent design that decomposes planning, execution, and verification across separate models.
Like the Muse Glimmer 30B recently released by Meta Superintelligence Labs as open weights, it is becoming difficult for models of this size to compete at the top in overall intelligence comparisons. The competitive ground has shifted to "which domain to become an expert in," and the next battlefield for the 30B class looks set to be their role as specialized agents. The day when domain-specialized Lightning models line up in the DGX Spark at hand doesn't feel too far off.
Summary
Nemotron 3.5 Lightning clearly stands out in speed as a 30B-class model for the DGX Spark. 79.59 tok/s standalone without speculative decoding, 115.75 tok/s with DSpark loaded, and a combined 421.85 tok/s for 8 simultaneous instances. The response speed of 0.071 seconds to the first token also makes a difference when used in conversation. With a KV cache pool of 18 million tokens, the headline 1M context was realistic in terms of capacity as well.
There are quirks to handling it. It is a model that thinks a great deal, and with a narrow token budget, it never reaches the body text. When measured at 4,096 for side-by-side comparison, thinking consumed the entire budget on 3 tasks. Conversely, expanding to NVIDIA's intended 16,000 eliminated cutoffs, code generation achieved a perfect 5/5 (all 49 tests passed), and fixed format improved from 0/5 to 10/15. The correct understanding was that given enough budget, it delivers.
Even so, fixed format breaks in structure 1–2 times out of every 5. Yet disabling thinking causes it to stop following 50-character constraints or JSON formatting. It is all-or-nothing, with no knob on the model side to dial in a middle ground. At first, this seemed like a weakness.
However, I think the premise of having a single model handle everything was wrong to begin with. NVIDIA's division of labor is: complex planning for upper-tier models like Nemotron 3 Ultra, high-volume execution for Lightning. Rather than varying reasoning intensity within a single model, the design switches models per task. The routing role is NeMo Switchyard, and LangChain validation showed that routing Lightning and Claude Opus 4.8 kept only 7% of requests going to the upper-tier model, producing a 74% cost reduction figure.
With this division of labor as the premise, the actual measurements fell neatly into place. Tasks involving calling tools and verifying results had no misses, code generation was 5/5, and speed was 27% faster than the previous generation. As a "handler for rapidly cycling through large volumes of routine work," what arrived was what was expected. Conversely, for work like continuously producing fixed-format documents unattended, the previous-generation Nano with the same skeleton was more stable. Selecting by use case is the correct way to read this model.
NVIDIA's claim of "up to 4x output speed compared to the same class" also came in at 6.8–11.6x against dense 30B-class models in our testing — actually a conservative figure. Regarding the claim of being 30% faster than Qwen3.6 35B in agent task completion time, since raw generation speed was roughly on par, any difference would likely come from harness optimization effects. That was not measured this time.
What I want to do as a follow-up is exactly that combination with Switchyard. Since I previously tested Switchyard standalone, setting up a configuration with Lightning as the execution layer routing with upper-tier models should allow the speeds measured in this article to directly take effect. Filling the 1M context in practice and measuring performance at that point also remains as homework.
Reference Links
- NVIDIA Nemotron 3.5 Lightning announcement blog (2026-08-11)
- Developer explanation of Nemotron 3.5 Lightning — Positioning as execution layer and harness optimization explanation here
- Routing agent workloads across models with NeMo Switchyard — Explanation from the side that divides use with upper-tier models
- Specialized finance and healthcare models released by Fastino based on Lightning — Apache 2.0. A real example of how far domain specialization can reach
- nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 — The subject of this measurement. DGX Spark startup recipe also here
- nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4-DSpark — Draft for speculative decoding for DGX Spark
- nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-NVFP4 — Previous generation used for comparison
- NVIDIA-NeMo/Nemotron vLLM cookbook — Speculative decoding settings are more detailed here than the model card, but the description of lookahead steps differed from the actual machine
- NVIDIA-NeMo/Nemotron SGLang cookbook — The DSpark flag for SGLang is not in the model card; it is listed here
- MiaAI-Lab SGLang recipe for DGX Spark — Starting point for SGLang verification this time. Includes memory settings for GB10
- vLLM v0.27.1 release (2026-08-11) — Includes support for quantized DSpark head
- OpenMDW-1.1 License
- Article measuring Muse Glimmer 30B on DGX Spark — Muse Glimmer / Gemma4-31B figures in the comparison table were measured here
- Article measuring Qwen3.6-35B-A3B-NVFP4 on DGX Spark — Comparison with the same Active 3B MoE
