A new champion among partner LLMs for DGX Spark? Comparing Qwen3.6-35B-A3B-NVFP4 in two quantized versions

Introduction

Hello, I'm Morishige from Classmethod's Manufacturing Business Technology Department.

I just recently wrote an article thinking about model configurations to keep resident on a single DGX Spark, and Qwen3.6-35B-A3B, which differs in parameter count from the star of that article, Qwen3.6-27B, has been getting a lot of attention lately. Unsloth is pushing a new NVFP4 quantized version claiming it's "2.5x faster," while the NVIDIA forums have seen pushback saying "actually, when measured empirically, the official NVIDIA version is faster." Many of you are probably wondering which to believe.

https://huggingface.co/unsloth/Qwen3.6-35B-A3B-NVFP4

My interest is less in the debate itself and more in whether this 35B-A3B can replace the "partner for a single DGX Spark" position. In my previous article, the single fastest was Nemotron Nano plus an external drafter added to Gemma at 61–62 tok/s, while the balanced pick for smarts and image input was Qwen3.6-27B at 29 tok/s. The 35B-A3B is a MoE with Active 3B, comes bundled with MTP, and also supports image input — specs that could unify the speed slot and balance slot into one. Since vLLM's new stable v0.25.0 just came out, I downloaded both the NVIDIA version and the Unsloth version and re-measured them under the same conditions.

To state the conclusion upfront: the title of single fastest for my use case has changed hands. The NVIDIA version of 35B-A3B achieves 77 tok/s out of the box, and with the bundled MTP enabled it reaches single 108 tok/s — 1.75x the previous speed champions. And in the much-discussed quantized version showdown, the empirical results under identical conditions showed the NVIDIA version was 12–17% faster across all configurations. The measurements also explain where the "2.5x" claim came from, and why the Unsloth version's benchmark numbers weren't a lie either.

My previous article is here (written as of 2026-07-06). This article is a continuation, adding 35B-A3B under the same harness and same measurement conditions.

https://dev.classmethod.jp/articles/dgx-spark-single-node-best-llm-qwen3-6-27b-nvfp4/

This article presents the results of empirically testing two quantized versions of Qwen3.6-35B-A3B-NVFP4 on a single DGX Spark, covering speed, quality, and image input comprehensively. I hope it's useful for anyone looking to update their local LLM selection for DGX Spark.

What kind of model is Qwen3.6-35B-A3B

First, let's confirm its characteristics. The previous star, Qwen3.6-27B, was a "dense-leaning hybrid" that reads 27B worth of weights for every token. Today's 35B-A3B is, as the name suggests, a MoE with 35B total parameters, with only 3B Active parameters used per token. It operates by selecting 8 + 1 shared experts from 256. Since decode speed on the DGX Spark is largely determined by "the amount of weights read per token," speed is practically guaranteed at this point.

What's nice is that all the virtues of the 27B remain intact. An MTP module for speculative decoding is bundled in the checkpoint, it supports image input (VL), context extends to 262K, and the license is Apache 2.0. The publicly stated intelligence benchmark is also 88.8 on AIME 2025, maintaining a level close to the 27B's 92.7.

One thing that caught my attention is that there are two lineages of NVFP4 quantized versions. Looking inside, it's not just a difference in distributor — the quantization method itself differs.

The Unsloth version claims to be "1.56x faster than other NVFP4 quants," and Mia's repository testing this on a DGX Spark reported approximately 80 tok/s single. On the other hand, the NVIDIA forums also have follow-up tests claiming "the NVIDIA version is 15% faster under identical conditions," leaving the numbers split. It seemed fastest to just download both and measure them with the same harness.

There were 2 pitfalls to getting it running on vLLM v0.25.0

The execution environment was based on the stable vllm/vllm-openai:v0.25.0-aarch64, which had just been released the day before verification. Since my previous article was based on v0.24.0, I first re-measured the 27B that was still in cache with v0.25.0 to bridge the versions. It came in at base 12.43 / MTP 28.91 tok/s, within 2.5% of the previous measurements (12.13 / 29.38), so I judged that numbers could be compared directly across builds.

When I then launched 35B-A3B on top of that, it immediately crashed. There were 2 stumbling blocks, so I'll note them first.

The first is an assertion error at launch. Launching with default settings hits the following error and stops:

AssertionError: In Mamba cache align mode, block_size (2096) must be <= max_num_batched_tokens (2048).

35B-A3B is a hybrid with linear attention (GDN), and the block size of 2096 for its Mamba-style cache barely exceeds the v0.25.0 default max_num_batched_tokens of 2048. The fix is simply to add --max-num-batched-tokens 4096 to the launch flags. The 27B, which is also a GDN hybrid, doesn't hit this assertion, so it's a 35B-A3B-specific stumbling block.

The second is the recommended flags for the Unsloth version. The model card specifies --moe-backend flashinfer_b12x --linear-backend flashinfer_b12x for DGX Spark, but specifying --linear-backend flashinfer_b12x with the official v0.25.0 image causes it to fail to launch:

ValueError: --linear-backend=flashinfer_b12x was requested but no 'flashinfer_b12x' kernel exists for this layer type.

There are the same reports in the forums, and it appears this flag only works with special builds like Mia's Docker image (vLLM 0.24.1-dev + FlashInfer 0.6.13). So I measured three variations of how much the backend specification changes things with the NVIDIA version in its default state:

The difference is 1.5%, within the margin of error. Looking at the launch logs, v0.25.0's auto internally selects the MARLIN kernel, landing in the same place as the NVIDIA model card's recommendation (marlin). In other words, as long as you're using the official image, there's no need to worry about backend specification. All subsequent measurements use auto with no flag specification.

To summarize, the launch command for v0.25.0 is as follows. The only difference is that the NVIDIA version uses --quantization modelopt, while the Unsloth version in compressed-tensors format leaves out the quantization specification and relies on auto-detection.

vllm serve nvidia/Qwen3.6-35B-A3B-NVFP4 \
  --host 0.0.0.0 --port 8000 \
  --quantization modelopt \
  --kv-cache-dtype fp8 \
  --max-model-len 65536 \
  --max-num-seqs 8 \
  --max-num-batched-tokens 4096 \
  --gpu-memory-utilization 0.85 \
  --reasoning-parser qwen3 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3}' \
  --enable-prefix-caching --trust-remote-code

It already exceeded the previous speed champions out of the box

The measurements use exactly the same harness as last time. Force-generating 256 tokens, measuring decode speed excluding TTFT, with single being the median of 3 runs and parallel looking at the combined throughput of 2/4/8 simultaneous requests.

Starting with the NVIDIA version results. The bare state produced single 76.7 tok/s. This already exceeds the previous fastest — Nemotron Nano's 61.2 and Gemma with external drafter's 62.4 — without any MTP. Being 25% faster than Nano at the same Active 3B is likely attributable to the additional lightness of NVFP4 weights compared to bf16 equivalent.

And enabling the bundled MTP, speeds stretched to 98.0 at spec=2 and 108.3 tok/s at spec=3. The speculative decoding acceptance rates at spec=3 were 0.86 / 0.67 / 0.55 from the first stage — the descending staircase distribution you see when things are working well. Previously with Gemma's external drafter, increasing to 3 lookaheads didn't change speed at all, but it's interesting that 35B-A3B's bundled MTP provides a net positive even at stage 3's acceptance rate of 0.55.

Lined up against the 5 models from last time, the changing of the guard is immediately obvious:

Dark blue is bare speed, light blue is the MTP uplift. The newcomer 35B-A3B already exceeds the previous speed champions at baseline, reaching 108.3 tok/s with MTP enabled.

The hypothesis I laid out in the previous article — "decode speed approaches the value of memory bandwidth 273 GB/s divided by bytes read per token" — holds just as well for the newcomer. With a footprint of around 23GB, comparable to Nano and Gemma, it also fits right into the shared sub-machine configuration from last time where you throttle gpu-memory-utilization to around 0.35 to co-reside with RAG or ComfyUI.

Which is faster, the NVIDIA version or the Unsloth version?

Now for the main event. I measured the much-discussed quantized version showdown across 3 configurations — bare, MTP spec=2, and spec=3 — using the same vLLM v0.25.0, same backend (auto), and same harness.

Blue is NVIDIA version (modelopt), green is Unsloth version (compressed-tensors). NVIDIA version was 12–17% faster across all configurations.

The NVIDIA version wins all single sessions. The results are consistent with forum follow-up reports (Unsloth version averaging 15.2% lower decode speed, single 75 vs 90 tok/s), and at least in my DGX Spark environment's single sessions, "the Unsloth version is faster" did not reproduce.

What's interesting is that the MTP acceptance rates were nearly identical between both versions. spec=2 showed 0.87 / 0.69 for both, spec=3 showed 0.86–0.87 / 0.67–0.69 / 0.54–0.55 for both. This means there's no difference in drafter prediction quality — the difference is purely in execution efficiency of the quantization format. The Unsloth version's compressed-tensors dynamically quantizes activations every token, and in single sessions that computation adds directly to latency. The fact that the gap nearly disappears at 8-parallel load (in MTP spec=2 the Unsloth version actually edges ahead by a slim margin) makes sense if this overhead is amortized across batches.

As for the "2.5x" claim, it's an assertion that includes B200 throughput, and Mia's ~80 tok/s can be substantially reproduced with the Unsloth version's MTP spec=2 result of 81.6 tok/s in this measurement. The numbers themselves are correct; it appears the truth is simply that the comparison against the NVIDIA version under identical conditions was missing.

Checking quality and image input too

Making a switch based on speed alone invites problems, so I ran the same 4-question Japanese quality check from last time (50-character instruction compliance, free description, code generation, numerical calculation) through both versions.

The NVIDIA version passed all 4 questions as expected. It answered the "explain Japan's capital in 50 characters or fewer including punctuation" in 11 characters, code generation was correct through execution verification, and the apple and orange calculation was right. The Unsloth version was correct on code generation and calculation, but on just the 50-character instruction question, the thinking process ran long and was cut off before reaching an answer. Since it was only one trial, I can't call it a statistically significant quality difference, but the tendency toward longer thinking does show up in the numbers.

One caution here: 35B-A3B is a reasoning model, and thinking length is considerably longer than the 27B. When I first ran the quality check with max_tokens 512, thinking used up the entire budget on every question and answers came back empty. It's recommended to secure 2048 or more when checking quality.

I also confirmed image input. The method was to send image requests with MTP enabled and watch the speculative decoding counters in vLLM's metrics. The result was that speculative decoding remained active for both versions, with both text and image requests. MTP's 108 tok/s and image input are compatible. The same behavior confirmed with the 27B is maintained in 35B-A3B as well.

Conclusion

Empirically testing Qwen3.6-35B-A3B-NVFP4 on a single DGX Spark yielded two answers. First, the resident model slot for a single DGX Spark now belongs to the NVIDIA version of 35B-A3B + bundled MTP. At single 108 tok/s, it's 1.75x the previous speed champions, with smarts and image input both covered — it ended up being almost a pure superset of the 27B I said I'd use as my axis last time. I'm going to update my thinking from "choosing between the speed slot of Nano and instruction-following of Gemma" to "start with just 35B-A3B."

Second, the quantized version showdown on DGX Spark single sessions goes to the NVIDIA version. The Unsloth version nearly catches up in 8-parallel, and both versions' published numbers are correct for their respective conditions. However, if you switch without verifying "which conditions the numbers apply to," you'll lose 17% in single sessions.

Next I'd like to incorporate the new champion into a resident configuration and try out tool calling from Hermes Agent and co-residing with RAG in actual operation. I'm also planning to cover re-measurement of Nano and Gemma on the latest builds, and how much the full flashinfer_b12x path on the special build differs, in a follow-up article if the opportunity arises.

Reference Links

• nvidia/Qwen3.6-35B-A3B-NVFP4
• unsloth/Qwen3.6-35B-A3B-NVFP4
• MiaAI-Lab/Unsloth-Qwen3.6-35b-NVFP4-DGX-Spark — The Docker setup that was the starting point for verification
• New 2.5x Faster Qwen3.6 NVFP4 Unsloth quants — NVIDIA Developer Forums — The thread on the quantized version debate
• Benchmark Report: Qwen3.6-35B-A3B-NVFP4 — NVIDIA Developer Forums (Cross-platform benchmark covering DGX Spark / Jetson Thor / Blackwell 6000 Pro)
• nvidia/Qwen3.6-27B-NVFP4 — The previous star
• vLLM Releases