A new champion among partner LLMs for DGX Spark? Comparing Qwen3.6-35B-A3B-NVFP4 in two quantized versions

Introduction

Hello, I'm Morishige from Classmethod's Manufacturing Business Technology Department.

I just recently wrote an article about considering model configurations to run permanently on a single DGX Spark, but Qwen3.6-35B-A3B, a different parameter variant of the Qwen3.6-27B that was the star of that article, has been getting a lot of attention lately. Unsloth is promoting a new NVFP4 quantized version as "2.5x faster," while the NVIDIA forums have seen pushback saying "actually, real-world measurements show the official NVIDIA version is faster." Many people may be wondering which to believe.

https://huggingface.co/unsloth/Qwen3.6-35B-A3B-NVFP4

My interest is less in the debate itself and more in whether this 35B-A3B can replace Qwen3.6-27B as the "partner for a single DGX Spark." In my previous article, the single fastest was Nemotron Nano plus an external drafter added to Gemma at 61–62 tok/s, while the balance slot combining intelligence and image input was Qwen3.6-27B at 29 tok/s. The 35B-A3B is a MoE with Active 3B, bundled MTP, and image input support—specs that could consolidate the speed slot and balance slot into one. Since the new stable vLLM v0.25.0 had just been released, I downloaded both the NVIDIA and Unsloth versions and re-measured them under the same conditions.

To state the conclusion up front: for me, the top single-session speed crown has changed hands. The NVIDIA version of 35B-A3B delivers 77 tok/s out of the box, and with the bundled MTP enabled reaches single 108 tok/s—1.75x the previous speed leader. And in the hotly debated quantization showdown, the real-world measurements under the same conditions showed the NVIDIA version to be 12–17% faster across all configurations. The real-world measurements also explain where the "2.5x" claim comes from, and why the Unsloth version's benchmark numbers weren't actually wrong.

My previous article is here (as of 2026-07-06). This is a sequel using the same harness and measurement conditions with 35B-A3B added.

https://dev.classmethod.jp/articles/dgx-spark-single-node-best-llm-qwen3-6-27b-nvfp4/

This article presents the results of measuring two quantized versions of Qwen3.6-35B-A3B-NVFP4 on a standalone DGX Spark, checking speed, quality, and image input across the board. I hope it's useful for anyone looking to update their local LLM selection for DGX Spark.

What kind of model is Qwen3.6-35B-A3B

First, let's confirm its characteristics. The previous star, Qwen3.6-27B, was a "dense-leaning hybrid" that reads 27B worth of weights every token. Today's 35B-A3B is, as the name suggests, a MoE with 35B total parameters, with only 3B Active parameters used per token. It operates by selecting 8 experts plus 1 shared expert from 256. Since decode speed on the DGX Spark is essentially determined by "the amount of weights read per token," high speed is practically guaranteed from the outset.

What's great is that the strengths of the 27B are carried over intact. The MTP module for speculative decoding is bundled in the checkpoint, it supports image input (VL), and context extends up to 262K. The license is also Apache 2.0. The published intelligence benchmark of 88.8 on AIME 2025 also maintains a level close to the 27B's 92.7.

What's interesting here is that there are two lineages of NVFP4 quantized versions. Looking inside, it's not just a difference in distributor—the quantization methods themselves are different.

The Unsloth version claims to be "1.56x faster than other NVFP4 quants," and Mia's repository that tested this on a DGX Spark reports approximately single 80 tok/s. On the other hand, the NVIDIA forums also have follow-up tests claiming "under the same conditions, the NVIDIA version is 15% faster," leaving the numbers split. Downloading both and measuring them with the same harness seems the quickest way to settle it.

There were 2 pitfalls when running on vLLM v0.25.0

The execution environment was based on the stable vllm/vllm-openai:v0.25.0-aarch64 that had just been released the day before verification. Since the previous article was based on v0.24.0, I first re-measured the 27B that was still in cache under v0.25.0 to bridge the gap. The results of base 12.43 / MTP 28.91 tok/s were within 2.5% of the previous measurements (12.13 / 29.38), so I judged that numbers could be compared across builds.

When I then tried to launch 35B-A3B, it immediately crashed. There were 2 sticking points, so let me describe them first.

The first is an assertion at startup. Starting with default settings causes it to stop with the following error.

AssertionError: In Mamba cache align mode, block_size (2096) must be <= max_num_batched_tokens (2048).

The 35B-A3B is a hybrid with linear attention (GDN), and the root cause is that the Mamba-style cache block size of 2096 slightly exceeds the v0.25.0 default max_num_batched_tokens of 2048. The fix is simply adding --max-num-batched-tokens 4096 to the startup flags. Even though 27B is also a GDN hybrid, it doesn't hit this assertion, so this is a 35B-A3B-specific pitfall.

The second is the recommended flags for the Unsloth version. The model card lists --moe-backend flashinfer_b12x --linear-backend flashinfer_b12x for DGX Spark, but specifying --linear-backend flashinfer_b12x with the official v0.25.0 image prevents startup.

ValueError: --linear-backend=flashinfer_b12x was requested but no 'flashinfer_b12x' kernel exists for this layer type.

The same report appears in the forums, and this flag apparently only works with special builds like Mia's Docker image (vLLM 0.24.1-dev + FlashInfer 0.6.13). So I measured 3 backend configurations with the NVIDIA version in its bare state to see how much the backend specification matters.

The difference is within 1.5%, which is within the margin of error. Looking at the startup log, v0.25.0's auto internally selects the MARLIN kernel, landing at the same place as the NVIDIA model card's recommendation (marlin). This means there's no need to worry about backend specifications when using the official image. All subsequent measurements are unified with auto (no flag specified).

To summarize, the startup command for v0.25.0 looks like this. The only difference is that the NVIDIA version uses --quantization modelopt, while the Unsloth version in compressed-tensors format omits the quantization specification and relies on auto-detection.

vllm serve nvidia/Qwen3.6-35B-A3B-NVFP4 \
  --host 0.0.0.0 --port 8000 \
  --quantization modelopt \
  --kv-cache-dtype fp8 \
  --max-model-len 65536 \
  --max-num-seqs 8 \
  --max-num-batched-tokens 4096 \
  --gpu-memory-utilization 0.85 \
  --reasoning-parser qwen3 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3}' \
  --enable-prefix-caching --trust-remote-code

The bare speed already surpassed the previous fastest

The measurement used exactly the same harness as before. Force-generate 256 tokens, measure decode speed excluding TTFT, take the median of 3 runs for single, and look at total throughput for 2/4/8 concurrent sessions for parallel.

Starting with the NVIDIA version results. The bare state produced single 76.7 tok/s. This exceeds the previous fastest, Nemotron Nano at 61.2 and Gemma with an external drafter at 62.4, in the bare state without MTP. The fact that it's 25% faster than Nano with the same Active 3B is likely attributable to the additional savings from NVFP4 weights being even lighter than bf16.

And with the bundled MTP enabled, it stretched to 98.0 tok/s at spec=2 and 108.3 tok/s at spec=3. The speculative decoding acceptance rate at spec=3 was a descending staircase distribution of 0.86 / 0.67 / 0.55 from the first stage—the kind you see when things are working well. When using Gemma's external drafter before, increasing the lookahead to 3 didn't change speed, but 35B-A3B's bundled MTP works out to a net positive even at the 3rd stage acceptance rate of 0.55—an interesting difference.

Lined up against the 5 models from before, the changing of the guard is clear at a glance.

Dark blue is the bare speed, light blue is the MTP-enabled boost. The newcomer 35B-A3B exceeds the previous fastest even at bare speed, and reaches 108.3 tok/s with MTP.

The prediction from the previous article—"decode speed approaches memory bandwidth 273 GB/s divided by the bytes read per token"—holds up with the newcomer as well. The footprint of approximately 23GB is in the same class as Nano and Gemma, so it also fits into the shared sub-machine configuration from before, where gpu-memory-utilization is throttled to around 0.35 to coexist with RAG or ComfyUI.

Which is faster: NVIDIA version or Unsloth version?

Here's the main topic. I measured the hotly debated quantization showdown using the same vLLM v0.25.0, same backend (auto), and same harness across all 3 configurations: bare, MTP spec=2, and spec=3.

Blue is the NVIDIA version (modelopt), green is the Unsloth version (compressed-tensors). NVIDIA version was 12–17% ahead in all configurations.

The NVIDIA version wins in all single sessions. This result is also consistent with forum follow-up tests (Unsloth version 15.2% lower in average decode, single 75 vs 90 tok/s), and at least in my DGX Spark environment for single sessions, "the Unsloth version is faster" was not reproducible.

What's interesting is that the MTP acceptance rates were nearly identical between the two versions. At spec=2, both showed 0.87 / 0.69; at spec=3, both showed 0.86–0.87 / 0.67–0.69 / 0.54–0.55. This means there's no difference in drafter prediction quality, and the difference that does appear is purely in the execution efficiency of the quantization format. The Unsloth version's compressed-tensors dynamically quantizes activations every token, and that computation adds entirely to latency in single sessions. The fact that the difference nearly disappears at up to 8 parallel sessions (and actually reverses by a small margin at MTP spec=2) makes sense if you consider this overhead being diluted by batching.

Note that the "2.5x" claim is for B200 throughput, and Mia's approximately 80 tok/s is nearly reproduced by the Unsloth version MTP spec=2's 81.6 tok/s in this test. The numbers themselves are correct; it appears the truth is simply that the NVIDIA version as a comparison baseline under the same conditions was missing.

Also checking quality and image input

Making a switch based on speed alone is a recipe for trouble, so I also ran both versions through the same 4 Japanese quality check questions as before (instruction compliance within 50 characters, free-form writing, code generation, numerical calculation).

The NVIDIA version answered all 4 questions as expected. It answered the "describe Tokyo in 50 characters or fewer including punctuation" question in 11 characters, the code generation passed execution verification, and the apple and orange calculation was correct. The Unsloth version got the code generation and calculation right, but for the 50-character instruction question, the thinking process ran long and was cut off before reaching an answer. Since it was a single trial, this can't be called a meaningful quality difference, but the tendency toward longer thinking does show up in the numbers.

One important caveat here: 35B-A3B is a reasoning model, and the thinking length is considerably longer than with 27B. When I first ran the quality checks with max_tokens 512, the thinking used up the entire allocation on every question and the responses came out empty. It's recommended to allocate 2048 or more when evaluating quality.

I also confirmed image input. The method was to send image requests with MTP enabled and check the speculative decoding counters in vLLM metrics. The result was that speculative decoding remained active for both text and image in both versions. The same behavior confirmed with 27B is maintained with 35B-A3B.

Summary

Real-world measurements of Qwen3.6-35B-A3B-NVFP4 on a standalone DGX Spark yielded two answers. First, the NVIDIA version of 35B-A3B with bundled MTP is the new frontrunner for the single DGX Spark partner slot. At single 108 tok/s, it's 1.75x the previous speed leaders, with intelligence and image input both covered—it's become nearly a strict upgrade over the 27B I wrote about building around in my previous article. I'm going to update my approach from "split usage between Nano for speed and Gemma for instruction compliance" to "start with 35B-A3B as the single choice."

Second, in the quantization showdown, the NVIDIA version won on single-session DGX Spark. The Unsloth version is nearly on par at 8 parallel sessions, and each set of published numbers is correct for its respective conditions. However, switching without checking "which conditions the numbers apply to" means losing 17% in single sessions.

Next, I want to incorporate the new champion into a permanent configuration and test tool calling from Hermes Agent and RAG coexistence in real-world operation. I also plan to cover the latest build re-measurements for Nano and Gemma, and how far the full flashinfer_b12x path on the special build differs, in a follow-up article if the opportunity arises.

Reference Links

• nvidia/Qwen3.6-35B-A3B-NVFP4
• unsloth/Qwen3.6-35B-A3B-NVFP4
• MiaAI-Lab/Unsloth-Qwen3.6-35b-NVFP4-DGX-Spark — The Docker setup that was the starting point for verification
• New 2.5x Faster Qwen3.6 NVFP4 Unsloth quants — NVIDIA Developer Forums — The thread where the quantization version debate happened
• Benchmark Report: Qwen3.6-35B-A3B-NVFP4 — NVIDIA Developer Forums (Cross-platform benchmarks for DGX Spark / Jetson Thor / Blackwell 6000 Pro)
• nvidia/Qwen3.6-27B-NVFP4 — The previous article's star
• vLLM Releases