Tried running DeepSeek V4 Flash-DSpark on 2 DGX Spark units

Introduction

Hello, I'm Morishige from Classmethod's Manufacturing Business Technology Department.

In the latter half of June 2026, DeepSeek published a model with a somewhat unusual name, DeepSeek-V4-Flash-DSpark, on HuggingFace. The base is the same checkpoint as DeepSeek-V4-Flash (284B total / 13B active MoE) published in April, but dspark appears as-is in both the filename and config.

https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-DSpark

DeepSeek officially released a model named and optimized for specific hardware called DGX Spark. That alone was personally a bit of interesting news, but the difference from the base version turned out to be a DGX Spark-targeted version with an MTP (Multi-Token Prediction) speculative decoding module attached—a difference that delves into the internals of inference acceleration—which made me want to try it out even more.

Another reason I tried it this time was personal circumstances. I've been regularly using NVIDIA LLM Router v3 recently, and observing the routing results across a 9-model pool, I noticed that most of my everyday tasks get routed to around V4 Flash. The experience through Hermes Agent / Codex / Claude Code is running at a quality level that doesn't bother me. Then the DSpark version appeared at just the right time, so I thought "if I can run the same V4 Flash on my local DGX Spark, I won't have to worry about API costs, and it could become a realistic everyday environment"—that's what led me to bring it in.

https://dev.classmethod.jp/articles/dgx-spark-nvidia-llm-router-v3/

The prerequisite of two DGX Sparks is certainly not a light one, so I won't say it's a realistic option for everyone. However, if you're in an environment where owning two units is acceptable, given recent local LLM trends and LLM Router behavior, I think it's a combination worth considering.

Initially I was gathering materials with the assumption of running nvidia/DeepSeek-V4-Flash-NVFP4 on 2 nodes. The DSpark version appeared midway, so I swapped it in as the main subject. I'll separate the parallel comparison with NVFP4 into another article; this article focuses on getting DSpark alone running with "2 DGX Sparks + vLLM."

To state the conclusion upfront: for reasoning tasks with thinking disabled, I achieved decode 55.17 tok/s / MTP acceptance 40.4%, and was able to run the 13B active MoE at a reasonably practical speed on 2 DGX Sparks. On the other hand, realities also emerged: throughput drops by a factor of 1.32x the moment thinking is enabled, and stretching context to 900K causes decode to hit a ceiling due to attention computation.

Reading DSpark's True Nature from the Config

The first thing I was curious about was "what's the difference between the base V4 Flash and the DSpark version?" Looking at the HuggingFace file list, the main body is 48 shards / 167.57 GB total—the same size range as the base V4 Flash. However, there are directories called encoding/ and inference/, which contain the DSML (a tool-call markup defined by DeepSeek) encoder and a minimal reference implementation.

The substance of the difference is in the last 2 lines of config.json.

{
  "model_type": "deepseek_v4",
  "num_hidden_layers": 43,
  "n_routed_experts": 256,
  "num_experts_per_tok": 6,
  "max_position_embeddings": 1048576,
  "rope_scaling": {"type": "yarn", "factor": 16},
  "quantization_config": {
    "quant_method": "fp8",
    "fmt": "e4m3",
    "scale_fmt": "ue8m0",
    "weight_block_size": [128, 128]
  },
  "num_nextn_predict_layers": 1,
  "dspark_target_layer_ids": [40, 41, 42]
}

The key points are the combination of dspark_target_layer_ids: [40, 41, 42] and num_nextn_predict_layers: 1. This is a declaration that among the 43-layer MoE, the last 3 layers are treated for speculation purposes targeting DGX Spark. MTP doesn't maintain a separate draft model; instead, it repurposes the tail layers of the main network as prediction layers for speculative decoding, and the draft weights are already integrated into the main 48 shards rather than separate shards.

The quantization is FP8 (E4M3) base + FP4 only for the MoE portion mixed, weight_block_size is [128, 128], and scale_fmt is ue8m0, an unusual format. You can read this as tuning to fit the 13B active MoE into the DGX Spark's 128 GB unified memory by compressing expert weights down to FP4. In practice, specifying --gpu-memory-utilization 0.82 on 2 nodes (TP=2), it fit comfortably including the KV cache.

The base V4 Flash itself uses Hybrid Attention (a combination of Compressed Sparse Attention + Heavily Compressed Attention) + Manifold-Constrained Hyper-Connections—a new architecture DeepSeek introduced in papers. The design upper limit extends to 1M context (max_position_embeddings: 1048576, yarn factor 16), and reasoning_effort for switching inference depth is also available. The model card lists 3 levels: non-think / high / max, but through the vLLM API it accepts the OpenAI standard none / low / medium / high / xhigh / max, so note that sending non-think directly will return a 400 error. The DSpark version inherits all these features and requires --trust-remote-code at vLLM startup because it uses a custom tokenizer extension called encoding_dsv4.

The license is MIT. The DSpark version's position is as a 13B active MoE with tool calling and reasoning officially distributed by DeepSeek with DGX Spark named explicitly.

Bringing in the 2-Node Environment and tonyd2wild Recipe

Trying to straightforwardly load DSpark's 167 GB weights onto a single DGX Spark won't work with 128 GB of unified memory. The practical solution is to distribute across 2 units and start with TP=2, making inter-node communication the determinant of cluster speed.

The current configuration is a simple setup where the QSFP ports of 2 DGX Sparks (node1 / node2) are connected with a single direct cable. The QSFP spec is 200 Gbps, but actual measurements with iperf3 showed a ceiling of around 14.9 Gbps for single stream and 16.4 Gbps for 4 parallel streams. The kernel's streaming capability is the bottleneck, and the 200 Gbps can't be fully utilized—that's the premise we're working with. Still, compared to Wi-Fi (a few hundred Mbps) or Tailscale (~90 Mbps), it's 2 orders of magnitude faster, so it's sufficient for TP all-reduce communication. The ping RTT is 0.5~0.7 ms, giving the kind of response you'd expect from a direct link.

Green is QSFP direct connection (the path we want vLLM's TP all-reduce to use), purple is Wi-Fi, and yellow is Tailscale. The default route points to the Wi-Fi side, so if nothing is specified, vLLM will try to grab that. This becomes a trap later, but for now the physical layer is set up this way.

The Story of Going Around in Circles Choosing a Recipe

On the software side, I started with Aiden's Recipe (aidendle94/sparkrun-vllm-ds4-gb10:production-ready). This is a widely used image for running the base DeepSeek-V4-Flash on 2 nodes, built so that specifying --speculative-config '{"method":"mtp", ...}' enables MTP.

However, when I fed it DSpark weights and started with method=mtp, the vLLM loading process crashed like this:

File ".../vllm/models/deepseek_v4/nvidia/mtp.py", line 448, in load_weights
    params_dict[name]
KeyError: ...

It seems the MTP weight naming in the DSpark version and the MTP loader bundled in the Aiden image don't match. The plain method=mtp path can't read DSpark's MTP module as-is.

So I switched to tonyd2wild's Recipe (tonyd2wild/DeepSeek-v4-Flash-DSpark-60-tok-s-900K-ctx-2x-DGX-Spark). The repository name aggressively goes for it with the string "60 tok/s / 900K context / 2x DGX Spark," but the content was solid: it builds by layering a runtime overlay containing Rafael Caricio's vLLM PR (rafaelcaricio/vllm#1) on top of the base image (ghcr.io/bjk110/vllm-spark:unholy-fusion-prod-ready). With this overlay-included vLLM, you can select a DSpark-specific method --speculative-config '{"method":"dspark", "num_speculative_tokens":5}', which allows reading DSpark's MTP weights.

Running build-dspark-vllm-runtime.sh produces vllm-dspark-runtime:clean (22.7 GB) on both nodes. Up to here went smoothly.

The Trap of vLLM Grabbing Wi-Fi and 3 Essential Patches

After building, starting it caused vLLM's TP all-reduce to fail with this error:

RuntimeError: ifa != nullptr. Unable to find address for: enP7s7

The Gloo backend is trying to find an interface called enP7s7 that doesn't exist and failing. Tracing the cause, vLLM's multiproc_executor.py was selecting the Wi-Fi side IP (192.168.64.198) as mq_connect_ip based on the default route, and inferring the interface name from there.

The DGX Spark's interface configuration is as shown in the Mermaid diagram at the beginning—we want to use enp1s0f1np1 (QSFP), but without explicitly telling vLLM, it goes to the Wi-Fi side. I applied the following 3 patches to the Recipe.

WORKER_IP=192.168.0.14    # worker QSFP IP (head reuses MASTER_ADDR)
NCCL_IB_GID_INDEX=3         # set to 3 because default 0 stalls with RoCEv2

GLOO_SOCKET_IFNAME: '${GLOO_SOCKET_IFNAME:-${NCCL_SOCKET_IFNAME}}'
TP_SOCKET_IFNAME: '${TP_SOCKET_IFNAME:-${NCCL_SOCKET_IFNAME}}'
MN_IF_NAME: '${MN_IF_NAME:-${NCCL_SOCKET_IFNAME}}'
OMPI_MCA_btl_tcp_if_include: '${OMPI_MCA_btl_tcp_if_include:-${NCCL_SOCKET_IFNAME}}'
WORKER_IP: '${WORKER_IP:-}'

if [ "${NODE_RANK:-0}" = "0" ]; then
  export VLLM_HOST_IP="${MASTER_ADDR}"
else
  export VLLM_HOST_IP="${WORKER_IP:-}"
fi;

What's being done here is ultimately just "fixing the interface used for inter-node communication to the QSFP side." The 4 variables MN_IF_NAME / GLOO_SOCKET_IFNAME / TP_SOCKET_IFNAME / OMPI_MCA_btl_tcp_if_include cover vLLM's multiproc, Gloo, TP, and OpenMPI paths respectively, and VLLM_HOST_IP is branched based on NODE_RANK to head=MASTER_ADDR / worker=WORKER_IP.

After applying these and returning to the build → startup flow, it no longer stalled at TP initialization.

Getting a Rough Grasp of Capabilities with Official Benchmarks

Before measuring speed on actual hardware, I'll get a sense of V4 Flash's capabilities from the official numbers. Since the DSpark version shares the same checkpoint as the base V4 Flash, the benchmark quality values for V4 Flash apply directly.

The numbers DeepSeek published for V4 Flash Max (Think Max mode) in coding and long-context understanding are as follows:

Source: deepseek-ai/DeepSeek-V4-Flash model card

LiveCodeBench is on par with Gemini 3.1 Pro, and Codeforces rating 3052 matches as well. Looking at SWE Verified 79.0 and MRCR 1M 78.7 together, these look like quite aggressive numbers considering it's an Active 13B MoE with weights distributed under MIT.

The DSpark acceleration adds on top of this. According to paper §5.4 and the official README, in DeepSeek's official serving environment with DSpark-5 (γ=5 + Markov head) enabled, per-user generation speed increases by +60~85% relative to the MTP-1 baseline. One of the purposes of running this on 2 DGX Spark nodes is to confirm how much of this "faster with the same weights" can actually be felt locally.

The Breakdown of the 19 Minutes It Took to Start vLLM

After applying the patches and running docker run in the order worker → head, it took about 19 minutes until the API responded on port 8888. Tracking the logs revealed that the waiting time breaks down as follows:

The 162 seconds for loading 48 shards and the 24 seconds for mHC kernel warmup were within expected range; the remaining 15+ minutes were almost entirely consumed by sparse MLA autotune, TileLang JIT, and FlashInfer autotune. This is the process of searching for the fastest parameters empirically on actual hardware targeting DGX Spark's sm_121a (GB10), which inevitably makes the first startup heavy.

Fortunately, the results are persisted under the host's ~/.cache/huggingface, so subsequent startups shrink to about 7 minutes. For operations where you're constantly hitting the API from Hermes Agent or Codex, the basic approach is to start it once and keep it running, so you only need to brace for the first time.

At this point, curl http://127.0.0.1:8888/v1/models finally returned DeepSeek-V4-Flash-DSpark, and a light math smoke test also passed with a <think> block. The max_model_len is tonyd2wild's default of 262,144 (262K), which is just right for Hermes' 256K context everyday use.

Measuring Decode tok/s for Short to Medium Context

From here I'll look at actual speeds. To compare MTP effectiveness, I prepared 3 types of prompts:

• medium: a natural language question of about 200 characters (general chat / knowledge questions)
• long: a natural language explanation request of about 500 characters
• reasoning_code: a short 42-token code generation request saying "Write Python code to solve a maze with BFS"

For each, I measured decode tok/s in 2 modes: using <think> blocks (thinking ON) / not using them (thinking OFF). Context is the prod 262K setting (max_num_seqs=1, single user).

The unit for numbers is tok/s (output token speed during the decode stage). For medium / long, there's almost no difference between having thinking on or off. 35~39 tok/s is the honest performance when running a 13B active MoE on 2 DGX Sparks with 128 GB unified memory, and it falls within the practical range for local LLMs. For use cases like Q&A from Hermes Agent or Codex, this becomes the baseline.

On the other hand, reasoning_code behaves differently. From 41.90 tok/s with thinking ON, it jumps to 55.17 tok/s when thinking is turned OFF. Same prompt, just switching reasoning_effort=—and that's the difference. TTFT (time until the first token returns) was about 0.6~2.1 seconds for short prompts and 4.7~8.8 seconds for medium prompts, which scaled proportionally with the weight of prefilling prompt tokens all at once.

What's curious here is "why does only reasoning_code improve so much when thinking is turned off?" Normally you might think that thinking ON would be slower because the output length increases with <think> blocks, but this is directly connected to the MTP acceptance rate discussion, which becomes the core of the next chapter.

Note that tonyd2wild's README reports 62.48 tok/s using a code_completion-type 512-token prompt on the same DSpark configuration. Since conditions aren't identical, direct comparison isn't possible, but there's enough possibility to reach this range by aligning the prompt structure for draft-friendly input. I'll leave that as a topic for a separate article.

For reference, the median throughput (30-minute aggregate) of major providers pulling the same V4 Flash via OpenRouter sits around Baidu 66 / Fireworks 63 / Alibaba 62 / DeepSeek official 61 / SiliconFlow 61 tok/s. However, V4 Flash on OpenRouter defaults reasoning_effort to high, and specifying low / off is not supported. Comparing the same thinking ON mode, DSpark's 41 tok/s vs. the cloud top tier's 60+ tok/s still shows a gap.

Note that the 55 tok/s with thinking OFF is a number premised on directly calling reasoning_effort=none via curl. vLLM's DSpark implementation in this mode sets content to null and puts the final output in the reasoning field, so OpenAI-compatible clients (Hermes Agent / Codex / Claude Code etc. that read message.content) will see it as an empty response. For everyday use via Agent, it's currently safe to consider thinking ON's 41 tok/s as the effective value.

Looking at How MTP Works with Thinking On / Off

This is the core of this article. Why does decode speed increase 1.32x for reasoning_code when thinking is turned off? Digging deeper reveals that DSpark's MTP (Multi-Token Prediction) has the characteristic of being "hard to draft" for reasoning traces containing <think> blocks.

vLLM's /metrics endpoint outputs speculative decoding stats for DSpark (num_drafts_total / num_draft_tokens_total / num_accepted_tokens_total / num_accepted_tokens_per_pos_total). I ran reasoning_code once each with thinking ON / OFF, and took the diff between values before (before) and after (after) completion.

Turning off thinking improves the mean number of tokens accepted per draft from 1.21 → 2.02 (+67%), and per-token acceptance jumps from 24.2% → 40.4%. This directly shows that DSpark's draft (the last 3 layers of the main network + Markov head) struggles to predict "text where what comes next fluctuates every step" like reasoning traces. Conversely, for structured text like code itself, drafts tend to hit more often, and acceptance can be captured more generously even with γ=5 static draft.

DSpark paper §1 also states "Math/code naturally sustain higher acceptance rates," and that this doesn't necessarily apply to reasoning trace portions—which is the fact confirmed hands-on this time. "For tasks that don't truly need reasoning, turning off thinking actually leads to greater benefit from MTP" is a fairly useful operational guideline.

Note that the DSpark running in vLLM is not the full version from the paper; the dynamic verification scheduler proposed in paper §3.2 is off (VLLM_DSPARK_CONFIDENCE_SCHEDULER=off), and only the Markov head + γ=5 static draft portion is effective. DeepSeek's official serving's "+60~85% vs MTP-1 baseline" includes this scheduler, so there's still room to grow from here once the scheduler is ported to vLLM.

What Happens When You Naively Extend Long Context

DSpark is designed to handle up to 1M tokens. Keeping the prod 262K setup and changing the input context to 64K / 128K / 256K while measuring decode tok/s gave the following pattern:

Compared to the 30~55 tok/s from the short prompts in Chapter 6, decode drops in two stages just from expanding the input. The curve is roughly flat between 64K and 128K, then drops a step at 256K.

From here, increasing max_model_len to 900K, restarting, and appending "Write the next Python code" to a long text of about 247K tokens to generate 500 tokens resulted in a total elapsed time of about 4 minutes and 32 seconds, with decode around 2.4 tok/s. Yet DSpark's accept rate was 46.8%—not dropping from the short text value (40.4%)—and the output code incorporated function names and signatures from the input context, confirming that draft quality and output quality are maintained even in long context. The reason decode drops is that attention computation gets heavier proportional to context length; it's not that quality breaks down and it becomes unusable.

So personally, for use cases like Hermes Agent or Codex where you want to maintain context without cutting it, I think the most practical approach is to set max_model_len=262144 (262K) while keeping the actual context per turn to a few K to tens of K. The scenarios where there's merit in using 900K seem limited to single-shot long document summarization or code reading.

Summary

I brought DeepSeek's officially DGX Spark-named model DeepSeek-V4-Flash-DSpark to 2 DGX Sparks using a combination of the tonyd2wild Recipe and custom patches, and was able to see 55.17 tok/s / MTP acceptance 40.4% on actual hardware for reasoning code generation.

As mentioned at the beginning, the recent hands-on feeling has been that when using LLM Router v3, most everyday tasks get routed around V4 Flash. The speed range seen this time, as mentioned at the end of Chapter 6, is also approaching a line where comparison with OpenRouter's top providers (60 tok/s range with thinking ON) becomes possible. The effective value for Agent-based operation is thinking ON's 41 tok/s, and while you can draw out thinking OFF's 55 tok/s with direct curl calls, it's still just a bit short of being used as an everyday client. Still, the idea of swapping the path being routed to V4 Flash via LLM Router v3 directly to the DSpark side is close enough to come into view.

Compared to being accustomed to cloud API response, there are some moments where the speed feels a bit lacking, though it's manageable for everyday use. The barrier of owning 2 units is undeniably not light, so it's not a configuration I can recommend to everyone, but if the environment is set up, it's quite an interesting position to be in, don't you think?

Reference Links

• deepseek-ai/DeepSeek-V4-Flash-DSpark (HF)
• DSpark paper arXiv 2606.19348
• tonyd2wild/DeepSeek-v4-Flash-DSpark-60-tok-s-900K-ctx-2x-DGX-Spark
• rafaelcaricio/vllm#1 (DSpark method PR)
• NVIDIA DGX Spark Forum #374742 (2x DGX Spark Recipe)
• Flowtivity blog (DeepSeek V4 Flash 1M context dual DGX Spark)