Tried running DeepSeek V4 Flash 284B on DGX Spark with DwarfStar 4

Introduction

Hello, I'm Morishige from Classmethod's Manufacturing Business Technology Department.

ds4.c is a small inference engine written by Redis author antirez, specifically for DeepSeek V4 Flash. It's not a general-purpose GGUF runner or a llama.cpp wrapper — it's a C-based engine with the sole aim of running "one specific model, locally, end-to-end in a complete form." Just recently it was renamed "DwarfStar 4" and the CUDA backend was officially merged. What was previously Mac-only with Metal can now be built on Linux + NVIDIA with a single make.

https://github.com/antirez/ds4

DeepSeek V4 Flash is a 284B parameter MoE model. Of the total 284B, only around 13B are actually active at any time (per DeepSeek's announcement). The supported context is 1 million tokens. DwarfStar 4 is designed to fit this model onto a 128GB-class machine using 2-bit quantization, and to persist the internal memory used during generation (KV cache) to disk. Since I have a DGX Spark (GB10, 128GB unified memory, memory bandwidth 273 GB/s) on hand, I took the opportunity right after the CUDA support was merged to actually run it.

The three things I personally wanted to know were:

1. How does a 284B MoE actually fit on a 128GB DGX Spark — is it streaming weights from disk too?
2. Would the community-reported "DGX Spark generates around 12 tokens/sec, slower than M3 Max at ~27 tokens/sec" reproduce on real hardware?
3. Does the mechanism of offloading the KV cache to disk actually help in practice for long-context or repeated prompts?

To give the conclusion upfront: the build went through with just make, and 284B fit in 128GB at about 81GiB. Generation is 8–15 tokens/sec — slow in proportion to memory bandwidth — but prefill (prompt ingestion) comes out fast at 80–250 tokens/sec incrementally. The disk KV cache reduced a ~32K-token prompt re-send from 115 seconds to 7.7 seconds. The result was very characteristic of DGX Spark: "too slow for interactive chat, but realistic for batch processing of long documents."

How DwarfStar 4 and DeepSeek V4 Flash Work

The DeepSeek V4 Flash Model

DeepSeek V4 Flash is a MoE with fewer active parameters, so it runs lighter than you'd expect for a 284B scale model. According to antirez's analysis, even with long instructions, the thinking output tends to be shorter than other models (roughly 1/5 if you avoid maximum thinking mode).

The GGUF distributed by DwarfStar 4 is a custom one — it's not the case that any arbitrary DeepSeek/GGUF file will work. The q2-imatrix used here is about 81GiB on disk. The quantization is asymmetric: only the expert networks, which make up the majority of the model, are aggressively compressed to 2-bit, while the core parts that matter for quality (attention layers, output layers, etc.) are kept at higher precision. There's also a q4-imatrix (~153GB) for machines with 256GB or more, but for the 128GB DGX Spark, q2-imatrix is the only option.

The Idea of Putting the KV Cache on Disk

DeepSeek V4 Flash compresses its KV cache into a small representation using transformations learned during training, and varies the compression strength per layer (the most aggressive layers compress down to 1/128). Combined with a mechanism that instead of attending to all past tokens, focuses only on the most relevant subset (top 512 per token), the KV cache ends up extremely small.

Measured on DGX Spark, it's about 14KB per token. Even at 1 million tokens, the linear extrapolation is only around a dozen GiB, and even including the pre-allocated context buffers it fits within about 26GB (per the README's estimate, about 22GB of that is for the index used by the "attend to relevant tokens" mechanism).

The README contains the line: The KV cache is actually a first-class disk citizen. The thinking is that with KV this compressed and modern SSDs being this fast, there's no reason the KV cache must live in RAM. In practice, ds4-server hashes the prefix of a prompt and saves it as <sha1>.kv, so when a continuation of the same prompt arrives later, it reads back that checkpoint instead of re-running prefill from token 0. Files are written with ordinary read/write calls, so no extra memory maps are added to a process that already has 81GiB mapped.

The reason llama.cpp integration is said to be difficult is that the DeepSeek V4-specific KV structure and the design that assumes disk KV persistence are fundamentally incompatible with how a general-purpose GGUF runner is built. You could say this is something only possible because of the deliberate choice to make it a single-model-dedicated engine.

Prefill Is Compute-Bound, Generation Is Bandwidth-Bound

This is a pattern that comes up every time DGX Spark is discussed, and it holds here too. Prefill — reading the prompt — can be parallelized in batch, so the GPU's compute throughput is the bottleneck, and GB10's compute performance helps. Generation (decode), which produces one token at a time, requires re-reading parameters from memory at every step, so memory bandwidth is the bottleneck. DGX Spark's memory bandwidth is 273 GB/s, which is lower than M3 Max (~400 GB/s) or M3 Ultra (~819 GB/s), so the prediction is that Apple Silicon should have the advantage in generation.

Let's see how it actually played out.

Test Environment

Since this is just a few days after the CUDA support was merged, I'm noting the commit hash explicitly.

Building and Running on DGX Spark

Built with a Single make

Setup was surprisingly straightforward.

git clone https://github.com/antirez/ds4 ~/works/dwarfstar4/ds4
cd ~/works/dwarfstar4/ds4
export PATH=/usr/local/cuda/bin:$PATH
make

Looking at the make contents, on Linux it automatically builds the CUDA backend (ds4_cuda.cu) and uses CUDA_ARCH ?= native to target the visible GPU. With aarch64 + CUDA 13.0, no patches or environment variable tweaks were needed; in about 16 seconds, three binaries were produced: ds4 (CLI), ds4-server (HTTP API), and ds4-bench (benchmark). -arch=native correctly detected GB10 (sm_121), and there were no issues with Mac-specific code or dependency libraries.

The model is retrieved using the bundled script. I chose q2-imatrix for a 128GB machine.

./download_model.sh q2-imatrix   # Downloads ~81GiB from Hugging Face → linked as ./ds4flash.gguf

Startup and Memory Behavior

Let's try sending one short prompt.

./ds4 -p "In one short paragraph, what is Redis?" --cuda --nothink -n 64

ds4: context buffers 751.71 MiB (ctx=32768, backend=cuda, prefill_chunk=2048, raw_kv_rows=2304, compressed_kv_rows=8194)
ds4: CUDA backend initialized on NVIDIA GB10 (sm_121)
ds4: CUDA host registration skipped: operation not supported
ds4: CUDA loading model tensors into device cache
ds4: CUDA loading model tensors 16.02 GiB cached
...
ds4: CUDA startup model cache prepared 80.76 GiB of tensor spans in 26.432s
ds4: CUDA q8 fp16 cache budget exhausted; using q8 kernels (request=8.00 MiB cached=0.00 GiB free=3.54 GiB reserve=6.08 GiB total=121.69 GiB)
**Redis** is an open-source, in-memory data structure store that is commonly used as a high-speed cache, message broker, and database. ...
ds4: prefill: 17.29 t/s, generation: 14.81 t/s

The output is coherent text. Two things stand out.

First, loading the 80.76 GiB of weights into the "device cache" takes about 26 seconds the first time. With the file in the page cache, subsequent runs shrink this to about 10 seconds. Since GB10 is unified memory, this is physically a copy within the same RAM.

Second, the line CUDA q8 fp16 cache budget exhausted; using q8 kernels. With weights at 81GiB, reserved space at ~6GiB, and context buffers, there's only 3.5–6GiB free out of the 122GiB system memory, leaving no room for the 16-bit expanded cache that would speed up computation slightly, so it falls back to computing in 8-bit. The "limits of 128GB" show up in the logs from the very moment of startup. CUDA host registration skipped: operation not supported is also a message specific to GB10's unified memory environment, and has no impact on operation.

Result 1: Context Length vs. Throughput

Using the bundled ds4-bench, you can gradually extend the context and measure prefill and generation throughput "at that point." The setup places checkpoints at 2048, 4096, … and at each point measures "incremental prefill since the last checkpoint" and "generate 128 tokens using greedy decoding each time."

./ds4-bench -m ds4flash.gguf --prompt-file speed-bench/promessi_sposi.txt \
  --ctx-start 2048 --ctx-max 65536 --step-incr 2048 --gen-tokens 128 \
  --csv results/dgx-spark-q2-sweep.csv

This was supplemented with a coarser sweep from 65536 to 262144, doubling each time (64 tokens generated), and the result is the following graph.

Generation (red) starts around 13 tokens/sec and gradually falls to 8 tokens/sec at 260K tokens. Prefill (blue) rises in steps from a cold 65 tokens/sec at the start, settling around 200 tokens/sec.

Generation throughput starts at about 13 tokens/sec with short context and drops to about 8 tokens/sec at 260K tokens. This is consistent with the community-reported "~12 tokens/sec on DGX Spark," confirming the memory-bandwidth-limited prediction. The official benchmark table in the README shows DGX Spark GB10 (128GB, q2, 7047 tokens) at prefill 343.81 tokens/sec and generation 13.75 tokens/sec — generation matches closely, while prefill is somewhat higher than my incremental measurements (~200 tokens/sec). The README measures a one-shot prefill at a short ~7000-token context, whereas I'm measuring incrementally walking to a long context, so I attribute the difference to measurement methodology.

The reason prefill is low early on is likely that the pages for the expert networks within the 81GiB of weights are warming up gradually as different experts are activated. The stepwise increase up to around 30K tokens is visible as the warm-up region in the graph.

Peak memory when extending to 260K tokens was about 115GiB (roughly 7GiB free). According to ds4-bench logs, the context buffers alone took about 4.5GiB. The README also states "128GB machines can realistically handle 100K–300K tokens," so around 260K tokens feels like the practical upper limit on DGX Spark.

Result 2: The Compressed KV Cache Is Really Small

Now let's actually verify the idea that "the KV cache can live on disk." Looking at the kvcache_bytes column in the table above, it grows linearly at about 14KB per token (slope ~13.8KB). Extrapolating this to 1 million tokens:

Measured points fall almost perfectly on a line. Even linearly extrapolated, 1 million tokens would be only about 12.8 GiB — well within 128GB even combined with the 81GiB weights.

Even at 260K tokens, the compressed KV is only 3.6GB. In other words, for the context lengths DGX Spark can handle, the KV fits entirely in RAM. What clicked for me most clearly after trying this is that the disk KV cache is not "a mechanism to overflow what doesn't fit in RAM" — it's "a mechanism to reuse the expensive prefill cost paid once." Where TurboQuant/RotorQuant take the approach of "shrinking KV through compression," this takes a different angle: "compression + persisting the prefix to disk."

https://dev.classmethod.jp/articles/dgx-spark-turboquant-kv-cache/

So how much does that "reuse" actually help? I started ds4-server with disk KV cache enabled, sent a ~100KB (~32K token) text with a question, evicted the session with a small intermediate request, and sent the same large prompt again.

./ds4-server --ctx 200000 --kv-disk-dir /tmp/ds4-kv --kv-disk-space-mb 24576 --port 8000

Cold (no cache): 31,930-token prefill + 24-token generation took 115.2 seconds. Warm (same prompt re-sent): disk KV cache hit, 7.7 seconds. About 15× speedup.

Looking at the trace, the warm run shows cache_source: disk-text with cached_tokens: 30720 — of 31,930 tokens, 30,720 (rounded down to a nice multiple of 2048) are restored from the disk checkpoint, and only the remaining ~1,200 tokens plus generation are redone. /tmp/ds4-kv/ contains several hundred MB .kv files, which roughly matches the expected KV size for ~30K tokens (~475MB) seen earlier.

This is exactly the scenario described in the README: "Claude Code tends to send a large initial prompt of around 25K tokens before doing any real work. With --kv-disk-dir enabled, after the first expensive prefill, that saved prefix can be reused on continuation or session resume." In practice, 115 seconds became 7.7 seconds.

Result 3: Is It Practical for Agent Use Cases?

Next, let's look at whether it's practical for agent use cases. ds4-server speaks both OpenAI-compatible and Anthropic-compatible APIs, and the README even includes a wrapper script for redirecting Claude Code's endpoint to ds4-server. I measured what a 284B-class model running locally at 12–15 tokens/sec actually feels like for long-context tasks.

First, as a baseline for generation throughput, sending "output the numbers 1 to 120 separated by spaces" gave 24-token prompt + 239-token generation in 16.1 seconds — about 14.9 tokens/sec for generation. As interactive chat, that's indeed frustratingly slow.

Next, two batch long-context tasks. First, a code review: I concatenated DwarfStar 4's own C source files (ds4.h, ds4_gpu.h, ds4_bench.c, ds4_cli.c, and others) and asked it to "identify 5 bugs or dangerous assumptions." Second, a summarization: I concatenated 9 of my own published articles about DGX Spark and asked it to "write a one-paragraph summary in Japanese and list 5 recurring themes."

The code review output raised genuinely reasonable points: fseek(SEEK_END) + ftell giving wrong byte counts in Windows text mode, fread not accounting for embedded NUL bytes in the buffer, a raw-pointer-returning API with no mechanism to prevent freeing while a view is still live (use-after-free risk), and a case in the benchmark loop where a failed snapshot save could leave values uninitialized. The remarks were substantive for a C code review (it cited filenames and line numbers, though since no line numbers were provided, some were inferred). Reading 35,088 tokens and responding took about 2 minutes 45 seconds.

For the summary, it read 56,662 tokens — nine articles' worth — and returned a well-targeted Japanese summary starting with a title like "DGX Spark's Performance Limits and MoE Model Advantages — Local AI Workflows with 128GB Unified Memory." It covered: 273 GB/s memory bandwidth as the ceiling for long-context generation, the good fit with MoE models that have fewer active parameters, lightweight techniques like LoRA SFT, MTP, and NVFP4 quantization, and even local code completion with Continue.dev + Ollama. Honestly, it was a surprisingly accurate summary. It's genuinely interesting that a 2-bit-quantized 284B can summarize Japanese technical writing this well. Total time: about 5 minutes.

In short: too slow for back-and-forth conversation, but for "feed a large input once and get a complete answer in one shot" — code review, long-form summarization, final-answer generation in a RAG pipeline — it falls squarely in DGX Spark's strong zone. And with --kv-disk-dir, the initial prefill cost doesn't have to be paid again on subsequent requests, which makes it a good fit for agent use cases where system prompts and tool definitions are prepended to every request.

https://dev.classmethod.jp/articles/dgx-spark-claude-code-local/

Summary Matrix

Putting all the results in one table:

Summary

• DwarfStar 4, just a few days after CUDA support was merged (commit a97e7a3, 2026-05-12), built on DGX Spark with just make. No issues with aarch64 + CUDA 13.0.
• 284B DeepSeek V4 Flash fit within 128GB using q2-imatrix (~81GiB), which aggressively compresses only the expert networks to 2-bit. During operation it uses 110–115GiB, and the "128GB limit" shows up in logs from the very start.
• Generation is ~8–15 tokens/sec, limited by 273 GB/s memory bandwidth — as predicted, Apple Silicon has the advantage (roughly 40% of M3 Max, 30% of M3 Ultra). On the other hand, prefill at 80–250 tokens/sec incrementally is competitive on the compute side.
• The compressed KV cache is extremely small at ~14KB per token, and fits entirely in RAM for the context lengths DGX Spark can handle. The disk KV cache is not "overflow relief" but "expensive prefill reuse," and made a ~32K-token re-send about 15× faster.
• Claude Code and agent use cases: too slow for interactive chat, but realistic for batch long-context workloads (code review, summarization, RAG final answers). Combined with --kv-disk-dir, repeated re-prefill of fixed prompts can also be avoided.

The clean behavior that comes from the deliberate choice of "single-model-dedicated engine" is genuinely satisfying. If DeepSeek releases an updated version of V4 Flash, I'd definitely want to try it again.

Reference Links

• DwarfStar 4 (formerly ds4 / ds4.c) repository: https://github.com/antirez/ds4
• DeepSeek V4 Flash dedicated GGUF: https://huggingface.co/antirez/deepseek-v4-gguf
• Verification scripts and raw data: https://github.com/himorishige/dgx-spark-blog (dwarfstar4-deepseek-v4-flash-bench/)

Mac (M3 Max / M3 Ultra) numbers were not measured in this study and are cited from the benchmark table in the DwarfStar 4 README.