I measured the value of running 320B GLM-5.3-Flash on 2 DGX Spark units to see if having 2 units is worth it

Introduction

Hello, I'm Morishige from Classmethod's Manufacturing Business Technology Department.

Last time, I ran GLM-5.3-Flash 320B on a single DGX Spark using 2-bit GGUF, achieving 17.7 tok/s. Watching a 320B MoE return responses at readable speed on a single machine sitting on a desk was quite a sight in itself. However, the quantization was compressed down to 2-bit, and running with the default max thinking setting took 15 minutes for 10 Japanese tasks. This time's topic is what happens when you connect a second machine of the same model.

https://huggingface.co/LibertAIDAI/GLM-5.3-Flash-NVFP4

With two machines, the combined unified memory is 242 GiB, which can fit the community-published NVFP4 checkpoint (181 GiB) directly. The compression is much shallower than 2-bit, and the engine is vLLM tensor parallelism. I've written about running large models by connecting two DGX Sparks twice before with DeepSeek, but this is the first time I can compare one machine versus two side by side using the same harness and same tasks as measured on a single machine. I'll use GLM-5.3-Flash to answer the question of whether a second machine is worth buying. Since it's obvious that speed will improve, what I want to see isn't just whether it works, but whether it reaches a usable level on the same tasks as the single-machine edition.

To state the conclusion upfront: single-request speed went from 17.7 → 30.8 tok/s, a 1.7x improvement, and the 10 Japanese tasks went from 927 seconds to 344 seconds. It's not just functional—it's at a level usable as a working tool. Code generation answered all 5 questions correctly, repository bug fixing completed in 54 seconds, tool calls honored the contract including tool_choice=required, and recall across 10 turns of back-and-forth got every question right. However, the real value of two machines wasn't in the speed multiplier—that's my takeaway this time. With speculative decoding and 8 slots on 262K context, the total across 8 parallel requests is 76.4 tok/s, and it returns a keyword even when fed a 200K token prompt. The configuration can be chosen by use case—one person writing code, reading long documents, a few people sharing, many people sharing—so a personal verification machine becomes a team server. However, there's currently one pitfall: Japanese text corrupts about 2 kanji characters per 10,000 characters. I've written up the cause and countermeasures as well.

Since the single-machine edition (article as of 2026-08-30) already covered the basics, reasoning_effort, quantization, and images, this article focuses on what changes with two machines.

https://dev.classmethod.jp/articles/dgx-spark-glm-5-3-flash-first-touch/

The wiring for two machines hasn't changed from the DeepSeek V4 Flash-0731 two-machine edition (article as of 2026-08-10)—still QSFP direct connection with MTU 9000 RDMA.

This article covers running the GLM-5.3-Flash NVFP4 checkpoint on two DGX Sparks with vLLM, the process of varying speculative decoding and parallelism to determine configurations by use case, comparisons using the same harness as the single-machine edition, and the cause and countermeasures for Japanese character corruption. I hope it's useful for anyone on the fence about buying a second machine.

The test environment is two QSFP-connected DGX Sparks with vLLM tensor parallelism

The configuration uses vLLM TP=2 with one machine as head and the other as worker. The head handles OpenAI-compatible API requests, and the weights are split evenly across both machines. Here is a summary table of the environment.

The configuration in diagram form is as follows.

The compose file, env, startup/shutdown scripts, and patches are all published together. For readers who want to run it today, the README instructions there are the quickest path, and this article covers the actual measurements and pitfalls behind it.

https://github.com/himorishige/glm53-flash-2x-dgx-spark-recipe

I'll note the fixed values I didn't touch upfront. --block-size 2304 is the page size required by fp8 paged MQA; removing it corrupts output without errors. --language-model-only prevents loading the multimodal front-end; without it, 15.7 GiB extra is consumed. The tool call parser is glm47; specifying glm silently drops tool calls. Startup order is to bring up the worker first, wait 25 seconds, then start the head; when stopping, both ranks must be brought down together. Stopping only one leaves the remaining GPU stuck at 100%. On the host side, setting vm.swappiness=0 and running drop_caches before startup is about all you need.

The official image couldn't start; we ran it with a community image and patch

vLLM released a dedicated image vllm/vllm-openai:glm53-flash-arm64-cu130 to coincide with the GLM-5.3-Flash release, but we couldn't start this model with that image in a two-machine GB10 configuration. Weight loading and NCCL connection went through, but the following assert appeared during warmup.

RuntimeError: concat_and_cache_mla, /workspace/csrc/libtorch_stable/cache_kernels.cu:866, pe_dim must be 64 for fp8_ds_mla

Changing --kv-cache-dtype to fp8, fp8_e4m3, or auto made no difference. GLM-5.3-Flash is a NoPE model with rope dimension 0, and the stock kernel doesn't accept the cache path used by the DeepSeek Sparse Attention indexer. Every report of this working on two Sparks used a patched image or plugin; no example of the stock image working alone was found.

The approach I took was: the community image v11 published by tonyd2wild, with the SM121 top-k patch distributed by the same person applied via compose bind mount. The patch has 2 differences from stock and avoids persistent_topk that doesn't fit in GB10's shared memory. Without it, the engine reportedly crashes on long prompts; indeed, with this configuration, prompts up to 200K went through. Startup took 988 seconds, with three gates confirmed: model list, one math problem, and tool call.

Speed is determined by speculative decoding, and DFlash2 was the fastest

With the system up, I changed one factor at a time from the day-1 configuration. Using forced-length 256-token greedy decoding, C=1 is the median of 3 runs, and code generation is the effective tok/s when actually solving 5 problems with effort=low. All rows use the v11 image + top-k patch + fp8_e4m3 KV, 262K context, 2 slots.

The no-speculation baseline of 14.55 tok/s matches community reports (14.3–14.6), and this is our denominator. Just using the MTP heads built into the model yields 24.53–27.78 tok/s, a +76–91% improvement. k=3 was fastest for single requests, k=4 was best for 2-parallel totals, and k=5 dropped off with parallelism—the same pattern as when draft=3 worked best with a single-machine llama.cpp.

DFlash2 pushed even further. It's a dedicated drafter for GLM-5.3-Flash published by incoai, loading a separate 2.2 GB model alongside. Single-request was 30.84 tok/s, code generation 28.24 tok/s, which is +20% single-request and +27% code generation compared to MTP k=4. I attribute the speed difference to acceptance rate differences, though I didn't measure acceptance rate directly this time. One note: this drafter is distributed under CC BY-NC-ND 4.0. I used it for verification, but for commercial use, the MTP k=4 row is the upper limit.

KV cache type and context length had no effect on single-request speed. Switching KV to bf16 gives 25.62 tok/s single-request—same as k=4's 25.67—and 2-parallel total of 30.2, below fp8's 37.0. Whether keeping 262K or dropping to 65K, single-request is 27.2 vs 28.0 and 8-parallel total is 76.4 vs 76.0—no difference. There's no need to cut context to gain speed. I treat rows within 3% of each other as equivalent.

MoE backend flashinfer_cutlass, CUDA graph, batched 8192, autotune off, b12x, and util 0.90 were not measured this time.

8 slots fit with 262K context, and 2 configurations are enough for all use cases

Next, parallelism. In the single-machine edition, the limit was 8 slots with 65K context, but with two machines, 8 slots fit with 262K intact. Of GLM-5.3-Flash's 45 layers, 34 are linear attention (KDA) with no KV cache, and the KV pool fixed at 9 GiB holds 1,103,764 tokens according to vLLM's startup log. Here are the parallel totals by configuration.

Single-request speed doesn't drop with 8 slots, giving 8-parallel total of 76.4 tok/s with MTP k=4. A single-machine llama.cpp gave 63.97 tok/s at 8-parallel, so two machines come out ahead even in total. On the other hand, loading DFlash2 for 8-slot serving peaks at 70.8 tok/s at 4-parallel and drops to 58.9 at 8-parallel. I believe the drafter is competing for computation with parallel requests, but this is interpretation. The conclusion is to use DFlash2 for single-user workloads and MTP k=4 for shared serving.

From these measurements, the configuration boils down to two options. For single-user, configuration S with 262K, 2 slots, and DFlash2; for shared multi-user, 262K with 8 slots and MTP k=4. There's no need to drop context to 65K as in the single-machine edition—speed is the same at 262K. The per-use-case determination is summarized in the "Which to choose by use case" section after the quality results.

In the published recipe, configuration S is the default, and the shared configuration is switched via OVERRIDES at startup.

# Shared use: remove drafter, use MTP k=4 with 262K × 8 slots
OVERRIDES="SPEC_METHOD=mtp MTP_NUM_TOKENS=4 KV_CACHE_MEMORY_BYTES=9663676416 MAX_NUM_SEQS=8" scripts/start-worker.sh

Previously it was "a box for personal use," but when 76 tok/s total comes out while holding 8 threads of 262K context, it works as a server for several people in a department. The biggest change from going to two machines wasn't the speed numbers—it was this breadth of use cases.

A keyword was returned even with a 200K token prompt

Long-form testing was done with configuration S. A unique haystack was prepared per request with a keyword embedded at 50% depth, and TTFT, prefill, and decode were measured with forced-length 512 tokens, with recall verified by asking for the keyword in a separate request.

Prefill is flat at 1,391–1,510 tok/s all the way to 200K, and TTFT scales proportionally with length. For a 200K token prompt, the first character takes 139.2 seconds to return, and the keyword came back correctly for all 5 sizes. The decode spikes at 8K and 32K because the speculation acceptance rate improves—when the output recites the haystack, the drafter tends to guess correctly. Since decode with speculation depends on the content being generated, it's safer to look at TTFT and prefill as long-form indicators.

I also ran two simultaneous requests. Sending two 200K requests simultaneously gives TTFT of 218.9 seconds, per-request prefill of 1,052 tok/s, combined decode of 3.3 tok/s, and both complete in 310 seconds. That amounts to holding 400K total tokens of context simultaneously. The context limit is set to 262K, so I didn't test up to 1M this time.

What changes when measured with the same harness as the single-machine edition

From here, I ran the same tasks as the single-machine edition on a serve with 8 slots and DFlash2. Context length alone was matched to the single-machine 65K, but as shown in the previous section, it has no effect on speed. The single-machine column uses the previous values from running 2-bit GGUF on llama.cpp.

The KV cache type is fp8_e4m3 for two machines and f16 for one machine—this is the one thing I couldn't align. The effort=max row uses an 8192 token budget, and one task—summarizing while preserving proper nouns—on both machines ran out of thinking budget and returned an empty body. That task was re-measured in a separate row with budget 16,384, giving 6/9, 431.3 seconds for two machines.

Time is shorter across every item. Japanese 10 tasks effort=max went from 927 to 344 seconds, back-and-forth A from 433 to 354 seconds, code repair from 72 to 54 seconds. Even running with the default max, responses come back in just under 6 minutes, so the feeling from the single-machine edition that "max's timing doesn't match local speed" eases somewhat with two machines. For pass/fail, the tasks that fail are still the same faces—50-character constraint, specified words, proper nouns—as in the single-machine edition. effort=high's 5/9 looks lower than the single-machine's 7/9, but when I ran low 3 times in the single-machine edition, it also varied at 7/9, 7/9, 6/9, so you can't call it a capability difference from a single measurement. Machine-scored recall across 3 back-and-forth scenarios was perfect again.

To be upfront about parallelism: DFlash2's 8-parallel total is 58.85 tok/s, which is below the single-machine's 63.97. As seen in the previous section, DFlash2 is disadvantaged under parallelism. Per-request is 15.9 tok/s, faster than the single-machine's 8.42, and for shared use, MTP k=4 reaches 76.4. Please don't read the total alone and conclude that two machines are slower.

The agent aptitude table probes individual capabilities the harness depends on—it's not the result of running the actual harness. With two machines, only selection from 40+ tools fails across all 3 levels, while the long system prompt that failed low and high on the single machine now passes. These are also single measurements, so I'm not claiming it's due to quantization differences. The verdict is the same as the single-machine edition: challenging for clients with many tools like opencode, usable at Hermes Agent level.

Japanese corrupts about 2 kanji per 10,000 characters

Midway through the main testing, I noticed unfamiliar characters mixed into the back-and-forth output. In the 27,757 characters of back-and-forths A and B run on the main serve, there were 7 instances of U+FFFD replacement characters. Cases like "許容<U+FFFD>囲" and "<U+FFFD>実的" showed single kanji missing, while the same probes run on the single-machine 2-bit GGUF across 28,000+ characters had zero. Having decided configuration based on speed, I found there's a separate condition for Japanese use.

I eliminated conditions one by one. The U+FFFD count in the table is the number of replacement characters in the entire output.

Changing or removing the speculation type doesn't help. Trimming the tail distribution with top_k 40 or min_p 0.05 to match llama.cpp defaults doesn't eliminate it, and switching KV to bf16 doesn't either. In other words, it's not a sampler tail issue or KV precision issue—the model is selecting invalid tokens as primary candidates.

I made a wrong call here. When I decoded the token sequence myself from logprobs bytes, the corruption disappeared, so I concluded it was a display bug in vLLM's incremental detokenizer and wrote that in the table. However, when I took the token IDs received via return_token_ids and decoded them offline with the tokenizer.json bundled with the checkpoint, replacement characters reappeared at the same positions as in content (3 out of 3 cases). The bytes in logprobs is the diff after passing through the detokenizer—not the raw vocabulary bytes. Since I was using output from the same pipeline to judge a defect in that pipeline, it was naturally invisible. I retracted that conclusion and decided to suspect the token sequence itself.

The cause is the tokenizer's split of 2-byte fragments and 1-byte continuations

The tokenizer has the same md5 as the zai-org original, so it's not a checkpoint modification. Looking at the vocabulary with the tokenizers library, GLM-5.3's byte-level BPE doesn't have most modern Japanese kanji as single tokens; instead it spells them as 2 tokens: "2-byte fragment + 1-byte continuation." For example: 測 is e6b8 + ac, 範 is e7af + 84, 陥 is e999 + a5. Of the 154,820 vocabulary items, 1,095 tokens are not valid UTF-8 on their own. The model sometimes outputs a 2-byte fragment and then skips the 1-byte continuation to move on to the next character, producing replacement characters.

Statistics on corrupted words match this split pattern. "現実" (reality) was corrupted 11 of 66 occurrences, "効果測定" (effectiveness measurement) 5 times, and others including 許容範囲 (tolerance range), 陥る (fall into), 桁 (digit), 継ぎ (splice), 併用 (combined use), 拡大 (expansion), and 毀損 (damage)—all containing characters spelled via the fragment path. Meanwhile "現場" (site/scene), which can be spelled as 1 character per token, appeared 95 times without a single corruption. It hasn't been isolated which layer the continuation judgment breaks down in, whether it's in the NVFP4 marlin path or the v11 kernel. Running the same probe on a single-machine 2-bit GGUF with llama.cpp gives zero, so it's somewhere in this configuration, not in the model weights themselves.

UTF-8 guard brings it to 0, but can't be used with speculation

Even without knowing the cause layer, the symptom can be blocked. I wrote utf8_guard_lp.py as a vLLM v1 logits processor that masks tokens causing byte sequence corruption at every step. Immediately after a token ending mid-character, only tokens starting with the missing continuation bytes are allowed; at character boundaries, tokens starting with continuation bytes are forbidden; and tokens that are individually invalid are always masked. The model is forced to finish any character it starts writing.

OVERRIDES="MTP_NUM_TOKENS=0 LOGITS_PROCESSORS=utf8_guard_lp:Utf8GuardLogitsProcessor" scripts/start-worker.sh

This gives 0 replacement characters in 24,590 characters, all back-and-forth recall questions answered correctly, and Japanese character ratio maintained at 0.945 or above. Targeted words also spell correctly: 現実 15 times, 測定 14 times, 範囲 11 times.

The tradeoff is speed. Since vLLM doesn't accept custom logits processors during speculative decoding, using the guard requires disabling speculation, and single-request speed drops back to the no-speculation 14.6 tok/s. Comparing 10-turn back-and-forth A: DFlash2's 353.8 seconds becomes 478.6 seconds with the guard. For use cases where a single missing character is unacceptable—contracts, procedures—use the guard; for conversation and code, use speculation. This is the current split. I've added the same option as a fourth configuration in the recipe.

Which to choose by use case

Let me draw a line between "it works" and "it's usable." Here, I define usable as meeting all 4 criteria: not failing harness tasks, waiting times appropriate for the use case, honoring contracts like tool_choice, and no character corruption. If one condition applies, it's conditional.

The top 3 rows in the table all pass the tasks. Speed-wise, 17.7 tok/s on a single machine was already readable, but when code repair goes from 72 to 54 seconds and the default effort=max Japanese 10 tasks go from 15 minutes to just under 6 minutes, the feeling of waiting changes considerably. Personally, I think writing and fixing code as a single user, and shared back-and-forth use by several people, were the most satisfying use cases this time.

What resonated most was that a 320B model becomes a team tool with two machines. It fixes a Python repository with failing tests in 6 turns and 54 seconds, answers 8 questions over a 12,242-character document in 62 seconds, and returns keywords even from a 200K token context. It maintains 15.5 tok/s per person with 4 simultaneous users, running not in the cloud but on two machines on a desk.

To be honest about the two conditional rows: the use case requiring zero dropped Japanese characters works, since replacing speculation with the guard brings replacement characters to zero, but single 14.6 tok/s is slower than the single-machine 2-bit, and the waiting feeling comes back. Agent residency remains challenging for clients with many tools, since the probe for selecting the right tool from 40+ tools fails across all 3 levels—this didn't change with two machines.

For reference, I'll compare against sending the same prompts to Fireworks' GLM-5.3-Flash (glm-5p3-flash) from my local Mac. Single-request median is 46.6 tok/s (10 runs, 34.0–51.8), 1.5x the two-machine 30.8; at 8-parallel, 60 tok/s per request, 397 tok/s total—5x the two-machine result. The gap is largest for long inputs: 200K token TTFT is 7.49 seconds on the service versus 139.2 seconds locally. This is one measurement in one time slot on a shared tenant with large run-to-run variance, but it's safe to read as: there are almost no scenarios where two local machines beat the service on speed. If there's still a reason to choose two local machines, it's three things: data doesn't leave your premises, you hold 262K context in your own box, and there's no token billing. The "usable as a team tool" in this article is premised on those conditions.

Summary

I ran GLM-5.3-Flash NVFP4 on two DGX Sparks with vLLM, varied speculative decoding and parallelism, and compared against the same tasks from the single-machine edition.

Is it usable? Yes. The four use cases—single user writing and fixing code, reading long documents, back-and-forth with a few people, and sharing among many—all passed the tasks and ran faster than a single machine. Configuration is chosen by use case. For single user: configuration S with 262K and DFlash2 at 30.8 tok/s. For shared multi-user: 262K with 8 slots and MTP k=4 at 76.4 tok/s total. The value of two machines was in this breadth, not the 1.7x single-request speed increase.

There are two conditional cases. For use cases requiring zero dropped Japanese characters, disable speculation and add the UTF-8 guard to get 14.6 tok/s with zero replacement characters. Agent clients with many tools remain challenging since the multi-tool selection weakness persists with two machines.

I'll also note the limitations. The official stock image couldn't start; what worked was a community image and patch. The DFlash2 drafter is CC BY-NC-ND 4.0, so MTP k=4 is the upper limit for commercial use. The corruption cause layer hasn't been isolated, and the guard can't be combined with speculation. MoE backend cutlass, CUDA graph, and batched 8192 weren't measured. Single-request speed also fluctuates in the 30.8–32.3 tok/s range.

Next, I'd like to run the same checkpoint on SGLang to see if corruption appears and isolate the cause layer. Testing with the separate NVFP4 checkpoint from RedHat should also let me check whether it's checkpoint-specific.

Reference Links

• LibertAIDAI/GLM-5.3-Flash-NVFP4 - Hugging Face
• zai-org/GLM-5.3-Flash - Hugging Face
• incoai/GLM-5.3-Flash-DFlash2 - Hugging Face — DFlash2 drafter (CC BY-NC-ND 4.0)
• tonyd2wild/GLM-5.3-Flash-NVFP4-DFlash2-2x-DGX-Spark — v11 image and SM121 top-k patch
• Libertai/glm53-flash-vllm-gb10 — stock image assert and cutlass util floor
• himorishige/glm53-flash-2x-dgx-spark-recipe — compose, scripts, and patches used this time
• GLM 5.3 Flash API & Playground | Fireworks AI — API used for reference comparison
• Running 320B GLM-5.3-Flash on a Single DGX Spark to Measure the Boundary of Practical Use (Published 2026-08-30)
• Running 284B DeepSeek V4 Flash-0731 on Two DGX Sparks (Published 2026-08-10)
• Connecting Two DGX Sparks with a Cluster Cable (Published February 2026)