I tested the practical boundary by running 320B GLM-5.3-Flash on a single DGX Spark

Introduction

Hello, I'm Morishige from Classmethod's Manufacturing Business Technology Department.

I'm sure many of you are curious about GLM-5.3-Flash, released by Z.ai on 2026-08-26. It runs a 320B parameter MoE with an active 18B, is the first native multimodal model in the GLM-5 series, and has a context of 1M tokens — all released under an MIT license with public weights, making it quite an aggressive offering.

https://huggingface.co/zai-org/GLM-5.3-Flash

However, at 320B, the weights are 642GB in BF16. My local DGX Spark is a single-unit configuration with 128GB of unified memory (effective 121GiB), so there's no way to load it as-is. The lifeline is Unsloth's Dynamic GGUF, where low-bit quantizations of 1-bit 93GB, 2-bit 109GB, and 3-bit 120GB would physically fit. Setting aside whether it loads at all, the real question is "can it be used for everyday tasks in that state?"

To give the conclusion upfront: the 2-bit (UD-Q2_K_XL, 109GB) fit on a single DGX Spark and ran at 17.7 tok/s with C=1. Code fixes scored 5/5, code generation also scored 5/5 complete answers with all 49/49 tests passing — even at this low bit depth, it reaches practical usability. However, the deciding factor wasn't the quantization. The reasoning_effort runs at max by default, taking 29 minutes for 10 Japanese tasks. When explicitly set to low, the same tasks finish in 98 seconds with no score drop. Whether you grasp this single point determines whether the same model is "usable" or "unusable."

This is part of a fixed-point observation series measuring the same tasks under the same conditions. The previous installment measured Nemotron 3.5 Lightning (article dated 2026-08-12).

https://dev.classmethod.jp/articles/dgx-spark-nemotron-3-5-lightning-first-touch/

This article introduces the characteristics of GLM-5.3-Flash and presents actual measured results from varying effort and quantization on a single DGX Spark.

An article verifying operation on 2 DGX Spark units is available here:

https://dev.classmethod.jp/articles/dgx-spark-2node-glm-5-3-flash-nvfp4-vllm/

GLM-5.3-Flash Characteristics

First, let me organize the characteristics from primary sources (Hugging Face model card, config.json, and chat_template).

What's interesting is the attention architecture. Of the 45 layers, 34 are linear attention (KDA) layers that hold no KV cache, and only 11 are DeepSeek Sparse Attention layers inserted every 4 layers in the traditional sense. The official documentation states this reduces attention computation by 3.01x and KV cache by 4.44x compared to GLM-5.3. It's a design intended to handle 1M context with realistic memory usage.

The next question is which quantization fits on a single DGX Spark. Here I list Unsloth's Dynamic GGUF sizes alongside my actual download and verification results.

The 3-bit loaded with context 65536 and 8 slots, but with only 1GiB of free memory and 15.17 tok/s — slower than the 2-bit. Since there's no room to co-locate other processes, I stopped at confirming it loads. The main evaluation uses the primary 2-bit and the more compressed 1-bit to see where quality degrades with quantization.

Getting It Running

The execution environment is a single DGX Spark (GB10, unified memory 121GiB, ~273GB/s bandwidth), with llama.cpp as the engine. However, since GLM-5.3-Flash's glm5_next architecture hasn't been merged into the upstream llama.cpp yet, I build from the branch Unsloth has published.

git clone --branch glm5next/upstream --depth 1 https://github.com/unslothai/llama.cpp
cmake llama.cpp -B llama.cpp/build \
    -DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=121
cmake --build llama.cpp/build --config Release -j \
    --target llama-cli llama-mtmd-cli llama-server llama-gguf-split

Since GB10 has compute capability 12.1, I specify CMAKE_CUDA_ARCHITECTURES=121. The build itself completed without issues. I obtained the two quantizations and the mmproj for image input from Hugging Face. The total is approximately 203GB.

hf download unsloth/GLM-5.3-Flash-GGUF \
  --include "UD-Q2_K_XL/*" --local-dir ~/models/GLM-5.3-Flash-GGUF
hf download unsloth/GLM-5.3-Flash-GGUF \
  --include "UD-IQ1_S/*" --local-dir ~/models/GLM-5.3-Flash-GGUF
hf download unsloth/GLM-5.3-Flash-GGUF \
  --include "mmproj-F16.gguf" --local-dir ~/models/GLM-5.3-Flash-GGUF

For serving, I use llama-server with --jinja to use the chat template as-is, and pass in the mmproj.

llama-server \
  --model ~/models/GLM-5.3-Flash-GGUF/UD-Q2_K_XL/GLM-5.3-Flash-UD-Q2_K_XL-00001-of-00004.gguf \
  --mmproj ~/models/GLM-5.3-Flash-GGUF/mmproj-F16.gguf \
  --host 0.0.0.0 --port 8080 \
  --ctx-size 65536 --parallel 8 \
  --n-gpu-layers 999 --jinja

With the 2-bit loaded at context 65536 and 8 slots, memory usage was 110/121GiB. Thanks to the KDA layers having no KV cache by design, even loading a 109GB model leaves enough room for context. The load log shows that tensors for blk.45.nextn.* were ignored, revealing that the model itself contains an MTP head for speculative decoding but llama.cpp doesn't support it yet.

For reasoning strength, the chat template accepts reasoning_effort at three levels: low / high / max. Since llama-server supports chat_template_kwargs per request, you can pass it from the API caller as follows:

{ "chat_template_kwargs": { "reasoning_effort": "low" } }

It's important to note that leaving it unspecified defaults to max. If you want to lighten the thinking, you need to explicitly pass low — and as I'll describe later, this is the deciding factor for practical usability.

Speed

I measured while varying concurrency with a fixed output length of 256 tokens at temperature 0. Values are in tok/s.

At C=1, the 2-bit runs at 17.72 tok/s with a TTFT of 0.333 seconds. Scaling up to 8 parallel slots brings the total to 63.97 tok/s. For a 320B-class model running on a single unit, this speed is sufficiently practical for individual interactive use.

What was surprising here was the difference from 1-bit. While the weights shrank by 14% from 108.7GB to 93.1GB, the speed only changed by 5% to 18.68 tok/s. The main cost of decoding is concentrated in reading and dequantizing the active 18B portion, so the difference in total size doesn't translate directly into speed. This means the motivation for choosing 1-bit isn't speed but rather the ability to use the freed 15GB for more context or co-located processes.

Note that if llama.cpp gains support for the MTP head (blk.45.nextn.*) that was ignored at load time, there's room for improvement through speculative decoding. A draft PR #27917 implementing the NextN head for GLM-5.3-Flash is up on the main repository, and the author reports an acceptance rate of 0.74 with a 15–30% speedup in CPU offload environments. However, the main glm5_next architecture needs to be merged first, and this isn't yet available in the Unsloth branch I used (as of 2026-08-30). My early results from trying this draft PR are added in the next section.

Trying the MTP Draft PR Early

(Added 2026-08-30) I built the branch from PR #27917 mentioned in the previous section to test how much MTP helps on a single DGX Spark. Out of the box, it couldn't read the Unsloth GGUF because the branch registers the architecture name as glm5-next while Unsloth uses glm5next — a mismatch in identifiers. After rewriting one line in src/llama-arch.cpp to glm5next and rebuilding, it loaded. Running with multiple slots requires --kv-unified, and the mmproj can't be read with this branch, so testing was text-only.

I measured three conditions on the same binary — no MTP, draft 3, and draft 5 — running at effort=low with parallelism 1 on real generation. Rather than a fixed-length speed benchmark, I calculated effective tok/s from the time taken for 4 Japanese core tasks and 5 code generation problems.

For code generation, draft 3 was 33% faster. On the other hand, Japanese prose showed only a 3.6% improvement — within the margin of error — while draft 5 had a lower acceptance rate and was actually 7% slower for Japanese. This is a straightforward result: it only helps for outputs where the draft head predicts well. The author-reported 15–30% is for CPU offload environments; on the Spark where everything fits on GPU, the baseline cost per token is lower, making the verification cost of rejected drafts relatively heavier.

It wasn't usable for parallel operation. Running draft 3 with 8 slots caused the total at C=8 to drop from this branch's no-MTP 41.09 tok/s to 24.19 tok/s — half as fast — followed immediately by a CUDA illegal memory access crash. This is expected at the draft PR stage, and similar issues have been filed on the main repository. Whether draft 5's 4/5 correctness is a side effect of MTP or variance from temperature 1.0 can't be determined from a single measurement.

For now, the sweet spot is "worth trying draft 3 when writing code in isolation; disable it for Japanese conversation and parallel operation." I plan to remeasure with aligned conditions after it's merged into the main repository.

Japanese Language

I measured 10 Japanese tasks (9 machine-graded + 1 free-response) at 3 levels of effort. This is the most deeply explored section in this article.

The effort=max rows used an output budget of 16384 tokens and 2 server slots; the low and high rows used a budget of 8192 and 8 slots. The 2-bit low was run 3 times; the table shows the first run (scores across 3 runs were 7 / 7 / 6).

First, the time required. Low took 98 seconds, high took 141 seconds, and max took 1736 seconds. Low and high tied at 7/9, while max spent 18x longer and actually scored lower at 6/9. Looking at the "tasks with thinking" column, low showed no thinking at all for 8 out of 10 tasks, effectively operating as a non-thinking mode. High thought through 7 tasks while finishing in 141 seconds, making it the realistic option if you want to retain thinking.

What happened with max is best illustrated by the task of summarizing while retaining proper nouns. The thinking consumed the full 16384 tokens and the response was cut off with an empty body — this occurred consistently regardless of the budget size (8192 or 16384) and for both 2-bit and 1-bit. Doubling the budget caused the thinking to double as well (the thinking for that task grew from roughly 14,000 characters to roughly 28,000 characters), so increasing the budget doesn't solve the problem. Z.ai's own evaluation gives a maximum generation of 163,840 tokens on HLE and 65,536 tokens on Terminal-Bench — max is a setting premised on thinking budgets of that magnitude. It simply doesn't match the scale of a local environment running at 17.7 tok/s.

Let me also look at the thinking content. Despite questions being asked in Japanese, the thinking was entirely in English. All 10 tasks under max began with The user is asking me in Japanese..., and for numerical reasoning it even wrote Let me translate the problem before solving in English. This didn't affect the Japanese output itself, but looking at the tasks that failed, it's apparent that the structure of "thinking in English while counting Japanese" is affecting instruction precision.

The failure patterns fell into three categories. The first was a task requiring answers within 50 characters, where the 2-bit failed all 5 times. However, looking at the content, 3 of those 5 times the body text was actually 42–47 characters, but the self-reported character count appended at the end was pushing it over the limit. Here is the high output:

Japan's capital is Tokyo. Once called Edo, this city developed as the political center after the Meiji Restoration.

(43 characters)

The body is 47 characters. The self-reported number is also wrong, and counting the annotation itself brings the total to 55 characters, which fails. The other 2 failures were a 3-character overshoot and one instance where the model ignored the constraint entirely and wrote 258 characters. Meanwhile, the 1-bit passed both times with 38–41 characters and no self-reported count appended.

The second failure type was an internal announcement with required and prohibited terms, where 1 of the 3 runs and the max run failed. The body text in both cases was actually correct. The failure came from a self-check appended at the end: writing "Prohibited words: '革命' and 'シームレス' were not used" caused the checker to flag it, since the prohibited words were explicitly named. The model's diligent compliance report became the very violation.

The third type was the summarization task requiring proper nouns to be retained within 150 characters, where all 6 completed runs failed to satisfy both "within 150 characters" and "6/6 proper nouns" simultaneously. Runs that fit within exactly 150 characters dropped 2 names, and the run that kept 6/6 ran to 235 characters. In all 5 runs that failed to keep 6/6, the person's name Shiraze Toru was consistently dropped, while company and product names were never dropped. When trading off length against proper nouns, the model appears to prefer dropping personal names first.

Let me also note the successes. The 4 tasks of honorific conversion, JSON schema output, transcribing figures from tables, and fact extraction from long text never failed — across both quantizations, all effort levels, and all runs. Let me also quote one free-response output: this is the 2-bit, effort=low output.

Advantages

1. Reduced commute time — Since daily travel becomes unnecessary, you can save time and transportation costs and redirect them toward work or personal life.
2. Flexible work style — Working in an environment and at times that suit you makes it easier to improve concentration and productivity.

The Japanese prose itself is natural and the honorific level doesn't break down. The weakness isn't "the ability to write in Japanese" but rather "the ability to count in Japanese" and the habit of appending unrequested self-reports. Operationally, for use cases with character count constraints, I think the practical approach is to either add "please don't write the character count" or plan to count and trim downstream.

Is It Usable as a Thinking Partner?

Single tasks alone won't reveal degradation over multiple turns or recall while holding a long document. So I created 3 article-specific probes and ran them with 2-bit at effort=low and high. Thinking was not returned to history, following the same approach as a typical chat UI.

All machine checks were perfect for both effort levels. In Scenario A, the responsible person, budget, target lines, and deadline planted in turn 1 were reproduced correctly in a table at turn 7, and the budget reduced in turn 5 was correctly used again in the summary at turn 10. Throughout all 10 turns, the Japanese ratio remained above 0.95, overlap with past utterances was at most 0.07, and no signs of degradation or recycling were visible.

The quality of the thinking partner exchanges has to be judged by reading the content, so let me share my impressions. From the very first turn, it opens with "Let me be candid" and lists the weaknesses: the definition of foreign objects is undefined, there's overlap with existing inspections, success metrics are subjective, and the quality assurance department alone can't drive this forward — then concludes with "First, let me confirm whether you have any analysis results of foreign object data. That is the starting point." In turn 3, when pushed back with "I have 400 photos of defective products," it doesn't yield — it reasons that 400 photos divided by 5 types of foreign objects is only 80 per type, and moreover, what's actually needed are photos of good products, repositioning the 400 photos as material for requirements definition rather than training data. With high effort, it goes a step further and proposes a sensible alternative: switch to an anomaly detection approach trained only on normal products and redirect the 400 photos as ground truth for validation. When the budget was cut in turn 5, it set a policy of "cut feature richness, not scope" and connected it coherently to its own previous positions.

In Scenario B, it identified the 3 planted contradictions with section numbers, and then found 2 additional contradictions I hadn't intended. One was a conflict between "notify management within 5 minutes" and "only notify management for stoppages of 10 minutes or more"; another was a conflict between "real-time display" and "cloud transfer is on a 5-minute cycle" — both of which are genuine breakdowns in the document as written. As the person who planted the contradictions, I came out the loser.

In Scenario C, the first response to pasting the roughly 10,000-token document took 46 seconds, but after that, llama.cpp's prompt cache kicked in and short questions returned in 1–8 seconds, with even turn 7 requesting 5 proposals coming back in 24 seconds. With a 10,000-token context held over 8 turns, all 4 fact recalls were correct, and the turn 6 request to "list all questions and answers so far" was also perfectly accurate for all 4 items. Pasting a long document and asking questions feels light after the initial wait.

The cost was 17–75 seconds per response with low and 17–92 seconds with high for Scenario A. It's not instant-reply chat, but for a thinking partner where you're also thinking while the model thinks, I consider that acceptable. Note that all 3 scenarios were single measurements, and the sharpness of rebuttals is a human impression.

Code Generation

Five zero-shot tasks from Japanese specifications (graded with 49 local pytest tests) and a bug-fix task on a mini repository with planted bugs. Starting with zero-shot:

All configurations except 1-bit with max scored 5/5 complete answers and 49/49 tests passing. The task set contains clauses that cause failures if skimmed over, so this is genuinely impressive. The only exception, 1-bit at effort=max scoring 4/5, was because the thinking consumed all 16384 tokens on one date parsing problem — the code couldn't be written not because the model couldn't code, but because the thinking never finished. This is the same symptom as in the Japanese section. For code generation, raising or lowering effort doesn't change results, so the conclusion here is also that low is sufficient.

For existing bug fixes, I measure agentic behavior (reading files with tools, editing them, and running tests).

Against a 30-turn limit, all 3 bugs were fixed in 5–6 turns in 72–84 seconds, with the model declaring final_answer on its own. Here, even at effort=max, the thinking didn't run away — behavior befitting the published Terminal-Bench 2.1 score of 84.3. What's interesting is that while free-response tasks cause unbounded thinking, agentic behavior with tool execution results returned every turn keeps the thinking short and contained.

Is It Usable as a Tool Caller?

Let's start with the tool call format. GLM-5.3-Flash's tool calls use a GLM-specific XML format: <tool_call>{function name}<arg_key>...<arg_value>...</tool_call>, but llama.cpp's --jinja interprets this natively and converts it to OpenAI-compatible JSON. No additional parser configuration was needed.

Single calls scored 5/5 for both quantizations, multiple tools were returned in parallel within a single message, and both completed the pseudo-filesystem agent task. The difference emerged with tool_choice=required. The 2-bit honored the forced specification and returned 1 tool call, but the 1-bit accepted the request yet returned 0 tool calls. Since it's not an error but a silent contract violation, using 1-bit in clients that rely on required is best avoided.

Next, the agent suitability probes. Note that these are probes targeting individual capabilities that harnesses depend on — they are not results from running a harness itself.

The only probe that failed across all configurations was the tool selection task at 40+ tools; everything else passed with 2-bit at effort=max. Per-harness judgments also rated 2-bit + effort=max as Hermes Agent equivalent, i.e., "usable". For clients like opencode with many tools, the multi-tool selection weakness will matter, earning a "difficult" rating there.

This is the opposite trend from Japanese and code generation — the more effort is lowered, the more probes fail. Low and high couldn't honor long system prompts of about 2,500 characters, and high also dropped template stability. Although individual measurements carry some variance, for unattended operation with fixed outputs, max seems to be required. This suggests a usage pattern of effort=low for everyday conversation and code generation, and effort=max for agentic deployment.

The shadow of quantization shows up here too for 1-bit. With effort=max, language lock breaks and it starts responding to Japanese questions in English; with effort=low, template stability also drops. The 1-bit degradation that wasn't visible in core capability tests surfaces at the edges of agentic operation.

Images

I verify the native multimodal capability through image input via mmproj. Since the tasks use synthetic images, the ground truth is precisely known.

Both scored 4/4 for transcribing 4 bar graph values, and OCR on a Japanese form showed a character error rate of 0.0 — even low-bit quantization reads figures and text perfectly. Dates, equipment numbers, and vibration values on an equipment inspection record were all transcribed without a single error.

The object enumeration task (listing protective equipment worn by workers) was auto-judged ❌ for both, but the content was completely different. The 2-bit output correctly listed 2 helmets and a reflective vest and then accurately stated "safety glasses, gloves, and masks are not visible and therefore not listed" — the ❌ was a false positive where the grader misdetected this summary negation as a hallucination. Meanwhile, the 1-bit missed the reflective vest entirely. While figures and OCR show no difference, free visual recognition reveals 1-bit degradation.

Summary

Based on actual measurements on a single DGX Spark, 2-bit (UD-Q2_K_XL) with explicit effort=low is the answer that emerged from this evaluation. This configuration runs at 17.7 tok/s, code generation scores 5/5 complete answers, bug fixing completes in 72 seconds, and tool calls including tool_choice=required honored the contract. Even across 10 turns of thinking-partner conversation, it didn't confuse premises and didn't yield under pushback — it works as a planning consultant as well. The fact that a 320B MoE is running at this level on a single desktop machine is, from a perspective one year ago, a rather remarkable sight.

The limitations were also clear. The default effort=max is configured with a thinking budget in the 100K-token range, which simply doesn't match the scale of a local 17.7 tok/s environment. Moreover, perhaps because of the structure of "thinking in English while counting Japanese," longer thinking leads to more breakdowns with character count constraints and self-annotations. The Japanese prose itself is natural — as evidenced by the honorific conversion and announcement tasks — and the weakness isn't "the ability to write in Japanese" but "the ability to count in Japanese." As for quantization, dropping to 1-bit keeps core capabilities roughly on par with 2-bit, but tool_choice contract violations, language lock failures, and image recognition misses start to appear at the edges first. Whether saving 15GB is worth that tradeoff depends on the use case, but I'd personally keep 2-bit for regular use.

The MTP results from the early draft PR were promising enough that I'll remeasure with aligned conditions after it's merged into the main repository. I'd also like to explore in a follow-up how the multi-tool selection weakness plays out when connected to real clients like opencode.

Reference Links

• GLM-5.3-Flash - Hugging Face
• GLM-5.3-Flash: How to Run Locally - Unsloth Documentation
• unsloth/GLM-5.3-Flash-GGUF - Hugging Face
• GLM-5.3-Flash - Z.AI Developer Documentation
• GLM-5: from Vibe Coding to Agentic Engineering - arXiv