I tried running the 320B GLM-5.3-Flash on a single DGX Spark to measure the dividing line of practical usability

Introduction

Hello, I'm Morishige from Classmethod's Manufacturing Business Technology Department.

Many of you are probably curious about GLM-5.3-Flash, released by Z.ai on 2026-08-26. It runs a 320B parameter MoE with an active 18B, features the first native multimodal in the GLM-5 series, and offers a context of 1M tokens. And it's quite aggressive in that the weights are released under the MIT license.

https://huggingface.co/zai-org/GLM-5.3-Flash

However, since it's 320B, the weights are 642GB in BF16. My DGX Spark is a single-unit configuration with 128GB of unified memory (effective 121GiB), so there's no way to fit it as-is. The lifeline is Unsloth's Dynamic GGUF, where low-bit quantizations of 1-bit at 93GB, 2-bit at 109GB, and 3-bit at 120GB should physically fit. While they do fit, the question is "can it be used on a daily basis in that state."

Let me state the conclusion upfront: the 2-bit (UD-Q2_K_XL, 109GB) fits on a single DGX Spark and ran at 17.7 tok/s with C=1. Code fixes were 5/5, code generation was also 5/5 complete answers with 49/49 tests all passing, reaching practical levels even at this low bit depth. However, the dividing line was not quantization. reasoning_effort runs at max by default, taking 29 minutes for 10 Japanese tasks. Explicitly setting it to low completes the same tasks in 98 seconds with no score drop. Whether you grasp this single point determines whether the same model is "usable" or "unusable."

This is one entry in a fixed-point observation series measuring with the same tasks and same conditions; the previous entry measured Nemotron 3.5 Lightning (article dated 2026-08-12).

https://dev.classmethod.jp/articles/dgx-spark-nemotron-3-5-lightning-first-touch/

This article introduces GLM-5.3-Flash's characteristics and presents actual measurement results from varying effort and quantization on a single DGX Spark.

GLM-5.3-Flash Characteristics

First, let me organize the characteristics from primary sources (Hugging Face model card, config.json, chat_template).

What's interesting is the attention architecture. Of the 45 layers, 34 are linear attention (KDA) without KV cache, and only 11 layers of DeepSeek Sparse Attention inserted every 4 layers have attention in the conventional sense. The official documentation states a 3.01x reduction in attention computation and 4.44x reduction in KV cache compared to GLM-5.3. It's a design to handle 1M context with realistic memory.

The question here is which quantization will fit on a single DGX Spark. Here are the Unsloth Dynamic GGUF sizes alongside judgments confirmed by actually downloading them.

3-bit did fit with context 65536 and 8 slots, but only 1GiB remained and it ran at 15.17 tok/s, slower than 2-bit. Since there's no room to run other processes alongside it, I stopped at just confirming the load. The main tests run the two tiers of 2-bit as the primary target and the further-reduced 1-bit, also examining where things break down with quantization.

Getting It Running

The execution environment is a single DGX Spark (GB10, unified memory 121GiB, bandwidth ~273GB/s), using llama.cpp as the engine. However, since GLM-5.3-Flash's glm5_next architecture hasn't been merged into mainline llama.cpp yet, we build from the branch published by Unsloth.

git clone --branch glm5next/upstream --depth 1 https://github.com/unslothai/llama.cpp
cmake llama.cpp -B llama.cpp/build \
    -DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=121
cmake --build llama.cpp/build --config Release -j \
    --target llama-cli llama-mtmd-cli llama-server llama-gguf-split

Since GB10 has compute capability 12.1, specify CMAKE_CUDA_ARCHITECTURES=121. The build itself went through without issues. For the model, download both quantizations and the mmproj for image input from Hugging Face. A total of about 203GB.

hf download unsloth/GLM-5.3-Flash-GGUF \
  --include "UD-Q2_K_XL/*" --local-dir ~/models/GLM-5.3-Flash-GGUF
hf download unsloth/GLM-5.3-Flash-GGUF \
  --include "UD-IQ1_S/*" --local-dir ~/models/GLM-5.3-Flash-GGUF
hf download unsloth/GLM-5.3-Flash-GGUF \
  --include "mmproj-F16.gguf" --local-dir ~/models/GLM-5.3-Flash-GGUF

For serving, use llama-server, passing --jinja to use the chat template as-is and mmproj.

llama-server \
  --model ~/models/GLM-5.3-Flash-GGUF/UD-Q2_K_XL/GLM-5.3-Flash-UD-Q2_K_XL-00001-of-00004.gguf \
  --mmproj ~/models/GLM-5.3-Flash-GGUF/mmproj-F16.gguf \
  --host 0.0.0.0 --port 8080 \
  --ctx-size 65536 --parallel 8 \
  --n-gpu-layers 999 --jinja

With 2-bit started at context 65536 and 8 slots, memory usage was 110/121GiB. Perhaps thanks to the KDA layers not having KV, even loading a 109GB model leaves room for the context. The load log shows that tensors from blk.45.nextn.* were ignored, revealing that the model itself has a built-in MTP head for speculative decoding, but llama.cpp doesn't support it yet.

For thinking strength, the chat template accepts reasoning_effort in three levels: low / high / max. Since llama-server supports chat_template_kwargs per request, you can pass it from the API call side like this:

{ "chat_template_kwargs": { "reasoning_effort": "low" } }

Note that if unspecified, it resolves to max. If you want "lighter thinking," you need to explicitly pass low, and as I'll describe later, this becomes the dividing line for practical use.

Speed

Measured with forced length of 256 tokens and temperature 0, varying concurrency. Numbers are tok/s.

At C=1, 2-bit runs at 17.72 tok/s with TTFT of 0.333 seconds. Stacking up to 8 parallel extends total to 63.97 tok/s. For a 320B-class model running on a single unit at this speed, it's sufficiently practical for single-user dialogue.

What was surprising here was the gap from 1-bit. Even though the weights dropped 14% from 108.7GB to 93.1GB, speed only changed 5% to 18.68 tok/s. The primary decode cost concentrates on reading and dequantizing the active 18B portion, so the total size difference doesn't translate directly to speed. This means the motivation to choose 1-bit is not speed but rather freeing up 15GB for context or co-resident processes.

Also, once llama.cpp can use the MTP head (blk.45.nextn.*) that was being ignored at load time, there's room to grow with speculative decoding. PR #27917, which implements a NextN head for GLM-5.3-Flash, is out as a draft on mainline, with the author reporting an acceptance rate of 0.74 and 15-30% speedup in CPU offload environments. However, the glm5_next body needs to merge first, and the Unsloth branch used this time (as of 2026-08-30) doesn't support it yet. Results from trying this draft PR ahead of time are added in the next chapter.

Trying Out MTP with the Draft PR

(Added 2026-08-30) I built the branch from PR #27917 mentioned in the previous chapter and tested how much MTP affects things on a single DGX Spark. It couldn't read the Unsloth GGUF as-is. The branch side registers the architecture name as glm-5-next while Unsloth's side registers it as glm5next, causing identifier mismatch, but rewriting the one line in the name table in src/llama-arch.cpp to glm5next and rebuilding got it loaded. --kv-unified is required to run with multiple slots, and mmproj cannot be loaded with this branch, so text only.

What I measured was actual generation running three conditions—no MTP, draft 3, and draft 5—with the same binary at effort=low and parallel 1. Rather than a forced-length speed benchmark, I derived effective tok/s from the time taken for 4 Japanese core tasks and 5 code generation problems.

For code generation, draft 3 was 33% faster. On the other hand, Japanese prose was up 3.6%, within the margin of error, and draft 5's acceptance rate dropped, making it 7% slower for Japanese. A straightforward result that it only works for outputs the draft head can predict well. The author's reported 15-30% is for CPU offload environments; on a Spark where everything fits in GPU, the raw cost per token is smaller, making the verification cost of rejected drafts relatively heavier.

It couldn't be used for parallel operation. Running draft 3 with 8 slots halved the C=8 total from 41.09 tok/s for this branch's no-MTP to 24.19 tok/s, then it crashed immediately after with a CUDA illegal memory access. Expected at the draft PR stage, and similar issues have been filed on the mainline. Whether draft 5's 4/5 correctness is a side effect of MTP or temperature 1.0 variance can't be determined from a single measurement.

The current use case is "worth trying draft 3 when generating code in isolation; disable it for Japanese dialogue and parallel operation." I plan to measure again with aligned conditions after the mainline merge.

Japanese

Measured 10 Japanese tasks (9 machine-judged + 1 open-ended) across 3 effort levels. This is the most deeply explored chapter in this article.

The effort=max rows were measured with output budget 16384 tokens and 2 server slots; low and high rows with budget 8192 and 8 slots. 2-bit low was run 3 times; the table shows run 1 (scores across 3 runs: 7 / 7 / 6).

First, time taken. low is 98 seconds, high is 141 seconds, max is 1736 seconds. low and high have the same score of 7/9, while max takes 18x longer and scores 6/9, actually lower. Looking at the "tasks with thinking" column, low wrote no thinking at all for 8 out of 10 tasks, effectively operating as a non-thinking mode. high thinks through 7 tasks while fitting within 141 seconds, making it the practical level if you want to keep thinking.

What happens with max is well illustrated by the task of summarizing while preserving proper nouns. The thinking consumes all 16384 tokens with the main text left empty and cut off, and this phenomenon reproduced with an 8192 budget as well, with both 2-bit and 1-bit. Doubling the budget also doubles the thinking (the thinking for the same task expanded from about 14,000 to about 28,000 characters), so increasing the budget doesn't solve it. Z.ai's own evaluation gives GLM-5.3-Flash a maximum generation of 163,840 tokens on HLE and 65,536 tokens on Terminal-Bench; max is a setting that assumes a thinking budget of that magnitude. It's orders of magnitude incompatible with local speed at 17.7 tok/s.

Let me also look at the thinking content. Even though the questions were in Japanese, the thinking was entirely in English. All 10 tasks at max started with The user is asking me in Japanese..., and for numerical reasoning it even translated with Let me translate the problem before solving. This itself doesn't affect the Japanese in the output, but looking at the tasks that failed, you can see how the structure of "thinking in English and counting in Japanese" affects instruction accuracy.

The failure patterns can be organized into 3 types. The first is tasks requiring answers within 50 characters, where 2-bit failed all 5 times. However, looking at the content, 3 of those times the body was 42-47 characters, and the self-reported character count appended at the end pushed it past the constraint. Here is high's output:

Japan's capital is Tokyo. Once called Edo, this city developed as the political center after the Meiji Restoration.

(43 characters)

The body is 47 characters. The self-reported number is also wrong, and counting including that annotation comes to 55 characters, failing. The remaining 2 times were one that exceeded the limit by 3 characters, and one that ignored the constraint and wrote 258 characters. Meanwhile, 1-bit passed both times at 38-41 characters without appending a self-report.

The second type is an internal notice with required and forbidden words, where 1 out of 3 runs and the max run failed. The body itself passed in both cases. The reason for failure was the self-check appended at the end: "Forbidden words: 'revolution' and 'seamless' are not used" — naming the forbidden words caused the judgment to flag it. The careful verification report itself became the violation.

The third type is the task of summarizing within 150 characters while preserving proper nouns, where all 6 completed runs failed to satisfy both "within 150 characters" and "6/6 proper nouns." Runs that fit within exactly 150 characters dropped 2 names, while runs preserving 6/6 had 235 characters. All 5 runs that failed to preserve 6/6 dropped the personal name Shirasei Toru, while company names and product names were never dropped. There appears to be a tendency to trim personal names first when balancing length against proper nouns.

Let me also write about what passed. The 4 tasks of keigo conversion, JSON schema output, numerical transcription from tables/charts, and fact extraction from long text never failed across both quantizations, all effort levels, and all runs. Let me also quote one free-form answer. This is from 2-bit with effort=low:

Merits

1. Reduced commute time — Since daily travel is no longer needed, you can save time and transportation costs, using that time for work or personal activities.
2. Flexible working style — You can work in an environment and at hours that suit you, making it easier to improve concentration and productivity.

The Japanese text itself is natural, and keigo remains consistent. The weakness is not "ability to write in Japanese" but "ability to count in Japanese," combined with a habit of adding unrequested self-annotations. As a practical measure, for use cases with character count constraints, adding a note like "please don't write the character count" or planning to count and trim downstream seems realistic.

Can It Be Used for Brainstorming?

Single tasks alone don't tell us about degradation over several turns or recall while holding long documents. So I put together 3 article-specific probes and ran 2-bit with effort=low and high. Thinking was not returned to history, using the same method as a typical chat UI.

Machine verification was fully correct for both effort levels. In A, the person in charge, budget, target line, and deadline planted in turn 1 were accurately reproduced in a table at turn 7, and the budget reduced at turn 5 was correctly used in the summary at turn 10. Across all 10 turns, Japanese ratio was 0.95 or higher, maximum overlap with past utterances was 0.07 with no rehashing, and no signs of degradation were visible.

The quality of brainstorming can only be judged by reading the full text, so let me share my impressions. From turn 1, it opened with "I'll give you my frank assessment" and listed weaknesses: the definition of foreign objects is undecided, overlap with existing inspection, subjective success metrics, and that quality assurance alone can't move things forward, then responded with "first, let me check whether there are analysis results for foreign object data. That's the starting point." Even when pushed back at turn 3 with "we have 400 photos of defective products," it didn't back down, reasoning that 400 divided by 5 types of foreign objects is 80 per type, and actually what's needed is normal product images, repositioning the 400 photos not as training data but as material for requirements definition. With high, it goes a step further and proposes a sensible alternative of switching to an anomaly detection approach that trains only on normal products and redirecting the 400 photos to serve as ground truth validation data. For the budget cut at turn 5, it set the policy of "cut the feature richness, not the scope," staying consistent with its own previous arguments without contradiction.

In B, it picked up all 3 planted contradictions with chapter numbers, and additionally found 2 contradictions I hadn't intended. One was a discrepancy between "notify management department within 5 minutes" and "only notify management department for stoppages of 10 minutes or more," and the other was a discrepancy between "real-time display" and "cloud transfer is on a 5-minute cycle" — both are genuine document failures. As a reviewer, the one who planted them came out the loser.

In C, the first response after pasting a roughly 10,000-token document took 46 seconds, but thereafter llama.cpp's prompt cache kicked in, returning short questions in 1-8 seconds and even turn 7 asking for 5 proposals in 24 seconds. Carrying a 10,000-token context through 8 turns, recall of all 4 facts was correct, and the turn 6 request to "list the questions and answers so far" was also accurate for all 4. The feel of pasting a long document and asking questions is that you only wait on the first one and it's light thereafter.

Cost was 17-75 seconds per response in A for low and 17-92 seconds for high. It's not instant-reply chat speed, but for brainstorming where you're also thinking while the model thinks, it feels within acceptable range. Note that all 3 scenarios were measured once, and the sharpness of pushback is a subjective impression from reading.

Code Generation

5 problems of zero-shot writing from Japanese specifications (scored by local pytest with 49 tests), and a bug-fixing task on a mini-repository with injected bugs. Starting with zero-shot writing.

All configurations except 1-bit max scored 5/5 complete with 49/49 all passing. Since the task set includes items that will fail if you skip over them, this is genuinely strong. The only case of 4/5 for 1-bit effort=max was because for one date parsing problem the thinking consumed all 16384 tokens, so it wasn't that the code couldn't be written but that the thinking didn't finish — the same symptom as in the Japanese chapter. For code generation, raising or lowering effort doesn't change the result, so the conclusion here too is that low is sufficient.

Fixing existing bugs is measured with agentic operation (reading files with tools, editing, running tests).

Against a 30-turn limit, it fixed all 3 bugs in 5-6 turns over 72-84 seconds and declared final_answer on its own. Here, even with effort=max, the thinking didn't run away, living up to the published score of 84.3 on Terminal-Bench 2.1. What's interesting is that while thinking doesn't stop in open-ended tasks, it stays short in agentic operation where tool execution results return every turn.

Tool Use

Let's start by looking at the tool call format. GLM-5.3-Flash's tool calls use GLM's proprietary XML format <tool_call>{function name}<arg_key>...<arg_value>...</tool_call>, but llama.cpp's --jinja natively interprets this and returns it converted to OpenAI-compatible JSON. No additional parser configuration was needed.

Single calls were 5/5 for both quantizations, multiple tools return as parallel in one message, and both completed the agent task of operating a pseudo-filesystem. The difference appeared with tool_choice=required. 2-bit honored the forced specification and returned 1 tool call, but 1-bit accepted the request and returned 0 tool calls. Since it fails silently rather than with an error, using 1-bit with clients that assume required is best avoided.

Next are probes for agent suitability. Please note that these are probes targeting individual capabilities that harnesses depend on, not results from running the harness itself.

The only probe failing across all configurations is the selection task from around 40 tools, and 2-bit effort=max passes everything else. Even in per-harness judgment, 2-bit + effort=max gets a "usable" rating equivalent to Hermes Agent. For clients like opencode with many tools, the weakness in multi-tool selection has an impact, resulting in a "tough" rating.

This shows the opposite trend from Japanese and code generation — the more effort is lowered, the more probes fail. low and high couldn't follow the long ~2,500-character system prompt, and high also dropped template stability. Though there's variance in single measurements, the reading this time is that max is required for unattended operation with formatted output. It looks like the use case will be switching effort by purpose: low for everyday dialogue and code generation, max when running as an agent.

The 1-bit shadow appears here too. With effort=max, language locking breaks down and it starts answering Japanese questions in English, and with effort=low, template stability also drops. Degradation in 1-bit that wasn't visible in core capability metrics shows up in the edges of agent operation.

Images

Let's verify the native multimodal claim via image input through mmproj. The tasks use synthetic images so the correct answers are strictly determined.

Bar chart transcription of 4 values was 4/4 for both, and OCR on Japanese forms had a character error rate of 0.0 — even at low-bit quantization, tables and text can be read perfectly. Not a single character was missed from dates, equipment numbers, or vibration values on the equipment inspection record.

Object enumeration (a task to list protective gear on workers) was auto-judged ❌ for both, but the content was entirely different. The 2-bit output correctly listed 2 helmets and a reflective vest, then correctly denied with "protective goggles, gloves, and masks are not visible in the image, so they are not listed," and the judge falsely detected this summary denial as hallucination — a false positive. Meanwhile, 1-bit missed the reflective vest itself. Even though there's no difference in tables and OCR, 1-bit degradation appears in open-ended visual recognition.

Summary

For using GLM-5.3-Flash on a single DGX Spark, the answer from this round of measurements is 2-bit (UD-Q2_K_XL) + explicit effort=low. With this configuration, it runs at 17.7 tok/s, code generation was 5/5 complete, bug fixing ran to completion in 72 seconds, and tool calls including tool_choice=required worked as contracted. Over 10 turns of brainstorming, it didn't lose track of premises and didn't back down from pushback, making it viable as a planning partner. A 320B MoE running at this level on a single desktop machine is quite a surreal sight compared to how things felt a year ago.

Limitations were also clear. The default effort=max is a setting that assumes a thinking budget in the 100,000-token range, which is orders of magnitude out of sync with local speed. And perhaps due to the structure of thinking in English and counting in Japanese, the longer it thinks, the more it breaks down on character count constraints and self-annotations. The Japanese text itself is natural judging by keigo conversion and notice writing; the weakness is not "ability to write in Japanese" but "ability to count in Japanese." For quantization, even cutting down to 1-bit keeps core capabilities nearly on par with 2-bit, but the edges break first: breaking the tool_choice contract, losing language lock, and missing objects in images. Whether to accept that for 15GB of savings depends on use case, but for personal daily use, I'd go with 2-bit.

I got a feel for MTP with the draft PR, so I'll measure again with aligned conditions after the mainline merge. I also want to try in a follow-up how the multi-tool selection weakness plays out when connected to actual clients like opencode.

Reference Links

• GLM-5.3-Flash - Hugging Face
• GLM-5.3-Flash: How to Run Locally - Unsloth Documentation
• unsloth/GLM-5.3-Flash-GGUF - Hugging Face
• GLM-5.3-Flash - Z.AI Developer Documentation
• GLM-5: from Vibe Coding to Agentic Engineering - arXiv