After 2 Months of Using DGX Spark: "Jobs It's Good At" and "Jobs It's Not Good At"
This page has been translated by machine translation. View original
Introduction
Hello, I'm Morishige from Classmethod's Manufacturing Business Technology Department.
It's been about two months since I started using the DGX Spark. During this time, I've run just about every workload I could think of: LLM inference, fine-tuning, video AI, robotics, and image generation. Before I knew it, I had published over 20 articles...
After trying everything out, my takeaway is: "128GB isn't a silver bullet, but there are definitely situations where nothing else can substitute for it." In this article, I'll organize what the DGX Spark is good at and what it's not, based on measured data and lessons learned from failures. I hope this is useful for those considering a purchase, or those who already own one but are still figuring out how to use it.
The DGX Spark in a Nutshell
The defining feature of the DGX Spark is its 128GB unified memory (UMA: Unified Memory Architecture). Because the CPU and GPU share memory, models that physically can't fit in the 32GB VRAM of an RTX 5090 can run as-is.
Here's a comparison with commonly referenced configurations:
| Item | DGX Spark | Mac Studio M3 Ultra | RTX 4090 | RTX 5090 |
|---|---|---|---|---|
| Price | $4,699 | From $3,999 (96GB) | $1,599 (MSRP) | $1,999 (MSRP) |
| Memory | 128GB LPDDR5x | Up to 512GB | 24GB GDDR6X | 32GB GDDR7 |
| Memory Bandwidth | 273 GB/s | 819 GB/s | 1,008 GB/s | 1,792 GB/s |
| Power Consumption | 170W (TDP) | ~200W | ~450W | ~575W |
| OS | Ubuntu 24.04 ARM | macOS | Windows/Linux | Windows/Linux |
What stands out in this table is the difference in memory bandwidth. The 273 GB/s figure is about one-third of the Mac Studio (819 GB/s), about one-quarter of the RTX 4090 (1,008 GB/s), and roughly 6.5 times lower than the RTX 5090 (1,792 GB/s). On the other hand, even the RTX 5090 is limited to 32GB of VRAM, so the constraint of not being able to fit large models remains unchanged. This bandwidth gap directly ties into the "strengths and weaknesses" I'll discuss later.
To put it simply, the DGX Spark's strength isn't "speed" — it's "capacity." The greatest value is being able to load an entire large model into 128GB of memory and run it completely locally. At conferences, there's been a metaphor comparing AI computing environments to F1 cars vs. SUVs, and by that analogy, the DGX Spark would sit firmly on the SUV side.
Where It Excels
Here are 5 use cases where I felt "this is what the DGX Spark is uniquely suited for" after two months of use.
Inference with Large Models Loaded Entirely in Memory
Models in the 70B–120B class that can't run on an RTX 5090's 32GB can run on a single DGX Spark.
When I ran Nemotron 3 Super (120B-A12B) with Ollama, it consumed about 87GB of memory during model loading and achieved a 94.4% accuracy on JCommonsenseQA (Japanese commonsense reasoning). The lightweight model from the same family, Nano (30B-A3B), scored 87.0%, showing that the difference in parameter count is clearly reflected in accuracy.
NVIDIA's gpt-oss:120b (a MoE model with 5.1B active parameters) also ran on a single unit, and Qwen3-235B-A22B (MoE, 22B active) just barely fit on one unit as well. "A desktop machine running a 100B+ model" — once you experience it, the impact is real.
Fine-Tuning Beyond 30GB
Fine-tuning is more memory-intensive than inference. In addition to the model itself, you need to hold gradients and optimizer states. Training tasks that would require offloading or memory-saving techniques on the RTX 5090's 32GB can be run with headroom on 128GB of unified memory.
For example, Dreambooth LoRA fine-tuning of FLUX.1 (12B) consumed 71GB of GPU memory and 96GB of system memory during training. I was able to set LoRA Rank to 256 instead of the usual 8–64, which also allowed me to push quality further. Training time was about 3.4 hours for 500 steps.
In the robotics domain, fine-tuning GR00T N1.6 completed in 5 hours and 47 minutes with a memory consumption of 90.8GB / 128GB. It was using nearly all of the 128GB, but it never hit OOM.
RAFT LoRA for Nemotron 9B-v2-Japanese, a Japanese character LoRA for Qwen3.5 4B, and imitation learning with LeRobot ACT all ran without issues. The 128GB unified memory really shines when you want to run training tasks entirely locally.
Local Video AI Pipeline
A video AI pipeline using NVIDIA VSS (Video Search & Summarization) is one of the use cases that pairs well with the DGX Spark.
With VSS 2.4.x, I ran a video search agent combining LLM + VLM + Embedding + Reranker. GPU memory consumption was about 37GB for the VLM, 18GB for the LLM, and 6GB for NIM (Embedding + Reranker) — a total of about 61GB, which is a fairly heavy configuration — yet video search and summarization with Japanese queries worked successfully.
Lighter configurations are also possible. Event Reviewer uses a two-stage setup of CV (GroundingDINO) + VLM (Cosmos-Reason2-8B), with 8 containers and GPU memory usage of 45GB / 128GB (35%). In a use case detecting cardboard boxes on a conveyor belt and having the VLM judge damage, it ran with a comfortable load of 1–37% GPU utilization.
For environments where video data can't leave the premises, or for always-on monitoring use cases, a fully local DGX Spark configuration seems like a strong candidate.
Training Infrastructure for Robotics
I trained a SO-ARM101 robot arm using imitation learning (ACT: Action Chunking with Transformers) on the DGX Spark, and it completed in about 7 hours (100,000 steps) with 18GB of GPU memory consumption.
What was particularly interesting was how inference FPS directly impacted success rate. With Mac's MPS (15Hz), the success rate was 40%, while with the DGX Spark's CUDA (30Hz), it jumped to 90%. For robotics workflows where you're cycling through "train → infer → real-world evaluation" locally, both the GPU memory headroom and the stable CUDA inference speed make a real difference.
Always-On Development Infrastructure
The DGX Spark has low power consumption (40–45W at idle, 135–140W during inference) and a near-fanless, quiet design. It has a small enough footprint that you can leave it on all the time without it being a bother, making it well-suited to serve as always-on development infrastructure.
I've found it invaluable as "local AI that's ready whenever you need it" — setting up local LLM code completion with Continue.dev + VS Code, keeping Cosmos-Reason2-8B (32GB) resident for video analysis, and spinning up a local agent environment with NemoClaw.
Not having to worry about cloud API usage costs and being able to experiment freely whenever an idea strikes is mentally refreshing.
Where It Falls Short
On the other hand, there were also moments where I felt "I shouldn't have tried this on the DGX Spark." I'll share these honestly, hoping to save time for anyone considering the same.
Use Cases Requiring Fast Token Generation
LLM inference is broadly divided into two phases: prefill, where the entire prompt is read in at once, and decode, where tokens are generated one at a time.
Prefill benefits from parallel processing, so the DGX Spark's compute capacity shines here. With Nemotron 3 Super, I got 112.4 tok/s for prompt eval. However, decode is bottlenecked by memory bandwidth, so the DGX Spark's 273 GB/s limits it to 17.9 tok/s.
An intuitive way to put it: "It's good at reading prompts, but slow at writing text." For long-form batch generation or real-time chat where response speed matters, cloud GPUs with HBM will be much more comfortable.
Models That Max Out 128GB
128GB is large, but it's still a ceiling. When I attempted the Cosmos Predict 2.5 14B model, the model loaded successfully (using about 51GB), but during inference, intermediate tensors consumed memory all at once and the system froze completely. I tried three times and got the same result every time. The "14B" label may look small at first glance, but video generation models carry more activation memory per frame than image generation models, and there are cases where even 128GB isn't enough.
During inference, intermediate tensors and activations require more memory than the model size alone, so it's important to remember that "the model fits ≠ inference will work." From my experience, for training tasks, around 90GB (70% of 128GB) feels like the safe zone — going beyond that raises the risk of OOM.
Using a Two-Unit Cluster for Speed
The DGX Spark can connect two units directly via ConnectX-7 (200Gbps). It's true that you can expand the memory space to 256GB and run models that don't fit on a single unit.
However, the speed improvement was limited. Measuring with Qwen3-235B-A22B, the single-unit configuration achieved 15.51 tok/s, while the two-unit configuration actually slowed down slightly to 14.57 tok/s (RPC overhead). With the dense 123B model Devstral 2, the two-unit configuration achieved a decode speed of 2.64 tok/s — hard to call practical.
The value of a two-unit cluster lies in "being able to run larger models," not in "getting faster." It's useful if you want to run models that don't fit on a single unit, such as Qwen3-Coder-480B (168GB) or Llama 4 Maverick (143GB), but I wouldn't recommend buying a second unit expecting a speed boost.
Note that an official NVIDIA blog post from March 2026 introduced cluster configurations of up to four units, describing scenarios where models like DeepSeek-R1 671B could be run in a 512GB memory space. I haven't tried a four-unit configuration myself yet, but if you're interested, check out the blog below.
Local Replacement for Claude Code
I was hopeful that with 128GB on the DGX Spark, I could replace Claude Code with a local LLM — but the conclusion was that it's difficult at this point.
Even with the gpt-oss 120B model, tool call accuracy was inconsistent and Edit tool success rates varied. This seems to be more of a model-side limitation than a hardware issue — even at 120B, there's a ceiling when it comes to agentic task accuracy. The situation may change as local LLMs evolve, but for now, relying on cloud-based Claude is the practical choice.
Understanding Prefill vs. Decode
Throughout this discussion of strengths and weaknesses, the underlying issue is memory bandwidth. Let me dig into this a little.
I mentioned earlier that LLM inference is split into two phases.
Prefill processes the entire prompt in one batch. Since matrix multiplication and parallel computation are the main operations, GPU compute (FLOPS) is the bottleneck. Decode, on the other hand, generates tokens one at a time, requiring all parameters to be read from memory every step — so memory bandwidth is the bottleneck.
The DGX Spark's GB10 has sufficient compute (estimated ~62 TFLOPS FP16), but its memory bandwidth is a modest 273 GB/s. This results in an asymmetric performance profile: fast prefill (112.4 tok/s with Super) but slow decode (17.9 tok/s with the same model).
Understanding this characteristic helps clarify where the DGX Spark fits.
- Use cases where prefill matters (reading large inputs and returning short outputs) → DGX Spark excels
- RAG search query processing, VLM video analysis, code completion suggestion generation
- Use cases where decode matters (sequentially generating long text) → Cloud GPUs are more comfortable
- Full blog post generation, bulk translation, chatbot streaming responses
Use Case Decision Flowchart
I put together a simple decision flow to help determine whether the DGX Spark fits your use case.
This is just a rough guide, but I hope it's a useful starting point for making decisions.
Reflections After Two Months
Finally, a few thoughts on the experience that don't easily show up in specs or measured data.
The quietness exceeded expectations. At 40–45W idle, and with the GPU quickly returning to idle after inference completes, it doesn't bother me even when left on continuously. I keep it next to my desk as development infrastructure, and sometimes I forget it's even there.
Be prepared for ARM64 environment pitfalls. I ran into situations several times where container images or Python packages built for x86 wouldn't run. There was a case where NVIDIA's NIM (Nemotron-Nano-9B-v2) ARM64 image contained x86 binaries, and another where vLLM's CUTLASS kernel didn't support sm_121 and wouldn't run. There's usually a workaround, but don't assume "everything will just work."
The Playbook ecosystem is maturing. Using the official Playbooks provided by NVIDIA, you can set up TRT-LLM inference configurations and FLUX.1 training environments by following step-by-step instructions. Even if you're not sure where to start, using a Playbook as a jumping-off point makes it much easier to get hands-on.
Summary
Over two months, I've run a wide variety of workloads on the DGX Spark. What's become clear is that the essence of this machine is not "speed" but "fitting into 128GB unified memory."
Inference with 120B models and fine-tuning exceeding 70GB — physically impossible on a single RTX 5090 — can be done entirely on a $4,699 desktop. At the same time, token generation speed is bottlenecked by memory bandwidth, so it can't match cloud GPUs. In situations where you need to break through the "fits or doesn't fit" barrier, the DGX Spark has become an irreplaceable option.
Much of the data presented in this article is covered in detail in the individual verification articles. If there's a particular area that interests you, feel free to follow the links below.
Related Articles
| Category | Article |
|---|---|
| Unboxing & Setup | NVIDIA DGX Spark Has Arrived |
| LLM Inference | Running Nemotron 3 Super on DGX Spark |
| Clustering | Connecting Two DGX Sparks for Distributed Inference of Large Models |
| Image Generation | Fine-Tuning FLUX.1 with Dreambooth LoRA on DGX Spark |
| World Models | Running NVIDIA Cosmos World Foundation Model on DGX Spark |
| Video AI | Running a Video Search AI Agent on DGX Spark |
| Video AI | Real-Time Video Monitoring with VSS Event Reviewer on DGX Spark |
| Robotics | Training SO-ARM101 with LeRobot ACT Imitation Learning |
| Dev Environment | Setting Up Local LLM Code Completion with Continue.dev on DGX Spark |
| Dev Environment | Another Attempt at Running Claude Code + Ollama Locally on DGX Spark |
| Agents | Running NVIDIA NemoClaw on DGX Spark |
