After 2 months of using DGX Spark, here's what I found about "jobs it's suited for" and "jobs it's not suited for"
This page has been translated by machine translation. View original
Introduction
Hello, I'm Morishige from Classmethod's Manufacturing Business Technology Department.
It's been about 2 months since I started using the DGX Spark. During this time, I've run every workload I could think of: LLM inference, fine-tuning, video AI, robotics, and image generation. Before I knew it, I had published over 20 articles...
After trying everything out, what I felt was: "128GB isn't a silver bullet, but there are definitely situations where nothing else can substitute for it." In this article, I'll organize what the DGX Spark is good at and what it struggles with, based on actual measurement data and my failures. I hope this is helpful for those considering a purchase, or those who already have one but are still figuring out how to use it.
The DGX Spark in One Sentence
The defining feature of the DGX Spark is its 128GB of unified memory (UMA: Unified Memory Architecture). Since the CPU and GPU share memory, models that physically can't fit in the 32GB VRAM of an RTX 5090 can run as-is.
Here's a comparison with commonly referenced configurations.
| Item | DGX Spark | Mac Studio M3 Ultra | RTX 4090 | RTX 5090 |
|---|---|---|---|---|
| Price | $4,699 | From $3,999 (96GB) | $1,599 (MSRP) | $1,999 (MSRP) |
| Memory | 128GB LPDDR5x | Up to 512GB | 24GB GDDR6X | 32GB GDDR7 |
| Memory Bandwidth | 273 GB/s | 819 GB/s | 1,008 GB/s | 1,792 GB/s |
| Power Consumption | 170W (TDP) | ~200W | ~450W | ~575W |
| OS | Ubuntu 24.04 ARM | macOS | Windows/Linux | Windows/Linux |
What stands out in this table is the difference in memory bandwidth. The figure of 273 GB/s is about one-third of the Mac Studio (819 GB/s), about one-quarter of the RTX 4090 (1,008 GB/s), and there's roughly a 6.5x gap compared to the RTX 5090 (1,792 GB/s). On the other hand, even the RTX 5090 only has 32GB of VRAM, so the constraint of not being able to fit large models doesn't change. This bandwidth difference directly ties into the "strengths and weaknesses" discussed later.
Simply put, the DGX Spark's strength is not "speed" but "capacity." The greatest value is being able to load an entire large model into the 128GB memory space and run everything locally. At conferences, there was a metaphor comparing AI computing environments to F1 cars vs. SUVs — using that analogy, the DGX Spark would be on the SUV side.
What It's Good At
Here are 5 use cases where I felt "this is uniquely DGX Spark" after 2 months of use.
Inference with Large Models Loaded Entirely in Memory
Models in the 70B–120B class that can't run on the 32GB of an RTX 5090 run on a single DGX Spark.
When I ran Nemotron 3 Super (120B-A12B) with Ollama, loading the model consumed about 87GB of memory, and it scored 94.4% accuracy on JCommonsenseQA (Japanese commonsense reasoning). The lightweight model in the same family, Nano (30B-A3B), scored 87.0%, so the difference in parameter count is clearly reflected in accuracy.
NVIDIA's gpt-oss:120b (a MoE with 5.1B active parameters) also ran on a single unit, and Qwen3-235B-A22B (MoE, 22B active) just barely fit on one unit as well. "A 100B+ model running on a single desktop" has real impact once you experience it.
Fine-Tuning Exceeding 30GB
Fine-tuning consumes more memory than inference. This is because, in addition to the model itself, gradients and optimizer states need to be retained. For training tasks where memory-saving techniques and offloading would be essential on an RTX 5090's 32GB, the 128GB of unified memory handles them with room to spare.
For example, during Dreambooth LoRA fine-tuning of FLUX.1 (12B), GPU memory usage reached 71GB and system memory reached 96GB during training. The LoRA Rank could also be set to 256 instead of the usual 8–64, allowing for higher quality. Training time was about 3.4 hours for 500 steps.
In the robotics field as well, fine-tuning of GR00T N1.6 completed in 5 hours and 47 minutes with memory consumption of 90.8GB / 128GB. It used nearly all of the 128GB, but didn't hit OOM.
RAFT LoRA for Nemotron 9B-v2-Japanese, a Japanese character LoRA for Qwen3.5 4B, and imitation learning with LeRobot ACT all worked without issues. For use cases where you want to complete training tasks locally, the 128GB of unified memory makes a real difference.
Local Video AI Pipeline
A video AI pipeline using NVIDIA VSS (Video Search & Summarization) is one use case that pairs well with the DGX Spark.
With VSS 2.4.x, I ran a video search agent combining LLM + VLM + Embedding + Reranker. GPU memory consumption was about 37GB for VLM, 18GB for LLM, and 6GB for NIM (Embedding + Reranker), totaling about 61GB — a fairly heavy configuration — but video search and summarization with Japanese queries worked.
Lighter configurations are also possible. Event Reviewer uses a two-stage setup of CV (GroundingDINO) + VLM (Cosmos-Reason2-8B), with 8 containers and GPU memory usage of 45GB / 128GB (35%). In a use case detecting cardboard boxes on a conveyor belt and having the VLM assess damage, it ran with GPU utilization of 1–37%, well within comfortable limits.
For environments where video data can't be sent to the cloud, or for always-on monitoring use cases, a locally self-contained DGX Spark configuration seems like a viable option.
Training Infrastructure for Robotics
When I trained imitation learning (ACT: Action Chunking with Transformers) for a robot arm called SO-ARM101 on the DGX Spark, it completed in about 7 hours (100,000 steps) with 18GB of GPU memory consumption.
What was interesting was that inference FPS directly correlated with success rate. On Mac's MPS (15Hz), the success rate was 40%, while on DGX Spark's CUDA (30Hz), it rose to 90%. For robotics use cases where you cycle through "training → inference → physical evaluation" locally, having ample GPU memory and stable CUDA inference speed makes a real difference.
Always-On Development Infrastructure
The DGX Spark has low power consumption (40–45W at idle, 135–140W during inference) and a near-fanless, quiet design. With it running 24/7, it's unobtrusive enough to keep as always-on development infrastructure.
I've found it invaluable as "local AI ready whenever you need it" — building a local LLM code completion environment with Continue.dev + VS Code, keeping Cosmos-Reason2-8B (32GB) running persistently for video analysis, and spinning up a local agent environment with NemoClaw.
Not having to worry about cloud API usage fees and being able to experiment whenever an idea strikes is also a mental relief.
What It's Not Good At
On the other hand, there were moments where I felt "I shouldn't have tried this on the DGX Spark." I'll share these honestly, hoping to save time for others considering the same.
Use Cases Requiring Fast Token Generation
LLM inference is broadly divided into 2 phases: prefill, which processes the prompt all at once, and decode, which generates tokens one by one.
Prefill benefits from parallel processing, so the DGX Spark's computational power shines here. With Nemotron 3 Super, I got 112.4 tok/s for prompt evaluation. However, decode is bottlenecked by memory bandwidth, so the DGX Spark's 273 GB/s limits it to 17.9 tok/s.
An intuitive way to think about it: "good at reading prompts, but slow at writing text." For batch generation of long text or real-time chat where response speed matters, cloud GPUs with HBM will be more comfortable.
Models That Max Out the 128GB
128GB is large, but it has a ceiling. When I attempted the Cosmos Predict 2.5 14B model, the model loaded successfully (about 51GB), but intermediate tensors consumed memory all at once during inference, freezing the entire system. I tried 3 times with the same result each time. The number 14B looks small at first glance, but video generation models have much larger per-frame activations than image generation models, and there are cases where even 128GB isn't enough.
Since inference requires more memory than the model size alone due to intermediate tensors and activations, the point that "model fits ≠ inference works" is worth noting. In practice, for training tasks I'd say about 90GB (70% of 128GB) is the safe zone, and exceeding that raises the risk of OOM.
Gaining Speed with a 2-Node Cluster
The DGX Spark can directly connect two units via ConnectX-7 (200Gbps). It's true that you can expand the memory space to 256GB and run models that don't fit on one unit.
However, speed improvements were limited. Measuring with Qwen3-235B-A22B, the 2-node configuration actually dropped slightly to 14.57 tok/s from the single-node 15.51 tok/s (due to RPC overhead). With the dense 123B model Devstral 2, decode was 2.64 tok/s in 2-node configuration, which is hard to call practical.
The value of a 2-node cluster lies not in "going faster" but in "being able to run larger models." It's useful if you want to run models like Qwen3-Coder-480B (168GB) or Llama 4 Maverick (143GB) that don't fit on one unit, but I wouldn't recommend buying a second unit expecting speed improvements.
Note that an NVIDIA official blog from March 2026 introduces cluster configurations of up to 4 units, with a scenario of running models like DeepSeek-R1 671B in a 512GB memory space. I haven't tried a 4-unit configuration myself yet, but if you're interested, check out the blog below.
Local Replacement for Claude Code
I had hoped that with the DGX Spark's 128GB I could replace Claude Code with a local LLM, but the conclusion was that it's currently difficult.
Even with the gpt-oss 120B model, tool call accuracy was inconsistent and the Edit tool success rate varied. This is more of a model-side issue than a hardware problem — even at 120B, there seems to be a ceiling on accuracy for agentic tasks. The situation may change as local LLMs evolve, but for now, relying on cloud Claude is the practical choice.
Understanding Prefill vs. Decode
Throughout the discussion of "strengths and weaknesses," the underlying issue is memory bandwidth. Let me dig in a little deeper.
I mentioned earlier that LLM inference is divided into 2 phases.
Prefill is the phase that processes the entire prompt at once. Matrix multiplication parallelism is the main operation, so GPU computational power (FLOPS) is the bottleneck. Decode, on the other hand, is the phase that generates tokens one by one, requiring all parameters to be read from memory at every step, making memory bandwidth the bottleneck.
The DGX Spark's GB10 has sufficient computational power (estimated FP16 ~62 TFLOPS), but its memory bandwidth is modest at 273 GB/s. This is why it shows an asymmetric performance characteristic: prefill is fast (112.4 tok/s on Super) but decode is slow (17.9 tok/s on the same model).
Understanding this characteristic helps clarify the DGX Spark's use cases.
- Use cases where prefill matters (reading large inputs and returning short outputs) → DGX Spark excels
- RAG search query processing, VLM analysis of video, code completion suggestion generation
- Use cases where decode matters (sequentially generating long text) → Cloud GPUs are more comfortable
- Full blog post generation, bulk translation, chatbot streaming responses
Use Case Decision Flowchart
I put together a quick decision flow for figuring out whether the DGX Spark fits your use case.
It's just a rough guide, but I hope it provides a starting point for your decision.
What I've Felt After 2 Months
Finally, a few thoughts on the user experience that don't easily show up in specs or measurement data.
The quietness exceeds expectations. At idle it's 40–45W, and the GPU quickly returns to idle once inference is done, so having it on constantly doesn't bother me. I keep it next to my desk as development infrastructure and sometimes forget it's there.
You should prepare yourself for ARM64 pitfalls. I encountered several cases where container images or Python packages built with x86 assumptions wouldn't run. There were instances where NVIDIA's NIM (Nemotron-Nano-9B-v2) ARM64 image contained x86 binaries inside, and cases where vLLM's CUTLASS kernel didn't support sm_121 and wouldn't run. You can usually find a workaround, but it's best not to expect "everything works smoothly."
The Playbook ecosystem is maturing. Using the officially provided NVIDIA Playbooks, you can build TRT-LLM inference settings and FLUX.1 training environments by following the steps. Even for those who aren't sure where to start, using Playbooks as a starting point makes it easier to get hands-on.
Summary
Over 2 months, I've run a wide variety of workloads on the DGX Spark. What became clear is that the essence of this machine lies not in "speed" but in "fitting into the 128GB unified memory."
Inference of 120B models and fine-tuning exceeding 70GB — both physically impossible on a single RTX 5090 — run on a $4,699 desktop. On the other hand, token generation speed is limited by memory bandwidth, so it can't match cloud GPUs. When you need to break through the "will it fit or won't it" wall, the DGX Spark has become an irreplaceable option.
Much of the data introduced in this article is covered in detail in individual verification articles. If there's an area you're interested in, follow the links below.
Related Articles
| Category | Article |
|---|---|
| Unboxing & Setup | NVIDIA DGX Spark Has Arrived |
| LLM Inference | Running Nemotron 3 Super on DGX Spark |
| Clustering | Connecting Two DGX Sparks for Distributed Inference of Large Models |
| Image Generation | Fine-Tuning FLUX.1 with Dreambooth LoRA on DGX Spark |
| World Models | Running NVIDIA Cosmos World Foundation Model on DGX Spark |
| Video AI | Running a Video Search AI Agent on DGX Spark |
| Video AI | Real-Time Video Monitoring with VSS Event Reviewer on DGX Spark |
| Robotics | Imitation Learning for SO-ARM101 with LeRobot ACT |
| Dev Environment | Building a Local LLM Code Completion Environment with Continue.dev on DGX Spark |
| Dev Environment | Another Attempt at Running Claude Code + Ollama Locally on DGX Spark |
| Agents | Running NVIDIA NemoClaw on DGX Spark |
