I tried running NeMo AutoModel, which can train Hugging Face models as-is, on DGX Spark

Introduction

Hello, I'm Morishige from Classmethod's Manufacturing Business Technology Department.

NVIDIA's training library NeMo AutoModel had, before I knew it, become normally usable on DGX Spark. When I previously wrote an article surveying the NeMo Framework, I had left AutoModel's ARM64 support as "needs verification," but upstream recipes for DGX Spark are now in place, and CI is even running on actual GB10 hardware.

https://github.com/NVIDIA-NeMo/Automodel

Until now, I hadn't used AutoModel for fine-tuning on DGX Spark. For Japanese training of Nemotron 3 Nano, I used Hugging Face's PEFT and TRL directly, and when I set up Nemotron 3.5 Lightning as a judge for an LLM router, the cookbook assumed 8x H100s, so I ran Megatron-Bridge in the cloud. The motivation for this article is to verify whether it has become something that can be used straightforwardly on a single DGX Spark.

https://dev.classmethod.jp/articles/dgx-spark-nvidia-nemo-framework-overview-and-index/

I'll update the AutoModel row that I had marked △ in the survey article above (which is an article as of 2026-04-21) with this article.

This article introduces an overview of NeMo AutoModel and the current state of DGX Spark support, as well as the initial results from running Qwen3-8B LoRA and full SFT in the official container 26.08. I hope it resonates with people who have a single DGX Spark and want to train Hugging Face models locally.

NeMo AutoModel is a mechanism for training Hugging Face models as-is

NeMo AutoModel is a PyTorch-native training library published by NVIDIA. Borrowing the official description, it is "a PyTorch DTensor-native SPMD training library that supports Hugging Face models from day-0." It accepts models from Hugging Face Hub as-is via from_pretrained and handles SFT, PEFT such as LoRA and QLoRA, knowledge distillation, and VLM training all within the same framework. Training results also come out as Hugging Face-format safetensors, so they can be passed directly to inference engines like vLLM without any conversion step.

Since tools with "NeMo" in their name keep multiplying, let me first organize the positioning. The LLM-facing parts of NeMo Framework are split into separate repositories by role.

The training layer has two options—AutoModel and Megatron-Bridge—and this is the most confusing part. Megatron-Bridge is Megatron-Core based, converts checkpoints to Megatron format, and is suited for large-scale pre-training and deep customization. AutoModel handles Hugging Face-compatible checkpoints as-is and lets you write the same recipe format from single-GPU LoRA to multi-node training. My understanding is: if you want to start from weights on Hugging Face and end in Hugging Face format, use AutoModel; if you want to organize training in the Megatron world, use Megatron-Bridge.

The skeleton of usage is simple: write a single YAML recipe specifying "which model, with which data, with which parallelization," and launch with automodel <recipe.yaml> --nproc-per-node N. The model is specified via nemo_automodel.NeMoAutoModelForCausalLM.from_pretrained, a Hugging Face-compatible class, and the recipe contents are readable in ordinary PyTorch and Hugging Face vocabulary. Distributed training is built on FSDP2 and DTensor, designed to layer tensor parallel, context parallel, and expert parallel according to model size and GPU count. Checkpoints support both DCP and safetensors, and FP8 training and speculative decoding drafter training are also among the recipe options. Hugging Face also has documentation on fine-tuning using AutoModel, and NVIDIA has published an article showing 3.4–3.7x throughput improvement over Transformers v5 for MoE models.

Reasons to choose AutoModel

If all you want is to LoRA a Hugging Face model, there's the path of using PEFT and TRL directly, or lightweight tools like Unsloth. I also used PEFT and TRL directly when training Nemotron 3 Nano in Japanese. Even so, I find myself choosing AutoModel more often now, for four reasons.

The first is that the entry and exit stay closed in Hugging Face format. The training input is a Hub model ID or a local snapshot, and the output is safetensors or a PEFT-format adapter. In separate verification, I was able to feed the adapter output by AutoModel directly into vLLM and get back a chat completion. Not having to insert format conversion between training and serving makes it easier to increase iteration speed.

The second is that NVIDIA takes care of the hardware. The GB10 in DGX Spark has compute capability 12.1, which is different from data center Blackwell. Keeping up with things like bitsandbytes builds tuned for GB10, or recipe settings that avoid paths cuDNN can't handle, quietly consumes time when you chase them yourself. With AutoModel, GB10 physical machines are in CI, and this kind of support arrives pre-baked into the container and recipes.

The third is that a recipe written for 1 GPU can be taken to a larger environment as-is. LoRA rank and learning rate are YAML values, and parallelization settings are isolated in the distributed block. There's no rewriting needed when you take a recipe you've validated on your local machine to cloud GPUs later.

The fourth is connectivity with other NeMo tools. I'm building a flow where I generate synthetic data with Data Designer, clean it with Curator, train with AutoModel, and score with Evaluator. Each is an independent tool so any can be swapped out individually, but the data formats and evaluation APIs align, so there are fewer seams when assembled as a pipeline.

Diagramming the flow I've built looks like this. AutoModel is the training box in the middle, with both inputs and outputs in Hugging Face format.

On the other hand, training requiring deep customization is Megatron-Bridge territory, and since the container contents update every two months, recipe compatibility needs to be verified per version. The version display quirk I mention later is one example of this. Excessive expectations aren't warranted, but for the use case of simply wanting to train Hugging Face models on DGX Spark, I think this is currently the least labor-intensive path.

DGX Spark support is visible through accumulated PRs, not release notes

The reason I missed the support is simple: there was only one line in the release notes saying "DGX Spark support," in v0.2.0. The actual support accumulated through PRs and recipe YAMLs, and tracing the GitHub history it looks like this:

The turning point is the two entries on August 14, 2026. GB10 physical machines entered CI runners, and Qwen3-8B SFT, GPT-OSS-20B LoRA, and Llama-3.3-70B QLoRA became GB10 physical CI targets. On the same day, a change was merged to natively build bitsandbytes for SM121. These two were cherry-picked into v0.6.0's release branch, v0.6.0 was published on August 26, and the corresponding NGC tag is 26.08.

The DGX Spark-oriented recipes included in v0.6.0 are four: Qwen3-8B SQuAD SFT, GPT-OSS-20B single-GPU LoRA, Llama-3.3-70B QLoRA, and Nemotron 3 Nano 30B-A3B single-GPU LoRA. The last one is written with ep_size 1, and the MoE LoRA that I had previously avoided by judging "the cookbook assumes 8x H100s" officially exists as a single-machine recipe.

There are 3 installation paths but the container is the only choice

There are three paths to run AutoModel on DGX Spark: the NGC container, pip wheel, and the build.nvidia.com playbook. Lining up what I confirmed on my end:

The playbook is published as "Fine-tune with NeMo," and the procedure itself is the same structure as this time—enter the container via docker run and run a recipe. However, the specified tag is still 26.02, a version before GB10 CI and the bitsandbytes SM121 build were added. The pip wheel is py3-none-any and is pure Python inside, so the real issue becomes how to set up Transformer Engine and bitsandbytes for aarch64, not AutoModel itself. I didn't dive into that this time and used only the container.

Personally, I think if you're using DGX Spark, the container is the only choice. The NGC image has a linux/arm64 manifest and is ready with just docker pull, and recipes are also bundled at /opt/Automodel/examples/.

Making a short run of Qwen3-8B by converting the official Spark recipe to LoRA

The Spark-oriented recipe bundled in the container, examples/llm_finetune/qwen/qwen3_8b_squad_spark.yaml, is a full SFT configuration. Since my goal this time was to understand how to use it, I took this recipe as a base, added LoRA, and made a short version that stops at 30 steps. What I changed were: step count, batch size, validation and save intervals, the peft and checkpoint blocks, number of data samples, and model path. The ci block for CI use was removed. The GB10-specific settings for attention, loss function, and sequence packing were left as-is.

step_scheduler:
  global_batch_size: 2 # Official is 64
  local_batch_size: 1 # Official is 2
  ckpt_every_steps: 30
  val_every_steps: 30
  num_epochs: 1
  max_steps: 30

model:
  _target_: nemo_automodel.NeMoAutoModelForCausalLM.from_pretrained
  pretrained_model_name_or_path: /workspace/models/Qwen3-8B # Read pre-fetched weights as read-only mount
  # Keep packed training on the HF FlashAttention path for GB10. TE's packed THD
  # backward requests ragged LSE, which cuDNN does not support on SM12x GPUs.
  force_hf: true
  attn_implementation: flash_attention_2

peft:
  _target_: nemo_automodel.components._peft.lora.PeftConfig
  target_modules: '*_proj'
  dim: 8
  alpha: 32
  use_triton: false

checkpoint:
  enabled: true
  checkpoint_dir: /workspace/run/checkpoints
  model_save_format: safetensors
  save_consolidated: final

dataset:
  _target_: nemo_automodel.components.datasets.llm.squad.make_squad_dataset
  dataset_name: rajpurkar/squad
  split: train
  limit_dataset_samples: 512

The two lines force_hf: true and flash_attention_2 are GB10-specific settings included with comments in the official recipe. Because the cuDNN on GB10 cannot handle the backward used by Transformer Engine's packed attention, it's fixed to use Hugging Face's FlashAttention 2 path. I think the fact that this kind of background is written as comments in the recipe is a byproduct of running CI on physical hardware.

Execution is just launching the container once. I mounted pre-fetched model weights and retrieved only SQuAD from inside the container. The following is extracted from the actual run.sh I used, in a reproducible form, assuming config/ is placed directly under the working directory.

docker run --rm --gpus all --network=host --shm-size=32g \
  --ulimit memlock=-1 --ulimit stack=67108864 \
  --workdir /opt/Automodel \
  -v "$PWD:/workspace/run" \
  -v "$HOME/models/qwen3-8b-b968826d:/workspace/models/Qwen3-8B:ro" \
  -v "$HOME/.cache/huggingface:/root/.cache/huggingface" \
  nvcr.io/nvidia/nemo-automodel:26.08 \
  automodel /workspace/run/config/qwen3-8b-squad-spark-lora.yaml --nproc-per-node 1

The results were as follows. From loading through 30 steps, validation, and checkpoint saving, it took 90 seconds inside the container.

The loss going down is the result of running just 30 steps with SQuAD limited to 512 samples, so please don't read it as a measure of training quality. What I wanted to confirm here were three things: that the official Spark recipe path works as-is with the 26.08 arm64 image, that the LoRA adapter comes out as safetensors, and whereabouts within 128GB the memory usage lands. Under the conditions of this run, the maximum CUDA allocation for Qwen3-8B BF16 LoRA was 20.78 GiB.

Lining up my hands-on results for how much fits in 128GB unified memory

Since just 30 steps isn't very satisfying, I'll also line up my own results from running the previous AutoModel container (26.06.00). The conditions differ, so this isn't an apples-to-apples comparison—treat it purely as a guide for whether things will fit on a single machine.

8B LoRA at 20.78 GiB, 14B at 42.85 GiB, 30B MoE at 63.25 GiB—all fit on a single GB10 under the conditions in the table. On the other hand, 8B full SFT used up to 77.70 GiB with the official recipe settings, and each step took about 72.54 seconds.

I ran this full SFT to completion. 305 steps (1 epoch of SQuAD) finished in 6 hours 11 minutes, with training loss dropping from 7.0378 to 0.0637 and validation loss from 4.4213 to 0.0960. Since validation loss was nearly flat from step 39 onward, the latter half was just running due to recipe settings. Upon completion, 5 safetensors files (~16.4GB total) consolidated in Hugging Face format plus split weights and optimizer state totaling 62GB were written as checkpoints. The consolidated weights are in a form that can be passed directly to vLLM. For 8B BF16 full SFT, a single DGX Spark is in the range of not just "fits" but "finishes if you wait 6 hours."

So how large can full SFT realistically go? Since 8B used 77.70 GiB, applying the same calculation to 14B gives about 136 GiB, which exceeds the 121 GiB visible to Linux. Options like offloading optimizer state to CPU or using 8-bit optimizers exist, but I haven't tried them on my end. The official playbook also presents the DGX Spark combination as "8B for full SFT, 70B for QLoRA," so my current assessment is: BF16 full SFT up to the 8B class, and for larger models, LoRA or QLoRA.

What personally surprised me was Lightning. I had previously judged "BF16 LoRA won't fit in unified memory" and chose 4-bit QLoRA or H100, but building on the ep_size 1 and target module narrowing from the bundled Nemotron 3 Nano recipe, and removing Lightning-specific MTP layers, it ran at 63.25 GiB. I plan to cover this Lightning run in a separate article, so I'll leave just the numbers here.

Things to watch out for in 26.08, noted upfront

That covers the usage, but I noticed a few things while looking inside the container that I'll note upfront for anyone trying the same thing.

First, the version display. The NGC tag is 26.08, which should correspond to the release v0.6.0, but checking nemo_automodel.__version__ inside the container shows 0.5.0+922e83e42. The startup banner also says "NVIDIA Release 26.06." Tracing on GitHub, this commit is on v0.6.0's release branch and was built three commits before the tag. The diff consists of 1 LoRA backward fix and 2 version number bumps, and the GB10 CI and bitsandbytes SM121 build are included. It seems safe to consider the contents equivalent to v0.6.0, but trusting the version string will lead to confusion.

Next, CUDA. The container uses CUDA 13.3, and the kernel driver on my DGX Spark was 580.159.03. At startup "CUDA Forward Compatibility mode ENABLED" appears, and this combination completed 30 steps without issue. This is the mechanism for running a newer container without upgrading the driver, and it wasn't a problem this time.

Log output also has quirks. Training loss, mem, and tps appear as INFO lines for each step, but these go to stderr. Stdout receives recipe contents and progress bars. If you split with 2>, the loss becomes invisible, so it's safest to keep both.

There are two points regarding checkpoints. Since the container runs as root, files under checkpoints/ are written as root-owned. I ran chown from the same image before retrieving them. The other point is a warning at startup: specifying save_consolidated: final produces a warning that "since v4_compatible=False, there may be incompatibility with transformers v4." The container's transformers is version 5, so I proceeded as-is this time, but if you plan to read with older transformers, it's worth looking at checkpoint.v4_compatible.

Note that the single-GPU LoRA recipe for Lightning added in PR #3533 exists in main but is not included in the 26.08 image. You'll need to write it yourself based on the bundled nemotron_nano_v3_singlegpu_lora.yaml.

Summary

NeMo AutoModel has reached a state where it has official recipes for DGX Spark and physical GB10 CI. The NGC 26.08 container has a linux/arm64 image, Qwen3-8B training ran with a YAML that merely added LoRA to the official Spark recipe at 20.78 GiB max CUDA allocation, and full SFT with the same recipe also completed 305 steps at 77.70 GiB. The row I had marked △ in my previous survey article can be advanced to "8B LoRA and full SFT complete with the official container."

On the other hand, what I confirmed this time extends only to 30 steps of LoRA and full SFT completion, plus a rough memory guide for what fits on one machine. I haven't looked at post-training model quality, pip-based installation, or FP8. The fact that the container's version display points to a midpoint in the r0.6.0 branch requires caution when communicating numbers to others.

I've already taken Nemotron 3.5 Lightning single-GPU LoRA through the full cycle from training to serving with vLLM using the previous container. I plan to cover that in a separate article, including training and evaluation using Japanese Tool Calling as a subject.

Reference Links

• NVIDIA-NeMo/Automodel (GitHub)
• NeMo AutoModel Documentation
• NeMo AutoModel Container (NGC)
• Fine-tune with NeMo (DGX Spark playbook) — Updated 2026-03-04, specifies 26.02
• PR #3538 ci: route DGX Spark recipes to GB10 (merged 2026-08-14)
• PR #3553 fix(docker): build bitsandbytes for SM121 (merged 2026-08-14)
• PR #3533 fix(checkpoint): restore single-GPU custom-model DCP loading (merged 2026-08-18, main only)
• Discussion #640 DGX Spark + NeMo: Model Training Power in the Palm of Your Hand
• Accelerating Transformers Fine-Tuning with NVIDIA NeMo AutoModel (Hugging Face Blog)
• A Survey of NVIDIA NeMo Framework (article as of 2026-04-21)
• Setting Up Nemotron 3.5 Lightning as an LLM Router Judge via LoRA Post-Training
• Japanese Fine-Tuning of Nemotron 3 Nano on DGX Spark