developersIO
I organized the NVIDIA NeMo Framework

I organized the NVIDIA NeMo Framework

The NVIDIA NeMo Framework ecosystem is an area where similarly named products keep increasing — Nemotron, NeMo, NIM, Cosmos, NemoClaw — making it easy to get confused about what each one actually is. In this article, we have organized the overall picture of NeMo as of April 2026 and compiled an index of series articles verified on actual hardware using DGX Spark. We hope this can serve as a starting point for those who want to begin following NeMo.
NVIDIA生成AILLM
2026.04.21

This page has been translated by machine translation. View original

Introduction

Hello, I'm Morishige from Classmethod's Manufacturing Business Technology Department.

In NVIDIA's generative AI stack, products with "NeMo" in their name keep multiplying, and I've been noticing more situations where people wonder "What's the difference between Curator, Automodel, and Megatron-Bridge again?" or "Is NIM part of NeMo, or is it something separate?"

In this article, I've organized a map of the entire NVIDIA NeMo Framework ecosystem as of April 2026, also serving as an index of DGX Spark series articles where I tried out each component hands-on. I hope this can serve as an entry point for those who think "I haven't gotten a grasp of NeMo as a whole, but I want to start following it now."

Since this is a fast-moving space, this article is a snapshot as of 2026-04-20. Because release cycles turn monthly, some details may change in a few months. Please keep that in mind.

Brand Clarification — Nemotron, NeMo, NIM, Cosmos, and NemoClaw

First, let's sort out the brand names that tend to cause the most confusion.

NVIDIA's AI-related brands have many similar names, and the fact that "Nemotron," "NeMo," "NIM," "Cosmos," and "NemoClaw" are all separate things is a pretty significant source of confusion. Roughly speaking, keeping the following in mind should help things click:

  • Nemotron is a model. It refers to the LLM/VLM family (Nemotron 3 Nano / Super, etc.) that NVIDIA publishes as open weights
  • NeMo is a framework. It's an umbrella for the generative AI development software stack covering training, customization, evaluation, guardrails, and agent building
  • NIM is an inference microservice. It's a package that distributes trained and optimized models as API endpoints, and is part of NVIDIA AI Enterprise rather than NeMo
  • Cosmos is a foundational model group for Physical AI. It centers on World Foundation Models (Cosmos-Predict / Cosmos-Transfer) and a VLM (Cosmos-Reason), and is an independent project with a separate GitHub organization from NeMo
  • NemoClaw is an agent execution environment. It's a safe agent infrastructure combining a sandbox (OpenShell) and a policy engine with NeMo Guardrails

To put the relationships in order: you customize a Nemotron model with the NeMo Framework, serve it in production via NIM microservices, and if you're running it as an agent, you use NeMo Agent Toolkit or NemoClaw — that's the intended combination. Cosmos runs alongside as a world model for Physical AI.

The Boundary Between NVIDIA AI Enterprise and NeMo

Going one step further, understanding the license boundary will help you avoid confusion in practical work.

The NeMo Framework itself is published as OSS on GitHub and can be freely used by individuals and enterprises alike. This includes Megatron-Bridge, Automodel, RL, Curator, Guardrails, Agent Toolkit, Evaluator, and others. Even if you've personally bought a DGX Spark and have it at home, these are areas you can freely explore.

On the other hand, NIM and NeMo Microservices (Customizer / Evaluator Microservice / Retriever / Guardrails Service, etc.) are commercial products under the NVIDIA AI Enterprise umbrella. Deploying NIM Operator (for Kubernetes) or Customizer Microservice in production requires an AI Enterprise subscription.

This article acknowledges this boundary and, in principle, covers the OSS versions. Topics that assume commercial use will be noted as "AI Enterprise required," so please read accordingly.

Overview of NeMo Framework

Let's take another look at what components are under the NeMo umbrella, starting with a bird's-eye view in a single diagram.

It can be broadly organized into 5 layers: Training & Customization Infrastructure (Megatron-Bridge / Automodel / RL / Run), Data & Evaluation (Curator / Evaluator), Safety & Control (Guardrails), Agents (Agent Toolkit / NemoClaw), and Inference & Serving (NIM / Microservices, under AI Enterprise).

The monolithic NVIDIA/NeMo repository from the NeMo 1.0 era has now been slimmed down to focus on Speech AI, with LLM/VLM responsibilities distributed across independent repositories like Megatron-Bridge and Automodel.

Part of why "NeMo" tends to cause confusion is that we're in a transitional period of this modular decomposition. When information from a year ago conflicts with current information, that's not because NeMo is broken — it's just that kind of turning point.

Component List by Category

From here, I'll introduce the key components in each category in 2-3 paragraphs each. Articles with more detail are linked inline within the paragraphs, so feel free to follow whichever ones interest you.

Training & Customization Infrastructure

Megatron-Bridge adds bidirectional conversion with Hugging Face on top of the Megatron-Core-based training library. It allows you to leverage parallelization such as Tensor Parallelism and Pipeline Parallelism while moving weights back and forth with models on HF Hub. Its primary domain is serious training of large-scale models including Mamba layers and MoE. I ran experiments in an article applying Megatron-Bridge to Nemotron 9B on DGX Spark, though because the container is x86_64 only, I rented a Brev H100 to run it.

Automodel is an SPMD distributed training library native to PyTorch DTensor. It can accept HF models out of the box and use FSDP/DDP with simple configuration. While it doesn't offer the deep customization of Megatron-Bridge, its training configurations are compact and easy to read. Since entering 2026, support for Nemotron 3 Super 120B and Mistral Small 4 119B has been added, making it an actively growing repository.

NeMo RL is the successor to the former NeMo Aligner. Aligner was archived on 2025-11-19 and its functionality was consolidated into NeMo RL. The focus is on Ray-based post-training (RLHF / DPO / GRPO, etc.), and since v0.4 it has adopted Megatron-Bridge as its backend.

NeMo Run is a tool for configuring, launching, and managing ML experiments. Rather than being a standalone training CLI, it's used as a command center for running experiments with Megatron-Bridge and RL.

Data & Evaluation

NeMo Curator is a library dedicated to preprocessing and curating pre-training data. It has expanded its coverage from text to include image and video curation, and can run deduplication and quality filtering at high speed using RAPIDS. Containers are distributed on NGC, but ARM64 support is not officially stated, so trying it on DGX Spark would require verification.

NeMo Evaluator is an LLM evaluation infrastructure. It can call up 23 harnesses and 421 tasks — including lm-evaluation-harness / bigcode-eval / simple-evals — from a single YAML, and supports BYOB (Bring Your Own Benchmark) to integrate custom evaluations. ARM64 support has been confirmed for 4 types — lm-eval / bigcode / simple-evals / garak — with only vlmevalkit being amd64-only. The detailed article is summarized here.

Safety & Control

NeMo Guardrails is an OSS that wraps rules around LLM inputs and outputs. It's designed with 5 layers: Input Rails (prompt filtering), Output Rails (response filtering), Dialog Rails (conversation flow control), Retrieval Rails (RAG filtering), and Execution Rails (tool call control), written in a dedicated language called Colang or Python custom actions.

In the implementation article for Nemotron 3 Nano with safety equipment for Japanese LLMs, I also cover issues like the fixed English keyword problem with is_content_safe and quirks of Colang 2.x, which may be helpful as pitfalls to watch out for when you start working with it.

Agents

NeMo Agent Toolkit (formerly AIQ Toolkit, nvidia-nat on PyPI) is an agent development SDK that provides plugin integrations with major frameworks like LangChain / LangGraph / LlamaIndex / CrewAI and lets you define workflows in YAML. It comes with operations-aware CLIs such as nat mcp serve for publishing agents as MCP servers and nat eval for running evaluations.

NemoClaw is a safe agent execution environment that combines the OpenShell sandbox runtime with NeMo Guardrails. It's used as a sandbox that runs AI agents locally while imposing constraints on network access and destructive operations. NVIDIA officially announced it at GTC 2026, and it's also featured in the DGX Spark Playbook.

Inference & Serving (Under AI Enterprise)

NIM is a package that distributes models as optimized inference microservices. Launching it via Docker / Kubernetes brings up an OpenAI-compatible API. NeMo Microservices is an enterprise-oriented service group used in combination with NIM, and includes Customizer (fine-tuning including LoRA), Evaluator Microservice, Retriever, Guardrails Service, Data Designer, Safe Synthesizer, and more.

In NIM Operator 2.0, a "train LoRA with Customizer → dynamically load into NIM" Data Flywheel is proposed as an official pattern, and that's essentially where AI Enterprise's value proposition lies.

DGX Spark / ARM64 Support Status Matrix

When it comes to trying things on DGX Spark (GB10, ARM64), support status varies by component, so here's a summary based on actual measurements.

Component ARM64 Support How Confirmed Notes
Megatron-Bridge container ❌ Not supported Confirmed x86_64 only in Nemotron 9B × Megatron-Bridge article Offloading to Brev H100 is the practical solution
Automodel △ Needs verification Worked around with direct TRL+PEFT configuration for Gemma 4 VLM FT; ARM64 verification of Automodel itself not performed Theoretically should work since it's PyTorch native
NeMo RL △ Needs verification linux/arm64 image available on NGC but not verified on actual DGX Spark hardware v0.5.0+
NeMo Curator △ Needs verification No explicit ARM64 mention in NGC container; actual RAPIDS dependency needs checking Left as a candidate article topic
NeMo Evaluator ✅ Supported ARM64 manifest confirmed for major harnesses in detailed article Only vlmevalkit is amd64-only
NeMo Guardrails ✅ Supported Confirmed working on actual hardware in Nemotron safety equipment article Pure Python, no ARM64 issues
NeMo Agent Toolkit ✅ Supported Confirmed working on DGX Spark with uv pip install nvidia-nat in Agent Toolkit local article nemotron_h series requires --trust-remote-code
NemoClaw (OpenShell) ✅ Supported Confirmed working via DGX Spark Playbook in hands-on article Early Preview; cgroup v2 setup required via nemoclaw setup-spark
Cosmos-Reason2 ✅ Supported Both 2B/8B models confirmed in structured reasoning article Custom wheel from repository required for vLLM
Cosmos-Predict 2.5 △ Partial support 2B works, 14B OOM, Multiview requires 8 GPUs and not possible Updates after W1 article being verified in W2 draft
NIM △ Model-dependent ARM64 manifest availability varies by model Requires AI Enterprise; check individually

As a rough guideline: OSS Python packages (Guardrails / Agent Toolkit / Evaluator) should work fine on ARM64 without issue, while container-based components (Megatron-Bridge / NIM) require checking the manifest beforehand.

Here I'll arrange by series the articles where I ran hands-on verification of each NeMo component, primarily using DGX Spark. I hope this can serve as an entry point for when you want to "learn about this component" or "re-read the topic touched on in that article."

Nemotron Model Series (7 articles)

A series following training, deployment, RAG, and RLHF around NVIDIA's Japanese-enhanced LLM "Nemotron 9B-v2-Japanese" and the large-scale MoE "Nemotron 3 Super."

Article Content Published
Trying Nemotron 9B-v2-Japanese in Various Cases Basic operations, CLI, OpenAI-compatible inference 2026-02-21
Deploying Nemotron 9B to SageMaker Tokyo Region with VPC Closed Network Configuration Cloud deployment, security design 2026-02-21
Improving RAG Accuracy with NTA FAQ × RAFT RAFT (Retrieval-Augmented FT) 2026-02-22
Japanese Fine-tuning of Nemotron 3 Nano on DGX Spark LoRA FT, Japanese data 2026-02-16
Teaching Classmethod's Culture (CLP) to Nemotron 9B Constitutional AI + SimPO 2026-02-23
Training Full-Layer LoRA Including Mamba-2 with Megatron-Bridge on H100 Megatron-Bridge v0.2.0, Brev H100 2026-02-25
Running Nemotron 3 Super on DGX Spark 120B Mamba-Transformer-MoE 2026-03-12

Evaluation, Safety & Agents (4 articles)

A group of articles covering one each of NeMo's "evaluation, safety, and agent" domains. The agent domain has 2 articles.

Cosmos Series (3 articles)

A series of hands-on verification on DGX Spark of Cosmos Predict 2.5 and Cosmos Reason2, the Physical AI model group.

GTC 2026 Report

Upcoming Plans

Beyond the already-published articles, I'm also preparing a follow-up on NeMo Guardrails targeting Cosmos-Reason2, and an evaluation edition of Agent Toolkit using nat eval. In areas not yet covered, I'm also planning to write standalone articles on NeMo Curator, NeMo RL, and Automodel, as well as NeMo Retriever, NIM-based fine-tuning and service deployment, VLM safety evaluation combining ShieldGemma 2 and Llama Guard 4, and a deeper dive into Cosmos Predict 2.5.

External Resources for Learning NeMo

Finally, here are some pointers to official documentation and primary sources.

Summary

NeMo Framework might seem like a "confusing mess of brands" at first glance, but if you view it as a flow from Models (Nemotron / Cosmos) → Training (Megatron-Bridge / Automodel / RL) → Data & Evaluation (Curator / Evaluator) → Safety (Guardrails) → Agents (Agent Toolkit / NemoClaw) → Inference (NIM / Microservices), the positioning of each component isn't all that complex.

If you're starting out on DGX Spark, here are 3 recommended starting patterns:

  1. I want to run an LLM — Start by running Nemotron 3 Nano or Cosmos-Reason2 with Ollama / vLLM (start with the Nemotron Japanese FT article)
  2. I want to build an agent — Build a local agent with NeMo Agent Toolkit + Nemotron 3 Nano (start with the Agent Toolkit local article)
  3. I want to operate safely — Apply Input/Output rails with NeMo Guardrails (start with the Guardrails introductory article)

While NeMo's brand and component names change frequently, the underlying philosophy — "separate models, frameworks, and inference services, and combine each layer flexibly" — is actually quite straightforward. I hope this article can serve as a map of the NeMo landscape as of April 2026.

AI白書2026 配布中

クラスメソッドが独自に行なったAI診断調査をもとに、企業のAI活用の現在地を調査レポートとしてまとめました。企業規模別の活用度傾向に加え、規模を超えてAI活用を進める企業に共通する取り組みまで、自社の現在地を捉えるためのヒントにぜひ。

AI白書2026

無料でダウンロードする

Share this article

DevelopersIO 2026