developersIO
I organized the NVIDIA NeMo Framework

I organized the NVIDIA NeMo Framework

The NVIDIA NeMo Framework ecosystem is an area where similarly named products such as Nemotron, NeMo, NIM, Cosmos, and NemoClaw keep increasing, making it easy to get confused about what each one does differently. In this article, we have organized an overview of the entire NeMo landscape as of April 2026 and compiled an index of a series of articles verified on actual hardware using DGX Spark. We hope this can serve as an entry point for those who are just starting to follow NeMo.
NVIDIA生成AILLM
2026.04.21

This page has been translated by machine translation. View original

Introduction

Hello, I'm Morishige from Classmethod's Manufacturing Business Technology Department.

In NVIDIA's generative AI stack, the number of products with "NeMo" in their name keeps growing, and I wonder if you've found yourself in situations like "Wait, what's the difference between Curator, Automodel, and Megatron-Bridge again?" or "Is NIM part of NeMo, or is it something separate?"

In this article, I've put together a map of the entire NVIDIA NeMo Framework ecosystem as of April 2026, which also serves as an index of DGX Spark series articles where I tried each component hands-on. I hope it can serve as an entry point for those who haven't fully grasped NeMo yet but want to start catching up now.

Since this is a fast-moving area, this article is a snapshot as of 2026-04-20. Given that release cycles run monthly, details may change a few months from now. Please keep that in mind.

Brand Breakdown — Nemotron, NeMo, NIM, Cosmos, and NemoClaw

First, let's sort out the brand names that tend to cause the most confusion.

NVIDIA's AI-related brands have a lot of similar names, and the fact that "Nemotron," "NeMo," "NIM," "Cosmos," and "NemoClaw" are all separate things is a pretty significant source of confusion. Here's a rough breakdown that should help clear things up.

  • Nemotron is a model. It refers to the family of LLMs and VLMs that NVIDIA publishes as open weights (Nemotron 3 Nano / Super, etc.)
  • NeMo is a framework. It's the umbrella for the software stack covering generative AI development, including training, customization, evaluation, guardrails, and agent building
  • NIM is an inference microservice. It's a package that delivers trained and optimized models as API endpoints, and it's part of NVIDIA AI Enterprise rather than NeMo
  • Cosmos is a collection of foundation models for Physical AI. It centers on World Foundation Models (Cosmos-Predict / Cosmos-Transfer) and a VLM (Cosmos-Reason), and it's an independent project with a separate GitHub organization from NeMo
  • NemoClaw is an agent execution environment. It's a secure agent platform that combines a sandbox (OpenShell) and a policy engine with NeMo Guardrails

To summarize the relationships: you customize Nemotron models with NeMo Framework, deploy them for production with NIM microservices, and if you're running them as agents, you use NeMo Agent Toolkit or NemoClaw. Cosmos runs alongside all of this as a world model for Physical AI.

The Boundary Between NVIDIA AI Enterprise and NeMo

Going one step further, it's worth understanding the license boundaries to avoid confusion in practical use.

The NeMo Framework itself is published as OSS on GitHub and can be freely used by individuals and companies alike. This includes Megatron-Bridge, Automodel, RL, Curator, Guardrails, Agent Toolkit, Evaluator, and more. Even if you've personally bought a DGX Spark and have it at home, these are all areas you can freely explore.

On the other hand, NIM and NeMo Microservices (Customizer / Evaluator Microservice / Retriever / Guardrails Service, etc.) are commercial products under NVIDIA AI Enterprise. If you want to deploy NIM Operator (for Kubernetes) or Customizer Microservice in production, you'll need an AI Enterprise subscription.

This article acknowledges this boundary while primarily covering the OSS versions. Topics that require AI Enterprise will be noted as "AI Enterprise required," so please read with that in mind.

Overview of NeMo Framework

Let's step back and take a bird's-eye view of what components fall under NeMo, starting with a single diagram.

It can be broadly divided into 5 layers: Training & Customization Infrastructure (Megatron-Bridge / Automodel / RL / Run), Data & Evaluation (Curator / Evaluator), Safety & Control (Guardrails), Agents (Agent Toolkit / NemoClaw), and Inference & Deployment (NIM / Microservices, under AI Enterprise).

The monolithic NVIDIA/NeMo repository from the NeMo 1.0 era has been slimmed down to focus on Speech AI, with LLM/VLM responsibilities being distributed across independent repositories like Megatron-Bridge and Automodel.

A big reason people get confused when they hear "NeMo" is that we're in the middle of this modular transition. When information from a year ago conflicts with information today, it's not that NeMo is poorly organized — it's just that we're at that kind of turning point.

Component List by Category

From here, I'll introduce the main components in each category with 2–3 paragraphs each. Articles with more detail are linked inline throughout the paragraphs, so feel free to follow whichever interests you.

Training & Customization Infrastructure

Megatron-Bridge adds bidirectional conversion with Hugging Face on top of the Megatron-Core-based training library. It lets you leverage parallelism like Tensor Parallelism and Pipeline Parallelism while moving weights back and forth with models on HF Hub. Its primary use case is seriously training large-scale models that include Mamba layers or MoE. I experimented with this in an article applying Megatron-Bridge to Nemotron 9B on DGX Spark, but since the container is x86_64-only, I borrowed a Brev H100 to run it.

Automodel is a PyTorch DTensor-native SPMD distributed training library. It accepts HF models out of the box and lets you use FSDP/DDP with simple configuration. While it doesn't offer the same depth of customization as Megatron-Bridge, the training configuration is compact and easy to read. Support for Nemotron 3 Super 120B and Mistral Small 4 119B has been added since the start of 2026, and it's an actively growing repository.

NeMo RL is the successor to the former NeMo Aligner. Aligner was archived on 2025-11-19, and its functionality was consolidated into NeMo RL. The main focus is Ray-based post-training (RLHF / DPO / GRPO, etc.), and from v0.4 onward it has adopted Megatron-Bridge as its backend.

NeMo Run is a tool for configuring, launching, and managing ML experiments. Rather than being a standalone training CLI, it's used as the command center for running experiments with Megatron-Bridge or RL.

Data & Evaluation

NeMo Curator is a library dedicated to preprocessing and curating pre-training data. Its scope has expanded to include image and video curation in addition to text, and it can run deduplication and quality filtering at high speed using RAPIDS. A container is distributed on NGC, but ARM64 support is not officially stated, so if you're trying it on DGX Spark, it's a territory that needs verification.

NeMo Evaluator is an LLM evaluation platform. It lets you invoke lm-evaluation-harness / bigcode-eval / simple-evals and 23 other harnesses covering 421 tasks from a single YAML file, and it supports BYOB (Bring Your Own Benchmark) to integrate custom evaluations. ARM64 support has been confirmed for 4 types — lm-eval / bigcode / simple-evals / garak — while vlmevalkit was amd64-only. The explanatory article is here.

Safety & Control

NeMo Guardrails is an OSS that layers rules on top of LLM inputs and outputs. It's designed with 5 layers: Input Rails (prompt filtering), Output Rails (response filtering), Dialog Rails (conversation flow control), Retrieval Rails (RAG filtering), and Execution Rails (tool call control), described using a dedicated language called Colang or Python custom actions.

In the implementation article adding safety features to a Japanese LLM using Nemotron 3 Nano, I also cover quirks like the fixed English keyword issue in is_content_safe and the peculiarities of Colang 2.x, so it may be useful as a reference for pitfalls if you're starting to work with it.

Agents

NeMo Agent Toolkit (formerly AIQ Toolkit, nvidia-nat on PyPI) is an agent development SDK that provides plugin integrations with major frameworks like LangChain / LangGraph / LlamaIndex / CrewAI and lets you define workflows in YAML. It comes with operations-focused CLIs like nat mcp serve to expose agents as MCP servers and nat eval to run evaluations.

NemoClaw is a secure agent execution environment that combines the OpenShell sandbox runtime with NeMo Guardrails. It's used as a sandbox that lets you run AI agents locally while imposing constraints on network access and destructive operations. NVIDIA officially announced it at GTC 2026, and it's also featured in the DGX Spark Playbook.

Inference & Deployment (Under AI Enterprise)

NIM is a package that delivers models as optimized inference microservices. Launching it with Docker / Kubernetes brings up an OpenAI-compatible API. NeMo Microservices is a suite of enterprise-focused services used in conjunction with NIM, including Customizer (fine-tuning including LoRA), Evaluator Microservice, Retriever, Guardrails Service, Data Designer, Safe Synthesizer, and more.

NIM Operator 2.0 proposes a "train LoRA with Customizer → dynamically load into NIM" Data Flywheel as an official pattern, and I think it's fair to say that's where the value proposition of AI Enterprise lies.

DGX Spark / ARM64 Compatibility Matrix

When working with DGX Spark (GB10, ARM64), compatibility varies by component, so here's a summary based on actual measurements.

Component ARM64 Support How Verified Notes
Megatron-Bridge Container ❌ Not Supported Confirmed x86_64 only in Nemotron 9B × Megatron-Bridge article Offloading to Brev H100 is the practical solution
Automodel △ Needs Verification For Gemma 4 VLM FT, worked around by using TRL+PEFT directly; ARM64 verification of Automodel itself not done Theoretically should work since it's PyTorch-native
NeMo RL △ Needs Verification NGC provides linux/arm64 image but not verified on actual DGX Spark hardware v0.5.0+
NeMo Curator △ Needs Verification No ARM64 indication in NGC container; RAPIDS dependency needs real-world confirmation Kept as a candidate for a future article
NeMo Evaluator ✅ Supported ARM64 manifests for major harnesses confirmed in explanatory article Only vlmevalkit is amd64-only
NeMo Guardrails ✅ Supported Confirmed working on actual hardware in Nemotron safety features article Pure Python, no ARM64 issues
NeMo Agent Toolkit ✅ Supported Confirmed working directly on DGX Spark via uv pip install nvidia-nat in Agent Toolkit local article nemotron_h series requires --trust-remote-code
NemoClaw (OpenShell) ✅ Supported Confirmed working via DGX Spark Playbook in hands-on article Early Preview; nemoclaw setup-spark requires cgroup v2 configuration
Cosmos-Reason2 ✅ Supported Both 2B/8B models confirmed working in structured reasoning article vLLM requires a custom wheel from the repository
Cosmos-Predict 2.5 △ Partial Support 2B works, 14B OOM, Multiview requires 8 GPUs so not possible Updates since W1 article are being verified in W2 draft
NIM △ Model-Dependent ARM64 manifest availability varies by model Requires AI Enterprise; check individually

As a rough guideline: OSS Python package types (Guardrails / Agent Toolkit / Evaluator) should work on ARM64 without issues, while container types (Megatron-Bridge / NIM) require you to check the manifest in advance.

Here I've organized articles that verified each NeMo component hands-on, primarily using DGX Spark, by series. I hope this serves as an entry point when you want to know about a particular component or revisit a topic from a previous article.

Nemotron Model Series (7 articles)

A series that covers training, deployment, RAG, and RLHF for NVIDIA's Japanese-enhanced LLM "Nemotron 9B-v2-Japanese" and the large-scale MoE "Nemotron 3 Super."

Article Content Published
Trying Nemotron 9B-v2-Japanese in Various Scenarios Basic operations, CLI, OpenAI-compatible inference 2026-02-21
Deploying Nemotron 9B to SageMaker Tokyo Region in a Closed VPC Configuration Cloud deployment, security design 2026-02-21
Improving RAG Accuracy with NTA FAQ × RAFT RAFT (Retrieval-Augmented FT) 2026-02-22
Fine-Tuning Nemotron 3 Nano in Japanese on DGX Spark LoRA FT, Japanese data 2026-02-16
Teaching Classmethod's Culture (CLP) to Nemotron 9B Constitutional AI + SimPO 2026-02-23
Training Full-Layer LoRA Including Mamba-2 with Megatron-Bridge on H100 Megatron-Bridge v0.2.0, Brev H100 2026-02-25
Running Nemotron 3 Super on DGX Spark 120B Mamba-Transformer-MoE 2026-03-12

Evaluation, Safety & Agents (4 articles)

A collection of articles covering NeMo's three domains of "evaluation, safety, and agents," one article each. The agent domain has two articles.

Cosmos Series (3 articles)

A series of hands-on verification articles on DGX Spark for the Physical AI model family, Cosmos Predict 2.5 and Cosmos Reason2.

GTC 2026 Report

Upcoming Plans

In addition to already-published articles, I'm preparing a follow-up on NeMo Guardrails targeting Cosmos-Reason2, and an evaluation edition of Agent Toolkit using nat eval. In untouched areas, I'm also planning to write standalone articles on NeMo Curator, NeMo RL, Automodel, NeMo Retriever, NIM-based fine-tuning and service deployment, VLM safety evaluation combining ShieldGemma 2 and Llama Guard 4, and a deep dive into Cosmos Predict 2.5.

External Resources for Learning NeMo

Finally, here are some pointers to official documentation and primary sources.

Summary

NeMo Framework is a space that can feel like "a confusing proliferation of brands" at first glance, but if you look at it as a flow from Models (Nemotron / Cosmos) → Training (Megatron-Bridge / Automodel / RL) → Data & Evaluation (Curator / Evaluator) → Safety (Guardrails) → Agents (Agent Toolkit / NemoClaw) → Inference (NIM / Microservices), I think the positioning of each component isn't all that complicated.

If you're working with DGX Spark, here are 3 recommended starting points.

  1. Want to run an LLM — Start by running Nemotron 3 Nano or Cosmos-Reason2 with Ollama / vLLM (start from the Nemotron Japanese FT article)
  2. Want to build an agent — Build a local agent with NeMo Agent Toolkit + Nemotron 3 Nano (start from the Agent Toolkit local article)
  3. Want to operate safely — Layer Input/Output rails with NeMo Guardrails (start from the Guardrails introductory article)

Although NeMo's brand and component names change frequently, the underlying philosophy is a straightforward structure of "separating models, frameworks, and inference services, and combining each layer flexibly." I hope this article serves as a map of the NeMo ecosystem as of April 2026.

国内企業 AI活用実態調査2026 配布中

クラスメソッドが独自に行なったAI診断調査をもとに、企業のAI活用の現在地を調査レポートとしてまとめました。企業規模別の活用度傾向に加え、規模を超えてAI活用を進める企業に共通する取り組みまで、自社の現在地を捉えるためのヒントにぜひ。

国内企業 AI活用実態調査2026

無料でダウンロードする

Share this article