I tried to analyze the updated NVIDIA Physical AI in Cosmos 3 through the three-layer structure of factory AI

Hello, I'm Morishige from Classmethod's Manufacturing Business Technology Division.

My motivation for this article is that with the official release of Cosmos 3 at GTC Taipei, I wanted to take a bird's-eye view of the articles I've been accumulating as individual points throughout the DGX Spark series.

In the time series foundation model series, I covered a comparison of Chronos-2 and TimesFM 2.5, an experiment connecting Chronos-2 to a PLC-like simulator, and anomaly detection with the SKAB dataset — I've gotten a decent feel for the "numerical time series layer." On the other hand, the video-focused VSS Agent and Cosmos Reason, the simulation-focused Cosmos Predict, and the omnimodel integrated in Cosmos 3 are all running on the same DGX Spark, and I hadn't properly taken stock of what these ultimately contribute to factory AI and how.

With the public release of Cosmos 3 at the GTC Taipei keynote, the models covering each layer are now mostly in place. This article is an overview piece with zero hands-on verification, reading factory AI through a three-layer structure of numerical, visual, and simulation layers, and organizing the roles and division of responsibilities among the corresponding NVIDIA model groups (time series foundation models / Cosmos Reason / world models).

What Makes the 3 Systems Different in the First Place

When you break down "learning-based models" used in factory AI by their primary inputs and outputs, they naturally fall into 3 systems.

When laid out like this, it becomes clear that the 3 systems are not in opposition — they simply have different roles in terms of latency, memory, and real-time capability. The numerical layer handles control at the millisecond scale, the visual layer returns semantic interpretation at the second scale, and the simulation layer creates "plausible futures" as video at the minute scale.

Plotting the 3 systems on two axes of latency and memory makes their operational roles even clearer.

Let's look at what each layer can do, referring to the articles covered in the series.

Numerical Layer — The Domain of Time Series Foundation Models

This is the layer that directly feeds numerical time series data — temperature, flow rate, pressure, current, etc. — flowing from factory PLCs and sensors into models to produce future values and anomaly scores. In the series, I lined up 3 models in the comparison article for Chronos-2 / TimesFM 2.5 / NV-Tesseract.

Listing the characteristics of the 3 major models in order: Chronos-2 has an encoder + 1 forward pass structure, meaning latency barely changes even as the horizon extends (96→720 steps: 6.5ms → 6.8ms). The 28M model at 4ms / 84MB is practical for edge deployments at the Jetson class. TimesFM 2.5, on the other hand, has an autoregressive structure that gets heavier as the horizon grows (h=96→192: 86→229ms), but benefits from context scaling — extending from c=512 → 15,360 improves MASE from 1.106 → 0.770. NV-Tesseract is an industrial time series specialized model announced at GTC 2026 in partnership with Cognite and Celanese. It's evaluation-license based, and hands-on verification remains as homework for a future article.

In the SKAB follow-up article focused on anomaly detection, we saw the asymmetry that while TimesFM mean outperforms Chronos-2 by +18–22 AUC points on some datasets, making Chronos-2 multivariate makes it 7–8x faster in latency. A practical division would be Chronos-2 28M for edge deployment and TimesFM 2.5 mean when prioritizing accuracy.

In the experiment connecting Chronos-2 to a PLC-like simulator, spike detection across 72h × 16,177 windows achieved AUC 0.999 / F1 0.83. On the other hand, gradual drift in wear+spike mode only reached AUC ≈ 0.51, revealing through actual measurement that slowly progressing degradation is hard to catch with the numerical layer alone. This gap is addressed by the visual and simulation layers described next.

Visual Layer — The Domain of Cosmos Reason / VSS Agent

This is the layer that reads "what is happening" from surveillance cameras, manufacturing line camera footage, and still images for visual inspection. In NVIDIA's stack, this layer has two systems: standalone VLM inference (Cosmos Reason) and video search with an agent loop (VSS Agent).

Cosmos Reason 2 and the Reasoner Tower of Cosmos 3 Nano are VLMs with 4 structured output capabilities (2D Grounding / Robot CoT / Embodied Reasoning / Temporal Localization), suited for returning JSON answers to one-shot questions like "Is PPE being worn correctly in this image?" or "Which process does this action correspond to?" In the series, I measured 6 features + PPE detection + video benchmarks in the article testing structured reasoning with Cosmos Reason 2. VSS Agent + Skills, the NVIDIA Video Search and Summarization Blueprint, uses Cosmos Reason as its VLM backend while being a larger package that includes video search + summarization + agent loop + MCP. This is covered in the re-verification article for VSS 3.1.0 EA and the article organizing the VSS in the age of Agents + Skills as a reading piece.

The division is simple: use Cosmos Reason for standalone VLM inference, and VSS when you need video search, summarization, and an agent loop. With Cosmos 3's Reasoner Tower now matching Cosmos Reason 2 in performance, it's natural to expect the Cosmos 3 series to be adopted as the VLM for VSS going forward. I'll revisit this in more detail in the "Three Layers Updated by Cosmos 3" section later.

While the numerical layer answers "is the current value anomalous?" in sub-second time, the visual layer returns "how do we interpret the currently visible situation?" in a few seconds. In real projects, the most effective pattern is one where the visual layer adds meaning to alerts from the numerical layer.

Simulation Layer — The Domain of World Models

This is the layer that generates "plausible next videos" from past footage or a single image. On the article side, I covered world foundation models in the verification article for Cosmos Predict 2.5 + Reason2.

There are mainly 3 scenarios where world models are effective in factory AI. For synthetic data generation, they're useful for generating abnormal patterns that can't be collected sufficiently in real machines as video, to pre-train Cosmos Reason-type visual layer models. The Sim2Real bridge is the context of augmenting robot arm motion data in simulation and transferring it to real-machine ACT / VLA. And as a Digital Twin, reproducing physical phenomena in processes as video for impact prediction before equipment changes or for training materials is also within scope.

We had actual measurements on DGX Spark of Cosmos Predict 2.5 (2B model) generating 1280×704 video in 36 steps, taking about 30 minutes. This is not a layer for real-time control; the practical approach is to use it to create data upstream or to produce review materials downstream.

With Cosmos 3, the world model has been restructured into Cosmos 3 Nano / Super as an omnimodel, evolving to handle text / image / video / audio / action input and output in a single model.

Collaboration Scenarios for the 3 Systems

Mapping the 3 systems to real manufacturing use cases naturally reveals collaboration patterns.

Writing out the flow for Scenario A in sequence:

Looking at this sequence, it's easy to grasp the structure where the 3 systems are "viewing the same event from different angles." It's a layered design where the numerical layer responds immediately, the visual layer complements, and the LLM verbalizes. The Cosmos-series world models don't enter this direct operational loop, but they contribute by mass-producing anomaly footage during the pre-training stage to raise the overall accuracy floor.

Three Layers as Seen Through NVIDIA's Strategy Map

NVIDIA has intentionally positioned the 3 systems as complementary, and the strategic outline has become clear at GTC 2026 / GTC Taipei. The numerical sensor prediction and anomaly detection area is handled by NV-Tesseract (announced in partnership with Cognite and Celanese). Video-based factory visualization centers on VSS + Cosmos Reason (expanding through partner cases like Invisible AI / Tulip / Fogsphere / Pegatron and others). The synthetic data, Digital Twin, and Policy Model space is handled by the world model, restructured from Cosmos Predict to the Cosmos 3 omnimodel. This is the three-pillar structure.

At the GTC Taipei keynote, Cosmos 3 was announced alongside Alpamayo 2 (a reasoning VLA for autonomous driving) and Isaac GR00T (a humanoid robot reference) as a foundation model for Physical AI. NVIDIA is organizing Physical AI into 3 areas — "general AI / autonomous driving / humanoid" — with foundation models and reference designs for each. In the context of factory AI, this can be read as NVIDIA intentionally mapping models to the three layers of numerical, visual, and simulation.

Three Layers Updated by Cosmos 3

With the release of Cosmos 3 at the GTC Taipei keynote, both the visual and simulation layers have been updated. Here's a summary of the key points.

Cosmos 3 is released as Nano (16B) and Super (64B) omnimodels, with Edge (4B) coming soon. The license is OpenMDW 1.1 (Linux Foundation, commercially available). The Reasoner Tower for understanding and the Generator Tower for generation, which were separate in the previous generation, are integrated into a single omnimodel — though at inference time, only the Reasoner Tower can be extracted and used as a VLM. For actual behavior and benchmark figures of each model, please also refer to the following articles.

https://dev.classmethod.jp/articles/dgx-spark-cosmos3-omni-world-model-policy/

https://dev.classmethod.jp/articles/dgx-spark-cosmos3-family-usecase-map/

When loading onto DGX Spark's 128GB unified memory, running Cosmos Reason 2, Cosmos 3 Nano, and Chronos-2 all together fits within about 50GB, making it practical to have the visual layer + numerical layer fully self-contained on a single machine. Running world model generation seriously would still be best done at a separate time.

Manufacturing 5-Layer Organization — Rule-Based Layers and Learning-Based Layers

So far I've been focusing on the "3 systems of learning-based models," but in actual manufacturing settings, these sit on top of existing rule-based layers like PLC / SCADA / MES. Organizing both into 5 layers gives a more three-dimensional view of the AI side's role division.

Viewing the contrast as "rule-based layer = the world of existing equipment" vs. "learning-based layer = today's three layers + LLM assistance" makes it easier to organize what the manufacturing AI proposal is actually about.

The judgment materials to incorporate when designing a quality stabilization model for manufacturing can also be organized with the same three layers + LLM assistance.

Equipment signals and subtle anomaly signs go to the numerical layer, images and video go to the visual layer, and reference information and human expertise go to LLM assistance — that's the basic mapping. The world model doesn't directly correspond to these 5 items, but functions as a behind-the-scenes contributor by retroactively generating synthetic data for equipment signals and images/video to fill data gaps.

When a real project comes in with "please build a quality stabilization model," going through this 5-item inventory first makes it easier to move the discussion forward on what should be placed on which layer — that's the practical value of this organization.

Summary

I've broken down the learning-based layers of factory AI into 3 systems and realigned them at the current state after the release of Cosmos 3. There are 4 key takeaways.

First, the 3 systems have different roles in terms of latency, memory, and real-time capability. The numerical layer engages with control at the millisecond scale, the visual layer returns semantic interpretation at the second scale, and the simulation layer handles data generation at the minute scale. Second, the 3 systems are strongest in collaboration. The visual layer adds meaning to anomaly scores from the numerical layer, and the simulation layer acts as a behind-the-scenes supplier of pre-training data. Third, Cosmos 3 has significantly updated both the visual and simulation layers — Cosmos 3 Nano's Reasoner Tower is competitive with Cosmos Reason 2, and as an omnimodel, the world model domain has taken a major step forward with 4-modality integration and Policy Model. Fourth, with learning-based layers (prediction / optimization) sitting on top of rule-based layers (PLC / SCADA / MES), this organization serves as a useful reference axis when designing quality stabilization models for manufacturing.

Reference Links

Time Series Foundation Model Series

• Running and Comparing Time Series Foundation Models on DGX Spark
• Predicting PLC-Style Time Series Data with Chronos-2 and Generating Maintenance Comments with Nemotron
• Trying Industrial Sensor Anomaly Detection with SKAB and Time Series Foundation Models

VSS-Related Articles

• Looking into VSS 3.1.0 EA and the Current State of Manufacturing VSS as Seen at Hannover Messe
• Thinking About Everyday Use Cases for NVIDIA VSS + AI Agents + Skills in the Field

Cosmos-Related Articles

• Trying Structured Image and Video Analysis with Cosmos-Reason2 on DGX Spark
• Running NVIDIA Cosmos 3 on DGX Spark
• Organizing the Cosmos 3 Family Use Case Map on DGX Spark

NeMo Framework Overview

• A Bird's-Eye View of NVIDIA NeMo Framework — Spring 2026 Ecosystem Map and DGX Spark Series Index