I thought about a Mini-FOX configuration to start small with the NVIDIA FOX Blueprint

Introduction

Hello, I'm Morishige from Classmethod's Manufacturing Business Technology Division.

Many of you may be curious about the Factory Operations Blueprint announced by NVIDIA, commonly known as FOX. It's a concept that connects factory sensors, machine signals, video, work procedures, and robots, with a Factory Manager AI overseeing the entire shop floor. It's an exciting read, and with case studies from Foxconn and Pegatron included, I got the impression that a large-scale AI Blueprint for manufacturing has finally arrived.

https://blogs.nvidia.com/blog/factory-operations-fox-blueprint-ai-brain/

However, after reading through the article, I was a bit taken aback by the assumption that the hardware would be at the DGX Station level. Building an AI Brain for an entire factory right from the start is somewhat heavy as a PoC.

In this article, I'll try to break down the FOX concept into something as small as 1 line, 1 camera, and 1 use case, and think through an approach I'm calling "Mini-FOX" — a way to get started by combining DGX Spark, PCs, Jetson, and AWS. Mini-FOX is not an official NVIDIA term; it's simply a label I'm using within this article for organizational purposes.

The Factory-Wide AI Brain Envisioned by NVIDIA FOX Blueprint

First, let me briefly summarize the outline of FOX within the scope of official announcements.

FOX is a reference design that integrates machine signals, quality systems, work procedures, and operational alerts within a factory, with a Factory Manager AI orchestrating specialized agents and machinery. The Factory Manager AI acts as the "brain of the shop floor," with specialized agents hanging below it responsible for individual domains such as safety, quality, maintenance, and operations.

Here, at a rough level of granularity, I'll list the FOX elements covered in this article. With NemoClaw at the center, AI-Q Blueprint and Nemotron-series open models sit inside, and a model improvement loop via TAO sits on the outside. Video is handled by Metropolis VSS, and the design is intended to work with NVIDIA stacks such as Cosmos for world models, Omniverse, and OpenShell as a sandbox foundation — with the reference optimized to run on DGX Station hardware. On the case study side, Foxconn, Pegatron, Advantech, and Wistron are featured.

I won't go into detail on each component here. Trying to break down FOX on its own would fill an entire article. The main topic is "how to start small without simply replicating the official configuration."

Carving Out a Mini-FOX Instead of Targeting the Entire Factory at Once

This is where the main topic begins.

If the official FOX is the "fully equipped version," Mini-FOX is the image of a "trial version" scaled down to a single line. When you shrink the key elements of FOX, you get the following correspondence:

Personally, I think this way of carving things out is the most realistic approach. With AI for manufacturing, the first hurdle is whether you can capture one event that the shop floor is truly struggling with, rather than "connecting everything." If you stand up many specialized agents in parallel from the start, the difficulty of operations and data collection jumps dramatically all at once.

The Minimal Configuration of Mini-FOX Starts with Event Capture

The minimal configuration of Mini-FOX follows a flow where video and sensor data are converted into events before being passed to an LLM or VLM. To get a feel for the overall picture, let me lay out the full diagram first.

One thing I want to emphasize here is: don't send every frame to the VLM. If you stream video directly into a VLM, inference costs and bandwidth will quickly become burdensome. The approach is to first thin out frames at the edge, convert only anomaly candidates into events using lightweight detection or rule judgment, and then have the LLM or VLM produce situation descriptions, candidate causes, and next verification actions for those events.

As a concrete example of an event JSON, let me show a case where a cart was left unattended in an aisle. The idea is to format the detector's output directly as JSON and pass it to the reasoning side.

{
  "timestamp": "2026-06-17T10:15:30+09:00",
  "camera_id": "line-a-camera-01",
  "line_id": "line-a",
  "event_type": "aisle_obstruction_candidate",
  "confidence": 0.82,
  "frame_uri": "s3://example-bucket/events/2026/06/17/frame-001.jpg",
  "detected_objects": ["cart", "box"],
  "rule_triggered": "cart_stayed_in_aisle_over_30s",
  "llm_summary": "A cart and box are placed in the aisle, potentially obstructing worker movement.",
  "recommended_action": "Please ask a nearby worker to confirm removal.",
  "human_feedback": null
}

Keeping records at this level of granularity allows you to write human review results into human_feedback later, and the data can also be used as a dataset for retraining. Even without locking down a detailed schema from the start, I think having at least timestamp, camera_id, event_type, frame_uri, llm_summary, and human_feedback is sufficient.

A Locally Oriented Configuration Centered on DGX Spark

When the hardware side leans toward DGX Spark, it works well for PoCs where shop floor video is difficult to send externally. The idea is to run the VLM and LLM on DGX Spark, while the PC or Mac mini side handles the UI and API.

The advantage of this configuration is that it keeps data local while still giving the feel of the NVIDIA stack. It connects naturally with existing DGX Spark validation assets such as Cosmos-series VLMs, Nemotron-series LLMs, VSS-style video summarization, and NemoHermes. When there's an internal PoC requirement like "we'd prefer not to send shop floor video outside," I think starting with this form is the most practical approach.

That said, even here it's better not to immediately aim for the massive factory manager that's assumed on DGX Station. Starting with 1–3 cameras and implementing at a scale where you cycle through event capture and human review will make things easier to manage, both as an article and in practice.

PCs and Jetson Divide Labor Between Lightweight Detection and Cloud Reasoning

If you don't have a DGX Spark, or if you want to start from truly low cost, it's probably better not to try doing everything on a PC or Jetson alone. The idea is to run lightweight object detection and rule judgment on the edge side, and pass only anomaly candidates to an LLM or VLM in the cloud.

Rather than continuously sending normal video, the approach is to send only representative frames or short clips from around the time of an anomaly to the cloud. This makes it easier to balance bandwidth, cost, and privacy, and also avoids the accident of shockingly large cloud bills during a PoC.

Here's an image of the division of roles between edge and cloud:

Keeping lightweight judgment on the edge means that even if the network goes down, the first-tier alert can still fire, which gives the shop floor considerably more peace of mind. If only the reasoning is cloud-dependent, setting up degraded operation during outages can be done relatively straightforwardly.

On AWS, Greengrass Serves as the Center of Edge Management

When building around AWS, it's natural to use AWS IoT Greengrass as the edge runtime. Greengrass is a management platform that can run Lambda functions and containers on edge devices, and combined with AWS IoT Core, it makes it easier to securely operate edge devices across multiple locations. AWS's official blog also introduces a configuration that uses IoT Greengrass and IoT Core to perform video analysis for industrial safety from existing CCTV and edge gateways, connecting to S3 and SageMaker.

In terms of roles, Greengrass handles the distribution and management of edge applications, IoT Core receives events, Lambda and Step Functions advance the workflow, and Bedrock generates explanatory text and response recommendations. Placing model improvement with SageMaker makes it easier to set up retraining and incorporation of review results later on.

If you have plans to deploy across multiple locations, it's easier to incorporate Greengrass from the beginning. Even for a single-site PoC, if you have horizontal expansion in mind afterward, placing a control point here means you won't be scrambling later, I think.

Narrow Down to a Single Use Case First

The first subject I recommend for running Mini-FOX is either "aisle obstruction detection" or "SOP deviation candidate explanation."

Aisle obstruction detection is easy to explain using video alone, easy to convert into events, and connects well with shop floor safety and 5S activities. The detection side can also be run with a simple combination of object detection and loitering detection, and false positives can be discussed with clear examples like "a cart in the aisle that's fine from a work operations standpoint."

SOP deviation candidates bring out more of the FOX character, but they require organizing work instruction documents and shop floor rules, making them a bit heavy for an initial PoC. In terms of the flow for this article, I think it reads more naturally to make aisle obstruction detection the main example first, then expand toward SOP verification as a development step.

Here too, it's safer not to aim for autonomous control from the start. Rather than jumping all the way to stopping machines or issuing instructions to equipment based on detections, first building a flow to record "what was found, how it was explained, and how a person judged it" is easier to demonstrate PoC value and tends to make it easier to get buy-in from the shop floor.

Specialized Agent Separation Can Wait

While the FOX concept introduces many specialized agents, it's easier to operate if you don't separate them too much right after starting Mini-FOX. Start by placing the processing within a single Supervisor Agent, and only split it out once you can clearly see from looking at the logs that roles are distinctly divided.

Even when splitting, I think starting with around the following 4 is sufficient: safety_agent handles classification of safety events and risk explanation; sop_agent handles checking against work procedures and rules; report_agent handles generation of daily and weekly reports; and learning_queue_agent focuses on collecting false positives and missed detections.

Here too, avoid diving into autonomous control, and keep human confirmation as a prerequisite. In PoCs for manufacturing, rather than immediately reaching for control of the machinery side, first leaving a record of "what was found, how it was explained, and how a person judged it" makes it easier to establish evaluation criteria at the shop floor.

Summary

Here are the three configurations — DGX Spark-centered, PC and cloud sharing, and AWS IoT Greengrass-centered — laid out side by side by key criteria.

As for expanding from here, possibilities might include: actually running Mini-FOX's image event analysis on DGX Spark; building a PoC that explains factory events using AWS IoT Greengrass and Bedrock; turning aisle obstruction detection into a dataset with human review; auto-generating daily reports from Mini-FOX event logs; or thinking through a configuration that connects VSS Blueprint with Mini-FOX.

Reference Links

• NVIDIA Factory Operations Blueprint Gives Factories a New AI Brain
• Improving industrial safety with video analytics, AWS IoT Core, and AWS IoT Greengrass
• Build machine learning at the edge applications using Amazon SageMaker Edge Manager and AWS IoT Greengrass V2