I thought about practical use cases for NVIDIA VSS + AI Agents + Skills in familiar workplaces

Introduction

Hello, I'm Morishige from Classmethod's Manufacturing Business Technology Division.

A little while ago, I summarized the VSS 3.1.0 EA setup differences and Hannover Messe 2026 manufacturing use cases in the article below.

https://dev.classmethod.jp/articles/dgx-spark-vss-3-1-revisit/

The day after that article was published (2026-05-13), a follow-up article appeared on the NVIDIA Developer Blog.

https://developer.nvidia.com/blog/transform-video-into-instantly-searchable-actionable-intelligence-with-ai-agents-and-skills/

The main subject is "handling VSS with AI Agents + Skills," and it introduces a configuration for calling VSS in natural language from four types of agents: Codex, Claude Code, OpenClaw, and NemoClaw. It's an extension of what I wrote in the previous article — that "VSS doesn't appear on the surface as a standalone product" — and now the interface on the user side has become one step more accessible.

In this article, stepping back a bit from the heavy-duty manufacturing use cases, I'll organize some scenarios imagining how "VSS Skills could fit" in familiar settings such as retail stores, reception areas, small warehouses, and restaurants.

VSS Has Entered the Era of AI Agents + Skills

The main message from the NVIDIA Developer Blog is that "VSS can now be handled with AI Agents + Skills." Specifically, Skills — units of functionality — are distributed to the agent side, and when an agent sends a video query in natural language, the Skills call the VSS API and return the results.

Skills are "small folders centered around SKILL.md" defined by the specification at agentskills.io, and are distributed in a form that agents can read and execute. VSS Skills are published on GitHub at NVIDIA-AI-Blueprints/video-search-and-summarization/tree/main/skills, and as of May 2026, 10 skills are available.

skills/
├── alerts/              ← Adding, managing, and monitoring alerts on video streams
├── deploy/              ← Deploying and removing VSS profile docker compose
├── report/              ← Generating analysis reports via /generate endpoint
├── rt-vlm/              ← Real-time VLM (caption / alert / stream / OpenAI compatible)
├── video-analytics/     ← Querying video metrics in Elasticsearch via VA-MCP
├── video-search/        ← Multi-angle embeddings + VLM critique for video search
├── video-summarization/ ← Video summarization via LVS microservice
├── video-understanding/ ← Q&A about video content using VLM
├── vios/                ← Video IO + Storage (recording, timeline, clip extraction)
└── vss-frag/            ← Extended long-form summarization, Enterprise RAG, HITL via video_search_frag

The VSS core itself offers 5 Developer Profiles to choose from based on which workflow you want to run.

In other words, it's a two-layer structure where Skills are "the interface on the user side" and Profiles are "how VSS is run." As shown in the diagram below, from the agent's perspective, Skills are like the handles of drawers, and VSS Profiles are the contents inside those drawers.

Installation is done by sending a single natural language prompt to the agent, which creates symbolic links for the entire Skill folder into ~/.claude/skills/<name>/ or ~/.codex/skills/<name>/. For generic hosts without agent-specific paths, ~/.agents/skills/<name>/ compliant with the agentskills.io specification is also available. Since it uses symbolic links, doing a git pull on the repository side simultaneously updates the Skills for all agents — a very convenient design.

The Positioning of the 4 Supported Agents

The official blog explicitly states that "VSS Skills can be called from any of Codex, Claude Code, OpenClaw, or NemoClaw." Simply placing the same Skill folder according to each agent's conventions allows any agent to hit VSS through the same natural language interface.

Multi-Angle Embedding Search and the Agentic Reasoning Layer

Digging a little into the search profile reveals the sophisticated engineering behind it as a video search engine. The official blog prefaces it by saying "video search is one of the most challenging areas in modern information retrieval," then highlights two core capabilities.

The first is a method combining embedding vectors by angle (referred to in the official blog as Multi-Embedding Search), which creates separate embedding indexes for different angles such as objects, events, and attributes, then integrates the results for ranking. If you try to capture parallel conditions like "worker in a red uniform," "person climbing a ladder," and "not wearing a helmet" using only a single type of vector similarity, the priorities of each element tend to conflict. The idea is that separating indexes by type and then combining them allows you to maintain balanced conditions without sacrificing recall.

The second is the Agentic Reasoning Layer, which decomposes complex queries into subqueries (Query Decomposition), runs verification loops for each subquery (Verification Loops), and finally eliminates semantic duplicates (Semantic Deduplication). The flow looks like the diagram below.

The representative example shown in the official blog involves asking OpenClaw to perform the following on three 10-minute warehouse videos:

I have a set of warehouse videos located at ~/warehouse_videos. I need to find any instances of a worker climbing a ladder and verify they are wearing a hardhat and safety vest. Can you do this with the VSS Search profile that is deployed?

OpenClaw hits the search profile via the Skill, uses Query Decomposition to verify the three conditions — "ladder use," "helmet worn," and "safety vest worn" — as separate subqueries, retrieves candidates using multi-angle embeddings, has the VLM double-check, and finally organizes the "ladder + missing gear" results to consolidate them into a single report. The Verification Loop's ability to filter out false positives like "is that a cart rather than a worker?" seems to make a quiet but meaningful difference.

GPU-specific latency figures are also officially provided for the Alert Verification workflow (RT-DETR + Cosmos Reason 2, assuming 1 alert per minute).

The reason it's a combination of DGX Spark with AGX Thor rather than DGX Spark alone is that the CV pipeline side (DeepStream + RT-DETR) is intended to be offloaded to the AGX Thor. In familiar real-world settings, 14 streams should be sufficient in many cases, so it's nice to see DGX Spark positioned without looking inferior compared to the enterprise H100 configuration.

What Could Be Done in Everyday Workplaces

Rather than the large-scale cases like Invisible AI or Pegatron covered in the previous article, I tried to imagine scenarios where VSS Skills could be useful in more familiar, everyday settings — assuming a scale of 1-2 cameras per location. I'll save actual hands-on verification for another occasion, but I think a reasonable scope can be inferred from the combination of Developer Profiles and the roles of each Skill.

Reviewing Misplaced Products and Customer Flow in Retail Stores

For a drugstore or convenience store in town (assuming around 3-4 cameras per store), it seems interesting to use in-store traffic cameras to retrieve in natural language queries things like "customers who stood still in front of the red shelf for more than 3 minutes" or "shelves that went out of stock again right after restocking." The image is combining the search profile with video-search + report Skills, and asking Claude Code something like, "Pull out the time slots last Saturday afternoon when customers seemed confused in the cosmetics section." Behaviors invisible from POS data alone — like "picked up a product and put it back" or "looked for a staff member but couldn't find one and left" — could become useful material for staff shift reviews.

However, from a privacy perspective, customers' faces and personally identifiable elements are an obvious challenge. Since VSS alone doesn't cover masking functionality, separate mosaic processing and data retention period design would be needed for real-world deployment.

Recording Visitor Movement at Reception Areas and Entrances

For a reception area at an office or hospital (a scale fully coverable with 1-2 cameras), consider a configuration using the alerts (VLM) profile + video-analytics Skill. The idea is having the VLM perform real-time judgment on conditions like "an unscheduled visitor was at the counter for more than 5 minutes" or "a figure appeared at the entrance after 8 PM," with metrics stored in Elasticsearch.

Rather than for security purposes, using it to review visitor handling (average visitor dwell time, trends during peak hours) seems more likely to gain consensus at the workplace. The vision is using the report Skill to auto-generate monthly reports and feed into operational decisions like "delivery drivers concentrate on Tuesday mornings, so we should staff one more person at reception."

Monitoring Sorting Mistakes and Safe Behavior in Small Warehouses

Let me bring the official blog's ladder + PPE example down to a mid-sized local warehouse scale. Not a massive fulfillment center, but a regional logistics hub with 1 camera per line (5-10 cameras for the whole site). Combining the alerts (verification) profile with rt-vlm + alerts Skills could enable operations like "immediately alert when a helmet is not worn during ladder work" or "send a caution when more than 5 items are loaded simultaneously onto a picking cart."

The Pegatron PCB line case covered in the previous article achieved a major result of "67% reduction in defect rate," but at a local warehouse scale, the targets would more likely be "reduce monthly near-misses from 1 to 0" or "reduce weekly picking errors from 5 to 3." Even so, I believe the judgment accuracy of Cosmos Reason 2 can cover this range sufficiently.

Supplementing SOP Compliance in Restaurant Kitchens with Video

Finally, let's talk about chain restaurant kitchens (1-2 cameras per store, only inside the kitchen). HACCP self-inspections are centered on paper records, but combining rt-vlm + video-understanding could supplement items like "did they wash hands for at least 15 seconds before cooking," "was the refrigerator left open for more than 3 minutes," and "did they switch cutting boards after handling raw meat" using video.

The premise is that camera positions are limited to inside the kitchen, with a clear boundary of not pointing toward customer seating or personally identifiable areas. Since the video remains as evidence of SOP compliance, the image is using it for guidance from headquarters to store staff and as review material for new employee training. Even if the VLM's judgment occasionally wavers, there seems to be value in creating "an opportunity for humans to go back and review."

Aggregating Multiple Stores at Headquarters Quickly Gives an Enterprise Feel

Up to this point, I've been talking at the single-location level, but if you configure multiple stores and locations to aggregate camera footage to headquarters via the cloud for centralized analysis, it quickly scales up to enterprise level. For example, in a chain retail or restaurant operation, you could place Jetson Orin Nano Super or AGX Thor on the edge at each store to handle real-time judgment and primary filtering, while the headquarters-side DGX Spark or H100 runs cross-location analysis using the search profile + lvs profile — a two-stage Edge-to-Cloud setup.

The Edge-to-Fog-to-Cloud architecture from Fogsphere and the Invisible AI deployment across 6 automakers introduced in the previous article are, at their core, this same structure. Analyses like aggregating "customers who were confused at the red shelf" across all stores, or visualizing "stores with frequent handwashing non-compliance" on a headquarters dashboard, become callable in natural language with just the report Skill. Since VSS Skills can be used with the same interface for both single-store and headquarters-aggregation use cases, it's a nice point that scaling up from PoC to production deployment is a seamless continuum.

Summary

In this article, I've explored what the AI Agents + Skills integration of VSS brings, alongside potential applications in familiar everyday settings.

The biggest takeaway in the context of this series is that the accessibility of handling VSS from familiar coding agents has come one step closer. While the picture I described in the previous article — that "VSS doesn't appear on the surface as a standalone product" — remains the same, I think it's accurate to say that only the interface on the user side has become one step more accessible.

For the four familiar workplace scenarios (retail, reception, small warehouse, restaurant), I assumed a scale of 1-2 cameras per location throughout. The benchmark of 14 streams for Alert Verification with DGX Spark + AGX Thor provides more than enough headroom for these small-scale settings. Stepping back from the heavy-duty manufacturing cases and imagining VSS Skills running in shops and offices around town feels both realistic and exciting.

Reference Links

• NVIDIA blog: Transform Video Into Instantly Searchable, Actionable Intelligence with AI Agents and Skills (2026-05-13)
• VSS Skills GitHub (NVIDIA-AI-Blueprints/video-search-and-summarization/tree/main/skills)
• agentskills.io specification
• VSS Documentation 3.0.0
• VSS Warehouse Blueprint 3.1.0 Release Notes
• VSS Brev Launchable
• Previous article: Investigating the Current State of Manufacturing VSS as Seen Through VSS 3.1.0 EA and Hannover Messe
• Running NemoClaw on DGX Spark
• Running NeMo Agent Toolkit with DGX Spark + vLLM in a Local Configuration
• An Overview of the NVIDIA NeMo Framework — Spring 2026 Ecosystem Map and DGX Spark Series Index