I tried real-time analysis of camera footage with the Live VLM WebUI on DGX Spark
This page has been translated by machine translation. View original
Introduction
Hello, I'm Morishige from Classmethod's Manufacturing Business Technology Department.
Up until now, I've been using DGX Spark primarily for text-based applications such as local code completion and integration with Claude Code, but this time I'd like to try something a bit different and work with "video."
I used the Live VLM WebUI from Playbooks to analyze webcam footage with a VLM (Vision-Language Model).
What is Live VLM WebUI?
Live VLM WebUI is a tool included in the Multimodal/Vision category of DGX Spark Playbooks. It analyzes camera footage in real time using a VLM and displays the results in a browser.
You can choose from multiple backend engines including Ollama, vLLM, SGLang, and NVIDIA NIM. What's interesting is that by customizing the prompt, you can ask various questions about the same camera footage from different angles — such as object detection, scene recognition, OCR, and safety monitoring.
Normally it accesses the camera directly via the browser's WebRTC API, and it also supports RTSP stream input (Beta). This time, I tested it with webcam input via WebRTC.
By the way, NVIDIA has also released an AI Blueprint called "Video Search and Summarization (VSS)" that specializes in searching and summarizing recorded footage. Live VLM WebUI, which describes "what is currently in view" in real time, and VSS, which "searches for specific scenes from past footage," are tools you'd use differently depending on the use case.
Configuration for This Test
This is a configuration that runs entirely within the local network. For this test, I used Ollama as the backend and compared two models: gemma3:4b and llama3.2-vision:11b.
Setup
Preparing the VLM Model
First, prepare the VLM model with Ollama. I recommend starting with the lightweight gemma3:4b to confirm everything works.
ollama pull gemma3:4b
ollama pull llama3.2-vision:11b # for comparison
Installing Live VLM WebUI
You can install it without polluting your system Python environment using uv tool install.
uv tool install live-vlm-webui
In v0.2.1, 43 packages are installed and two commands become available: live-vlm-webui and live-vlm-webui-stop.
Starting Up
Start the service with Ollama specified as the backend.
live-vlm-webui --model gemma3:4b --api-base http://localhost:11434/v1 --port 8090
When started, a self-signed SSL certificate is automatically generated, and the DGX Spark's GPU (NVIDIA GB10) is also automatically detected.
Initialized VLM service:
Model: gemma3:4b
API: http://localhost:11434/v1 (Local)
Starting server on 0.0.0.0:8090
Auto-detected NVIDIA GPU (NVML available)
Detected NVIDIA DGX Spark (Version 7.2.3)
Access https://<DGX Spark IP>:8090 in your browser to open the WebUI.
Trying It Out
Pointing the Camera at the Desk
I pointed the webcam at my desk and asked the VLM "what is visible in this footage." On the desk I had arranged the DGX Spark and its box, a mechanical keyboard, Ghana chocolate, and a Xiaomi monitor.
With a one-sentence English prompt (Describe what you see in this image in one sentence.), the following output was returned.
A NVIDIA DGX Spark monitor is sitting on a desk next to a Xiaomi laptop, a grey keyboard, and a box of Ghana Milk Chocolate.
The DGX Spark box was misidentified as a "monitor," but the Xiaomi monitor, keyboard, and Ghana chocolate were correctly identified. The latency was about 1,100ms. For real-time analysis, this level of accuracy in just over a second is quite practical.
Changing the Prompt
I tried changing the prompt for the same scene.
Switching to object enumeration (List all objects visible in this image.) changes the output to describe each object individually.
- NVIDIA DGX Spark Box: A green cardboard box with the NVIDIA logo.
- Lotte Ghana Milk Chocolate Bar: A rectangular chocolate bar with Japanese writing.
- Keyboard: A silver and grey mechanical keyboard.
- Computer Monitor: A black Xiaomi monitor.
- Wall Decor: A leafy, vertical wall decoration monitor.
I was surprised that it even recognized the Japanese packaging on the Ghana chocolate.
Japanese prompts (この映像に映っているものを説明してください。) also return natural Japanese output.
- NVIDIA DGX Spark：NVIDIA のゲーミンググラフィックボードです。
- キーボード：白色のメカニカルキーボードです。
- Lotte Ghana Chocolate：ロッテのミルクチョコレートの箱です。
- Xiaomi モニター：Xiaomi のモニターです。
- 背景には植物の模様の壁飾りが写っています。
Describing the DGX Spark as a "gaming graphics board" is a charming mistake, but the overall recognition accuracy is high.
Inference Speed
I measured latency with gemma3:4b. The setting queries the VLM once every 30 frames (approximately 1-second intervals).
| Prompt | Latency | Feel |
|---|---|---|
| English, 1 sentence | ~1,100ms | Nearly real-time |
| English, list | ~2,000ms | Slight wait |
| Japanese | ~1,100–2,500ms | Depends on output length |
The experience is like "AI narrating the situation every few seconds." It feels more like periodic status reports than real-time video surveillance.
Comparing Models
I compared the output of gemma3:4b and llama3.2-vision:11b on the same desk scene.
The English output from llama3.2-vision:11b was more detailed than gemma3:4b, recognizing even the jacket in the background and the curtain pattern. However, when switching to a Japanese prompt, the output breaks down completely.
| Aspect | gemma3:4b | llama3.2-vision:11b |
|---|---|---|
| English accuracy | Accurately identifies major objects | More detailed (9 items), picks up background |
| Japanese support | Natural Japanese output | Grammar breaks down, not practical |
| Latency (English, 1 sentence) | ~1,100ms | ~3,000–4,000ms |
| GPU utilization | 13–14% | ~90% |
| Memory usage | ~15 GB | ~54 GB |
For real-time video analysis, bigger isn't always better. gemma3:4b has a good balance of speed, Japanese support, and resource efficiency — with 13% GPU utilization against DGX Spark's 128GB memory, there's plenty of headroom. The remaining resources can be used for other tasks simultaneously.
llama3.2-vision:11b is suitable for cases where detailed recognition is needed in English only, but the latency is 3–4x higher, making the trade-off with real-time performance significant.
Comparison with Edge AI (YOLO)
Here I'd also like to clarify the difference from another approach. Edge devices that can perform object detection with YOLO on the camera itself — like the Seeed Studio ReCamera 2002w — are increasingly available. I actually tested the same desk scene with ReCamera's YOLO detection.
With ReCamera's YOLOTIn Detection, the mouse was correctly detected as "mouse," but the Ghana chocolate was misclassified as "book." Since "chocolate" is not included in the COCO dataset's 80 classes, it was assigned to "book" due to similar shape. No bounding boxes appeared for the DGX Spark box or keyboard.
On the other hand, the gemma3:4b VLM described the same scene with natural language: "NVIDIA DGX Spark Box," "Lotte Ghana Milk Chocolate Bar," "Keyboard," and "Xiaomi Monitor." This difference is exactly what distinguishes the YOLO and VLM approaches.
| Aspect | Edge YOLO (ReCamera) | DGX Spark (VLM) |
|---|---|---|
| Output format | Bounding boxes + labels | Natural language sentences |
| Vocabulary | Fixed (COCO 80 classes, etc.) | Unlimited |
| Contextual understanding | Individual object detection only | Grasps relationships across the entire scene |
| Custom questions | Not possible | Free to ask via prompts |
| Inference speed | Tens of ms | 1–4 seconds |
| Required hardware | Camera alone (USB powered) | GPU server |
It's not a question of "which is better" — they serve completely different purposes. If you want to quickly know "where is that object," use YOLO; if you want to ask "describe this situation in words," use VLM. YOLO is like eyes, VLM is closer to a brain.
In a production environment, a pipeline where YOLO filters at high speed before passing to VLM is also conceivable. For example, a camera in a factory corridor selects "only frames with a person," and DGX Spark then checks whether they're wearing helmets and safety equipment. Since the video stays within your local network without being sent to the cloud, it's an easy-to-prototype setup even in privacy-sensitive environments.
Bonus
The Live VLM WebUI logs record VLM responses with timestamps. By saving them to a file with tee at startup, you can later trace how recognition results change over time.
live-vlm-webui --model gemma3:4b --api-base http://localhost:11434/v1 --port 8090 2>&1 | tee vlm-log.txt
As a test, I spent about 2 minutes putting objects on and removing them from the desk. The prompt was List all objects visible in this image. Below is an excerpt of recognition results extracted from the log.
18:25:41 Nvidia DGX Spark box, notebook, chocolate box (Lotte Ghana Milk Chocolate),
keyboard, computer monitor, desk, jacket. (1307ms)
18:25:59 Nvidia DGX Spark box, Keyboard, Mouse, Bottle of Kirin Lemon Tea,
Box of Japanese sweets (1253ms)
18:26:03 NVIDIA DGX Spark box, bottle of Kirin tea, keyboard, mouse, desk,
plant, plant pot, tea bag. (1538ms)
18:26:09 NVIDIA DGX Spark box, water bottle, LOTTE Ghana Milk Chocolate box,
keyboard, mouse, monitor, USB cable, reflection (1377ms)
18:26:33 Nvidia DGX Spark box, monitor, keyboard, mouse, desk, cardboard,
shadows (1181ms)
18:26:53 Nvidia DGX Spark box, computer monitor, computer keyboard,
computer mouse, two paper cups, plant, cables (1329ms)
18:27:41 Nvidia DGX Spark box, plush toy (blue whale), keyboard, mouse,
computer monitor, cable, gray speaker. (1366ms)
18:27:56 Box, Keyboard, Mouse, Phone, Computer monitor, Wall, Tree (1166ms)
18:28:05 NVIDIA DGX Spark box, bottle of tea, computer monitor,
computer keyboard, computer mouse, desk, water, background blur. (1351ms)
When I placed a drink, it recognized "Bottle of Kirin Lemon Tea"; a stuffed animal became "plush toy (blue whale)"; a smartphone became "Phone" — recognition updated within 1–2 seconds as objects were added or removed. The fact that the stuffed animal's type varied between "blue whale," "plush shark," and "plush dolphin" across frames is an interesting glimpse of the limits of a 4B parameter model.
On the other hand, objects that were consistently visible in the frame — like the DGX Spark box and keyboard — were stably recognized throughout. Latency also stayed within the 1,100–1,500ms range, and analysis of 58 frames was logged over approximately 2 minutes.
By aggregating these logs, you could perform quantitative analysis such as "which frames recognized a specific object" and "how stable was the recognition." It seemed like a solid enough foundation to use as a simple video monitoring prototype.
Summary
I installed Live VLM WebUI on DGX Spark and tested real-time VLM analysis of camera footage.
Setup is straightforward. Installation completes with a single uv tool install command, and you can get it running immediately in combination with Ollama. With gemma3:4b, footage can be analyzed at approximately 1-second intervals, and Japanese prompts are properly supported. Since GPU utilization stays at around 13%, being able to share DGX Spark's 128GB memory with other workloads is a significant advantage.
On the other hand, while llama3.2-vision:11b offers detailed recognition in English, the fact that Japanese output breaks down and latency increases 3–4x makes it a tough fit for real-time use cases. Personally, discovering that "bigger model ≠ always better" was the most interesting finding this time.
The evolution of VLMs for video is moving fast, so as models advance to the next generation, these limitations should change. There are plenty of ideas to explore — combining with IP camera RTSP streams, integrating with edge-side YOLO, and more.
Test Environment
| Item | Specs |
|---|---|
| DGX Spark | 128GB LPDDR5x, GB10 (Grace Blackwell) |
| Live VLM WebUI | v0.2.1 |
| VLM Models | gemma3:4b / llama3.2-vision:11b |
| Backend | Ollama |
| Camera | HD Pro Webcam C920 (1080p) |
| Comparison Device | Seeed Studio ReCamera 2002w (YOLO) |
