I tried running Gemma 4 on DGX Spark and benchmarked Japanese and multimodal capabilities

Introduction

Hello, I'm Morishige from Classmethod's Manufacturing Business Technology Department.

Google has released Gemma 4. Under the Apache 2.0 license, four models ranging from the small E2B (2.3B) to the large 31B Dense have been released at once. All models support multimodal input and can handle image input natively.

Since DGX Spark has 128GB of unified memory, even the largest 31B model can run in BF16 (without quantization). In fact, NVIDIA's official blog explicitly states that Gemma-4-31B BF16 runs on DGX Spark. Being able to test large models at full precision without quantization is something only 128GB makes possible.

In this article, I'll share the results of deploying all 4 Gemma 4 models on DGX Spark via Ollama and benchmarking them across two dimensions: Japanese text (JCommonsenseQA) and multimodal (image understanding). I've also included data on the effectiveness of the "Thinking mode" introduced in Gemma 4, as well as comparisons with the Gemma 3 and Nemotron series.

Overview of Gemma 4

Model Family

"E" stands for Effective, referring to small models that improve parameter efficiency through a technique called Per-Layer Embeddings (PLE). By giving each decoder layer an auxiliary embedding table, these models achieve greater expressive power with fewer parameters. 26B-A4B uses Mixture-of-Experts (MoE), where the total is 26B but only 3.8B is used during inference, making it an efficiency-focused design.

E2B and E4B support audio input in addition to text and images (though Ollama does not support audio as of April 2026).

License

Gemma 4 is released under the Apache 2.0 license. Up through Gemma 3, it used the proprietary "Gemma Terms of Use," which placed some restrictions on commercial use. With Apache 2.0, commercial use including modification and redistribution is now fully unrestricted. This is a significant change that makes it much easier for companies to consider adoption.

Key Changes from Gemma 3

Gemma 3 came in 4 sizes — 1B / 4B / 12B / 27B — with multimodal support for 4B and above. Here's a summary of the major changes in Gemma 4.

• A MoE model called 26B-A4B has been added. With Active 3.8B, it aims for performance close to a 27B Dense model
• Thinking mode, which reasons step-by-step before responding, has been added to all models
• Shared KV Cache allows the latter N layers to reuse the KV states from the earlier layers, reducing memory and computational cost during inference

Thinking Mode

In Ollama, it can be enabled simply by passing "think": true as an API parameter. In the response, the reasoning process is separated into message.thinking and the final answer into message.content, making post-processing straightforward.

Test Environment

Setup

Gemma 4 is supported in Ollama 0.20.0 and later.

# Gemma 4 — BF16 (E2B / E4B / 31B) + Q4_K_M (26B MoE)
ollama pull gemma4:e2b-it-bf16     # 10 GB
ollama pull gemma4:e4b-it-bf16     # 16 GB
ollama pull gemma4:26b             # 17 GB (Q4_K_M)
ollama pull gemma4:31b-it-bf16     # 62 GB

Since a BF16 tag for 26B-A4B was not available in Ollama, measurements were taken using Q4_K_M. The other 3 models are at full precision without quantization.

VRAM Usage

Actual measured values when models were loaded in Ollama (the SIZE column of ollama ps, including KV cache for the context length).

31B BF16 uses 117 GB, nearly filling the 128 GB of unified memory.

Text Benchmark: JCommonsenseQA

Benchmark Method

To measure Japanese commonsense reasoning ability, I used JCommonsenseQA v1.1. I evaluated 1,119 five-choice questions using a 3-shot prompt. Since previous articles (on the Nemotron series) used the same dataset, cross-model comparisons are possible.

Results

Gemma 3 latency is omitted because previous articles used lm-evaluation-harness, which cannot be directly compared with the Ollama API. Also note that Gemma 4's E2B / E4B / 31B are BF16, while 26B-A4B and comparison models are Q4_K_M, so the quantization conditions differ.

31B tops all models at 97.9%. In a previous article, Gemma 3 27B scored 93.9% as the highest score among DGX Spark local models, so this represents a 4-point improvement within the same series.

Worth noting is the 26B MoE. Despite having only 3.8B Active parameters, it achieves 96.4%. This surpasses Nemotron 3 Super (120B, Active 12B) at 94.4%, and its latency of 0.33s/q is on par with Nemotron 3 Nano (0.31s/q).

E4B also achieves 93.5% with 4.5B Effective parameters, surpassing the Japanese-specialized Nemotron Nano 9B v2 JP (91.2%). This suggests a high level of Japanese language capability as a general-purpose model.

Effect of Thinking Mode

* Reference value based on 370/1119 questions

For knowledge-based multiple-choice tasks like JCQ, Thinking mode was counterproductive across all models. The 26B MoE dropped by 9.2 points in particular. It gives the impression of overthinking — "maybe this option could also be correct."

Since Thinking mode is designed for tasks requiring step-by-step reasoning such as math and coding, tasks like commonsense reasoning where it's a matter of "knowing or not knowing" actually benefit from answering intuitively.

Multimodal Benchmark

All Gemma 4 models support multimodal input. Using photos taken at the GTC 2026 exhibition venue, I evaluated two aspects: Japanese caption generation and structured data extraction, and PPE (personal protective equipment) detection.

Japanese Image Captioning and Structured Tag Extraction

For 10 photos, I performed scene description in Japanese (captions) and structured tag information extraction (JSON format).

The 26B MoE is the fastest, with captioning at 4.0s and tag extraction at 2.4s. It being faster than E2B is likely because Q4_K_M quantization leaves more memory bandwidth headroom. The 31B BF16's massive model size of 117 GB is the bottleneck, taking about 37 seconds per image.

It's encouraging that all models achieved a 100% JSON parse success rate. If you specify JSON format in the prompt, it looks like structured data extraction can be used reliably.

This image captures the scene at a booth at an indoor exhibition or technology event.

In the center is a humanoid robot with a black body and silver joints on display,
with two men who appear to be staff standing behind it. On the counter in the foreground,
there is a basket containing snacks that appears to be for visitors.

{
  "location": "Exhibition Hall",
  "event_type": "Booth Demo",
  "subjects": ["Humanoid Robot", "Visitors", "Display Counter", "Basket"],
  "technologies": ["Humanoid Robot"],
  "people_count": 2,
  "atmosphere": "Demo in Progress"
}

PPE (Personal Protective Equipment) Detection

As a VLM application for the manufacturing industry, I also tested detection of safety protective equipment (helmets, safety vests, etc.) from images. For 5 scene images, I had the model determine wearing status and compliance in JSON format.

{
  "workers_count": 1,
  "ppe_items": [
    {
      "worker_id": 1,
      "hard_hat": false,
      "safety_vest": true,
      "safety_glasses": false,
      "gloves": false,
      "safety_shoes": false,
      "other": []
    }
  ],
  "compliance_score": "Low",
  "observations": "The worker is wearing a high-visibility safety vest, but other important personal protective equipment such as a helmet, safety shoes, and gloves cannot be confirmed. For forklift operations, head protection and foot safety are important."
}

It's convenient that such detailed judgments can be returned in a zero-shot manner. While I don't think it can match specialized models like Cosmos-Reason2-8B fine-tuned for the task (which achieved 90% accuracy after SFT in a previous article), the accuracy is sufficient for prototyping at the "let's try it first" stage.

DGX Spark-Specific Notes

• Ollama 0.20.0 or later is required. Gemma 4 was first supported in version 0.20.0, released on April 2, 2026
• 26B-A4B does not have a BF16 tag in Ollama. Only Q4_K_M is available. If you want to try BF16, you'll need a different runtime such as vLLM
• NVIDIA's DGX Spark Playbook also provides instructions for deploying Gemma 4 with vLLM
• An NVFP4 quantized checkpoint for Blackwell is reportedly in preparation. If 31B can run in NVFP4, improvements in both memory and speed can be expected
• The NVFP4 quantized checkpoint (nvidia/Gemma-4-31B-IT-NVFP4) has been released.

Model Selection Guide

Addendum (2026-04-04): Lightening 31B with NVFP4 Quantization

After the article was published, NVIDIA released Gemma-4-31B-IT-NVFP4. This is a checkpoint with weights and activations quantized to FP4 using NVIDIA Model Optimizer, and the official benchmark reports less than 0.5% accuracy degradation.

I was able to confirm it works with the vLLM container for DGX Spark vllm/vllm-openai:gemma4-cu130 (vLLM 0.18.2rc1, CUDA 13.0).

docker run -d --gpus all --network host --ipc=host \
  -e HF_TOKEN=$HF_TOKEN \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai:gemma4-cu130 \
  --model nvidia/Gemma-4-31B-IT-NVFP4 \
  --quantization modelopt \
  --tensor-parallel-size 1 \
  --max-model-len 4096 \
  --gpu-memory-utilization 0.85 \
  --trust-remote-code

The BF16 decode speed is cited from the NVIDIA Developer Forum benchmark (for a like-for-like vLLM comparison). Since this differs from Ollama BF16 in the main text (4.6 tok/s) in terms of inference engine, I'm using the vLLM values here to keep conditions consistent.

With model memory fitting within 31 GB, it's now possible to allocate 65 GB to the KV Cache. Maximum context length support has also increased to 141K tokens, easing the constraints of the BF16 version.

Note that the generic vllm/vllm-openai:latest (0.18.0) fails to load due to compatibility issues with Gemma 4's transformers implementation. When using NVFP4 on DGX Spark, use the gemma4-cu130 tag. DGX Spark-specific information is also being actively shared in the NVIDIA Developer Forum community.

Summary

I ran all 4 Gemma 4 models on DGX Spark and benchmarked them across both Japanese text and multimodal dimensions.

On JCommonsenseQA, the 31B achieved 97.9%, setting a new high score among DGX Spark local models. However, what impressed me most personally was the 26B MoE (Active 3.8B) at 96.4%. It was surprising to see it surpass Nemotron 3 Super (120B, Active 12B) at 94.4% with roughly one-third the Active parameters. Its latency of 0.33s/q puts it in the same speed tier as "small models."

In multimodal tasks as well, the 26B MoE was the fastest (captioning at 4.0s/image), and all models achieved 100% JSON structured output parse success. It demonstrated practical accuracy even on zero-shot tasks like PPE detection, making it sufficient for prototyping.

On the other hand, Thinking mode was counterproductive across all models for commonsense reasoning tasks like JCQ. It seems best to disable it for anything other than math or coding.

With the release of NVFP4 quantized checkpoints, latency improvements for 31B can also be expected. If a BF16 version of 26B MoE comes to Ollama, the results could become even more interesting.

Reference Links

• Google Blog: Gemma 4
• HuggingFace: Gemma 4
• Unsloth: Gemma 4 Docs
• Ollama: Gemma 4
• NVIDIA Developer Blog: Gemma 4 on DGX Spark
• NVIDIA Blog: RTX AI Garage — Gemma 4
• JCommonsenseQA v1.1
• Verification Scripts (GitHub)