I tried the non-generative diffusion model Nemotron Deciffusion 8B on DGX Spark

Introduction

Hello, I'm Morishige from Classmethod's Manufacturing Business Technology Department.

On October 14, 2026, a model called Nemotron Deciffusion 8B was released. The name appears to be a blend of "decision" and "diffusion," though the release page offers no explanation of its origin. The base is NVIDIA's Nemotron-Labs-Diffusion-8B — a diffusion language model. Yet it doesn't write a single character of text. You pass it a situation, a question, and choices, and it returns only the probability of "which choice is most plausible." The one who published it is Alex Steiner, a Solutions Architect at NVIDIA. It comes from a personal GitHub account and a dedicated Hugging Face organization, so I'll note upfront that this is not an official NVIDIA release.

https://pst2154.github.io/nemotron-deciffusion/

When I thought of diffusion language models, I assumed it was all about the speed of generation — filling in masked positions in parallel. This discards that generation entirely, using it only for judgment. Honestly, this direction caught me off guard. Moreover, the API is /v1/systemone, compatible with TypeSafe's Jev. Over the past month, I've been following non-generative judgment across three articles. There's the article comparing OpenJev implementations (as of 2026-09-18), the article reworking NeMo Switchyard's judge role into single-token judgment (as of 2026-09-23), and the article putting the decision model CLM — which does nothing but choose without generating — on the same measuring stick (as of 2026-09-27). The 528-item evaluation set and comparison method remain right here with me. If I can call it with the same API, there's no reason not to run it through.

https://dev.classmethod.jp/articles/clm-contrastive-language-model-switchyard-judge/

https://dev.classmethod.jp/articles/dgx-spark-nemotron-labs-diffusion-tri-mode-bench/

To state the conclusion upfront: zero-shot Deciffusion, given the same Japanese questions as Jev, reached the same level as Jev 1.13 measured on the same 528 items. Zero misroutes out of 87 production items, all 10 design consultation items rated strong, 10 misroutes out of 237 unseen items, and AUC 0.858. The p50 for a single judgment was 138 ms at the median input of 961 tokens, with FP8 occupying 9 GiB. I also re-ran the same 528 items with Jev 1.14, released the day before measurement, but my reading that they are comparable didn't change. However, the weaknesses also align with Jev's. 10 out of 14 discriminating questions were routed to weak, and adding an execution environment section reduced the weak count. It hasn't quite reached the point of directly replacing the local judge role.

This article organizes the nature and mechanism of Nemotron Deciffusion 8B using primary sources. I then run it on DGX Spark and present results comparing it against Jev, CLM, and the current judge role on the same 528 items from the 9/23 and 9/27 articles. I hope this is useful for those who want to place a small decision model in the judgment section of a router or agent.

The Nature of Nemotron Deciffusion 8B

First, I'll summarize the available primary information in a table. The publisher is a personal account and dedicated organization, not NVIDIA's organization. The README only states "Built on NVIDIA Nemotron-Labs-Diffusion-8B" and does not present itself as an NVIDIA product.

The list of training data and model card contain no mention of Japanese or multilingual support. Whether Japanese passes through as-is is confirmed with a single example in a later section.

From the listed figures, I'll pick out those most relevant to this article's interests. The numbers come from release/v2/SOURCES.md in the public repository, with citations.

A Diffusion Model That Fills Only One Mask

A diffusion language model fills masks (blanks) scattered throughout text in parallel, reading surrounding context. When I measured generation on DGX Spark in a May article, the speed of this blank-filling was the main topic. What Deciffusion does is narrow that blank-filling down to just one mask. Imagine an answer sheet with the situation and question written out, and only one answer blank left open. Whether A, B, or C goes there is decided after reading everything around it. That's all this model does.

I'll diagram the difference from a conventional LLM, as far as I can read from the distribution code's serving/app/scoring.py.

The single-token judgment I built in the 9/23 article reads the probability of a conventional LLM outputting "weak" or "strong" as its "next single word." Deciffusion's final readout works the same way. Single-token symbols like A, B, C are assigned to choices, and the probability of which symbol fills the answer blank is read. The difference is that this answer blank is not the "next word" at the end of the text, but rather a "single masked position," and the entire answer sheet attends to each other before filling that mask. A conventional LLM can only read left to right, but a diffusion model sees the situation, question, and choices all at once.

This determines two characteristics. The first is determinism. The ordering of choices is fixed by the serving side, with no random numbers. Since the same input always returns the same probabilities, Deciffusion measurements in this article were taken just once. The second is how speed is achieved. Because it reads forward and backward simultaneously, prefix caching — the mechanism that reuses previously read text — cannot be used. There's no shortcut like CLM's "second judgment in 1 ms"; it reads everything from scratch each time. Instead, up to 512 questions can be batched for a single situation, making it a design that pays off when there are many questions. I'll look at numbers for this in the speed section later.

On DGX Spark, It Ran with One Docker Command

Startup is a single docker run from the README. The only thing I changed locally was the memory limit. On my Spark, several Switchyard judge containers were already running, leaving about 31 GB free, so I lowered GPU_MEMORY_UTILIZATION to 0.18.

mkdir -p ~/.cache/deciffusion
docker run -d --name deciffusion --gpus all --shm-size=8g -p 127.0.0.1:8770:8770 \
  -v $HOME/.cache/deciffusion:/models \
  -e GPU_MEMORY_UTILIZATION=0.18 -e MAX_MODEL_LEN=16384 \
  ghcr.io/pst2154/nemotron-deciffusion:8b-v2
until curl -sf localhost:8770/health > /dev/null; do sleep 10; done

There's an arm64 image available; on first launch it downloads the 16 GB weights from Hugging Face and loads them while quantizing to FP8. After about 5 minutes from startup, /health returned 200. The resident model occupies 9.03 GiB.

One small but welcome discovery: the startup script already contained a branch for DGX Spark. When it detects GB10, it disables the startup auto-tuning (FlashInfer's autotune) and sets memory limits and context length to Spark-specific defaults. The comment explains that autotune pushed the GB10 to 86°C and caused it to freeze. The author had apparently also encountered the phenomenon of GB10 stopping from heat, which made me feel a slight kinship. My local log also shows "Skipping FlashInfer autotune because it is disabled," confirming this branch is in effect.

For the operational check, I submitted the README's example ticket using all three question types: choice for "which counter to route to," noul for "is it urgent," and score for "priority." With the situation and choices translated into Japanese, answers in the same direction were returned.

{
  "answers": {
    "route": {
      "type": "choice",
      "probabilities": { "billing": 0.9875, "shipping": 0.0063, "tech": 0.0063 },
      "confidence": 0.9875,
      "choice": "billing"
    },
    "urgent": { "type": "noul", "noul": 0.531 },
    "priority": { "type": "score", "probabilities": { "0": 0.096, "1": 0.38, "2": 0.488, "3": 0.035 }, "score": 1.463 }
  },
  "usage": { "input_tokens": 503, "decision_positions": 3 },
  "metrics": { "seconds": 0.489, "engine_seconds": 0.471, "questions": 3 }
}

choice returns per-choice probabilities and appends the highest as confidence. noul is the probability of "yes," and score is the expected value across the scale. In the Japanese version, route's probability was 0.9835, nearly matching the English version's 0.9875, while urgent shifted from 0.531 to 0.349. Since this is just one item, I can't say it makes the same judgment in Japanese generally. I'll simply note here: Japanese input passes through without breaking.

The Measuring Stick for This Article

Let me explain my local use case first. NeMo Switchyard is a mechanism that decides whether to route coding agent requests to a small, cheap model (weak) or a large, expensive model (strong). The role that makes this routing decision is the judge, currently handled by an LLM that writes JSON. What the judge receives is the request conversation and a card describing what the weak-side agent can do (available tools, permissions, execution environment). The question is a binary choice: "Route this request to weak or strong?" Here's a diagram of the judge's position:

The comparison method is the same as the CLM article from 9/27: I run the same 528 frozen inputs that the production judge received.

I'll define three scoring terms. A misroute is when a request requiring strong is sent to weak. Safe-side means a request that could go to weak is sent to strong — no harm done, but no savings either. AUC represents on a scale of 0.5 to 1 how well the model's "probability of being weak" rankings are correct, with values closer to 1 meaning it can better separate weak from strong by probability.

The rule for where to cut the probability to make it weak is fixed before looking at results, using the development 40 items and half of production. Candidates are thresholds like "if the top probability is 0.9 or higher, accept; otherwise send to strong as a precaution" — choosing the one with the fewest misroutes, then fewest safe-side. The recorded values for Jev and CLM are what were scored with this rule on 9/27.

Running the Same 528 Items as Jev Matched Jev

Here's the main part. I passed the same Japanese questions and choice descriptions as Jev. The rule selected was "accept at probability 0.9 or higher, otherwise strong" — the same rule chosen for Jev and CLM zero-shot. Only the CLM post-training column uses "weak at weak probability 0.7 or higher."

Numbers are in correct / misrouted / safe-side order. Deciffusion is a single measurement; Jev is a majority vote over 3 repetitions; CLM is single-run. Jev, CLM, and current judge values are from the 9/27 records.

Honestly, I didn't expect it to match Jev this closely. The 87 production items had 0 misroutes — the same as Jev — and safe-side was 6, fewer than Jev's 12. All 10 design consultations stayed on strong, and all 17 environment pairs had their probabilities assigned in the correct direction. Among the 237 unseen items, 10 misroutes matched Jev exactly, and safe-side was 57, fewer than Jev's 71. AUC was 0.858 on 237 unseen items and 0.902 overall — virtually the same curve as Jev's 0.856 and 0.901.

At the same time, Jev's weaknesses were inherited intact. The 14 discriminating questions are hard ones that look like implementation tasks on the surface but are beyond what weak can handle. Deciffusion sent 10 of them to weak with probability 0.9 or higher. The median weak probability was 0.932 — it wasn't even hesitating. Jev had 12, the current judge had 3, CLM zero-shot had 0. General-purpose decision models read "whether the request looks like an implementation task" but not "whether this implementation exceeds this agent's capabilities." This is the same reading as in the 9/23 article.

One more number came out in an unexpected direction. The 30 implementation tasks have a version with a card section describing the execution environment (noting that read_file and run_tests are available) and a version without it. Adding the section should make it easier to route to weak — that's the expected direction — and Jev confirmed it: 19 with section, 9 without. Deciffusion showed 28 without section and 20 with section — adding the section reduced weak by 8 items. Those 8 aren't misroutes; they're items where the probability fell below 0.9 and were sent to strong as a precaution. It can read the difference of one environment line (17/17 on environment pairs), yet adding the section itself reduces confidence. The reason for this is not clear from this round of measurements.

Incidentally, while writing this article, I noticed that Jev 1.14.0 had been released the previous day, on 10/14. Since I didn't want the comparison target to be stale, I also ran the same 528 items through typesafe/jev-1.14 in 3 repetitions: 1,584 calls via OpenRouter for $0.09. Following the same procedure to determine the rule, 1.14 selected "weak at weak probability 0.5 or higher," different from 1.13's "accept at probability 0.9 or higher." So I'm reading it both ways.

Jev 1.14's AUC was 0.847 on 237 unseen items and 0.902 overall — barely moving from 1.13's 0.856 and 0.901. Environment pairs remained 17/17. With its own rule, safe-side dropped sharply but unseen misroutes increased to 16, and all 14 discriminating questions were sent to weak. With the same rule as 1.13, discriminating questions were 7/14 and unseen misroutes were 4, but safe-side grew to 85. Under either reading, Deciffusion was at the same level as Jev 1.14, and the weakness of sending the 14 discriminating questions to weak was shared by all three.

Only the Threshold Side Moved with Different Question Wording

In the CLM article, zero-shot results shifted significantly with different question wording. I verified the same with Deciffusion. I added two variants: an English question (en) and an English sentence with model names written in (model-names), with rules determined separately for each.

AUC stayed within a band of 0.89 to 0.90 overall regardless of question wording. The probability rankings don't shift much with question wording; what shifts is the absolute value of probabilities — i.e., where to cut the threshold. The English question leaned strongly toward strong overall, sending 49 of the 56 production items that should be weak to strong. 12 of the 14 discriminating questions stayed on strong, but this wasn't the model choosing strong — the probability didn't reach 0.9 and they were sent to strong as a precaution. A judge that sends the majority of weak-eligible requests to strong doesn't suit the goal of saving costs. model-names leaned the opposite way toward weak, routing almost all implementation tasks to weak, but unseen 237-item misroutes rose to 21 — the highest.

Japanese input: the production 15 items and all 126 Japanese items among the 237 unseen items passed through normally. For production 87-item AUC and overall AUC, Japanese questions performed best. That's all I can say as a fact — why Japanese questions worked best on a model whose model card doesn't mention Japanese is not clear from this round of measurements.

Single Judgment 138 ms, but Concurrency 8 Was Problematic

Speed was measured on Spark with the calling side also on the same machine. Since server-side processing time is included in the response, I recorded both. The input was the 87 production items (median 961 tokens, max 1,803 tokens).

At concurrency 1, p50 was 137.6 ms, the same value on the second pass. Since there's no prefix reuse, there's no shortcut like CLM's "second judgment in 1 ms." In return, the probabilities from both passes matched exactly across all 87 items, confirming deterministic behavior. Compared to the 0.25 seconds of the 9/23 single-token judgment, it sits on the shorter side within the same order of magnitude. Jev's 0.25 seconds includes network latency, so I won't put them side by side. The claimed GB200 figure of 24.1 ms uses ~362 token inputs on a different GPU, so that's also a different playing field.

The surprise was concurrency 8. Per-item time stretched to 2.7 seconds, and overall throughput dropped to 2.96 items per second — below the 5.43 items per second when processing sequentially at concurrency 1. Concurrent requests were slower than sequential ones — a reversal. My guess is that the bidirectional reading design can't split prompts into small chunks, making a single batch of eight ~1,000-token prompts too heavy for the GB10, but I haven't isolated the cause. Since my local judge makes one call per session the practical impact is small, but for use cases that send many judgments concurrently, you can't expect to match the claimed figures.

Bundling 3 questions for one situation took 336.6 ms — 2.4 times the cost of 1 question. That's less than the 3× cost of sending them separately, so the bundling benefit holds on Spark too. Increasing choices from 2 to 100 stretched time from 208 ms to 1,434 ms, since the choice descriptions go directly into the prompt and longer input means longer processing. CLM reused candidate embeddings to return 1,000 candidates in 110 ms, so the takeaway is that CLM's design suits cases with many candidates, while Deciffusion's design suits cases with many questions.

What to Entrust from Switchyard to a General-Purpose Decision Model

Here I'll line up the judges from the past month in the same format as the 9/23 article.

Reading against the adoption criteria set in the 9/23 article: it meets the condition of keeping all 10 design consultations on strong; it fails the condition of keeping 9 or more of the 14 discriminating questions on strong (only 4); the condition that adding an environment section doesn't increase weak is violated in the wrong direction; and it meets the condition that all items return normally and judgment is within 0.35 seconds. Since it stalls at the same places as Jev, the conclusion for local judge use is the same as Jev. A general-purpose decision model unfamiliar with the card can read the form of the request but doesn't know what this particular agent struggles with.

That said, among the judges I've looked at over this past month, Deciffusion's position is a little different. Jev is an external API, and CLM wasn't usable zero-shot. Deciffusion has weights locally, achieves scores matching Jev without training, and inference runs in 138 ms on a single Spark. The repository has a finetune/ directory, stating LoRA starts from a single 80 GB GPU and full fine-tuning from 8. I have 4,511 judgment data points from the 9/23 and 9/27 articles. Whether those can move the 14 discriminating questions is a harder assignment than CLM training, but I think it's one worth attempting.

For use cases where general judgment is sufficient — say, tool selection, escalation decisions, triage — cases where "not knowing the card is fine" — the numbers from this run apply directly. The ability to bundle questions for one situation, the fact that probabilities return deterministically, and the fact that Japanese passes through as-is are all convenient properties for components of a router.

Summary

I ran Nemotron Deciffusion 8B v2 on DGX Spark and measured it as a judge role on the same 528 items as Jev. Startup was a single docker command with a GB10-specific branch already built in: 9 GiB in FP8, 138 ms per judgment. Zero-shot performance with the same Japanese questions as Jev matched the level of Jev 1.13: 0 misroutes on 87 production items, 10/10 design consultations, 10 misroutes and AUC 0.858 on 237 unseen items. Re-running the same 528 items with Jev 1.14, which was released the day before, didn't change this reading. The biggest takeaway this time was discovering that diffusion models have this use case too. Just narrowing the blank-filling down to a single answer position turns it into a readout interface for non-generative judgment. The weights are local, it gets this far without training, and the same input always returns the same probability. The properties I wanted in a router component were all there from the start.

On the other hand, the shortcomings as a local judge were the same as Jev's. It routed 10 of the 14 discriminating questions to weak, and adding an execution environment section reduced weak routing. I believe a general-purpose model unfamiliar with the card being unable to read this particular agent's limits comes from the shape of the training data. Only the number tripped by the threshold moved with different question wording. This was a Day 0 measurement — single run — and while the claimed values are BF16, my local run used FP8. I also haven't tried starting the NVFP4 version.

Next, I'd like to train on local judgment data using finetune/ and see whether the 14 discriminating questions move. Since LoRA requires a single 80 GB GPU, I'm planning to cover in a follow-up whether that fits within Spark's 128 GB.

Reference Links

• Nemotron Deciffusion 8B Public Page
• pst2154/nemotron-deciffusion (GitHub) — Distribution code, finetune/, release/v2/SOURCES.md
• nemotron-deciffusion/Nemotron-Deciffusion-8B-v2 (Hugging Face)
• nemotron-deciffusion/Nemotron-Deciffusion-8B-v2-NVFP4 (Hugging Face)
• nvidia/Nemotron-Labs-Diffusion-8B (Hugging Face) — Base model
• vllm-project/vllm#59299 — General-purpose endpoint for /v1/systemone (merged 2026-10-08)
• vllm-project/vllm#60283 — Diffusion decision reads for Deciffusion (not yet merged)
• Jev: For the builders AND dreamers (TypeSafe AI Blog) — Jev 1.14.0 release announcement (2026-10-14)
• Models (TypeSafe docs) — Targets for jev-latest / jev-preview
• Reading OpenJev and Thinking About Alternative Approaches to Non-Generative AI (2026-09-18)
• Turning the Judge Role in Nemotron 3.5 Lightning into a Non-Generative Single-Token Decision Like Jev (2026-09-23)
• Trying a Non-Generative Decision Model CLM as the Judge Role in Switchyard on DGX Spark (2026-09-27)
• Running Nemotron-Labs-Diffusion on DGX Spark and Benchmarking Tri-Mode Generation (2026-05-21)