
I Tried Running "Inkling 2 Mini," a 26B MoE That Passes Through Its Layers 3 Times, on a DGX Spark
This page has been translated by machine translation. View original
Introduction
Hello, I'm Mori from the Manufacturing Business Technology Department at Classmethod.
Thinking Machines Lab released Inkling 2 Mini on November 20. It is a 26.4B multimodal model licensed under Apache 2.0. The model card lists "1x NVIDIA DGX Spark" under supported hardware, and vLLM provides a recipe for DGX Spark.
What caught my attention was the structure. Of its 24 layers, the 12 in the middle are passed through 3 times with the same weights. The distributed weights are about 20GB, much smaller than the 120B Nemotron 3.5 Super VL I measured last month, and about the same as the Qwen3.8-27B I normally use on DGX Spark. However, unlike the dense Qwen3.8-27B, this one is an MoE, so far fewer weights are used for computation. The model card names these two as comparison targets, so I decided to line them up on the same tasks I have at hand.
To state the conclusion first: the recipe worked as written, started in 380 seconds with a 262K context, and reached 37.83 tok/s single-stream. On the 10 Japanese tasks, the automated judgment stays at 9/9 even when I lower the reasoning effort to low. On the other hand, the task of choosing the correct one out of 40 tools passed only 1 of 6 questions, and even with tool_choice: required specified, there were 0 tool calls.
Here is the article where I measured Super VL (as of 2026-10-18).
This article presents the results of running Inkling 2 Mini, the day after its release, on vLLM on DGX Spark. I check whether the recipe works as is, how fast it is, and measure Japanese, code, tools, and images on the same tasks as Qwen3.8-27B and Super VL.
What Inkling 2 Mini Is
First, let me cover the primary information. The numbers are taken from the model card, config.json, and chat_template.jinja.
| Item | Value |
|---|---|
| Provider | Thinking Machines Lab (released 2026-11-20) |
| Parameters | 26.4B (3.4B active) |
| Architecture | 24-layer decoder-only, 8 of 128 experts used per MoE layer, 23-layer ViT for images |
| Loop | The 12 middle layers are passed through 3 times with the same weights. 48 layers' worth are applied per token |
| Quantization | Mix of NVFP4 and MXFP8 (the distributed weights are only this one kind) |
| Context | 1,048,576 tokens |
| Input | Text, images |
| Supported languages | English and multilingual in general |
| License | Apache 2.0 (there is also a separate Model Acceptable Use Policy) |
| Recommended sampling | No dedicated section. generation_config.json has temperature 1.0 |
| Distribution | 19.5GB main body, and 636MB mtp.safetensors for the speculative decoding draft DSpark |
| Intended deployment | 1× DGX Spark, Apple silicon with 32GB or more, GPUs with 32GB or more VRAM |
The model card calls itself a looped transformer. Since it is an MoE, the weights used per token (active parameters) are 3.4B. However, because the 12 middle layers are passed through 3 times, the card explains that each loop adds about 1.0B of effective parameters, reaching an effective 5.4B after 3 loops. It is a design that adds only computational depth without adding weights.

The top shows the flow for producing one token: only the middle 12 of the 24 layers (blue) are passed through 3 times with the same weights. The first and last 6 layers are passed through only once. The bottom shows the length with the loop unrolled: the weights stay at 24 layers' worth, but the layers traversed amount to 6 + 12 × 3 + 6 = 48 layers.
Reasoning effort is specified with reasoning_effort. It can be passed either as a name or as a number from 0.0 to 0.99, and inside the template it is converted to a number.
| Name | Value | Note |
|---|---|---|
| none | 0.0 | |
| minimal | 0.1 | |
| low | 0.2 | The weak side in this article |
| medium | 0.7 | |
| high | 0.9 | Default when unspecified |
| xhigh, max | 0.99 |
The model card has a note saying "even with reasoning effort at minimum, thinking does not stop entirely." There is no switch to turn thinking off; it is a model that only reduces the amount.
As for language, the card's limitations section says, "Output quality may degrade in languages other than English. In Chinese tests, we observed truncated-sounding phrasing and mixing of languages." I will check how Japanese comes out on my own machine.
Getting It Running
The vLLM recipe says "vLLM 0.35.0 or later is required (unreleased)" and instructs using the dedicated image vllm/vllm-openai:inkling-2-mini until support lands in a release. As of November 21, the latest release, v0.34.0, does not yet include the definition for this model. The dedicated image has an arm64 build, and the vLLM inside was 0.1.dev25855+g7cb5dae22.
I downloaded the weights locally with hf download and passed the recipe's DGX Spark arguments as is.
docker run -d --name inkling --gpus all --ipc=host \
-p 127.0.0.1:8000:8000 \
-v "$MODEL_DIR":/model:ro \
vllm/vllm-openai:inkling-2-mini /model \
--served-model-name inkling-2-mini \
--tensor-parallel-size 1 \
--max-model-len 262144 \
--reasoning-parser inkling \
--enable-auto-tool-choice \
--tool-call-parser inkling \
--speculative-config '{"method":"dspark","num_speculative_tokens":7,"draft_sample_method":"probabilistic"}'
The differences from the recipe are that the weights are read from a local directory, that I added --served-model-name, and that I removed --privileged from the recipe's docker line. It started even without --privileged.
The context length is held to 262,144 rather than the model's upper limit of 1M. The recipe explains that this value is used on DGX Spark and H100 to fit in memory.
The result: it started without changing anything.
| Item | Value |
|---|---|
| Startup | 380 seconds (102 seconds for loading weights) |
| KV cache | 1,550,098 tokens' worth. Works out to holding 5.91 requests of 262,144 |
| Free host memory | 113.2 GiB before startup, 9.1 GiB after |
| Temperature during measurement | GPU peak 61 °C, hottest zone in /sys/class/thermal at 72.7 °C |
Even though the weights are only 20GB, the free memory after startup drops to 9.1 GiB. This is because the recipe has no --gpu-memory-utilization setting, so vLLM reserves memory at its default ratio and devotes the rest to the KV cache. If you want to run it alongside other processes, you need to lower --gpu-memory-utilization yourself.
The speculative decoding DSpark uses mtp.safetensors, which is in the same repository as the main model. You do not need to download an external draft separately. The startup log shows Resolved architecture: Inkling2DSparkModel and SpeculativeConfig(method='dspark', model='/model', num_spec_tokens=7). There was also one more warning line saying that the draft does not receive image embeddings and looks ahead using text only.
Measurement Conditions
The first row of the speed section was measured with the recipe launched verbatim. The rest use an 8-sequence launch modeled on past measurements: --max-model-len 131072 --max-num-seqs 8 --max-num-batched-tokens 8192 --gpu-memory-utilization 0.75, no speculation, same image. This one started in 265 seconds with a KV cache of 485,332 tokens' worth, and 19.0 GiB free after startup.
Sampling uses temperature 1.0 from generation_config.json, and the budget (upper limit on output tokens) is 8,192. No task was cut off by this budget. I measured at two reasoning effort levels: the default high, and low on the weaker side.
The comparison targets are Qwen3.8-27B (NVFP4) and Nemotron 3.5 Super VL 120B-A12B (NVFP4), which I measured earlier on the same tasks. Both were measured on vLLM on DGX Spark, but the launch settings are not aligned. Qwen3.8-27B used xhigh reasoning effort, a context length of 262,144, and --gpu-memory-utilization of 0.85, while Super VL used an fp8 KV cache. Only the Inkling 2 Mini side was adjusted to match past measurements, so the tables place these as measured values that include these differences.
Speed
These are medians of 3 runs with a forced length of 256 tokens at temperature 0.
| Configuration | Single-stream tok/s | TTFT (s) |
|---|---|---|
| Recipe verbatim (262,144, DSpark lookahead of 7 tokens) | 37.83 | 0.212 |
| 8-sequence launch (131,072, no speculation) | 39.43 | 0.164 |
The recipe verbatim gives 37.83 tok/s single-stream. The 3 values range from 37.63 to 37.87, with almost no fluctuation.
The 8-sequence launch without speculation gave 39.43 tok/s. The context length and memory settings differ between these two rows, so please do not read the difference as the effect of DSpark.
The acceptance rate (the fraction of lookahead candidates that the main model adopted) can be read from the /metrics diff. In this speed measurement interval, 402 of 2,541 lookahead tokens were adopted, 16%. In the interval right after startup, where I ran Japanese questions and tool calls as a check, it was 935 of 3,549 tokens, 26%. The recipe also says that the acceptance rate depends on the prompt and temperature.
The measurements with varying concurrency use the 8-sequence launch. The single-stream value is the median of 3 runs, and the values for 2 or more streams are one measurement each.
| Concurrency | tok/s per stream | Total tok/s | TTFT (s) |
|---|---|---|---|
| 1 | 39.43 | 39.43 | 0.164 |
| 2 | 31.85 | 61.7 | 0.247 |
| 4 | 25.99 | 86.92 | 1.928 |
| 8 | 27.27 | 133.35 | 5.968 |
With 8 concurrent streams the total is 133.35 tok/s. What concerns me is the time to first response (TTFT), which stretches to 1.9 seconds with 4 streams and 6.0 seconds with 8. If you share it among several people for interactive use, this would likely matter before the total throughput does.
Let me line it up against the two measured the same way.
| Model | Single-stream tok/s | Total tok/s at 8 concurrent | Per stream |
|---|---|---|---|
| Inkling 2 Mini | 39.43 | 133.35 | 27.27 |
| Qwen3.8-27B | 12.37 | 85.86 | 10.93 |
| Nemotron 3.5 Super VL 120B-A12B | 19.18 | 79.32 | 10.12 |

The top is single-stream (forced length of 256 tokens, median of 3 runs) and the bottom is the total at 8 concurrent (one measurement); the horizontal axis scales differ between top and bottom. The recipe verbatim uses a context length of 262,144 with DSpark, and the 8-sequence launch uses 131,072 with no speculation. The two gray models are values measured earlier on the same tasks without speculation, and their context length and KV cache settings are not aligned.
All three are values without speculation. For Qwen3.8-27B as well, I aligned on the row with the built-in MTP turned off. Single-stream is 3.2 times Qwen3.8-27B and 2.1 times Super VL. Since the launch settings are not aligned, the ratios include differences in configuration. Even so, a model that passes through the same layers 3 times running at nearly 40 tok/s single-stream was faster than I expected.
Japanese
I threw 10 Japanese tasks at it. Judgment is scored by conditions that a machine can verify, such as character count, presence of required words, and whether the JSON parses.
| Task | Inkling 2 Mini (high) | Tokens used | Time (s) |
|---|---|---|---|
| Answer within 50 characters | ○ (37 characters) | 753 | 21.2 |
| Free-form writing | ○ | 290 | 9.1 |
| Code generation | ○ | 165 | 5.9 |
| Numerical reasoning | ○ | 430 | 12.8 |
| Vocabulary constraint | ○ (166 characters) | 808 | 21.2 |
| JSON formatting | ○ | 346 | 12.5 |
| Summary with proper nouns | ○ (6/6 words) | 7,737 | 212.3 |
| Transcribing a chart | ○ (5/5) | 2,719 | 74.2 |
| Rephrasing into honorific language | ○ (5/5) | 703 | 21.0 |
| Extraction from long text | ○ (4/4) | 540 | 23.0 |
| Total of automated judgment | 9/9 | 14,491 | 413.1 |
Free-form writing cannot be scored by a machine, so I excluded it from the denominator of 9/9. Tokens used and time are totals across all 10 tasks.
The automated judgment is 9/9. On the same tasks with the same output token budget of 8,192, Qwen3.8-27B at xhigh and Super VL at high both scored 8/9.
| Model (reasoning effort) | Automated judgment | Total tokens used | Time for 10 tasks |
|---|---|---|---|
| Inkling 2 Mini (high) | 9/9 | 14,491 | 413.1 s |
| Inkling 2 Mini (low) | 9/9 | 4,095 | 110.2 s |
| Qwen3.8-27B (xhigh) | 8/9 | 26,205 | 2,168.6 s |
| Qwen3.8-27B (low) | 8/9 | 5,083 | 419.7 s |
| Nemotron 3.5 Super VL 120B-A12B (high) | 8/9 | 13,932 | 736.5 s |
The comparison rows need a note. The one question missed by Qwen3.8-27B at xhigh and Super VL at high reached the output token budget of 8,192 and produced no body text. Super VL scored 9/9 in the row where thinking was set to medium. This table alone cannot decide which is better at Japanese. All that can be said is that Inkling 2 Mini, at the default high, returned body text for all 10 tasks within the same 8,192.
The tightest task within that budget was the summary with proper nouns, which used 7,737 tokens. It thought for 11,858 characters and then returned a 150-character summary that kept all 6 words. This is a task where the three Nemotron 3.5 models all used up the whole budget at high, and Inkling 2 Mini too had only 455 tokens left before the limit.
The thinking was in English for every task. It parses the Japanese question in English and writes the Japanese body text at the end. For free-form writing, it returned two numbered merits and two demerits of remote work, and reading it with my own eyes I saw no unnatural phrasing or language mixing.
Having It Write Code
This is a two-part set: 5 problems written from scratch from Japanese specifications, and a task of fixing a mini repository containing bugs. Grading is done by running pytest on my side, without using the model's self-reports.
Writing from Scratch from Specifications
| Problem | Inkling 2 Mini | Time (s) | Qwen3.8-27B | Super VL |
|---|---|---|---|---|
| slugify | ○ 11/11 | 20.8 | ○ 11/11 | ○ 11/11 |
| parse_duration | ○ 16/16 | 40.4 | ○ 16/16 | ○ 16/16 |
| merge_intervals | ○ 8/8 | 17.4 | ○ 8/8 | ○ 8/8 |
| group_by_month | ○ 6/6 | 14.2 | ○ 6/6 | ○ 6/6 |
| top_k_frequent | ○ 8/8 | 15.6 | ○ 8/8 | ○ 8/8 |
| Total | 5/5 | 5/5 | 5/5 |
All 49 tests across the 5 problems pass, and the result is the same for all three. This task did not separate them. Inkling 2 Mini returned even the longest one, parse_duration, in 40 seconds.
Fixing Existing Bugs
This task hands over a mini repository in which 5 of 12 tests fail, and has the model fix it using only file read/write tools.
| Model | Fixed | Turns | Tool calls | Time (s) |
|---|---|---|---|---|
| Inkling 2 Mini | 5/5 | 8 | 14 | 60.0 |
| Qwen3.8-27B | 5/5 | 5 | 11 | 119.4 |
| Super VL | 5/5 | 6 | 14 | 99.7 |
All three get to all 12 tests passing. Inkling 2 Mini has the most turns at 8, but the shortest time at 60.0 seconds, and had 0 broken calls (outputs that could not be parsed). It often called multiple tools together in one turn, putting 2 to 3 calls, such as running tests and reading files, in a single message.
The model card cautions that thinking can get stuck in a long loop when debugging code. In this task it finished in 8 turns and no loop occurred.
Can It Be Used as a Tool
I look at whether it can be used as a component of an agent. Inkling 2 Mini's tool calls use a proprietary format, and on the vLLM side --tool-call-parser inkling converts them to OpenAI-compatible JSON.
One thing that concerned me was how arguments are handled. This model's template raises an exception if the arguments of past tool calls remain JSON strings. OpenAI-compatible APIs return arguments as strings, so I was worried that stacking returned calls into the history as is would make it fail. In practice, it round-tripped even when sent back as strings. The server side fixes it, so the client can write things as usual.
All 5 basic call questions passed. A task that hands over a pseudo file system and has it hunt for a bug was also solved in 4 turns and 6 tool calls, with 0 broken calls. With streaming too, both tool calls and thinking come through in the chunks.
required Gives 0 Calls, and Queries for Two Places Are Combined in One Message
tool_choice: required is accepted but was not honored. When I passed one weather tool with the input "Hello. Please don't talk about the weather." and specified required, it thought for 12,393 characters over 82.4 seconds and then replied with a greeting in the body without calling the tool. There is no error; a response with 0 calls is returned. Qwen3.8-27B and Super VL called once and stopped on the same input. A client that assumes required needs a branch for the 0-call case.
For a query about two places such as "Tell me the weather in Tokyo and Osaka," it returned the 2 calls combined in a single message. This is the same as the two comparison models.
Suitability as an Agent Component
I check with 6 probes whether it can be loaded into harnesses such as opencode or Hermes Agent. I should state up front that what is measured here are the capabilities each harness depends on, not results of running the harnesses themselves.
| Probe | high | low |
|---|---|---|
| Correctly choose from 40 tools | 1/6 | 2/6 |
| Rule compliance in a long system prompt | ○ | × |
| Reproduce a fixed format (5 times) | ○ 5/5 | ○ 5/5 |
| Fixing the response language | ○ (3/3) | ○ (3/3) |
| Suppressing excessive tool calls | ○ (2/2) | ○ (2/2) |
| Multi-turn round trips | ○ | ○ |
At high it passed 5 of 6, and the per-harness verdicts were "usable" for Hermes Agent and "difficult" for opencode and OpenClaw. This is the same verdict as Super VL at high.
The one it failed is the task of choosing from 40 tools, where at high only 1 of 6 questions passed. The way it misses has a pattern. For "append one line to this file," it called file search rather than the append tool, and for "set this up to run every morning at 9," it called a directory listing and a wiki search. It does not fail to call tools; it starts with research before going to the target tool. The probe looks at the first move, so the behavior of inserting research first is counted as incorrect as is.
On this task, with the same output token budget of 8,192, Qwen3.8-27B at xhigh and Super VL at high also failed with 4/6. Still, Inkling 2 Mini's 1/6 is even lower than that. Super VL scored 6/6 in the row with thinking at medium and in the row with the limit widened to 16,384. If you plan to load it into a harness that stacks 40 tools, I would check it in practice first.
Images
Inkling 2 Mini is a model that can also read images, so I ran the same 3 probes. The output token budget is 4,096 and thinking is high.
| Probe | Inkling 2 Mini | Qwen3.8-27B | Super VL |
|---|---|---|---|
| Listing protective gear (picture of 2 workers) | ○ | × (false positive in judgment) | ○ |
| Reading values from a bar chart | ○ (4/4) | ○ (4/4) | ○ (4/4) |
| OCR of an inspection record | ○ (CER 0.0) | ○ (CER 0.0) | ○ (CER 0.0) |
Here are two of the images I actually passed in, along with the answers returned. Both images are synthetic images I drew for verification.

The top is the image passed in, and the bottom is the answer from Inkling 2 Mini (high). On the left, for the question "list all protective gear worn by the people shown, and do not list anything that is not shown," it listed the helmets and reflective vests along with their colors. On the right, for the question "read out the number attached to each bar from left to right," it read all 4 correctly.
All three passed, and hallucinations (listing things not shown) were 0. In listing protective gear, it added colors, saying the helmets were "white on the left, yellow on the right" and the vests "orange and white stripes," and then answered that other protective gear is not shown. Qwen3.8-27B has × on the automated judgment, but reading its answer, it listed only helmets and safety vests. The scoring script counted as a hallucination the single sentence that disclaimed, "I do not list gloves, goggles, and so on because they cannot be identified," a false positive; judged by eye, all 3 models pass. Note that only the Qwen3.8-27B row had an output token budget of 2,048, so the conditions are not aligned.
For the OCR of the inspection record, it transcribed the date, equipment number AZ-4471, 7.5 mm/s, the judgment, and the inspector without dropping a single character. Time across the 3 probes was 5.7 to 30.9 seconds.
What Changes When You Vary the Amount of Thinking
I first checked whether reasoning_effort takes effect. The template turns the specification into a one-line system message like Thinking effort level: 0.9 and inserts it at the start of the conversation. When I threw the same question at none and high, the input token counts differed, 94 versus 96, so the specification reaches the template.
Here I got stuck on one point. For a question solvable in one step, like "calculate 17 × 23," the same 93 tokens come back at both none and high. When I switched to a word problem requiring many steps, in 3 measurements none split into 426 to 488 tokens and high into 811 to 914 tokens. Since simple questions show no difference, you need to choose the task when checking whether it takes effect. As the card's note says, even at none, 1,046 to 1,311 characters' worth of thinking remained.
Here are high and low side by side on the same tasks.
| Item | high | low |
|---|---|---|
| Japanese automated judgment | 9/9 | 9/9 |
| Tokens used across 10 Japanese tasks | 14,491 | 4,095 |
| Time for 10 Japanese tasks | 413.1 s | 110.2 s |
| Tokens used for summary with proper nouns | 7,737 | 948 |
| Correctly choose from 40 tools | 1/6 | 2/6 |
| Rule compliance in a long system prompt | ○ | × |
| Harnesses judged "usable" | Hermes Agent | None |

Totals from running the 10 Japanese tasks at high (default, 0.9) and low (0.2). Output tokens are the sum of thinking and body text. The automated judgment (9 tasks excluding free-form writing) stays at 9/9 for both high and low, while output tokens go from 14,491 to 4,095 and time from 413.1 seconds to 110.2 seconds. The output token budget is 8,192, one measurement each.
On the Japanese tasks, even lowering to low, it stays at 9/9 while the time goes from 413.1 seconds to 110.2 seconds. The biggest difference is in the summary with proper nouns: 7,737 tokens at high and 948 tokens at low. Both returned a summary keeping all 6 words, and with the same judgment the output tokens are down 88%. However, against the "within 150 characters" instruction, high packed it into exactly 150 characters, while low came to 182 characters, helped by the tolerance cap of 200 characters. The long thinking is being used to squeeze down the character count.
What failed at low was the long system prompt task. It followed the rules of answering in Japanese and not using bullet points, but dropped "put a one-line summary at the end." This also changes the Hermes Agent verdict to "difficult." The 40-tool choice task passed one more question at low, but a 1-question difference out of 6 in a single measurement; I do not read it as a real difference.
Within the range of these tasks, dialogue and text formatting had no change in verdict at low, while a system prompt stacked with long rules lost one rule unless at high. There is only one kind of system prompt task, so when you try low, it seems best to check with your own rules. Since this is a model whose thinking cannot be turned off, this setting is the first thing to touch when you want to shorten the wait.
One more thing, about reproducibility. When I fixed the seed at temperature 0 and threw the same question twice, the outputs split into 467 tokens and 488 tokens (with DSpark, one observation). The recipe's DSpark has draft_sample_method set to probabilistic, so that may be the cause. For uses where you want to compare with fixed outputs, it seems wise to check beforehand.
Summary
Inkling 2 Mini ran with the vLLM recipe for DGX Spark exactly as written, and started in 380 seconds with a 262,144 context. Single-stream is 37.83 tok/s with the recipe verbatim, 39.43 tok/s with the 8-sequence launch, and the 8-concurrent total is 133.35 tok/s. Qwen3.8-27B measured the same way was 12.37 tok/s single-stream and Super VL was 19.18 tok/s.
On quality: Japanese automated judgment 9/9, code 5 problems with all 49 tests passing, bug fixing 5/5, tool calls 5/5, and all 3 image probes passed. Japanese stays at 9/9 even with thinking lowered to low, and the time for the 10 tasks went from 413.1 seconds to 110.2 seconds. I had braced myself because the card noted quality degradation outside English, but in this set of tasks I saw no breakdown in Japanese.
The weak points are also clear. The 40-tool choice task is 1/6 at high, and tool_choice: required is accepted but gave 0 calls. At low it dropped one rule in the long system prompt, and at 8 concurrent the time to first response is 6.0 seconds. Note that the effect of DSpark alone and the 1M context are outside the scope of this article.
It is a model with 20GB of weights that reaches nearly 40 tok/s single-stream on a single DGX Spark and handles Japanese and images. Personally, I think the best entry point that makes the most of this model's speed is to first run dialogue and text formatting at low.
References
- thinkingmachines/Inkling-2-Mini — Model card. See here for the structure, supported hardware, and limitations
- Introduction article on the Hugging Face blog
- vLLM recipe (Inkling 2 Mini) — Launch command for DGX Spark and notes on DSpark
- vLLM support PR (#69028) (unmerged as of 2026-11-21)
- Model Acceptable Use Policy
- Article measuring Nemotron 3.5 Super VL 120B-A12B on DGX Spark — See here for descriptions of the tasks used for comparison
- Article comparing 4 configurations of Qwen3.8 and GLM-5.3-Flash on DGX Spark by use case — The Qwen3.8-27B comparison values come from measurement data from that time (the 85.86 tok/s at 8 concurrent is the row with the built-in MTP turned off)
