I Reconciled Document Values with Gemini Decision 1 and Compared It to the Jev Decisions API

Hello, I'm Keima.

Checking whether values match has long been a struggle for engineers, who have fought with regular expressions and scripts only to be troubled by notation variations and exceptions.
On the other hand, entrusting it to a general-purpose LLM allows flexible judgment, but processing takes time and API call costs add up.
To address these issues, decision models, which do not generate text and return only a judgment and a probability value, are attracting attention as an option that solves both speed and cost.

However, among the decision models I have tested so far, some output probabilities that stick to extremes of 0 or 1, with almost nothing in between. I often found myself wondering, "Is this probability really appropriate?" Among those I tried before, TypeSafe's Jev scattered probabilities reasonably well between 0 and 1, and I could tell the cases where it hesitated by their low probabilities (I tried matching document values with OpenAI's Decisions API and compared it with Jev and Clef | DevelopersIO).

So in this article, I used "Gemini Decision 1", newly released by Google on November 11, 2026, and investigated how it behaves on the same cross-matching task (the test date was November 13, 2026).
Beyond the number of correct answers, I also focus on whether the output probabilities stick to extremes of 0 or 1, compared with Jev and the Decisions API.

1. Test method

1.1 Data

I prepared 107 pairs, each consisting of two values of a document field.
I deliberately mixed in cases that are easy to get wrong, such as the following.

• Differences in date notation (Sep 12, 2026 and 12/09/2026)

• Differences in number separators (1.250,00 and 1,250.00)

• Differences between Japanese and English notation of company names and addresses

• OCR misreads (the O in 1,25O.00)

• Differences in how a missing value is expressed (blank, N/A, Not applicable)

• Matching a total against line items (Total 500 KG and Details: 500 KG / 500 KG)

For each pair, I asked the following two questions in Japanese.

• Exact match: Are value A and value B identical as strings?

• Semantic match: Even if the notation differs, do value A and value B indicate the same value?

1.2 Request

I sent the same values and the same question text to the three models.
Because Gemini Decision 1 does not accept criteria (the conditions for yes and no) on noul questions, I appended the conditions to the end of the question text (the same treatment applies to the Decisions API).

{
  "model": "gemini-decision-1",
  "input": "{\"値A\": \"1,250.00\", \"値B\": \"1250\"}",
  "questions": {
    "exact": {
      "type": "noul",
      "instructions": "値Aと値Bは、文字列として完全に同一か。空白・記号・大文字小文字・全角半角の違いも「同一ではない」と扱う。\n「はい」となる条件：1文字も違わず完全に同じ\n「いいえ」となる条件：1文字でも違う"
    },
    "semantic": {
      "type": "noul",
      "instructions": "値Aと値Bは、表記が違っていても同じ値を指しているか。これは2つの書類の同じ項目を突合している場面での判断である。片方が明細の羅列で、もう片方が合計値の場合は、明細をすべて足し合わせてから合計値と比べること。\n「はい」となる条件：書式・言語・単位・略記の違いだけで、指している値は同じ。または明細の合計が合計値と一致する\n「いいえ」となる条件：指している値そのものが違う、または片方にしか値がない。または明細の合計が合計値と一致しない"
    }
  }
}

For Jev, I sent the same content using state and criteria, and for the Decisions API, using the predicate type.
In the test, the three models were called in sequence for each case.

1.3 Metrics

• Correct answers: the number of cases where a yes probability of 0.8 or higher was judged "same" and matched the correct answer

• Slip-through: the number of cases judged "same" even though the values are actually different. This is the error to avoid most in cross-matching

• Number of cases with probability exactly 0 or 1: the more there are, the harder it is to distinguish hesitant cases by probability

• Cost: converted at 150 yen per dollar

2. Results

2.1 Correct answers, speed, and cost

For semantic matching, Jev had the highest number of correct answers, with only 1 slip-through.
Although Gemini Decision 1 had the fewest input tokens, its unit price is about 2.4 times that of Jev, so its cost came out slightly higher than Jev's.

2.2 Probability distribution

I plotted the semantic match probabilities, separated into 69 cases whose correct answer is "same" and 37 cases whose correct answer is "different". This is what I care about most. When I tested the Decisions API in the past, its probabilities stuck to 0 and 1.
Each dot represents one value pair, and the dashed line is the 0.8 threshold.

Semantic match probabilities (Japanese question text, 106 cases)

Wow, this is the first decision model since Jev to produce plausible-looking probabilities!
They don't stick to 0 or 1.
There is also a decent middle range.

2.3 When changing the threshold

When I varied the threshold, the number of correct semantic matches and the number of slip-throughs changed as follows (correct / slip-through).

For Gemini Decision 1, raising the threshold to 0.9 reduced slip-throughs to 1 and also improved the number of correct answers to 95.

2.4 Cases Gemini Decision 1 got wrong

Of the 12 semantic match cases Gemini Decision 1 got wrong, the 3 that were slip-throughs are as follows.

3. Results for all cases

Bold indicates cases that were wrong at the 0.8 threshold.

4. Summary

I had Gemini Decision 1, Jev, and the Decisions API solve the same 107 cross-matching cases on the same day.

For the number of correct semantic matches at the 0.8 threshold, Jev was highest with 100, followed by the Decisions API with 96 and Gemini Decision 1 with 94.
Jev was also the most advantageous in speed and cost; Gemini Decision 1 had the slowest median response time of the three at about 0.54 seconds, and its cost was about 1.4 times Jev's. I think the speed difference is mostly negligible.

On the other hand, the tendencies of the output probabilities were very similar between Gemini Decision 1 and Jev.
While the Decisions API stuck to exactly 0 or 1 in 83 cases, Gemini Decision 1 had 0 such cases, making it possible to pick up hesitant cases by their low probabilities. This is really great!

Although Gemini Decision 1 costs somewhat more than Jev, including the advantage of handling images and PDFs with the same API, I feel it is a good candidate to consider adopting.

However, at the moment Gemini Decision 1 is available only on the Gemini Developer API (the API called with a Google AI Studio API key).

Available on the Gemini Developer API (v1beta) with Gemini Decision 1 only.

Source: Decisions API | Google AI Studio

It would be perfect if it also became available on Google Cloud's Gemini Enterprise Agent Platform (formerly Vertex AI; Vertex AI name changes | Google Cloud)!