I compared document value matching using Cloudflare's Clef with Jev

Hello, I'm Keema.

In a previous article, I had TypeSafe's Jev perform cross-matching to determine whether the same field values written in two documents match (【TypeSafe】How Far Can Jev Go for Document Field Matching? Measuring Accuracy, Speed, and Cost | DevelopersIO).

In document matching operations, for example when reconciling invoices and purchase orders, the following types of "notation variations that humans can recognize as the same value but don't match as strings" frequently occur:

• Date notation: 2026/09/12 and 2026年9月12日, Sep 12, 2026

• Amount and currency notation: $1,250.00 and USD 1,250.00

• Counterparty name notation: 株式会社サンプル商事 and サンプル商事株式会社, ACME CO., LTD. and Acme Co., Ltd.

If you try to determine these using simple string comparison alone, they all end up as mismatches.
However, covering them with regular expressions or rule-based approaches is difficult, so an approach of using a model such as an LLM to determine semantic matches is effective.

Cloudflare Workers AI's Clef is API-compatible with Jev, so this time I sent the same test data and the same questions to compare the results.

1. Verification Conditions

• Data: The same 107 cases as last time (10 categories including dates, amounts, counterparty names, addresses, OCR misreads, and missing values). All values are fictitious and include both Japanese and English notations.

• Evaluation axes: The same 2 questions as last time. Both exact (whether the strings are completely identical) and semantic (whether they refer to the same value even with different notations) are judged by probability of yes.

• Question (prompt) language: 2 patterns were prepared: one with instructions and questions passed to the model in "English" and one in "Japanese." The data being matched (Value A and Value B) is the same, but this verifies whether accuracy differs depending on the language of the prompt given to the model.

• Threshold: Same as last time, 0.8. If the probability is 0.8 or higher, it is judged as "match."

• Models: clef (27B) and clef-flash (9B), each run once with English and Japanese question prompts.

• Jev: Results from the previous run (September 20, 2026, jev-1.13.0) are used.

The instruction content of the prompts and the content of the cases are the same as the previous article.
Note that after the previous run, the correct answer for case 3-6 was corrected, so the Jev results have also been rescored according to the current correct answers.

2. Results

2.1 Overall

※ "English prompt" and "Japanese prompt" in the table refer to the language of the instructions and questions (prompts) passed to the model. The data being matched (Value A and Value B) itself is a mix of Japanese and English and is the same across all conditions.

For exact, which checks string matches, clef achieved accuracy equal to or better than Jev, with a perfect score on English prompts.
On the other hand, for semantic, which checks semantic matches, both clef and clef-flash had fewer correct answers than Jev.

2.2 Clef Outputs Conservative Probabilities

Many of the cases that were incorrect for semantic were ones where the output probability did not reach 0.8, even though the correct answer was "match."
For example, even looking at the semantic probabilities output for completely identical strings (9 cases of exact duplicates), Jev was 0.94 or higher, while clef (Japanese prompt) had a minimum of 0.73, and clef-flash (English prompt) had a minimum of 0.57.

The results after rescoring with different thresholds are as follows.
As with the overall results in 2.1 (threshold 0.8), the number of correct answers and the "number of false positives (incorrectly judging different values as matching)" are summarized by threshold.

Threshold 0.5

Threshold 0.6

Threshold 0.7

Threshold 0.8 (default)

When the threshold is lowered to 0.6 for clef (English prompt), it achieves 97/106, matching the number of correct answers as Jev at threshold 0.8.
However, the number of cases where different values were incorrectly judged as "matching" is 4, which is 1 more than Jev's 3 (at threshold 0.8).
If you apply the threshold set for Jev directly to Clef, it tends to judge items that should match as non-matching.

2.3 Cost

Although clef had fewer input tokens than Jev, the token unit price is approximately 6 times higher, so the total cost was approximately 4 times higher.

2.4 Probabilities for All Cases

3. Summary

For exact string matching (exact), clef achieved accuracy equal to or better than Jev, but for semantic matching that absorbs notation variations (semantic), clef's accuracy did not appear as high as Jev's.

While Jev does not officially support Japanese either, clef showed a particularly large drop in accuracy when the question prompt was in Japanese, and I felt that even when using clef, it is safer to write the question prompt to the model in English.

Regarding execution speed, clef was faster in this actual measurement.
However, response speed is also affected by server load and network conditions, so it cannot be stated categorically that clef is always faster.
Considering operational scenarios where a threshold is set to route low-confidence data to human review, I felt that Jev is currently easier to work with in practical operations, also from the perspectives of output probability stability and cost.