I tried forcibly turning Jev into a language model
This page has been translated by machine translation. View original
Jev is getting a lot of buzz. Apparently it's a model that only returns "Yes probability," "probability per choice," and "score" for any input.
When told it won't generate, it makes you want to make it generate.
Forced it to generate
I set the choices to 88 options — all hiragana characters + punctuation + END — and pass the question and "answer so far" as input, having it pick the next character. The picked character is added to the answer, and this repeats until END appears. It's a language model that generates one character at a time.
The code can be opened in Colab (API key required).
Results
The question is "What is the capital of Japan?" (in Japanese).
step 1 pick='し' top3: 'し':0.11 'た':0.10 'こ':0.09
step 2 pick='い' top3: 'し':0.08 'こ':0.08 'い':0.08
step 3 pick='し' top3: 'し':0.08 'こ':0.08 'と':0.07
step 4 pick='END' top3: 'END':0.18 'し':0.09 'と':0.08
A: 「しいし」
The capital of Japan turned out to be "shiishi." The distribution is nearly flat, with a three-way tie between "shi," "ta," and "ko" from the very first character.
It's not that it doesn't know
Asking the same question in Yes/No format (Noul) — "The answer is Tokyo" — returns 0.96, and "Written in hiragana it is 'tōkyō'" returns 0.92. Given five choices — tōkyō, ōsaka, kyōto, nagoya, sapporo — tōkyō scores 1.00.
At the very least, it knows the answer itself. It just can't write it out one character at a time.
Just to be sure, I tried passing it the answer and asking "Write the word 'Tokyo' in hiragana one character at a time."
A: 「たこ」
Splitting into rows and columns
Thinking 88 choices might be too many, I first had it pick a row (a-row through wa-row), then pick one character from within that row (for the sa-row: sa, shi, su, se, so) — a two-stage approach. The number of choices per step drops to 17 and 5.
A: 「あいいいいいいいいいいい」
The row distribution remained nearly flat — a-row 0.22, ka-row 0.21, ta-row 0.18. When unsure, the default locked onto "a-row" and "i," writing "i" all the way up to the 12-character limit.
What about English?
With 32 choices — a–z + symbols + SPACE + END — I asked "What is the capital of Japan?"
A: "tai"
When told "Spell the word 'tokyo'," giving it the word to spell directly, it gets as far as "tokoyo." Copying characters it can see works reasonably well, but recalling something and then spelling it out is impossible regardless of language.
Why can't it write?
Probably because the probabilities Jev returns are the probabilities of "which choice is correct" as a judgment — not the probabilities of "what comes next" as a prediction. (Well, of course.)
Finally
The official documentation wasn't lying. Jev truly does not generate text.
Use Jev if you want it to judge, use an LLM if you want it to generate, and use anything other than "shiishi" if you want to know the capital of Japan.
It feels odd saying this after writing an article like this, but at DevelopersIO 2026 Tokyo on Saturday, October 24th, I'll be giving a more serious talk. I'll be presenting under the title "AI Agent Design Learned from Stock Trading — Context, Harness, and LLMOps," so please come check it out.
See you 👋
