I had Fable 5, Opus 4.8, and GPT 5.6 Sol solve difficult Project Euler problems and compared them

Hello. I'm Takeda from the Service Development Department.

I compared the latest Claude models, Fable 5 and Opus 4.8, along with OpenAI's GPT 5.6 Sol, using the latest Project Euler problems. I've done model comparisons in this same format before, and tried it again this time. I had each of the 3 models solve 2 problems each at both medium and high reasoning effort.

Let me note upfront: this is not a rigorous benchmark. Each setting was run only once, and the execution environments were not perfectly matched. This is a casual comparative test to see what happens when the same problems are given with the same instructions.

Also, Project Euler requires that solutions and methods for problems 101 and beyond not be shared externally. Therefore, this article only covers correctness, time taken, and differences in model behavior — it does not touch on the actual answer values or solution methods.

Study Design

Problem Selection

I selected 2 problems from the latest Project Euler problems. The reason for using the latest problems is that no solutions exist in training data or on the web. This structurally prevents any possibility of cheating.

Solved by (number of correct solvers) serves as a measure of difficulty. Around 100 people after several weeks since publication is quite difficult even by Project Euler standards overall.

Prompt and Constraints

All models were given the same prompt.

Can you solve this problem? Consulting others or looking at existing sources is completely prohibited.
https://projecteuler.net/problem=<problem number>

Only retrieving the problem statement was permitted; viewing or searching explanations, forums, or others' solutions was prohibited. Each model declared in its final report that it "referenced nothing other than the problem statement," and since these are the latest problems, no actual solvable methods exist to reference.

Execution Environment

All ran in agent environments capable of code execution and web retrieval. Correctness was verified by submitting answers to the Project Euler site.

Results

First, a note on how to read the numbers. Time and tokens are listed as reference values. Time is self-reported by each agent; tokens represent output_tokens (including thinking tokens) for Claude-based models, and the Codex CLI reported value for GPT 5.6 Sol — the two use different measurement metrics. They cannot be used for absolute cross-model comparisons, so treat them as a way to see the medium/high difference within the same model.

Problem 1004 (Somewhat Difficult)

The only failure was Opus 4.8 on medium. Moreover, it cut off at 4.3 minutes and 8.2K tokens — shorter than any other run. Even with the same Opus 4.8 model, switching to high caused both time and tokens to roughly triple, and it got the correct answer.

Opus 4.8 on medium had passed the small verification values given in the problem statement, but submitted an answer with part of the range that should have been counted missing. Since it passed the small verification, the error couldn't be caught at the time. This is the same structure as 998, discussed later — answers can still be wrong even after passing the verification values.

Problem 998 (Among the Hardest)

The failure here was GPT 5.6 Sol on medium. It ran through in about 10 minutes, but the cause of failure was that among candidates mechanically generated from the approach, there were items that "looked correct but actually didn't satisfy the conditions," and it compiled results without performing final verification on those. Moreover, since this mixing doesn't occur within the range of verification values in the problem statement, all verification checks passed. Switching to high, it invested double the time and tokens in verification and got the correct answer.

The two Claude-based models, on the other hand, got 998 correct on both medium and high. Without being instructed to, both spontaneously incorporated cross-checks with independent implementations and broad brute-force verification. As a result, Opus 4.8 took an exceptionally long 168 minutes on medium.

Observations

The clearest finding this time was that what separates success from failure on difficult problems is not finding the solution approach, but the thoroughness of verification. Of the 12 runs (3 models × 2 effort levels × 2 problems), only 2 failed — both on medium, and for different models on different problems. Opus 4.8 cut off on 1004 at 4 minutes and 8K tokens and gave the wrong answer; GPT 5.6 Sol skipped candidate verification on 998 and gave the wrong answer. In both cases the problem's verification values passed, so the error couldn't be caught at the time. Switching to high caused both to invest time and tokens in verification and get the correct answers.

The two Claude-based models showed a tendency to spontaneously incorporate cross-checks with independent implementations and brute-force verification without being instructed to. This comes at a time cost (Opus 4.8 took 168 minutes on 998 medium). Fable 5 got all 4 runs correct and was on the faster end among the Claude-based models. GPT 5.6 Sol showed a straightforward response to effort settings — on medium it ran quickly but with thin verification; on high it properly accumulated verification. If you're using Codex for coding or mathematical tasks, the effort setting is something to be mindful of.

Let me also touch on the design of problem 998. Thanks to its structure where "you can pass all the provided verification values and still give the wrong answer," differences in verification habits showed up directly in correctness. Problem 1004 trapped medium Opus 4.8 in the same pitfall. It was a renewed reminder that passing verification checks and having the correct answer are two different things. It's also suggestive as evaluation material for LLMs.

To reiterate, this is a casual comparison with only one run per setting. That said, the format of "latest Project Euler problems × prohibition on referencing existing solutions" is convenient for seeing a model's raw mathematical and implementation ability along with verification habits, so I plan to try the same format again when new models come out.