Post

The Smartest Model Is the Wrong Question

By Promit Bhattacherjee·July 25, 2026·18 min read#Frontier Models#AI Benchmarks#Agentic AI

A benchmark-grounded field guide to GPT-5.6 Sol, Claude Opus 5, and Claude Fable 5—and the work I would actually give each one.

Three frontier models now sit within two points of one another on one of the better independent intelligence indices.

That sounds like a close race. It is not.

GPT-5.6 Sol, Claude Opus 5, and Claude Fable 5 may occupy the same narrow band on a leaderboard, but they are not interchangeable products. One behaves like a fast technical operator with a first-class tool belt. One is an unusually good senior colleague for difficult professional judgment. One is a costly research instrument built for problems so large that ordinary model economics stop making sense.

The mistake is to ask, Which one is smartest?

The useful question is: Which failure am I least willing to tolerate on this particular job?

Do I care most about tool execution, turnaround time, and cost? Do I need a model to inspect a repository, operate a shell, revise an artifact, and verify the result? Do I need better judgment across a messy professional brief? Or am I giving an agent an almost absurd amount of context and asking it to remain useful for hours or days?

Those are three different buying decisions.

This comparison uses official model information from OpenAI, Anthropic's Opus 5 announcement, and Anthropic's Fable 5 announcement, alongside independent evaluations from Artificial Analysis. Benchmark results and prices are current as of 25 July 2026.

The short answer

If I had to choose before knowing anything else about the workload:

Model	I would choose it for	I would hesitate when
GPT-5.6 Sol	Coding agents, terminal work, tool-heavy orchestration, computer use, research pipelines, artifact generation, and production systems where cost and turnaround matter	The task is primarily ambiguous executive judgment rather than execution
Claude Opus 5	High-stakes professional analysis, difficult reviews, strategy, architecture decisions, senior-level coding judgment, and daily frontier work	I need the fastest or cheapest agentic execution at scale
Claude Fable 5	The hardest long-horizon research, enormous codebases, deep factual recall, vision-heavy reconstruction, and projects where the ceiling matters more than the bill	The workload touches guarded cyber or biological domains, cannot accept 30-day retention, or has normal commercial economics

My default for building would be Sol. My default for thinking would be Opus 5. I would reserve Fable 5 for the rare assignment where I could explain, in one sentence, why neither of the other two was enough.

That last condition matters. A premium model should not be a personality preference. It should be an architectural decision.

First, the benchmark table—and the warning label

Here is the most useful independent snapshot I could assemble. It deliberately mixes no vendor claims into the main quality comparison.

Independent evaluation	GPT-5.6 Sol	Claude Opus 5	Claude Fable 5
Artificial Analysis Intelligence Index v4.1, max effort	59	61	60
GDPval-AA v2, Elo	~1,748	1,861	1,747
Intelligence Index cost per task	$1.04	$2.03	$2.75
Coding Agent Index v1.3	67 in Codex	67 in Claude Code, xhigh	66 in Claude Code, max with fallback
DeepSWE	69%	60%	66%
Terminal-Bench v2	88%	85%	83%
SWE-Atlas-QnA	43%	55%	49%
Coding-agent cost per task	$7.08	$8.23	$11.71
Coding-agent wall time per task	10.2 min	23.6 min	23.4 min

The intelligence and cost results come from Artificial Analysis's Opus 5 report and its earlier GPT-5.6 evaluation. The coding breakdown comes from its current Claude Code versus Codex comparison.

Now the warning label.

This is not a horse race between three naked neural networks. It is a comparison between systems:

Sol is running inside the Codex harness; Opus and Fable are running inside Claude Code.
Reasoning effort is not identical. The leading Opus coding result uses xhigh; the Sol result uses max.
Anthropic's Fable result uses a fallback. Some guarded requests may be answered by another Claude model.
The harness decides how context is assembled, which tools are exposed, when the model is allowed to retry, and how the task ends.
Prompt caching changes effective cost dramatically. Artificial Analysis measured the cache behaviour it observed, not an imaginary perfect-cache scenario.
A difference of one or two index points is not a law of nature. It is often smaller than the change produced by a better system prompt, a better retrieval layer, or a less foolish tool definition.

The individual coding scores make the point beautifully. Sol leads DeepSWE and Terminal-Bench. Opus 5 leads SWE-Atlas-QnA. Both finish with the same composite score.

That is not a tie in the ordinary sense. It means they arrive at the same postcode by different roads.

Sol is the better operator. Opus is the better reader of the engineering situation. Fable has the highest long-horizon ceiling, but it is slower and more expensive in this harness.

The commercial reality

Token pricing is not the same as task pricing, but it is still where the invoice begins.

Model	Input per 1M tokens	Output per 1M tokens	Context
GPT-5.6 Sol	$5	$30	1.05M tokens
Claude Opus 5	$5	$25	1M tokens
Claude Fable 5	$10	$50	1M tokens

OpenAI lists a 1,050,000-token context window for Sol, with up to 922,000 input tokens and 128,000 output tokens. It also applies higher rates to requests above 272,000 input tokens: twice the input price and 1.5 times the output price for the entire request. That is not a footnote if your design regularly drops a repository or data room into context.

At list price, a fresh request containing 200,000 input tokens and producing 20,000 output tokens costs approximately:

Model	Approximate uncached request cost
GPT-5.6 Sol	$1.60
Claude Opus 5	$1.50
Claude Fable 5	$3.00

That simple calculation makes Opus look slightly cheaper than Sol. The independent task results make Sol look substantially cheaper. Both can be true.

Agents do not consume a fixed prompt and emit a fixed answer. They loop. They read tool results, reconsider plans, repeat commands, revise files, and accumulate context. A model that takes half as many turns can cost less despite a more expensive output token. A model with a better caching pattern can reverse the calculation again.

The economically correct unit is therefore not price per token. It is:

Cost per accepted outcome, including model calls, tools, retries, verification, human review, and the price of failure.

On the Artificial Analysis Intelligence Index, Sol reaches 59 for $1.04 per task. Opus 5 reaches 61 for $2.03. Fable reaches 60 for $2.75. Opus wins the score; Sol wins the frontier of intelligence per dollar.

In production, I would not debate that difference abstractly. I would run twenty to fifty representative tasks, score them blind, include reviewer time, and calculate the cost of the accepted result. Anything else is model fandom with a spreadsheet attached.

GPT-5.6 Sol: the technical operator

Sol's most important feature is not that it scores 59 instead of 58. It is that OpenAI has designed the surrounding platform for work that unfolds through tools.

The Responses API supports programmatic tool calling, allowing the model to write and run lightweight programs that coordinate tools and filter intermediate results. Sol can use web and file search, code execution, hosted shell, patch application, computer use, MCP, and tool search. OpenAI also exposes a multi-agent beta, while higher-effort experiences can coordinate several agents in parallel.

That changes the architecture.

In a conventional agent loop, every tool response returns to the model. The context grows, the model reads the same noise repeatedly, and the bill becomes a biography of every intermediate mistake. Programmatic tool calling lets computation remain computation. A small program can query ten systems, discard nine irrelevant results, aggregate the useful pieces, and send only the decision-grade residue back into reasoning.

That is why I would choose Sol for an end-to-end cloud delivery path:

Ingest a meeting transcript and identify requirements, constraints, assumptions, risks, and missing decisions.
Produce an HLD for human review.
Convert the approved HLD into an LLD, network plan, identity model, security controls, observability plan, and deployment sequence.
Inspect an existing Terraform estate rather than assuming a greenfield fantasy.
Build or modify modules, run fmt, validate, tests, policy checks, and plans.
Read failures, revise the implementation, and produce an evidence pack for the human approval gate.

That workload rewards terminal competence, persistence, structured tools, low turn count, and the ability to create and inspect real artifacts. The current coding-agent results reflect exactly those strengths: Sol leads the terminal and repository implementation components, finishes the composite evaluation in half the wall time of the leading Opus configuration, and does so at lower task cost.

Sol also has a strong case for computer-use workflows, browser research, cybersecurity, scientific work, and visual artifact creation. OpenAI reports 62.6% on OSWorld 2.0, 90.4% on BrowseComp, and strong results in terminal, CAD, and professional document tasks. Those are vendor-reported results, so I would not treat them as a procurement verdict. But they agree with the architecture of the product: Sol is meant to operate.

Where would I not automatically choose it? A board memo assembled from contradictory documents. A strategic recommendation whose core difficulty is deciding which evidence deserves weight. A senior code review where the implementation already works and the real question is whether the abstraction should exist at all.

Sol can do those things. Opus 5 currently has the stronger evidence for them.

Claude Opus 5: the senior colleague

Opus 5 is the surprise of this comparison.

Anthropic positions Fable as the tier above Opus, yet Opus 5 narrowly leads Fable on the Artificial Analysis Intelligence Index and beats both Fable and Sol by more than 100 Elo on GDPval-AA v2. It also leads AA-Briefcase, an evaluation of agentic professional outputs, by 146 Elo over Fable.

That is not evidence that Opus contains more facts than Fable. Artificial Analysis explicitly finds that it does not: Fable remains stronger on factual knowledge, while Opus 5's hallucination rate in one omniscience evaluation rises as it becomes more willing to answer.

What Opus appears to do unusually well is turn messy information into a professional decision and a coherent artifact.

That is a different capability from recall.

A senior architect is valuable not because she can recite more Azure limits than the documentation. She is valuable because she can notice that the business has asked for three mutually incompatible things, find the actual constraint, explain the trade-off without hiding behind jargon, and design a path that the organisation can operate after the consultants leave.

The Opus 5 evaluations point toward that kind of work:

strategy and operational analysis;
finance and consulting artifacts;
ambiguous, multistep knowledge work;
codebase comprehension and engineering judgment;
verification, revision, and self-correction;
tasks where the answer must be not merely correct but useful to a professional reader.

I would use Opus 5 to review a cloud architecture before a design authority meeting. I would ask it to attack the assumptions in an AI Landing Zone. I would give it five vendor proposals, security exceptions, meeting notes, and cost estimates, then ask for the decision memo that nobody has managed to write. I would use it as the second model in a high-stakes review pipeline precisely because it does not fail in the same shape as Sol.

It is also the pragmatic Claude choice. Opus costs half as much as Fable at list price, carries no special data-retention requirement for general access, and Anthropic says its cyber classifiers intervene roughly 85% less often than Fable's. It remains guarded—especially for penetration testing, exploit generation, and some binary analysis—but ordinary secure-code review is less likely to be rerouted.

If one model had to sit beside me for a full working day, reading, questioning, reviewing, and occasionally writing code, I would pick Opus 5.

Not because it is universally smarter. Because judgment is its most convincing advantage.

Claude Fable 5: the research instrument

Fable 5 is easier to misunderstand because its best use cases sound like marketing until the scale becomes concrete.

Anthropic reports that it migrated a 50-million-line Ruby codebase in a day, a job Stripe estimated would otherwise require a team for more than two months. It can remain focused across extremely long-running tasks, use persistent notes, reconstruct applications from screenshots, and pursue scientific or engineering projects with far less scaffolding than earlier models.

This is not the model I would use to draft a normal article, refactor an API, or analyse a weekly sales report. It is the model I would consider when the unit of work has stopped being a task and become a campaign.

Examples:

a codebase-wide migration across tens of millions of lines;
a multi-day research effort combining hundreds of papers, datasets, notes, and experiments;
reconstruction of a complicated visual product from screenshots and behaviour;
autonomous exploration where the agent must maintain its own working memory;
factual and analytical work where a modest improvement at the ceiling is worth twice the list price.

Fable's difficulty is that the capability and the product policy cannot be separated.

Fable is the generally available version of Anthropic's Mythos-class model. The underlying model is deliberately placed behind classifiers for dual-use cyber, biology, chemistry, and model-distillation requests. When a classifier triggers, the request may fall back to another model rather than receive a Fable answer. Anthropic originally routed many such requests to Opus 4.8; biology requests blocked on Fable now route to Opus 5.

So a Fable benchmark with fallback is not always a Fable benchmark. And a Fable production request is not guaranteed to be answered by Fable.

There is also a governance issue: Anthropic requires 30-day retention for traffic on Mythos-class models, including Fable, across first- and third-party surfaces. Anthropic says the data is not used to train models and is retained for safety purposes, but for regulated workloads the purpose does not remove the control requirement. Some organisations simply cannot place that class of data there.

This makes Fable a consciously chosen specialist, not the premium button a team presses whenever a task looks difficult.

I would ask three questions before approving it:

Is there evidence that Opus 5 or Sol cannot reach the acceptance threshold?
Is the expected value of the additional capability larger than the higher task cost and latency?
Are the retention and safety-routing behaviours compatible with the data and the domain?

If any answer is vague, I would not use Fable yet.

Six workloads, six decisions

Benchmarks become useful only when they change an actual routing rule.

1. Build an Azure platform from an approved LLD

Choose: GPT-5.6 Sol.

The workload is dominated by repository inspection, Terraform changes, shell commands, Azure queries, policy validation, plan analysis, diagrams, and evidence capture. Sol's agentic coding, terminal performance, tool coordination, and lower observed wall time matter more than Opus's advantage in professional judgment.

I would still place human approval before apply, and I would use an independent reviewer—often Opus 5—to examine the plan, identity boundaries, destructive changes, and unstated assumptions.

2. Turn a board pack, transcripts, and market evidence into a strategy

Choose: Claude Opus 5.

This is classic GDPval and AA-Briefcase territory: inconsistent evidence, ambiguous intent, professional presentation, and a recommendation that must survive intelligent objection. The model must decide what matters before it can write.

The deliverable should still cite its evidence and expose uncertainty. A polished hallucination is simply a more dangerous hallucination.

3. Migrate a vast legacy codebase

Choose: Fable 5—after a pilot.

If the estate is truly vast and the work can run for a day or more, Fable's long-horizon behaviour may justify the cost. But I would first run the same vertical slice through Sol and Opus. The winner is the model that produces an acceptable migrated slice with the fewest human corrections, not the one with the grandest launch story.

4. Operate a high-volume coding agent

Choose: GPT-5.6 Sol, probably below max effort for routine work.

Sol max and Opus xhigh tie on the current coding-agent composite. Sol finishes the evaluation much faster and cheaper. More importantly, GPT-5.6 exposes multiple effort levels, and many production tickets do not deserve maximum cognition. Route routine fixes to a cheaper setting; escalate only when confidence, tests, or complexity cross a threshold.

Using the most expensive model for every ticket is not quality engineering. It is the absence of routing.

5. Perform secure code review or scientific analysis

Choose: Sol or Opus 5, depending on the action boundary.

Sol has broad technical and scientific evaluation strength. Opus 5 is compelling for careful review and is less restricted than Fable. Fable may be the most capable underlying system for parts of biology or cyber, but its safeguards can reroute the request, making it an unreliable choice unless the organisation has the appropriate trusted-access arrangement.

In regulated science, add a domain expert and provenance checks. None of these models converts an unverified hypothesis into evidence.

6. Review an AI or cloud architecture

Choose: Opus 5 first, Sol second.

Ask Opus to find conceptual weakness: missing boundaries, incoherent assumptions, operational blind spots, responsibility gaps, and choices the diagram conceals. Ask Sol to test the architecture against implementation reality: service limits, network paths, IAM configuration, policies, deployment order, monitoring, and failure recovery.

That pair is more valuable than asking either model the same question twice.

The routing architecture I would actually deploy

I would not choose one of these models for an enterprise. I would build a small routing system.

Default path: use a lower-cost model or lower effort for classification, extraction, summarisation, and routine changes.

Execution path: send tool-heavy, terminal-heavy, computer-use, and implementation work to GPT-5.6 Sol.

Judgment path: send ambiguous professional analysis, design review, and high-stakes synthesis to Claude Opus 5.

Frontier escalation: send only empirically difficult, long-horizon tasks to Claude Fable 5 after checking domain safeguards, retention policy, and budget.

Cross-review: use a model from the other family to criticise high-impact output. Correlated failure is the enemy. Two answers from the same model are not two independent opinions.

Human gate: require a named person to approve financial commitments, production changes, security-sensitive actions, regulated decisions, and irreversible operations.

The router should not rely on a poetic prompt that asks the model whether the problem is difficult. It should use measurable signals:

context size;
number and type of tools;
reversibility of proposed actions;
domain and data classification;
expected task duration;
confidence from automated evaluators;
test and policy results;
cost ceiling;
previous failure on a cheaper route.

Every route should log the model, snapshot, effort, fallback behaviour, prompt version, tools, token usage, latency, evaluation outcome, and human decision. Without that record, model selection is not architecture. It is gossip.

Why the benchmark winner keeps changing

Opus 5 arrived on 24 July and immediately changed the leaderboard. Another release will change it again.

That does not make the research useless. It means the durable asset is not the answer. It is the evaluation discipline.

Build a private benchmark from your own work:

twenty representative tasks;
five deliberately ugly edge cases;
a blind acceptance rubric;
factual and citation checks;
executable tests where possible;
security and policy checks;
human correction time;
total task cost;
latency to an accepted result.

Then rerun it when a model, harness, system prompt, or reasoning mode changes.

Public benchmarks tell me which models deserve a trial. Private evaluations tell me which model deserves production traffic.

The distinction is the difference between reading a restaurant review and running a kitchen.

My verdict

GPT-5.6 Sol is the best engineering default. It combines near-frontier intelligence with the strongest current evidence for efficient coding-agent execution, a broad native tool surface, programmatic tool orchestration, and faster completion in the independent coding comparison. If the work must touch systems and come back with proof, Sol is my first choice.

Claude Opus 5 is the best general frontier model for professional judgment. It currently leads the independent intelligence index by a narrow margin and leads agentic knowledge-work evaluations by a meaningful one. It is the model I would want to challenge a design, write the decision memo, or make sense of a complicated professional problem.

Claude Fable 5 has the highest specialist ceiling, not the best default value. Its factual depth, vision, and long-horizon autonomy matter for unusually large projects. Its double price, slower agentic economics, safety routing, and 30-day retention requirement mean that using it casually is difficult to defend.

So which is better?

For an architect, that is the wrong level of abstraction.

Sol is better when the machine must do. Opus 5 is better when it must judge. Fable is better when it must endure.

The model is only one component. The tools, harness, context, evaluator, safety policy, and human approval boundary decide whether intelligence becomes a result—or merely an expensive performance of competence.

Sources and further reading

← All posts