Compare 104 models released since 2024 across 5 task families. Model records include linked official sources, while benchmark results retain their compatible evaluation snapshot.
Choose one task family, then filter its validated inventory. Charts and comparisons use compatible benchmark snapshots and pricing units; missing values remain visible as Not reported.
Choose a task
Compare models only within one compatible task family.
Decision summary
Facts from one compatible snapshot, measured 2026-10-08.
View detailsStrengths, cautions, and provenance for Claude Haiku 5.5
Strengths
The cheapest model in this catalogue at $0.10 per million input and $0.50 per million output for prompts up to 100,000 tokens
Carries the full 1M-token context window and a 128,000-token output cap, which earlier Haiku releases did not
Cache reads at $0.01 per million make high-volume classification and routing nearly free on the input side
Cautions
Priced by prompt length, and it is a cliff rather than a taper: a request whose prompt passes 100,000 tokens is billed at $0.50 input and $2.50 output for the whole request, five times the short rate
Cache reads and cache writes both count toward that 100,000-token measurement, so a large cached prefix can tip a request into the higher band on its own
It is the only current Claude model whose 1M window is not priced uniformly; every other 4.6-and-later model bills a 900k request at the same rate as a 9k one
View detailsStrengths, cautions, and provenance for Mistral Large 4
Strengths
A natively multimodal mixture-of-experts model with 1.05 trillion total and 52 billion active parameters, a 1.6B vision encoder and a 1M-token context window
Mistral reports fluency in more than 160 languages, including every official EU language, and serves the preview from European data centres
Mistral says open weights follow by the end of October 2026, which would make it one of the largest open-weight models available
Cautions
This is a public preview: Mistral says the reinforcement learning run is still in flight and the model will keep changing
Weights and their licence were not published at the time of this update, so it is recorded as API-only
Mistral offered a temporary 50% launch discount; the recorded rates are the list prices
Epoch AI measured 20.0% on SimpleQA Verified, low for its size, so check factual recall on your own questions
View detailsStrengths, cautions, and provenance for Claude Sonnet 5.5
Strengths
Anthropic's speed-and-intelligence tier at $2 per million input and $10 per million output, with the same 1M-token window and 128,000-token output cap as Opus 5.5
Cache reads at 0.05x base input ($0.10 per million), half the usual multiplier
Keeps the high effort default, unlike Claude Opus 5.5
Cautions
Same list price as Claude Sonnet 5, so the gain is capability and cache economics rather than a cheaper rate card
Anthropic positions Opus 5.5 rather than this as the default, so reach for Sonnet 5.5 when latency matters more than peak capability
View detailsStrengths, cautions, and provenance for Claude Opus 5.5
Strengths
Anthropic's recommended default for most workloads, taking that role from Claude Opus 5, at $4 per million input and $20 per million output
1M-token context window with a 128,000-token maximum output, and adaptive thinking that is always on
Cache reads are 0.05x base input ($0.20 per million) rather than the usual 0.1x, which matters most in agent loops that replay a large prefix
Cautions
The API effort default is medium here, not high as on Claude Fable 5.1 and Claude Sonnet 5.5, so a like-for-like comparison has to set effort explicitly
The 4.7-generation tokenizer produces roughly 30% more tokens for the same text than Claude Sonnet 4.6 and earlier, so per-token rates do not compare directly across generations
View detailsStrengths, cautions, and provenance for GPT-6 Luna
Strengths
OpenAI's budget tier at $0.10 per million input and $0.50 per million output, matching Claude Haiku 5.5 on both
Keeps the full 1,050,000-token context window rather than cutting it down for the cheap tier
The long-context surcharge starts at 272,000 tokens rather than Claude Haiku 5.5's 100,000, which is the more forgiving threshold of the two budget options
Cautions
Prompts over 272,000 tokens are rebilled at $0.20 input and $0.75 output for the whole request
A twentieth of GPT-6 Sol's price reflects a capability gap, not a discount on the same model
View detailsStrengths, cautions, and provenance for DeepSeek-V4.1-Flash
Strengths
A 552B-parameter mixture-of-experts model that activates 8B parameters for input and 16B for output, with native visual understanding and a 1M-token context window
Released as MIT-licensed open weights on Hugging Face as well as on the DeepSeek API, so the same model can be hosted or self-run
DeepSeek says it beats its own former flagship, DeepSeek-V4-Pro, and halves KV-cache memory needs; Epoch AI measured 89.8% on GPQA Diamond
Cautions
The recorded API rates are the peak-hour prices; DeepSeek charges half outside 01:00-04:00 and 06:00-10:00 UTC on weekdays
Running the open weights moves cost to your own infrastructure, which the token rates here do not represent
View detailsStrengths, cautions, and provenance for GPT-6 Astra
Strengths
OpenAI positions GPT-6 Astra as its frontier model for reasoning, agentic work, and computer use, with a 1,050,000-token context window and 128,000-token maximum output
OpenAI reports the strongest published deep-context retention of any frontier model on MRCR v2 (8 needles): 100% up to 512K tokens and 96.3% in the 512K-1M band
OpenAI reports saturation-level scores on FrontierMath Tier 4 (97.6%) and, under its own provider adapter harness, ARC-AGI-3 (99.9%)
Cautions
Long-context billing is a cliff, not a taper: a request over the long-context threshold is rebilled in full at $20 per million input and $75 per million output, so a prompt that crosses it costs roughly double
GPT-6 Astra is the first publicly released model OpenAI classifies as crossing the Critical cybersecurity threshold under its own Preparedness Framework, which carries additional deployment safeguards
Vendor-reported benchmark figures come from OpenAI’s own launch material and its own harness; they are not independently reproduced here
Compare declared coverage and compatible operational values for the current task.
Capability coverage
Capabilities are not compared for this task. Providers declare them inconsistently, so a missing entry would read as unsupported when it is only unrecorded. Compare the benchmark results above instead.
Operational comparisons
Each panel has its own unit, direction, and fixed displayed range. Values are never combined across panels.
Arena Text
Elo · Higher is better · Range 800–1,600
Price per 1M input tokens
USD / 1M input tokens · Lower is better for cost · Range 0–10
Price per 1M output tokens
USD / 1M output tokens · Lower is better for cost · Range 0–50
Context window
tokens · Higher supports larger inputs · Range 0–1,050,000
View exact operational data
Operational metric data
Metric
Model
Value
Direction
Arena Text
Mistral Large 4
1,429 Elo
Higher is better
Arena Text
GPT-6.1 Sol
1,484 Elo
Higher is better
Arena Text
Claude Sonnet 5.5
1,476 Elo
Higher is better
Arena Text
Claude Opus 5.5
1,507 Elo
Higher is better
Arena Text
GPT-6 Luna
1,443 Elo
Higher is better
Arena Text
GPT-6 Sol
1,456 Elo
Higher is better
Arena Text
Grok 4.7
1,443 Elo
Higher is better
Arena Text
DeepSeek-V4.1-Flash
1,475 Elo
Higher is better
Arena Text
GPT-6 Astra
1,475 Elo
Higher is better
Arena Text
Gemini 3.8 Flash
1,497 Elo
Higher is better
Arena Text
Muse Spark 1.3
1,494 Elo
Higher is better
Arena Text
Claude Fable 5.1
1,501 Elo
Higher is better
Arena Text
Gemini 3.7 Flash
1,486 Elo
Higher is better
Arena Text
Grok 4.6
1,454 Elo
Higher is better
Arena Text
Qwen3.8-Max
1,483 Elo
Higher is better
Arena Text
Claude Opus 5
1,490 Elo
Higher is better
Arena Text
Gemini 3.5 Flash-Lite
1,455 Elo
Higher is better
Arena Text
Gemini 3.6 Flash
1,482 Elo
Higher is better
Arena Text
Grok 4.5
1,466 Elo
Higher is better
Arena Text
Kimi K3
1,488 Elo
Higher is better
Arena Text
GPT-5.6 Luna
1,452 Elo
Higher is better
Arena Text
GPT-5.6 Sol
1,485 Elo
Higher is better
Arena Text
GPT-5.6 Terra
1,466 Elo
Higher is better
Arena Text
Claude Sonnet 5
1,461 Elo
Higher is better
Arena Text
Claude Fable 5
1,504 Elo
Higher is better
Arena Text
Claude Opus 4.8
1,481 Elo
Higher is better
Arena Text
Gemini 3.5 Flash
1,478 Elo
Higher is better
Arena Text
GPT-5.5
1,482 Elo
Higher is better
Arena Text
Grok 4.20
1,472 Elo
Higher is better
Arena Text
GPT-5.4
1,475 Elo
Higher is better
Arena Text
Gemini 3.1 Pro Preview
1,487 Elo
Higher is better
Arena Text
Gemini 3 Flash Preview
1,473 Elo
Higher is better
Arena Text
Kimi K2 Instruct
1,418 Elo
Higher is better
Arena Text
GPT-4.1
1,415 Elo
Higher is better
Arena Text
GPT-4.1 mini
1,383 Elo
Higher is better
Arena Text
GPT-4.1 nano
1,322 Elo
Higher is better
Arena Text
Gemini 2.5 Pro
1,445 Elo
Higher is better
Arena Text
DeepSeek-R1
1,398 Elo
Higher is better
Arena Text
Grok 4.3
1,442 Elo
Higher is better
Price per 1M input tokens
Claude Haiku 5.5
0.1 USD / 1M input tokens
Lower is better for cost
Price per 1M input tokens
Mistral Large 4
1.36 USD / 1M input tokens
Lower is better for cost
Price per 1M input tokens
GPT-6.1 Sol
2 USD / 1M input tokens
Lower is better for cost
Price per 1M input tokens
Claude Sonnet 5.5
2 USD / 1M input tokens
Lower is better for cost
Price per 1M input tokens
Claude Opus 5.5
4 USD / 1M input tokens
Lower is better for cost
Price per 1M input tokens
GPT-6 Luna
0.1 USD / 1M input tokens
Lower is better for cost
Price per 1M input tokens
GPT-6 Sol
2 USD / 1M input tokens
Lower is better for cost
Price per 1M input tokens
Grok 4.7
2 USD / 1M input tokens
Lower is better for cost
Price per 1M input tokens
DeepSeek-V4.1-Flash
0.3 USD / 1M input tokens
Lower is better for cost
Price per 1M input tokens
GPT-6 Astra
10 USD / 1M input tokens
Lower is better for cost
Price per 1M input tokens
Gemini 3.8 Flash
0.75 USD / 1M input tokens
Lower is better for cost
Price per 1M input tokens
Muse Spark 1.3
1.25 USD / 1M input tokens
Lower is better for cost
Price per 1M input tokens
Claude Fable 5.1
10 USD / 1M input tokens
Lower is better for cost
Price per 1M input tokens
Claude Mythos 5.1
10 USD / 1M input tokens
Lower is better for cost
Price per 1M input tokens
Gemini 3.7 Flash
0.75 USD / 1M input tokens
Lower is better for cost
Price per 1M input tokens
Grok 4.6
2 USD / 1M input tokens
Lower is better for cost
Price per 1M input tokens
Qwen3.8-Max
2 USD / 1M input tokens
Lower is better for cost
Price per 1M input tokens
Claude Opus 5
5 USD / 1M input tokens
Lower is better for cost
Price per 1M input tokens
Gemini 3.5 Flash-Lite
0.3 USD / 1M input tokens
Lower is better for cost
Price per 1M input tokens
Gemini 3.6 Flash
1.5 USD / 1M input tokens
Lower is better for cost
Price per 1M input tokens
Grok 4.5
2 USD / 1M input tokens
Lower is better for cost
Price per 1M input tokens
Kimi K3
3 USD / 1M input tokens
Lower is better for cost
Price per 1M input tokens
GPT-5.6 Luna
0.2 USD / 1M input tokens
Lower is better for cost
Price per 1M input tokens
GPT-5.6 Sol
4 USD / 1M input tokens
Lower is better for cost
Price per 1M input tokens
GPT-5.6 Terra
2 USD / 1M input tokens
Lower is better for cost
Price per 1M input tokens
Claude Sonnet 5
2 USD / 1M input tokens
Lower is better for cost
Price per 1M input tokens
Claude Fable 5
10 USD / 1M input tokens
Lower is better for cost
Price per 1M input tokens
Claude Opus 4.8
5 USD / 1M input tokens
Lower is better for cost
Price per 1M input tokens
Gemini 3.1 Flash-Lite
0.25 USD / 1M input tokens
Lower is better for cost
Price per 1M input tokens
GPT-5.5
5 USD / 1M input tokens
Lower is better for cost
Price per 1M input tokens
Grok 4.20
1.25 USD / 1M input tokens
Lower is better for cost
Price per 1M input tokens
GPT-5.4
2.5 USD / 1M input tokens
Lower is better for cost
Price per 1M input tokens
Gemini 3 Flash Preview
0.5 USD / 1M input tokens
Lower is better for cost
Price per 1M input tokens
GPT-4.1
2 USD / 1M input tokens
Lower is better for cost
Price per 1M input tokens
GPT-4.1 mini
0.4 USD / 1M input tokens
Lower is better for cost
Price per 1M input tokens
GPT-4.1 nano
0.1 USD / 1M input tokens
Lower is better for cost
Price per 1M input tokens
Grok 4.3
1.25 USD / 1M input tokens
Lower is better for cost
Price per 1M output tokens
Claude Haiku 5.5
0.5 USD / 1M output tokens
Lower is better for cost
Price per 1M output tokens
Mistral Large 4
4.18 USD / 1M output tokens
Lower is better for cost
Price per 1M output tokens
GPT-6.1 Sol
10 USD / 1M output tokens
Lower is better for cost
Price per 1M output tokens
Claude Sonnet 5.5
10 USD / 1M output tokens
Lower is better for cost
Price per 1M output tokens
Claude Opus 5.5
20 USD / 1M output tokens
Lower is better for cost
Price per 1M output tokens
GPT-6 Luna
0.5 USD / 1M output tokens
Lower is better for cost
Price per 1M output tokens
GPT-6 Sol
10 USD / 1M output tokens
Lower is better for cost
Price per 1M output tokens
Grok 4.7
6 USD / 1M output tokens
Lower is better for cost
Price per 1M output tokens
DeepSeek-V4.1-Flash
1.2 USD / 1M output tokens
Lower is better for cost
Price per 1M output tokens
GPT-6 Astra
50 USD / 1M output tokens
Lower is better for cost
Price per 1M output tokens
Gemini 3.8 Flash
3.75 USD / 1M output tokens
Lower is better for cost
Price per 1M output tokens
Muse Spark 1.3
4.25 USD / 1M output tokens
Lower is better for cost
Price per 1M output tokens
Claude Fable 5.1
50 USD / 1M output tokens
Lower is better for cost
Price per 1M output tokens
Claude Mythos 5.1
50 USD / 1M output tokens
Lower is better for cost
Price per 1M output tokens
Gemini 3.7 Flash
3.75 USD / 1M output tokens
Lower is better for cost
Price per 1M output tokens
Grok 4.6
6 USD / 1M output tokens
Lower is better for cost
Price per 1M output tokens
Qwen3.8-Max
6 USD / 1M output tokens
Lower is better for cost
Price per 1M output tokens
Claude Opus 5
25 USD / 1M output tokens
Lower is better for cost
Price per 1M output tokens
Gemini 3.5 Flash-Lite
2.5 USD / 1M output tokens
Lower is better for cost
Price per 1M output tokens
Gemini 3.6 Flash
7.5 USD / 1M output tokens
Lower is better for cost
Price per 1M output tokens
Grok 4.5
6 USD / 1M output tokens
Lower is better for cost
Price per 1M output tokens
Kimi K3
15 USD / 1M output tokens
Lower is better for cost
Price per 1M output tokens
GPT-5.6 Luna
1.2 USD / 1M output tokens
Lower is better for cost
Price per 1M output tokens
GPT-5.6 Sol
20 USD / 1M output tokens
Lower is better for cost
Price per 1M output tokens
GPT-5.6 Terra
12 USD / 1M output tokens
Lower is better for cost
Price per 1M output tokens
Claude Sonnet 5
10 USD / 1M output tokens
Lower is better for cost
Price per 1M output tokens
Claude Fable 5
50 USD / 1M output tokens
Lower is better for cost
Price per 1M output tokens
Claude Opus 4.8
25 USD / 1M output tokens
Lower is better for cost
Price per 1M output tokens
Gemini 3.1 Flash-Lite
1.5 USD / 1M output tokens
Lower is better for cost
Price per 1M output tokens
GPT-5.5
30 USD / 1M output tokens
Lower is better for cost
Price per 1M output tokens
Grok 4.20
2.5 USD / 1M output tokens
Lower is better for cost
Price per 1M output tokens
GPT-5.4
15 USD / 1M output tokens
Lower is better for cost
Price per 1M output tokens
Gemini 3 Flash Preview
3 USD / 1M output tokens
Lower is better for cost
Price per 1M output tokens
GPT-4.1
8 USD / 1M output tokens
Lower is better for cost
Price per 1M output tokens
GPT-4.1 mini
1.6 USD / 1M output tokens
Lower is better for cost
Price per 1M output tokens
GPT-4.1 nano
0.4 USD / 1M output tokens
Lower is better for cost
Price per 1M output tokens
Grok 4.3
2.5 USD / 1M output tokens
Lower is better for cost
Context window
Claude Haiku 5.5
1,000,000 tokens
Higher supports larger inputs
Context window
Mistral Large 4
1,000,000 tokens
Higher supports larger inputs
Context window
GPT-6.1 Sol
1,050,000 tokens
Higher supports larger inputs
Context window
Claude Sonnet 5.5
1,000,000 tokens
Higher supports larger inputs
Context window
Claude Opus 5.5
1,000,000 tokens
Higher supports larger inputs
Context window
GPT-6 Luna
1,050,000 tokens
Higher supports larger inputs
Context window
GPT-6 Sol
1,050,000 tokens
Higher supports larger inputs
Context window
Grok 4.7
500,000 tokens
Higher supports larger inputs
Context window
DeepSeek-V4.1-Flash
1,000,000 tokens
Higher supports larger inputs
Context window
GPT-6 Astra
1,050,000 tokens
Higher supports larger inputs
Context window
Gemini 3.8 Flash
1,048,576 tokens
Higher supports larger inputs
Context window
Muse Spark 1.3
1,048,576 tokens
Higher supports larger inputs
Context window
Claude Fable 5.1
1,000,000 tokens
Higher supports larger inputs
Context window
Gemini 3.7 Flash
1,048,576 tokens
Higher supports larger inputs
Context window
Grok 4.6
500,000 tokens
Higher supports larger inputs
Context window
Qwen3.8-Max
1,000,000 tokens
Higher supports larger inputs
Context window
Claude Opus 5
1,000,000 tokens
Higher supports larger inputs
Context window
Gemini 3.5 Flash-Lite
1,048,576 tokens
Higher supports larger inputs
Context window
Gemini 3.6 Flash
1,048,576 tokens
Higher supports larger inputs
Context window
Grok 4.5
500,000 tokens
Higher supports larger inputs
Context window
Kimi K3
1,048,576 tokens
Higher supports larger inputs
Context window
GPT-5.6 Luna
1,050,000 tokens
Higher supports larger inputs
Context window
GPT-5.6 Sol
1,050,000 tokens
Higher supports larger inputs
Context window
GPT-5.6 Terra
1,050,000 tokens
Higher supports larger inputs
Context window
Claude Sonnet 5
1,000,000 tokens
Higher supports larger inputs
Context window
Claude Fable 5
1,000,000 tokens
Higher supports larger inputs
Context window
Claude Opus 4.8
1,000,000 tokens
Higher supports larger inputs
Context window
Gemini 3.5 Flash
1,048,576 tokens
Higher supports larger inputs
Context window
Gemini 3.1 Flash-Lite
1,048,576 tokens
Higher supports larger inputs
Context window
GPT-5.5
1,050,000 tokens
Higher supports larger inputs
Context window
Grok 4.20
1,000,000 tokens
Higher supports larger inputs
Context window
GPT-5.4
1,050,000 tokens
Higher supports larger inputs
Context window
Gemini 3.1 Pro Preview
1,048,576 tokens
Higher supports larger inputs
Context window
Gemini 3 Flash Preview
1,048,576 tokens
Higher supports larger inputs
Context window
Kimi K2 Base
128,000 tokens
Higher supports larger inputs
Context window
Kimi K2 Instruct
128,000 tokens
Higher supports larger inputs
Context window
GPT-4.1
1,047,576 tokens
Higher supports larger inputs
Context window
GPT-4.1 mini
1,047,576 tokens
Higher supports larger inputs
Context window
GPT-4.1 nano
1,047,576 tokens
Higher supports larger inputs
Context window
Kimi-VL-A3B-Instruct
128,000 tokens
Higher supports larger inputs
Context window
Gemini 2.5 Pro
1,048,576 tokens
Higher supports larger inputs
Context window
Qwen2.5-VL 72B Instruct
128,000 tokens
Higher supports larger inputs
Context window
DeepSeek-R1
128,000 tokens
Higher supports larger inputs
Context window
Grok 4.3
1,000,000 tokens
Higher supports larger inputs
Arena Text · 39 models · Measured 2026-10-08
Benchmark observations by release
Arena Text, snapshot Arena leaderboard published 2026-10-08, measured 2026-10-08. Scores are shown from newest release to oldest. Eligible models (39): Mistral Large 4, GPT-6.1 Sol, Claude Sonnet 5.5, Claude Opus 5.5, GPT-6 Luna, GPT-6 Sol, Grok 4.7, DeepSeek-V4.1-Flash, GPT-6 Astra, Gemini 3.8 Flash, Muse Spark 1.3, Claude Fable 5.1, Gemini 3.7 Flash, Grok 4.6, Qwen3.8-Max, Claude Opus 5, Gemini 3.5 Flash-Lite, Gemini 3.6 Flash, Grok 4.5, Kimi K3, GPT-5.6 Luna, GPT-5.6 Sol, GPT-5.6 Terra, Claude Sonnet 5, Claude Fable 5, Claude Opus 4.8, Gemini 3.5 Flash, GPT-5.5, Grok 4.20, GPT-5.4, Gemini 3.1 Pro Preview, Gemini 3 Flash Preview, Kimi K2 Instruct, GPT-4.1, GPT-4.1 mini, GPT-4.1 nano, Gemini 2.5 Pro, DeepSeek-R1, Grok 4.3.
Takeaway: Claude Opus 5.5, GPT-6 Luna, DeepSeek-V4.1-Flash, Gemini 3.8 Flash, GPT-5.6 Luna are not dominated on both published 1M input tokens price and this direct benchmark score among the 34 eligible models. Higher benchmark scores are better.
Prices are provider list prices verified 2026-10-12.
Side-by-side comparison
Compare 2–4 models using directly sourced capability, cost, and access facts.
Select models from the active task. A shared benchmark score is shown only when an audited snapshot covers every selected model.
How to use this comparison
01Filter or search for the models that fit your workload.
02Pick them in the selector above, or Select Compare on a row in the inventory.
03Read the matrix, source evidence, and trade-offs before choosing a default.
0 models selected for comparison
Pick 2 more models above to build the matrix.
Methodology: models with missing scores or prices are omitted from the applicable visual. Prices are provider list prices at the dataset verification date. Benchmark results should not be compared across dataset versions.
Van Data Team analysis
Recommended by use case
Editorial shortlists for evaluation, not automatic winners. Dataset 2026-10-12, last verified 2026-10-12.
AI agents and tool use
GPT-5.6 Sol, Claude Sonnet 5, Gemini 3.6 Flash
Start a current agent-workflow evaluation with GPT-5.6 Sol, Claude Sonnet 5, and Gemini 3.6 Flash.
Review rationale and evidence
Why
All three active API records are positioned by their providers for agentic or tool-using workflows.
GPT-5.6 Sol and Gemini 3.6 Flash publish context windows above one million tokens, while Claude Sonnet 5 launches with a lower introductory input price than GPT-5.6 Sol.
Watch for
The pinned LiveBench snapshot predates GPT-5.6 Sol and Gemini 3.6 Flash and does not measure tool-use success, latency, uptime, or end-to-end agent reliability; Claude Sonnet 5 pricing changes after 2026-08-31.
Fits when: Test the models with the actual tools, permissions, retry rules, and human approval steps used in production.
Compare GPT-5.5 and Claude Fable 5 for current high-capability analysis, then test both against your review standard.
Review rationale and evidence
Why
Both providers position these active API records for demanding reasoning or professional knowledge work.
Both records publish context windows of at least 1,000,000 tokens.
Watch for
The pinned LiveBench snapshot does not provide a compatible Claude Fable 5 result and does not establish factuality, domain expertise, citation quality, latency, or production reliability.
Fits when: Use representative documents, expected citations, and a defined human review rubric for the production trial.
Open only the methodology, production guidance, release history, sources, or answers you need. Complete records remain available in the page HTML.
How to read benchmark data
How to read the benchmark data
Use the catalog to screen candidates, then verify them on your workload. Linked official sources support each record as a whole; they are not a field-by-field audit trail. Benchmark results retain their publisher and compatible snapshot.
Keep task families separate
Scores and operational metrics should not be compared across task families. Text reasoning, image quality, video quality, and speech error metrics answer different questions.
Match one benchmark snapshot
Compare results only when the task family, benchmark version, measurement date, and eligible model set match. Different snapshots may use different prompts, evaluators, or datasets.
Follow the score direction
Higher-is-better metrics reward a larger result; lower-is-better metrics such as error rate reward a smaller result. The direction belongs to the named benchmark, not to every task.
Match the pricing unit
A price per million tokens, image, audio minute, video second, or realtime minute is a different cost basis. Compare only the same unit and tier, then add retries, review, tooling, and infrastructure.
Check lifecycle and source status
Current models are active provider offerings; Legacy models remain for historical context. Official records come from providers, while independent results come from benchmark publishers, each with a visible verification date.
Keep missing data missing
Not reported is not zero. The explorer excludes missing values from rankings and does not infer them from a related model.
Scope begins 2024-01-01. Dataset version 2026-10-12. Last verified 2026-10-12. 144 source records.
Production fit
Why a benchmark winner may not fit production
Benchmark rank and production fit measure different things. A result can support a shortlist without deciding the system design.
A public score may cover one narrow task while your system combines retrieval, tools, permissions, and review.
Production acceptance also depends on latency, reliability, safety, data handling, regional availability, and total cost.
Van Data Team recommendations are transparent editorial shortlists. They use published fields and results, apply no hidden weighting, and must be tested on representative work.
Release timeline
Model release timeline since 2024
Generated from exact explorer release dates. Models without a verified exact date remain in the inventory and are omitted here rather than assigned an inferred date.
2026 Q4
Claude Haiku 5.52026-10-07
Mistral Large 42026-10-06
Nano Banana 2.12026-10-06
2026 Q3
GPT-6.1 Sol2026-09-29
Claude Sonnet 5.52026-09-28
Claude Opus 5.52026-09-22
Gemini 3.8 Flash TTS2026-09-22
Gemini 3.8 Flash-Lite TTS2026-09-22
GPT-6 Luna2026-09-22
GPT-6 Sol2026-09-22
Grok 4.72026-09-21
Qwen3.8-Omni-Flash2026-09-20
Gemini 3.8 Live2026-09-15
DeepSeek-V4.1-Flash2026-09-10
GPT-Live 12026-09-10
GPT Image 2.5 Flare2026-09-08
GPT Image 2.5 Sunburst2026-09-08
GPT-6 Astra2026-09-03
Gemini 3.8 Flash2026-09-02
Muse Spark 1.32026-09-02
Claude Fable 5.12026-09-01
Claude Mythos 5.12026-09-01
Gemini 3.7 Flash2026-08-13
Grok 4.62026-08-12
Qwen3.8-Max2026-08-03
Claude Opus 52026-07-24
Gemini 3.5 Flash-Lite2026-07-21
Gemini 3.6 Flash2026-07-21
Grok 4.52026-07-16
Kimi K32026-07-16
GPT-5.6 Luna2026-07-09
GPT-5.6 Sol2026-07-09
GPT-5.6 Terra2026-07-09
2026 Q2
Claude Sonnet 52026-06-30
Gemini 3.1 Flash Lite Image2026-06-30
Gemini Omni Flash Preview2026-06-30
Claude Fable 52026-06-09
Claude Opus 4.82026-05-28
Gemini 3 Pro Image2026-05-28
Gemini 3.1 Flash Image2026-05-28
Gemini 3.5 Flash2026-05-19
Gemini 3.1 Flash-Lite2026-05-07
GPT-5.52026-04-23
Gemini 3.1 Flash TTS Preview2026-04-15
2026 Q1
Gemini 3.1 Flash Live Preview2026-03-26
Grok 4.202026-03-10
GPT-5.42026-03-05
Gemini 3.1 Pro Preview2026-02-19
Scribe v22026-01-09
2025 Q4
Gemini 3 Flash Preview2025-12-17
Scribe v2 Realtime2025-11-11
Veo 3.1 Preview2025-10-15
2025 Q3
Kimi K2 Base2025-07-11
Kimi K2 Instruct2025-07-11
2025 Q2
Kimi-Dev-72B2025-06-17
GPT-4.12025-04-14
GPT-4.1 mini2025-04-14
GPT-4.1 nano2025-04-14
Kimi-VL-A3B-Instruct2025-04-09
2025 Q1
Gemini 2.5 Pro2025-03-25
Claude 3.7 Sonnet2025-02-24
Grok 32025-02-19
Qwen2.5-VL 72B Instruct2025-01-26
DeepSeek-R12025-01-20
2024 Q4
DeepSeek-V32024-12-26
Gemini 2.0 Flash2024-12-11
Ministral 8B2024-10-16
2024 Q3
Llama 3.2 11B Vision2024-09-25
Qwen2.5 32B Instruct2024-09-19
Pixtral 12B2024-09-17
Grok 22024-08-13
Mistral Large 22024-07-24
Llama 3.1 405B2024-07-23
2024 Q2
DeepSeek-Coder-V2 Instruct2024-06-17
Qwen2 72B Instruct2024-06-07
Qwen2 7B Instruct2024-06-07
Codestral 22B2024-05-29
Gemini 1.5 Flash2024-05-14
GPT-4o2024-05-13
DeepSeek-V2 Chat2024-05-06
Llama 3 70B2024-04-18
Llama 3 8B2024-04-18
Grok-1.5V2024-04-12
2024 Q1
Grok-1.52024-03-28
Claude 3 Haiku2024-03-13
Claude 3 Opus2024-03-04
Claude 3 Sonnet2024-03-04
Gemini 1.5 Pro2024-02-15
Source catalog · 144 records
Source catalog
Official sources come from model providers. Independent sources come from a separate benchmark publisher. Each row shows its publisher, verification date, version, license status, and coverage; missing metadata is not filled or inferred.
Full-duplex voice model; audio and text in and out; $0.05 per voice-session minute billed per second; reasoning and tools delegated to a separately billed backend model.
Official release announcement verifies Claude Fable 5 identity, 2026-06-09 release date, API access, standard token pricing, and general availability after redeployment.
Official current-model documentation verifies Fable's text-and-image inputs, 1,000,000-token context window, 128,000-token output limit, and current API availability.
Official release notes verify the dated Gemini 3.5 Flash, Gemini 3.1 Flash Live, Gemini 3.1 Flash TTS, Gemini 3.1 Flash Lite Image, Gemini 3 Pro Image, and Gemini Omni Flash releases represented in this catalog.
Official Gemini Developer API pricing table used only for exact published token, image, audio-minute, and video-second prices represented in model records.
Official model documentation verifies the stable Gemini 3.7 Flash code, text/image/video/audio inputs, text output, token limits, structured outputs, thinking, and URL context.
Official Google announcement verifies the 2026-08-13 release, coding and agent positioning, and introductory token prices through 2026-12-31 before the documented 2027 rate change.
Model card verifies the Gemini 3.1 Flash Live preview endpoint, supported realtime multimodal inputs, text and audio output, token limits, and capabilities.
Model card verifies the stable Gemini 3.1 Flash Lite Image endpoint, text and image inputs, text and image output, token limits, and image generation and editing capabilities.
Model card verifies the Gemini Omni Flash preview endpoint, text, image, and video inputs, video output, context window, and 3-to-10-second output range.
Official API reference exposes Grok 4.3 modalities, aliases, and token pricing fields; its object creation timestamp is not treated as a public release date.
Official 2026-05-18 announcement confirms Grok 4.3 was live across grok.com, iOS, and Android by that date; it is not treated as the model's exact release date.
Official current pricing table verifies active Grok 4.6, Grok 4.5, and Grok 4.20 variants, their documented context windows, and standard and long-context token rates.
Official model documentation verifies modalities, context window, and standard short-context token prices; the model caution discloses higher long-context rates.
Official model documentation verifies the exact model ID, text and image input, text output, 500,000-token context window, reasoning, function calling, structured outputs, and standard token prices.
Pinned implementation averages task scores within each of the seven categories, then averages the category scores; the paired categories_2026_06_25.json file defines category membership.
Snapshot and measurement date: 2026-06-25. Overall scores are imported only for unambiguous model mappings and are recomputed with the pinned category-weighted averaging implementation.
Snapshot and measurement date: 2024-11-25. Exact published snapshot; only direct Web of Lies V2 percentages with unambiguous model mappings are imported.
Results Epoch AI ran itself (the GPQA Diamond, FrontierMath Tiers 1-3 v2 and SimpleQA Verified files), not the external files it collects from other publishers. A model maps only when its Epoch row is unambiguous: exact name, or a dated version pinned by the record release date. Where Epoch lists several reasoning-effort settings, the best published configuration is used and its row is recorded in sourceRow. Scores are converted from 0-1 to 0-100 and rounded to one decimal. Epoch evaluated rows between 2023 and 2026-10-09; every result is dated to this capture. Credit: Epoch AI, Capabilities and benchmarking, epoch.ai, CC BY 4.0.
198 four-option graduate-level questions in biology, physics and chemistry, written by domain experts; random guessing scores 25%. Epoch runs the evaluation itself.
Original research-level mathematics problems written and vetted by expert mathematicians. Tiers 1-3 v2 (released 2026-06-12) is the 295-problem base set after a revision that corrected errors in 42% of problems. Scores come from Epoch-run private sets. OpenAI funded the benchmark and has access to part of it.
A 1,000-question factoid benchmark from Google, building on SimpleQA from OpenAI. Epoch runs it itself and reports the proportion answered correctly, not the F1-based score Google reports.
Ratings from blind pairwise human votes, published by Arena (formerly LMArena) in its public leaderboard dataset and pinned here to one commit. The overall category of the latest split is used; for text, the style-controlled leaderboard, which is the default view on arena.ai. A model maps only when its Arena row names the same model; where Arena lists several quality or resolution settings of one model, the best rated row is used and recorded in sourceRow. Ratings are rounded to whole points. Credit: Arena, lmarena-ai/leaderboard-dataset on Hugging Face, CC BY 4.0.
People compare two anonymous images generated from the same prompt and vote for the better one. Arena fits a Bradley-Terry model to the votes and reports ratings on an Elo-like scale.
People compare two anonymous edits of the same image from the same instruction and vote for the better one. Arena fits a Bradley-Terry model to the votes and reports ratings on an Elo-like scale.
People compare two anonymous videos generated from the same prompt and vote for the better one. Arena fits a Bradley-Terry model to the votes and reports ratings on an Elo-like scale.
People compare two anonymous videos animated from the same image and prompt and vote for the better one. Arena fits a Bradley-Terry model to the votes and reports ratings on an Elo-like scale.
People put the same prompt to two anonymous models and vote for the better answer. Arena fits a Bradley-Terry model to the votes; the rating used here is the style-controlled one, which is the default view on arena.ai.
FAQ
AI model benchmark FAQ
What does an AI model benchmark score mean?
A benchmark score reports performance on one defined evaluation. Compare scores only when the task family, benchmark version, measurement date, scoring method, and eligible model set match; the result is a screening signal, not a prediction for every workload.
How fresh are the model and source records?
Each source shows a verification date, and the page shows the dataset version and last verified date. A verification date confirms when Van Data Team checked the linked record; it does not promise that a provider page has stayed unchanged since then.
Why are some model values marked Not reported?
Not reported means the catalog does not have a compatible, directly sourced value for that exact model record. Missing values remain empty rather than being estimated, copied from a related model, or treated as zero.
How should I compare AI model pricing?
Compare only the same pricing unit and tier, such as a million tokens, image, audio minute, video second, character, or realtime minute. Prices are provider list prices checked on the source verification date, not live quotes; retries, review, tooling, and infrastructure also affect production cost.
Does open-weight access mean a model is free to run?
No. Open weights means the model parameters are available for deployment under the provider license. This can provide infrastructure control, but compute, serving, monitoring, security, and engineering still carry costs.
How should I choose models for a production evaluation?
Start with the use case, select a small evidence-backed shortlist, and test it on representative work. Set acceptance thresholds for quality, latency, cost, reliability, safety, and human review before comparing results.
Where do the benchmark scores come from, and how often are they updated?
Scores come only from independent publishers that release reproducible results: Epoch AI runs GPQA Diamond, FrontierMath and SimpleQA Verified itself, and LiveBench publishes its task results. Vendor launch numbers are never mixed in. Where a model was tested at several reasoning-effort settings, the best published setting is shown with its exact leaderboard row. The dataset is updated when a review verifies material model, price, or benchmark changes, and the last verified date is the update record.
How can Van Data Team help with model evaluation?
Van Data Team can turn a shortlist into a production evaluation with representative cases, measurable acceptance rules, model routing, workflow controls, observability, and cost tracking tailored to your system.
Production evaluation
Turn a model shortlist into a production decision
We design representative evaluations and the workflow controls needed to operate the selected model with clear quality, cost, and review boundaries.
Evaluation cases and acceptance thresholds tied to real work
Model routing, tool permissions, observability, and human review design
