Dataset

LLM benchmark scores by model, since 2019

Public

6,843 distinct values, no chart

  • GPQA diamond5%
  • OTIS Mock AIME 2024-20254%
  • ARC-AGI4%
  • Other (70)
Timeline per year: 2016-01-01, 80; 2017-01-01, 115; 2018-01-01, 213; 2019-01-01, 913; 2020-01-01, 249; 2021-01-01, 343; 2022-01-01, 206; 2023-01-01, 319; 2024-01-01, 1,119; 2025-01-01, 1,970; 2026-01-01, 1,301
2016-01-012026-01-01

per year

  • GPT-5.6 Sol1%
  • GPT-5.51%
  • GPT-5.21%
  • Other (698)

1,176 distinct values, no chart

  • OpenAI28%
  • Anthropic17%
  • Google DeepMind10%
  • Other (51)
  • United States of America75%
  • China19%
  • France3%
  • API access62%
  • Open weights (unrestricted)20%
  • Open weights (restricted use)10%
Timeline per year: 2019-01-01, 15; 2020-01-01, 13; 2021-01-01, 72; 2022-01-01, 176; 2023-01-01, 905; 2024-01-01, 805; 2025-01-01, 1,918; 2026-01-01, 2,565
2019-01-012026-01-01

per year

150 distinct values, no chart

3,630 distinct values, no chart

  • fraction88%
  • percent5%
  • Elo rating2%
Histogram: 0 to 0.10000000000000001, 807; 0.10000000000000001 to 0.20000000000000001, 520; 0.20000000000000001 to 0.29999999999999999, 565; 0.29999999999999999 to 0.40000000000000002, 611; 0.40000000000000002 to 0.5, 690; 0.5 to 0.59999999999999998, 724; 0.59999999999999998 to 0.69999999999999996, 637; 0.69999999999999996 to 0.80000000000000004, 804; 0.80000000000000004 to 0.90000000000000002, 690; 0.90000000000000002 to 1, 368
01
  • true100%
  • false0%

1,235 distinct values, no chart

Timeline per quarter: 2024-10-01, 38; 2025-01-01, 247; 2025-04-01, 130; 2025-07-01, 116; 2025-10-01, 257; 2026-01-01, 156; 2026-04-01, 202; 2026-07-01, 905
2024-10-012026-07-01

per quarter

  • Leaderboard41%
  • Epoch AI24%
  • Paper or report21%
  • Epoch AI28%
  • conceptualreasoning.ai7%
  • artificialanalysis.ai3%
  • Other (94)

1,102 distinct values, no chart

  • Shots: 518%
  • Shots: 014%
  • Shots: 14%
  • Other (206)

74 distinct values, no chart

82 distinct values, no chart

262FWwUwrjPpxVN8G8sDZTMystery Game Puzzles2026-07-30GPT-5gpt-5-2025-08-07_lowOpenAIUnited States of AmericaAPI access2025-08-076.6000000000000002e+250.16fraction0.16true0.036845294917747062026-08-27Epoch AIEpoch AI∅∅∅mystery_game_puzzles.csv
26q7yB8XHAcmxWgEMFPMHPChess Puzzles2025-12-10Nemotron 3 Ultranemotron-3-ultraNvidiaUnited States of AmericaOpen weights (unrestricted)2026-06-046.5999999999999998e+240.12fraction0.12true0.0326598632371090452026-08-10Epoch AIEpoch AI∅∅chess_puzzles.csv
272CatAfNpHUw44w7oJ2MMChess Puzzles2025-12-10Gemma 4 26B A4Bgemma-4-26b-a4b-it_minimalGoogle DeepMindUnited States of AmericaOpen weights (unrestricted)2026-04-02∅0.059999999999999998fraction0.059999999999999998true0.0238683256575942012026-08-06Epoch AIEpoch AI∅∅chess_puzzles.csv
298stsbTPpZp7toZsS5cp8OTIS Mock AIME 2024-20252024-12-19GPT-5 nanogpt-5-nano-2025-08-07_lowOpenAIUnited States of AmericaAPI access2025-08-07∅0.46666666666666667fraction0.46666666666666667true0.0752101433090354852026-07-13Epoch AIEpoch AI∅∅otis_mock_aime_2024_2025.csv
2AbeCjy9w4KJWKhbKjtFTJMystery Game Puzzles2026-07-30Claude Opus 4.1claude-opus-4-1-20250805_24KAnthropicUnited States of AmericaAPI access2025-08-05∅0.20999999999999999fraction0.20999999999999999true0.0409360180740332362026-07-25Epoch AIEpoch AI∅∅∅mystery_game_puzzles.csv
2AxSH8gojpyyzADeWfuSquChess Puzzles2025-12-10DeepSeek-R1-Distill-Qwen-14BDeepSeek-R1-Distill-Qwen-14BDeepSeekChinaOpen weights (unrestricted)2025-01-20∅0.01fraction0.01true0.00999999999999999852026-08-28Epoch AIEpoch AI∅∅chess_puzzles.csv
2CdomqtBVz2it7a2Zx2G2tGPQA diamond2023-11-20GPT-5.5gpt-5.5_lowOpenAIUnited States of AmericaAPI access2026-04-23∅0.90656565656565657fraction0.90656565656565657true0.0175326583777217082026-05-05Epoch AIEpoch AI∅∅gpqa_diamond.csv
2RiSB2ii5kf2Q6fDBv2P3EMATH level 52021-03-05o3-minio3-mini-2025-01-31_mediumOpenAIUnited States of AmericaAPI access2025-01-31∅0.95166163141993954fraction0.95166163141993954true0.00465191785762157812025-01-31Epoch AIEpoch AI∅∅∅math_level_5.csv
2UmuBfhBvYbbnTg5BnQqovFrontierMath-Tier-4-v2-Private2026-06-12GPT-5.5 Instantgpt-5.5-instantOpenAIUnited States of AmericaAPI access2026-05-05∅0.024390243902439025fraction0.024390243902439025true0.0243902439024390222026-08-02Epoch AIEpoch AI∅∅frontiermath_tier_4_v2.csv
2WxRbjq5ZWTgiXZPqYUG4RFrontierMath-Tier-4-v2-Private2026-06-12GPT-5 minigpt-5-mini-2025-08-07_highOpenAIUnited States of AmericaAPI access2025-08-07∅0.12195121951219512fraction0.12195121951219512true0.0517395205746254352026-06-12Epoch AIEpoch AI∅∅frontiermath_tier_4_v2.csv
2a2uhidAfksbnAJC8yUrZKMystery Game Puzzles2026-07-30GPT-5.1gpt-5.1-2025-11-13_noneOpenAIUnited States of AmericaAPI access2025-11-13∅0.14999999999999999fraction0.14999999999999999true0.0358870281282636792026-08-27Epoch AIEpoch AI∅∅∅mystery_game_puzzles.csv
2c4rvmf5N7QXTugrFLMZ83SimpleQA Verified2025-09-09Claude Opus 4.7claude-opus-4-7_xhighAnthropicUnited States of AmericaAPI access2026-04-16∅0.51700000000000002fraction0.51700000000000002true0.0158101537298332742026-08-27Epoch AIEpoch AI∅∅simpleqa_verified.csv
2e2NePZeU3iB4wvhwCaTciMystery Game Puzzles2026-07-30GPT-5.4 Nanogpt-5.4-nano-2026-03-17_highOpenAIUnited States of AmericaAPI access2026-03-17∅0.050000000000000003fraction0.050000000000000003true0.0219042913557590332026-08-27Epoch AIEpoch AI∅∅∅mystery_game_puzzles.csv
2eMHpXXR9h7P2oNPDQkTBYFrontierMath-2025-02-28-Private2025-02-28GPT-5 nanogpt-5-nano-2025-08-07_highOpenAIUnited States of AmericaAPI access2025-08-07∅0.082758620689655171fraction0.082758620689655171true0.0162068838583676912025-10-30Epoch AIEpoch AI∅∅∅frontiermath.csv
2ekT2UzaebvmGprKvmeeEaFrontierMath-2025-02-28-Private2025-02-28o1-minio1-mini-2024-09-12_highOpenAIUnited States of AmericaAPI access2024-09-12∅0.013793103448275862fraction0.013793103448275862true0.00686066309342302662025-03-06Epoch AIEpoch AI∅∅∅frontiermath.csv
2f8Vn5QBdJnc9i5y9jzXS7OTIS Mock AIME 2024-20252024-12-19Qwen3.8 Max (0902)qwen3.8-max-0902_xhighAlibabaChinaAPI access2026-09-01∅1fraction1true02026-09-02Epoch AIEpoch AI∅∅∅otis_mock_aime_2024_2025.csv
2fQJP3brRApfBdQq7Zy3bmOTIS Mock AIME 2024-20252024-12-19Llama 3.1-405BLlama-3.1-405B-InstructMeta AIUnited States of AmericaOpen weights (restricted use)2024-07-233.8e+250.097222222222222224fraction0.097222222222222224true0.032186446091623392025-02-25Epoch AIEpoch AI∅∅otis_mock_aime_2024_2025.csv
2gzNF2d53h2oCKDnojLtSPGPQA diamond2023-11-20Qwen3-30B-A3B-Thinking (Jul 2025)qwen3-30b-a3b-thinking-2507AlibabaChinaOpen weights (unrestricted)2025-07-306.4800000000000005e+230.7007575757575758fraction0.7007575757575758true0.0283278976135454932026-08-30Epoch AIEpoch AI∅∅gpqa_diamond.csv
2hH2GAPUFaJR7pfVKhhicgSWE-Bench verified2024-08-13Claude Opus 4.6claude-opus-4-6AnthropicUnited States of AmericaAPI access2026-02-05∅0.78719008264462809fraction0.78719008264462809true0.018623537303052882026-02-18Epoch AIEpoch AI∅∅swe_bench_verified.csv
2hHzyijzk3jKujCSxd5d2sChess Puzzles2025-12-10DeepSeek-V4-Prodeepseek-v4-pro_noneDeepSeekChinaOpen weights (unrestricted)2026-04-249.7019999999999994e+240.070000000000000007fraction0.070000000000000007true0.025643239997624282026-08-06Epoch AIEpoch AI∅∅chess_puzzles.csv
2hbbp8z6PhB7c2T33os9q5FrontierMath-Tier-4-v2-Private2026-06-12Claude Opus 4.6claude-opus-4-6_maxAnthropicUnited States of AmericaAPI access2026-02-05∅0.26829268292682928fraction0.26829268292682928true0.0700556420309515762026-06-11Epoch AIEpoch AI∅∅frontiermath_tier_4_v2.csv
2jTRxz7rW3B66a8smtz7YaChess Puzzles2025-12-10Claude Opus 4.5claude-opus-4-5-20251101AnthropicUnited States of AmericaAPI access2025-11-24∅0.040000000000000001fraction0.040000000000000001true0.0196946385566932372026-08-06Epoch AIEpoch AI∅∅chess_puzzles.csv
2mepC8oUjPQRjq4XcBMpbZChess Puzzles2025-12-10Qwen2.5-32Bqwen2.5-32b-instructAlibabaChinaOpen weights (unrestricted)2024-09-173.5099999999999999e+240fraction0true02026-08-30Epoch AIEpoch AI∅∅chess_puzzles.csv
2o8XZ8V6W5jKD7TuY4fruxFrontierMath-Tier-4-2025-07-01-Private2025-07-11Claude Sonnet 4.5claude-sonnet-4-5-20250929AnthropicUnited States of AmericaAPI access2025-09-29∅0.020833333333333332fraction0.020833333333333332true0.0208333333333333322025-09-29Epoch AIEpoch AI∅∅∅frontiermath_tier_4.csv
2pSDjMSGDh4HsAxkqUuNQPGPQA diamond2023-11-20GPT-5 nanogpt-5-nano-2025-08-07_highOpenAIUnited States of AmericaAPI access2025-08-07∅0.69444444444444442fraction0.69444444444444442true0.0275945089982510452025-10-30Epoch AIEpoch AI∅∅gpqa_diamond.csv
2t26VSNdsAE5pefQEMP25xGPQA diamond2023-11-20Claude 3.7 Sonnetclaude-3-7-sonnet-20250219_64KAnthropicUnited States of AmericaAPI access2025-02-243.3500000000000002e+250.79734848484848486fraction0.79734848484848486true0.0265068831230600642025-05-26Epoch AIEpoch AI∅∅gpqa_diamond.csv
2vg9qw9pd8zSyRMu9YNiVmOTIS Mock AIME 2024-20252024-12-19Claude 2.1claude-2.1AnthropicUnited States of AmericaAPI access2023-11-21∅0.019444444444444445fraction0.019444444444444445true0.0104775158766286622025-03-07Epoch AIEpoch AI∅∅otis_mock_aime_2024_2025.csv
2wd9khVKiBzsAMUquz2R5sMATH level 52021-03-05DeepSeek-R1 (May 2025)DeepSeek-R1-0528DeepSeekChinaOpen weights (unrestricted)2025-05-284.0200100000000008e+240.96638972809667678fraction0.96638972809667678true0.00421561617356365152025-05-29Epoch AIEpoch AI∅∅∅math_level_5.csv
2xMn6dbY9mw4pmkh8Tz9GnOTIS Mock AIME 2024-20252024-12-19QwQ-32BQwQ-32BAlibabaChinaOpen weights (unrestricted)2025-03-053.5099999999999999e+240.59166666666666667fraction0.59166666666666667true0.0634896396543116472026-08-28Epoch AIEpoch AI∅∅otis_mock_aime_2024_2025.csv
2xNPhKLsiWLHQBLLDGU4VJOTIS Mock AIME 2024-20252024-12-19Claude 3.5 Sonnetclaude-3-5-sonnet-20240620AnthropicUnited States of AmericaAPI access2024-06-202.7000000000000009e+250.065277777777777782fraction0.065277777777777782true0.0222931318156600162025-02-25Epoch AIEpoch AI∅∅otis_mock_aime_2024_2025.csv
2xfFf6KZAp2gUWYhHGkFgsSimpleQA Verified2025-09-09DeepSeek V4 Pro 0813deepseek-v4-pro-0813_maxDeepSeekChinaOpen weights (unrestricted)2026-08-139.7019999999999994e+240.5290581162324649fraction0.5290581162324649true0.0158083946657065972026-08-27Epoch AIEpoch AI∅∅simpleqa_verified.csv
2y5jXmHGjyEyZvGjNDtRZeMystery Game Puzzles2026-07-30Qwen 3.6 Flashqwen3.6-flashAlibabaChina∅2026-04-27∅0.12fraction0.12true0.0326598632371090382026-08-27Epoch AIEpoch AI∅∅∅mystery_game_puzzles.csv
2z7kL9DpfvY44T9Wo99FYxFrontierMath-Tier-4-v2-Private2026-06-12Gemini 2.5 Pro (Jun 2025)gemini-2.5-proGoogle DeepMindUnited States of AmericaAPI access2025-06-17∅0fraction0true02026-06-11Epoch AIEpoch AI∅∅frontiermath_tier_4_v2.csv
3542DFKGpjD6k8AZfGKD7WEBR-bench2026-07-01GPT-5.2gpt-5.2-2025-12-11_xhighOpenAIUnited States of AmericaAPI access2025-12-11∅0.230159fraction0.230159true∅2026-06-26Epoch AIEpoch AI∅∅∅ebr_bench.csv
35hRAbkceJM3Dpb2pWXnx9Chess Puzzles2025-12-10Kimi K3kimi-k3_maxMoonshotChinaOpen weights (non-commercial)2026-07-162.0000999999999998e+250.39000000000000001fraction0.39000000000000001true0.0490207130000197282026-07-16Epoch AIEpoch AI∅∅∅chess_puzzles.csv
37pJqPpXTE6ia9ikmmGPJ4OTIS Mock AIME 2024-20252024-12-19Claude Opus 4.8claude-opus-4-8_maxAnthropicUnited States of AmericaAPI access2026-05-28∅0.98333333333333328fraction0.98333333333333328true0.0141019018704441542026-06-07Epoch AIEpoch AI∅∅∅otis_mock_aime_2024_2025.csv
3CGChL5vNDM6Df5C8Z8owNOTIS Mock AIME 2024-20252024-12-19Gemma 3 1Bgemma-3-1b-itGoogle DeepMindUnited States of AmericaOpen weights (restricted use)2025-03-121.2e+220.011111111111111112fraction0.011111111111111112true0.00536281832868568122026-08-28Epoch AIEpoch AI∅∅otis_mock_aime_2024_2025.csv
3D8aaFBLZwQcnWHYKwvnmqOTIS Mock AIME 2024-20252024-12-19GPT-5gpt-5-2025-08-07_highOpenAIUnited States of AmericaAPI access2025-08-076.6000000000000002e+250.91388888888888886fraction0.91388888888888886true0.0365887218675541372025-10-29Epoch AIEpoch AI∅∅otis_mock_aime_2024_2025.csv
3GzYKQW2K4iijm6a73rVehFrontierMath-2025-02-28-Private2025-02-28Claude Sonnet 4.5claude-sonnet-4-5-20250929_59KAnthropicUnited States of AmericaAPI access2025-09-29∅0.13494809688581316fraction0.13494809688581316true0.0201329873756760922025-11-13Epoch AIEpoch AI∅∅∅frontiermath.csv
3L22tfAid5ZM42p92tqC73Mystery Game Puzzles2026-07-30Gemini 3.6 Flashgemini-3.6-flash_minimalGoogle DeepMindUnited States of AmericaAPI access2026-07-21∅0.25fraction0.25true0.0435194139889244632026-08-05Epoch AIEpoch AI∅∅∅mystery_game_puzzles.csv
3NYyCGnM6XhgytjyVGtwh8MATH level 52021-03-05GPT-5 minigpt-5-mini-2025-08-07_mediumOpenAIUnited States of AmericaAPI access2025-08-07∅0.9679003021148036fraction0.9679003021148036true0.0038534757473536612025-08-20Epoch AIEpoch AI∅∅∅math_level_5.csv
3PVdaSEYFJKgYUgHnF4DBuMystery Game Puzzles2026-07-30Claude Sonnet 5.5claude-sonnet-5-5_maxAnthropicUnited States of AmericaAPI access2026-09-28∅0.65000000000000002fraction0.65000000000000002true0.0479372485441102172026-09-29Epoch AIEpoch AI∅∅∅mystery_game_puzzles.csv
3Q6osT8EYezayG7yfxkSprGPQA diamond2023-11-20GPT-5.2gpt-5.2-2025-12-11_highOpenAIUnited States of AmericaAPI access2025-12-11∅0.88194444444444442fraction0.88194444444444442true0.0191385827763392412025-12-11Epoch AIEpoch AI∅∅gpqa_diamond.csv
3QBsxKkbF2R2LEXTd6HDrJSimpleQA Verified2025-09-09GPT-5.4 Nanogpt-5.4-nano-2026-03-17_highOpenAIUnited States of AmericaAPI access2026-03-17∅0.11700000000000001fraction0.11700000000000001true0.0101692878027133452026-08-27Epoch AIEpoch AI∅∅simpleqa_verified.csv
3QjP5HPi8tKUzHtNJktfv4OTIS Mock AIME 2024-20252024-12-19gpt-oss-120bopenai/gpt-oss-120b_highOpenAIUnited States of AmericaOpen weights (unrestricted)2025-08-054.9400000000000001e+240.88888888888888884fraction0.88888888888888884true0.0446334365561327292025-12-11Epoch AIEpoch AI∅∅otis_mock_aime_2024_2025.csv
3QtkFWzSLzd4hgYGsNd5cLMystery Game Puzzles2026-07-30Qwen 3.6 Flashqwen3.6-flash_noneAlibabaChina∅2026-04-27∅0.17999999999999999fraction0.17999999999999999true0.0386122919665369132026-08-27Epoch AIEpoch AI∅∅∅mystery_game_puzzles.csv
3S6fjGAMKWKK3zcdxkmh9HOTIS Mock AIME 2024-20252024-12-19Gemini 1.5 Pro (May 2024)gemini-1.5-pro-001Google DeepMindUnited States of AmericaAPI access2024-05-14∅0.06805555555555555fraction0.06805555555555555true0.0189303429551487792025-02-25Epoch AIEpoch AI∅∅otis_mock_aime_2024_2025.csv
3UbYVQELDHAcQgu7sTTnRfGPQA diamond2023-11-20Magistral Small 1.2magistral-small-2509Mistral AIFranceOpen weights (unrestricted)2025-09-18∅0.47601010101010099fraction0.47601010101010099true0.0259934626984533982026-08-30Epoch AIEpoch AI∅∅gpqa_diamond.csv
3UzTUnn6wx79jU3XoP7VFAChess Puzzles2025-12-10Qwen3-1.7Bqwen3-1.7b_noneAlibabaChinaOpen weights (unrestricted)2025-04-293.6719999999999997e+230fraction0true02026-08-28Epoch AIEpoch AI∅∅chess_puzzles.csv
3VDhPT5WvzjJZkRc36KuviMATH level 52021-03-05o3-minio3-mini-2025-01-31_highOpenAIUnited States of AmericaAPI access2025-01-31∅0.96487915407854985fraction0.96487915407854985true0.00426526346198466932025-02-13Epoch AIEpoch AI∅∅∅math_level_5.csv
Page 1 of 137
Rows
Updated 6,843 rows
Searches benchmark, model, model_version, organization, country, model_access, reported_by, source, evaluation_setup, notes · Keyword search

Use this data for free

A free workspace gives you an API key to query whole tables, download Parquet and connect your AI tools.

We are in open beta, so expect changes week to week.

Create a free workspace

Already have an account? Log in

DocsPricingTermsPrivacyStatus