{"schema_version":"mostlyright-public-catalog.v1","dataset":{"id":"f644292a-2f98-415c-b28a-560181d74f1c","slug":"llm-benchmark-scores-by-model-since-2019","title":"LLM benchmark scores by model, since 2019","summary":"**Compare large language models on more than 80 benchmarks, from GPQA Diamond and SWE-bench Verified to FrontierMath and Humanity's Last Exam.** The table gathers every result published in Epoch AI's Benchmarking Hub, with each score in its own unit, who ran the evaluation and a link to the source. Use it to rank models, follow how scores rose with each release, or set lab and paper numbers beside independent runs.\n\nOn 1 October 2026 the hub held about 6,800 results for about 1,200 model versions of 710 models from OpenAI, Anthropic, Google DeepMind, Meta, Alibaba and about 60 other developers. Each row is one model version on one benchmark under one evaluation setup.\n\n## Coverage\n- Window: models released from October 2019 to Epoch AI's latest update, most of them in 2025 and 2026. Dates are UTC calendar dates.\n- Grain: one row per published result, so a model can have several rows on one benchmark when the shots, harness, reasoning effort or source differ.\n- Cadence: the table is rebuilt every day from Epoch AI's download. Epoch AI changed 36 of its benchmark tables between 29 September and 1 October 2026.\n- New benchmarks: a benchmark Epoch AI adds to its Capabilities Index appears automatically, with one best score per model family, until its full table is added to the build.\n\n## Columns\n- `result_id`: Epoch AI's record id for the result, or a hash of the row where Epoch AI publishes none.\n- `benchmark`, `benchmark_release_date`: the benchmark as Epoch AI names it, and the date it was published.\n- `model`, `model_version`: the model family and the exact version evaluated, usually the developer's API name plus Epoch AI's reasoning-setting suffix.\n- `organization`, `country`, `model_access`, `model_release_date`: the developer, its country, how the model is released (API or open weights) and when.\n- `training_compute_flop`: Epoch AI's estimate of training compute, in floating-point operations.\n- `score`, `score_unit`, `higher_is_better`: the score as published, its unit and its direction. Units are fraction, percent, points out of 10, Elo rating, rating, speedup factor, Brier score, index points or US dollars.\n- `score_fraction`: the score as a fraction from 0 to 1, for benchmarks scored as a proportion.\n- `stderr`: the standard error the source reports, in the unit of `score`.\n- `result_date`: the run, grading, addition or update date, where the source gives one.\n- `reported_by`: Epoch AI for its own runs, Leaderboard, Model developer, Paper or report, or Not stated.\n- `source`, `source_url`: the cited paper, leaderboard or site and its link, or Epoch AI's evaluation log for its own runs.\n- `evaluation_setup`: the shots, harness, agent, reasoning effort or provider that tell several results for one model apart.\n- `notes`, `epoch_file`: Epoch AI's note on the result and the file in its download the row comes from.\n\n## Missing values\nAn empty cell means the source did not publish that value. Rows without a numeric score or a model name are left out, about 70 of the published rows on 1 October 2026. Results for older models often lack an organisation, access type or release date. Most results carry no standard error or result date. Epoch AI does not flag lab self-reports, so labs' own numbers appear under Paper or report and Model developer, next to academic papers.\n\n## Suitable for\n- Ranking models on one benchmark\n- Tracking how the best scores changed with model release dates\n- Comparing Epoch AI's independent runs with numbers from leaderboards, papers and labs\n- Relating benchmark scores to training compute and model access\n\n## Source and rights\nEvery row comes from the Epoch AI Benchmarking Hub at epoch.ai/benchmarks, read daily from its download. Epoch AI publishes its own evaluation runs under the Creative Commons Attribution 4.0 licence. Results Epoch AI compiled from other projects keep those projects' licences, so credit Epoch AI and the source linked in each row.","topics":["llm benchmarks","large language models","ai evaluation","leaderboards","epoch ai","frontier models","ai capabilities"],"publisher":{"handle":"mostlyrightmd","display_name":"Mostly Right","avatar_url":"https://mostlyright.md/api/public/avatars/e70f8c973d16894fcee5cb5aba887feafc75cc0d67419bd0ed321cfde9d0d967.webp","mark":{"background":"#FFC613","facets":["#2B5FE3","#F15B22","#17160F"],"pattern":2,"rotation":90}},"star_count":0,"table_count":1,"row_count":7121,"published_at":"2026-10-02T10:15:04.269Z","updated_at":"2026-10-02T10:15:05.508Z","cover_image_url":"https://storage.googleapis.com/mostlyright-dataset-covers/v1/c6b2d272-c0d7-4c36-bbf3-81cb79d07b7e/4e1146d03a59cc1809a5f86600c5f5d882392418ce812b2f6ddd3ede40a2f65a.webp","cover_tile_url":"https://storage.googleapis.com/mostlyright-dataset-covers/v1/c6b2d272-c0d7-4c36-bbf3-81cb79d07b7e/4e1146d03a59cc1809a5f86600c5f5d882392418ce812b2f6ddd3ede40a2f65a.tile.webp","activity":{"views":1881,"likes":0},"canonical_url":"https://mostlyright.md/datasets/llm-benchmark-scores-by-model-since-2019","tables":[{"id":"9687578e-158b-4a35-a894-0152d5fba679","slug":"llm-benchmark-results","title":"llm_benchmark_results","license":null,"schema":{"columns":22,"column_details":[{"name":"result_id","unit":null,"meaning":"Identifier of the result: Epoch AI's own record id where it publishes one, otherwise a hash of the file name and the row, stable while the row is unchanged.","nullable":null,"logical_type":"string"},{"name":"benchmark","unit":null,"meaning":"The benchmark, named as Epoch AI names it, for example GPQA diamond or SWE-bench Verified.","nullable":null,"logical_type":"string"},{"name":"benchmark_release_date","unit":null,"meaning":"The date the benchmark was published, from Epoch AI's benchmark list.","nullable":null,"logical_type":"date"},{"name":"model","unit":null,"meaning":"The model the result belongs to, as Epoch AI groups versions of one model, for example GPT-5 or Claude Opus 4.5.","nullable":null,"logical_type":"string"},{"name":"model_version","unit":null,"meaning":"The exact model version evaluated, usually the developer's API identifier with Epoch AI's suffix for the reasoning setting; empty for one-per-family rows from the Capabilities Index.","nullable":null,"logical_type":"string"},{"name":"organization","unit":null,"meaning":"The organisation that developed the model; joint developers are listed together.","nullable":null,"logical_type":"string"},{"name":"country","unit":null,"meaning":"The country of the developing organisation.","nullable":null,"logical_type":"string"},{"name":"model_access","unit":null,"meaning":"How the model is released, for example API access or Open weights (unrestricted), from Epoch AI's model list.","nullable":null,"logical_type":"string"},{"name":"model_release_date","unit":null,"meaning":"The date the model version was released.","nullable":null,"logical_type":"date"},{"name":"training_compute_flop","unit":null,"meaning":"Epoch AI's estimate of the compute used to train the model, in floating-point operations, where it publishes one.","nullable":null,"logical_type":"number"},{"name":"score","unit":null,"meaning":"The score as the source publishes it, in the unit named by score_unit.","nullable":null,"logical_type":"number"},{"name":"score_unit","unit":null,"meaning":"The unit of score: fraction, percent, points out of 10, Elo rating, rating, speedup factor, Brier score, index points (0-100) or US dollars.","nullable":null,"logical_type":"string"},{"name":"score_fraction","unit":null,"meaning":"The score as a fraction between 0 and 1, for benchmarks scored as a proportion; empty for ratings, money and other units.","nullable":null,"logical_type":"number"},{"name":"higher_is_better","unit":null,"meaning":"Whether a higher score is better on this benchmark; false only for error measures such as the Brier score.","nullable":null,"logical_type":"boolean"},{"name":"stderr","unit":null,"meaning":"The standard error the source reports for the score, in the same unit as score.","nullable":null,"logical_type":"number"},{"name":"result_date","unit":null,"meaning":"The date the source attaches to the result: the run date for Epoch AI's own runs, otherwise the evaluation, grading, addition or last-update date the source states.","nullable":null,"logical_type":"date"},{"name":"reported_by","unit":null,"meaning":"Who ran or published the result: Epoch AI (its own run), Leaderboard (a benchmark or evaluator leaderboard), Model developer (the developer's own page), Paper or report (a research paper or technical report, where labs publish their own numbers), or Not stated.","nullable":null,"logical_type":"string"},{"name":"source","unit":null,"meaning":"The source Epoch AI cites for the result: a paper or leaderboard name, or the web host of its link.","nullable":null,"logical_type":"string"},{"name":"source_url","unit":null,"meaning":"The link to the cited source, or to Epoch AI's evaluation log for its own runs.","nullable":null,"logical_type":"string"},{"name":"evaluation_setup","unit":null,"meaning":"Settings that distinguish several results for one model on one benchmark, such as the number of shots, the agent harness or the reasoning effort.","nullable":null,"logical_type":"string"},{"name":"notes","unit":null,"meaning":"Epoch AI's note on the result, where it gives one.","nullable":null,"logical_type":"string"},{"name":"epoch_file","unit":null,"meaning":"The file in Epoch AI's download the row comes from.","nullable":null,"logical_type":"string"}],"schema_version":"mostlyright.cloud.released-schema-summary.v1"},"preview_enabled":true,"download_enabled":true,"version_id":"dde2b5b0-4267-5ba8-80d1-91188ea9f96c"}]}}