We value your privacy

We use cookies to enhance your browsing experience, serve personalized content and analyze our traffic. By clicking “Accept all” you consent to our use of cookies. Cookie Policy

More options
Skip to content
FacthoryFacthory
FacthoryFacthory
    • Workspace
    • Agent
    • Knowledge
    • Analytics
    • Office apps
    • Desktop
    All features
    What’s new
    Download
  • By work

    • Organizational Knowledge Risk & Gap Intelligence
    • Scientific Research & Experimentation
    • Autonomous Analytics & Insight Discovery
    • Workforce Readiness & Adaptive Learning
    • SOP Governance & Operational Standardization
    • Process Discovery & Continuous Optimization
    • All solutions

    By industry

    • Manufacturing
    • Automotive
    • Construction & Infrastructure
    • Real Estate
    • Public Sector
    • Life Sciences
    • All industries
  • Facthory for

    • Regulated industries
    • On-premises
    • Developers

    Discover

    • What’s new
    • Blog
    • Case studies
    • Webinars
    • Models

    Learn

    • Documentation
    • Help centre
  • Pricing
  • Download
Sign inGet started
Models

Benchmarks

The benchmarks two or more models in the catalog report, with the results their model cards publish. Facthory has not run them; each result names its source.

Independent evaluations

Results Epoch AI ran or collected from published sources, open and closed models alike, each with its evidence.

Data: Epoch AI, ‘Capabilities & benchmarking’, epoch.ai/benchmarks, licensed CC BY 4.0.

BenchmarkPublishedModels
GPQA diamond20 Nov 2023339
OTIS Mock AIME 2024-202519 Dec 2024316
ARC-AGI5 Nov 2019287
ARC-AGI-224 Mar 2025275
DTBench12 Aug 2026241
LMCA12 Aug 2026203
WeirdML16 Jan 2025172
Chess Puzzles10 Dec 2025164
Mystery Game Puzzles30 Jul 2026146
MMLU7 Sept 2020136
FrontierMath-Tiers-1-3-v2-Private12 Jun 2026125
SimpleBench31 Oct 2024124
MATH level 55 Mar 2021108
SimpleQA Verified9 Sept 2025103
GSM8K27 Oct 202193
ProofBench30 Jan 202685
FrontierMath-Tier-4-v2-Private12 Jun 202681
Winogrande24 Jul 201980
ARC AI214 Mar 201877
HellaSwag19 May 201976
Aider polyglot21 Dec 202471
APEX-Agents21 Jan 202665
Fiction.LiveBench21 Feb 202561
PIQA26 Nov 201960
Terminal Bench19 May 202559
BBH17 Oct 202250
GeoBench1 Mar 202549
FrontierCode8 Jun 202646
METR Time Horizons19 Mar 202546
MirrorCode26 Jun 202646
GSO-Bench29 May 202542
OpenBookQA8 Sept 201842
DeepResearch Bench13 Jun 202541
Balrog20 Nov 202440
Furniture Assembly23 Sept 202639
TriviaQA9 May 201739
VPCT30 Jan 202538
EBR-bench1 Jul 202631
Surface Evolver Bench19 Jun 202629
LAMBADA20 Jun 201626
WeirdML v318 Sept 202626
CL-bench3 Feb 202623
FrontierSWE2 Sept 202623
Cybench15 Aug 202422
PostTrainBench11 Mar 202620
Remote Labor Index29 Oct 202519
CL-bench Life29 Apr 202617
CadEval22 Apr 202515
OSWorld 2.026 Jun 202614
The Agent Company18 Dec 202414
ANLI31 Oct 201911
GDPval25 Sept 202511
ExploitBench13 May 20269
OSWorld11 Apr 20249
SpireClimb14 Oct 20269
SuperGLUE2 May 20198
FrontierMath-2025-02-28-Private
superseded
28 Feb 2025101
FrontierMath-Tier-4-2025-07-01-Private
superseded
11 Jul 202572

Self-reported in model cards

Results the models’ publishers report in their cards. Kept apart from the independent evaluations: setups differ.

BenchmarkMetricModels
LibriSpeech (clean)
other
Test WER9
LibriSpeech (clean)
clean
Test WER7
Common Voice 11.0
hi
Test WER5
MTEB AmazonCounterfactualClassification (en)
en
accuracy5
MTEB AmazonCounterfactualClassification (en)
en
ap5
MTEB AmazonCounterfactualClassification (en)
en
f15
MTEB AmazonPolarityClassification
default
accuracy5
MTEB AmazonPolarityClassification
default
ap5
MTEB AmazonPolarityClassification
default
f15
MTEB AmazonReviewsClassification (en)
en
accuracy5
MTEB AmazonReviewsClassification (en)
en
f15
GIFT-EvalCRPS4
GIFT-EvalMASE4
TIMECRPS4
TIMEMASE4
fev-benchSQL skill score4
BBH (3-Shot)normalized accuracy3
FLEURS
de_de
Test WER (De)3
FLEURS
en_us
Test WER (En)3
FLEURS
es_419
Test WER (Es)3
FLEURS
fr_fr
Test WER (Fr)3
FLEURS
it_it
Test WER (It)3
FLEURS
pt_br
Test WER (Pt)3
GPQA (0-shot)acc_norm3
IFEval (0-Shot)strict accuracy3
MATH Lvl 5 (4-Shot)exact match3
MMLU-PRO (5-shot)
main
accuracy3
MTEB AmazonCounterfactualClassification (en)
en
ap_weighted3
MTEB AmazonCounterfactualClassification (en)
en
f1_weighted3
MTEB AmazonCounterfactualClassification (en)
en
main_score3
MTEB AmazonCounterfactualClassification (en-ext)
en-ext
accuracy3
MTEB AmazonCounterfactualClassification (en-ext)
en-ext
ap3
MTEB AmazonCounterfactualClassification (en-ext)
en-ext
ap_weighted3
MTEB AmazonCounterfactualClassification (en-ext)
en-ext
f13
MTEB AmazonCounterfactualClassification (en-ext)
en-ext
f1_weighted3
MTEB AmazonCounterfactualClassification (en-ext)
en-ext
main_score3
MTEB AmazonPolarityClassification (default)
default
ap_weighted3
MTEB AmazonPolarityClassification (default)
default
f1_weighted3
MTEB AmazonPolarityClassification (default)
default
main_score3
MTEB AmazonReviewsClassification (en)
en
f1_weighted3
MTEB AmazonReviewsClassification (en)
en
main_score3
MTEB AppsRetrieval (default)
default
main_score3
MTEB AppsRetrieval (default)
default
map_at_13
MTEB AppsRetrieval (default)
default
map_at_103
MTEB AppsRetrieval (default)
default
map_at_1003
MTEB AppsRetrieval (default)
default
map_at_10003
MTEB AppsRetrieval (default)
default
map_at_203
MTEB AppsRetrieval (default)
default
map_at_33
MTEB AppsRetrieval (default)
default
map_at_53
MTEB AppsRetrieval (default)
default
mrr_at_13
MTEB AppsRetrieval (default)
default
mrr_at_103
MTEB AppsRetrieval (default)
default
mrr_at_1003
MTEB AppsRetrieval (default)
default
mrr_at_10003
MTEB AppsRetrieval (default)
default
mrr_at_203
MTEB AppsRetrieval (default)
default
mrr_at_33
MTEB AppsRetrieval (default)
default
mrr_at_53
MTEB AppsRetrieval (default)
default
nauc_map_at_1000_diff13
MTEB AppsRetrieval (default)
default
nauc_map_at_1000_max3
MTEB AppsRetrieval (default)
default
nauc_map_at_1000_std3
MTEB AppsRetrieval (default)
default
nauc_map_at_100_diff13
MTEB AppsRetrieval (default)
default
nauc_map_at_100_max3
MTEB AppsRetrieval (default)
default
nauc_map_at_100_std3
MTEB AppsRetrieval (default)
default
nauc_map_at_10_diff13
MTEB AppsRetrieval (default)
default
nauc_map_at_10_max3
MTEB AppsRetrieval (default)
default
nauc_map_at_10_std3
MTEB AppsRetrieval (default)
default
nauc_map_at_1_diff13
MTEB AppsRetrieval (default)
default
nauc_map_at_1_max3
MTEB AppsRetrieval (default)
default
nauc_map_at_1_std3
MTEB AppsRetrieval (default)
default
nauc_map_at_20_diff13
MTEB AppsRetrieval (default)
default
nauc_map_at_20_max3
MTEB AppsRetrieval (default)
default
nauc_map_at_20_std3
MTEB AppsRetrieval (default)
default
nauc_map_at_3_diff13
MTEB AppsRetrieval (default)
default
nauc_map_at_3_max3
MTEB AppsRetrieval (default)
default
nauc_map_at_3_std3
MTEB AppsRetrieval (default)
default
nauc_map_at_5_diff13
MTEB AppsRetrieval (default)
default
nauc_map_at_5_max3
MTEB AppsRetrieval (default)
default
nauc_map_at_5_std3
MTEB AppsRetrieval (default)
default
nauc_mrr_at_1000_diff13
MTEB AppsRetrieval (default)
default
nauc_mrr_at_1000_max3
MTEB AppsRetrieval (default)
default
nauc_mrr_at_1000_std3
MTEB AppsRetrieval (default)
default
nauc_mrr_at_100_diff13
MTEB AppsRetrieval (default)
default
nauc_mrr_at_100_max3
MTEB AppsRetrieval (default)
default
nauc_mrr_at_100_std3
MTEB AppsRetrieval (default)
default
nauc_mrr_at_10_diff13
MTEB AppsRetrieval (default)
default
nauc_mrr_at_10_max3
MTEB AppsRetrieval (default)
default
nauc_mrr_at_10_std3
MTEB AppsRetrieval (default)
default
nauc_mrr_at_1_diff13
MTEB AppsRetrieval (default)
default
nauc_mrr_at_1_max3
MTEB AppsRetrieval (default)
default
nauc_mrr_at_1_std3
MTEB AppsRetrieval (default)
default
nauc_mrr_at_20_diff13
MTEB AppsRetrieval (default)
default
nauc_mrr_at_20_max3
MTEB AppsRetrieval (default)
default
nauc_mrr_at_20_std3
MTEB AppsRetrieval (default)
default
nauc_mrr_at_3_diff13
MTEB AppsRetrieval (default)
default
nauc_mrr_at_3_max3
MTEB AppsRetrieval (default)
default
nauc_mrr_at_3_std3
MTEB AppsRetrieval (default)
default
nauc_mrr_at_5_diff13
MTEB AppsRetrieval (default)
default
nauc_mrr_at_5_max3
MTEB AppsRetrieval (default)
default
nauc_mrr_at_5_std3
MTEB AppsRetrieval (default)
default
nauc_ndcg_at_1000_diff13
MTEB AppsRetrieval (default)
default
nauc_ndcg_at_1000_max3
MTEB AppsRetrieval (default)
default
nauc_ndcg_at_1000_std3
MTEB AppsRetrieval (default)
default
nauc_ndcg_at_100_diff13
MTEB AppsRetrieval (default)
default
nauc_ndcg_at_100_max3
MTEB AppsRetrieval (default)
default
nauc_ndcg_at_100_std3
MTEB AppsRetrieval (default)
default
nauc_ndcg_at_10_diff13
MTEB AppsRetrieval (default)
default
nauc_ndcg_at_10_max3
MTEB AppsRetrieval (default)
default
nauc_ndcg_at_10_std3
MTEB AppsRetrieval (default)
default
nauc_ndcg_at_1_diff13
MTEB AppsRetrieval (default)
default
nauc_ndcg_at_1_max3
MTEB AppsRetrieval (default)
default
nauc_ndcg_at_1_std3
MTEB AppsRetrieval (default)
default
nauc_ndcg_at_20_diff13
MTEB AppsRetrieval (default)
default
nauc_ndcg_at_20_max3
MTEB AppsRetrieval (default)
default
nauc_ndcg_at_20_std3
MTEB AppsRetrieval (default)
default
nauc_ndcg_at_3_diff13
MTEB AppsRetrieval (default)
default
nauc_ndcg_at_3_max3
MTEB AppsRetrieval (default)
default
nauc_ndcg_at_3_std3
MTEB AppsRetrieval (default)
default
nauc_ndcg_at_5_diff13
MTEB AppsRetrieval (default)
default
nauc_ndcg_at_5_max3
MTEB AppsRetrieval (default)
default
nauc_ndcg_at_5_std3
MTEB AppsRetrieval (default)
default
nauc_precision_at_1000_diff13
MTEB AppsRetrieval (default)
default
nauc_precision_at_1000_max3
MTEB AppsRetrieval (default)
default
nauc_precision_at_1000_std3
MTEB AppsRetrieval (default)
default
nauc_precision_at_100_diff13
MTEB AppsRetrieval (default)
default
nauc_precision_at_100_max3
MTEB AppsRetrieval (default)
default
nauc_precision_at_100_std3
MTEB AppsRetrieval (default)
default
nauc_precision_at_10_diff13
MTEB AppsRetrieval (default)
default
nauc_precision_at_10_max3
MTEB AppsRetrieval (default)
default
nauc_precision_at_10_std3
MTEB AppsRetrieval (default)
default
nauc_precision_at_1_diff13
MTEB AppsRetrieval (default)
default
nauc_precision_at_1_max3
MTEB AppsRetrieval (default)
default
nauc_precision_at_1_std3
MTEB AppsRetrieval (default)
default
nauc_precision_at_20_diff13
MTEB AppsRetrieval (default)
default
nauc_precision_at_20_max3
MTEB AppsRetrieval (default)
default
nauc_precision_at_20_std3
MTEB AppsRetrieval (default)
default
nauc_precision_at_3_diff13
MTEB AppsRetrieval (default)
default
nauc_precision_at_3_max3
MTEB AppsRetrieval (default)
default
nauc_precision_at_3_std3
MTEB AppsRetrieval (default)
default
nauc_precision_at_5_diff13
MTEB AppsRetrieval (default)
default
nauc_precision_at_5_max3
MTEB AppsRetrieval (default)
default
nauc_precision_at_5_std3
MTEB AppsRetrieval (default)
default
nauc_recall_at_1000_diff13
MTEB AppsRetrieval (default)
default
nauc_recall_at_1000_max3
MTEB AppsRetrieval (default)
default
nauc_recall_at_1000_std3
MTEB AppsRetrieval (default)
default
nauc_recall_at_100_diff13
MTEB AppsRetrieval (default)
default
nauc_recall_at_100_max3
MTEB AppsRetrieval (default)
default
nauc_recall_at_100_std3
MTEB AppsRetrieval (default)
default
nauc_recall_at_10_diff13
MTEB AppsRetrieval (default)
default
nauc_recall_at_10_max3
MTEB AppsRetrieval (default)
default
nauc_recall_at_10_std3
MTEB AppsRetrieval (default)
default
nauc_recall_at_1_diff13
MTEB AppsRetrieval (default)
default
nauc_recall_at_1_max3
MTEB AppsRetrieval (default)
default
nauc_recall_at_1_std3
MTEB AppsRetrieval (default)
default
nauc_recall_at_20_diff13
MTEB AppsRetrieval (default)
default
nauc_recall_at_20_max3
MTEB AppsRetrieval (default)
default
nauc_recall_at_20_std3
MTEB AppsRetrieval (default)
default
nauc_recall_at_3_diff13
MTEB AppsRetrieval (default)
default
nauc_recall_at_3_max3
MTEB AppsRetrieval (default)
default
nauc_recall_at_3_std3
MTEB AppsRetrieval (default)
default
nauc_recall_at_5_diff13
MTEB AppsRetrieval (default)
default
nauc_recall_at_5_max3
MTEB AppsRetrieval (default)
default
nauc_recall_at_5_std3
MTEB AppsRetrieval (default)
default
ndcg_at_13
MTEB AppsRetrieval (default)
default
ndcg_at_103
MTEB AppsRetrieval (default)
default
ndcg_at_1003
MTEB AppsRetrieval (default)
default
ndcg_at_10003
MTEB AppsRetrieval (default)
default
ndcg_at_203
MTEB AppsRetrieval (default)
default
ndcg_at_33
MTEB AppsRetrieval (default)
default
ndcg_at_53
MTEB AppsRetrieval (default)
default
precision_at_13
MTEB AppsRetrieval (default)
default
precision_at_103
MTEB AppsRetrieval (default)
default
precision_at_1003
MTEB AppsRetrieval (default)
default
precision_at_10003
MTEB AppsRetrieval (default)
default
precision_at_203
MTEB AppsRetrieval (default)
default
precision_at_33
MTEB AppsRetrieval (default)
default
precision_at_53
MTEB AppsRetrieval (default)
default
recall_at_13
MTEB AppsRetrieval (default)
default
recall_at_103
MTEB AppsRetrieval (default)
default
recall_at_1003
MTEB AppsRetrieval (default)
default
recall_at_10003
MTEB AppsRetrieval (default)
default
recall_at_203
MTEB AppsRetrieval (default)
default
recall_at_33
MTEB AppsRetrieval (default)
default
recall_at_53
MTEB ArguAna (default)
default
map_at_13
MTEB ArguAna (default)
default
map_at_103
MTEB ArguAna (default)
default
map_at_1003
MTEB ArguAna (default)
default
map_at_10003
MTEB ArguAna (default)
default
map_at_203
MTEB ArguAna (default)
default
map_at_33
MTEB ArguAna (default)
default
map_at_53
MTEB ArguAna (default)
default
mrr_at_13
MTEB ArguAna (default)
default
mrr_at_103
MTEB ArguAna (default)
default
mrr_at_1003
MTEB ArguAna (default)
default
mrr_at_10003
MTEB ArguAna (default)
default
mrr_at_203
MTEB ArguAna (default)
default
mrr_at_33
MTEB ArguAna (default)
default
mrr_at_53
MTEB ArguAna (default)
default
nauc_map_at_1000_diff13
MTEB ArguAna (default)
default
nauc_map_at_1000_max3
MTEB ArguAna (default)
default
nauc_map_at_1000_std3
MTEB ArguAna (default)
default
nauc_map_at_100_diff13

A workspace for what your organization knows, with an agent that works on it.

Product

  • All features
  • Workspace
  • Agent
  • Knowledge
  • Analytics
  • Office apps
  • Desktop
  • Download
  • Pricing

Solutions

  • Organizational Knowledge Risk & Gap Intelligence
  • Scientific Research & Experimentation
  • Autonomous Analytics & Insight Discovery
  • Workforce Readiness & Adaptive Learning
  • SOP Governance & Operational Standardization
  • Process Discovery & Continuous Optimization
  • All solutions
  • All industries

Resources

  • Blog
  • Case studies
  • Webinars
  • Documentation
  • Help centre
  • Models
  • ROI calculator

Company

  • Who we are
  • Careers
  • Press hub
  • Brand kit
  • Philosophy
  • Contact us

Join our newsletter

Product news from Facthory, by email.

​

© 2026 Facthory. All rights reserved.
  • Privacy policy
  • Cookie policy
  • Terms and conditions
  • Imprint
  • Cookie settings