MMLU
Each model’s result as Epoch AI’s Benchmarking Hub publishes it, run by Epoch or collected from the source it cites. Facthory has not run them.
Data: Epoch AI, ‘Capabilities & benchmarking’, epoch.ai/benchmarks, licensed CC BY 4.0. Read on 16 Nov 2026.
The chart shows the 25 best and the table the 100 best of 136 results.
| # | Model | Organization | Result | Evidence |
|---|---|---|---|---|
| 1 | GPT-4o (Nov 2024) API access | OpenAI | 88.1% | – |
| 2 | Claude 3.5 Sonnet (Oct 2024) API access | Anthropic | 87.3% | Log or source |
| 3 | DeepSeek-V3 Open weights (restricted use) | DeepSeek | 87.2% | Log or source |
| 4 | gemini-1.5-pro-002 API access | Google DeepMind | 86.9% | Log or source |
| 5 | Claude 3.5 Sonnet (Jun 2024) API access | Anthropic | 86.5% | Log or source |
| 6 | GPT-4 (Mar 2023) API access | OpenAI | 86.4% | Log or source |
| 7 | gemini-1.5-pro-001 API access | Google DeepMind | 85.9% | Log or source |
| 8 | qwen2.5-72b-instruct Open weights (unrestricted) | Alibaba | 85.3% | – |
| 9 | Qwen2.5-72B Open weights (unrestricted) | Alibaba | 85% | Log or source |
| 10 | phi-4 Open weights (unrestricted) | Microsoft Research | 84.8% | – |
| 11 | claude-3-opus-20240229 API access | Anthropic | 84.6% | Log or source |
| 12 | Llama-3.1-405B-Instruct Open weights (restricted use) | Meta AI | 84.5% | Log or source |
| 13 | Llama-3.1-405B Open weights (restricted use) | Meta AI | 84.4% | Log or source |
| 14 | GPT-4o (Aug 2024) API access | OpenAI | 84.3% | Log or source |
| 15 | GPT-4o (May 2024) API access | OpenAI | 84.2% | Log or source |
| 16 | gemini-1.5-pro-001-feb24 API access | Google DeepMind | 82.7% | Log or source |
| 17 | GPT-4 (Jun 2023) API access | OpenAI | 82.4% | Log or source |
| 18 | qwen2-72b-instruct Open weights (unrestricted) | Alibaba | 82.4% | Log or source |
| 19 | amazon.nova-pro-v1:0 API access | Amazon | 82% | Log or source |
| 20 | gpt-4o-mini-2024-07-18 API access | OpenAI | 81.8% | – |
| 21 | gpt-4-turbo-2024-04-09 API access | OpenAI | 81.3% | Log or source |
| 22 | Llama-3.2-90B-Vision-Instruct Open weights (restricted use) | Meta AI | 80.3% | Log or source |
| 23 | Llama-3.1-70B-Instruct Open weights (restricted use) | Meta AI | 80.1% | Log or source |
| 24 | mistral-large-2407 Open weights (non-commercial) | Mistral AI | 80% | Log or source |
| 25 | qwen2.5-14b-instruct Open weights (unrestricted) | Alibaba | 79.9% | – |
| 26 | gemini-2.0-flash-exp API access | Google DeepMind,Google | 79.7% | Log or source |
| 27 | gpt-4-turbo API access | OpenAI | 79.6% | Log or source |
| 28 | Meta-Llama-3-70B-Instruct Open weights (restricted use) | Meta AI | 79.3% | Log or source |
| 29 | Yi-large API access | 01.AI | 79.3% | Log or source |
| 30 | Llama-3.3-70B-Instruct Open weights (restricted use) | Meta AI | 79.1% | Log or source |
| 31 | claude-2.0 API access | Anthropic | 78.5% | Log or source |
| 32 | DeepSeek-V2 Open weights (restricted use) | DeepSeek | 78.4% | Log or source |
| 33 | Gemini 1.5 Flash (May 2024) API access | Google DeepMind | 77.9% | Log or source |
| 34 | Mixtral-8x22B-v0.1 Open weights (unrestricted) | Mistral AI | 77.8% | Log or source |
| 35 | gemini-1.5-flash-0514 API access | Google DeepMind | 77.8% | Log or source |
| 36 | Phi-3-medium-128k-instruct Open weights (unrestricted) | Microsoft | 77.5% | Log or source |
| 37 | amazon.nova-lite-v1:0 API access | Amazon | 77% | Log or source |
| 38 | claude-1.3 API access | Anthropic | 77% | Log or source |
| 39 | Yi-34B Open weights (restricted use) | 01.AI | 76.2% | Log or source |
| 40 | claude-3-sonnet-20240229 API access | Anthropic | 75.9% | Log or source |
| 41 | Phi-3-small-8k-instruct Open weights (unrestricted) | Microsoft | 75.7% | – |
| 42 | gemma-2-27b-it Open weights (restricted use) | Google DeepMind | 75.7% | Log or source |
| 43 | Qwen2.5-Coder-14B | – | 75.2% | – |
| 44 | qwen1.5-32B Open weights (restricted use) | Alibaba | 74.4% | Log or source |
| 45 | Claude 3.5 Haiku (Oct 2024) API access | Anthropic | 74.3% | Log or source |
| 46 | gemini-1.5-flash-002 API access | Google DeepMind | 73.9% | Log or source |
| 47 | claude-3-haiku-20240307 API access | Anthropic | 73.8% | Log or source |
| 48 | claude-2.1 API access | Anthropic | 73.5% | Log or source |
| 49 | claude-instant-1.1 API access | Anthropic | 73.4% | Log or source |
| 50 | qwen2.5-7b-instruct Open weights (unrestricted) | Alibaba | 72.9% | Log or source |
| 51 | Inflection-1 Hosted access (no API) | Inflection AI | 72.7% | Log or source |
| 52 | gemma-2-9b-it Open weights (restricted use) | Google DeepMind | 72.1% | Log or source |
| 53 | GPT-3.5 Turbo (Nov 2023) API access | OpenAI | 71.4% | – |
| 54 | amazon.nova-micro-v1:0 API access | Amazon | 70.8% | Log or source |
| 55 | Mixtral-8x7B-v0.1 Open weights (unrestricted) | Mistral AI | 70.6% | Log or source |
| 56 | falcon-180B Open weights (restricted use) | Technology Innovation Institute | 70.6% | – |
| 57 | gemini-1.0-pro-001 API access | Google DeepMind | 70% | Log or source |
| 58 | text-davinci-002 API access | OpenAI | 70% | Log or source |
| 59 | Llama-2-70b-hf Open weights (restricted use) | Meta AI | 69.5% | Log or source |
| 60 | c4ai-command-r-plus-08-2024 Open weights (non-commercial) | Cohere,Cohere Labs (formerly Cohere for AI) | 69.4% | Log or source |
| 61 | PaLM 540B Unreleased | Google Research | 69.3% | – |
| 62 | GPT-3.5 Turbo (June 2023) API access | OpenAI | 68.9% | Log or source |
| 63 | Meta-Llama-3-8B-Instruct Open weights (restricted use) | Meta AI | 68.8% | Log or source |
| 64 | Phi-3-mini-4k-instruct Open weights (unrestricted) | Microsoft | 68.8% | – |
| 65 | claude-instant-1.2 API access | Anthropic | 68.8% | Log or source |
| 66 | mistral-large-2402 API access | Mistral AI | 68.8% | Log or source |
| 67 | mistral-small-2402 | – | 68.7% | Log or source |
| 68 | StableBeluga2 Open weights (non-commercial) | Stability AI | 68.6% | Log or source |
| 69 | qwen1.5-14B Open weights (unrestricted) | Alibaba | 68.6% | Log or source |
| 70 | Yi-9B | – | 68.4% | – |
| 71 | Qwen2.5-Coder-7B Open weights (unrestricted) | Alibaba | 68% | – |
| 72 | Yi-34B-Chat Open weights (restricted use) | 01.AI | 67.6% | – |
| 73 | Chinchilla (70B) Unreleased | DeepMind | 67.5% | – |
| 74 | GPT-3.5 Turbo (Jan 2024) API access | OpenAI | 67.3% | Log or source |
| 75 | Qwen-14B Open weights (restricted use) | Alibaba | 66.3% | – |
| 76 | gemma-7b Open weights (restricted use) | Google DeepMind | 66.1% | Log or source |
| 77 | c4ai-command-r-08-2024 | – | 65.2% | Log or source |
| 78 | starcoder2-15b Open weights (restricted use) | Hugging Face,ServiceNow,Nvidia,BigCode | 64.1% | – |
| 79 | Qwen-14B-Chat Open weights (restricted use) | Alibaba | 64% | – |
| 80 | Yi-6B Open weights (restricted use) | 01.AI | 64% | Log or source |
| 81 | LLaMA-65B Open weights (non-commercial) | Meta AI | 63.4% | – |
| 82 | Llama-2-34b Unreleased | Meta AI | 62.6% | – |
| 83 | Qwen1.5-7B Open weights (unrestricted) | Alibaba | 62.6% | Log or source |
| 84 | Mistral-7B-Instruct-v0.2 Open weights (unrestricted) | Mistral AI | 62.5% | Log or source |
| 85 | Mistral-7B-v0.1 Open weights (unrestricted) | Mistral AI | 62.5% | Log or source |
| 86 | Qwen2.5-Coder-32B Open weights (unrestricted) | Alibaba | 61.2% | – |
| 87 | DeepSeek-Coder-V2-Lite-Base | – | 60.5% | – |
| 88 | Gopher (280B) Unreleased | DeepMind | 60% | – |
| 89 | Mistral-7B-Instruct-v0.3 Open weights (unrestricted) | Mistral AI | 59.9% | Log or source |
| 90 | Llama-2-70b-chat Open weights (restricted use) | Meta AI | 59.4% | – |
| 91 | Baichuan-2-13B-Base Open weights (restricted use) | Baichuan | 59.2% | – |
| 92 | Nemotron-4 15B Unreleased | Nvidia | 58.7% | – |
| 93 | Falcon 2-11B Open weights (restricted use) | Technology Innovation Institute | 58.4% | – |
| 94 | phi-2 Open weights (unrestricted) | Microsoft | 58.4% | Log or source |
| 95 | Yi-6B-Chat Open weights (restricted use) | 01.AI | 58.2% | – |
| 96 | LLaMA-33B Open weights (non-commercial) | Meta AI | 57.8% | – |
| 97 | falcon-40b Open weights (unrestricted) | Technology Innovation Institute | 56.9% | – |
| 98 | Llama-3.2-11B-Vision-Instruct Open weights (restricted use) | Meta AI | 56.5% | Log or source |
| 99 | Llama-3.1-8B-Instruct Open weights (restricted use) | Meta AI | 56.1% | Log or source |
| 100 | internlm-chat-20b | – | 55.6% | – |
