Winogrande
Each model’s result as Epoch AI’s Benchmarking Hub publishes it, run by Epoch or collected from the source it cites. Facthory has not run them.
Data: Epoch AI, ‘Capabilities & benchmarking’, epoch.ai/benchmarks, licensed CC BY 4.0. Read on 16 Nov 2026.
The chart shows the 25 best of 80 results; the table lists them all.
| # | Model | Organization | Result | Evidence |
|---|---|---|---|---|
| 1 | Llama-3.1-405B Open weights (restricted use) | Meta AI | 89.2% | Log or source |
| 2 | claude-3-opus-20240229 API access | Anthropic | 88.5% | Log or source |
| 3 | GPT-4 (Mar 2023) API access | OpenAI | 87.5% | Log or source |
| 4 | gpt-4-32k-0314 API access | OpenAI | 87.5% | – |
| 5 | falcon-180B Open weights (restricted use) | Technology Innovation Institute | 87.1% | – |
| 6 | DeepSeek-V2 Open weights (restricted use) | DeepSeek | 86.3% | Log or source |
| 7 | DeepSeek-V3 Open weights (restricted use) | DeepSeek | 85.2% | Log or source |
| 8 | PaLM 540B Unreleased | Google Research | 85.1% | Log or source |
| 9 | DeepSeek-Coder-V2-Base Open weights (restricted use) | DeepSeek | 83.7% | Log or source |
| 10 | Meta-Llama-3-70B Open weights (restricted use) | Meta AI | 83.5% | Log or source |
| 11 | PaLM 2-L | – | 83% | Log or source |
| 12 | Qwen2.5-72B Open weights (unrestricted) | Alibaba | 82.3% | Log or source |
| 13 | Llama-3.1-405B-Instruct Open weights (restricted use) | Meta AI | 82.2% | Log or source |
| 14 | GPT-3.5 Turbo (June 2023) API access | OpenAI | 81.6% | – |
| 15 | text-davinci-002 API access | OpenAI | 81.6% | Log or source |
| 16 | Phi-3-medium-128k-instruct Open weights (unrestricted) | Microsoft | 81.5% | – |
| 17 | Phi-3-small-8k-instruct Open weights (unrestricted) | Microsoft | 81.5% | – |
| 18 | Qwen2.5-Coder-32B Open weights (unrestricted) | Alibaba | 80.8% | Log or source |
| 19 | Llama-2-70b-hf Open weights (restricted use) | Meta AI | 80.2% | Log or source |
| 20 | PaLM 2-M | – | 79.2% | Log or source |
| 21 | gemma-7b Open weights (restricted use) | Google DeepMind | 79% | Log or source |
| 22 | Megatron-Turing NLG 530B Unreleased | Microsoft,Nvidia | 78.9% | – |
| 23 | Falcon 2-11B Open weights (restricted use) | Technology Innovation Institute | 78.3% | – |
| 24 | Nemotron-4 15B Unreleased | Nvidia | 78% | – |
| 25 | PaLM 2-S | – | 77.9% | Log or source |
| 26 | Mixtral-8x7B-v0.1 Open weights (unrestricted) | Mistral AI | 77.2% | Log or source |
| 27 | LLaMA-65B Open weights (non-commercial) | Meta AI | 77% | Log or source |
| 28 | PaLM 62B | – | 77% | Log or source |
| 29 | falcon-40b Open weights (unrestricted) | Technology Innovation Institute | 76.9% | Log or source |
| 30 | Qwen2.5-Coder-14B | – | 76.8% | Log or source |
| 31 | Llama-2-34b Unreleased | Meta AI | 76.7% | Log or source |
| 32 | LLaMA-33B Open weights (non-commercial) | Meta AI | 76% | Log or source |
| 33 | Meta-Llama-3-8B Open weights (restricted use) | Meta AI | 75.7% | Log or source |
| 34 | Mistral-7B-v0.1 Open weights (unrestricted) | Mistral AI | 75.3% | Log or source |
| 35 | claude-3-sonnet-20240229 API access | Anthropic | 75.1% | Log or source |
| 36 | Chinchilla (70B) Unreleased | DeepMind | 74.9% | Log or source |
| 37 | claude-3-haiku-20240307 API access | Anthropic | 74.2% | Log or source |
| 38 | GLaM (MoE) Unreleased | 73.5% | – | |
| 39 | phi-1_5 Open weights (unrestricted) | Microsoft | 73.4% | – |
| 40 | LLaMA-13B Open weights (non-commercial) | Meta AI | 73% | Log or source |
| 41 | Yi-9B | – | 73% | – |
| 42 | DeepSeek-Coder-V2-Lite-Base | – | 72.9% | Log or source |
| 43 | Qwen2.5-Coder-7B Open weights (unrestricted) | Alibaba | 72.9% | Log or source |
| 44 | Llama-2-13b Open weights (restricted use) | Meta AI | 72.8% | Log or source |
| 45 | Yi-6B Open weights (restricted use) | 01.AI | 71.3% | – |
| 46 | mpt-30b Open weights (unrestricted) | MosaicML | 71% | Log or source |
| 47 | Phi-3-mini-4k-instruct Open weights (unrestricted) | Microsoft | 70.8% | – |
| 48 | vicuna-13b-v1.1 | – | 70.8% | – |
| 49 | text-davinci-001 API access | OpenAI | 70.2% | Log or source |
| 50 | Gopher (280B) Unreleased | DeepMind | 70.1% | Log or source |
| 51 | LLaMA-7B Open weights (non-commercial) | Meta AI | 70.1% | Log or source |
| 52 | Llama-2-7b Open weights (restricted use) | Meta AI | 69.2% | Log or source |
| 53 | GPT-3.5 Turbo (Nov 2023) API access | OpenAI | 68.8% | – |
| 54 | mpt-7b Open weights (unrestricted) | MosaicML | 68.3% | Log or source |
| 55 | Qwen2.5-Coder-3B | – | 67.4% | Log or source |
| 56 | open_llama_7b | – | 67% | – |
| 57 | falcon-7b Open weights (unrestricted) | Technology Innovation Institute | 66.3% | Log or source |
| 58 | gpt-neox-20b Open weights (unrestricted) | EleutherAI | 66.1% | – |
| 59 | INTELLECT-1-Instruct | Prime Intellect,Hugging Face,Arcee AI | 65.8% | Log or source |
| 60 | gemma-2b Open weights (restricted use) | Google DeepMind | 65.4% | Log or source |
| 61 | Meta-Llama-3-8B-Instruct Open weights (restricted use) | Meta AI | 65% | – |
| 62 | xgen-7b-8k-base Open weights (unrestricted) | Salesforce | 64.9% | – |
| 63 | opt-13b | – | 64.7% | – |
| 64 | gpt-j-6b Open weights (unrestricted) | EleutherAI,LAION | 64.5% | – |
| 65 | starcoder2-15b Open weights (restricted use) | Hugging Face,ServiceNow,Nvidia,BigCode | 64.3% | Log or source |
| 66 | RedPajama-INCITE-7B-Base | – | 63.8% | – |
| 67 | deepseek-coder-33b-base Open weights (restricted use) | DeepSeek,Peking University | 62% | Log or source |
| 68 | dolly-v2-12b Open weights (unrestricted) | Databricks | 61.8% | – |
| 69 | opt-1.3b Open weights (non-commercial) | Meta AI | 61% | – |
| 70 | Cerebras-GPT-13B Open weights (unrestricted) | Cerebras Systems | 60.8% | – |
| 71 | Qwen2.5-Coder-1.5B Open weights (unrestricted) | Alibaba | 60.7% | Log or source |
| 72 | CodeQwen1.5-7B | – | 59.8% | Log or source |
| 73 | gpt2-xl Open weights (unrestricted) | OpenAI | 58.3% | – |
| 74 | deepseek-coder-6.7b-base Open weights (restricted use) | DeepSeek,Peking University | 57.6% | Log or source |
| 75 | starcoder2-3b Open weights (restricted use) | Hugging Face,ServiceNow,Nvidia,BigCode | 57.1% | Log or source |
| 76 | starcoder2-7b Open weights (restricted use) | Hugging Face,ServiceNow,Nvidia,BigCode | 57.1% | Log or source |
| 77 | Qwen2.5-Coder-0.5B | – | 54.8% | Log or source |
| 78 | phi-2 Open weights (unrestricted) | Microsoft | 54.7% | – |
| 79 | deepseek-coder-1.3b-base Open weights (restricted use) | DeepSeek,Peking University | 53.3% | Log or source |
| 80 | stablelm-tuned-alpha-7b | – | 51.5% | – |
