OpenBookQA
Each model’s result as Epoch AI’s Benchmarking Hub publishes it, run by Epoch or collected from the source it cites. Facthory has not run them.
Data: Epoch AI, ‘Capabilities & benchmarking’, epoch.ai/benchmarks, licensed CC BY 4.0. Read on 16 Nov 2026.
The chart shows the 25 best of 42 results; the table lists them all.
| # | Model | Organization | Result | Evidence |
|---|---|---|---|---|
| 1 | Phi-3-mini-4k-instruct Open weights (unrestricted) | Microsoft | 88% | – |
| 2 | Phi-3-small-8k-instruct Open weights (unrestricted) | Microsoft | 88% | – |
| 3 | Phi-3-medium-128k-instruct Open weights (unrestricted) | Microsoft | 87.4% | – |
| 4 | GPT-3.5 Turbo (Nov 2023) API access | OpenAI | 86% | – |
| 5 | Mixtral-8x7B-v0.1 Open weights (unrestricted) | Mistral AI | 85.8% | – |
| 6 | Meta-Llama-3-8B-Instruct Open weights (restricted use) | Meta AI | 82.6% | – |
| 7 | Mistral-7B-v0.1 Open weights (unrestricted) | Mistral AI | 79.8% | – |
| 8 | gemma-7b Open weights (restricted use) | Google DeepMind | 78.6% | – |
| 9 | phi-2 Open weights (unrestricted) | Microsoft | 73.6% | – |
| 10 | PaLM 540B Unreleased | Google Research | 68% | Log or source |
| 11 | text-davinci-001 API access | OpenAI | 65.4% | – |
| 12 | falcon-180B Open weights (restricted use) | Technology Innovation Institute | 64.2% | – |
| 13 | GLaM (MoE) Unreleased | 63% | – | |
| 14 | LLaMA-65B Open weights (non-commercial) | Meta AI | 60.2% | – |
| 15 | Llama-2-70b-hf Open weights (restricted use) | Meta AI | 60.2% | – |
| 16 | LLaMA-33B Open weights (non-commercial) | Meta AI | 58.6% | – |
| 17 | Llama-2-7b Open weights (restricted use) | Meta AI | 58.6% | – |
| 18 | Llama-2-34b Unreleased | Meta AI | 58.2% | – |
| 19 | PaLM 2-M | – | 57.4% | Log or source |
| 20 | LLaMA-7B Open weights (non-commercial) | Meta AI | 57.2% | – |
| 21 | Llama-2-13b Open weights (restricted use) | Meta AI | 57% | – |
| 22 | falcon-40b Open weights (unrestricted) | Technology Innovation Institute | 56.6% | – |
| 23 | LLaMA-13B Open weights (non-commercial) | Meta AI | 56.4% | – |
| 24 | PaLM 2-S | – | 56.2% | Log or source |
| 25 | mpt-30b Open weights (unrestricted) | MosaicML | 52% | – |
| 26 | falcon-7b Open weights (unrestricted) | Technology Innovation Institute | 51.6% | – |
| 27 | mpt-7b Open weights (unrestricted) | MosaicML | 51.4% | – |
| 28 | PaLM 62B | – | 50.4% | – |
| 29 | xgen-7b-8k-base Open weights (unrestricted) | Salesforce | 40.2% | – |
| 30 | RedPajama-INCITE-7B-Base | – | 40% | – |
| 31 | opt-13b | – | 39.8% | – |
| 32 | dolly-v2-12b Open weights (unrestricted) | Databricks | 39.2% | – |
| 33 | open_llama_7b | – | 39% | – |
| 34 | gpt-neox-20b Open weights (unrestricted) | EleutherAI | 38.8% | – |
| 35 | gpt-j-6b Open weights (unrestricted) | EleutherAI,LAION | 38.2% | – |
| 36 | phi-1_5 Open weights (unrestricted) | Microsoft | 37.2% | – |
| 37 | Cerebras-GPT-13B Open weights (unrestricted) | Cerebras Systems | 35.8% | – |
| 38 | vicuna-13b-v1.1 | – | 33% | – |
| 39 | stablelm-tuned-alpha-7b | – | 32.4% | – |
| 40 | opt-1.3b Open weights (non-commercial) | Meta AI | 24% | – |
| 41 | gpt-neo-2.7B Open weights (unrestricted) | EleutherAI | 23.2% | – |
| 42 | gpt2-xl Open weights (unrestricted) | OpenAI | 22.4% | – |
