Aider polyglot
Each model’s result as Epoch AI’s Benchmarking Hub publishes it, run by Epoch or collected from the source it cites. Facthory has not run them.
Data: Epoch AI, ‘Capabilities & benchmarking’, epoch.ai/benchmarks, licensed CC BY 4.0. Read on 16 Nov 2026.
The chart shows the 25 best of 71 results; the table lists them all.
| # | Model | Organization | Result | Evidence |
|---|---|---|---|---|
| 1 | GPT-5 (high) API access | OpenAI | 88% | Log or source |
| 2 | GPT-5 (medium) API access | OpenAI | 86.7% | Log or source |
| 3 | o3-pro-2025-06-10_high API access | OpenAI | 84.9% | Log or source |
| 4 | Gemini 2.5 Pro Preview (Jun 2025) API access | Google DeepMind | 83.1% | Log or source |
| 5 | GPT-5 (low) API access | OpenAI | 81.3% | Log or source |
| 6 | o3 (high) API access | OpenAI | 81.3% | Log or source |
| 7 | grok-4-0709 API access | xAI | 79.6% | Log or source |
| 8 | grok-4-0709_high API access | xAI | 79.6% | – |
| 9 | Gemini 2.5 Pro Preview (Jun 2025) API access | Google DeepMind | 79.1% | Log or source |
| 10 | Gemini 2.5 Pro Preview (Jun 2025) API access | Google DeepMind | 76.9% | Log or source |
| 11 | o3 (medium) API access | OpenAI | 76.9% | Log or source |
| 12 | o3 (unknown thinking) API access | OpenAI | 76.9% | – |
| 13 | DeepSeek-V3.2-Exp_thinking Open weights (unrestricted) | DeepSeek | 74.2% | Log or source |
| 14 | DeepSeek-V3.2 (Thinking) Open weights (unrestricted) | DeepSeek | 74.2% | – |
| 15 | Gemini 2.5 Pro Exp (Mar 2025) API access | Google DeepMind | 72.9% | Log or source |
| 16 | Gemini 2.5 Pro Preview (Mar 2025) API access | Google DeepMind | 72.9% | – |
| 17 | Claude Opus 4 API access | Anthropic | 72% | Log or source |
| 18 | o4-mini (high) API access | OpenAI | 72% | Log or source |
| 19 | DeepSeek-R1 (May 2025) Open weights (unrestricted) | DeepSeek | 71.4% | Log or source |
| 20 | Claude Opus 4 API access | Anthropic | 70.7% | Log or source |
| 21 | DeepSeek-V3.2-Exp Open weights (unrestricted) | DeepSeek | 70.2% | Log or source |
| 22 | deepseek-chat Open weights (unrestricted) | DeepSeek | 70.2% | – |
| 23 | Claude 3.7 Sonnet (32k thinking) API access | Anthropic | 64.9% | Log or source |
| 24 | o1 (high) API access | OpenAI | 61.7% | Log or source |
| 25 | claude-sonnet-4-20250514_32K API access | Anthropic | 61.3% | Log or source |
| 26 | claude-3-7-sonnet-20250219 API access | Anthropic | 60.4% | Log or source |
| 27 | o3-mini (high) API access | OpenAI | 60.4% | Log or source |
| 28 | Qwen3-235B-A22B Open weights (unrestricted) | Alibaba | 59.6% | Log or source |
| 29 | Qwen3-235B-A22B-Instruct-2507 Open weights (unrestricted) | Alibaba | 59.6% | – |
| 30 | Kimi K2 Instruct Open weights (restricted use) | Moonshot | 59.1% | Log or source |
| 31 | Kimi K2 0905 (Novita) Open weights (restricted use) | Moonshot | 59.1% | – |
| 32 | DeepSeek-R1 Open weights (unrestricted) | DeepSeek | 56.9% | Log or source |
| 33 | claude-sonnet-4-20250514 API access | Anthropic | 56.4% | Log or source |
| 34 | DeepSeek-V3 (Mar 2025) Open weights (restricted use) | DeepSeek | 55.1% | Log or source |
| 35 | gemini-2.5-flash-preview-05-20_23K API access | Google DeepMind | 55.1% | Log or source |
| 36 | o3-mini (medium) API access | OpenAI | 53.8% | Log or source |
| 37 | grok-3-beta API access | xAI | 53.3% | Log or source |
| 38 | GPT-4.1 API access | OpenAI | 52.4% | Log or source |
| 39 | Claude 3.5 Sonnet (Oct 2024) API access | Anthropic | 51.6% | Log or source |
| 40 | grok-3-mini-beta_high API access | xAI | 49.3% | Log or source |
| 41 | DeepSeek-V3 Open weights (restricted use) | DeepSeek | 48.4% | Log or source |
| 42 | Gemini 2.5 Flash Preview (Apr 2025) API access | Google DeepMind | 47.1% | Log or source |
| 43 | chatgpt-4o-03-27-2025 API access | OpenAI | 45.3% | Log or source |
| 44 | GPT-4.5 Preview (Feb 2025) API access | OpenAI | 44.9% | Log or source |
| 45 | gemini-2.5-flash-preview-05-20 API access | Google DeepMind | 44% | Log or source |
| 46 | gpt-oss-120b_high Open weights (unrestricted) | OpenAI | 41.8% | – |
| 47 | openai/gpt-oss-120b_high Open weights (unrestricted) | OpenAI | 41.8% | – |
| 48 | Qwen3-32B Open weights (unrestricted) | Alibaba | 40% | Log or source |
| 49 | gemini-exp-1206 API access | Google DeepMind,Google | 38.2% | Log or source |
| 50 | Gemini 2.0 Pro Exp (Feb 2025) Hosted access (no API) | Google DeepMind | 35.6% | Log or source |
| 51 | grok-3-mini-beta_low API access | xAI | 34.7% | Log or source |
| 52 | o1-mini (unknown thinking) API access | OpenAI | 32.9% | – |
| 53 | GPT-4.1 mini API access | OpenAI | 32.4% | Log or source |
| 54 | Claude 3.5 Haiku (Oct 2024) API access | Anthropic | 28% | Log or source |
| 55 | chatgpt-4o-01-29-2025 API access | OpenAI | 27.1% | Log or source |
| 56 | GPT-4o (Aug 2024) API access | OpenAI | 23.1% | Log or source |
| 57 | gemini-2.0-flash-exp API access | Google DeepMind,Google | 22.2% | Log or source |
| 58 | qwen-max-2025-01-25 API access | Alibaba | 21.8% | Log or source |
| 59 | QwQ-32B Open weights (unrestricted) | Alibaba | 20.9% | Log or source |
| 60 | Gemini 2.0 Flash Thinking Exp API access | Google DeepMind,Google | 18.2% | Log or source |
| 61 | GPT-4o (Nov 2024) API access | OpenAI | 18.2% | Log or source |
| 62 | DeepSeek-V2.5 Open weights (restricted use) | DeepSeek | 17.8% | Log or source |
| 63 | Llama 4 Maverick Open weights (restricted use) | Meta AI | 15.6% | Log or source |
| 64 | yi-lightning API access | 01.AI | 12.9% | Log or source |
| 65 | c4ai-command-a-03-2025 Open weights (non-commercial) | Cohere | 12% | Log or source |
| 66 | codestral-2501 Open weights (non-commercial) | Mistral AI | 11.1% | Log or source |
| 67 | openhands-lm-32b-v0.1 | – | 10.2% | Log or source |
| 68 | GPT-4.1 nano API access | OpenAI | 8.9% | Log or source |
| 69 | Qwen2.5-Coder-32B-Instruct Open weights (unrestricted) | Alibaba | 8% | Log or source |
| 70 | gemma-3-27b-it Open weights (restricted use) | Google DeepMind | 4.9% | Log or source |
| 71 | gpt-4o-mini-2024-07-18 API access | OpenAI | 3.6% | Log or source |
