AI coding model comparison
📅 Data snapshot: March 2026
| Model | Family | Copilot | $/task | SWE-bench | Aider | Arena | LiveBench |
|---|---|---|---|---|---|---|---|
| GPT-5.4 | OpenAI | 1× | $0.28 | - | - | - | 80.3 |
| Gemini 3.1 Pro | 1× | $0.22 | 69.6% | - | - | 79.9 | |
| Claude Opus 4.6 | Anthropic | 3× | $0.50 | 75.6% | - | - | 76.3 |
| Claude Opus 4.5 thinking-32k | Anthropic | - | $0.50 | 76.8% | 72.0% | 1497 | 76.0 |
| Claude Sonnet 4.6 | Anthropic | 1× | $0.30 | - | - | - | 75.5 |
| Claude Opus 4.5 | Anthropic | 3× | $0.50 | 76.8% | 70.7% | 1468 | 59.1 |
| Minimax M2.5 | Minimax | - | $0.07 | 75.8% | - | - | 60.1 |
| GPT-5.2 high reasoning | OpenAI | 1× | $0.23 | 72.8% | 88.0% | 1470 | 74.8 |
| GLM-5 | Zhipu | - | $0.05 | 72.8% | - | - | 68.9 |
| Claude Sonnet 4.5 | Anthropic | 1× | $0.30 | 71.4% | 82.4% | 1383 | 53.7 |
| Kimi K2.5 | Moonshot | - | $0.15 | 70.8% | - | - | 69.1 |
| DeepSeek V3.2 Reasoner | DeepSeek | - | $0.02 | 70.0% | 74.2% | 1350 | 62.2 |
| GPT-5.2 | OpenAI | 1× | $0.23 | 72.8% | 88.0% | 1432 | 48.9 |
| Claude Opus 4.1 | Anthropic | 10× | $1.50 | 67.6% | 82.1% | 1431 | 54.5 |
| Claude Haiku 4.5 | Anthropic | 0.33× | $0.10 | 66.6% | 73.5% | 1290 | 45.3 |
| GPT-5 | OpenAI | 1× | $0.16 | 65.0% | 88.0% | 1407 | 70.5 |
| Kimi K2 Thinking Turbo | Moonshot | - | $0.06 | 63.4% | 59.1% | 1356 | 61.6 |
| Minimax M2 | Minimax | - | $0.03 | 61.0% | - | 1408 | - |
| o3 | OpenAI | - | $0.18 | 58.4% | 81.3% | 1417 | - |
| GPT-5 mini | OpenAI | 0× | $0.03 | 56.2% | 50.2% | 1145 | - |
| GLM-4.6 | Zhipu | - | $0.05 | 55.4% | - | - | 55.2 |
| Devstral 2 | Mistral | - | - | 53.8% | - | 1363 | 41.2 |
| Gemini 2.5 Pro | 1× | $0.16 | 53.6% | 83.1% | 1372 | 58.3 | |
| GPT-5.1 | OpenAI | 1× | $0.16 | - | - | - | 72.0 |
| Grok 4 | xAI | - | - | - | 79.6% | - | 62.0 |
| Grok 4.1 Fast | xAI | 0.25× | - | - | - | 1393 | 60.0 |
| Gemini 3 Flash | 0.33× | $0.06 | 75.8% | - | 1443 | 72.4 | |
| GPT-4o | OpenAI | 0× | $0.23 | 48.9% | 72.9% | 1372 | - |
| GLM-4.7 | Zhipu | - | $0.05 | - | - | 1440 | 58.1 |
| Minimax M2.1 preview | Minimax | - | $0.03 | - | - | 1408 | - |
| o4-mini | OpenAI | - | $0.10 | 45.0% | 72.0% | 1310 | - |
| GPT-4.1 | OpenAI | 0× | $0.18 | 39.6% | 52.4% | 1305 | - |
| DeepSeek V3.2 Chat | DeepSeek | - | $0.02 | 39.0% | 70.2% | 1287 | 51.8 |
| Gemini 2.5 Flash | - | $0.04 | 28.7% | 55.1% | 1233 | 47.7 | |
| Gemini 2.0 Flash | - | $0.01 | 22.0% | 58.0% | 1214 | - | |
| GPT-4o-mini | OpenAI | - | $0.01 | 18.6% | 55.6% | 1176 | - |
Column guide
| Column | What it means |
|---|---|
| Copilot | GitHub Copilot premium request multiplier (0× = free, 1× = standard, 3× = expensive, - = not available) |
| $/task | Estimated cost per task if using APIs directly (50K in + 10K out tokens). Useful for comparing relative model costs - Copilot users pay via the multiplier instead. |
| SWE-bench | % of real GitHub issues the model can fix autonomously (source) - February 2026 data (standardized harness, high reasoning mode) |
| Aider | % correct on multi-language code editing (source) - October 2025 data |
| Arena | Elo rating from human preference voting on Code category (source) - February 2026 data |
| LiveBench | Global average score across 23 diverse tasks (source) - January 2026 data, contamination-free |
Data sources:
SWE-bench (Feb 2026) ·
Aider (Oct 2025) ·
Arena Code (Feb 2026, not refreshed) ·
LiveBench (Jan 2026) ·
GitHub Copilot
API pricing: Anthropic · OpenAI · Google · DeepSeek · Zhipu (GLM)
API pricing: Anthropic · OpenAI · Google · DeepSeek · Zhipu (GLM)