Knowledge Cutoff Bench
Every model answers the same questions about things that happened in one known month, from memory, with web search off and no date in the prompt. Run 2026-10-15.
Leaderboard
| # | Model | Lab | Real cutoff | Lab says | Recall | Cost of one run |
|---|---|---|---|---|---|---|
| 1 | GPT-6 Sol | OpenAI | Apr 2026 | Apr 2026 | 25.0 | $0.069 |
| 2 | Claude Opus 5.5 | Anthropic | Mar 2026 | Jun 2026 | 22.2 | $0.17 |
| 3 | GPT-6 Astra | OpenAI | Feb 2026 | Apr 2026 | 16.7 | $0.26 |
| 4 | DeepSeek V4.1 Flash | DeepSeek | Dec 2025 | Not stated | 8.3 | $0.11 |
| 5 | GPT-6 Luna | OpenAI | Feb 2026 | May 2026 | 8.3 | $0.006 |
| 6 | Claude Sonnet 5 | Anthropic | Dec 2025 | Jan 2026 | 2.8 | $0.040 |
| 7 | Kimi K3 | Moonshot | Nov 2025 | Not stated | 2.8 | $0.40 |
| 8 | Qwen3.8 Max | Qwen | Sept 2025 | Not stated | 2.8 | $0.14 |
| 9 | Gemini 3.5 Flash-Lite | Feb 2025 | Mar 2026 | 0.0 | $0.058 | |
| 10 | Claude Fable 5.1 | Anthropic | Dec 2025 | Jun 2026 | 0.0 | $0.20 |
| 11 | GLM-5.3 | Z.ai | Nov 2025 | Not stated | 0.0 | $0.028 |
| 12 | GLM-5.3 Flash | Z.ai | Dec 2025 | Not stated | 0.0 | $0.003 |
| 13 | DeepSeek V4 Pro | DeepSeek | Jun 2025 | Not stated | 0.0 | $0.15 |
| 14 | Gemini 3.1 Pro | Jan 2025 | Jan 2025 | 0.0 | $0.45 | |
| 15 | Grok 4.7 | xAI | Aug 2025 | May 2026 | 0.0 | $0.35 |
| 16 | Gemini 3.8 Flash | Mar 2025 | Mar 2026 | 0.0 | $0.068 | |
| 17 | Qwen3.8 Flash | Qwen | Mar 2025 | Not stated | 0.0 | $0.048 |
Sample questions
Real questions from the bench, with the right answer and what every model replied: all 8 questions and answers.
How the bench works
Four questions a month, one each on world news, science and technology, sport and culture, and money and prices. Every answer and its date come from the organiser's or the authority's own record, re-derived by a second agent without seeing the first. The real cutoff is the last month whose three month average is at least half right. A model that says it does not know scores zero, the same as a wrong answer, and is counted apart.