Highest GPQA Diamond score ever at 94.3%, doubled ARC-AGI-2 to 77.1%
SI v3 uses weighted percentages from one Artificial Analysis configuration. Ranking requires all five benchmarks. Partial scores average only available weights. Scores are not comparable to SI v2 or a probability of reaching singularity.
Compared with 58 models released within six months of it, across 10 benchmarks where scores spread out.
| Benchmark | Score | Rank |
|---|---|---|
SWE-bench Real-world GitHub issue resolution | 80.6% | #4 / 39 |
MMLU-Provals.ai Harder 10-option successor to MMLU; more reasoning-focused | 91% | #4 / 61 |
GPQA PhD-level science questions even experts struggle with | 94.3% | #5 / 96 |
Arena Elo Human preference ranking via blind comparisons | 1520 | #6 / 51 |
LiveCodeBenchvals.ai Contamination-free competitive programming (filtered by cutoff date) | 88.5% | #6 / 62 |
MMLU Tests knowledge across 57 subjects from STEM to humanities | 92.6% | #7 / 54 |
MMMUvals.ai College-level multimodal reasoning across 30+ disciplines | 88.2% | #10 / 51 |
Terminal Agentic terminal coding tasks requiring multi-step execution | 68.5% | #11 / 58 |
ARC-AGI Novel reasoning tasks requiring fluid intelligence | 77.1% | #13 / 40 |
HumanEval Coding ability - generating correct Python functions | 94.2% | #15 / 50 |
MATH Competition-level mathematics problems | 95.1% | #17 / 50 |
HLEArtificial Analysis Challenging multidisciplinary questions evaluated by Artificial Analysis | 47% | #18 / 100 |
| Benchmark / source | Score |
|---|---|
HLE Artificial AnalysisGemini 3.1 Pro Preview | 47% |
MMMU-Pro Artificial AnalysisGemini 3.1 Pro Preview | 82.4% |
Terminal-Bench 2.1 Artificial AnalysisGemini 3.1 Pro Preview | 73.8% |
Terminal-Bench 4.0 Artificial AnalysisGemini 3.1 Pro Preview | 4% |
SciCode Artificial AnalysisGemini 3.1 Pro Preview | 58.7% |
AA-LCR Artificial AnalysisGemini 3.1 Pro Preview | 82% |
CritPt Artificial AnalysisGemini 3.1 Pro Preview | 17.7% |
ITBench SRE Artificial AnalysisGemini 3.1 Pro Preview | 30.3% |
IFBench Artificial AnalysisGemini 3.1 Pro Preview | 77.1% |