First Flash model with thinking, near-Pro performance at one-eighth the price
SI v3 uses weighted percentages from one Artificial Analysis configuration. Ranking requires all five benchmarks. Partial scores average only available weights. Scores are not comparable to SI v2 or a probability of reaching singularity.
Compared with 27 models released within six months of it, across 7 benchmarks where scores spread out.
| Benchmark | Score | Rank |
|---|---|---|
SWE-bench Real-world GitHub issue resolution | 60.4% | #30 / 39 |
MMLU Tests knowledge across 57 subjects from STEM to humanities | 88.4% | #32 / 54 |
MMMUvals.ai College-level multimodal reasoning across 30+ disciplines | 80.8% | #36 / 51 |
TerminalArtificial Analysis Agentic terminal coding tasks requiring multi-step execution | 16.7% | #49 / 58 |
LiveCodeBenchvals.ai Contamination-free competitive programming (filtered by cutoff date) | 76.2% | #50 / 62 |
MMLU-Provals.ai Harder 10-option successor to MMLU; more reasoning-focused | 83.7% | #51 / 61 |
GPQA PhD-level science questions even experts struggle with | 82.8% | #69 / 96 |
HLEArtificial Analysis Challenging multidisciplinary questions evaluated by Artificial Analysis | 13.8% | #84 / 100 |
| Benchmark / source | Score |
|---|---|
HLE Artificial AnalysisGemini 2.5 Flash Preview (Sep '25) (Reasoning) | 13.8% |
MMMU-Pro Artificial AnalysisGemini 2.5 Flash Preview (Sep '25) (Reasoning) | 73.1% |
AA-LCR Artificial AnalysisGemini 2.5 Flash Preview (Sep '25) (Reasoning) | 71% |
CritPt Artificial AnalysisGemini 2.5 Flash Preview (Sep '25) (Reasoning) | 0.3% |
IFBench Artificial AnalysisGemini 2.5 Flash Preview (Sep '25) (Reasoning) | 52.3% |