Full reasoning model with tool use, first to break 80% on GPQA Diamond
SI v3 uses weighted percentages from one Artificial Analysis configuration. Ranking requires all five benchmarks. Partial scores average only available weights. Scores are not comparable to SI v2 or a probability of reaching singularity.
Compared with 22 models released within six months of it, across 8 benchmarks where scores spread out.
| Benchmark | Score | Rank |
|---|---|---|
MATH Competition-level mathematics problems | 91.6% | #23 / 50 |
SWE-bench Real-world GitHub issue resolution | 71.7% | #25 / 39 |
ARC-AGIARC Prize Novel reasoning tasks requiring fluid intelligence | 4.9% | #39 / 40 |
LiveCodeBenchvals.ai Contamination-free competitive programming (filtered by cutoff date) | 71.5% | #53 / 62 |
TerminalArtificial Analysis Agentic terminal coding tasks requiring multi-step execution | 6.1% | #56 / 58 |
MMLU-Provals.ai Harder 10-option successor to MMLU; more reasoning-focused | 78.7% | #57 / 61 |
GPQA PhD-level science questions even experts struggle with | 83.3% | #67 / 96 |
HLEArtificial Analysis Challenging multidisciplinary questions evaluated by Artificial Analysis | 12% | #88 / 100 |
| Benchmark / source | Score |
|---|---|
HLE Artificial Analysiso3-mini (High) | 12% |
Terminal-Bench 2.1 Artificial Analysiso3-mini (High) | 4.5% |
Terminal-Bench 4.0 Artificial Analysiso3-mini (High) | 0% |
SciCode Artificial Analysiso3-mini (High) | 42.8% |
AA-LCR Artificial Analysiso3-mini (High) | 43% |
CritPt Artificial Analysiso3-mini (High) | 0.3% |
IFBench Artificial Analysiso3-mini (High) | 67.1% |