Opus-level performance at one-fifth the cost, default model on claude.ai
SI v3 uses weighted percentages from one Artificial Analysis configuration. Ranking requires all five benchmarks. Partial scores average only available weights. Scores are not comparable to SI v2 or a probability of reaching singularity.
Compared with 57 models released within six months of it, across 11 benchmarks where scores spread out.
| Benchmark | Score | Rank |
|---|---|---|
OSWorld Computer use in real desktop environments | 72.5% | #6 / 13 |
SWE-bench Real-world GitHub issue resolution | 79.6% | #7 / 39 |
Terminal Agentic terminal coding tasks requiring multi-step execution | 59.1% | #15 / 58 |
Arena Elo Human preference ranking via blind comparisons | 1459 | #18 / 51 |
MMLU Tests knowledge across 57 subjects from STEM to humanities | 89.3% | #24 / 54 |
ARC-AGI Novel reasoning tasks requiring fluid intelligence | 60.4% | #24 / 40 |
HumanEval Coding ability - generating correct Python functions | 92.1% | #27 / 50 |
MMMUvals.ai College-level multimodal reasoning across 30+ disciplines | 83.6% | #28 / 51 |
MMLU-Provals.ai Harder 10-option successor to MMLU; more reasoning-focused | 87.3% | #29 / 61 |
MATH Competition-level mathematics problems | 85.3% | #30 / 50 |
LiveCodeBenchvals.ai Contamination-free competitive programming (filtered by cutoff date) | 82.1% | #38 / 62 |
HLEArtificial Analysis Challenging multidisciplinary questions evaluated by Artificial Analysis | 33.6% | #60 / 100 |
GPQA PhD-level science questions even experts struggle with | 74.1% | #77 / 96 |
| Benchmark / source | Score |
|---|---|
HLE Artificial AnalysisClaude Sonnet 4.6 (Max) | 33.6% |
MMMU-Pro Artificial AnalysisClaude Sonnet 4.6 (Max) | 73.3% |
Terminal-Bench 2.1 Artificial AnalysisClaude Sonnet 4.6 (Max) | 71.2% |
Terminal-Bench 4.0 Artificial AnalysisClaude Sonnet 4.6 (Max) | 3% |
SciCode Artificial AnalysisClaude Sonnet 4.6 (Max) | 50.1% |
AA-LCR Artificial AnalysisClaude Sonnet 4.6 (Max) | 80% |
CritPt Artificial AnalysisClaude Sonnet 4.6 (Max) | 3.1% |
ITBench SRE Artificial AnalysisClaude Sonnet 4.6 (Max) | 39.8% |
IFBench Artificial AnalysisClaude Sonnet 4.6 (Max) | 56.6% |