Open-weight reasoning model that triggered the DeepSeek market shock
SI v3 uses weighted percentages from one Artificial Analysis configuration. Ranking requires all five benchmarks. Partial scores average only available weights. Scores are not comparable to SI v2 or a probability of reaching singularity.
Compared with 21 models released within six months of it, across 5 benchmarks where scores spread out.
| Benchmark | Score | Rank |
|---|---|---|
MATH Competition-level mathematics problems | 97.3% | #10 / 50 |
ARC-C Grade-school science questions requiring reasoning | 97.1% | #19 / 40 |
MMLU Tests knowledge across 57 subjects from STEM to humanities | 90.8% | #20 / 54 |
HumanEval Coding ability - generating correct Python functions | 92.8% | #25 / 50 |
Arena Elo Human preference ranking via blind comparisons | 1358 | #34 / 51 |
SWE-bench Real-world GitHub issue resolution | 49.2% | #35 / 39 |
TerminalArtificial Analysis Agentic terminal coding tasks requiring multi-step execution | 15.9% | #50 / 58 |
GPQA PhD-level science questions even experts struggle with | 71.5% | #80 / 96 |
HLEArtificial Analysis Challenging multidisciplinary questions evaluated by Artificial Analysis | 15.8% | #82 / 100 |
| Benchmark / source | Score |
|---|---|
HLE Artificial AnalysisDeepSeek R1 0528 (May '25) | 15.8% |
AA-LCR Artificial AnalysisDeepSeek R1 0528 (May '25) | 55.7% |
CritPt Artificial AnalysisDeepSeek R1 0528 (May '25) | 1.4% |
IFBench Artificial AnalysisDeepSeek R1 0528 (May '25) | 39.6% |