Four specialized agents debate before answering, 65% hallucination reduction
SI v3 uses weighted percentages from one Artificial Analysis configuration. Ranking requires all five benchmarks. Partial scores average only available weights. Scores are not comparable to SI v2 or a probability of reaching singularity.
Compared with models released within six months of it.
| Benchmark | Score | Rank |
|---|---|---|
Arena Elo Human preference ranking via blind comparisons | 1491 | #12 / 51 |
SWE-bench Real-world GitHub issue resolution | 75% | #15 / 39 |
ARC-AGIARC Prize Novel reasoning tasks requiring fluid intelligence | 65.1% | #20 / 40 |
TerminalArtificial Analysis Agentic terminal coding tasks requiring multi-step execution | 37.9% | #35 / 58 |
MMLU Tests knowledge across 57 subjects from STEM to humanities | 86.6% | #38 / 54 |
GPQA PhD-level science questions even experts struggle with | 88% | #55 / 96 |
HLEArtificial Analysis Challenging multidisciplinary questions evaluated by Artificial Analysis | 34.5% | #56 / 100 |
| Benchmark / source | Score |
|---|---|
HLE Artificial AnalysisGrok 4.20 0309 v2 (Reasoning) | 34.5% |
MMMU-Pro Artificial AnalysisGrok 4.20 0309 v2 (Reasoning) | 74.6% |
AA-LCR Artificial AnalysisGrok 4.20 0309 v2 (Reasoning) | 69% |
CritPt Artificial AnalysisGrok 4.20 0309 v2 (Reasoning) | 6.6% |
IFBench Artificial AnalysisGrok 4.20 0309 v2 (Reasoning) | 81.2% |