First 10M token context window, 17B active params from 109B MoE
SI v3 uses weighted percentages from one Artificial Analysis configuration. Ranking requires all five benchmarks. Partial scores average only available weights. Scores are not comparable to SI v2 or a probability of reaching singularity.
Compared with models released within six months of it.
| Benchmark | Score | Rank |
|---|---|---|
MATH Competition-level mathematics problems | 50.3% | #45 / 50 |
MMLU Tests knowledge across 57 subjects from STEM to humanities | 79.6% | #48 / 54 |
TerminalArtificial Analysis Agentic terminal coding tasks requiring multi-step execution | 1.5% | #58 / 58 |
GPQA PhD-level science questions even experts struggle with | 57.2% | #86 / 96 |
HLEArtificial Analysis Challenging multidisciplinary questions evaluated by Artificial Analysis | 3.8% | #97 / 100 |
| Benchmark / source | Score |
|---|---|
HLE Artificial AnalysisLlama 4 Scout | 3.8% |
MMMU-Pro Artificial AnalysisLlama 4 Scout | 52.9% |
Terminal-Bench 2.1 Artificial AnalysisLlama 4 Scout | 3.7% |
Terminal-Bench 4.0 Artificial AnalysisLlama 4 Scout | 0% |
SciCode Artificial AnalysisLlama 4 Scout | 21.3% |
AA-LCR Artificial AnalysisLlama 4 Scout | 27.7% |
CritPt Artificial AnalysisLlama 4 Scout | 0% |
IFBench Artificial AnalysisLlama 4 Scout | 39.5% |