Open-weight 397B MoE with visual agentic capabilities, 201 languages
SI v3 uses weighted percentages from one Artificial Analysis configuration. Ranking requires all five benchmarks. Partial scores average only available weights. Scores are not comparable to SI v2 or a probability of reaching singularity.
Compared with models released within six months of it.
| Benchmark | Score | Rank |
|---|---|---|
OSWorld Computer use in real desktop environments | 62.2% | #8 / 13 |
SWE-bench Real-world GitHub issue resolution | 76.4% | #12 / 39 |
Terminal Agentic terminal coding tasks requiring multi-step execution | 52.5% | #16 / 58 |
Arena Elo Human preference ranking via blind comparisons | 1451 | #21 / 51 |
LiveCodeBenchvals.ai Contamination-free competitive programming (filtered by cutoff date) | 85.3% | #28 / 62 |
MMLU Tests knowledge across 57 subjects from STEM to humanities | 88.5% | #31 / 54 |
MMLU-Provals.ai Harder 10-option successor to MMLU; more reasoning-focused | 87.2% | #32 / 61 |
MMMUvals.ai College-level multimodal reasoning across 30+ disciplines | 81.9% | #33 / 51 |
HumanEval Coding ability - generating correct Python functions | 84.8% | #39 / 50 |
GPQA PhD-level science questions even experts struggle with | 88.4% | #49 / 96 |
HLEArtificial Analysis Challenging multidisciplinary questions evaluated by Artificial Analysis | 14.9% | #83 / 100 |
| Benchmark / source | Score |
|---|---|
HLE Artificial AnalysisQwen3.5 Omni Plus | 14.9% |
MMMU-Pro Artificial AnalysisQwen3.5 Omni Plus | 70.5% |
AA-LCR Artificial AnalysisQwen3.5 Omni Plus | 63.7% |
CritPt Artificial AnalysisQwen3.5 Omni Plus | 0.6% |
IFBench Artificial AnalysisQwen3.5 Omni Plus | 51.2% |