Iteration on Meta's proprietary Superintelligence Labs model, with gains in reasoning and coding.
| Benchmark | Score | Rank |
|---|---|---|
hleArtificial Analysis | 46.2% | #12 / 85 |
aaAgentic | 37.5% | #15 / 16 |
MMLU-Provals.ai Harder 10-option successor to MMLU; more reasoning-focused | 88.7% | #17 / 61 |
MMMUvals.ai College-level multimodal reasoning across 30+ disciplines | 86.6% | #17 / 51 |
AA Coding Independent composite of implementation, terminal use, and real-codebase agent performance | 71.3% | #18 / 22 |
AA Intelligence Independent composite of agentic work, coding, scientific reasoning, and general capability | 50.6% | #20 / 22 |
LiveCodeBenchvals.ai Contamination-free competitive programming (filtered by cutoff date) | 85.9% | #24 / 62 |
GPQAArtificial Analysis PhD-level science questions even experts struggle with | 89.8% | #41 / 95 |