Leaderboard Archive

To view historical performance data for previous models, select a version below.
Back to current leaderboard

  • July 8th 2026 Android LLM Benchmark

    Model Score (%) info Average percentage of 100 test cases successfully resolved across 5 runs for each model
    arrow_range Cl range (%) info Expected performance range, reflecting the results' statistical reliability (p-value < 0.05)
    Avg latency (h) info Average time taken to solve 100 tasks across 5 runs
    Avg cost ($) info Average cost per full benchmark run
    Claude Fable 5 84.5
    		79.9 — 88.8 8.0 133ドル.2
    GPT 5.5 80.2
    		73.5 — 86.6 11.4 138ドル.3
    Claude Sonnet 5 76.2
    		69.0 — 82.1 12.3 99ドル.9
    GPT 5.4 74.1
    		66.0 — 80.9 8.4 83ドル.4
    Gemini 3.1 Pro Preview 73.7
    		66.1 — 80.4 10.6 87ドル.4
    Claude Opus 4.8 72.4
    		65.8 — 79.3 6.7 88ドル.0
    GLM 5.2 72.2
    		65.3 — 78.7 38.9 117ドル.0
    Gemini 3.5 Flash 71.1
    		63.6 — 78.2 28.3 165ドル.6
    Kimi K2.7 Code 70.4
    		63.2 — 77.0 31.8 48ドル.1
    Claude Opus 4.7 68.7
    		60.9 — 76.4 7.0 96ドル.5
    Kimi K2.6 67.6
    		60.2 — 74.3 57.2 49ドル.4
    Claude Sonnet 4.6 67.0
    		58.3 — 75.4 16.9 127ドル.6
    MiniMax M3 63.6
    		56.3 — 70.3 26.0 41ドル.7
    GLM 5.1 63.2
    		56.0 — 71.3 17.6 53ドル.5
    Gemini 3 Flash Preview 62.5
    		54.2 — 70.0 13.1 30ドル.1
    MiMo-V2.5-Pro 60.8
    		53.1 — 68.3 13.6 9ドル.2
    Deepseek V4 Pro 59.5
    		51.7 — 66.9 9.0 3ドル.7
    Qwen 3.7 Plus 57.7
    		49.5 — 65.5 18.5 18ドル.6
    Deepseek V4 Flash 54.7
    		46.6 — 62.8 8.9 1ドル.5
    Qwen 3.7 Max 54.2
    		46.3 — 61.8 14.2 58ドル.3
    Qwen 3.6 27B 45.1
    		38.2 — 53.0 25.8 97ドル.3
    MiniMax M2.7 41.6
    		34.4 — 49.0 18.2 14ドル.9
    Qwen 3.6 35B A3B 37.0
    		29.5 — 44.4 16.3 17ドル.8
    Gemma 4 31B IT 36.3
    		29.3 — 43.2 38.9 10ドル.6
    Gemma 4 26B A4B IT 25.1
    		18.6 — 31.8 21.4 3ドル.3
    Latest results as of July 8th.
    View archived leaderboards and check back periodically for updates.

  • June 9th 2026 Android LLM Benchmark

    Model Score (%) info Average percentage of 100 test cases successfully resolved across 5 runs for each model
    arrow_range Cl range (%) info Expected performance range, reflecting the results' statistical reliability (p-value < 0.05)
    Avg latency (h) info Average time taken to solve 100 tasks across 5 runs
    Avg cost ($) info Average cost per full benchmark run
    GPT 5.5 74.0
    		66.9 — 80.6 15.7 134ドル.2
    GPT 5.4 72.4
    		65.4 — 79.0 21.2 91ドル.7
    Gemini 3.1 Pro Preview 72.4
    		65.2 — 79.0 11.1 47ドル.9
    Claude Opus 4.7 68.7
    		61.0 — 76.0 11.6 124ドル.3
    Claude Opus 4.6 66.6
    		59.2 — 74.0 9.9 84ドル.4
    Gemini 3.5 Flash 63.7
    		56.3 — 70.7 14.2 147ドル.1
    GLM 5.1 59.7
    		52.1 — 67.4 33.4 46ドル.7
    Kimi K2.6 58.6
    		51.3 — 66.1 29.9 42ドル.5
    Claude Sonnet 4.6 58.4
    		50.3 — 66.3 8.2 40ドル.4
    DeepSeek V4 Pro 55.4
    		47.9 — 63.5 35.8 13ドル.7
    Claude Sonnet 4.5 53.7
    		46.1 — 61.4 13.1 61ドル.0
    DeepSeek V4 Flash 52.7
    		45.1 — 60.2 28.1 8ドル.4
    MiMo 2.5 Pro 52.0
    		43.6 — 59.4 33.1 74ドル.5
    Qwen 3.6 Max Preview 51.4
    		44.1 — 59.2 20.5 222ドル.4
    Gemini 3 Flash Preview 42.0
    		36.2 — 48.2 16.5 34ドル.2
    Qwen 3.6 27B 37.4
    		30.3 — 44.7 20.7 64ドル.6
    MiniMax M2.7 37.2
    		30.2 — 44.3 20.3 10ドル.1
    Gemma 4 31B IT 33.2
    		26.0 — 40.8 14.2 2ドル.5
    Qwen 3.6 35B A3B 31.7
    		24.7 — 39.0 12.5 10ドル.7
    Gemma 4 26B A4B IT 25.1
    		18.6 — 31.8 21.4 3ドル.3
    GPT OSS 120B 18.9
    		13.3 — 24.7 25.9 7ドル.6
    Qwen 3.5 9B 15.5
    		10.4 — 21.1 16.6 15ドル.6
    GPT OSS 20B 2.4
    		1.1 — 3.8 3.8 0ドル.2
    Latest results as of June 9th.
    View archived leaderboards and check back periodically for updates.

  • May 18th 2026 Android LLM Benchmark

    Model Score (%) info Average percentage of 100 test cases successfully resolved across 5 runs for each model
    arrow_range Cl range (%) info Expected performance range, reflecting the results' statistical reliability (p-value < 0.05)
    Avg latency (h) info Average time taken to solve 100 tasks across 5 runs
    Avg cost ($) info Average cost per full benchmark run
    GPT 5.5 74.0
    		66.8 — 80.5 15.5 133ドル.9
    GPT 5.4 72.4
    		65.4 — 79.3 21.2 91ドル.7
    Gemini 3.1 Pro Preview 72.4
    		65.1 — 78.8 11.5 49ドル.0
    Claude Opus 4 7 68.7
    		60.5 — 75.9 11.6 124ドル.3
    GPT 5.3 Codex 67.7
    		59.9 — 75.6 11.2 42ドル.6
    Claude Opus 4 6 66.6
    		59.1 — 74.1 9.9 84ドル.4
    GPT 5.2 Codex 62.5
    		54.4 — 70.0 24.3 121ドル.9
    Claude Opus 4.5 61.9
    		53.9 — 70.2 12.5 102ドル.5
    Gemini 3 Pro Preview 60.4
    		52.3 — 67.7 9.8 63ドル.7
    GLM 5.1 59.7
    		52.4 — 67.4 33.4 46ドル.7
    Claude Sonnet 4.6 58.4
    		50.3 — 66.4 8.2 40ドル.4
    Kimi K2.6 58.6
    		51.3 — 66.5 29.9 42ドル.5
    DeepSeek V4 Pro 55.4
    		47.5 — 63.6 35.8 13ドル.7
    Claude Sonnet 4.5 54.2
    		45.9 — 62.2 13.1 60ドル.3
    DeepSeek V4 Flash 52.7
    		45.3 — 60.7 28.1 8ドル.4
    MiMo 2.5 Pro 52.0
    		43.8 — 60.0 33.1 74ドル.5
    Qwen 3.6 Max Preview 51.4
    		43.5 — 59.3 20.5 222ドル.4
    Gemini 3 Flash Preview 42.0
    		36.6 — 47.3 16.5 34ドル.2
    MiniMax M2.7 37.2
    		30.3 — 44.9 20.3 10ドル.1
    Qwen 3.6 27B 37.4
    		30.5 — 44.5 20.7 64ドル.6
    Gemma 4 31B IT 33.2
    		26.2 — 40.8 14.2 2ドル.5
    Qwen 3.6 35B A3B 31.7
    		24.4 — 39.0 12.5 10ドル.7
    Gemini 2.5 Pro 29.1
    		22.3 — 36.1 8.4 35ドル.8
    Gemma 4 26B A4B IT 25.1
    		18.8 — 31.8 21.4 3ドル.3
    GPT OSS 120B 18.9
    		13.1 — 25.1 25.9 7ドル.6
    Gemini 2.5 Flash 15.9
    		10.7 — 21.1 4.9 11ドル.2
    Qwen 3.5 9B 15.5
    		10.1 — 20.9 16.6 15ドル.6
    GPT OSS 20B 2.4
    		1.2 — 3.9 3.8 0ドル.2
    Latest results as of May 18th.
    View archived leaderboards and check back periodically for updates.

  • May 5th 2026 Android LLM Benchmark

    Model Score (%) info Average percentage of 100 test cases successfully resolved across 5 runs for each model
    arrow_range Cl range (%) info Expected performance range, reflecting the results' statistical reliability (p-value < 0.05)
    GPT 5.5 74.0
    		66.8 — 80.5
    GPT 5.4 72.4
    		65.4 — 79.3
    Gemini 3.1 Pro Preview 72.4
    		65.1 — 78.8
    Claude Opus 4.7 68.7
    		61.2 — 76.0
    GPT 5.3 Codex 67.7
    		59.9 — 75.1
    Claude Opus 4.6 66.6
    		59.5 — 73.9
    GPT 5.2 Codex 62.5
    		54.6 — 70.1
    Claude Opus 4.5 61.9
    		53.0 — 70.1
    Gemini 3 Pro Preview 60.4
    		52.3 — 68.2
    Claude Sonnet 4.6 58.4
    		50.4 — 66.5
    Claude Sonnet 4.5 53.8
    		45.5 — 62.2
    Gemini 3 Flash Preview 42.0
    		36.5 — 47.6
    Gemini 2.5 Flash 16.7
    		11.5 — 22.1
    Latest results as of May 5th.
    View archived leaderboards and check back periodically for updates.

  • April 7th 2026 Android LLM Benchmark

    Model Score (%) info Average percentage of 100 test cases successfully resolved across 5 runs for each model
    arrow_range Cl range (%) info Expected performance range, reflecting the results' statistical reliability (p-value < 0.05)
    GPT-5.4 72.4
    		65.1 — 79.3
    Gemini 3.1 Pro Preview 72.4
    		64.8 — 79.3
    GPT-5.3-Codex 67.7
    		60.1 — 74.8
    Claude Opus 4.6 66.6
    		58.5 — 74.0
    GPT-5.2-Codex 62.5
    		54.8 — 69.8
    Claude Opus 4.5 61.9
    		53.8 — 70.3
    Gemini 3 Pro Preview 60.4
    		52.4 — 68.1
    Claude Sonnet 4.6 58.4
    		50.9 — 66.5
    Claude Sonnet 4.5 54.2
    		46.0 — 62.1
    Gemini 3 Flash Preview 42.0
    		36.4 — 47.7
    Gemini 2.5 Flash 16.1
    		11.2 — 21.2
    Latest results as of April 7th.
    View archived leaderboards and check back periodically for updates.

  • March 5th 2026 Android LLM Benchmark

    Model Score (%) info Average percentage of 100 test cases successfully resolved across 5 runs for each model
    arrow_range Cl range (%) info Expected performance range, reflecting the results' statistical reliability (p-value < 0.05)
    Gemini 3.1 Pro Preview 72.4
    		65.3 — 79.8
    Claude Opus 4.6 66.6
    		58.9 — 73.9
    GPT-5.2-Codex 62.5
    		54.7 — 70.3
    Claude Opus 4.5 61.9
    		53.9 — 69.6
    Gemini 3 Pro Preview 60.4
    		52.6 — 67.8
    Claude Sonnet 4.6 58.4
    		51.1 — 66.6
    Claude Sonnet 4.5 54.2
    		45.5 — 62.4
    Gemini 3 Flash Preview 42.0
    		36.3 — 47.9
    Gemini 2.5 Flash 16.1
    		10.9 — 21.9
    Latest results as of March 5th.
    View archived leaderboards and check back periodically for updates.