Tasks Traces V1 Blog Changelog Contribute GitHub

Traces

Every run: the agent's full trajectory, its submissions over the 20-hour budget, and the final score.

Astronomy Toolkit

task page →110 runs · indexed 2026-10-08

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Opus 5.5	0.9999	$74.48	265.3M	811k	19h 2m	653	24
2	Gemini 4 Argon	0.9999	$11.62	30.2M	300k	2h 33m	131	3
3	Claude Opus 5.5	0.9931	$69.52	233.4M	859k	17h 56m	587	25
4	Claude Opus 5.5	0.9907	$77.01	274.8M	827k	18h 8m	669	37
5	Claude Fable 5	0.9249	$134.09	102.7M	413k	9h 12m	289	20
6	Claude Opus 5	0.9214	$95.87	150.8M	548k	9h 35m	337	5
7	Claude Opus 5	0.9179	$89.22	139.0M	522k	13h 10m	320	6
8	Claude Sonnet 5.5	0.9039	$64.46	256.3M	1.0M	13h 39m	599	11
9	Claude Sonnet 5.5	0.9012	$70.07	279.4M	1.1M	11h 52m	618	11
10	Gemini 4 Argon	0.8999	$8.32	27.3M	281k	1h 26m	117	2
11	Claude Fable 5	0.8978	$116.44	85.6M	417k	5h 53m	271	12
12	Claude Fable 5.1	0.8953	$64.06	109.1M	391k	4h 49m	305	17
13	Claude Fable 5.1	0.8935	$57.22	97.5M	350k	6h 25m	298	24
14	Claude Opus 5	0.8866	$79.79	122.4M	498k	14h 37m	303	4
15	Claude Fable 5	0.8818	$104.89	77.6M	364k	13h 47m	261	9
16	Claude Opus 5.5	0.8205	$49.70	176.5M	532k	9h 15m	362	10
17	GPT-6 Astra	0.8045	$135.65	67.7M	278k	2h 40m	277	5
18	Claude Fable 5.1	0.8000	$36.76	59.7M	285k	3h 53m	220	11
19	Claude Fable 5.1	0.7999	$32.01	51.0M	267k	3h 57m	202	12
20	Claude Fable 5	0.7998	$113.89	82.7M	426k	7h 1m	260	9
21	GPT-6 Astra	0.7969	$2365.47	1121.6M	2.3M	19h 43m	2548	49
22	Claude Fable 5.1	0.7955	$42.10	68.8M	317k	4h 11m	232	12
23	Claude Opus 5	0.7648	$96.45	153.7M	524k	15h 7m	358	5
24	Claude Fable 5	0.7616	$120.57	89.4M	424k	12h 14m	280	10
25	Gemini 3.7 Flash	0.7294	$8.39	49.1M	408k	2h 12m	218	1
26	GPT-6 Astra	0.7058	$233.15	111.6M	353k	2h 41m	356	10
27	Gemini 3.8 Flash	0.7044	$5.05	41.3M	227k	1h 51m	218	2
28	Claude Opus 5.5	0.7024	$55.16	202.1M	545k	11h 37m	413	16
29	GPT-6 Astra	0.7013	$86.82	45.8M	224k	2h 2m	214	5
30	GLM-5.3	0.6985	$72.52	246.9M	707k	19h 16m	599	6
31	GPT-6 Astra	0.6917	$365.75	173.3M	402k	3h 17m	457	7
32	Claude Sonnet 5.5	0.6877	$35.84	138.7M	621k	5h 52m	269	5
33	Claude Opus 5	0.6837	$80.75	120.3M	550k	6h 30m	272	7
34	Kimi K3	0.6250	$60.98	152.4M	478k	16h 31m	414	12
35	Claude Sonnet 5.5	0.5306	$67.58	269.9M	1.1M	17h 27m	630	9
36	Grok 4.6	0.5299	$15.57	15.6M	323k	1h 52m	106	4
37	Gemini 4 Argon	0.5135	$11.83	39.0M	318k	1h 48m	154	2
38	GPT-5.6 Sol	0.4849	$43.38	52.6M	186k	3h 16m	263	5
39	Claude Haiku 5.5	0.4030	$8.08	127.7M	586k	6h 4m	275	2
40	Claude Sonnet 5.5	0.4029	$56.04	216.5M	989k	10h 27m	546	8
41	GLM-5.3	0.4028	$57.26	195.8M	552k	12h 7m	426	8
42	Qwen3.8-Max-0902	0.4022	$62.34	193.6M	1.6M	19h 35m	663	3
43	GLM-5.3	0.4017	$72.89	226.5M	514k	17h 29m	476	6
44	Gemini 3.7 Flash	0.4014	$3.09	23.0M	225k	38m	153	3
45	Grok 4.6	0.3993	$48.11	46.2M	395k	3h 31m	208	6
46	Kimi K3	0.3983	$90.41	238.0M	505k	20h 1m	587	20
47	Kimi K3	0.3959	$20.08	46.3M	279k	6h 2m	205	5
48	GLM-5.3	0.3930	$77.97	220.3M	559k	15h 22m	481	2
49	Kimi K3	0.3804	$70.49	167.8M	452k	19h 35m	468	8
50	Claude Haiku 5.5	0.3093	$8.98	142.5M	643k	5h 2m	290	4
51	Grok 4.7	0.3076	$28.45	28.6M	218k	1h 15m	142	5
52	Claude Haiku 5.5	0.3019	$6.55	100.4M	539k	4h 3m	248	1
53	Gemini 3.8 Flash	0.2999	$4.78	40.8M	198k	1h 11m	225	4
54	Grok 4.7	0.2999	$23.31	23.1M	224k	1h 29m	118	3
55	Kimi K3	0.2995	$53.30	137.6M	418k	11h 54m	430	7
56	GLM-5.3 Flash	0.2975	$5.82	179.7M	476k	11h 27m	453	4
57	Qwen3.8-Max-0902	0.2897	$62.30	208.2M	1.4M	20h 2m	506	1
58	GLM-5.3	0.2893	$64.65	196.5M	549k	19h 1m	419	8
59	Muse Spark 1.3	0.2056	$84.96	112.8M	364k	5h 13m	428	19
60	Grok 4.6	0.2000	$34.68	34.5M	354k	2h 42m	175	8
61	Grok 4.6	0.1971	$39.66	35.3M	400k	3h 43m	165	11
62	GPT-5.6 Sol	0.1823	$39.28	48.9M	195k	2h 7m	255	4
63	GLM-5.3 Flash	0.1158	$3.08	93.0M	371k	3h 57m	320	3
64	Claude Haiku 5.5	0.1077	$9.75	147.2M	915k	6h 7m	380	7
65	Gemini 3.8 Flash	0.1037	$5.88	54.2M	233k	1h 9m	255	3
66	Muse Spark 1.3	0.1015	$57.05	70.7M	278k	5h 16m	361	11
67	Muse Spark 1.2	0.1013	$11.07	59.7M	393k	2h 7m	259	1
68	Muse Spark 1.2	0.1000	$12.16	66.0M	402k	2h 34m	231	4
69	Muse Spark 1.3	0.0976	$81.13	81.9M	258k	12h 38m	397	2
70	Muse Spark 1.2	0.0563	$26.33	90.8M	427k	5h 8m	297	2
71	Claude Haiku 5.5	0.0349	$11.89	182.6M	1.1M	7h 13m	458	3
72	Muse Spark 1.3	0.0279	$94.74	98.7M	274k	19h 49m	426	38
73	Grok 4.7	0.0108	$18.18	18.2M	206k	1h 1m	105	3
74	Grok 4.7	0.0075	$224.04	218.5M	550k	3h 42m	699	15
75	Qwen3.8-Max-0902	0.0063	$53.54	172.9M	1.4M	19h 54m	408	1
76	Grok 4.7	0.0032	$13.33	14.3M	178k	52m	90	2
77	Muse Spark 1.3	0.0028	$18.89	27.3M	239k	2h 57m	175	4
78	Qwen3.8-Max-0902	0.0016	$62.25	179.2M	2.0M	19h 45m	513	2
79	GLM-5.3 Flash	0.0006	$4.33	118.3M	395k	10h 5m	388	3
80	Qwen3.8-Max-0902	0.0003	$33.82	90.4M	1.2M	20h 0m	309	1
81	Qwen3.8-Max	0.0000	$42.82	145.6M	925k	17h 28m	524	2
82	GLM-5.3 Flash	0.0000	$10.24	291.8M	558k	13h 32m	631	2
83	Inkling	0.0000	$4.59	4.7M	24k	22m	105	2
84	DeepSeek V4 Flash Vision Exp	0.0000	$4.02	226.3M	587k	18h 16m	503	5
85	Inkling	0.0000	$5.98	6.1M	33k	29m	144	1
86	DeepSeek V4 Flash Vision Exp	0.0000	$4.14	229.6M	637k	20h 0m	483	1
87	GPT-5.6 Sol	0.0000	$29.74	37.3M	170k	2h 5m	211	8
88	Gemini 3.8 Flash	0.0000	$5.19	43.5M	254k	50m	231	3
89	Qwen3.8-Max	0.0000	$48.24	157.1M	1.3M	19h 42m	518	2
90	Gemini 4 Argon	0.0000	$8.85	35.0M	322k	1h 26m	137	2
91	Gemini 3.7 Flash	0.0000	$5.44	38.0M	299k	1h 47m	197	3
92	Inkling	0.0000	$2.97	2.9M	45k	43m	109	2
93	Muse Spark 1.2	0.0000	$53.10	223.4M	907k	9h 16m	535	3
94	DeepSeek V4 Flash Vision Exp	0.0000	$3.11	165.3M	552k	9h 15m	409	6
95	GLM-5.3 Flash	0.0000	$6.17	161.9M	462k	9h 59m	452	3
96	Muse Spark 1.2	0.0000	$11.05	58.0M	328k	3h 11m	241	2
97	Gemini 4 Argon	0.0000	$27.00	17.6M	236k	2h 6m	93	2
98	Qwen3.8-Max	0.0000	$42.25	133.5M	1.3M	19h 46m	412	2
99	GPT-5.6 Sol	0.0000	$23.28	33.7M	166k	2h 33m	207	5
100	DeepSeek V4 Flash Vision Exp	0.0000	$3.44	189.2M	548k	10h 29m	453	4
101	GPT-5.6 Sol	0.0000	$47.11	54.0M	192k	2h 34m	262	4
102	Gemini 3.8 Flash	0.0000	$13.11	116.9M	410k	2h 14m	355	6
103	Inkling	0.0000	$6.49	6.6M	47k	1h 53m	142	1
104	Qwen3.8-Max	0.0000	$56.35	191.3M	1.2M	19h 55m	589	1
105	Grok 4.6	0.0000	$23.09	23.9M	317k	4h 4m	152	3
106	Gemini 3.7 Flash	0.0000	$3.91	24.5M	212k	1h 28m	155	2
107	Gemini 3.7 Flash	0.0000	$3.82	23.6M	220k	46m	162	2
108	Inkling	0.0000	$6.06	6.2M	36k	1h 0m	118	2
109	Qwen3.8-Max	0.0000	$28.32	86.1M	1.0M	14h 43m	386	3
110	DeepSeek V4 Flash Vision Exp	0.0000	$3.67	205.7M	547k	18h 16m	510	3

Cranelift Codegen Optimization

task page →109 runs · indexed 2026-10-08

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Sonnet 5.5	0.5217	$188.61	792.1M	2.3M	19h 20m	1836	19
2	Claude Sonnet 5.5	0.4700	$138.58	570.3M	1.8M	19h 30m	1309	10
3	Gemini 4 Argon	0.4043	$256.70	491.0M	1.5M	20h 49m	1116	24
4	Claude Opus 5.5	0.3651	$135.88	506.9M	1.2M	19h 1m	1217	34
5	Claude Sonnet 5.5	0.3536	$165.14	693.2M	2.0M	18h 53m	1535	3
6	Claude Fable 5.1	0.3484	$166.03	303.2M	604k	16h 29m	825	13
7	Claude Fable 5.1	0.3356	$233.87	425.4M	876k	16h 37m	1021	18
8	GPT-6 Astra	0.3006	$2589.93	1231.7M	1.9M	19h 38m	2587	122
9	Claude Opus 5.5	0.2984	$145.49	543.9M	1.3M	18h 43m	1227	39
10	Claude Opus 5	0.2915	$236.87	396.3M	962k	16h 44m	955	7
11	Claude Fable 5.1	0.2852	$199.91	365.0M	723k	15h 17m	974	15
12	Claude Opus 5.5	0.2828	$138.56	513.8M	1.3M	19h 19m	1211	30
13	Claude Sonnet 5.5	0.2621	$131.90	545.0M	1.7M	19h 41m	1250	18
14	Claude Fable 5	0.2501	$498.77	425.9M	919k	16h 22m	1061	44
15	Claude Opus 5.5	0.2500	$129.78	475.1M	1.3M	18h 48m	1162	27
16	GPT-6 Astra	0.2357	$2732.07	1287.7M	2.4M	19h 34m	2829	97
17	Gemini 4 Argon	0.2201	$298.10	572.4M	2.1M	20h 43m	1256	24
18	GPT-6 Astra	0.2194	$2585.69	1232.6M	2.0M	18h 22m	2643	79
19	Claude Opus 5.5	0.2063	$105.12	402.9M	850k	17h 7m	905	27
20	Claude Fable 5.1	0.2009	$170.71	311.4M	625k	17h 11m	813	17
21	Gemini 4 Argon	0.1959	$361.37	454.4M	1.3M	17h 22m	1002	27
22	Claude Sonnet 5.5	0.1812	$160.59	674.3M	1.9M	17h 6m	1513	4
23	Claude Fable 5	0.1789	$444.68	370.1M	951k	17h 14m	918	22
24	GPT-6 Astra	0.1650	$2812.60	1334.7M	2.3M	19h 56m	2934	92
25	Claude Opus 5	0.1550	$162.82	258.1M	858k	16h 52m	679	10
26	Claude Haiku 5.5	0.1347	$36.63	614.9M	2.1M	17h 29m	1443	8
27	Gemini 4 Argon	0.1238	$407.04	524.2M	1.6M	20h 47m	1255	28
28	Claude Opus 5	0.1213	$221.80	357.8M	1.1M	16h 0m	789	12
29	Claude Opus 5	0.0902	$217.40	355.4M	1.0M	14h 55m	890	16
30	Claude Haiku 5.5	0.0815	$13.27	217.3M	900k	5h 19m	586	1
31	Gemini 3.7 Flash	0.0802	$35.86	231.6M	242k	6h 15m	574	20
32	Kimi K3	0.0603	$148.59	441.7M	786k	16h 22m	1028	15
33	Claude Fable 5	0.0600	$397.22	333.0M	807k	17h 59m	852	28
34	Kimi K3	0.0478	$126.89	362.2M	801k	17h 53m	936	9
35	DeepSeek V4 Flash Vision Exp	0.0237	$8.27	504.6M	723k	9h 18m	1260	11
36	Muse Spark 1.3	0.0231	$411.74	433.6M	529k	18h 32m	875	4
37	Gemini 3.8 Flash	0.0215	$33.75	333.3M	357k	7h 19m	852	10
38	Muse Spark 1.3	0.0192	$411.77	452.6M	517k	18h 56m	944	8
39	Gemini 3.7 Flash	0.0189	$20.57	157.5M	204k	2h 49m	432	17
40	GPT-5.6 Sol	0.0176	$225.59	218.6M	216k	6h 56m	506	6
41	Muse Spark 1.3	0.0168	$495.72	581.2M	737k	19h 59m	1530	3
42	GLM-5.3	0.0162	$93.99	346.3M	474k	11h 28m	766	2
43	Claude Haiku 5.5	0.0152	$18.59	304.3M	1.2M	8h 22m	679	2
44	Kimi K3	0.0132	$145.36	415.9M	1.0M	20h 3m	1079	7
45	GLM-5.3	0.0129	$109.37	311.0M	484k	17h 22m	770	9
46	Qwen3.8-Max	0.0117	$41.08	108.0M	2.2M	18h 38m	484	6
47	GLM-5.3 Flash	0.0115	$9.01	238.7M	447k	10h 39m	679	4
48	Gemini 3.7 Flash	0.0110	$23.00	164.8M	188k	4h 11m	495	19
49	GLM-5.3 Flash	0.0109	$12.92	356.8M	427k	14h 26m	958	4
50	DeepSeek V4 Flash Vision Exp	0.0107	$5.01	303.8M	439k	7h 50m	723	8
51	GLM-5.3	0.0102	$60.68	223.7M	392k	7h 14m	615	4
52	Qwen3.8-Max-0902	0.0080	$64.48	159.6M	3.9M	19h 22m	453	1
53	Gemini 3.8 Flash	0.0078	$39.55	408.0M	435k	8h 49m	1021	22
54	DeepSeek V4 Flash Vision Exp	0.0030	$5.08	311.7M	419k	8h 3m	781	6
55	Qwen3.8-Max-0902	0.0007	$62.21	152.4M	3.8M	19h 36m	511	2
56	Grok 4.6	0.0000	$113.60	103.4M	918k	9h 35m	426	6
57	Inkling	0.0000	$8.38	8.8M	12k	55m	94	1
58	Gemini 3.8 Flash	0.0000	$49.95	454.4M	486k	9h 11m	1075	17
59	Muse Spark 1.3	0.0000	$472.59	577.9M	275k	18h 58m	1276	12
60	Grok 4.7	0.0000	$645.99	652.2M	2.1M	20h 0m	2589	15
61	GPT-5.6 Sol	0.0000	$297.21	276.6M	238k	7h 17m	549	5
62	Muse Spark 1.2	0.0000	$33.79	127.0M	205k	10h 23m	356	5
63	Gemini 3.8 Flash	0.0000	$54.64	599.7M	676k	11h 22m	1488	23
64	Claude Haiku 5.5	0.0000	$17.35	279.6M	1.2M	8h 20m	635	4
65	GLM-5.3 Flash	0.0000	$9.07	226.2M	412k	9h 11m	645	4
66	Qwen3.8-Max-0902	0.0000	$41.40	90.2M	3.0M	20h 1m	362	2
67	Muse Spark 1.2	0.0000	$18.72	115.3M	194k	3h 23m	350	4
68	Qwen3.8-Max	0.0000	$34.88	81.5M	2.3M	20h 0m	413	1
69	Kimi K3	0.0000	$136.80	389.4M	1.1M	20h 0m	1006	19
70	GLM-5.3 Flash	0.0000	$9.52	253.8M	421k	12h 41m	720	2
71	Claude Opus 5	0.0000	$223.36	369.6M	980k	18h 26m	878	18
72	Qwen3.8-Max-0902	0.0000	$33.03	53.9M	3.1M	20h 1m	263	1
73	Gemini 4 Argon	0.0000	$212.69	506.0M	1.7M	20h 46m	1184	27
74	Gemini 3.7 Flash	0.0000	$30.61	210.1M	211k	5h 15m	593	21
75	GPT-5.6 Sol	0.0000	$234.61	222.5M	240k	7h 38m	461	5
76	Qwen3.8-Max-0902	0.0000	$33.18	49.8M	3.3M	20h 0m	260	1
77	Grok 4.6	0.0000	$240.25	234.7M	1.3M	16h 47m	895	24
78	Inkling	0.0000	$13.75	14.4M	22k	1h 40m	135	2
79	Muse Spark 1.3	0.0000	$394.96	454.8M	322k	15h 45m	1021	12
80	Claude Fable 5	0.0000	$391.33	323.6M	852k	17h 7m	849	24
81	Kimi K3	0.0000	$136.72	403.0M	752k	19h 8m	1147	11
82	Grok 4.7	0.0000	$228.14	230.9M	694k	6h 44m	948	7
83	Inkling	0.0000	$11.79	12.4M	15k	1h 2m	121	1
84	Muse Spark 1.2	0.0000	$87.58	303.3M	711k	10h 13m	562	5
85	GPT-5.6 Sol	0.0000	$212.19	205.4M	208k	5h 52m	461	5
86	Inkling	0.0000	$8.24	8.6M	21k	30m	104	2
87	DeepSeek V4 Flash Vision Exp	0.0000	$3.68	220.8M	344k	8h 44m	670	8
88	Inkling	0.0000	$13.82	14.5M	21k	59m	137	2
89	Grok 4.6	0.0000	$138.74	131.9M	1.2M	8h 47m	587	6
90	Gemini 3.8 Flash	0.0000	$43.81	479.7M	493k	7h 38m	1152	19
91	Claude Fable 5.1	0.0000	$232.11	427.5M	765k	16h 0m	1023	35
92	Grok 4.7	0.0000	$477.48	483.7M	1.5M	16h 55m	1975	20
93	Grok 4.6	0.0000	$138.58	127.2M	927k	7h 29m	488	9
94	Muse Spark 1.2	0.0000	$76.77	223.4M	655k	10h 10m	450	7
95	Muse Spark 1.2	0.0000	$49.18	242.6M	532k	8h 28m	495	9
96	Claude Haiku 5.5	0.0000	$32.91	562.1M	1.7M	19h 9m	1233	14
97	Qwen3.8-Max	0.0000	$43.78	119.5M	2.2M	18h 46m	496	2
98	Qwen3.8-Max	0.0000	$50.74	159.1M	1.7M	16h 16m	624	4
99	GLM-5.3	0.0000	$110.08	353.7M	515k	18h 43m	761	18
100	Grok 4.7	0.0000	$635.61	634.0M	1.8M	19h 42m	2545	20
101	Gemini 3.7 Flash	0.0000	$47.46	341.2M	378k	8h 44m	828	22
102	GLM-5.3 Flash	0.0000	$13.40	389.9M	465k	13h 31m	921	2
103	Grok 4.6	0.0000	$108.30	103.9M	740k	6h 37m	447	6
104	DeepSeek V4 Flash Vision Exp	0.0000	$4.87	296.6M	422k	6h 43m	730	3
105	Claude Fable 5	0.0000	$346.79	290.6M	710k	18h 28m	737	21
106	GLM-5.3	0.0000	$69.79	258.7M	392k	7h 49m	703	7
107	Grok 4.7	0.0000	$640.87	669.7M	1.8M	19h 25m	2877	27
108	GPT-5.6 Sol	0.0000	$355.12	344.3M	254k	10h 32m	627	3
109	Qwen3.8-Max	0.0000	$42.54	115.4M	2.2M	20h 0m	491	1

Crash-Proof Flash Filesystem

task page →110 runs · indexed 2026-10-08

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Sonnet 5.5	0.9994	$65.32	251.2M	1.2M	7h 54m	507	9
2	Claude Sonnet 5.5	0.9991	$90.59	355.9M	1.5M	17h 4m	824	6
3	Claude Sonnet 5.5	0.9988	$62.92	245.6M	1.1M	10h 2m	522	11
4	Claude Fable 5	0.9988	$187.27	145.7M	561k	6h 17m	294	8
5	Claude Sonnet 5.5	0.9981	$69.13	270.2M	1.2M	18h 11m	576	4
6	Claude Opus 5.5	0.9981	$27.48	78.1M	449k	2h 59m	213	5
7	Claude Opus 5.5	0.9981	$26.95	76.7M	440k	2h 11m	202	7
8	Claude Sonnet 5.5	0.9978	$42.06	150.5M	930k	4h 45m	371	2
9	Claude Opus 5.5	0.9978	$40.02	124.8M	561k	3h 28m	245	7
10	Claude Opus 5.5	0.9978	$56.18	167.0M	853k	5h 25m	375	8
11	Claude Opus 5.5	0.9978	$31.57	92.4M	498k	2h 55m	217	4
12	Claude Opus 5	0.9977	$235.26	378.0M	1.2M	15h 3m	814	17
13	Claude Fable 5	0.9975	$300.62	253.7M	627k	16h 21m	456	10
14	Claude Fable 5.1	0.9970	$49.34	77.1M	443k	2h 57m	208	6
15	Claude Fable 5.1	0.9927	$43.72	70.9M	340k	3h 17m	229	4
16	Claude Fable 5.1	0.9927	$61.85	100.4M	479k	3h 5m	243	10
17	Claude Fable 5	0.9894	$286.95	239.2M	640k	18h 38m	417	7
18	Claude Fable 5	0.9784	$330.54	261.4M	927k	18h 19m	649	14
19	Claude Fable 5	0.8904	$290.91	231.9M	787k	18h 31m	576	9
20	Claude Fable 5.1	0.8898	$81.53	137.5M	526k	5h 0m	326	10
21	Claude Opus 5	0.8852	$226.31	360.9M	1.2M	15h 56m	756	18
22	Claude Opus 5	0.8705	$196.81	310.4M	1.1M	15h 31m	680	10
23	Claude Opus 5	0.8628	$231.46	372.7M	1.2M	17h 30m	794	16
24	Gemini 4 Argon	0.7378	$152.44	168.7M	1.1M	7h 34m	429	3
25	Claude Opus 5	0.6631	$126.31	185.3M	890k	10h 20m	498	10
26	GPT-6 Astra	0.6090	$92.20	45.6M	190k	1h 44m	177	4
27	GPT-6 Astra	0.5968	$497.70	237.1M	397k	4h 16m	516	14
28	GPT-6 Astra	0.5902	$154.84	75.5M	253k	1h 42m	259	5
29	GPT-6 Astra	0.5835	$114.98	55.8M	198k	1h 55m	208	5
30	GPT-6 Astra	0.5519	$228.77	111.2M	286k	2h 11m	332	5
31	Gemini 4 Argon	0.5467	$54.25	127.4M	708k	4h 23m	300	2
32	Gemini 4 Argon	0.2570	$245.29	429.7M	1.7M	13h 47m	960	3
33	Claude Haiku 5.5	0.1981	$23.67	373.5M	1.8M	4h 55m	822	2
34	Gemini 4 Argon	0.1980	$156.36	254.9M	1.4M	8h 56m	624	2
35	Gemini 4 Argon	0.1446	$77.78	221.0M	1.0M	7h 15m	532	2
36	Grok 4.7	0.0806	$696.28	696.9M	2.0M	12h 34m	2584	17
37	Claude Haiku 5.5	0.0687	$47.91	775.4M	3.3M	12h 44m	1650	6
38	Grok 4.6	0.0610	$858.44	859.7M	2.4M	19h 46m	2990	501
39	Claude Haiku 5.5	0.0528	$36.17	586.2M	2.5M	9h 44m	1238	4
40	Kimi K3	0.0431	$197.20	550.4M	1.5M	20h 1m	1197	1
41	Gemini 3.8 Flash	0.0394	$80.97	751.6M	1.2M	15h 46m	1693	10
42	Claude Haiku 5.5	0.0255	$45.61	737.3M	3.2M	16h 38m	1572	8
43	Grok 4.6	0.0241	$445.27	424.9M	2.8M	19h 59m	1572	200
44	Gemini 3.8 Flash	0.0229	$99.02	916.4M	1.5M	20h 0m	2136	2
45	GPT-5.6 Sol	0.0225	$66.23	68.5M	180k	1h 47m	274	8
46	GPT-5.6 Sol	0.0225	$81.19	79.4M	199k	1h 55m	291	7
47	GPT-5.6 Sol	0.0224	$73.40	72.5M	171k	1h 41m	273	8
48	Gemini 3.7 Flash	0.0217	$93.80	813.5M	2.3M	19h 56m	2141	1
49	Gemini 3.8 Flash	0.0186	$116.31	1099.4M	1.7M	19h 58m	2658	1
50	Gemini 3.8 Flash	0.0186	$181.15	1820.6M	3.3M	20h 0m	4383	2
51	Grok 4.7	0.0184	$312.58	313.6M	1.2M	9h 0m	1280	10
52	Grok 4.7	0.0184	$793.49	801.3M	2.8M	20h 0m	3084	23
53	Muse Spark 1.3	0.0177	$523.70	619.4M	1.1M	20h 0m	1409	1
54	Claude Fable 5.1	0.0173	$56.44	90.9M	453k	4h 18m	228	8
55	Grok 4.7	0.0173	$406.03	410.3M	1.5M	9h 50m	1624	19
56	GPT-5.6 Sol	0.0173	$159.04	153.2M	259k	4h 31m	440	16
57	GPT-5.6 Sol	0.0173	$92.72	87.0M	212k	1h 59m	268	7
58	GLM-5.3	0.0173	$171.41	627.5M	1.2M	19h 32m	1385	14
59	Muse Spark 1.3	0.0173	$532.50	628.8M	1.0M	20h 0m	1552	1
60	GLM-5.3	0.0173	$236.98	874.4M	1.4M	19h 51m	1908	31
61	Claude Haiku 5.5	0.0173	$35.40	562.3M	2.6M	7h 29m	1185	5
62	Grok 4.7	0.0164	$349.50	343.9M	1.4M	11h 51m	1349	8
63	Grok 4.6	0.0117	$262.00	255.9M	1.7M	13h 11m	1039	20
64	Kimi K3	0.0112	$202.76	569.3M	1.5M	19h 50m	1237	2
65	Gemini 3.7 Flash	0.0112	$89.95	756.9M	1.7M	20h 0m	1907	1
66	GLM-5.3	0.0111	$148.21	526.6M	1.0M	18h 49m	1175	16
67	DeepSeek V4 Flash Vision Exp	0.0110	$29.12	1761.2M	2.9M	19h 19m	3911	24
68	Qwen3.8-Max-0902	0.0057	$36.03	58.6M	3.4M	19h 58m	271	1
69	DeepSeek V4 Flash Vision Exp	0.0053	$22.61	1343.5M	2.5M	19h 20m	2867	26
70	GLM-5.3	0.0049	$108.31	386.0M	1.3M	16h 54m	851	5
71	DeepSeek V4 Flash Vision Exp	0.0049	$20.65	1205.0M	2.5M	19h 56m	2498	19
72	GLM-5.3 Flash	0.0049	$18.82	518.9M	1.2M	16h 54m	1227	7
73	Muse Spark 1.2	0.0049	$36.55	207.9M	989k	3h 16m	456	11
74	Kimi K3	0.0049	$158.40	411.0M	1.7M	20h 0m	1010	1
75	GLM-5.3 Flash	0.0049	$17.24	495.9M	945k	18h 11m	1222	7
76	DeepSeek V4 Flash Vision Exp	0.0049	$17.94	1097.8M	1.6M	18h 58m	2348	14
77	Gemini 3.7 Flash	0.0049	$62.15	472.1M	1.5M	20h 0m	1139	1
78	DeepSeek V4 Flash Vision Exp	0.0049	$20.78	1251.0M	2.1M	19h 4m	2804	15
79	Kimi K3	0.0049	$190.49	508.0M	1.5M	19h 54m	1111	9
80	GLM-5.3 Flash	0.0049	$25.92	740.1M	1.3M	17h 17m	1469	14
81	GLM-5.3 Flash	0.0047	$19.42	522.2M	1.2M	17h 11m	1230	5
82	Muse Spark 1.2	0.0047	$53.94	336.5M	554k	4h 31m	660	7
83	Grok 4.6	0.0046	$267.41	252.9M	1.7M	10h 33m	927	132
84	GLM-5.3	0.0039	$154.89	566.6M	1.0M	19h 45m	1350	18
85	Muse Spark 1.2	0.0038	$13.60	58.8M	346k	1h 50m	189	3
86	Muse Spark 1.2	0.0033	$35.75	222.8M	379k	2h 39m	489	7
87	Grok 4.6	0.0031	$181.59	175.3M	1.4M	8h 20m	693	24
88	Gemini 3.7 Flash	0.0024	$124.70	1220.5M	2.9M	20h 0m	2795	1
89	Qwen3.8-Max	0.0013	$12.18	16.0M	1.3M	19h 33m	130	1
90	Gemini 3.8 Flash	0.0013	$151.24	1551.3M	2.4M	20h 0m	3613	1
91	Gemini 3.7 Flash	0.0010	$64.73	520.0M	1.4M	20h 0m	1356	1
92	Qwen3.8-Max-0902	0.0008	$39.79	67.5M	3.7M	19h 56m	275	6
93	Qwen3.8-Max-0902	0.0008	$36.61	60.3M	3.5M	20h 1m	294	2
94	Kimi K3	0.0008	$195.88	529.5M	1.7M	20h 0m	1266	2
95	GLM-5.3 Flash	0.0008	$18.99	548.2M	1.2M	18h 12m	1172	11
96	Qwen3.8-Max-0902	0.0008	$33.78	44.2M	3.7M	20h 1m	226	1
97	Qwen3.8-Max-0902	0.0006	$33.01	43.7M	3.6M	20h 1m	236	1
98	Muse Spark 1.3	0.0002	$572.11	638.9M	1.2M	19h 59m	1384	1
99	Qwen3.8-Max	0.0001	$13.81	19.3M	1.5M	19h 35m	164	3
100	Qwen3.8-Max	0.0000	$14.39	19.8M	1.5M	20h 1m	160	1
101	Inkling	0.0000	$14.16	14.7M	37k	1h 30m	184	2
102	Muse Spark 1.3	0.0000	$486.57	544.5M	845k	19h 59m	1421	1
103	Qwen3.8-Max	0.0000	$9.18	10.7M	1.0M	19h 50m	100	1
104	Inkling	0.0000	$9.30	9.7M	22k	11m	123	2
105	Muse Spark 1.3	0.0000	$536.44	601.3M	1.3M	19h 59m	1324	1
106	Inkling	0.0000	$12.08	12.6M	23k	14m	161	2
107	Qwen3.8-Max	0.0000	$13.72	18.7M	1.5M	18h 42m	146	3
108	Muse Spark 1.2	0.0000	$18.50	97.7M	207k	1h 10m	269	4
109	Inkling	0.0000	$16.45	17.2M	21k	16m	173	2
110	Inkling	0.0000	$13.17	13.7M	28k	1h 27m	186	2

Dart Style in Haskell

task page →110 runs · indexed 2026-10-08

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Opus 5.5	0.9894	$168.41	622.4M	1.6M	10h 59m	1389	39
2	Claude Opus 5.5	0.9885	$192.52	720.9M	1.8M	13h 2m	1670	53
3	Claude Opus 5.5	0.9853	$159.11	577.5M	1.6M	9h 35m	1338	18
4	Claude Opus 5.5	0.9829	$190.35	691.9M	2.0M	9h 34m	1602	24
5	Claude Opus 5.5	0.9733	$318.71	1272.8M	2.4M	16h 51m	2868	99
6	Claude Sonnet 5.5	0.9726	$294.32	1267.9M	3.1M	19h 20m	2767	15
7	Gemini 4 Argon	0.9654	$218.42	430.1M	2.0M	15h 3m	1010	21
8	Claude Sonnet 5.5	0.9628	$264.23	1140.1M	2.8M	19h 20m	2386	18
9	GPT-6 Astra	0.9601	$2439.09	1157.2M	2.2M	16h 56m	2671	30
10	Claude Sonnet 5.5	0.9599	$231.53	982.7M	2.7M	19h 6m	2099	8
11	Claude Sonnet 5.5	0.9594	$281.27	1200.2M	3.2M	19h 38m	2675	25
12	Gemini 4 Argon	0.9568	$288.60	454.6M	2.0M	16h 22m	1093	14
13	Gemini 4 Argon	0.9534	$274.81	581.3M	2.0M	17h 49m	1368	15
14	Gemini 4 Argon	0.9529	$147.73	628.9M	2.2M	16h 46m	1478	23
15	GPT-6 Astra	0.9508	$2034.34	972.8M	1.7M	15h 13m	2216	42
16	Claude Fable 5.1	0.9491	$569.86	1069.9M	1.4M	17h 10m	2285	28
17	Claude Sonnet 5.5	0.9484	$282.69	1214.1M	3.1M	19h 50m	2712	18
18	GPT-6 Astra	0.9472	$895.41	422.2M	827k	6h 22m	958	14
19	Gemini 4 Argon	0.9457	$337.69	476.2M	2.1M	17h 14m	1055	18
20	GPT-6 Astra	0.9426	$2766.39	1322.7M	2.4M	19h 53m	2984	74
21	Claude Fable 5.1	0.9284	$552.06	1031.1M	1.5M	18h 56m	2091	46
22	Claude Fable 5.1	0.9275	$456.14	848.4M	1.3M	18h 57m	2026	39
23	Claude Opus 5	0.8972	$636.78	1144.6M	1.6M	19h 47m	2383	4
24	GPT-6 Astra	0.8965	$171.46	81.1M	258k	2h 1m	277	7
25	Claude Fable 5.1	0.8919	$415.43	774.8M	1.2M	19h 19m	1796	47
26	Claude Fable 5	0.8720	$1092.68	955.8M	1.8M	19h 38m	2037	16
27	Claude Fable 5.1	0.8712	$313.43	577.0M	1.0M	18h 15m	1283	12
28	Claude Opus 5	0.8643	$420.46	752.1M	1.2M	19h 22m	1536	5
29	Claude Fable 5	0.8484	$882.98	764.1M	1.5M	19h 42m	1840	23
30	Claude Opus 5	0.8376	$279.11	485.1M	948k	19h 29m	1196	3
31	Claude Haiku 5.5	0.8374	$77.49	1375.8M	3.3M	18h 52m	2942	36
32	Claude Fable 5	0.8275	$664.26	571.5M	1.2M	19h 26m	1197	10
33	Claude Opus 5	0.8261	$296.13	517.9M	961k	18h 59m	1277	26
34	Claude Opus 5	0.8114	$315.87	552.3M	1.0M	19h 44m	1262	18
35	Claude Fable 5	0.8095	$644.11	555.7M	1.2M	19h 31m	1237	15
36	Claude Haiku 5.5	0.8018	$62.96	1119.5M	2.6M	19h 39m	2502	41
37	Claude Haiku 5.5	0.7992	$77.73	1399.9M	2.9M	18h 46m	2994	20
38	GPT-5.6 Sol	0.7949	$998.14	972.7M	983k	19h 18m	2146	16
39	Claude Fable 5	0.7696	$331.51	264.9M	908k	19h 45m	637	2
40	Grok 4.7	0.7672	$810.72	832.8M	2.4M	19h 51m	3315	144
41	Claude Haiku 5.5	0.7663	$58.71	1042.8M	2.4M	19h 8m	2192	17
42	Claude Haiku 5.5	0.7528	$39.53	699.1M	1.7M	19h 36m	1507	18
43	Grok 4.7	0.7101	$934.33	959.7M	2.7M	20h 0m	3786	76
44	Grok 4.7	0.6935	$819.81	827.3M	2.4M	19h 30m	3292	79
45	Grok 4.7	0.6906	$879.91	896.0M	2.8M	19h 59m	3561	82
46	GPT-5.6 Sol	0.6779	$433.00	423.6M	391k	6h 1m	844	4
47	Gemini 3.8 Flash	0.6582	$120.09	1254.6M	1.6M	19h 48m	2946	161
48	Gemini 3.7 Flash	0.6517	$199.28	2123.1M	2.6M	19h 55m	4917	62
49	GPT-5.6 Sol	0.6344	$89.80	91.5M	201k	1h 55m	362	5
50	Muse Spark 1.2	0.6241	$2.29	11.0M	77k	1h 6m	128	3
51	Gemini 3.8 Flash	0.6130	$135.28	1312.0M	1.7M	19h 37m	2953	110
52	GPT-5.6 Sol	0.6068	$48.47	52.9M	204k	1h 35m	274	3
53	Grok 4.6	0.5864	$513.86	512.7M	2.7M	20h 0m	2054	59
54	Kimi K3	0.5650	$209.81	607.6M	1.3M	20h 1m	1454	19
55	Muse Spark 1.3	0.5513	$525.73	756.2M	1.1M	19h 57m	1637	8
56	Kimi K3	0.5395	$190.44	541.2M	1.2M	19h 50m	1336	3
57	Gemini 3.7 Flash	0.5376	$121.24	1110.9M	1.5M	19h 27m	2524	37
58	GLM-5.3	0.5299	$165.74	599.8M	1.1M	19h 6m	1398	51
59	Muse Spark 1.3	0.5258	$522.21	631.2M	1.1M	19h 59m	1757	5
60	Muse Spark 1.3	0.5020	$463.61	501.9M	1.1M	19h 20m	1265	9
61	Gemini 3.8 Flash	0.4900	$51.76	544.5M	634k	8h 8m	1241	24
62	Kimi K3	0.4874	$196.59	547.1M	1.3M	19h 46m	1241	8
63	Qwen3.8-Max	0.4828	$83.97	303.2M	1.2M	16h 38m	893	2
64	GLM-5.3	0.4696	$158.77	595.9M	645k	19h 33m	1177	12
65	GLM-5.3	0.4653	$144.84	516.1M	611k	19h 39m	1066	22
66	GLM-5.3	0.4595	$150.71	560.0M	706k	19h 32m	1263	35
67	GPT-5.6 Sol	0.4547	$71.79	84.5M	237k	2h 25m	406	7
68	Gemini 3.7 Flash	0.4365	$75.56	629.3M	934k	19h 55m	1331	3
69	Grok 4.7	0.4324	$263.14	264.6M	900k	6h 3m	996	20
70	Kimi K3	0.4305	$123.27	338.1M	738k	19h 52m	670	1
71	Gemini 3.7 Flash	0.4242	$33.92	324.0M	891k	3h 25m	780	8
72	GLM-5.3 Flash	0.4031	$18.46	598.3M	640k	20h 1m	1200	1
73	DeepSeek V4 Flash Vision Exp	0.4014	$18.12	1122.6M	1.6M	19h 51m	2602	23
74	GLM-5.3 Flash	0.4009	$22.72	722.1M	1.2M	19h 26m	1667	27
75	GLM-5.3	0.3788	$98.82	359.7M	600k	20h 0m	738	1
76	Muse Spark 1.3	0.3680	$506.26	577.2M	925k	19h 53m	1567	1
77	GLM-5.3 Flash	0.3666	$15.53	455.8M	658k	20h 1m	864	1
78	Qwen3.8-Max	0.3212	$156.83	579.3M	1.7M	20h 1m	1506	1
79	Gemini 3.7 Flash	0.3137	$25.24	282.5M	578k	1h 28m	590	2
80	Grok 4.6	0.3015	$389.00	386.7M	2.2M	19h 53m	1645	42
81	Qwen3.8-Max-0902	0.2762	$68.64	209.5M	2.5M	19h 58m	650	1
82	Qwen3.8-Max	0.2726	$83.47	304.6M	1.0M	17h 28m	843	2
83	Qwen3.8-Max	0.2674	$50.86	177.6M	941k	18h 25m	662	12
84	DeepSeek V4 Flash Vision Exp	0.2342	$7.03	436.2M	602k	8h 13m	914	3
85	DeepSeek V4 Flash Vision Exp	0.2234	$21.23	1304.8M	2.0M	15h 38m	2836	7
86	GLM-5.3 Flash	0.2155	$15.10	395.1M	560k	20h 1m	851	1
87	Qwen3.8-Max-0902	0.1979	$53.96	142.4M	2.8M	19h 56m	444	1
88	Grok 4.6	0.1785	$547.31	552.2M	2.4M	19h 52m	2177	74
89	Gemini 3.8 Flash	0.1718	$3.02	22.3M	97k	43m	198	3
90	DeepSeek V4 Flash Vision Exp	0.1638	$15.31	956.5M	1.3M	19h 4m	2023	3
91	GLM-5.3 Flash	0.1396	$17.07	519.3M	626k	20h 0m	1015	1
92	Gemini 3.8 Flash	0.1167	$2.13	14.3M	77k	33m	176	2
93	Muse Spark 1.2	0.1043	$2.53	9.5M	123k	1h 21m	101	1
94	Muse Spark 1.2	0.1040	$2.18	10.0M	104k	1h 4m	113	2
95	Muse Spark 1.2	0.1040	$3.41	16.0M	109k	59m	110	2
96	Qwen3.8-Max	0.0853	$104.77	373.0M	1.7M	19h 28m	957	1
97	DeepSeek V4 Flash Vision Exp	0.0766	$15.69	953.5M	1.5M	16h 48m	2199	5
98	Qwen3.8-Max-0902	0.0514	$56.12	143.5M	3.2M	20h 2m	493	1
99	Muse Spark 1.2	0.0377	$5.55	30.5M	146k	1h 16m	174	2
100	Muse Spark 1.3	0.0010	$582.91	707.6M	1.1M	19h 56m	1740	10
101	Inkling	0.0000	$2.95	3.1M	10k	11m	87	2
102	Inkling	0.0000	$20.49	21.4M	45k	32m	241	1
103	Qwen3.8-Max-0902	0.0000	$93.37	320.2M	1.9M	20h 1m	755	2
104	Grok 4.6	0.0000	$485.05	493.3M	2.2M	19h 52m	1992	109
105	Qwen3.8-Max-0902	0.0000	$44.97	92.5M	2.9M	20h 1m	511	1
106	Grok 4.6	0.0000	$402.09	394.7M	2.1M	17h 23m	1524	78
107	Inkling	0.0000	$19.88	20.7M	49k	1h 2m	297	1
108	Inkling	0.0000	$7.80	8.1M	20k	1h 3m	140	1
109	Inkling	0.0000	$4.62	4.8M	11k	1h 10m	101	2
110	Kimi K3	0.0000	$86.57	238.4M	589k	20h 1m	593	1

FFmpeg libswscale Optimization

task page →110 runs · indexed 2026-10-08

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Opus 5.5	0.8444	$169.75	555.3M	2.2M	12h 14m	1230	42
2	Claude Fable 5.1	0.8080	$283.04	487.0M	1.6M	13h 9m	1070	46
3	Claude Sonnet 5.5	0.5593	$210.44	829.8M	3.5M	18h 18m	1662	32
4	Claude Fable 5.1	0.5509	$202.06	343.4M	1.3M	9h 13m	819	28
5	Claude Sonnet 5.5	0.5128	$191.15	731.8M	3.5M	16h 45m	1549	18
6	Claude Fable 5.1	0.5106	$148.93	254.0M	906k	6h 11m	641	18
7	Claude Fable 5	0.3296	$651.27	511.9M	1.9M	17h 29m	1099	44
8	GPT-6 Astra	0.2755	$169.85	82.4M	307k	2h 7m	290	7
9	Claude Opus 5	0.2730	$386.74	584.2M	2.5M	17h 26m	1236	48
10	Claude Opus 5	0.2685	$323.38	505.1M	1.9M	18h 50m	1055	36
11	Claude Haiku 5.5	0.2461	$32.33	512.1M	2.4M	8h 34m	1174	9
12	Claude Fable 5	0.2400	$694.93	559.7M	1.8M	18h 56m	1226	65
13	Claude Opus 5	0.2348	$359.20	577.0M	1.9M	15h 26m	1190	24
14	Claude Fable 5	0.2293	$457.04	364.2M	1.2M	18h 33m	758	37
15	Claude Haiku 5.5	0.2204	$29.73	450.8M	2.6M	7h 6m	996	2
16	Claude Opus 5	0.2127	$514.92	809.2M	2.9M	17h 33m	1760	45
17	GPT-6 Astra	0.2021	$313.12	153.5M	377k	2h 54m	458	9
18	GPT-6 Astra	0.1983	$306.45	149.1M	351k	2h 48m	444	8
19	Claude Opus 5	0.1899	$481.68	774.7M	2.5M	14h 53m	1592	54
20	GLM-5.3	0.1752	$226.40	818.4M	2.0M	19h 33m	1884	18
21	GPT-6 Astra	0.1741	$262.36	125.2M	349k	2h 37m	379	6
22	Claude Fable 5	0.1712	$501.72	411.4M	1.2M	18h 49m	877	42
23	Claude Haiku 5.5	0.1587	$30.27	466.6M	2.5M	11h 4m	1072	17
24	Gemini 4 Argon	0.1572	$160.32	346.6M	1.9M	11h 31m	765	13
25	Claude Fable 5	0.1436	$530.26	432.3M	1.3M	19h 40m	864	28
26	GLM-5.3	0.1410	$148.35	524.2M	1.7M	16h 26m	1303	8
27	GLM-5.3	0.1369	$139.76	505.8M	1.2M	11h 14m	1097	9
28	Claude Fable 5.1	0.1368	$219.11	385.2M	1.1M	10h 27m	835	22
29	Muse Spark 1.3	0.1277	$499.70	617.7M	951k	19h 17m	1401	24
30	GLM-5.3 Flash	0.1268	$16.00	489.1M	1.2M	13h 53m	1179	110
31	Kimi K3	0.1260	$127.79	342.6M	1.2M	13h 59m	827	13
32	Kimi K3	0.1252	$211.22	606.6M	1.4M	19h 43m	1382	3
33	GLM-5.3	0.1231	$175.55	619.5M	1.7M	16h 47m	1509	8
34	Claude Opus 5.5	0.1224	$83.47	264.7M	1.1M	5h 11m	549	14
35	Claude Opus 5.5	0.1185	$175.12	575.5M	2.3M	11h 51m	1203	46
36	GPT-6 Astra	0.1179	$275.12	132.2M	355k	2h 35m	396	6
37	Muse Spark 1.3	0.1028	$438.57	557.6M	1.0M	16h 30m	1340	23
38	Claude Sonnet 5.5	0.1012	$189.51	720.9M	3.5M	18h 17m	1538	35
39	Muse Spark 1.3	0.1003	$186.76	239.3M	502k	7h 59m	559	14
40	Muse Spark 1.3	0.0996	$157.06	200.2M	465k	6h 34m	504	12
41	GLM-5.3 Flash	0.0819	$11.44	312.2M	1.0M	11h 47m	786	2
42	Kimi K3	0.0799	$127.97	356.2M	1.1M	15h 4m	914	11
43	GLM-5.3	0.0783	$177.95	625.3M	1.7M	17h 1m	1420	8
44	Muse Spark 1.3	0.0779	$279.58	336.6M	595k	9h 26m	688	17
45	Claude Haiku 5.5	0.0742	$11.46	167.4M	1.1M	3h 2m	421	4
46	GLM-5.3 Flash	0.0721	$12.54	348.2M	1.1M	9h 45m	838	4
47	Grok 4.7	0.0717	$958.45	972.7M	3.5M	19h 0m	3810	51
48	GLM-5.3 Flash	0.0710	$14.74	418.0M	1.0M	13h 2m	963	6
49	Claude Sonnet 5.5	0.0703	$164.01	627.4M	3.0M	15h 19m	1268	23
50	Grok 4.7	0.0692	$75.94	76.6M	393k	2h 13m	349	5
51	DeepSeek V4 Flash Vision Exp	0.0668	$8.52	493.3M	1.0M	6h 44m	1095	8
52	Kimi K3	0.0639	$138.03	382.3M	1.1M	17h 32m	902	2
53	DeepSeek V4 Flash Vision Exp	0.0636	$9.11	528.7M	1.1M	6h 10m	1172	5
54	DeepSeek V4 Flash Vision Exp	0.0602	$13.77	800.3M	1.6M	9h 14m	1831	19
55	GLM-5.3 Flash	0.0591	$10.52	294.6M	1.1M	15h 39m	768	1
56	Grok 4.7	0.0562	$84.27	86.0M	423k	2h 12m	353	5
57	DeepSeek V4 Flash Vision Exp	0.0483	$9.39	554.1M	1.0M	5h 54m	1197	16
58	Grok 4.7	0.0482	$1089.41	1092.6M	3.0M	15h 42m	3929	49
59	Claude Sonnet 5.5	0.0471	$76.69	272.4M	1.7M	6h 51m	612	9
60	Claude Opus 5.5	0.0441	$106.98	317.9M	1.6M	6h 52m	693	18
61	Gemini 3.8 Flash	0.0421	$22.33	232.8M	629k	2h 36m	656	8
62	Qwen3.8-Max	0.0373	$86.44	286.2M	2.3M	17h 43m	706	6
63	Grok 4.6	0.0360	$419.25	404.2M	2.4M	19h 56m	1517	114
64	Qwen3.8-Max	0.0181	$64.05	193.9M	2.4M	18h 48m	555	1
65	Gemini 3.7 Flash	0.0125	$20.83	184.0M	617k	2h 35m	524	2
66	Qwen3.8-Max	0.0053	$46.03	124.6M	2.3M	19h 58m	424	1
67	Gemini 3.7 Flash	0.0032	$19.94	167.8M	795k	2h 4m	496	7
68	Grok 4.7	0.0022	$90.46	89.2M	451k	3h 23m	379	9
69	Grok 4.6	0.0000	$1121.42	1126.0M	2.4M	18h 11m	3693	434
70	Muse Spark 1.2	0.0000	$15.42	90.0M	327k	41m	286	2
71	Grok 4.6	0.0000	$68.40	59.6M	603k	3h 50m	229	4
72	Muse Spark 1.2	0.0000	$16.97	85.6M	384k	1h 38m	227	4
73	Gemini 3.8 Flash	0.0000	$23.73	247.4M	548k	2h 37m	674	3
74	Claude Fable 5.1	0.0000	$217.24	363.8M	1.5M	10h 22m	893	27
75	Gemini 4 Argon	0.0000	$156.84	296.0M	1.6M	9h 55m	668	11
76	Gemini 4 Argon	0.0000	$106.17	273.4M	1.3M	8h 28m	605	17
77	Qwen3.8-Max	0.0000	$54.69	158.2M	2.4M	18h 58m	475	1
78	Grok 4.6	0.0000	$132.56	129.2M	778k	5h 28m	562	27
79	Inkling	0.0000	$11.24	11.7M	27k	1h 20m	165	2
80	Inkling	0.0000	$15.45	16.1M	35k	22m	188	2
81	Gemini 3.8 Flash	0.0000	$19.00	192.6M	496k	2h 5m	536	4
82	Qwen3.8-Max-0902	0.0000	$39.61	68.0M	3.6M	20h 1m	275	1
83	Qwen3.8-Max	0.0000	$30.12	66.3M	2.1M	19h 55m	298	1
84	Inkling	0.0000	$39.66	41.4M	77k	41m	354	2
85	Gemini 3.7 Flash	0.0000	$25.27	228.2M	1.1M	2h 14m	655	5
86	GPT-5.6 Sol	0.0000	$66.82	75.1M	179k	1h 59m	318	8
87	Gemini 3.8 Flash	0.0000	$31.46	339.3M	663k	5h 43m	881	17
88	Qwen3.8-Max-0902	0.0000	$44.84	72.8M	3.4M	20h 2m	345	1
89	Inkling	0.0000	$15.51	16.2M	36k	32m	190	1
90	Muse Spark 1.2	0.0000	$11.90	64.7M	340k	1h 14m	223	2
91	GPT-5.6 Sol	0.0000	$78.43	84.8M	190k	2h 4m	348	5
92	Grok 4.6	0.0000	$380.75	368.6M	1.8M	11h 52m	1370	67
93	Muse Spark 1.2	0.0000	$35.70	203.8M	978k	3h 29m	468	8
94	Claude Haiku 5.5	0.0000	$13.45	205.4M	1.1M	2h 38m	488	3
95	GPT-5.6 Sol	0.0000	$123.31	124.9M	233k	2h 12m	422	3
96	Kimi K3	0.0000	$171.14	500.1M	1.1M	19h 41m	1175	47
97	Qwen3.8-Max-0902	0.0000	$51.17	115.3M	3.5M	20h 1m	358	1
98	Claude Opus 5.5	0.0000	$168.91	529.9M	2.4M	15h 41m	1150	37
99	GPT-5.6 Sol	0.0000	$76.67	86.4M	186k	3h 5m	369	10
100	Gemini 4 Argon	0.0000	$58.88	167.7M	1.1M	5h 59m	405	7
101	Qwen3.8-Max-0902	0.0000	$43.43	82.1M	3.6M	20h 1m	306	1
102	DeepSeek V4 Flash Vision Exp	0.0000	$12.34	718.0M	1.4M	8h 32m	1678	16
103	GPT-5.6 Sol	0.0000	$117.96	120.8M	233k	3h 42m	405	8
104	Inkling	0.0000	$11.17	11.6M	27k	16m	167	1
105	Qwen3.8-Max-0902	0.0000	$72.15	209.7M	3.0M	19h 38m	543	1
106	Gemini 4 Argon	0.0000	$190.93	224.2M	1.2M	7h 43m	531	8
107	Gemini 3.8 Flash	0.0000	$22.37	231.7M	581k	2h 9m	683	12
108	Gemini 3.7 Flash	0.0000	$16.99	131.7M	937k	2h 45m	448	6
109	Muse Spark 1.2	0.0000	$27.98	159.1M	568k	1h 40m	345	2
110	Gemini 3.7 Flash	0.0000	$44.74	251.9M	922k	8h 1m	758	6

Fitness-Recap Video in Remotion

task page →110 runs · indexed 2026-10-08

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	GPT-6 Astra	0.7901	$2673.96	1271.2M	2.5M	19h 58m	2991	31
2	GPT-6 Astra	0.7705	$2639.52	1259.7M	2.5M	19h 50m	2961	27
3	GPT-6 Astra	0.7148	$2733.41	1304.4M	2.5M	19h 53m	2977	29
4	GPT-6 Astra	0.6873	$2846.47	1353.6M	2.5M	19h 54m	3114	36
5	GPT-6 Astra	0.6437	$2584.51	1240.4M	2.4M	19h 47m	2920	23
6	Claude Sonnet 5.5	0.3659	$232.86	951.4M	2.7M	19h 46m	2188	3
7	Claude Opus 5.5	0.3605	$270.94	1051.2M	2.1M	19h 10m	2403	12
8	Claude Fable 5	0.3226	$587.70	496.9M	1.2M	18h 28m	1118	8
9	Claude Sonnet 5.5	0.3212	$193.06	791.5M	2.2M	19h 6m	1718	1
10	Claude Sonnet 5.5	0.3201	$185.52	762.3M	2.2M	19h 19m	1724	2
11	Claude Sonnet 5.5	0.3192	$216.22	909.3M	2.5M	19h 44m	2030	2
12	Claude Opus 5.5	0.2926	$307.93	1148.9M	1.9M	19h 20m	2553	28
13	Claude Opus 5.5	0.2905	$179.59	686.0M	1.3M	19h 22m	1645	21
14	Claude Sonnet 5.5	0.2754	$168.83	706.6M	2.1M	19h 28m	1679	3
15	Claude Opus 5.5	0.2637	$181.24	683.8M	1.4M	19h 15m	1667	19
16	Gemini 4 Argon	0.2451	$233.36	596.2M	1.7M	16h 49m	1260	12
17	Claude Opus 5.5	0.2443	$84.28	312.7M	521k	9h 32m	660	10
18	Gemini 4 Argon	0.2401	$112.07	365.5M	1.1M	10h 13m	860	5
19	Claude Haiku 5.5	0.2316	$28.48	497.9M	1.3M	13h 17m	1085	8
20	Claude Opus 5	0.2280	$444.54	757.9M	1.7M	18h 35m	1644	23
21	Claude Opus 5	0.2251	$524.60	906.1M	1.9M	19h 37m	1949	17
22	Gemini 4 Argon	0.2185	$165.03	335.8M	998k	11h 8m	805	2
23	Claude Haiku 5.5	0.2169	$33.38	569.6M	1.7M	9h 44m	1300	6
24	Claude Fable 5	0.2168	$885.34	754.4M	1.7M	18h 17m	1721	16
25	Claude Opus 5	0.2158	$509.67	881.7M	1.8M	18h 48m	1852	8
26	Claude Opus 5	0.2155	$438.52	750.5M	1.6M	18h 17m	1648	14
27	GPT-5.6 Sol	0.2152	$950.51	938.4M	1.0M	19h 3m	2343	45
28	Claude Opus 5	0.2129	$501.05	832.6M	1.6M	18h 19m	1912	14
29	Gemini 4 Argon	0.2055	$210.50	290.4M	915k	10h 7m	717	2
30	Claude Fable 5.1	0.2055	$157.69	293.5M	491k	6h 51m	665	8
31	Claude Fable 5	0.2027	$655.49	552.4M	1.4M	18h 20m	1203	14
32	Claude Fable 5.1	0.2018	$157.38	293.1M	476k	6h 7m	644	7
33	Gemini 4 Argon	0.1980	$159.82	263.8M	756k	8h 13m	636	3
34	Claude Haiku 5.5	0.1957	$25.71	439.2M	1.3M	6h 34m	947	5
35	GPT-5.6 Sol	0.1930	$656.93	636.3M	650k	19h 12m	1478	11
36	Claude Fable 5.1	0.1927	$120.12	223.7M	414k	5h 14m	587	7
37	GPT-5.6 Sol	0.1913	$298.28	288.7M	261k	7h 23m	651	12
38	GPT-5.6 Sol	0.1895	$283.99	275.5M	274k	5h 43m	687	6
39	GLM-5.3	0.1894	$78.50	212.7M	442k	18h 20m	530	6
40	Claude Haiku 5.5	0.1842	$16.30	276.2M	923k	15h 8m	666	11
41	GLM-5.3	0.1828	$113.14	407.8M	483k	19h 2m	804	9
42	Muse Spark 1.3	0.1827	$179.29	1045.0M	1.1M	19h 56m	2290	8
43	Kimi K3	0.1767	$191.75	571.0M	982k	19h 53m	1356	3
44	GLM-5.3 Flash	0.1750	$19.23	554.0M	928k	19h 35m	1394	1
45	Claude Fable 5.1	0.1730	$560.60	1048.2M	1.5M	18h 2m	2274	26
46	Kimi K3	0.1723	$171.83	504.1M	1.1M	19h 44m	1234	8
47	Kimi K3	0.1716	$266.42	802.2M	1.3M	20h 1m	1618	1
48	Claude Haiku 5.5	0.1704	$8.47	145.8M	443k	15h 41m	419	9
49	GLM-5.3 Flash	0.1702	$12.92	416.9M	481k	11h 49m	1002	5
50	DeepSeek V4 Flash Vision Exp	0.1621	$10.73	658.6M	994k	19h 13m	1651	18
51	Qwen3.8-Max-0902	0.1486	$33.48	89.8M	1.0M	20h 8m	410	2
52	Qwen3.8-Max	0.1474	$77.77	271.1M	994k	13h 7m	800	4
53	Claude Fable 5	0.1467	$662.31	571.3M	1.2M	19h 23m	1221	12
54	Qwen3.8-Max-0902	0.1462	$31.95	68.8M	1.2M	20h 2m	351	1
55	GLM-5.3	0.1426	$118.44	380.0M	784k	18h 55m	986	18
56	GPT-5.6 Sol	0.1374	$190.12	189.9M	230k	5h 21m	589	2
57	Muse Spark 1.3	0.1338	$375.85	806.9M	1.0M	20h 0m	1927	3
58	Claude Fable 5	0.1112	$587.97	497.7M	1.2M	19h 20m	1094	10
59	GLM-5.3	0.1053	$110.52	388.9M	908k	15h 36m	961	9
60	GLM-5.3 Flash	0.1000	$22.01	692.4M	913k	17h 25m	1635	6
61	Grok 4.7	0.0961	$439.29	445.6M	1.3M	9h 16m	1738	103
62	GLM-5.3 Flash	0.0914	$16.05	372.4M	537k	19h 9m	828	3
63	Grok 4.7	0.0840	$675.93	699.4M	1.7M	12h 52m	2790	89
64	GLM-5.3	0.0837	$90.93	307.2M	485k	16h 10m	636	8
65	DeepSeek V4 Flash Vision Exp	0.0738	$6.53	405.2M	565k	19h 26m	994	10
66	Qwen3.8-Max	0.0733	$163.48	458.4M	1.7M	19h 54m	1249	1
67	Claude Fable 5.1	0.0694	$335.29	627.1M	960k	15h 45m	1456	19
68	Kimi K3	0.0657	$243.01	727.7M	1.1M	20h 1m	1633	1
69	Kimi K3	0.0536	$202.63	587.9M	1.0M	19h 57m	1479	12
70	Grok 4.7	0.0491	$750.66	786.8M	2.0M	15h 22m	3238	130
71	DeepSeek V4 Flash Vision Exp	0.0483	$6.64	411.0M	525k	16h 0m	983	21
72	Grok 4.7	0.0481	$814.19	833.8M	1.7M	19h 58m	3225	109
73	Grok 4.7	0.0422	$931.62	946.8M	1.9M	20h 0m	3560	153
74	GLM-5.3 Flash	0.0395	$21.46	680.4M	950k	19h 31m	1613	12
75	Muse Spark 1.3	0.0362	$213.98	610.8M	874k	19h 50m	1636	3
76	Grok 4.6	0.0338	$1762.94	1204.0M	2.8M	20h 0m	4175	289
77	DeepSeek V4 Flash Vision Exp	0.0324	$12.46	767.8M	1.1M	18h 4m	1789	24
78	Qwen3.8-Max-0902	0.0314	$43.03	134.8M	748k	20h 9m	405	1
79	Grok 4.6	0.0279	$646.10	557.5M	1.5M	19h 48m	2043	149
80	Muse Spark 1.3	0.0269	$429.57	453.8M	790k	19h 52m	1245	2
81	Muse Spark 1.3	0.0263	$363.14	496.5M	863k	19h 59m	1337	1
82	Grok 4.6	0.0245	$555.19	490.3M	1.5M	19h 56m	2020	90
83	Grok 4.6	0.0242	$1149.83	792.5M	1.8M	19h 58m	2836	250
84	Gemini 3.8 Flash	0.0241	$38.64	291.2M	430k	9h 43m	866	3
85	Grok 4.6	0.0216	$746.94	531.5M	1.4M	19h 55m	1941	163
86	Gemini 3.7 Flash	0.0129	$13.97	114.7M	211k	4h 23m	382	2
87	Qwen3.8-Max	0.0127	$196.38	482.4M	1.8M	19h 51m	1048	1
88	DeepSeek V4 Flash Vision Exp	0.0121	$4.01	242.4M	395k	12h 19m	777	16
89	Gemini 3.7 Flash	0.0091	$2.94	21.2M	86k	1h 20m	169	2
90	Gemini 3.7 Flash	0.0089	$10.44	67.7M	164k	2h 48m	328	4
91	Gemini 3.7 Flash	0.0080	$7.68	61.7M	148k	2h 7m	300	1
92	Muse Spark 1.2	0.0079	$12.46	62.2M	180k	3h 28m	270	3
93	Qwen3.8-Max	0.0062	$52.83	157.6M	675k	9h 28m	578	4
94	Muse Spark 1.2	0.0060	$91.70	184.1M	1.3M	8h 42m	509	3
95	Gemini 3.7 Flash	0.0060	$4.67	30.1M	112k	2h 7m	222	1
96	Qwen3.8-Max-0902	0.0051	$35.22	96.8M	1.1M	20h 1m	342	1
97	Muse Spark 1.2	0.0032	$14.71	29.2M	224k	3h 30m	183	2
98	Muse Spark 1.2	0.0031	$37.85	75.3M	303k	4h 52m	285	4
99	Qwen3.8-Max-0902	0.0029	$28.31	69.9M	1.1M	20h 5m	355	1
100	Inkling	0.0006	$14.05	14.6M	38k	1h 0m	183	1
101	Inkling	0.0001	$7.96	8.3M	26k	31m	141	2
102	Gemini 3.8 Flash	0.0000†	$3.21	19.3M	43k	1h 20m	160	1
103	Gemini 3.8 Flash	0.0000†	$3.95	27.7M	51k	1h 46m	183	1
104	Inkling	0.0000	$13.72	14.2M	49k	59m	205	1
105	Inkling	0.0000	$10.65	11.1M	29k	1h 18m	172	2
106	Inkling	0.0000	$11.15	11.6M	44k	40m	138	1
107	Qwen3.8-Max	0.0000†	$11.91	31.2M	238k	4h 2m	292	2
108	Muse Spark 1.2	0.0000†	$12.66	38.1M	209k	4h 41m	190	1
109	Gemini 3.8 Flash	0.0000†	$1.73	10.5M	40k	42m	105	1
110	Gemini 3.8 Flash	0.0000†	$4.92	31.8M	62k	1h 28m	213	1

† score adjusted by manual review; the trace's Judge tab carries the measured score and the rationale.

Flight-Sim Renderer in OpenGL

task page →110 runs · indexed 2026-10-08

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	GPT-6 Astra	0.9965	$2541.33	1210.9M	2.5M	19h 50m	2830	27
2	GPT-6 Astra	0.9845	$2685.88	1278.3M	2.6M	20h 0m	2961	38
3	GPT-6 Astra	0.9844	$2735.71	1297.4M	2.6M	19h 42m	2884	29
4	GPT-6 Astra	0.9841	$2589.72	1228.2M	2.6M	19h 59m	2801	34
5	GPT-6 Astra	0.9841	$2728.19	1288.8M	2.7M	19h 57m	2941	47
6	Gemini 4 Argon	0.9334	$206.78	382.0M	1.3M	15h 47m	877	5
7	Claude Opus 5.5	0.9097	$156.04	545.9M	1.3M	19h 52m	1288	19
8	Claude Sonnet 5.5	0.9083	$181.73	740.0M	2.3M	18h 47m	1546	6
9	Gemini 4 Argon	0.9080	$207.90	379.4M	1.5M	15h 34m	854	10
10	Claude Sonnet 5.5	0.8981	$169.21	677.4M	2.4M	19h 40m	1549	4
11	Claude Opus 5.5	0.8821	$144.22	511.3M	1.0M	18h 11m	1074	26
12	Claude Opus 5.5	0.8618	$140.79	553.3M	1.1M	19h 44m	1191	4
13	Claude Opus 5.5	0.8540	$182.36	671.0M	1.3M	19h 5m	1637	23
14	Gemini 4 Argon	0.8458	$261.90	442.3M	1.8M	19h 2m	1056	6
15	Claude Sonnet 5.5	0.7941	$169.89	694.5M	2.3M	19h 11m	1580	2
16	Gemini 4 Argon	0.7148	$130.35	185.9M	816k	8h 32m	449	3
17	Gemini 4 Argon	0.7072	$170.81	258.5M	1.2M	8h 45m	673	3
18	Claude Sonnet 5.5	0.7057	$164.69	656.9M	2.0M	19h 44m	1566	6
19	Claude Sonnet 5.5	0.6877	$201.48	823.9M	2.7M	19h 38m	1897	8
20	Claude Opus 5.5	0.6632	$158.77	603.5M	1.1M	18h 8m	1203	7
21	Claude Fable 5.1	0.4716	$208.91	376.5M	866k	17h 50m	1017	10
22	Claude Fable 5.1	0.3532	$168.41	314.0M	539k	12h 53m	608	4
23	Claude Fable 5.1	0.3481	$106.70	193.6M	429k	9h 8m	454	13
24	Claude Fable 5.1	0.3402	$125.75	229.9M	473k	10h 25m	482	6
25	Claude Opus 5	0.2471	$159.25	261.3M	595k	19h 14m	530	3
26	GPT-5.6 Sol	0.2171	$270.86	274.0M	362k	12h 30m	806	4
27	Claude Haiku 5.5	0.2045	$31.05	511.9M	1.8M	18h 49m	1138	3
28	Claude Opus 5	0.2039	$238.32	344.9M	699k	19h 25m	735	12
29	Claude Fable 5.1	0.1985	$113.40	205.2M	463k	5h 28m	430	3
30	Claude Fable 5	0.1934	$427.55	353.3M	996k	18h 35m	924	3
31	Claude Haiku 5.5	0.1872	$31.43	521.7M	1.9M	17h 21m	1152	5
32	Claude Opus 5	0.1567	$190.60	308.2M	776k	19h 30m	742	5
33	GPT-5.6 Sol	0.1552	$336.67	340.6M	347k	11h 28m	906	14
34	Claude Fable 5	0.1550	$396.50	323.2M	982k	19h 5m	848	7
35	Grok 4.7	0.1336	$487.10	480.1M	1.3M	20h 1m	1837	77
36	Claude Fable 5	0.1319	$351.84	291.9M	807k	19h 38m	711	8
37	Claude Opus 5	0.1302	$226.32	342.2M	722k	19h 35m	760	14
38	GPT-5.6 Sol	0.1222	$253.61	257.1M	305k	8h 28m	760	3
39	Grok 4.7	0.1182	$509.72	506.8M	1.5M	18h 52m	2068	85
40	Grok 4.7	0.1138	$588.60	597.8M	1.8M	19h 55m	2409	106
41	GPT-5.6 Sol	0.1119	$309.74	312.3M	342k	9h 22m	857	12
42	Muse Spark 1.3	0.1055	$206.91	689.2M	886k	10h 35m	1681	1
43	GPT-5.6 Sol	0.1032	$420.32	406.2M	354k	11h 35m	880	13
44	Claude Fable 5	0.0950	$314.88	250.5M	673k	18h 55m	484	7
45	Claude Haiku 5.5	0.0884	$18.09	294.8M	1.2M	8h 15m	682	4
46	Claude Haiku 5.5	0.0726	$19.59	319.7M	1.3M	12h 4m	710	5
47	Grok 4.7	0.0713	$540.56	547.5M	1.5M	19h 14m	2247	102
48	Grok 4.6	0.0526	$317.62	274.4M	1.8M	19h 55m	1078	110
49	Muse Spark 1.3	0.0410	$299.15	649.2M	932k	19h 37m	1588	2
50	Muse Spark 1.3	0.0386	$233.75	628.1M	517k	18h 2m	1282	5
51	Grok 4.6	0.0382	$286.38	260.8M	1.4M	19h 57m	984	103
52	Gemini 3.8 Flash	0.0349	$41.18	386.0M	562k	10h 22m	970	4
53	Grok 4.6	0.0273	$488.84	437.7M	2.0M	19h 56m	1536	307
54	Grok 4.6	0.0266	$454.46	403.4M	2.0M	19h 58m	1435	75
55	Muse Spark 1.3	0.0254	$147.68	673.6M	1.0M	19h 47m	1561	8
56	Muse Spark 1.3	0.0250	$351.72	511.0M	807k	19h 59m	1300	1
57	Kimi K3	0.0249	$126.07	350.4M	1.1M	19h 56m	936	7
58	Claude Opus 5	0.0249	$181.57	297.4M	666k	19h 31m	602	6
59	Kimi K3	0.0214	$168.60	476.4M	1.3M	20h 1m	1079	5
60	Grok 4.7	0.0196	$518.13	515.0M	1.7M	19h 40m	2079	58
61	GLM-5.3	0.0166	$91.30	316.9M	852k	19h 35m	891	7
62	Grok 4.6	0.0147	$328.36	283.8M	1.8M	19h 57m	1084	57
63	Claude Haiku 5.5	0.0140	$10.65	174.9M	638k	2h 59m	351	5
64	Gemini 3.8 Flash	0.0130	$23.89	196.2M	401k	6h 33m	507	3
65	Gemini 3.7 Flash	0.0116	$11.28	68.7M	237k	4h 7m	304	2
66	Kimi K3	0.0104	$167.00	460.9M	1.4M	20h 1m	1039	1
67	Gemini 3.8 Flash	0.0089	$32.30	275.0M	453k	6h 8m	610	4
68	Kimi K3	0.0079	$184.18	516.4M	1.3M	19h 37m	1094	2
69	DeepSeek V4 Flash Vision Exp	0.0069	$10.96	663.9M	1.1M	19h 2m	1527	2
70	DeepSeek V4 Flash Vision Exp	0.0065	$10.60	633.4M	1.1M	15h 56m	1516	8
71	Gemini 3.7 Flash	0.0064	$35.75	195.3M	360k	8h 28m	555	1
72	GLM-5.3	0.0054	$111.06	362.5M	755k	19h 29m	893	10
73	Gemini 3.7 Flash	0.0054	$6.52	51.3M	224k	2h 40m	274	2
74	GLM-5.3	0.0038	$97.66	297.0M	573k	19h 4m	633	5
75	DeepSeek V4 Flash Vision Exp	0.0028	$12.40	756.6M	1.2M	19h 8m	1756	39
76	DeepSeek V4 Flash Vision Exp	0.0027	$6.77	415.2M	638k	18h 28m	837	7
77	Gemini 3.7 Flash	0.0020	$17.05	105.9M	300k	5h 32m	381	1
78	Qwen3.8-Max	0.0018	$183.02	558.1M	1.7M	19h 56m	1375	1
79	Qwen3.8-Max	0.0016	$153.43	448.8M	1.7M	19h 59m	1202	1
80	GLM-5.3 Flash	0.0015	$14.67	306.8M	449k	20h 1m	746	1
81	Qwen3.8-Max	0.0010	$149.46	419.0M	1.2M	19h 25m	993	2
82	GLM-5.3 Flash	0.0010	$16.55	339.1M	548k	17h 44m	740	5
83	Gemini 3.7 Flash	0.0009	$11.28	77.4M	230k	5h 1m	369	3
84	Gemini 3.8 Flash	0.0007	$23.00	212.3M	376k	5h 7m	526	3
85	GLM-5.3	0.0006	$94.18	300.9M	621k	18h 57m	592	10
86	GLM-5.3 Flash	0.0002	$12.16	275.0M	487k	19h 36m	628	3
87	GLM-5.3 Flash	0.0001	$17.12	399.9M	616k	15h 49m	828	5
88	Qwen3.8-Max	0.0001	$169.64	519.3M	1.6M	20h 0m	1239	1
89	Muse Spark 1.2	0.0000	$7.77	40.8M	287k	2h 9m	182	5
90	Inkling	0.0000	$6.72	7.0M	25k	22m	138	2
91	Qwen3.8-Max	0.0000	$189.91	559.5M	1.9M	19h 51m	1464	3
92	Kimi K3	0.0000	$182.81	498.0M	1.4M	20h 1m	1063	2
93	Muse Spark 1.2	0.0000	$8.07	38.9M	412k	3h 7m	202	4
94	DeepSeek V4 Flash Vision Exp	0.0000	$11.01	666.7M	1.1M	19h 11m	1467	11
95	Qwen3.8-Max-0902	0.0000	$58.14	147.3M	957k	20h 6m	434	1
96	Qwen3.8-Max-0902	0.0000	$33.94	103.0M	952k	20h 7m	367	1
97	Muse Spark 1.2	0.0000	$6.30	24.2M	135k	1h 37m	159	4
98	Qwen3.8-Max-0902	0.0000	$41.85	96.9M	886k	20h 4m	414	1
99	Claude Fable 5	0.0000	$411.65	293.0M	731k	19h 45m	698	3
100	GLM-5.3	0.0000	$103.09	375.1M	554k	19h 46m	790	12
101	Gemini 3.8 Flash	0.0000	$53.67	394.0M	628k	13h 26m	1068	4
102	Inkling	0.0000	$6.59	6.8M	31k	2h 10m	146	2
103	Muse Spark 1.2	0.0000†	$2.64	12.4M	42k	2h 25m	90	2
104	Qwen3.8-Max-0902	0.0000	$50.29	127.1M	914k	20h 7m	479	1
105	Inkling	0.0000	$6.57	6.8M	31k	35m	156	1
106	GLM-5.3 Flash	0.0000	$17.57	389.3M	753k	20h 0m	891	8
107	Qwen3.8-Max-0902	0.0000	$56.38	146.6M	1.2M	20h 0m	468	1
108	Inkling	0.0000	$4.24	4.4M	17k	41m	109	1
109	Inkling	0.0000	$8.87	9.2M	43k	1h 49m	170	2
110	Muse Spark 1.2	0.0000†	$1.72	8.1M	39k	1h 17m	88	1

† score adjusted by manual review; the trace's Judge tab carries the measured score and the rationale.

FrogsGame Post-Training

task page →110 runs · indexed 2026-10-08

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Sonnet 5.5	0.9920	$41.94	172.9M	571k	18h 38m	358	1
2	Claude Sonnet 5.5	0.9920	$37.06	145.6M	621k	18h 34m	289	4
3	Claude Sonnet 5.5	0.9860	$40.55	166.2M	572k	19h 5m	363	1
4	Claude Fable 5.1	0.9740	$14.41	21.9M	144k	17h 0m	153	5
5	GPT-6 Astra	0.9700	$116.67	60.2M	205k	18h 38m	255	8
6	Claude Opus 5.5	0.9380	$18.10	54.0M	277k	17h 56m	203	3
7	Claude Sonnet 5.5	0.9380	$31.79	125.6M	517k	19h 26m	294	2
8	Claude Opus 5.5	0.9340	$25.67	82.0M	341k	18h 34m	248	6
9	Claude Opus 5.5	0.9320	$21.18	64.6M	311k	18h 52m	228	5
10	GPT-6 Astra	0.9160	$256.74	124.3M	279k	19h 10m	365	11
11	Claude Sonnet 5.5	0.9160	$33.87	130.0M	616k	16h 46m	279	3
12	Claude Opus 5.5	0.9060	$27.72	93.7M	337k	17h 43m	292	4
13	GPT-6 Astra	0.9000	$49.97	27.6M	160k	19h 17m	158	6
14	Gemini 4 Argon	0.8920	$46.04	49.7M	235k	14h 34m	172	5
15	Claude Fable 5.1	0.8920	$12.71	19.5M	122k	16h 12m	146	4
16	Claude Haiku 5.5	0.8900	$4.98	80.9M	361k	18h 50m	247	5
17	GPT-6 Astra	0.8840	$265.88	128.3M	310k	19h 14m	390	8
18	Claude Haiku 5.5	0.8580	$5.65	92.0M	404k	18h 43m	272	1
19	Claude Haiku 5.5	0.8540	$5.01	78.6M	413k	17h 14m	235	1
20	GLM-5.3	0.8380	$21.95	47.4M	206k	19h 31m	233	2
21	Claude Opus 5.5	0.8240	$28.19	95.1M	341k	18h 8m	296	5
22	Gemini 4 Argon	0.8020	$82.91	62.7M	281k	14h 57m	166	4
23	Claude Fable 5.1	0.7800	$16.40	25.5M	151k	15h 15m	165	3
24	Claude Fable 5	0.7400	$44.21	30.2M	189k	19h 22m	191	4
25	GLM-5.3	0.7220	$12.91	44.6M	176k	19h 17m	247	2
26	Claude Opus 5	0.7140	$29.77	41.4M	245k	19h 28m	207	2
27	Gemini 4 Argon	0.6900	$75.10	61.9M	250k	17h 44m	191	4
28	Claude Opus 5	0.6840	$35.58	51.9M	259k	19h 2m	225	2
29	GPT-6 Astra	0.6760	$444.71	214.4M	392k	19h 40m	521	10
30	Claude Opus 5	0.6660	$32.36	44.1M	277k	18h 32m	192	4
31	Claude Fable 5.1	0.6180	$15.24	23.4M	145k	17h 43m	162	6
32	Muse Spark 1.3	0.6080	$42.76	36.7M	222k	19h 41m	241	2
33	Gemini 4 Argon	0.6040	$30.56	35.0M	214k	7h 42m	148	3
34	Gemini 4 Argon	0.5940	$87.69	65.7M	221k	8h 37m	175	3
35	Claude Fable 5	0.5920	$48.49	32.7M	211k	19h 14m	177	3
36	Claude Haiku 5.5	0.5500	$7.58	127.1M	460k	19h 19m	337	6
37	Claude Opus 5	0.5280	$40.04	56.7M	315k	17h 56m	222	4
38	Claude Haiku 5.5	0.5220	$5.68	92.2M	413k	18h 50m	284	2
39	Grok 4.7	0.4940	$60.79	36.4M	219k	19h 41m	169	1
40	Kimi K3	0.4620	$12.73	24.9M	159k	19h 34m	172	1
41	Claude Fable 5.1	0.4080	$13.04	19.3M	141k	15h 46m	135	4
42	Grok 4.7	0.3640	$259.38	236.3M	353k	19h 57m	795	2
43	GLM-5.3	0.3120	$16.40	57.5M	195k	19h 21m	314	2
44	Kimi K3	0.3000	$16.00	38.1M	240k	20h 0m	187	1
45	Grok 4.7	0.2980	$90.99	63.2M	209k	18h 43m	277	2
46	Claude Fable 5	0.2940	$55.28	38.9M	218k	19h 5m	196	4
47	Grok 4.7	0.2920	$118.94	66.1M	194k	19h 26m	282	2
48	Claude Opus 5	0.2740	$33.23	45.6M	283k	19h 1m	200	2
49	Claude Fable 5	0.2640	$49.52	34.1M	206k	18h 52m	186	4
50	Grok 4.6	0.2460	$18.57	19.8M	283k	19h 59m	152	1
51	Grok 4.6	0.2420	$9.68	12.9M	262k	19h 22m	132	2
52	Qwen3.8-Max	0.2160	$11.97	38.7M	277k	19h 34m	297	3
53	Kimi K3	0.1900	$13.63	32.0M	177k	20h 0m	198	1
54	GLM-5.3	0.1640	$20.82	41.0M	222k	19h 4m	214	3
55	GLM-5.3	0.1580	$16.59	58.5M	233k	19h 38m	265	1
56	Claude Fable 5	0.1360	$52.26	35.5M	227k	18h 52m	183	4
57	Muse Spark 1.3	0.1340	$12.49	16.7M	201k	19h 20m	179	3
58	Grok 4.6	0.1280	$47.67	45.1M	287k	18h 35m	259	5
59	DeepSeek V4 Flash Vision Exp	0.1100	$1.27	65.0M	240k	20h 4m	306	1
60	GLM-5.3 Flash	0.1040	$2.48	43.2M	239k	18h 52m	206	2
61	Kimi K3	0.1040	$11.13	20.9M	147k	19h 47m	174	1
62	GLM-5.3 Flash	0.1020	$2.48	58.2M	263k	18h 49m	267	2
63	Qwen3.8-Max	0.0980	$10.69	32.2M	339k	19h 46m	272	1
64	Gemini 3.8 Flash	0.0960	$23.82	139.9M	200k	8h 48m	450	31
65	Gemini 3.8 Flash	0.0900	$36.72	152.5M	162k	13h 5m	420	3
66	GPT-5.6 Sol	0.0900	$19.46	28.7M	100k	17h 8m	206	2
67	Qwen3.8-Max	0.0880	$5.85	16.5M	229k	19h 7m	195	1
68	DeepSeek V4 Flash Vision Exp	0.0860	$1.47	79.6M	227k	19h 17m	390	2
69	Qwen3.8-Max-0902	0.0680	$32.05	102.3M	870k	19h 37m	406	2
70	Kimi K3	0.0660	$8.10	17.2M	123k	19h 18m	153	1
71	Grok 4.6	0.0640	$49.02	43.5M	397k	19h 5m	221	4
72	Gemini 3.7 Flash	0.0620	$5.03	14.3M	76k	5h 27m	128	1
73	Muse Spark 1.3	0.0580	$11.26	13.8M	164k	19h 42m	163	3
74	GPT-5.6 Sol	0.0540	$20.87	32.1M	115k	12h 56m	241	3
75	Grok 4.7	0.0460	$50.68	36.5M	192k	18h 50m	196	2
76	GLM-5.3 Flash	0.0420	$2.01	26.5M	194k	19h 34m	169	1
77	Gemini 3.7 Flash	0.0420	$30.72	72.9M	130k	16h 34m	301	1
78	GPT-5.6 Sol	0.0380	$22.11	34.6M	103k	18h 7m	223	2
79	Muse Spark 1.3	0.0380	$29.75	31.6M	256k	19h 58m	261	1
80	GPT-5.6 Sol	0.0360	$61.31	64.5M	145k	15h 2m	277	2
81	Qwen3.8-Max	0.0360	$35.74	120.9M	650k	19h 55m	546	2
82	Gemini 3.8 Flash	0.0340	$11.75	59.4M	141k	7h 14m	276	4
83	Grok 4.6	0.0340	$48.29	46.2M	363k	16h 43m	237	5
84	Qwen3.8-Max-0902	0.0320	$32.58	103.5M	819k	19h 35m	415	2
85	Gemini 3.7 Flash	0.0320	$45.33	138.6M	173k	19h 58m	396	1
86	Muse Spark 1.3	0.0320	$20.03	29.7M	151k	19h 22m	241	1
87	Gemini 3.8 Flash	0.0300	$54.93	193.3M	322k	19h 11m	517	2
88	Qwen3.8-Max	0.0300	$11.98	37.6M	292k	18h 44m	323	4
89	Muse Spark 1.2	0.0300	$18.36	45.0M	166k	4h 11m	176	5
90	GLM-5.3 Flash	0.0280	$2.02	52.4M	227k	19h 53m	271	1
91	Gemini 3.8 Flash	0.0240	$6.62	28.1M	109k	3h 35m	158	2
92	Qwen3.8-Max-0902	0.0220	$35.21	102.2M	1.1M	19h 56m	399	—
93	Qwen3.8-Max-0902	0.0220	$33.10	101.6M	958k	19h 40m	430	3
94	Gemini 3.7 Flash	0.0220	$3.80	24.6M	101k	2h 41m	142	2
95	Muse Spark 1.2	0.0220	$21.74	48.9M	286k	5h 11m	239	11
96	Gemini 3.7 Flash	0.0200	$37.07	96.2M	140k	17h 26m	321	1
97	GLM-5.3 Flash	0.0160	$3.03	57.8M	247k	19h 17m	254	2
98	DeepSeek V4 Flash Vision Exp	0.0160	$1.33	68.9M	247k	19h 56m	345	2
99	DeepSeek V4 Flash Vision Exp	0.0160	$1.13	55.9M	221k	19h 37m	264	2
100	GPT-5.6 Sol	0.0120	$91.18	51.7M	86k	17h 41m	228	4
101	Muse Spark 1.2	0.0100	$18.40	105.2M	433k	4h 43m	251	5
102	Inkling	0.0080	$2.60	2.6M	22k	28m	84	2
103	Muse Spark 1.2	0.0060	$13.73	65.6M	312k	5h 25m	183	9
104	Inkling	0.0000	$8.37	8.7M	31k	38m	169	2
105	DeepSeek V4 Flash Vision Exp	0.0000	$1.61	87.5M	249k	20h 2m	395	1
106	Inkling	0.0000	$3.54	3.7M	16k	24m	97	2
107	Inkling	0.0000	$1.63	1.6M	18k	24m	71	2
108	Muse Spark 1.2	0.0000	$196.24	1245.9M	587k	4h 6m	2389	8
109	Qwen3.8-Max-0902	0.0000	$4.94	9.5M	357k	5h 14m	93	1
110	Inkling	0.0000	$2.52	2.6M	17k	28m	103	2

Git to Zig

task page →110 runs · indexed 2026-10-08

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	GPT-6 Astra	0.9864	$3111.97	1407.9M	3.4M	19h 58m	3119	48
2	GPT-6 Astra	0.9049	$3006.21	1380.7M	3.3M	20h 0m	3129	60
3	GPT-6 Astra	0.8950	$2950.79	1350.4M	3.3M	19h 50m	3031	52
4	GPT-6 Astra	0.8938	$3057.89	1408.3M	3.3M	19h 59m	3204	96
5	GPT-6 Astra	0.8893	$3214.36	1493.6M	3.2M	19h 59m	3347	99
6	Claude Opus 5.5	0.6812	$313.47	1048.3M	4.0M	19h 32m	2436	25
7	Claude Opus 5.5	0.6644	$241.59	773.6M	3.3M	19h 40m	1843	33
8	Claude Opus 5.5	0.6378	$222.91	717.3M	3.1M	19h 51m	1607	19
9	Claude Opus 5.5	0.6246	$242.07	814.4M	3.0M	19h 50m	1773	32
10	Claude Sonnet 5.5	0.6193	$286.71	1152.7M	4.4M	19h 47m	2662	17
11	Claude Sonnet 5.5	0.6146	$288.13	1151.1M	4.5M	19h 38m	2487	13
12	Claude Sonnet 5.5	0.6064	$328.10	1331.6M	4.8M	19h 55m	3071	8
13	Claude Opus 5.5	0.5824	$188.65	606.3M	2.6M	19h 43m	1437	34
14	Claude Fable 5.1	0.5599	$663.30	1199.7M	2.6M	18h 57m	2674	17
15	Claude Sonnet 5.5	0.5548	$219.87	857.0M	3.8M	19h 46m	2030	10
16	Gemini 4 Argon	0.5447	$362.36	783.2M	2.2M	20h 48m	1753	26
17	Claude Sonnet 5.5	0.5357	$185.56	725.4M	3.2M	19h 53m	1763	8
18	Gemini 4 Argon	0.5225	$190.93	727.6M	2.5M	20h 51m	1657	23
19	Claude Fable 5.1	0.5146	$566.08	1019.0M	2.3M	19h 11m	2360	13
20	Claude Fable 5.1	0.5049	$432.88	762.8M	2.1M	18h 58m	1732	13
21	Claude Fable 5.1	0.4886	$363.29	631.2M	2.0M	18h 58m	1562	11
22	Claude Fable 5.1	0.4844	$390.64	686.7M	2.0M	19h 13m	1611	26
23	Gemini 4 Argon	0.4697	$330.68	663.7M	1.9M	20h 55m	1569	11
24	Gemini 4 Argon	0.4294	$398.78	647.4M	2.0M	20h 52m	1487	26
25	Gemini 4 Argon	0.4167	$333.71	642.4M	2.0M	20h 59m	1402	31
26	Claude Fable 5	0.4059	$841.49	687.0M	2.0M	19h 35m	1534	22
27	Claude Fable 5	0.4009	$1051.04	892.5M	2.1M	19h 47m	1943	25
28	Claude Opus 5	0.3856	$635.16	1128.1M	1.8M	19h 53m	2411	22
29	Claude Fable 5	0.3814	$1038.32	878.7M	2.1M	19h 45m	1942	22
30	Claude Opus 5	0.3683	$533.23	935.7M	1.7M	19h 23m	1826	15
31	Claude Fable 5	0.3565	$966.19	820.9M	1.9M	19h 53m	1710	14
32	GPT-5.6 Sol	0.3440	$1238.08	1225.0M	944k	19h 8m	2627	21
33	Claude Opus 5	0.3411	$614.97	1098.7M	1.7M	19h 37m	2206	22
34	Claude Fable 5	0.3411	$903.80	768.8M	1.8M	19h 40m	1672	10
35	Claude Haiku 5.5	0.3210	$69.20	1225.1M	2.9M	18h 51m	2564	3
36	Claude Haiku 5.5	0.3209	$63.68	1097.5M	3.2M	19h 26m	2348	44
37	Claude Opus 5	0.3150	$429.99	753.2M	1.4M	19h 44m	1660	18
38	Claude Opus 5	0.3034	$490.76	870.1M	1.4M	19h 44m	1974	13
39	Claude Haiku 5.5	0.2918	$66.19	1196.6M	2.4M	17h 47m	2539	17
40	Claude Haiku 5.5	0.2897	$71.59	1281.2M	2.8M	19h 16m	2753	24
41	Claude Haiku 5.5	0.2814	$60.90	1077.6M	2.7M	19h 17m	2251	20
42	GPT-5.6 Sol	0.2804	$1082.72	1080.1M	850k	19h 36m	2314	42
43	GPT-5.6 Sol	0.2801	$1175.97	1171.4M	884k	19h 37m	2602	29
44	Grok 4.7	0.2530	$1242.68	1266.7M	4.0M	20h 0m	5068	70
45	Grok 4.7	0.2469	$910.55	936.9M	2.7M	19h 58m	3768	53
46	GPT-5.6 Sol	0.2369	$955.56	945.7M	722k	19h 4m	1982	13
47	Grok 4.7	0.2342	$1107.04	1144.5M	3.0M	20h 0m	4638	66
48	Grok 4.7	0.2156	$741.65	767.0M	2.0M	19h 54m	3099	73
49	Gemini 3.8 Flash	0.1938	$193.40	2143.9M	2.9M	19h 15m	4751	23
50	Grok 4.7	0.1876	$623.38	637.1M	2.0M	13h 32m	2600	18
51	GPT-5.6 Sol	0.1874	$770.31	763.5M	629k	19h 0m	1668	6
52	Gemini 3.7 Flash	0.1808	$157.01	1517.8M	2.4M	19h 43m	3256	1
53	Gemini 3.8 Flash	0.1795	$179.04	1977.0M	2.7M	19h 44m	4293	17
54	Kimi K3	0.1768	$251.00	752.4M	1.2M	20h 2m	1619	1
55	Kimi K3	0.1733	$200.20	602.7M	965k	19h 43m	1483	13
56	Gemini 3.8 Flash	0.1706	$167.16	1718.1M	2.3M	19h 49m	3932	19
57	Grok 4.6	0.1693	$652.70	657.8M	2.2M	19h 59m	2502	122
58	Gemini 3.8 Flash	0.1621	$197.44	2159.7M	2.7M	19h 47m	4872	28
59	Grok 4.6	0.1618	$614.44	610.4M	2.6M	18h 0m	2324	73
60	Kimi K3	0.1616	$254.18	761.8M	1.1M	19h 47m	1668	9
61	Gemini 3.8 Flash	0.1604	$196.69	2138.2M	2.7M	20h 0m	4694	26
62	Kimi K3	0.1574	$259.84	779.9M	1.1M	19h 53m	1765	2
63	GLM-5.3	0.1556	$225.36	842.0M	964k	19h 45m	2012	17
64	GLM-5.3	0.1531	$195.54	692.6M	689k	19h 54m	1480	2
65	Grok 4.6	0.1516	$806.68	819.7M	2.2M	19h 59m	2981	186
66	Inkling	0.1510	$17.77	18.6M	32k	1h 29m	255	1
67	Gemini 3.7 Flash	0.1495	$79.59	838.0M	1.2M	20h 0m	1855	2
68	Kimi K3	0.1473	$222.64	660.4M	1.1M	19h 44m	1491	1
69	Gemini 3.7 Flash	0.1433	$101.57	1069.8M	1.5M	19h 58m	2372	1
70	GLM-5.3 Flash	0.1431	$24.60	751.1M	974k	19h 4m	1752	13
71	Muse Spark 1.3	0.1427	$402.31	656.1M	1.1M	18h 56m	1632	3
72	GLM-5.3	0.1390	$212.77	708.2M	852k	19h 49m	1640	49
73	GLM-5.3	0.1378	$248.79	932.5M	906k	19h 55m	2219	21
74	Grok 4.6	0.1352	$482.88	483.0M	1.6M	11h 22m	1767	84
75	GLM-5.3 Flash	0.1328	$26.79	827.0M	1.1M	19h 10m	1751	10
76	GLM-5.3 Flash	0.1304	$29.71	954.5M	1.0M	19h 55m	2035	10
77	DeepSeek V4 Flash Vision Exp	0.1304	$28.18	1759.3M	2.2M	19h 24m	3723	15
78	Muse Spark 1.3	0.1274	$518.45	593.9M	920k	19h 54m	1465	2
79	Grok 4.6	0.1269	$408.61	405.7M	1.8M	19h 55m	1558	48
80	GLM-5.3 Flash	0.1260	$27.25	852.3M	924k	19h 18m	1873	6
81	Gemini 3.7 Flash	0.1237	$71.55	746.3M	1.3M	19h 57m	1690	4
82	GLM-5.3	0.1219	$188.86	712.0M	621k	19h 43m	1318	16
83	DeepSeek V4 Flash Vision Exp	0.1213	$28.06	1756.1M	2.2M	19h 43m	3793	11
84	Qwen3.8-Max	0.1212	$184.24	696.5M	1.3M	19h 33m	1805	12
85	GLM-5.3 Flash	0.1195	$28.09	863.8M	1.0M	19h 28m	2000	13
86	DeepSeek V4 Flash Vision Exp	0.1193	$26.16	1630.6M	2.1M	19h 37m	3584	8
87	Qwen3.8-Max-0902	0.1143	$142.85	503.3M	2.3M	19h 27m	1335	8
88	DeepSeek V4 Flash Vision Exp	0.1079	$21.83	1364.0M	1.7M	17h 17m	2977	8
89	Muse Spark 1.3	0.1062	$484.87	561.8M	975k	20h 0m	1499	1
90	Qwen3.8-Max	0.1038	$196.74	742.9M	1.4M	19h 38m	1831	26
91	Qwen3.8-Max	0.1023	$180.44	685.7M	1.2M	19h 14m	1776	2
92	Muse Spark 1.3	0.1006	$482.98	557.2M	1.0M	20h 0m	1594	15
93	DeepSeek V4 Flash Vision Exp	0.0988	$20.50	1272.8M	1.7M	19h 17m	2732	11
94	Qwen3.8-Max	0.0987	$178.20	662.0M	1.7M	19h 50m	1710	6
95	Qwen3.8-Max	0.0927	$190.64	725.2M	1.3M	19h 52m	1620	35
96	Qwen3.8-Max-0902	0.0891	$199.99	712.3M	3.1M	19h 47m	1665	9
97	Qwen3.8-Max-0902	0.0536	$90.79	292.3M	2.5M	20h 1m	788	2
98	Qwen3.8-Max-0902	0.0529	$127.39	412.8M	3.6M	20h 1m	1158	1
99	Muse Spark 1.2	0.0185	$42.33	250.7M	911k	3h 1m	578	5
100	Inkling	0.0093	$17.52	18.3M	28k	1h 29m	208	3
101	Qwen3.8-Max-0902	0.0086	$101.78	346.1M	2.1M	20h 0m	916	2
102	Inkling	0.0019	$16.36	17.1M	30k	17m	280	1
103	Muse Spark 1.2	0.0000†	$2.45	9.3M	123k	59m	86	2
104	Inkling	0.0000	$3.15	3.3M	13k	5m	86	2
105	Muse Spark 1.3	0.0000	$641.52	782.0M	1.4M	19h 59m	1580	1
106	Muse Spark 1.2	0.0000†	$5.53	26.0M	135k	1h 56m	175	4
107	Muse Spark 1.2	0.0000†	$59.99	358.3M	658k	4h 22m	779	1
108	Gemini 3.7 Flash	0.0000†	$76.41	819.6M	1.5M	19h 43m	1734	1
109	Inkling	0.0000†	$5.57	5.8M	17k	1h 55m	118	2
110	Muse Spark 1.2	0.0000†	$2.14	7.7M	83k	1h 10m	93	3

† score adjusted by manual review; the trace's Judge tab carries the measured score and the rationale.

Granite Mamba2 Inference Optimization

task page →110 runs · indexed 2026-10-08

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Sonnet 5.5	0.9284	$98.15	363.7M	2.0M	12h 14m	758	2
2	Gemini 4 Argon	0.9104	$105.47	115.9M	895k	5h 40m	315	6
3	Qwen3.8-Max-0902	0.9095	$72.15	209.6M	2.3M	19h 47m	563	2
4	Gemini 4 Argon	0.9046	$83.33	130.1M	792k	5h 37m	359	3
5	Gemini 4 Argon	0.8993	$145.24	118.2M	769k	6h 54m	331	8
6	GLM-5.3	0.8927	$85.79	297.0M	636k	6h 7m	634	7
7	Claude Haiku 5.5	0.8911	$15.33	230.6M	1.3M	10h 52m	500	2
8	Gemini 4 Argon	0.8884	$91.03	70.1M	500k	3h 12m	195	3
9	Claude Opus 5.5	0.8846	$214.03	690.9M	2.9M	15h 36m	1506	34
10	Gemini 4 Argon	0.8832	$36.18	98.9M	556k	3h 28m	236	3
11	Claude Opus 5.5	0.8817	$132.91	426.3M	1.8M	6h 35m	922	25
12	Claude Opus 5.5	0.8769	$159.17	476.8M	2.5M	11h 3m	1094	29
13	GLM-5.3	0.8719	$51.39	174.8M	618k	4h 8m	382	8
14	DeepSeek V4 Flash Vision Exp	0.8708	$2.68	149.8M	376k	16h 23m	473	5
15	GPT-6 Astra	0.8707	$2626.81	1259.5M	2.3M	18h 25m	2839	58
16	GPT-6 Astra	0.8658	$272.49	128.6M	369k	2h 40m	367	7
17	Claude Sonnet 5.5	0.8655	$166.81	620.5M	3.4M	17h 36m	1346	14
18	Claude Opus 5	0.8619	$158.22	232.0M	1.1M	7h 45m	543	10
19	Grok 4.7	0.8555	$664.66	670.3M	2.4M	20h 0m	2515	55
20	Claude Fable 5.1	0.8458	$138.31	228.0M	1000k	9h 1m	580	15
21	GLM-5.3	0.8401	$45.93	158.7M	655k	5h 28m	343	6
22	Claude Opus 5	0.8400	$159.98	236.7M	1.1M	11h 10m	566	15
23	Claude Fable 5.1	0.8393	$189.96	320.0M	1.2M	8h 30m	718	28
24	Claude Haiku 5.5	0.8379	$8.93	123.1M	1.0M	4h 22m	301	3
25	Claude Fable 5	0.8372	$204.33	157.4M	642k	12h 58m	310	20
26	GPT-6 Astra	0.8334	$251.26	124.6M	331k	2h 35m	409	4
27	Claude Haiku 5.5	0.8309	$11.48	157.4M	1.3M	4h 35m	386	4
28	Grok 4.7	0.8303	$663.28	652.6M	2.1M	14h 34m	2345	24
29	GPT-6 Astra	0.8266	$344.95	163.8M	406k	3h 8m	459	8
30	GPT-6 Astra	0.8198	$254.82	122.3M	354k	2h 28m	374	7
31	Claude Fable 5.1	0.8171	$168.52	281.0M	1.2M	8h 8m	667	16
32	Muse Spark 1.2	0.8168	$5.16	25.8M	238k	1h 23m	169	3
33	Qwen3.8-Max-0902	0.8140	$43.20	91.8M	3.2M	19h 47m	354	1
34	Muse Spark 1.3	0.8136	$31.62	38.2M	374k	3h 49m	186	5
35	Qwen3.8-Max	0.8086	$63.86	200.9M	2.1M	18h 31m	572	2
36	Claude Fable 5.1	0.8047	$204.26	349.5M	1.2M	8h 27m	784	31
37	Muse Spark 1.3	0.8036	$155.17	181.2M	487k	18h 42m	545	11
38	Claude Haiku 5.5	0.8023	$12.99	188.9M	1.3M	8h 53m	451	2
39	GLM-5.3 Flash	0.7990	$11.45	322.1M	969k	12h 30m	791	4
40	Qwen3.8-Max-0902	0.7969	$68.75	217.9M	2.1M	19h 23m	595	2
41	Inkling	0.7968	$5.41	5.6M	24k	1h 10m	98	3
42	Claude Fable 5	0.7947	$155.09	112.6M	581k	5h 11m	247	9
43	Inkling	0.7910	$2.64	2.7M	12k	47m	61	2
44	DeepSeek V4 Flash Vision Exp	0.7876	$3.53	195.8M	540k	19h 32m	519	8
45	Qwen3.8-Max	0.7876	$54.77	177.9M	1.6M	16h 1m	520	5
46	Kimi K3	0.7872	$122.78	337.3M	986k	19h 10m	771	36
47	Claude Fable 5.1	0.7867	$157.78	258.9M	1.2M	7h 40m	582	11
48	GLM-5.3 Flash	0.7842	$10.09	304.8M	714k	7h 37m	595	8
49	Gemini 3.7 Flash	0.7839	$8.97	84.4M	442k	56m	281	3
50	Muse Spark 1.3	0.7809	$183.67	203.2M	648k	10h 29m	557	11
51	Kimi K3	0.7805	$103.11	286.2M	907k	14h 49m	651	10
52	Grok 4.7	0.7778	$748.71	746.1M	3.0M	20h 0m	2763	23
53	Claude Opus 5	0.7718	$80.48	113.3M	644k	3h 51m	243	5
54	Inkling	0.7670	$5.68	5.8M	36k	1h 22m	101	2
55	GPT-5.6 Sol	0.7649	$74.74	74.2M	207k	2h 36m	270	5
56	DeepSeek V4 Flash Vision Exp	0.7578	$8.78	501.2M	1.2M	13h 5m	1157	3
57	Kimi K3	0.7571	$91.87	248.8M	620k	20h 0m	531	50
58	GLM-5.3	0.7528	$38.94	126.7M	486k	4h 14m	351	5
59	Claude Opus 5.5	0.7510	$78.62	226.6M	1.3M	6h 44m	478	12
60	Kimi K3	0.7477	$74.48	205.5M	639k	8h 3m	456	20
61	Grok 4.7	0.7473	$45.19	45.2M	263k	1h 21m	181	3
62	DeepSeek V4 Flash Vision Exp	0.7467	$8.27	472.9M	1.1M	11h 58m	1140	23
63	GLM-5.3 Flash	0.7460	$13.06	377.4M	1.1M	12h 40m	970	2
64	Claude Haiku 5.5	0.7446	$23.17	349.9M	2.1M	10h 30m	763	4
65	GLM-5.3 Flash	0.7421	$9.37	261.1M	646k	9h 26m	535	2
66	GLM-5.3	0.7402	$85.05	310.6M	673k	18h 16m	645	8
67	Claude Fable 5	0.7331	$198.58	154.1M	599k	9h 40m	302	16
68	Grok 4.6	0.7305	$58.74	52.8M	626k	4h 20m	206	13
69	Grok 4.6	0.7245	$46.67	39.7M	692k	3h 57m	169	10
70	Qwen3.8-Max	0.7188	$63.39	207.4M	1.7M	17h 14m	557	9
71	Muse Spark 1.2	0.7107	$36.53	207.7M	800k	2h 20m	527	1
72	Qwen3.8-Max	0.7023	$29.64	70.4M	1.9M	19h 40m	330	1
73	Claude Opus 5	0.6979	$228.77	352.0M	1.4M	11h 8m	751	18
74	Claude Sonnet 5.5	0.6871	$127.04	479.4M	2.5M	17h 41m	1063	10
75	Qwen3.8-Max	0.6658	$53.16	162.1M	1.8M	18h 29m	495	2
76	Grok 4.6	0.6586	$55.70	47.5M	700k	4h 6m	194	10
77	Grok 4.6	0.6473	$57.88	51.1M	725k	4h 13m	210	5
78	Muse Spark 1.3	0.6457	$320.20	329.1M	590k	13h 47m	773	12
79	GPT-5.6 Sol	0.6303	$111.93	116.6M	261k	11h 2m	425	3
80	Muse Spark 1.3	0.6302	$29.92	88.8M	460k	3h 13m	325	9
81	Kimi K3	0.6169	$112.73	307.9M	980k	13h 16m	708	31
82	Qwen3.8-Max-0902	0.6109	$38.46	77.0M	3.0M	18h 44m	298	2
83	GPT-5.6 Sol	0.6093	$23.53	32.7M	162k	1h 28m	207	3
84	Claude Fable 5	0.5974	$289.56	218.1M	971k	11h 20m	529	29
85	Claude Opus 5.5	0.5826	$111.15	316.0M	1.9M	10h 19m	731	29
86	Claude Sonnet 5.5	0.5555	$156.83	594.5M	3.0M	18h 16m	1275	4
87	Claude Fable 5	0.5364	$304.87	227.0M	1.1M	13h 5m	532	22
88	Claude Opus 5	0.5059	$98.60	149.0M	645k	5h 53m	287	7
89	Gemini 3.8 Flash	0.5042	$12.98	108.1M	394k	2h 18m	329	5
90	Claude Sonnet 5.5	0.4810	$155.71	584.6M	3.1M	16h 20m	1290	8
91	DeepSeek V4 Flash Vision Exp	0.4680	$3.00	159.1M	520k	6h 30m	442	5
92	GPT-5.6 Sol	0.4658	$119.15	122.5M	308k	3h 17m	463	6
93	GLM-5.3 Flash	0.4647	$8.11	210.3M	541k	9h 13m	473	3
94	Grok 4.7	0.4410	$49.65	48.3M	283k	2h 1m	184	13
95	Gemini 3.8 Flash	0.4012	$12.80	133.1M	348k	1h 19m	360	2
96	Gemini 3.7 Flash	0.3912	$15.89	144.8M	798k	2h 17m	479	5
97	Gemini 3.8 Flash	0.3668	$11.65	115.4M	351k	1h 8m	353	6
98	GPT-5.6 Sol	0.3383	$180.46	175.3M	266k	4h 7m	474	10
99	Gemini 3.7 Flash	0.3149	$10.74	97.9M	514k	1h 12m	278	2
100	Gemini 3.8 Flash	0.2792	$7.71	71.8M	268k	1h 9m	283	2
101	Grok 4.6	0.2777	$27.64	23.0M	551k	3h 18m	127	2
102	Gemini 3.7 Flash	0.2425	$12.51	121.4M	559k	2h 10m	326	2
103	Gemini 3.7 Flash	0.2101	$6.70	49.8M	379k	47m	212	3
104	Inkling	0.0000	$7.50	7.7M	51k	1h 46m	135	1
105	Gemini 3.8 Flash	0.0000	$15.57	160.4M	440k	1h 35m	409	1
106	Qwen3.8-Max-0902	0.0000	$60.04	188.3M	1.9M	19h 52m	497	3
107	Muse Spark 1.2	0.0000	$19.40	92.0M	1.0M	4h 3m	307	3
108	Muse Spark 1.2	0.0000	$9.93	53.1M	354k	2h 11m	258	6
109	Muse Spark 1.2	0.0000	$8.05	39.0M	223k	1h 17m	201	4
110	Inkling	0.0000	$4.66	4.8M	18k	1h 13m	91	2

Higgs Uncertainty Inference

task page →110 runs · indexed 2026-10-08

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	GPT-6 Astra	1.0000	$164.97	80.6M	296k	2h 47m	306	8
2	Claude Opus 5	1.0000	$43.00	55.4M	421k	5h 4m	193	4
3	Claude Opus 5.5	1.0000	$25.57	70.3M	442k	6h 17m	211	11
4	Claude Opus 5	1.0000	$43.73	58.4M	394k	12h 22m	210	6
5	GPT-6 Astra	1.0000	$50.75	31.1M	211k	1h 50m	184	6
6	Claude Fable 5.1	0.9930	$40.59	65.5M	322k	8h 40m	241	9
7	Claude Opus 5.5	0.9897	$30.03	90.4M	459k	7h 8m	258	13
8	Claude Sonnet 5.5	0.9897	$29.45	110.7M	573k	6h 41m	259	4
9	Claude Fable 5.1	0.9895	$33.56	52.1M	309k	8h 2m	221	8
10	Qwen3.8-Max	0.9778	$23.74	79.2M	556k	12h 13m	428	14
11	GPT-6 Astra	0.9768	$52.93	33.2M	214k	1h 55m	200	5
12	DeepSeek V4 Flash Vision Exp	0.9597	$1.51	70.8M	347k	3h 23m	283	4
13	Claude Sonnet 5.5	0.9593	$30.61	116.2M	578k	14h 56m	275	2
14	Kimi K3	0.9593	$29.55	62.1M	341k	12h 49m	285	15
15	Claude Fable 5	0.9397	$59.14	36.3M	315k	5h 54m	162	8
16	Claude Fable 5	0.9375	$84.70	58.1M	370k	13h 38m	228	14
17	Claude Opus 5.5	0.9370	$28.53	84.2M	453k	7h 52m	260	12
18	Qwen3.8-Max-0902	0.9183	$45.24	139.4M	1.5M	19h 44m	428	1
19	Grok 4.6	0.9143	$37.23	34.8M	407k	3h 22m	158	10
20	Muse Spark 1.2	0.9041	$8.87	38.4M	436k	2h 57m	150	1
21	Grok 4.6	0.8991	$42.77	39.2M	529k	3h 38m	183	9
22	Claude Sonnet 5.5	0.8984	$37.57	147.8M	629k	11h 9m	314	4
23	Claude Haiku 5.5	0.8970	$5.84	85.5M	559k	11h 56m	222	2
24	Grok 4.6	0.8827	$39.92	37.7M	462k	3h 38m	181	8
25	Inkling	0.8760	$6.24	6.3M	65k	37m	140	1
26	Claude Haiku 5.5	0.8579	$4.65	65.6M	502k	5h 43m	188	2
27	Muse Spark 1.3	0.8206	$37.73	59.2M	320k	3h 38m	300	12
28	Grok 4.6	0.8188	$274.70	264.2M	1.9M	19h 38m	1078	67
29	GPT-5.6 Sol	0.7863	$99.90	97.4M	228k	4h 26m	327	20
30	Grok 4.7	0.7706	$11.93	13.0M	204k	1h 11m	90	5
31	Qwen3.8-Max	0.7685	$16.66	51.1M	571k	13h 28m	342	5
32	Muse Spark 1.3	0.7474	$18.67	43.3M	220k	4h 23m	250	6
33	DeepSeek V4 Flash Vision Exp	0.7308	$2.17	110.7M	423k	3h 32m	374	9
34	Muse Spark 1.2	0.7187	$4.40	15.5M	216k	1h 29m	110	2
35	Muse Spark 1.2	0.7132	$3.96	18.1M	225k	49m	126	4
36	Qwen3.8-Max-0902	0.5260	$31.31	94.3M	1.1M	19h 31m	341	1
37	Claude Sonnet 5.5	0.4600	$33.95	128.3M	649k	6h 32m	272	3
38	Gemini 3.8 Flash	0.4507	$3.63	26.5M	187k	49m	177	3
39	Claude Opus 5	0.4324	$63.20	91.3M	476k	9h 29m	267	11
40	GPT-6 Astra	0.4310	$132.03	68.3M	275k	2h 40m	289	5
41	Qwen3.8-Max	0.4289	$16.21	49.4M	555k	8h 58m	285	8
42	Kimi K3	0.4200	$32.18	58.0M	281k	15h 50m	280	8
43	Claude Opus 5	0.3824	$73.86	111.1M	500k	10h 52m	316	11
44	Muse Spark 1.3	0.3726	$625.66	665.8M	679k	19h 55m	1128	223
45	Muse Spark 1.3	0.3641	$40.72	52.7M	332k	3h 49m	257	8
46	Qwen3.8-Max	0.3449	$15.61	47.3M	543k	7h 37m	278	8
47	Gemini 4 Argon	0.3435	$96.85	59.0M	525k	6h 34m	155	2
48	Qwen3.8-Max-0902	0.3378	$32.65	97.4M	1.2M	19h 38m	376	1
49	Claude Fable 5.1	0.3345	$30.45	45.8M	311k	4h 14m	174	15
50	Gemini 3.8 Flash	0.3266	$3.89	26.2M	153k	53m	177	3
51	Gemini 4 Argon	0.3199	$35.62	42.7M	359k	5h 46m	155	3
52	Qwen3.8-Max	0.3078	$18.56	57.9M	578k	8h 28m	322	6
53	Claude Opus 5	0.3011	$69.90	100.7M	538k	8h 25m	268	5
54	Claude Opus 5.5	0.2834	$29.08	88.4M	439k	9h 41m	265	10
55	Claude Opus 5.5	0.2805	$36.69	117.7M	504k	7h 47m	310	12
56	Claude Sonnet 5.5	0.2521	$37.68	144.7M	683k	9h 35m	297	3
57	Claude Fable 5	0.2381	$63.35	41.3M	309k	8h 51m	197	11
58	Gemini 4 Argon	0.2379	$42.74	116.3M	781k	7h 14m	281	4
59	Claude Fable 5	0.2345	$86.58	63.1M	319k	16h 11m	264	7
60	Gemini 3.8 Flash	0.2264	$4.42	36.8M	191k	1h 14m	202	5
61	Claude Haiku 5.5	0.2175	$5.41	75.1M	593k	4h 10m	203	2
62	Gemini 4 Argon	0.2135	$100.44	83.2M	842k	7h 14m	239	4
63	Grok 4.6	0.2104	$116.88	112.5M	824k	9h 43m	470	52
64	GPT-5.6 Sol	0.2084	$52.26	56.6M	152k	2h 42m	244	5
65	GPT-6 Astra	0.2020	$84.84	43.9M	239k	1h 53m	208	6
66	Claude Fable 5.1	0.2017	$30.51	47.2M	284k	6h 48m	194	9
67	Qwen3.8-Max-0902	0.1773	$37.51	112.2M	1.0M	19h 48m	461	6
68	Gemini 3.7 Flash	0.1681	$1.72	9.3M	135k	1h 3m	102	1
69	Claude Fable 5	0.1639	$81.13	55.6M	354k	15h 53m	224	11
70	GLM-5.3	0.1632	$24.11	84.1M	350k	8h 48m	310	8
71	Muse Spark 1.2	0.1494	$13.10	65.0M	597k	2h 52m	267	1
72	Claude Haiku 5.5	0.1334	$4.56	63.7M	502k	4h 40m	187	2
73	Gemini 3.7 Flash	0.1295	$1.98	10.2M	141k	50m	105	3
74	Gemini 4 Argon	0.1190	$36.03	53.9M	536k	3h 17m	162	3
75	Gemini 3.7 Flash	0.1075	$1.62	9.2M	146k	1h 7m	102	1
76	Claude Haiku 5.5	0.1032	$5.88	83.3M	608k	3h 20m	216	1
77	GLM-5.3	0.1005	$24.46	71.1M	367k	17h 31m	283	7
78	Kimi K3	0.0960	$32.35	77.4M	380k	13h 56m	311	15
79	Qwen3.8-Max-0902	0.0834	$33.12	98.6M	1.2M	19h 40m	371	1
80	Gemini 3.7 Flash	0.0798	$2.53	10.3M	161k	1h 26m	110	1
81	Claude Fable 5.1	0.0778	$29.88	44.4M	316k	2h 48m	183	13
82	GPT-5.6 Sol	0.0777	$44.33	50.5M	161k	1h 56m	229	6
83	DeepSeek V4 Flash Vision Exp	0.0718	$2.02	102.8M	397k	15h 29m	346	4
84	DeepSeek V4 Flash Vision Exp	0.0700	$1.50	70.3M	352k	6h 13m	290	6
85	Gemini 3.7 Flash	0.0647	$3.79	20.7M	194k	1h 41m	139	2
86	Kimi K3	0.0626	$34.08	68.6M	372k	14h 20m	275	12
87	DeepSeek V4 Flash Vision Exp	0.0559	$1.32	62.9M	305k	6h 32m	278	4
88	GLM-5.3	0.0517	$9.91	30.6M	263k	2h 10m	167	5
89	Muse Spark 1.2	0.0498	$2.47	8.0M	164k	1h 0m	83	2
90	GLM-5.3	0.0375	$14.94	49.5M	308k	3h 28m	223	2
91	Gemini 3.8 Flash	0.0340	$3.04	20.0M	136k	40m	164	4
92	Muse Spark 1.3	0.0300	$39.32	51.4M	265k	3h 54m	261	8
93	Kimi K3	0.0289	$90.08	230.0M	659k	20h 0m	555	1
94	GLM-5.3	0.0251	$17.68	60.0M	356k	3h 29m	246	3
95	GPT-5.6 Sol	0.0080	$56.44	60.4M	164k	6h 2m	259	12
96	Inkling	0.0078	$3.62	3.6M	51k	2h 38m	120	1
97	Gemini 3.8 Flash	0.0073	$2.13	14.4M	132k	40m	136	2
98	GPT-5.6 Sol	0.0049	$70.63	65.2M	124k	2h 29m	194	6
99	Inkling	0.0004	$3.37	3.3M	54k	1h 37m	104	1
100	Inkling	0.0004	$4.08	4.1M	54k	1h 12m	112	2
101	Grok 4.7	0.0000	$420.44	408.4M	739k	19h 20m	1293	18
102	Grok 4.7	0.0000	$7.84	8.3M	179k	58m	63	3
103	Inkling	0.0000	$2.47	2.5M	36k	1h 12m	106	1
104	GLM-5.3 Flash	0.0000	$3.36	53.3M	330k	10h 44m	230	6
105	GLM-5.3 Flash	0.0000	$3.57	84.9M	393k	10h 21m	284	2
106	Grok 4.7	0.0000	$17.21	17.6M	220k	1h 13m	99	5
107	GLM-5.3 Flash	0.0000	$2.05	35.6M	271k	7h 4m	202	3
108	Grok 4.7	0.0000	$9.29	10.1M	177k	1h 2m	76	2
109	GLM-5.3 Flash	0.0000	$1.70	38.2M	299k	3h 32m	183	4
110	GLM-5.3 Flash	0.0000	$4.90	95.4M	391k	15h 5m	312	7

Kolmogorov Audio Compression

task page →110 runs · indexed 2026-10-08

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	GPT-6 Astra	1.0000	$2526.29	1186.0M	2.6M	19h 52m	2748	60
2	GPT-6 Astra	1.0000	$2608.33	1211.7M	2.6M	19h 14m	2577	57
3	GPT-6 Astra	1.0000	$2728.80	1295.7M	2.6M	19h 47m	2920	59
4	GPT-6 Astra	1.0000	$1955.97	919.3M	1.8M	13h 34m	2031	27
5	Gemini 4 Argon	0.1051	$267.43	253.9M	1.3M	20h 43m	650	14
6	Claude Sonnet 5.5	0.0978	$108.60	441.2M	1.6M	19h 39m	1051	10
7	Claude Sonnet 5.5	0.0941	$68.80	277.4M	1.1M	19h 3m	695	1
8	Claude Sonnet 5.5	0.0934	$70.90	283.8M	1.1M	19h 20m	689	13
9	Claude Sonnet 5.5	0.0867	$89.81	366.1M	1.3M	19h 24m	779	15
10	Claude Opus 5.5	0.0856	$64.20	240.0M	620k	16h 10m	530	20
11	Claude Sonnet 5.5	0.0850	$134.99	552.5M	1.9M	17h 54m	1174	6
12	Claude Opus 5.5	0.0830	$83.18	303.7M	858k	18h 27m	759	19
13	Claude Opus 5.5	0.0789	$67.50	264.9M	554k	16h 28m	614	17
14	Gemini 4 Argon	0.0786	$189.34	268.5M	1.6M	19h 23m	661	14
15	Claude Opus 5.5	0.0783	$41.46	140.9M	515k	18h 1m	396	17
16	Claude Fable 5.1	0.0779	$45.19	77.6M	262k	17h 49m	334	16
17	Gemini 4 Argon	0.0759	$254.33	211.1M	1.1M	19h 34m	466	11
18	Claude Fable 5.1	0.0754	$52.35	88.6M	331k	12h 35m	339	10
19	Claude Fable 5.1	0.0711	$71.87	125.0M	385k	17h 43m	414	18
20	Claude Opus 5	0.0709	$49.79	73.0M	367k	18h 21m	287	9
21	Claude Opus 5	0.0708	$49.82	72.2M	372k	18h 28m	258	6
22	Claude Opus 5	0.0704	$79.01	124.4M	464k	18h 51m	366	13
23	GPT-6 Astra	0.0696	$683.33	329.9M	756k	19h 26m	879	16
24	Claude Fable 5.1	0.0695	$54.46	93.0M	327k	16h 46m	347	11
25	Claude Opus 5.5	0.0694	$57.79	216.0M	560k	18h 11m	500	14
26	Claude Fable 5	0.0684	$67.93	48.0M	274k	18h 50m	256	7
27	Claude Fable 5.1	0.0678	$47.98	81.1M	304k	16h 17m	333	10
28	Claude Haiku 5.5	0.0650	$9.87	161.3M	651k	11h 38m	367	1
29	Gemini 4 Argon	0.0648	$246.66	202.6M	896k	20h 43m	508	10
30	Claude Opus 5	0.0619	$46.80	66.5M	370k	18h 0m	250	9
31	Claude Fable 5	0.0592	$121.06	93.1M	386k	19h 25m	340	15
32	Claude Haiku 5.5	0.0578	$7.35	119.4M	510k	11h 49m	313	9
33	Claude Haiku 5.5	0.0569	$7.56	123.0M	501k	18h 46m	329	4
34	Claude Haiku 5.5	0.0566	$12.02	190.2M	959k	14h 47m	486	5
35	Claude Fable 5	0.0559	$85.54	62.1M	330k	17h 58m	277	11
36	Gemini 4 Argon	0.0474	$174.59	205.7M	1.1M	13h 22m	472	18
37	Grok 4.7	0.0462	$180.50	154.3M	515k	19h 45m	631	5
38	Claude Fable 5	0.0444	$66.84	47.5M	265k	18h 38m	234	5
39	GPT-5.6 Sol	0.0416	$183.22	165.9M	262k	15h 47m	471	12
40	Claude Haiku 5.5	0.0415	$8.31	137.0M	537k	14h 59m	362	2
41	GPT-5.6 Sol	0.0408	$43.32	47.2M	136k	19h 34m	249	10
42	Claude Fable 5	0.0392	$102.94	78.4M	342k	18h 53m	317	19
43	GLM-5.3	0.0369	$28.71	91.1M	330k	19h 38m	343	2
44	GLM-5.3	0.0349	$48.94	118.7M	352k	19h 6m	384	2
45	Grok 4.7	0.0329	$52.47	42.4M	229k	5h 56m	210	12
46	GLM-5.3 Flash	0.0316	$3.90	73.7M	369k	19h 44m	299	7
47	Grok 4.7	0.0313	$93.78	78.9M	256k	12h 4m	313	14
48	Kimi K3	0.0313	$38.45	102.2M	362k	20h 1m	397	10
49	GLM-5.3 Flash	0.0303	$10.53	223.0M	596k	19h 20m	502	15
50	GLM-5.3	0.0298	$35.18	91.1M	311k	19h 41m	366	13
51	GLM-5.3 Flash	0.0288	$2.96	83.8M	329k	18h 56m	365	2
52	GLM-5.3 Flash	0.0270	$4.30	96.9M	346k	19h 28m	348	3
53	DeepSeek V4 Flash Vision Exp	0.0268	$2.15	112.2M	397k	7h 43m	389	3
54	Kimi K3	0.0266	$26.15	60.2M	321k	19h 29m	303	13
55	GLM-5.3	0.0264	$29.01	75.1M	335k	17h 41m	311	6
56	Kimi K3	0.0251	$50.26	120.4M	460k	19h 45m	410	7
57	Kimi K3	0.0250	$34.53	93.2M	354k	19h 44m	395	8
58	Muse Spark 1.3	0.0247	$73.55	80.1M	351k	10h 48m	392	7
59	GLM-5.3	0.0243	$29.39	96.9M	323k	19h 43m	356	1
60	Gemini 3.7 Flash	0.0240	$13.44	120.0M	296k	7h 42m	465	7
61	Gemini 3.7 Flash	0.0232	$21.26	112.5M	588k	9h 45m	340	20
62	DeepSeek V4 Flash Vision Exp	0.0228	$5.13	304.7M	583k	13h 30m	712	7
63	GPT-5.6 Sol	0.0224	$49.42	77.2M	169k	19h 53m	422	11
64	Grok 4.6	0.0221	$127.10	131.5M	617k	19h 58m	617	30
65	Grok 4.7	0.0218	$33.81	32.7M	228k	2h 52m	171	8
66	Muse Spark 1.3	0.0215	$70.26	70.8M	222k	11h 3m	366	12
67	Muse Spark 1.3	0.0200	$64.21	65.3M	198k	19h 54m	404	5
68	Grok 4.6	0.0192	$168.00	174.0M	498k	19h 58m	768	68
69	Muse Spark 1.3	0.0192	$39.49	134.3M	306k	19h 56m	568	3
70	Gemini 3.7 Flash	0.0189	$26.85	83.8M	686k	9h 43m	267	16
71	Qwen3.8-Max-0902	0.0172	$73.87	221.2M	2.8M	19h 53m	570	2
72	Grok 4.6	0.0171	$114.81	118.3M	558k	19h 40m	599	47
73	Qwen3.8-Max	0.0171	$21.55	69.7M	606k	19h 26m	446	15
74	GPT-5.6 Sol	0.0163	$47.54	55.3M	185k	19h 43m	291	10
75	GPT-5.6 Sol	0.0159	$41.60	61.4M	164k	18h 50m	368	11
76	Qwen3.8-Max	0.0155	$18.80	60.7M	524k	19h 36m	400	4
77	Gemini 3.8 Flash	0.0140	$7.34	46.3M	196k	3h 13m	274	6
78	Qwen3.8-Max-0902	0.0136	$47.03	132.4M	2.1M	19h 23m	432	2
79	Grok 4.7	0.0128	$49.77	44.3M	205k	7h 40m	231	6
80	Muse Spark 1.2	0.0126	$49.13	229.2M	341k	8h 52m	575	6
81	Grok 4.6	0.0126	$162.44	156.4M	577k	19h 37m	634	61
82	Muse Spark 1.3	0.0124	$64.76	58.2M	224k	18h 13m	384	11
83	Kimi K3	0.0120	$62.95	172.7M	576k	20h 0m	473	15
84	Qwen3.8-Max	0.0118	$16.99	53.1M	556k	19h 0m	415	2
85	Gemini 3.7 Flash	0.0110	$6.63	19.2M	229k	5h 4m	166	5
86	Grok 4.6	0.0106	$206.03	201.8M	768k	19h 59m	845	63
87	Claude Opus 5	0.0105	$138.91	229.6M	656k	19h 23m	449	9
88	Muse Spark 1.2	0.0104	$27.73	109.9M	451k	7h 13m	358	10
89	Gemini 3.8 Flash	0.0102	$13.79	78.6M	180k	10h 32m	411	11
90	Gemini 3.8 Flash	0.0100	$7.64	32.7M	97k	6h 4m	255	5
91	Qwen3.8-Max	0.0098	$20.15	54.4M	994k	19h 8m	321	6
92	Gemini 3.8 Flash	0.0097	$6.87	33.3M	108k	7h 49m	251	7
93	Muse Spark 1.2	0.0097	$69.69	161.5M	457k	8h 41m	441	5
94	Gemini 3.7 Flash	0.0097	$3.47	13.3M	97k	5h 22m	153	4
95	DeepSeek V4 Flash Vision Exp	0.0094	$6.17	373.9M	627k	11h 53m	798	9
96	Gemini 3.8 Flash	0.0094	$15.15	81.6M	161k	9h 26m	425	11
97	Qwen3.8-Max	0.0089	$23.54	77.9M	588k	18h 20m	403	2
98	Qwen3.8-Max-0902	0.0087	$44.20	104.3M	2.8M	19h 20m	333	2
99	Muse Spark 1.2	0.0087	$13.25	53.2M	190k	6h 44m	295	3
100	DeepSeek V4 Flash Vision Exp	0.0081	$3.17	180.3M	434k	7h 52m	554	2
101	Qwen3.8-Max-0902	0.0080	$44.17	113.8M	2.4M	19h 47m	337	1
102	Inkling	0.0073	$7.83	8.2M	21k	1h 33m	141	2
103	Muse Spark 1.2	0.0065	$21.18	65.3M	269k	5h 22m	313	3
104	Inkling	0.0061	$5.43	5.5M	41k	1h 2m	147	2
105	Inkling	0.0061	$13.67	14.2M	40k	2h 24m	225	2
106	GLM-5.3 Flash	0.0056	$7.03	151.5M	495k	17h 26m	374	10
107	DeepSeek V4 Flash Vision Exp	0.0056	$3.93	222.7M	553k	14h 38m	530	7
108	Inkling	0.0039	$4.62	4.8M	24k	1h 6m	101	1
109	Inkling	0.0030	$5.70	5.9M	31k	2h 11m	145	1
110	Qwen3.8-Max-0902	0.0020	$43.94	119.8M	2.1M	19h 50m	371	2

Lean 4 Kernel Type Checker in Pascal

task page →110 runs · indexed 2026-10-08

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Sonnet 5.5	1.0000	$56.70	219.1M	1.0M	5h 58m	501	6
2	Claude Haiku 5.5	1.0000	$12.16	186.9M	1.0M	4h 28m	423	4
3	Claude Sonnet 5.5	1.0000	$55.34	209.5M	1.1M	7h 29m	541	6
4	Claude Sonnet 5.5	1.0000	$53.61	204.3M	990k	7h 10m	485	6
5	Claude Opus 5	1.0000	$174.84	280.2M	919k	15h 16m	714	20
6	Claude Opus 5.5	1.0000	$41.28	133.2M	560k	3h 31m	263	13
7	Claude Fable 5.1	1.0000	$34.17	49.9M	378k	2h 37m	164	6
8	Claude Opus 5	1.0000	$204.51	327.4M	1.1M	13h 23m	746	19
9	Gemini 4 Argon	1.0000	$45.68	47.8M	303k	2h 17m	135	2
10	Claude Opus 5.5	1.0000	$71.71	241.5M	889k	6h 55m	573	21
11	Claude Fable 5.1	1.0000	$45.30	70.2M	418k	4h 0m	200	9
12	Gemini 4 Argon	1.0000	$81.62	85.5M	322k	3h 41m	204	2
13	Claude Fable 5	1.0000	$203.81	160.6M	603k	9h 37m	335	11
14	Claude Opus 5.5	1.0000	$35.94	109.8M	528k	2h 59m	232	8
15	Gemini 4 Argon	1.0000	$56.56	92.0M	304k	3h 17m	203	2
16	Claude Haiku 5.5	1.0000	$12.93	195.5M	1.2M	5h 31m	459	5
17	Claude Sonnet 5.5	1.0000	$55.03	208.1M	1.1M	6h 24m	495	7
18	Claude Fable 5	1.0000	$201.80	156.2M	629k	7h 5m	319	12
19	Claude Opus 5.5	1.0000	$31.11	88.5M	510k	2h 27m	202	4
20	Claude Opus 5.5	1.0000	$37.66	116.3M	544k	3h 20m	243	8
21	Claude Fable 5.1	1.0000	$34.22	50.9M	360k	1h 59m	159	8
22	Claude Fable 5.1	1.0000	$43.08	67.7M	379k	4h 3m	206	11
23	Claude Fable 5.1	1.0000	$50.91	83.1M	385k	3h 9m	232	8
24	Claude Opus 5	0.9984	$103.75	161.1M	640k	5h 43m	315	4
25	Claude Opus 5	0.9841	$206.83	330.5M	1.1M	10h 19m	761	12
26	Claude Haiku 5.5	0.9841	$12.26	189.0M	1.0M	5h 12m	467	2
27	Claude Haiku 5.5	0.9841	$9.10	133.0M	903k	3h 32m	370	2
28	Claude Fable 5	0.9841	$232.48	180.5M	725k	7h 59m	313	8
29	Claude Sonnet 5.5	0.9841	$56.48	214.6M	1.1M	12h 59m	513	8
30	Claude Haiku 5.5	0.9841	$10.51	161.8M	906k	4h 31m	389	2
31	GPT-6 Astra	0.9807	$122.08	64.3M	290k	2h 50m	282	4
32	GPT-5.6 Sol	0.9807	$221.16	222.7M	429k	6h 13m	691	19
33	Gemini 4 Argon	0.9807	$13.24	39.9M	250k	1h 35m	117	2
34	GPT-6 Astra	0.9807	$62.98	37.7M	245k	1h 38m	203	5
35	Gemini 4 Argon	0.9807	$15.02	49.2M	292k	2h 22m	145	2
36	GPT-6 Astra	0.9807	$123.05	61.8M	300k	2h 8m	260	3
37	GPT-6 Astra	0.9791	$34.15	22.2M	179k	1h 14m	133	3
38	GPT-6 Astra	0.9791	$174.62	85.4M	285k	2h 50m	311	4
39	GPT-5.6 Sol	0.9791	$78.01	86.8M	272k	4h 32m	395	2
40	Muse Spark 1.3	0.9776	$411.77	410.4M	713k	19h 52m	799	2
41	GPT-5.6 Sol	0.9757	$207.31	199.1M	381k	7h 23m	620	10
42	Claude Opus 5	0.9635	$169.99	269.9M	940k	17h 12m	655	17
43	Grok 4.7	0.9589	$192.69	192.9M	802k	7h 19m	790	8
44	Grok 4.7	0.9461	$478.73	472.8M	1.7M	19h 19m	1887	5
45	Gemini 3.7 Flash	0.9448	$95.08	678.0M	1.4M	19h 49m	1660	1
46	Gemini 3.7 Flash	0.9361	$54.83	453.2M	1.3M	12h 43m	1224	1
47	GPT-5.6 Sol	0.9339	$340.15	337.0M	382k	8h 35m	789	11
48	Kimi K3	0.9198	$192.68	537.9M	1.4M	20h 0m	1218	1
49	Grok 4.7	0.9103	$351.92	346.3M	1.1M	20h 0m	1425	6
50	Grok 4.7	0.8682	$294.16	294.8M	1.1M	13h 7m	1229	6
51	GLM-5.3	0.8650	$139.71	481.8M	637k	18h 8m	859	6
52	GLM-5.3	0.8372	$111.12	401.3M	867k	19h 42m	904	10
53	Kimi K3	0.8313	$155.97	405.9M	1.5M	19h 47m	935	3
54	GPT-5.6 Sol	0.7874	$397.16	404.0M	396k	8h 54m	1134	7
55	Muse Spark 1.3	0.7421	$154.98	818.9M	1.3M	19h 49m	1805	2
56	Muse Spark 1.3	0.5609	$411.55	416.2M	856k	19h 58m	959	1
57	GLM-5.3 Flash	0.5457	$13.38	394.6M	1.0M	13h 39m	939	5
58	Kimi K3	0.5265	$147.65	387.7M	1.4M	20h 2m	894	1
59	GLM-5.3 Flash	0.5060	$14.37	406.2M	1.1M	16h 35m	1001	3
60	Muse Spark 1.3	0.5029	$384.24	384.5M	940k	19h 50m	937	1
61	Kimi K3	0.4229	$171.33	477.1M	1.3M	19h 55m	1130	12
62	GLM-5.3	0.4065	$133.87	385.6M	660k	19h 35m	677	3
63	Grok 4.6	0.3824	$105.13	105.0M	1.0M	6h 34m	490	2
64	DeepSeek V4 Flash Vision Exp	0.3599	$12.71	725.7M	1.6M	10h 38m	1614	4
65	Muse Spark 1.3	0.3530	$389.61	408.6M	860k	19h 59m	1015	1
66	DeepSeek V4 Flash Vision Exp	0.3164	$9.48	551.4M	1.2M	7h 58m	1228	8
67	GLM-5.3 Flash	0.3164	$17.89	532.1M	1.2M	17h 29m	1198	4
68	Kimi K3	0.2644	$151.43	423.8M	1.1M	20h 1m	928	1
69	DeepSeek V4 Flash Vision Exp	0.2513	$13.44	768.2M	1.8M	13h 43m	1760	7
70	GLM-5.3 Flash	0.2321	$15.97	465.4M	1.2M	13h 20m	1049	31
71	Qwen3.8-Max-0902	0.1653	$45.56	98.3M	3.3M	20h 1m	294	1
72	Gemini 3.7 Flash	0.1248	$74.81	557.5M	1.1M	14h 22m	1217	1
73	GLM-5.3	0.1056	$134.85	474.9M	964k	19h 7m	1105	3
74	DeepSeek V4 Flash Vision Exp	0.0143	$12.56	761.2M	1.2M	12h 33m	1627	35
75	Qwen3.8-Max-0902	0.0111	$40.06	77.6M	3.2M	20h 1m	244	1
76	Gemini 3.7 Flash	0.0087	$48.94	384.1M	939k	13h 20m	1118	1
77	Muse Spark 1.2	0.0000	$19.14	108.5M	529k	2h 29m	348	1
78	Qwen3.8-Max	0.0000	$14.14	28.2M	1.1M	19h 59m	169	1
79	Inkling	0.0000	$5.56	5.7M	37k	1h 16m	115	2
80	Inkling	0.0000	$5.46	5.6M	34k	55m	107	2
81	Muse Spark 1.2	0.0000	$21.09	53.6M	699k	3h 48m	205	2
82	Muse Spark 1.2	0.0000	$29.46	162.8M	830k	6h 1m	404	3
83	Qwen3.8-Max	0.0000	$8.31	10.6M	914k	19h 59m	109	2
84	Inkling	0.0000	$5.70	5.9M	22k	14m	80	2
85	Qwen3.8-Max-0902	0.0000	$35.73	44.7M	3.0M	20h 1m	312	1
86	Grok 4.6	0.0000	$114.14	110.5M	1.1M	7h 47m	492	4
87	Muse Spark 1.2	0.0000	$27.30	88.9M	859k	4h 46m	278	1
88	Grok 4.6	0.0000	$407.54	406.7M	2.0M	19h 47m	1527	137
89	Gemini 3.8 Flash	0.0000	$90.95	821.2M	1.4M	20h 0m	2038	1
90	Grok 4.6	0.0000	$250.10	240.1M	1.8M	13h 13m	970	43
91	Inkling	0.0000	$13.81	14.4M	43k	1h 26m	175	1
92	Qwen3.8-Max-0902	0.0000	$33.43	45.6M	3.5M	20h 1m	184	1
93	DeepSeek V4 Flash Vision Exp	0.0000	$11.64	694.6M	1.3M	11h 22m	1413	33
94	Claude Fable 5	0.0000	$288.50	223.6M	888k	18h 15m	536	13
95	Grok 4.6	0.0000	$233.84	217.8M	2.5M	19h 53m	929	30
96	Muse Spark 1.2	0.0000	$9.51	48.0M	382k	2h 35m	213	2
97	Grok 4.7	0.0000	$418.16	407.2M	1.6M	18h 42m	1640	7
98	Claude Fable 5	0.0000	$144.73	104.4M	563k	6h 16m	235	13
99	GLM-5.3 Flash	0.0000	$17.57	476.0M	1.4M	17h 22m	1023	5
100	Gemini 3.8 Flash	0.0000	$69.03	618.4M	870k	14h 14m	1477	3
101	Gemini 3.8 Flash	0.0000	$77.67	724.4M	1.0M	15h 44m	1676	2
102	Gemini 3.7 Flash	0.0000	$101.50	861.0M	1.8M	19h 58m	2092	1
103	Gemini 3.8 Flash	0.0000	$82.03	739.4M	1.1M	17h 33m	1797	1
104	Qwen3.8-Max	0.0000	$13.92	25.5M	1.2M	19h 56m	167	2
105	Gemini 3.8 Flash	0.0000	$66.46	629.7M	880k	14h 44m	1599	3
106	Qwen3.8-Max	0.0000	$19.68	45.8M	1.3M	20h 0m	211	1
107	GLM-5.3	0.0000	$113.59	400.7M	1.0M	18h 31m	962	5
108	Qwen3.8-Max	0.0000	$10.35	13.6M	1.1M	19h 57m	115	1
109	Qwen3.8-Max-0902	0.0000	$22.28	25.4M	2.5M	14h 13m	151	1
110	Inkling	0.0000	$6.56	6.8M	25k	12m	109	6

libexpat Optimization

task page →110 runs · indexed 2026-10-08

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Opus 5.5	0.8047	$233.16	850.8M	2.4M	15h 14m	1758	71
2	Claude Fable 5	0.7799	$660.65	527.2M	1.8M	19h 20m	1251	46
3	Claude Sonnet 5.5	0.7156	$145.32	582.0M	2.3M	15h 31m	1253	17
4	Claude Opus 5.5	0.7133	$179.28	606.1M	2.3M	14h 58m	1384	20
5	Claude Sonnet 5.5	0.7089	$104.14	409.0M	1.8M	9h 58m	905	8
6	Claude Opus 5.5	0.6801	$255.81	936.5M	2.6M	17h 42m	2064	87
7	Claude Sonnet 5.5	0.6732	$123.96	482.0M	2.2M	17h 39m	1064	21
8	Claude Sonnet 5.5	0.6432	$111.15	436.3M	1.9M	12h 32m	911	14
9	Gemini 4 Argon	0.6081	$51.02	180.3M	915k	5h 20m	388	8
10	Gemini 4 Argon	0.5881	$18.38	71.0M	446k	2h 16m	160	9
11	Gemini 4 Argon	0.5870	$86.35	125.1M	819k	4h 40m	340	4
12	Claude Fable 5.1	0.5674	$350.25	628.4M	1.5M	16h 3m	1456	82
13	Claude Sonnet 5.5	0.5673	$154.38	620.0M	2.4M	16h 46m	1326	6
14	Gemini 4 Argon	0.5630	$26.24	74.9M	691k	2h 55m	198	9
15	Claude Fable 5.1	0.5395	$313.80	554.2M	1.5M	11h 26m	1289	34
16	Claude Opus 5	0.5365	$422.53	667.4M	2.4M	15h 22m	1491	47
17	Claude Opus 5	0.5236	$756.50	1269.1M	3.2M	19h 11m	2743	94
18	Claude Opus 5	0.5183	$561.01	908.7M	2.8M	16h 42m	2090	66
19	Claude Opus 5	0.5083	$502.07	801.8M	2.6M	18h 34m	1852	60
20	Claude Opus 5	0.5030	$470.72	762.4M	2.3M	16h 43m	1656	45
21	Claude Fable 5.1	0.4885	$375.68	676.2M	1.5M	17h 3m	1505	50
22	Claude Opus 5.5	0.4775	$212.18	685.7M	2.9M	15h 26m	1659	63
23	Claude Opus 5.5	0.4760	$260.62	918.4M	2.9M	17h 55m	2043	64
24	Gemini 4 Argon	0.4643	$17.19	66.1M	478k	2h 17m	158	4
25	Claude Fable 5.1	0.4469	$289.07	503.7M	1.5M	13h 41m	1156	50
26	Claude Fable 5	0.4435	$891.44	742.6M	2.0M	15h 20m	1639	82
27	GPT-5.6 Sol	0.3847	$146.13	140.6M	311k	2h 25m	439	7
28	Claude Fable 5	0.3547	$517.42	421.8M	1.3M	18h 48m	902	36
29	Grok 4.7	0.3527	$861.01	886.1M	3.3M	19h 22m	3633	135
30	Claude Fable 5	0.3358	$480.59	387.1M	1.3M	13h 47m	845	26
31	Claude Haiku 5.5	0.3300	$46.90	739.9M	3.7M	9h 50m	1627	7
32	GPT-5.6 Sol	0.3225	$67.06	75.3M	223k	1h 44m	326	9
33	GPT-5.6 Sol	0.3025	$18.35	26.1M	127k	53m	144	4
34	GPT-6 Astra	0.2941	$510.01	240.5M	518k	4h 1m	542	7
35	Claude Fable 5.1	0.2880	$179.04	308.9M	1.0M	10h 29m	755	12
36	GPT-5.6 Sol	0.2868	$49.11	53.9M	193k	1h 32m	238	5
37	GPT-6 Astra	0.2776	$415.35	191.6M	517k	3h 32m	444	6
38	GPT-5.6 Sol	0.2751	$110.05	104.7M	243k	2h 24m	350	7
39	GPT-6 Astra	0.2744	$177.86	83.5M	336k	2h 10m	283	5
40	Claude Fable 5	0.2667	$556.53	463.3M	1.2M	18h 41m	912	25
41	Gemini 3.7 Flash	0.2399	$41.35	394.7M	960k	4h 30m	1009	6
42	GLM-5.3	0.2372	$181.79	673.8M	1.1M	8h 14m	1501	3
43	GLM-5.3	0.2303	$183.58	670.1M	1.1M	9h 50m	1460	18
44	Claude Haiku 5.5	0.2257	$53.36	858.1M	3.9M	16h 29m	1827	36
45	GLM-5.3	0.1823	$327.58	1209.3M	1.8M	18h 23m	2635	46
46	Kimi K3	0.1778	$238.01	679.3M	1.8M	19h 52m	1588	83
47	GLM-5.3	0.1583	$236.28	865.5M	1.6M	18h 53m	1991	34
48	Grok 4.7	0.1576	$386.43	383.9M	880k	5h 16m	1277	13
49	Gemini 3.7 Flash	0.1572	$41.30	358.7M	785k	6h 1m	899	5
50	Claude Haiku 5.5	0.1556	$46.19	751.2M	3.1M	15h 48m	1610	16
51	Gemini 3.7 Flash	0.1555	$24.69	216.5M	877k	2h 56m	624	2
52	GPT-6 Astra	0.1534	$13.19	6.7M	92k	53m	62	4
53	GPT-6 Astra	0.1516	$19.33	10.3M	134k	38m	78	3
54	Kimi K3	0.1513	$169.56	484.2M	1.2M	19h 34m	1155	16
55	Grok 4.6	0.1479	$150.76	148.7M	804k	4h 11m	580	26
56	Grok 4.6	0.1393	$108.10	108.1M	736k	4h 5m	435	11
57	Claude Haiku 5.5	0.1387	$35.67	574.8M	2.5M	9h 45m	1282	8
58	Kimi K3	0.1288	$173.14	509.0M	1.1M	13h 23m	1149	28
59	Gemini 3.8 Flash	0.1280	$22.80	229.1M	416k	2h 32m	541	2
60	Gemini 3.8 Flash	0.1047	$29.07	295.3M	541k	5h 49m	791	3
61	Grok 4.6	0.1022	$79.42	75.8M	516k	2h 56m	273	7
62	Muse Spark 1.3	0.0996	$565.41	641.8M	1.1M	19h 58m	1422	16
63	Gemini 3.8 Flash	0.0991	$27.80	282.7M	663k	2h 44m	771	17
64	DeepSeek V4 Flash Vision Exp	0.0891	$17.04	1034.3M	1.6M	6h 42m	2273	6
65	Claude Haiku 5.5	0.0877	$18.92	306.6M	1.3M	3h 20m	649	8
66	Muse Spark 1.3	0.0752	$635.38	703.6M	636k	19h 52m	1342	3
67	Gemini 3.8 Flash	0.0565	$31.04	299.7M	611k	3h 45m	818	4
68	DeepSeek V4 Flash Vision Exp	0.0545	$16.46	992.0M	1.7M	6h 43m	2222	11
69	Muse Spark 1.3	0.0491	$376.25	2119.7M	1.7M	19h 47m	4639	144
70	Muse Spark 1.3	0.0092	$609.32	704.0M	1.1M	19h 57m	1497	2
71	Qwen3.8-Max-0902	0.0000	$53.45	131.0M	3.2M	20h 1m	378	1
72	Muse Spark 1.2	0.0000	$40.00	150.5M	651k	3h 23m	388	3
73	Muse Spark 1.2	0.0000	$41.66	132.7M	956k	3h 49m	473	3
74	Inkling	0.0000	$23.47	24.5M	46k	39m	359	2
75	Muse Spark 1.2	0.0000	$16.54	87.9M	378k	1h 23m	309	3
76	Qwen3.8-Max	0.0000	$21.15	36.8M	1.9M	20h 0m	260	—
77	GLM-5.3 Flash	0.0000	$22.01	646.4M	1.1M	19h 18m	1514	1
78	GLM-5.3 Flash	0.0000	$23.47	725.1M	1.2M	15h 11m	1632	14
79	Inkling	0.0000	$3.70	3.7M	39k	18m	85	2
80	Qwen3.8-Max	0.0000	$16.28	22.3M	1.7M	19h 46m	202	—
81	Gemini 3.7 Flash	0.0000	$33.92	328.7M	960k	2h 58m	810	20
82	Qwen3.8-Max-0902	0.0000	$42.48	90.3M	3.1M	19h 53m	348	4
83	DeepSeek V4 Flash Vision Exp	0.0000	$13.51	827.8M	1.3M	5h 43m	1720	13
84	DeepSeek V4 Flash Vision Exp	0.0000	$21.93	1323.6M	2.2M	8h 50m	3095	7
85	Gemini 3.7 Flash	0.0000	$29.56	325.1M	599k	2h 10m	830	2
86	GLM-5.3 Flash	0.0000	$28.19	886.8M	1.2M	17h 12m	2083	25
87	DeepSeek V4 Flash Vision Exp	0.0000	$33.36	1965.0M	3.8M	19h 31m	4102	48
88	Kimi K3	0.0000	$214.44	584.8M	1.8M	20h 0m	1459	1
89	Inkling	0.0000	$18.36	19.0M	75k	47m	251	4
90	GLM-5.3 Flash	0.0000	$25.76	811.2M	1.4M	20h 0m	2421	7
91	Grok 4.7	0.0000	$77.71	79.6M	493k	2h 16m	325	4
92	Grok 4.7	0.0000	$35.69	33.2M	254k	1h 17m	139	5
93	Qwen3.8-Max	0.0000	$19.23	33.9M	1.7M	19h 53m	233	1
94	Qwen3.8-Max-0902	0.0000	$36.15	60.0M	3.4M	20h 0m	280	3
95	Grok 4.6	0.0000	$93.77	99.5M	677k	3h 26m	435	10
96	Muse Spark 1.2	0.0000	$39.83	172.4M	582k	3h 18m	559	6
97	Qwen3.8-Max-0902	0.0000	$95.18	281.1M	3.9M	19h 0m	632	3
98	Inkling	0.0000	$3.58	3.6M	31k	16m	74	2
99	Qwen3.8-Max	0.0000	$25.12	46.8M	2.2M	19h 55m	301	—
100	Gemini 3.8 Flash	0.0000	$35.42	341.5M	629k	6h 5m	907	2
101	Qwen3.8-Max-0902	0.0000	$38.42	73.2M	3.2M	20h 0m	340	2
102	Muse Spark 1.2	0.0000	$32.92	157.9M	806k	3h 52m	344	3
103	Grok 4.6	0.0000	$71.78	70.3M	399k	1h 58m	277	5
104	GLM-5.3 Flash	0.0000	$21.51	630.1M	971k	19h 59m	1458	2
105	Qwen3.8-Max	0.0000	$13.06	14.1M	1.6M	19h 35m	156	—
106	Inkling	0.0000	$3.11	3.2M	24k	13m	82	3
107	Kimi K3	0.0000	$256.75	739.1M	1.4M	20h 0m	1576	1
108	Muse Spark 1.3	0.0000	$571.21	716.3M	974k	19h 59m	1683	2
109	GLM-5.3	0.0000	$216.67	774.3M	1.6M	16h 7m	1754	19
110	Grok 4.7	0.0000†	$83.03	79.6M	516k	2h 36m	314	13

† score adjusted by manual review; the trace's Judge tab carries the measured score and the rationale.

Lua Native Compiler

task page →110 runs · indexed 2026-10-08

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Gemini 4 Argon	1.0000	$73.29	90.2M	317k	3h 26m	212	2
2	Claude Sonnet 5.5	1.0000	$63.17	240.2M	1.2M	8h 33m	574	4
3	Claude Fable 5.1	1.0000	$44.19	69.8M	381k	3h 24m	215	8
4	Claude Fable 5.1	1.0000	$51.03	82.5M	402k	3h 56m	245	12
5	Claude Sonnet 5.5	1.0000	$62.32	242.2M	1.1M	9h 42m	620	10
6	GPT-6 Astra	1.0000	$46.19	26.4M	213k	1h 23m	134	7
7	Claude Sonnet 5.5	1.0000	$65.89	255.0M	1.2M	8h 21m	630	3
8	Claude Fable 5	1.0000	$196.16	153.6M	590k	6h 11m	351	6
9	Claude Haiku 5.5	1.0000	$21.88	349.5M	1.7M	12h 38m	805	16
10	Claude Opus 5	1.0000	$209.27	343.2M	992k	13h 38m	826	10
11	Claude Opus 5	1.0000	$125.36	207.5M	580k	6h 45m	426	9
12	Claude Fable 5.1	1.0000	$55.84	88.8M	470k	3h 14m	235	6
13	Claude Haiku 5.5	1.0000	$14.18	224.8M	1.1M	5h 49m	554	3
14	Claude Opus 5	1.0000	$155.19	264.1M	628k	9h 49m	494	11
15	Claude Sonnet 5.5	1.0000	$84.18	344.6M	1.2M	16h 46m	746	14
16	GPT-6 Astra	1.0000	$35.97	23.6M	186k	1h 21m	133	4
17	GPT-6 Astra	1.0000	$131.76	64.1M	279k	2h 36m	232	7
18	Claude Opus 5	0.9992	$107.50	172.4M	579k	5h 52m	375	4
19	Claude Opus 5	0.9992	$128.06	213.3M	585k	7h 58m	454	2
20	Gemini 4 Argon	0.9976	$24.66	60.9M	275k	2h 36m	137	2
21	Gemini 4 Argon	0.9976	$34.72	50.2M	259k	2h 22m	136	2
22	Gemini 4 Argon	0.9976	$36.93	59.8M	371k	3h 58m	154	2
23	GLM-5.3	0.9976	$201.34	735.2M	1.4M	11h 37m	1660	3
24	GPT-6 Astra	0.9976	$79.59	42.6M	240k	2h 8m	195	9
25	Claude Haiku 5.5	0.9976	$23.67	398.7M	1.3M	8h 23m	819	7
26	Gemini 4 Argon	0.9976	$21.71	52.8M	265k	2h 10m	142	2
27	Muse Spark 1.3	0.9976	$451.94	515.6M	941k	16h 46m	1261	2
28	GPT-6 Astra	0.9976	$93.69	48.6M	237k	1h 56m	206	6
29	GLM-5.3	0.9953	$142.13	511.6M	1.2M	9h 22m	1178	2
30	DeepSeek V4 Flash Vision Exp	0.9788	$23.06	1383.4M	2.4M	13h 19m	2911	21
31	GLM-5.3	0.9599	$274.76	1000.1M	1.8M	19h 32m	2273	5
32	Kimi K3	0.9575	$183.51	524.3M	1.3M	19h 59m	1203	20
33	GLM-5.3	0.9527	$236.30	883.7M	1.1M	17h 20m	1835	7
34	Claude Haiku 5.5	0.9292	$10.74	177.0M	661k	2h 38m	328	1
35	Kimi K3	0.8750	$185.22	530.7M	1.3M	20h 0m	1207	1
36	Claude Fable 5.1	0.8463	$43.78	69.0M	381k	3h 39m	231	1
37	Claude Opus 5.5	0.7885	$68.38	213.8M	1.0M	9h 38m	482	11
38	Gemini 3.8 Flash	0.7382	$43.93	463.1M	839k	6h 53m	1136	1
39	Gemini 3.7 Flash	0.6998	$21.69	183.4M	580k	2h 58m	400	1
40	Claude Fable 5	0.6667	$216.90	168.1M	672k	7h 17m	337	5
41	Claude Fable 5	0.6667	$173.23	133.0M	557k	7h 50m	292	8
42	Claude Fable 5.1	0.6667	$57.38	92.1M	466k	4h 40m	248	7
43	Claude Haiku 5.5	0.6667	$17.99	286.4M	1.3M	10h 1m	661	4
44	Claude Opus 5.5	0.6667	$78.97	234.5M	1.3M	9h 24m	649	15
45	Gemini 3.7 Flash	0.6541	$62.39	519.3M	1.2M	7h 8m	1260	1
46	Gemini 3.7 Flash	0.6517	$26.08	255.3M	824k	1h 58m	670	1
47	Claude Opus 5.5	0.6447	$81.59	228.2M	1.4M	10h 19m	607	8
48	Gemini 3.7 Flash	0.6392	$29.13	278.2M	822k	3h 5m	751	1
49	Grok 4.6	0.6384	$176.45	167.1M	1.5M	7h 26m	653	8
50	Kimi K3	0.5063	$220.93	645.6M	1.2M	20h 2m	1461	1
51	Muse Spark 1.3	0.4969	$541.57	625.6M	1.2M	20h 0m	1447	1
52	Muse Spark 1.3	0.4002	$540.38	602.2M	792k	20h 0m	1400	1
53	GLM-5.3 Flash	0.3625	$33.26	1014.3M	1.2M	20h 1m	2002	1
54	Gemini 3.7 Flash	0.3270	$79.34	743.6M	1.7M	8h 42m	1815	1
55	Kimi K3	0.2862	$159.88	431.8M	1.4M	19h 11m	1009	1
56	Claude Fable 5	0.2123	$203.21	167.2M	496k	10h 45m	402	11
57	DeepSeek V4 Flash Vision Exp	0.2005	$21.27	1250.2M	2.4M	15h 16m	2676	17
58	GLM-5.3 Flash	0.1899	$23.36	672.0M	1.1M	19h 59m	1530	3
59	GLM-5.3 Flash	0.0339	$26.65	828.1M	1.4M	19h 57m	1984	2
60	Inkling	0.0000	$27.74	28.9M	72k	42m	300	1
61	Grok 4.7	0.0000	$74.24	76.4M	456k	2h 25m	373	1
62	Inkling	0.0000	$16.92	17.5M	67k	44m	221	1
63	Claude Sonnet 5.5	0.0000	$70.29	272.9M	1.3M	13h 50m	649	5
64	Grok 4.7	0.0000	$23.46	23.7M	161k	55m	121	1
65	Qwen3.8-Max	0.0000	$64.51	206.8M	2.0M	19h 8m	639	2
66	GLM-5.3 Flash	0.0000	$24.51	738.8M	1.3M	18h 5m	1545	29
67	DeepSeek V4 Flash Vision Exp	0.0000	$11.39	676.4M	1.3M	7h 45m	1507	15
68	Claude Fable 5	0.0000	$255.17	207.7M	659k	6h 43m	426	8
69	Grok 4.6	0.0000	$110.36	107.0M	986k	5h 11m	447	8
70	Muse Spark 1.3	0.0000†	$75.32	109.8M	406k	5h 42m	461	6
71	Inkling	0.0000	$48.62	50.6M	132k	1h 1m	367	2
72	GPT-5.6 Sol	0.0000	$35.07	44.0M	172k	2h 41m	211	10
73	Muse Spark 1.2	0.0000	$44.95	118.1M	616k	6h 24m	354	2
74	Qwen3.8-Max-0902	0.0000	$32.39	39.9M	3.6M	20h 0m	206	1
75	Kimi K3	0.0000	$228.55	670.7M	1.4M	19h 8m	1408	3
76	Qwen3.8-Max-0902	0.0000	$40.91	73.7M	3.6M	19h 57m	272	1
77	Inkling	0.0000	$14.81	15.3M	75k	53m	223	1
78	Claude Opus 5.5	0.0000	$124.07	397.7M	1.8M	12h 35m	893	28
79	GPT-5.6 Sol	0.0000†	$7.45	8.9M	65k	25m	65	3
80	GLM-5.3	0.0000	$251.10	909.6M	1.3M	19h 55m	1918	6
81	Gemini 3.8 Flash	0.0000†	$7.66	74.9M	188k	44m	304	2
82	Qwen3.8-Max	0.0000	$121.10	426.7M	2.2M	19h 59m	984	1
83	Claude Opus 5.5	0.0000	$93.26	296.3M	1.3M	8h 6m	673	12
84	Grok 4.6	0.0000	$271.55	264.4M	2.0M	9h 49m	1061	31
85	Muse Spark 1.2	0.0000	$10.77	39.0M	388k	1h 41m	164	4
86	DeepSeek V4 Flash Vision Exp	0.0000	$22.43	1385.4M	2.0M	11h 31m	2860	2
87	GLM-5.3 Flash	0.0000	$31.03	962.8M	1.5M	19h 47m	2160	10
88	Gemini 3.8 Flash	0.0000†	$9.10	88.8M	213k	1h 38m	328	2
89	Qwen3.8-Max	0.0000	$135.25	480.8M	2.3M	19h 7m	1081	2
90	Qwen3.8-Max	0.0000	$60.45	189.1M	2.1M	19h 48m	599	3
91	Grok 4.7	0.0000	$84.78	85.6M	413k	2h 11m	330	3
92	Qwen3.8-Max	0.0000	$31.00	87.6M	1.4M	13h 21m	385	6
93	Qwen3.8-Max-0902	0.0000	$31.88	33.4M	3.8M	20h 13m	209	1
94	Inkling	0.0000	$7.91	8.2M	23k	26m	165	2
95	Grok 4.6	0.0000	$676.54	663.0M	4.1M	19h 59m	2429	277
96	Gemini 3.8 Flash	0.0000†	$23.38	246.7M	422k	1h 50m	682	1
97	GPT-5.6 Sol	0.0000	$127.72	124.4M	217k	2h 5m	382	24
98	Muse Spark 1.2	0.0000	$4.46	22.6M	149k	52m	145	2
99	Muse Spark 1.3	0.0000	$637.70	724.8M	869k	20h 0m	1631	1
100	DeepSeek V4 Flash Vision Exp	0.0000	$18.34	1059.5M	2.2M	17h 1m	2383	9
101	Qwen3.8-Max-0902	0.0000	$34.76	50.3M	3.5M	20h 2m	251	1
102	GPT-5.6 Sol	0.0000	$24.84	37.1M	163k	1h 9m	219	13
103	Grok 4.6	0.0000	$250.76	240.6M	2.1M	11h 13m	930	16
104	Grok 4.7	0.0000	$10.18	12.5M	117k	39m	81	1
105	Muse Spark 1.2	0.0000	$22.61	124.4M	368k	1h 49m	323	2
106	Muse Spark 1.2	0.0000	$12.00	53.2M	328k	1h 52m	189	2
107	Grok 4.7	0.0000	$15.52	17.3M	144k	46m	103	1
108	Gemini 3.8 Flash	0.0000†	$32.38	312.3M	482k	4h 4m	832	1
109	Qwen3.8-Max-0902	0.0000	$91.51	260.0M	4.2M	19h 42m	578	1
110	GPT-5.6 Sol	0.0000	$20.50	39.9M	168k	1h 9m	231	7

† score adjusted by manual review; the trace's Judge tab carries the measured score and the rationale.

Machine-Learned Interatomic Potential

task page →110 runs · indexed 2026-10-08

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Sonnet 5.5	0.9988	$28.90	111.3M	520k	17h 22m	298	7
2	Claude Opus 5.5	0.9986	$19.92	56.8M	329k	17h 53m	217	9
3	Claude Sonnet 5.5	0.9974	$29.98	117.0M	513k	18h 55m	303	2
4	Gemini 4 Argon	0.9973	$144.92	105.5M	303k	19h 54m	273	10
5	Claude Haiku 5.5	0.9960	$3.07	44.7M	342k	18h 31m	181	3
6	Claude Haiku 5.5	0.9949	$5.38	85.9M	416k	15h 26m	274	3
7	Gemini 4 Argon	0.9944	$61.24	55.2M	244k	20h 19m	189	14
8	Gemini 4 Argon	0.9938	$60.34	45.2M	281k	20h 31m	167	13
9	Gemini 4 Argon	0.9931	$56.18	57.5M	250k	20h 47m	209	10
10	Qwen3.8-Max-0902	0.9925	$42.93	126.0M	1.3M	20h 0m	617	2
11	GPT-5.6 Sol	0.9925	$108.64	109.2M	151k	11h 16m	348	8
12	GPT-5.6 Sol	0.9918	$109.29	114.5M	152k	15h 47m	404	16
13	GLM-5.3	0.9901	$21.35	50.8M	160k	19h 5m	353	1
14	Claude Haiku 5.5	0.9898	$6.24	101.6M	445k	15h 29m	304	3
15	GPT-6 Astra	0.9884	$148.01	72.8M	245k	17h 13m	283	8
16	GPT-6 Astra	0.9884	$97.13	52.7M	210k	16h 8m	257	6
17	Qwen3.8-Max-0902	0.9860	$42.29	135.8M	786k	19h 35m	535	2
18	GPT-6 Astra	0.9842	$127.33	65.0M	249k	17h 10m	287	7
19	Gemini 4 Argon	0.9823	$48.43	49.9M	216k	20h 21m	199	20
20	Claude Opus 5	0.9772	$29.82	38.2M	293k	18h 29m	172	2
21	Muse Spark 1.3	0.9755	$18.59	16.6M	98k	19h 46m	219	3
22	Qwen3.8-Max	0.9731	$4.10	11.4M	180k	17h 54m	185	3
23	GPT-6 Astra	0.9702	$43.87	26.5M	154k	17h 20m	152	9
24	GPT-6 Astra	0.9688	$90.01	51.8M	262k	15h 58m	286	7
25	Claude Fable 5	0.9674	$40.40	26.4M	189k	18h 39m	181	7
26	Muse Spark 1.3	0.9671	$61.92	58.4M	240k	19h 36m	410	4
27	Claude Opus 5.5	0.9657	$21.32	60.2M	359k	15h 52m	219	7
28	GLM-5.3 Flash	0.9637	$1.52	38.6M	196k	15h 9m	253	2
29	GLM-5.3	0.9615	$15.62	37.8M	156k	19h 29m	294	2
30	Grok 4.7	0.9608	$60.45	36.2M	244k	15h 59m	187	2
31	Grok 4.7	0.9588	$76.24	57.2M	274k	16h 2m	238	12
32	Claude Fable 5.1	0.9586	$14.22	21.7M	140k	16h 34m	179	11
33	Claude Opus 5.5	0.9563	$17.15	48.3M	286k	18h 37m	196	8
34	Grok 4.7	0.9549	$72.03	45.6M	246k	19h 9m	215	7
35	Claude Opus 5	0.9539	$25.73	31.4M	274k	19h 1m	160	10
36	Muse Spark 1.3	0.9529	$5.47	10.6M	104k	19h 34m	177	2
37	Grok 4.7	0.9454	$102.72	67.6M	289k	18h 50m	254	8
38	GPT-5.6 Sol	0.9437	$26.20	41.3M	108k	14h 19m	237	3
39	Claude Fable 5.1	0.9369	$12.11	18.4M	119k	17h 11m	175	15
40	Kimi K3	0.9282	$14.81	21.6M	194k	19h 20m	175	3
41	Qwen3.8-Max-0902	0.9269	$37.45	125.1M	855k	19h 28m	494	1
42	Qwen3.8-Max	0.9220	$5.13	15.3M	180k	19h 14m	209	2
43	Claude Opus 5.5	0.9209	$19.00	56.2M	297k	18h 34m	231	12
44	GLM-5.3	0.9184	$31.31	62.0M	243k	18h 46m	307	2
45	Kimi K3	0.9177	$11.72	20.5M	161k	18h 47m	165	3
46	DeepSeek V4 Flash Vision Exp	0.9173	$3.12	179.4M	413k	18h 25m	558	6
47	Claude Opus 5	0.9168	$37.68	49.7M	354k	18h 11m	187	8
48	Claude Haiku 5.5	0.9161	$2.25	30.2M	316k	15h 29m	134	3
49	Qwen3.8-Max-0902	0.9150	$18.33	54.0M	689k	20h 5m	323	1
50	Claude Fable 5.1	0.9113	$11.26	16.5M	124k	17h 50m	150	5
51	Claude Fable 5	0.9086	$35.59	21.8M	190k	19h 5m	148	4
52	Gemini 3.8 Flash	0.9006	$5.19	42.9M	142k	3h 20m	258	6
53	Grok 4.6	0.8964	$27.94	18.3M	258k	19h 42m	147	2
54	Kimi K3	0.8916	$6.54	9.0M	95k	18h 28m	126	2
55	Muse Spark 1.2	0.8888	$4.08	20.1M	191k	2h 11m	165	12
56	Claude Fable 5	0.8825	$22.87	13.4M	129k	19h 3m	134	6
57	Claude Fable 5.1	0.8820	$17.05	26.5M	157k	16h 32m	179	7
58	Grok 4.6	0.8799	$71.42	58.5M	227k	19h 56m	307	9
59	Muse Spark 1.2	0.8642	$31.30	153.8M	371k	2h 47m	464	12
60	Gemini 3.8 Flash	0.8493	$4.81	37.8M	168k	3h 17m	232	5
61	Gemini 3.7 Flash	0.8422	$11.45	30.2M	159k	8h 5m	208	3
62	Muse Spark 1.3	0.8349	$17.65	15.6M	188k	19h 8m	174	1
63	Grok 4.6	0.8342	$80.88	55.9M	251k	19h 37m	265	10
64	Claude Fable 5.1	0.8229	$11.99	18.2M	119k	18h 9m	171	13
65	Muse Spark 1.2	0.8223	$3.45	15.9M	178k	49m	127	3
66	GLM-5.3	0.8163	$20.32	55.4M	207k	19h 30m	350	2
67	Qwen3.8-Max	0.8093	$6.94	20.8M	251k	18h 48m	231	2
68	Kimi K3	0.8033	$15.11	21.0M	176k	19h 11m	168	3
69	Claude Fable 5	0.8027	$24.76	14.6M	137k	19h 30m	133	7
70	GLM-5.3	0.7858	$25.28	49.5M	200k	18h 31m	285	2
71	Muse Spark 1.2	0.7533	$3.65	17.8M	164k	1h 53m	126	3
72	Kimi K3	0.7378	$9.88	13.2M	119k	19h 28m	143	2
73	Gemini 3.7 Flash	0.7285	$10.95	29.5M	154k	8h 16m	213	7
74	Grok 4.6	0.7267	$28.23	20.6M	259k	19h 44m	159	3
75	Muse Spark 1.2	0.7264	$3.55	17.0M	170k	2h 17m	144	3
76	Claude Haiku 5.5	0.6983	$3.46	51.0M	368k	17h 19m	192	1
77	DeepSeek V4 Flash Vision Exp	0.6906	$1.80	93.9M	332k	18h 0m	378	6
78	Qwen3.8-Max-0902	0.6529	$37.06	123.0M	877k	19h 28m	448	2
79	Claude Opus 5	0.5927	$30.96	39.3M	312k	17h 25m	184	3
80	Grok 4.6	0.5686	$61.59	46.3M	291k	18h 29m	250	4
81	Gemini 3.7 Flash	0.5113	$8.21	27.8M	178k	5h 29m	184	2
82	DeepSeek V4 Flash Vision Exp	0.4712	$1.05	54.4M	192k	19h 29m	342	2
83	Claude Opus 5.5	0.4697	$23.43	66.1M	395k	18h 40m	232	8
84	Gemini 3.7 Flash	0.4203	$5.25	21.8M	116k	4h 45m	178	1
85	Muse Spark 1.3	0.2516	$58.45	51.9M	245k	19h 22m	357	5
86	DeepSeek V4 Flash Vision Exp	0.2438	$1.59	84.5M	269k	19h 1m	389	5
87	Qwen3.8-Max	0.2071	$6.23	16.9M	295k	18h 26m	218	2
88	Gemini 3.8 Flash	0.0997	$3.73	23.7M	117k	2h 23m	194	6
89	Claude Sonnet 5.5	0.0768	$45.32	189.2M	581k	18h 26m	414	10
90	Inkling	0.0572	$3.46	3.5M	36k	2h 37m	136	1
91	DeepSeek V4 Flash Vision Exp	0.0562	$2.23	120.5M	374k	18h 41m	436	3
92	Gemini 3.8 Flash	0.0439	$5.15	42.3M	159k	2h 44m	260	3
93	Inkling	0.0431	$4.94	5.0M	59k	45m	120	1
94	Inkling	0.0393	$6.84	7.0M	43k	1h 26m	181	1
95	Claude Fable 5	0.0239	$24.85	14.9M	135k	19h 4m	143	7
96	Claude Sonnet 5.5	0.0201	$40.13	162.7M	594k	17h 49m	356	7
97	Inkling	0.0179	$5.35	5.4M	56k	1h 32m	126	2
98	Inkling	0.0126	$4.98	5.1M	42k	1h 33m	148	2
99	GPT-5.6 Sol	0.0076	$48.99	56.5M	122k	14h 29m	293	5
100	Claude Sonnet 5.5	0.0058	$42.44	171.8M	631k	18h 38m	372	1
101	GPT-5.6 Sol	0.0010	$36.55	49.8M	109k	7h 20m	272	3
102	Qwen3.8-Max	0.0010	$4.41	12.0M	206k	19h 18m	179	1
103	Gemini 3.7 Flash	0.0010	$6.01	20.4M	174k	4h 35m	154	2
104	Gemini 3.8 Flash	0.0009	$42.37	149.3M	250k	10h 22m	415	10
105	Claude Opus 5	0.0000	$33.24	46.7M	272k	18h 15m	233	6
106	GLM-5.3 Flash	0.0000	$1.89	27.2M	201k	18h 44m	197	3
107	Grok 4.7	0.0000	$87.65	66.0M	273k	12h 6m	260	15
108	GLM-5.3 Flash	0.0000	$1.76	24.4M	187k	19h 19m	185	2
109	GLM-5.3 Flash	0.0000	$2.29	40.6M	258k	17h 24m	221	4
110	GLM-5.3 Flash	0.0000	$1.92	27.6M	178k	19h 14m	216	5

Medium-Range Weather Forecast

task page →110 runs · indexed 2026-10-08

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Opus 5.5	0.6553	$21.31	71.4M	261k	19h 12m	313	10
2	Claude Sonnet 5.5	0.6529	$36.04	148.8M	488k	17h 33m	401	1
3	GPT-6 Astra	0.6474	$297.67	145.6M	335k	19h 47m	441	14
4	Claude Sonnet 5.5	0.6443	$32.66	133.0M	473k	13h 39m	382	9
5	Claude Opus 5.5	0.6405	$16.76	50.5M	249k	15h 23m	250	11
6	GPT-6 Astra	0.6387	$27.55	17.3M	152k	18h 9m	135	12
7	Claude Opus 5.5	0.6368	$21.64	71.5M	273k	18h 3m	288	10
8	GPT-6 Astra	0.6362	$98.77	53.1M	216k	17h 56m	257	14
9	GPT-6 Astra	0.6334	$244.67	123.1M	350k	18h 49m	445	13
10	Claude Opus 5	0.6321	$37.39	53.5M	286k	19h 19m	238	11
11	Claude Opus 5.5	0.6263	$14.11	42.0M	214k	18h 26m	225	7
12	Claude Fable 5.1	0.6214	$15.11	24.2M	125k	16h 58m	204	11
13	Grok 4.7	0.6200	$170.45	150.1M	788k	19h 40m	619	21
14	Claude Sonnet 5.5	0.6188	$32.99	135.6M	450k	18h 0m	394	14
15	Claude Fable 5.1	0.6178	$12.69	20.4M	104k	18h 1m	175	12
16	Claude Sonnet 5.5	0.6171	$43.64	183.7M	536k	18h 29m	447	5
17	Claude Opus 5.5	0.6141	$21.37	70.5M	271k	17h 20m	298	12
18	Claude Sonnet 5.5	0.6027	$41.54	172.7M	541k	18h 15m	423	9
19	Claude Opus 5	0.6013	$44.28	66.4M	300k	18h 19m	297	8
20	Claude Fable 5.1	0.6011	$16.81	27.4M	129k	16h 5m	208	16
21	Claude Fable 5.1	0.5960	$16.06	25.6M	135k	16h 36m	193	12
22	GPT-5.6 Sol	0.5936	$28.82	47.8M	113k	15h 13m	294	4
23	Grok 4.7	0.5877	$164.64	141.5M	437k	18h 53m	596	29
24	GLM-5.3	0.5873	$22.61	71.9M	228k	18h 20m	314	2
25	Claude Fable 5	0.5825	$47.76	32.3M	210k	18h 28m	199	10
26	Claude Opus 5	0.5817	$68.20	109.3M	364k	18h 25m	369	15
27	Grok 4.6	0.5804	$120.40	104.0M	444k	19h 13m	464	37
28	Gemini 3.8 Flash	0.5769	$6.44	56.5M	193k	2h 43m	275	11
29	Gemini 3.7 Flash	0.5747	$22.40	49.9M	145k	13h 34m	256	7
30	Gemini 4 Argon	0.5653	$75.42	53.6M	408k	7h 44m	162	9
31	Claude Haiku 5.5	0.5631	$7.09	116.4M	474k	17h 27m	332	9
32	GLM-5.3 Flash	0.5584	$3.61	51.8M	199k	18h 29m	284	2
33	Claude Haiku 5.5	0.5581	$5.34	84.1M	425k	18h 46m	272	10
34	Gemini 3.7 Flash	0.5566	$16.56	54.7M	160k	9h 58m	285	2
35	Grok 4.6	0.5522	$208.02	184.0M	911k	19h 55m	826	53
36	Grok 4.6	0.5495	$238.57	230.6M	626k	20h 0m	861	62
37	Claude Opus 5	0.5405	$40.40	59.3M	291k	18h 27m	261	16
38	Gemini 3.7 Flash	0.5365	$13.11	37.7M	147k	6h 21m	241	3
39	Gemini 3.7 Flash	0.5356	$19.36	48.1M	172k	8h 5m	252	4
40	Gemini 4 Argon	0.5347	$151.23	88.8M	463k	15h 22m	239	10
41	Grok 4.7	0.5339	$192.06	177.7M	824k	19h 10m	758	17
42	Gemini 3.7 Flash	0.5332	$4.99	21.4M	136k	4h 20m	171	1
43	Gemini 3.8 Flash	0.5330	$4.12	32.2M	140k	2h 31m	197	5
44	GLM-5.3 Flash	0.5292	$1.71	30.3M	154k	17h 56m	222	5
45	Gemini 3.8 Flash	0.5214	$13.54	89.8M	215k	6h 8m	390	14
46	Claude Fable 5	0.5192	$49.62	35.7M	184k	19h 9m	225	8
47	Claude Haiku 5.5	0.5171	$4.72	73.7M	409k	16h 51m	254	1
48	Qwen3.8-Max-0902	0.5127	$30.58	103.4M	640k	19h 23m	449	1
49	Claude Fable 5	0.5126	$51.69	34.8M	233k	18h 40m	219	14
50	Gemini 3.8 Flash	0.4927	$3.71	30.1M	141k	2h 9m	183	2
51	Claude Haiku 5.5	0.4915	$6.89	111.9M	491k	13h 38m	328	3
52	Claude Fable 5	0.4845	$59.30	44.0M	208k	17h 47m	259	6
53	Claude Fable 5	0.4839	$57.20	42.6M	198k	18h 53m	258	9
54	Gemini 4 Argon	0.4825	$53.28	50.9M	579k	8h 27m	159	4
55	GLM-5.3	0.4818	$19.75	47.5M	201k	18h 1m	273	5
56	GLM-5.3	0.4745	$28.03	96.3M	327k	19h 20m	363	7
57	GLM-5.3	0.4723	$21.03	47.8M	206k	18h 36m	293	2
58	Claude Haiku 5.5	0.4720	$4.54	74.1M	339k	17h 59m	276	1
59	Gemini 4 Argon	0.4704	$122.30	82.4M	652k	20h 41m	193	10
60	GLM-5.3	0.4679	$26.30	77.9M	252k	19h 29m	364	2
61	Muse Spark 1.3	0.4646	$17.40	32.8M	150k	19h 29m	250	11
62	Qwen3.8-Max-0902	0.4642	$26.04	87.1M	568k	18h 55m	401	2
63	DeepSeek V4 Flash Vision Exp	0.4639	$1.55	83.0M	253k	19h 5m	403	5
64	GLM-5.3 Flash	0.4629	$3.93	71.7M	248k	18h 47m	338	2
65	Grok 4.6	0.4530	$43.07	35.9M	245k	19h 35m	225	1
66	Qwen3.8-Max-0902	0.4423	$42.33	144.4M	856k	19h 32m	548	1
67	Grok 4.7	0.4365	$22.86	23.1M	216k	4h 0m	130	3
68	Muse Spark 1.3	0.4252	$39.97	63.7M	234k	18h 47m	401	10
69	Grok 4.7	0.4219	$34.89	31.9M	250k	3h 58m	135	11
70	Grok 4.6	0.4154	$92.48	66.7M	263k	19h 27m	303	6
71	Muse Spark 1.3	0.4057	$47.03	41.9M	228k	13h 41m	311	11
72	Qwen3.8-Max	0.4005	$8.94	29.1M	230k	19h 22m	281	1
73	Qwen3.8-Max	0.3989	$14.91	48.7M	387k	19h 25m	357	1
74	Muse Spark 1.3	0.3909	$44.60	41.6M	175k	17h 28m	283	7
75	Gemini 3.8 Flash	0.3886	$4.37	33.2M	181k	2h 4m	200	6
76	GLM-5.3 Flash	0.3763	$2.83	73.4M	262k	16h 13m	343	13
77	Qwen3.8-Max	0.3606	$13.12	44.7M	268k	19h 18m	385	3
78	GLM-5.3 Flash	0.3496	$2.98	85.3M	281k	19h 32m	394	4
79	Muse Spark 1.2	0.3456	$4.38	18.4M	145k	37m	104	2
80	Qwen3.8-Max-0902	0.3435	$56.59	197.4M	966k	18h 25m	518	6
81	Qwen3.8-Max	0.3410	$4.91	14.6M	172k	18h 25m	189	4
82	Qwen3.8-Max	0.3360	$8.01	25.7M	213k	18h 56m	239	8
83	Muse Spark 1.2	0.3227	$38.37	235.4M	385k	2h 26m	592	5
84	Muse Spark 1.3	0.3160	$38.63	81.9M	260k	18h 49m	434	13
85	Muse Spark 1.2	0.3018	$11.62	45.2M	202k	1h 9m	181	7
86	Muse Spark 1.2	0.2944	$2.12	8.7M	99k	50m	86	1
87	Muse Spark 1.2	0.2758	$11.17	51.5M	378k	2h 40m	211	6
88	Inkling	0.2568	$3.54	3.6M	37k	1h 43m	97	1
89	Inkling	0.2237	$3.91	3.9M	47k	1h 43m	120	1
90	Inkling	0.1542	$5.24	5.4M	39k	1h 50m	132	2
91	Inkling	0.1482	$3.49	3.6M	19k	1h 32m	102	1
92	GPT-5.6 Sol	0.0000	$122.35	137.1M	209k	17h 41m	576	7
93	GPT-6 Astra	0.0000	$488.12	238.0M	474k	17h 46m	610	16
94	Kimi K3	0.0000	$20.25	49.9M	257k	19h 31m	296	8
95	Kimi K3	0.0000	$16.03	42.3M	172k	19h 35m	282	1
96	Kimi K3	0.0000	$13.29	32.3M	193k	19h 22m	245	2
97	Gemini 4 Argon	0.0000	$44.59	35.1M	340k	6h 53m	144	5
98	Inkling	0.0000	$6.20	6.2M	72k	38m	132	1
99	DeepSeek V4 Flash Vision Exp	0.0000	$1.66	90.1M	258k	19h 29m	392	3
100	Claude Opus 5	0.0000	$33.36	46.8M	269k	19h 5m	229	5
101	DeepSeek V4 Flash Vision Exp	0.0000	$1.99	110.4M	289k	18h 13m	487	4
102	DeepSeek V4 Flash Vision Exp	0.0000	$1.55	84.2M	243k	18h 42m	398	5
103	GPT-5.6 Sol	0.0000	$63.62	70.4M	130k	17h 56m	290	3
104	Kimi K3	0.0000	$11.04	28.1M	135k	19h 31m	261	6
105	GPT-5.6 Sol	0.0000	$72.57	94.1M	189k	15h 53m	478	14
106	DeepSeek V4 Flash Vision Exp	0.0000	$1.68	93.8M	242k	19h 27m	452	2
107	Claude Fable 5.1	0.0000	$14.60	23.4M	121k	14h 28m	186	8
108	Qwen3.8-Max-0902	0.0000	$27.25	86.2M	783k	19h 31m	389	3
109	GPT-5.6 Sol	0.0000	$88.96	100.2M	138k	14h 52m	416	8
110	Kimi K3	0.0000	$19.09	48.5M	196k	18h 52m	326	6

MEG Speech Decoding

task page →103 runs · indexed 2026-10-08

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Sonnet 5.5	0.3980	$50.51	215.5M	568k	17h 35m	475	11
2	GPT-6 Astra	0.3919	$899.22	430.4M	993k	17h 31m	1064	15
3	Gemini 4 Argon	0.3815	$127.65	83.3M	493k	10h 58m	219	7
4	GPT-6 Astra	0.3729	$274.60	132.9M	344k	8h 16m	411	13
5	Claude Opus 5.5	0.3727	$24.06	81.6M	282k	17h 26m	303	14
6	GPT-6 Astra	0.3704	$154.28	76.5M	270k	4h 15m	297	9
7	Claude Opus 5.5	0.3678	$32.59	115.1M	354k	17h 32m	353	13
8	Claude Opus 5.5	0.3676	$34.48	123.3M	365k	14h 56m	393	16
9	Muse Spark 1.3	0.3657	$29.63	28.5M	123k	19h 14m	290	12
10	Claude Haiku 5.5	0.3592	$3.57	53.4M	356k	17h 25m	205	1
11	Claude Opus 5	0.3591	$41.25	57.4M	336k	16h 38m	244	11
12	Claude Haiku 5.5	0.3589	$6.16	99.7M	434k	15h 47m	312	1
13	GPT-6 Astra	0.3557	$452.78	217.0M	444k	15h 46m	527	14
14	Gemini 4 Argon	0.3541	$66.65	65.7M	370k	10h 16m	208	6
15	Gemini 4 Argon	0.3496	$73.61	73.2M	523k	11h 46m	215	9
16	GPT-6 Astra	0.3431	$526.36	253.0M	673k	17h 55m	694	20
17	Claude Sonnet 5.5	0.3387	$39.17	161.4M	527k	11h 58m	374	2
18	Claude Sonnet 5.5	0.3362	$42.30	176.9M	528k	15h 36m	407	2
19	Claude Sonnet 5.5	0.3295	$44.28	184.1M	576k	15h 25m	433	1
20	Gemini 4 Argon	0.3253	$89.40	63.7M	382k	14h 54m	195	7
21	Claude Haiku 5.5	0.3229	$5.52	86.2M	451k	9h 19m	265	3
22	Claude Opus 5	0.3182	$39.66	57.3M	295k	17h 37m	258	3
23	Claude Fable 5.1	0.3166	$19.44	30.2M	179k	16h 28m	214	8
24	Claude Fable 5.1	0.3123	$26.35	42.8M	204k	16h 51m	240	12
25	GPT-5.6 Sol	0.3119	$41.21	43.5M	133k	5h 23m	204	2
26	Claude Fable 5.1	0.3116	$13.99	21.5M	133k	17h 11m	192	11
27	Claude Sonnet 5.5	0.3091	$43.33	180.7M	552k	16h 17m	436	14
28	Claude Fable 5.1	0.3057	$16.57	26.2M	144k	13h 23m	202	11
29	Grok 4.6	0.3041	$128.46	122.4M	984k	15h 5m	545	23
30	GPT-5.6 Sol	0.3039	$14.10	19.5M	85k	2h 25m	137	2
31	Claude Opus 5	0.3001	$47.82	68.3M	371k	18h 4m	251	14
32	Claude Opus 5.5	0.2944	$25.47	87.0M	298k	19h 1m	298	14
33	Claude Fable 5	0.2869	$40.93	27.8M	172k	18h 49m	197	7
34	Claude Fable 5	0.2738	$32.57	21.0M	153k	17h 28m	176	7
35	Claude Haiku 5.5	0.2649	$5.00	78.8M	385k	13h 42m	250	1
36	Claude Haiku 5.5	0.2545	$5.81	88.4M	483k	16h 59m	251	6
37	Claude Fable 5	0.2462	$69.13	51.5M	236k	18h 13m	278	10
38	GPT-5.6 Sol	0.2311	$153.75	128.1M	180k	10h 45m	298	17
39	Grok 4.6	0.2265	$217.69	204.9M	1.2M	19h 25m	878	23
40	GLM-5.3	0.2244	$22.68	51.0M	188k	18h 6m	315	2
41	Claude Fable 5.1	0.2155	$25.98	41.7M	212k	14h 56m	237	10
42	Grok 4.6	0.2107	$132.86	129.3M	708k	19h 25m	571	33
43	GLM-5.3	0.2036	$34.96	74.2M	244k	14h 55m	372	2
44	GLM-5.3 Flash	0.2035	$2.39	39.2M	192k	18h 35m	253	3
45	Gemini 4 Argon	0.1941	$62.22	52.9M	318k	10h 43m	182	5
46	Claude Opus 5.5	0.1928	$24.23	78.6M	314k	16h 53m	281	9
47	GPT-5.6 Sol	0.1902	$64.35	67.7M	163k	7h 2m	279	4
48	Qwen3.8-Max-0902	0.1889	$35.75	119.7M	797k	19h 10m	470	2
49	Qwen3.8-Max-0902	0.1871	$50.73	177.9M	853k	19h 26m	592	1
50	GPT-5.6 Sol	0.1862	$104.80	98.3M	165k	7h 53m	318	8
51	Grok 4.6	0.1844	$95.16	83.3M	284k	18h 58m	362	11
52	DeepSeek V4 Flash Vision Exp	0.1749	$1.32	68.7M	240k	18h 36m	359	6
53	Grok 4.7	0.1697	$27.46	25.8M	199k	3h 25m	133	5
54	Muse Spark 1.2	0.1659	$17.10	36.3M	144k	3h 7m	130	6
55	GLM-5.3	0.1492	$27.34	90.5M	318k	15h 42m	371	3
56	Qwen3.8-Max	0.1460	$13.18	43.8M	302k	19h 37m	333	7
57	Muse Spark 1.3	0.1430	$33.98	30.2M	124k	18h 44m	278	4
58	Claude Fable 5	0.1418	$35.13	22.9M	161k	18h 17m	176	7
59	GLM-5.3 Flash	0.1356	$1.97	44.0M	221k	9h 31m	277	4
60	Muse Spark 1.3	0.1334	$8.50	30.2M	106k	18h 10m	317	7
61	Claude Opus 5	0.1306	$32.65	45.8M	261k	17h 4m	232	9
62	Claude Fable 5	0.1281	$60.54	43.7M	223k	17h 15m	249	11
63	Qwen3.8-Max-0902	0.1258	$56.59	193.9M	1.1M	18h 55m	561	4
64	Grok 4.6	0.1245	$71.38	67.2M	365k	9h 53m	293	10
65	GLM-5.3 Flash	0.1202	$2.02	38.5M	189k	18h 41m	272	4
66	Muse Spark 1.3	0.1187	$59.14	56.9M	180k	14h 50m	367	3
67	Muse Spark 1.2	0.1175	$9.52	47.7M	272k	3h 44m	183	8
68	Gemini 3.7 Flash	0.1130	$4.87	33.1M	132k	3h 51m	226	1
69	Gemini 3.7 Flash	0.1121	$12.12	51.3M	122k	7h 24m	312	4
70	Kimi K3	0.1102	$14.90	32.2M	173k	18h 22m	254	3
71	GLM-5.3 Flash	0.1039	$3.76	97.3M	271k	17h 33m	431	2
72	GLM-5.3	0.1032	$33.80	77.7M	262k	19h 7m	353	2
73	Gemini 3.8 Flash	0.1012	$4.22	32.1M	134k	3h 25m	220	3
74	Muse Spark 1.2	0.1011	$11.96	61.9M	415k	4h 37m	241	3
75	Qwen3.8-Max	0.0985	$20.47	70.2M	412k	18h 53m	478	2
76	Gemini 3.8 Flash	0.0958	$3.90	29.7M	119k	2h 37m	202	3
77	Gemini 3.7 Flash	0.0948	$9.93	38.7M	111k	7h 46m	266	5
78	Qwen3.8-Max-0902	0.0904	$20.15	52.8M	623k	19h 46m	279	2
79	Gemini 3.7 Flash	0.0854	$10.11	35.8M	110k	7h 53m	303	1
80	DeepSeek V4 Flash Vision Exp	0.0826	$1.73	93.5M	273k	12h 25m	457	2
81	Gemini 3.8 Flash	0.0714	$6.78	37.1M	131k	5h 53m	235	4
82	GLM-5.3	0.0000	$49.90	134.3M	340k	19h 4m	469	21
83	GLM-5.3 Flash	0.0000	$1.17	35.7M	150k	19h 11m	277	4
84	Muse Spark 1.3	0.0000	$32.86	29.1M	146k	18h 59m	269	6
85	Qwen3.8-Max	0.0000	$13.84	45.7M	331k	19h 21m	336	2
86	Inkling	0.0000	$5.29	5.5M	26k	1h 28m	151	1
87	Grok 4.7	0.0000	$17.01	16.9M	171k	3h 5m	110	2
88	Grok 4.7	0.0000	$33.74	29.5M	210k	5h 34m	157	5
89	Qwen3.8-Max	0.0000	$27.96	99.7M	412k	19h 32m	527	4
90	Muse Spark 1.2	0.0000	$13.11	77.0M	195k	5h 0m	202	3
91	Inkling	0.0000	$5.01	5.2M	28k	1h 29m	128	1
92	Qwen3.8-Max-0902	0.0000	$24.18	67.0M	860k	20h 1m	341	1
93	Gemini 3.7 Flash	0.0000	$12.20	48.7M	173k	7h 50m	285	2
94	Grok 4.7	0.0000	$9.82	11.5M	148k	1h 49m	85	2
95	Muse Spark 1.2	0.0000	$16.36	73.7M	275k	2h 42m	244	7
96	Gemini 3.8 Flash	0.0000	$6.01	39.8M	130k	3h 31m	254	7
97	Inkling	0.0000	$6.23	6.3M	53k	2h 6m	139	1
98	Gemini 3.8 Flash	0.0000	$6.66	41.6M	150k	3h 57m	251	6
99	Qwen3.8-Max	0.0000	$8.55	26.2M	281k	19h 56m	242	1
100	Grok 4.7	0.0000	$32.99	32.5M	199k	5h 31m	181	4
101	Inkling	0.0000	$5.31	5.5M	23k	2h 3m	134	2
102	Claude Opus 5	0.0000	$41.18	59.8M	304k	17h 59m	267	9
103	Inkling	0.0000	$5.42	5.6M	24k	1h 30m	116	1

MS/MS De Novo Generation

task page →100 runs · indexed 2026-10-08

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	GPT-6 Astra	0.1731	$851.54	402.0M	882k	16h 28m	928	15
2	GPT-6 Astra	0.1695	$1429.90	682.5M	1.4M	13h 9m	1631	15
3	GPT-6 Astra	0.1667	$818.15	390.6M	876k	18h 17m	988	21
4	GPT-6 Astra	0.1525	$440.96	211.9M	461k	8h 1m	546	7
5	Gemini 4 Argon	0.1524	$41.91	58.1M	431k	9h 28m	182	5
6	GPT-6 Astra	0.1519	$408.97	200.1M	427k	5h 57m	574	5
7	Claude Sonnet 5.5	0.1449	$60.59	259.5M	700k	17h 10m	500	4
8	Claude Opus 5.5	0.1412	$32.15	111.5M	374k	16h 42m	367	12
9	Claude Opus 5	0.1388	$55.40	79.2M	435k	18h 4m	247	7
10	Claude Opus 5.5	0.1378	$36.38	124.0M	438k	17h 32m	359	11
11	Claude Sonnet 5.5	0.1364	$65.26	259.5M	1.1M	17h 13m	616	11
12	Claude Opus 5	0.1349	$50.16	72.9M	377k	16h 47m	262	6
13	GPT-5.6 Sol	0.1316	$55.65	69.9M	170k	12h 29m	332	2
14	GPT-5.6 Sol	0.1312	$87.71	101.7M	199k	15h 19m	453	6
15	Gemini 4 Argon	0.1303	$43.39	66.9M	512k	10h 28m	195	7
16	Claude Opus 5	0.1301	$47.62	67.5M	384k	18h 51m	247	5
17	Claude Sonnet 5.5	0.1301	$50.58	209.6M	673k	13h 41m	408	10
18	Gemini 4 Argon	0.1252	$72.74	47.3M	391k	5h 42m	164	6
19	Claude Haiku 5.5	0.1246	$11.47	189.9M	707k	16h 52m	400	6
20	Claude Haiku 5.5	0.1232	$6.74	107.0M	520k	13h 56m	293	1
21	Muse Spark 1.3	0.1200	$62.04	57.0M	274k	16h 49m	329	3
22	Gemini 3.8 Flash	0.1144	$3.30	21.9M	158k	1h 16m	160	5
23	Claude Haiku 5.5	0.1135	$6.58	101.1M	551k	18h 27m	277	4
24	Muse Spark 1.3	0.1131	$41.90	45.4M	252k	18h 50m	304	7
25	Grok 4.7	0.1109	$12.56	12.3M	170k	1h 49m	85	2
26	Kimi K3	0.1101	$21.35	46.7M	229k	17h 35m	282	6
27	Grok 4.7	0.1098	$130.86	121.5M	481k	15h 47m	498	4
28	Kimi K3	0.1097	$26.90	62.8M	258k	19h 1m	325	7
29	Kimi K3	0.1091	$18.21	39.5M	209k	19h 3m	239	5
30	Muse Spark 1.3	0.1082	$55.03	49.7M	225k	19h 54m	361	1
31	Gemini 3.7 Flash	0.1064	$8.88	40.2M	172k	5h 7m	265	2
32	GLM-5.3	0.1050	$34.13	75.8M	305k	18h 42m	336	2
33	Muse Spark 1.3	0.1039	$52.80	46.7M	249k	19h 50m	351	6
34	GLM-5.3	0.1036	$36.94	94.3M	365k	18h 38m	359	3
35	Gemini 3.8 Flash	0.1027	$5.15	41.3M	171k	2h 11m	255	3
36	Grok 4.6	0.1023	$289.71	280.9M	1.3M	20h 0m	1109	91
37	Grok 4.7	0.0968	$55.01	43.9M	294k	8h 31m	190	6
38	Qwen3.8-Max-0902	0.0947	$60.29	210.7M	1.0M	19h 43m	575	1
39	GLM-5.3	0.0941	$44.44	143.9M	417k	17h 20m	450	8
40	Grok 4.7	0.0896	$57.29	42.5M	256k	5h 56m	203	6
41	Qwen3.8-Max	0.0879	$12.07	38.5M	344k	19h 44m	290	3
42	Gemini 3.7 Flash	0.0865	$5.25	20.8M	119k	3h 41m	172	1
43	Gemini 3.7 Flash	0.0846	$10.26	48.6M	211k	7h 0m	264	3
44	GLM-5.3 Flash	0.0784	$2.74	52.3M	271k	18h 56m	251	2
45	Grok 4.6	0.0709	$86.33	77.3M	437k	10h 44m	317	6
46	Qwen3.8-Max	0.0652	$23.10	77.1M	550k	18h 46m	436	2
47	Muse Spark 1.2	0.0557	$16.81	90.3M	275k	5h 28m	342	4
48	Muse Spark 1.2	0.0507	$33.64	201.0M	350k	2h 20m	488	3
49	Claude Opus 5.5	0.0000	$41.05	143.9M	474k	16h 43m	419	14
50	Inkling	0.0000	$4.78	4.9M	42k	32m	136	1
51	GLM-5.3	0.0000	$28.55	71.5M	295k	19h 12m	332	2
52	Claude Sonnet 5.5	0.0000	$60.75	240.5M	1.0M	19h 26m	568	3
53	Qwen3.8-Max-0902	0.0000	$71.57	251.7M	1.2M	18h 34m	626	3
54	GLM-5.3	0.0000	$43.34	125.4M	382k	18h 56m	463	2
55	Gemini 3.8 Flash	0.0000	$7.62	44.2M	197k	3h 25m	241	4
56	Claude Haiku 5.5	0.0000	$7.52	117.0M	598k	11h 15m	286	2
57	GPT-5.6 Sol	0.0000	$29.44	43.7M	172k	11h 11m	291	4
58	Inkling	0.0000	$9.60	9.7M	91k	1h 37m	187	1
59	Grok 4.7	0.0000	$47.01	35.3M	253k	6h 31m	169	4
60	Qwen3.8-Max	0.0000	$18.80	59.9M	558k	19h 4m	366	1
61	DeepSeek V4 Flash Vision Exp	0.0000	$1.89	103.3M	298k	19h 46m	435	3
62	DeepSeek V4 Flash Vision Exp	0.0000	$2.64	142.6M	438k	12h 9m	474	5
63	Claude Sonnet 5.5	0.0000	$49.78	205.3M	682k	14h 7m	417	8
64	Inkling	0.0000	$4.22	4.3M	40k	1h 20m	125	2
65	GLM-5.3 Flash	0.0000	$5.34	102.1M	341k	17h 52m	408	7
66	Muse Spark 1.2	0.0000	$8.11	42.2M	266k	2h 10m	169	2
67	Qwen3.8-Max	0.0000	$22.77	75.2M	580k	19h 13m	492	3
68	Qwen3.8-Max-0902	0.0000	$64.31	226.9M	1.0M	19h 20m	642	1
69	GLM-5.3 Flash	0.0000	$2.82	72.3M	274k	18h 58m	343	2
70	Claude Haiku 5.5	0.0000	$5.39	79.0M	529k	13h 40m	215	3
71	Grok 4.6	0.0000	$51.24	39.8M	426k	15h 47m	243	5
72	Claude Opus 5.5	0.0000	$25.87	82.9M	352k	17h 21m	296	8
73	GLM-5.3 Flash	0.0000	$2.38	59.1M	248k	15h 20m	309	5
74	Gemini 4 Argon	0.0000	$51.82	59.1M	521k	9h 41m	178	5
75	Gemini 3.8 Flash	0.0000	$7.44	46.3M	203k	3h 16m	253	4
76	Kimi K3	0.0000	$18.66	38.3M	209k	20h 0m	254	4
77	DeepSeek V4 Flash Vision Exp	0.0000	$2.54	140.9M	373k	17h 33m	473	5
78	Qwen3.8-Max-0902	0.0000	$63.28	220.8M	1.1M	19h 41m	571	2
79	Qwen3.8-Max	0.0000	$31.11	106.6M	635k	17h 29m	500	3
80	Inkling	0.0000	$9.21	9.4M	67k	2h 19m	235	1
81	GPT-5.6 Sol	0.0000	$115.28	137.3M	233k	11h 7m	603	10
82	Kimi K3	0.0000	$31.98	77.8M	334k	19h 25m	339	4
83	GLM-5.3 Flash	0.0000	$2.95	54.2M	261k	17h 55m	272	3
84	Claude Opus 5	0.0000	$35.03	47.7M	304k	18h 9m	206	7
85	Qwen3.8-Max-0902	0.0000	$57.59	198.5M	1.1M	19h 22m	561	3
86	DeepSeek V4 Flash Vision Exp	0.0000	$2.76	151.1M	447k	15h 16m	503	6
87	GPT-5.6 Sol	0.0000	$33.43	41.9M	177k	7h 37m	257	4
88	Gemini 3.7 Flash	0.0000	$6.93	28.3M	150k	3h 32m	203	4
89	Grok 4.6	0.0000	$192.75	185.2M	1.2M	14h 10m	766	64
90	Claude Opus 5	0.0000	$44.13	62.7M	352k	17h 5m	243	5
91	DeepSeek V4 Flash Vision Exp	0.0000	$2.75	153.9M	409k	18h 4m	510	2
92	Muse Spark 1.3	0.0000	$54.25	46.3M	268k	18h 59m	310	2
93	Grok 4.6	0.0000	$86.81	76.6M	468k	16h 53m	336	9
94	Gemini 3.7 Flash	0.0000	$15.58	62.6M	304k	13h 22m	285	2
95	Gemini 4 Argon	0.0000	$75.16	54.4M	420k	6h 54m	185	6
96	Muse Spark 1.2	0.0000	$7.49	39.5M	207k	4h 45m	218	2
97	Gemini 3.8 Flash	0.0000	$9.56	56.3M	191k	4h 7m	303	3
98	Muse Spark 1.2	0.0000	$9.64	33.9M	298k	3h 46m	194	5
99	Inkling	0.0000	$6.19	6.3M	42k	29m	120	1
100	Claude Opus 5.5	0.0000	$33.84	113.6M	428k	17h 31m	346	9

Multi-GPU Efficient Finetuning

task page →110 runs · indexed 2026-10-08

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Inkling	0.5238	$11.78	12.1M	59k	1h 35m	166	1
2	Claude Opus 5.5	0.4286	$18.55	51.6M	316k	17h 25m	201	4
3	Muse Spark 1.2	0.3810	$64.94	126.7M	662k	16h 40m	386	27
4	Claude Sonnet 5.5	0.3571	$29.39	111.1M	559k	18h 36m	264	2
5	DeepSeek V4 Flash Vision Exp	0.3333	$0.82	36.7M	207k	19h 58m	238	1
6	Kimi K3	0.3333	$19.15	18.0M	131k	19h 9m	167	6
7	Claude Opus 5.5	0.3333	$13.03	35.0M	224k	18h 31m	179	5
8	Claude Opus 5.5	0.3095	$18.49	50.1M	325k	18h 23m	192	5
9	Muse Spark 1.2	0.3095	$11.95	41.8M	358k	12h 44m	199	24
10	Muse Spark 1.3	0.3095	$25.50	21.9M	121k	19h 28m	197	2
11	Kimi K3	0.3095	$16.56	15.3M	148k	19h 10m	134	3
12	Gemini 4 Argon	0.3095	$59.08	60.8M	237k	16h 56m	198	5
13	Claude Fable 5	0.2857	$23.66	14.1M	128k	18h 18m	119	2
14	GLM-5.3	0.2857	$12.88	26.8M	144k	19h 0m	234	2
15	Claude Fable 5.1	0.2857	$5.65	7.6M	75k	17h 4m	98	2
16	Claude Opus 5.5	0.2857	$21.94	61.0M	375k	16h 29m	207	3
17	Qwen3.8-Max	0.2857	$6.42	17.1M	316k	18h 44m	213	5
18	Claude Fable 5.1	0.2857	$5.38	7.9M	60k	16h 16m	118	3
19	Claude Haiku 5.5	0.2857	$3.49	48.9M	402k	16h 53m	163	3
20	GLM-5.3 Flash	0.2857	$0.68	10.0M	99k	15h 21m	124	3
21	GLM-5.3	0.2857	$10.42	18.4M	127k	19h 10m	175	6
22	Claude Opus 5	0.2857	$20.67	27.0M	193k	18h 45m	177	5
23	Claude Haiku 5.5	0.2619	$1.78	24.5M	255k	17h 19m	135	2
24	Claude Opus 5.5	0.2619	$17.33	45.0M	317k	18h 13m	178	4
25	GLM-5.3 Flash	0.2619	$1.17	15.8M	143k	16h 53m	134	3
26	Claude Fable 5	0.2619	$17.72	9.8M	108k	18h 5m	106	4
27	Muse Spark 1.2	0.2619	$355.31	2070.9M	1.9M	15h 46m	5920	22
28	GLM-5.3 Flash	0.2619	$0.97	12.5M	122k	17h 37m	127	2
29	Claude Sonnet 5.5	0.2619	$34.13	135.6M	549k	19h 19m	338	2
30	Claude Fable 5	0.2619	$30.03	18.9M	149k	19h 3m	152	5
31	Claude Fable 5.1	0.2619	$6.12	8.6M	75k	18h 44m	112	4
32	Qwen3.8-Max	0.2619	$6.15	16.6M	296k	17h 38m	220	10
33	Claude Opus 5	0.2381	$22.41	28.9M	215k	18h 43m	177	2
34	GPT-5.6 Sol	0.2381	$19.33	23.5M	110k	15h 47m	167	3
35	Claude Fable 5	0.2381	$19.07	11.3M	104k	19h 17m	120	8
36	GLM-5.3 Flash	0.2381	$1.36	29.9M	170k	18h 53m	219	2
37	Claude Opus 5	0.2381	$21.91	28.5M	201k	18h 44m	180	5
38	Claude Fable 5	0.2381	$17.66	9.8M	106k	18h 38m	106	6
39	Claude Haiku 5.5	0.2143	$2.46	34.8M	304k	17h 30m	158	3
40	Gemini 4 Argon	0.2143	$71.77	65.5M	249k	18h 37m	200	8
41	Claude Fable 5.1	0.2143	$5.26	7.1M	71k	18h 19m	92	3
42	Gemini 3.8 Flash	0.2143	$4.91	23.2M	78k	4h 58m	186	6
43	GPT-6 Astra	0.2143	$155.59	78.1M	233k	19h 27m	299	6
44	Claude Opus 5	0.2143	$16.33	19.4M	178k	17h 33m	142	3
45	Gemini 3.7 Flash	0.2143	$10.24	29.6M	72k	9h 49m	181	31
46	Claude Haiku 5.5	0.1905	$3.24	45.3M	385k	19h 9m	157	1
47	GPT-6 Astra	0.1905	$229.70	113.1M	288k	19h 53m	364	6
48	Muse Spark 1.2	0.1905	$5.52	20.2M	214k	2h 15m	135	3
49	Gemini 4 Argon	0.1905	$119.59	80.8M	260k	19h 19m	230	3
50	GPT-6 Astra	0.1905	$125.10	64.1M	212k	19h 30m	261	6
51	Kimi K3	0.1905	$16.13	11.8M	128k	18h 47m	116	1
52	Gemini 3.7 Flash	0.1667	$11.85	31.1M	102k	19h 53m	202	10
53	Claude Sonnet 5.5	0.1667	$34.90	136.3M	593k	18h 28m	314	3
54	Claude Sonnet 5.5	0.1667	$37.45	150.3M	578k	14h 47m	343	3
55	Claude Fable 5.1	0.1667	$6.92	9.6M	88k	17h 1m	101	4
56	DeepSeek V4 Flash Vision Exp	0.1429	$0.92	41.7M	226k	19h 50m	251	2
57	Muse Spark 1.3	0.1429	$16.18	14.5M	230k	20h 0m	159	1
58	Gemini 4 Argon	0.1429	$59.35	68.3M	219k	18h 18m	207	7
59	Gemini 4 Argon	0.1190	$99.16	76.3M	304k	18h 34m	233	4
60	Gemini 3.8 Flash	0.0952	$10.19	37.4M	109k	8h 17m	227	11
61	GPT-6 Astra	0.0714	$291.48	141.3M	320k	19h 38m	402	6
62	Claude Haiku 5.5	0.0714	$4.82	71.2M	466k	18h 3m	219	1
63	GLM-5.3	0.0714	$17.61	33.2M	221k	18h 56m	211	2
64	Qwen3.8-Max-0902	0.0476	$21.69	51.3M	683k	18h 46m	334	3
65	Muse Spark 1.2	0.0476	$35.64	132.6M	581k	10h 53m	451	13
66	Qwen3.8-Max-0902	0.0476	$21.74	63.3M	874k	19h 49m	397	3
67	Kimi K3	0.0476	$25.13	35.1M	233k	19h 37m	236	1
68	Inkling	0.0476	$5.35	5.5M	24k	1h 26m	132	2
69	GPT-5.6 Sol	0.0238	$19.65	25.0M	71k	12h 53m	133	3
70	Inkling	0.0238	$14.22	14.8M	45k	2h 19m	219	1
71	DeepSeek V4 Flash Vision Exp	0.0238	$0.66	29.1M	168k	19h 41m	226	2
72	Grok 4.7	0.0000	$60.21	36.3M	189k	18h 34m	184	2
73	Muse Spark 1.3	0.0000	$22.96	20.6M	169k	19h 58m	157	5
74	Qwen3.8-Max	0.0000	$4.65	11.0M	287k	19h 49m	183	5
75	Gemini 3.7 Flash	0.0000	$22.18	47.8M	107k	18h 31m	299	2
76	Grok 4.7	0.0000	$37.89	24.1M	164k	18h 10m	140	2
77	GPT-5.6 Sol	0.0000	$16.45	25.9M	78k	16h 50m	180	2
78	GLM-5.3	0.0000	$20.26	34.9M	208k	19h 31m	211	4
79	Claude Opus 5	0.0000	$20.70	27.2M	184k	19h 13m	173	3
80	Inkling	0.0000	$6.66	6.8M	46k	2h 1m	135	1
81	GPT-5.6 Sol	0.0000	$24.13	34.7M	95k	16h 10m	181	2
82	GPT-5.6 Sol	0.0000	$30.41	39.1M	90k	14h 8m	199	3
83	Grok 4.6	0.0000	$47.03	50.4M	373k	19h 22m	261	13
84	Qwen3.8-Max	0.0000	$3.67	9.9M	171k	19h 13m	153	3
85	Qwen3.8-Max-0902	0.0000	$6.94	8.9M	369k	5h 21m	120	1
86	GLM-5.3	0.0000	$17.67	32.5M	199k	19h 29m	215	2
87	Muse Spark 1.3	0.0000	$12.91	12.2M	103k	19h 52m	142	1
88	DeepSeek V4 Flash Vision Exp	0.0000	$0.76	37.0M	151k	19h 59m	275	1
89	Gemini 3.8 Flash	0.0000	$11.92	58.4M	116k	9h 56m	301	7
90	Qwen3.8-Max	0.0000	$7.44	21.5M	284k	19h 29m	253	2
91	Grok 4.6	0.0000	$18.53	24.3M	327k	17h 14m	194	7
92	DeepSeek V4 Flash Vision Exp	0.0000	$1.23	60.8M	248k	19h 49m	323	1
93	Qwen3.8-Max-0902	0.0000	$21.73	60.3M	968k	19h 32m	366	3
94	Grok 4.6	0.0000	$32.88	28.8M	249k	18h 12m	196	5
95	Muse Spark 1.3	0.0000	$30.08	27.4M	216k	19h 49m	204	1
96	Gemini 3.7 Flash	0.0000	$8.30	27.1M	102k	9h 38m	188	10
97	GPT-6 Astra	0.0000	$142.95	74.8M	253k	19h 46m	311	7
98	Grok 4.6	0.0000	$23.95	18.0M	271k	19h 15m	145	4
99	Inkling	0.0000	$4.16	4.3M	15k	34m	107	3
100	Kimi K3	0.0000	$10.94	8.7M	101k	18h 40m	121	1
101	Gemini 3.8 Flash	0.0000	$9.46	51.9M	115k	6h 1m	268	6
102	Claude Sonnet 5.5	0.0000	$32.34	129.9M	492k	18h 23m	326	2
103	Grok 4.7	0.0000	$29.88	18.4M	159k	14h 59m	119	3
104	Gemini 3.7 Flash	0.0000	$30.31	73.0M	132k	18h 49m	310	13
105	GLM-5.3 Flash	0.0000	$2.31	42.4M	198k	17h 43m	261	2
106	Qwen3.8-Max-0902	0.0000	$37.59	115.3M	1.0M	19h 35m	541	3
107	Grok 4.7	0.0000	$36.18	19.8M	164k	20h 0m	126	5
108	Gemini 3.8 Flash	0.0000	$14.98	62.0M	118k	8h 40m	295	7
109	Grok 4.7	0.0000	$59.65	35.5M	185k	17h 44m	170	3
110	Grok 4.6	0.0000	$20.07	18.4M	217k	18h 24m	156	8

Notebook Compression

task page →110 runs · indexed 2026-10-08

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Sonnet 5.5	0.7077	$121.38	505.5M	1.6M	19h 29m	1140	23
2	Claude Opus 5.5	0.6404	$76.14	300.9M	607k	18h 32m	620	33
3	Claude Opus 5.5	0.6166	$96.89	380.6M	802k	18h 37m	872	33
4	GPT-6 Astra	0.6122	$2035.40	958.4M	1.9M	17h 45m	2132	53
5	GPT-6 Astra	0.6063	$2124.53	1006.7M	2.1M	18h 49m	2308	42
6	Claude Sonnet 5.5	0.6038	$113.09	459.8M	1.7M	19h 12m	1037	19
7	Claude Fable 5.1	0.6002	$67.85	117.2M	379k	14h 26m	401	17
8	Claude Sonnet 5.5	0.5947	$105.86	436.0M	1.5M	19h 24m	994	2
9	Claude Sonnet 5.5	0.5930	$98.59	408.5M	1.3M	17h 41m	827	1
10	Claude Opus 5.5	0.5926	$80.88	317.0M	668k	19h 35m	606	29
11	Claude Fable 5.1	0.5736	$77.68	134.5M	428k	13h 49m	450	16
12	Claude Sonnet 5.5	0.5678	$87.19	362.7M	1.2M	18h 49m	808	8
13	Claude Opus 5.5	0.5584	$79.55	318.4M	608k	18h 25m	649	27
14	Claude Opus 5.5	0.5566	$69.33	267.9M	597k	18h 1m	557	28
15	Claude Fable 5.1	0.5457	$68.00	118.2M	365k	13h 15m	419	21
16	GPT-6 Astra	0.5422	$1208.04	578.6M	1.3M	19h 17m	1389	24
17	Claude Haiku 5.5	0.5323	$15.39	247.7M	1.1M	15h 23m	618	3
18	Claude Fable 5.1	0.5279	$64.70	113.1M	334k	17h 24m	419	17
19	Claude Opus 5	0.5259	$112.64	181.6M	606k	17h 57m	454	17
20	Claude Haiku 5.5	0.5238	$18.15	307.8M	1.1M	18h 41m	707	23
21	Claude Fable 5	0.4971	$137.67	106.8M	434k	17h 17m	390	15
22	Claude Fable 5	0.4861	$123.21	90.8M	455k	18h 4m	318	16
23	Claude Fable 5.1	0.4824	$61.77	107.1M	338k	15h 49m	418	16
24	Gemini 4 Argon	0.4592	$350.30	290.6M	1.5M	20h 36m	690	25
25	Claude Opus 5	0.4532	$73.01	109.0M	516k	14h 40m	352	12
26	Claude Haiku 5.5	0.4519	$14.24	235.9M	949k	18h 16m	561	21
27	Gemini 4 Argon	0.4467	$232.09	321.0M	1.6M	20h 40m	706	28
28	Claude Opus 5	0.4439	$118.20	194.6M	584k	18h 55m	507	14
29	Claude Opus 5	0.4351	$112.61	183.8M	577k	18h 34m	471	19
30	GPT-6 Astra	0.4322	$274.97	132.6M	360k	5h 47m	407	7
31	Claude Opus 5	0.4296	$84.72	133.1M	506k	18h 0m	383	12
32	Claude Haiku 5.5	0.4204	$18.03	295.9M	1.2M	15h 35m	688	7
33	GPT-6 Astra	0.4121	$327.78	158.4M	411k	6h 16m	482	10
34	Gemini 4 Argon	0.4112	$223.42	270.8M	1.5M	21h 0m	634	23
35	Gemini 4 Argon	0.3971	$258.69	226.7M	1.1M	20h 39m	498	16
36	Gemini 4 Argon	0.3901	$194.88	235.0M	1.3M	15h 31m	566	13
37	Claude Fable 5	0.3654	$123.52	92.2M	440k	14h 28m	330	13
38	Claude Fable 5	0.3578	$103.79	73.7M	421k	17h 44m	273	13
39	Claude Haiku 5.5	0.3415	$9.66	154.9M	685k	7h 45m	327	9
40	Claude Fable 5	0.3346	$106.67	78.6M	394k	17h 8m	293	15
41	GLM-5.3	0.2770	$25.86	86.5M	327k	19h 49m	370	3
42	GLM-5.3	0.2744	$34.96	116.4M	391k	19h 2m	435	2
43	Muse Spark 1.3	0.2729	$60.32	65.3M	317k	6h 41m	357	11
44	Kimi K3	0.2647	$37.66	96.4M	363k	19h 31m	409	17
45	GPT-5.6 Sol	0.2623	$123.00	167.4M	279k	11h 19m	628	8
46	Qwen3.8-Max-0902	0.2600	$30.23	89.6M	1.1M	20h 1m	386	1
47	GPT-5.6 Sol	0.2599	$99.01	115.0M	257k	15h 3m	528	10
48	Grok 4.7	0.2584	$211.20	205.8M	904k	13h 1m	859	13
49	Qwen3.8-Max-0902	0.2513	$27.92	79.6M	1.2M	19h 13m	407	3
50	Grok 4.7	0.2464	$38.17	37.7M	250k	3h 43m	192	8
51	GPT-5.6 Sol	0.2438	$58.48	76.1M	190k	17h 55m	412	14
52	Grok 4.7	0.2422	$40.60	41.8M	253k	4h 6m	221	11
53	Qwen3.8-Max-0902	0.2416	$28.69	81.5M	1.3M	19h 35m	423	1
54	Kimi K3	0.2338	$43.76	111.9M	405k	17h 35m	403	12
55	GPT-5.6 Sol	0.2326	$88.19	110.8M	211k	11h 35m	518	6
56	GLM-5.3 Flash	0.2312	$4.45	118.7M	338k	13h 55m	477	3
57	Grok 4.6	0.2286	$85.31	84.1M	498k	7h 48m	331	14
58	Muse Spark 1.3	0.2285	$27.01	79.4M	314k	19h 7m	407	8
59	Kimi K3	0.2242	$36.27	87.9M	412k	11h 52m	382	12
60	GLM-5.3 Flash	0.2231	$4.83	113.6M	398k	17h 19m	455	18
61	Grok 4.7	0.2221	$20.96	23.3M	213k	3h 13m	149	5
62	Qwen3.8-Max-0902	0.2219	$31.23	88.3M	1.4M	19h 46m	478	1
63	Qwen3.8-Max	0.2190	$26.32	83.3M	818k	19h 14m	454	2
64	Qwen3.8-Max	0.2153	$10.82	27.5M	610k	19h 5m	314	2
65	Kimi K3	0.2151	$24.45	60.7M	274k	11h 45m	302	10
66	Qwen3.8-Max-0902	0.2150	$30.55	85.1M	1.4M	19h 31m	403	3
67	GLM-5.3 Flash	0.2148	$5.19	105.1M	375k	18h 46m	410	2
68	Grok 4.6	0.2146	$36.98	34.6M	412k	13h 20m	220	5
69	DeepSeek V4 Flash Vision Exp	0.2139	$1.97	101.8M	387k	13h 26m	420	1
70	GLM-5.3 Flash	0.2132	$3.44	59.6M	295k	17h 49m	300	2
71	Qwen3.8-Max	0.2125	$16.15	47.4M	646k	17h 46m	362	2
72	Gemini 3.8 Flash	0.2118	$6.55	47.0M	149k	3h 4m	286	6
73	Muse Spark 1.3	0.2112	$78.14	76.4M	260k	11h 8m	358	3
74	Grok 4.6	0.2098	$96.38	88.3M	636k	11h 39m	377	12
75	GLM-5.3	0.2089	$39.28	89.8M	362k	20h 0m	369	1
76	Grok 4.6	0.2060	$71.54	75.2M	388k	6h 34m	381	27
77	GLM-5.3	0.2058	$59.65	209.0M	565k	15h 11m	548	8
78	DeepSeek V4 Flash Vision Exp	0.2028	$2.58	144.1M	374k	19h 46m	508	2
79	Grok 4.6	0.2019	$87.92	83.6M	470k	8h 37m	361	10
80	Grok 4.7	0.2015	$41.06	38.3M	268k	3h 17m	188	5
81	Qwen3.8-Max	0.2005	$15.43	43.7M	693k	18h 44m	414	1
82	Kimi K3	0.1961	$13.59	21.1M	194k	9h 53m	179	9
83	DeepSeek V4 Flash Vision Exp	0.1948	$3.01	169.4M	449k	11h 0m	551	4
84	DeepSeek V4 Flash Vision Exp	0.1939	$2.93	164.0M	435k	14h 31m	526	4
85	Gemini 3.8 Flash	0.1896	$6.34	45.8M	152k	4h 56m	308	6
86	Qwen3.8-Max	0.1831	$20.18	58.6M	851k	19h 8m	443	2
87	GLM-5.3 Flash	0.1829	$5.24	90.6M	377k	19h 26m	352	2
88	DeepSeek V4 Flash Vision Exp	0.1818	$2.40	126.7M	415k	10h 56m	421	6
89	Gemini 3.8 Flash	0.1802	$4.63	34.5M	148k	3h 0m	225	6
90	Gemini 3.7 Flash	0.1775	$2.95	13.6M	108k	3h 17m	129	2
91	Gemini 3.7 Flash	0.1714	$6.92	27.7M	160k	6h 28m	236	5
92	Muse Spark 1.2	0.1646	$89.68	227.7M	1.2M	20h 2m	728	7
93	Gemini 3.8 Flash	0.1634	$11.15	62.0M	214k	5h 27m	315	7
94	GPT-5.6 Sol	0.1417	$64.27	80.6M	214k	10h 2m	389	3
95	Muse Spark 1.2	0.1353	$32.24	170.7M	487k	10h 37m	551	5
96	Muse Spark 1.3	0.1309	$65.04	67.7M	249k	11h 11m	371	6
97	GLM-5.3	0.1261	$28.93	86.9M	409k	13h 16m	328	2
98	Muse Spark 1.3	0.1170	$63.50	77.9M	313k	5h 49m	326	3
99	Muse Spark 1.2	0.0816	$32.40	192.3M	534k	7h 46m	570	4
100	Gemini 3.7 Flash	0.0417	$2.99	16.0M	112k	3h 8m	165	6
101	Gemini 3.7 Flash	0.0412	$3.05	19.8M	150k	3h 10m	194	10
102	Gemini 3.7 Flash	0.0403	$4.56	17.9M	128k	4h 20m	145	8
103	Muse Spark 1.2	0.0387	$183.05	621.0M	1.3M	19h 7m	1497	7
104	Muse Spark 1.2	0.0356	$37.95	196.7M	1.2M	19h 2m	669	3
105	Gemini 3.8 Flash	0.0292	$4.10	23.6M	91k	4h 7m	225	5
106	Inkling	0.0214	$5.30	5.4M	31k	3h 11m	184	2
107	Inkling	0.0209	$6.76	6.9M	44k	2h 24m	193	2
108	Inkling	0.0205	$8.12	8.4M	30k	3h 20m	185	1
109	Inkling	0.0203	$5.44	5.6M	27k	1h 39m	121	3
110	Inkling	0.0179	$5.86	6.0M	36k	2h 37m	166	2

Optimizer Design

task page →110 runs · indexed 2026-10-08

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Opus 5.5	0.3908	$28.07	87.7M	400k	17h 51m	286	12
2	Gemini 4 Argon	0.3700	$138.18	88.1M	358k	17h 59m	225	7
3	GPT-6 Astra	0.3513	$198.61	101.4M	290k	17h 55m	381	5
4	GPT-5.6 Sol	0.3487	$65.53	80.8M	133k	10h 22m	433	4
5	Claude Sonnet 5.5	0.3328	$24.92	96.2M	443k	18h 17m	313	2
6	Gemini 4 Argon	0.3302	$144.66	126.0M	552k	18h 30m	327	6
7	GPT-5.6 Sol	0.3194	$196.66	203.2M	202k	11h 49m	678	23
8	Kimi K3	0.3057	$15.06	37.2M	181k	19h 11m	275	4
9	Grok 4.7	0.2867	$182.37	175.0M	483k	20h 0m	744	15
10	GPT-5.6 Sol	0.2846	$55.29	72.7M	133k	15h 42m	423	5
11	Gemini 4 Argon	0.2842	$147.52	93.7M	408k	17h 44m	246	6
12	Claude Opus 5.5	0.2824	$26.98	82.6M	395k	14h 4m	282	6
13	GPT-5.6 Sol	0.2809	$71.65	93.2M	177k	14h 22m	523	21
14	Gemini 4 Argon	0.2729	$106.83	62.5M	361k	17h 2m	200	6
15	Claude Opus 5.5	0.2715	$26.87	83.2M	387k	12h 26m	281	8
16	Claude Sonnet 5.5	0.2671	$40.11	160.9M	619k	15h 53m	356	3
17	Gemini 4 Argon	0.2637	$128.39	105.4M	394k	12h 7m	261	6
18	Claude Opus 5.5	0.2621	$24.28	68.6M	403k	13h 10m	230	4
19	Claude Sonnet 5.5	0.2578	$28.93	110.0M	545k	14h 19m	282	2
20	Grok 4.6	0.2557	$99.14	98.0M	624k	19h 20m	424	18
21	GLM-5.3 Flash	0.2521	$2.91	48.3M	286k	16h 23m	241	3
22	Claude Opus 5	0.2499	$36.84	48.8M	334k	17h 28m	203	2
23	Claude Fable 5	0.2478	$59.89	43.2M	223k	15h 59m	229	10
24	Claude Sonnet 5.5	0.2467	$39.74	163.6M	545k	19h 1m	391	3
25	GLM-5.3	0.2349	$19.98	35.0M	224k	17h 41m	192	3
26	Qwen3.8-Max-0902	0.2342	$15.36	42.2M	714k	19h 22m	336	2
27	Claude Fable 5.1	0.2272	$15.45	22.4M	175k	11h 13m	164	5
28	GPT-5.6 Sol	0.2234	$31.29	52.9M	100k	13h 17m	373	2
29	GLM-5.3	0.2192	$17.37	36.9M	195k	19h 6m	241	2
30	Claude Sonnet 5.5	0.2147	$34.56	138.5M	535k	18h 9m	365	2
31	GLM-5.3	0.2082	$16.48	26.8M	198k	19h 23m	184	3
32	GLM-5.3 Flash	0.2047	$1.35	17.1M	199k	16h 48m	131	2
33	Grok 4.7	0.2033	$279.55	260.9M	639k	20h 0m	949	17
34	Grok 4.6	0.1957	$53.90	66.7M	503k	19h 24m	395	24
35	Muse Spark 1.3	0.1949	$17.67	14.5M	143k	19h 59m	150	1
36	GPT-6 Astra	0.1940	$334.17	159.8M	312k	18h 23m	412	7
37	Claude Haiku 5.5	0.1858	$3.96	58.4M	400k	17h 26m	217	2
38	Qwen3.8-Max-0902	0.1854	$14.00	33.7M	859k	19h 56m	315	1
39	GPT-6 Astra	0.1777	$89.95	48.3M	203k	19h 38m	244	10
40	Claude Opus 5	0.1760	$28.52	36.5M	275k	16h 52m	182	3
41	Claude Fable 5.1	0.1750	$17.15	26.3M	165k	12h 46m	182	6
42	Claude Opus 5.5	0.1731	$23.28	67.5M	372k	18h 2m	238	7
43	Claude Opus 5	0.1714	$43.94	57.7M	410k	17h 5m	200	4
44	Claude Haiku 5.5	0.1698	$7.09	114.6M	506k	14h 15m	325	3
45	Grok 4.6	0.1570	$59.53	57.5M	295k	19h 26m	258	11
46	Kimi K3	0.1475	$41.51	100.0M	508k	20h 1m	354	4
47	Claude Fable 5.1	0.1459	$14.90	22.4M	152k	15h 9m	171	4
48	Claude Haiku 5.5	0.1416	$5.30	82.3M	440k	13h 57m	256	1
49	Kimi K3	0.1370	$16.05	38.0M	228k	19h 25m	222	4
50	Grok 4.6	0.1368	$54.49	49.3M	462k	19h 16m	240	10
51	Qwen3.8-Max-0902	0.1351	$14.61	42.4M	583k	19h 28m	334	1
52	Claude Fable 5.1	0.1343	$11.08	15.3M	141k	13h 28m	134	3
53	Claude Opus 5	0.1316	$29.92	37.9M	299k	16h 54m	179	2
54	Gemini 3.7 Flash	0.1194	$22.13	241.0M	134k	10h 36m	834	2
55	Claude Fable 5	0.1170	$54.58	37.6M	230k	16h 38m	214	5
56	Claude Haiku 5.5	0.1133	$4.76	75.7M	379k	15h 43m	263	1
57	DeepSeek V4 Flash Vision Exp	0.1123	$1.12	52.3M	255k	18h 23m	256	2
58	Muse Spark 1.3	0.1110	$29.21	25.7M	102k	19h 53m	307	2
59	Claude Haiku 5.5	0.1082	$4.35	64.8M	433k	15h 17m	231	2
60	Muse Spark 1.3	0.1023	$23.80	20.6M	126k	19h 41m	260	2
61	Kimi K3	0.0990	$22.58	49.0M	297k	17h 53m	241	6
62	GLM-5.3	0.0984	$17.01	55.5M	261k	19h 4m	277	5
63	GLM-5.3	0.0965	$13.47	23.0M	159k	18h 59m	180	2
64	GLM-5.3 Flash	0.0964	$2.93	36.3M	234k	16h 39m	223	3
65	Qwen3.8-Max-0902	0.0958	$20.91	59.3M	909k	19h 56m	375	2
66	GPT-6 Astra	0.0929	$278.29	132.5M	303k	16h 52m	378	7
67	Claude Fable 5	0.0927	$57.48	40.3M	232k	16h 57m	210	4
68	GLM-5.3 Flash	0.0879	$2.67	37.1M	228k	16h 58m	207	2
69	Gemini 3.8 Flash	0.0861	$8.14	32.5M	139k	6h 7m	220	2
70	Gemini 3.8 Flash	0.0851	$12.84	46.8M	190k	10h 59m	216	5
71	Grok 4.6	0.0839	$80.45	75.7M	333k	19h 42m	279	3
72	Qwen3.8-Max	0.0827	$7.68	17.4M	516k	19h 49m	226	3
73	DeepSeek V4 Flash Vision Exp	0.0770	$1.22	63.5M	222k	19h 1m	334	2
74	Gemini 3.8 Flash	0.0755	$12.83	41.1M	192k	11h 46m	220	4
75	Qwen3.8-Max	0.0741	$7.61	18.8M	448k	19h 35m	241	3
76	Claude Fable 5	0.0728	$58.69	40.0M	246k	15h 56m	203	9
77	Qwen3.8-Max	0.0655	$6.87	15.9M	443k	19h 59m	203	3
78	Claude Fable 5	0.0619	$49.51	34.2M	205k	16h 37m	199	3
79	Gemini 3.7 Flash	0.0537	$10.37	85.1M	122k	13h 22m	435	2
80	GPT-6 Astra	0.0523	$53.35	35.6M	191k	19h 46m	218	6
81	Gemini 3.8 Flash	0.0506	$9.34	45.5M	175k	10h 6m	233	2
82	Gemini 3.7 Flash	0.0405	$25.34	217.9M	182k	10h 29m	744	2
83	DeepSeek V4 Flash Vision Exp	0.0348	$1.44	75.8M	242k	19h 19m	349	2
84	GLM-5.3 Flash	0.0292	$1.93	36.7M	215k	18h 0m	223	3
85	Claude Fable 5.1	0.0226	$15.74	21.7M	202k	14h 5m	141	4
86	Qwen3.8-Max	0.0143	$7.32	18.3M	421k	19h 31m	238	1
87	Grok 4.7	0.0119	$55.61	43.7M	222k	7h 51m	195	7
88	Gemini 3.8 Flash	0.0094	$13.99	75.0M	146k	11h 41m	384	5
89	Grok 4.7	0.0000	$308.24	304.6M	520k	20h 0m	1215	1
90	Muse Spark 1.2	0.0000	$39.25	85.0M	351k	13h 51m	275	7
91	Muse Spark 1.2	0.0000	$46.70	116.8M	475k	9h 50m	364	8
92	DeepSeek V4 Flash Vision Exp	0.0000	$1.51	79.4M	262k	19h 14m	406	2
93	Muse Spark 1.2	0.0000	$55.59	163.2M	672k	19h 37m	567	1
94	Gemini 3.7 Flash	0.0000	$28.88	259.5M	185k	18h 35m	908	1
95	Gemini 3.7 Flash	0.0000	$12.41	47.0M	144k	13h 48m	297	2
96	DeepSeek V4 Flash Vision Exp	0.0000	$1.33	65.8M	275k	19h 51m	313	1
97	Muse Spark 1.2	0.0000	$27.04	48.5M	345k	6h 22m	256	3
98	Grok 4.7	0.0000	$72.65	66.7M	251k	9h 7m	265	9
99	Muse Spark 1.3	0.0000	$14.04	11.9M	125k	19h 29m	179	1
100	Muse Spark 1.2	0.0000	$16.21	19.8M	148k	19h 57m	188	1
101	Inkling	0.0000	$1.08	1.1M	14k	1h 4m	58	1
102	Kimi K3	0.0000	$19.05	42.1M	297k	14h 45m	219	3
103	Inkling	0.0000	$1.53	1.5M	16k	1h 13m	73	2
104	Qwen3.8-Max-0902	0.0000	$24.43	66.6M	1.2M	19h 37m	381	2
105	Qwen3.8-Max	0.0000	$12.42	33.0M	636k	19h 59m	303	1
106	Inkling	0.0000	$3.84	3.9M	25k	5h 18m	124	1
107	Muse Spark 1.3	0.0000	$22.03	18.0M	123k	19h 58m	201	2
108	Inkling	0.0000	$2.54	2.6M	28k	2h 1m	88	1
109	Inkling	0.0000	$1.63	1.6M	27k	1h 46m	73	1
110	Claude Opus 5	0.0000	$34.43	42.6M	358k	17h 38m	173	3

PostgreSQL 18 on SQLite

task page →109 runs · indexed 2026-10-08

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	GPT-6 Astra	0.7308	$2982.00	1383.4M	2.9M	19h 56m	3073	67
2	GPT-6 Astra	0.7143	$3132.52	1453.4M	2.8M	19h 51m	3140	44
3	Gemini 4 Argon	0.6923	$41.63	62.6M	389k	2h 48m	192	2
4	Gemini 4 Argon	0.6923	$43.92	65.5M	430k	2h 31m	192	2
5	Gemini 4 Argon	0.6923	$26.32	82.0M	301k	2h 32m	224	2
6	Gemini 4 Argon	0.6923	$25.43	64.1M	365k	2h 39m	186	2
7	GPT-6 Astra	0.6593	$3139.16	1477.1M	2.8M	19h 52m	3280	73
8	Gemini 4 Argon	0.6484	$34.67	93.0M	375k	2h 50m	250	2
9	GPT-6 Astra	0.6484	$3119.08	1452.8M	2.8M	19h 56m	3108	62
10	GPT-6 Astra	0.6209	$3312.10	1548.3M	2.9M	20h 0m	3255	61
11	Claude Opus 5.5	0.5934	$433.39	1590.6M	4.3M	19h 2m	3640	18
12	Claude Opus 5.5	0.5659	$363.10	1344.0M	3.5M	19h 34m	3174	20
13	Claude Sonnet 5.5	0.5549	$310.17	1253.5M	4.5M	19h 16m	2929	12
14	GPT-5.6 Sol	0.5275	$209.42	198.9M	277k	4h 5m	589	12
15	Claude Sonnet 5.5	0.5000	$308.88	1278.7M	4.0M	19h 11m	2863	6
16	Claude Sonnet 5.5	0.5000	$284.21	1159.1M	4.0M	19h 17m	2709	1
17	Claude Sonnet 5.5	0.4780	$247.76	1000.6M	3.6M	19h 38m	2348	2
18	Claude Opus 5.5	0.4615	$283.28	1029.0M	2.9M	18h 59m	2312	13
19	Claude Sonnet 5.5	0.4451	$309.32	1277.3M	4.1M	19h 4m	2927	3
20	Claude Fable 5.1	0.4011	$588.09	1056.3M	2.5M	17h 46m	2515	13
21	Gemini 3.8 Flash	0.4011	$163.91	1747.0M	1.9M	19h 43m	3941	86
22	Claude Fable 5.1	0.3901	$721.01	1319.3M	2.5M	17h 44m	2872	14
23	GPT-5.6 Sol	0.3901	$654.39	640.1M	687k	19h 58m	1472	28
24	Gemini 3.8 Flash	0.3846	$244.90	2763.6M	3.1M	19h 16m	6084	124
25	Claude Fable 5.1	0.3516	$384.99	680.6M	1.8M	18h 59m	1688	9
26	Claude Opus 5	0.3462	$503.66	880.6M	1.6M	19h 10m	1883	12
27	Claude Fable 5	0.3407	$1287.66	1111.6M	2.3M	19h 25m	2503	12
28	Muse Spark 1.3	0.3407	$551.80	666.0M	898k	19h 58m	1714	4
29	Claude Fable 5.1	0.3352	$475.86	850.2M	2.1M	18h 4m	1894	7
30	Claude Opus 5.5	0.3352	$189.85	651.3M	2.3M	19h 20m	1478	8
31	Claude Opus 5	0.3352	$381.03	662.1M	1.3M	19h 37m	1475	7
32	Claude Opus 5	0.3352	$353.77	610.8M	1.3M	19h 43m	1274	5
33	Gemini 3.8 Flash	0.3352	$208.72	2281.9M	3.1M	19h 12m	4971	91
34	Gemini 3.8 Flash	0.3297	$180.31	1833.2M	2.5M	19h 54m	4050	100
35	Claude Opus 5	0.3242	$340.68	578.1M	1.3M	19h 18m	1332	6
36	Claude Opus 5.5	0.2967	$184.14	630.5M	2.2M	19h 31m	1487	10
37	Claude Opus 5	0.2912	$434.78	748.3M	1.6M	18h 39m	1774	5
38	Claude Haiku 5.5	0.2857	$70.44	1203.7M	4.0M	19h 19m	2653	34
39	Claude Haiku 5.5	0.2857	$79.48	1383.8M	3.8M	17h 44m	2900	5
40	Grok 4.7	0.2747	$697.58	718.7M	2.2M	19h 51m	2979	27
41	Grok 4.7	0.2637	$767.04	780.4M	2.4M	18h 58m	3117	30
42	Claude Fable 5	0.2637	$836.08	698.8M	1.8M	19h 25m	1580	13
43	Claude Haiku 5.5	0.2637	$80.98	1401.6M	4.1M	17h 22m	3095	1
44	Claude Fable 5	0.2637	$945.64	794.6M	2.0M	19h 56m	1718	6
45	Gemini 3.8 Flash	0.2637	$135.19	1385.8M	1.8M	16h 40m	3053	61
46	Gemini 3.7 Flash	0.2582	$124.97	1333.8M	2.5M	11h 51m	3014	37
47	Grok 4.7	0.2527	$961.79	986.1M	3.0M	20h 0m	3909	26
48	Gemini 3.7 Flash	0.2418	$144.25	1592.6M	2.6M	19h 46m	3450	39
49	Claude Fable 5.1	0.2308	$392.90	697.7M	1.8M	19h 16m	1679	4
50	Claude Haiku 5.5	0.2308	$58.07	999.5M	3.0M	18h 19m	2320	6
51	Gemini 3.7 Flash	0.2253	$193.89	2112.9M	3.0M	19h 55m	4680	16
52	Claude Haiku 5.5	0.2253	$57.46	996.8M	2.8M	17h 55m	2275	3
53	Grok 4.7	0.1978	$1004.11	1036.8M	3.4M	20h 0m	4217	22
54	Gemini 3.7 Flash	0.1758	$159.99	1689.8M	2.8M	19h 49m	3751	13
55	Grok 4.6	0.1593	$509.89	496.5M	1.6M	19h 56m	1696	70
56	GLM-5.3	0.1593	$191.86	711.5M	862k	19h 53m	1746	17
57	Claude Fable 5	0.1538	$641.49	517.5M	1.6M	19h 39m	1292	4
58	Grok 4.6	0.1538	$461.37	465.7M	1.9M	17h 59m	1842	49
59	DeepSeek V4 Flash Vision Exp	0.1484	$22.50	1379.3M	2.0M	19h 53m	3146	1
60	GLM-5.3	0.1484	$202.20	744.7M	1.0M	19h 42m	1789	7
61	GLM-5.3	0.1374	$201.89	706.7M	843k	19h 32m	1585	9
62	GLM-5.3	0.1374	$155.77	570.5M	772k	19h 52m	1384	2
63	Qwen3.8-Max	0.1374	$186.86	700.0M	1.6M	19h 30m	1822	2
64	Qwen3.8-Max	0.1319	$178.70	667.7M	1.7M	19h 47m	1549	4
65	DeepSeek V4 Flash Vision Exp	0.1264	$22.61	1383.3M	2.0M	19h 51m	3272	3
66	Muse Spark 1.3	0.1099	$305.43	1604.1M	2.2M	20h 0m	3690	2
67	DeepSeek V4 Flash Vision Exp	0.1044	$19.66	1214.6M	1.7M	19h 25m	2688	9
68	Grok 4.6	0.0989	$257.56	255.7M	1.5M	8h 44m	952	17
69	GLM-5.3	0.0989	$219.76	822.5M	966k	19h 50m	1903	11
70	GLM-5.3 Flash	0.0769	$23.42	737.2M	1.0M	19h 23m	1578	4
71	Qwen3.8-Max	0.0769	$170.86	638.4M	1.5M	19h 53m	1587	2
72	Grok 4.6	0.0714	$398.74	383.4M	2.7M	19h 51m	1457	34
73	Kimi K3	0.0714	$227.56	673.8M	1.2M	20h 0m	1528	1
74	DeepSeek V4 Flash Vision Exp	0.0659	$23.23	1428.3M	2.0M	19h 44m	3348	1
75	Qwen3.8-Max	0.0659	$174.14	653.5M	1.5M	19h 26m	1536	1
76	GLM-5.3 Flash	0.0659	$26.47	848.3M	1.3M	18h 3m	1663	11
77	Kimi K3	0.0659	$265.37	772.2M	1.4M	20h 1m	1427	1
78	Kimi K3	0.0604	$238.20	702.8M	1.4M	20h 1m	1513	1
79	Muse Spark 1.3	0.0549	$522.84	598.7M	1.2M	20h 0m	1525	1
80	Muse Spark 1.3	0.0549	$518.76	603.9M	1.1M	19h 59m	1572	1
81	Qwen3.8-Max	0.0549	$158.21	583.9M	1.7M	19h 23m	1405	3
82	Kimi K3	0.0495	$197.52	561.9M	1.3M	20h 0m	1168	1
83	GLM-5.3 Flash	0.0495	$25.01	730.3M	929k	19h 49m	1709	1
84	Grok 4.6	0.0495	$148.44	141.0M	985k	6h 17m	559	10
85	Gemini 3.7 Flash	0.0495	$17.96	182.1M	620k	1h 34m	402	2
86	Muse Spark 1.3	0.0495	$525.95	562.0M	986k	20h 0m	1387	1
87	Qwen3.8-Max-0902	0.0330	$183.61	654.3M	2.6M	19h 34m	1737	2
88	GLM-5.3 Flash	0.0330	$27.79	699.2M	1.1M	19h 38m	1496	10
89	Qwen3.8-Max-0902	0.0330	$116.07	407.7M	1.9M	20h 1m	1122	2
90	Qwen3.8-Max-0902	0.0220	$150.66	544.1M	1.9M	20h 0m	1434	1
91	GLM-5.3 Flash	0.0220	$24.50	755.8M	998k	19h 51m	1899	14
92	Qwen3.8-Max-0902	0.0165	$112.13	383.6M	1.8M	19h 52m	1132	5
93	DeepSeek V4 Flash Vision Exp	0.0165	$8.18	489.5M	856k	20h 0m	1194	2
94	Qwen3.8-Max-0902	0.0110	$104.91	366.4M	1.7M	20h 1m	1093	1
95	Muse Spark 1.2	0.0055	$37.82	217.7M	976k	3h 48m	496	10
96	Muse Spark 1.2	0.0055	$40.72	146.5M	561k	3h 14m	420	4
97	GPT-5.6 Sol	0.0055†	$293.05	285.8M	311k	4h 20m	671	6
98	Muse Spark 1.2	0.0000	$32.40	176.5M	473k	2h 53m	436	3
99	Inkling	0.0000	$22.34	23.3M	56k	1h 53m	340	3
100	Claude Fable 5	0.0000†	$385.63	316.2M	920k	5h 26m	738	3
101	Muse Spark 1.2	0.0000	$14.08	83.6M	240k	1h 8m	314	2
102	GPT-5.6 Sol	0.0000	$137.59	141.5M	243k	3h 27m	480	2
103	GPT-5.6 Sol	0.0000	$142.29	142.1M	259k	4h 34m	462	11
104	Grok 4.7	0.0000	$890.38	903.6M	3.1M	20h 0m	3660	12
105	Inkling	0.0000	$19.06	19.8M	53k	1h 41m	302	2
106	Inkling	0.0000	$19.04	19.9M	46k	23m	252	1
107	Inkling	0.0000	$16.56	17.2M	48k	32m	214	2
108	Inkling	0.0000	$16.62	17.3M	42k	34m	287	1
109	Muse Spark 1.2	0.0000	$33.19	162.4M	603k	2h 4m	458	1

† score adjusted by manual review; the trace's Judge tab carries the measured score and the rationale.

Quantum ESPRESSO pw.x in Rust

task page →110 runs · indexed 2026-10-08

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Opus 5.5	0.9756	$115.01	354.9M	1.7M	9h 7m	784	13
2	Claude Sonnet 5.5	0.9756	$170.72	658.9M	3.1M	17h 11m	1438	15
3	Claude Opus 5.5	0.9756	$120.52	382.4M	1.7M	11h 39m	867	25
4	Claude Opus 5.5	0.9594	$109.78	339.6M	1.6M	7h 33m	786	12
5	Claude Opus 5.5	0.9512	$58.52	152.7M	1.1M	4h 44m	350	6
6	Claude Sonnet 5.5	0.9512	$124.72	468.1M	2.5M	11h 1m	1054	5
7	Claude Fable 5.1	0.9431	$100.68	158.8M	876k	5h 35m	408	11
8	Claude Sonnet 5.5	0.9431	$92.52	348.6M	1.8M	7h 56m	748	15
9	Claude Sonnet 5.5	0.9431	$139.31	539.0M	2.5M	15h 11m	1141	12
10	GPT-6 Astra	0.9350	$571.91	270.6M	494k	3h 53m	570	4
11	Claude Opus 5.5	0.9350	$70.36	204.4M	1.1M	5h 26m	460	11
12	Claude Fable 5.1	0.9350	$63.80	99.1M	584k	3h 4m	218	8
13	Claude Sonnet 5.5	0.9350	$154.31	585.2M	2.9M	17h 40m	1295	15
14	GPT-6 Astra	0.9350	$540.43	256.4M	446k	3h 37m	544	4
15	Claude Fable 5.1	0.9268	$63.95	99.7M	578k	3h 23m	222	5
16	Claude Fable 5.1	0.9187	$72.59	116.9M	580k	4h 7m	245	5
17	Gemini 4 Argon	0.9187	$119.37	259.0M	1.1M	10h 5m	645	1
18	Gemini 4 Argon	0.9106	$303.04	318.6M	1.2M	11h 58m	691	2
19	Claude Fable 5.1	0.9106	$77.90	124.4M	643k	3h 55m	256	4
20	Gemini 4 Argon	0.9024	$154.81	373.7M	1.6M	12h 47m	887	2
21	Gemini 4 Argon	0.9024	$388.42	369.1M	1.6M	13h 39m	867	6
22	Gemini 4 Argon	0.8699	$138.25	365.2M	1.5M	10h 38m	856	2
23	GPT-6 Astra	0.8699	$378.55	179.5M	449k	3h 11m	459	5
24	Claude Fable 5	0.8699	$390.43	293.5M	1.3M	11h 49m	656	10
25	GPT-6 Astra	0.8699	$279.05	132.3M	363k	2h 37m	372	4
26	Claude Fable 5	0.8618	$392.73	295.7M	1.3M	10h 46m	658	9
27	GPT-6 Astra	0.8293	$299.55	142.9M	339k	2h 28m	402	4
28	Claude Opus 5	0.8211	$159.35	224.3M	1.3M	7h 45m	513	7
29	Claude Fable 5	0.8211	$384.44	289.9M	1.3M	12h 52m	660	5
30	Claude Opus 5	0.8211	$245.12	359.5M	1.8M	15h 10m	826	19
31	Claude Opus 5	0.8130	$286.81	434.1M	1.9M	15h 11m	970	12
32	Claude Opus 5	0.8130	$184.58	276.3M	1.3M	15h 20m	620	7
33	Claude Fable 5	0.8049	$335.99	247.3M	1.2M	10h 59m	581	7
34	Claude Fable 5	0.7967	$346.56	252.2M	1.3M	13h 4m	602	9
35	Claude Haiku 5.5	0.7886	$28.18	430.1M	2.5M	14h 14m	967	8
36	Claude Opus 5	0.7886	$115.90	152.7M	1.1M	6h 12m	390	8
37	Claude Haiku 5.5	0.7724	$22.74	342.2M	2.1M	12h 39m	720	5
38	Claude Haiku 5.5	0.7073	$23.96	363.4M	2.1M	14h 34m	775	3
39	Claude Haiku 5.5	0.6341	$29.37	441.9M	2.7M	11h 2m	943	8
40	Grok 4.7	0.6179	$248.05	251.2M	1.1M	11h 32m	1086	16
41	Claude Haiku 5.5	0.6179	$20.61	295.8M	2.1M	11h 25m	642	7
42	GPT-5.6 Sol	0.5203	$304.21	296.3M	260k	4h 50m	606	9
43	GPT-5.6 Sol	0.4634	$57.42	64.9M	189k	2h 47m	279	13
44	Grok 4.7	0.4553	$566.99	570.1M	2.1M	20h 0m	2281	1
45	Gemini 3.8 Flash	0.4553	$73.38	570.9M	924k	15h 34m	1436	4
46	GPT-5.6 Sol	0.4553	$120.99	116.9M	214k	5h 5m	404	8
47	Gemini 3.8 Flash	0.4390	$64.68	501.7M	861k	10h 34m	1229	2
48	Muse Spark 1.3	0.4228	$418.71	442.8M	1.1M	20h 0m	1119	1
49	Muse Spark 1.3	0.4228	$514.22	583.5M	1.3M	19h 59m	1415	1
50	GPT-5.6 Sol	0.4065	$104.97	107.5M	204k	3h 16m	391	10
51	Grok 4.6	0.3984	$196.95	181.0M	1.6M	13h 51m	681	14
52	Gemini 3.8 Flash	0.3577	$44.62	399.0M	781k	9h 40m	979	7
53	Muse Spark 1.3	0.3577	$352.28	387.2M	1.4M	19h 43m	1153	3
54	Grok 4.6	0.3008	$282.67	273.1M	1.6M	13h 49m	976	40
55	Grok 4.7	0.2764	$412.96	420.2M	1.8M	14h 39m	1779	17
56	Grok 4.6	0.2683	$1211.08	1191.6M	2.2M	19h 59m	3559	243
57	Kimi K3	0.2602	$75.00	147.3M	1.2M	19h 51m	428	1
58	GLM-5.3	0.1707	$145.94	507.1M	1.3M	19h 30m	1082	13
59	Gemini 3.7 Flash	0.1545	$46.71	314.3M	747k	10h 28m	740	2
60	Muse Spark 1.3	0.1463	$336.34	348.3M	1.1M	19h 35m	939	9
61	GLM-5.3	0.1382	$132.96	476.8M	1.3M	19h 42m	984	12
62	GLM-5.3	0.1301	$108.12	382.4M	1.3M	19h 32m	838	2
63	Gemini 3.8 Flash	0.1301	$76.16	654.7M	1.0M	15h 23m	1694	6
64	Grok 4.7	0.1138	$593.38	588.6M	2.1M	20h 0m	2376	1
65	Grok 4.6	0.1057	$123.16	111.5M	1.7M	10h 28m	429	13
66	Gemini 3.8 Flash	0.1057	$39.41	364.3M	690k	7h 35m	868	4
67	Grok 4.6	0.0976	$96.14	88.3M	1.1M	7h 19m	360	12
68	Gemini 3.7 Flash	0.0976	$71.79	423.8M	777k	17h 50m	948	3
69	Grok 4.7	0.0650	$449.65	416.6M	1.1M	20h 0m	1517	11
70	Gemini 3.7 Flash	0.0650	$36.49	288.8M	712k	6h 10m	725	2
71	GLM-5.3	0.0650	$123.53	403.7M	1.6M	19h 19m	957	11
72	GLM-5.3 Flash	0.0569	$14.19	350.7M	1.3M	17h 49m	793	3
73	Kimi K3	0.0488	$160.13	415.5M	1.7M	20h 0m	999	1
74	Muse Spark 1.2	0.0244	$48.27	287.1M	1.0M	3h 0m	598	1
75	Kimi K3	0.0244	$100.42	211.3M	1.8M	20h 0m	566	1
76	Muse Spark 1.3	0.0163	$435.17	464.8M	1.2M	19h 59m	1196	1
77	Kimi K3	0.0081	$131.87	355.0M	1.3M	19h 56m	785	6
78	GLM-5.3 Flash	0.0081	$17.03	435.1M	1.2M	17h 59m	1065	16
79	Qwen3.8-Max	0.0000	$48.53	146.5M	1.8M	19h 49m	431	1
80	Qwen3.8-Max-0902	0.0000	$27.89	47.8M	2.0M	20h 3m	236	1
81	DeepSeek V4 Flash Vision Exp	0.0000	$7.62	422.1M	1.1M	12h 5m	912	6
82	Gemini 3.7 Flash	0.0000	$51.00	393.0M	804k	11h 3m	1002	4
83	Kimi K3	0.0000	$72.76	158.2M	1.2M	20h 1m	418	1
84	Muse Spark 1.2	0.0000	$22.08	58.0M	685k	1h 30m	194	3
85	Qwen3.8-Max	0.0000	$54.35	159.6M	2.3M	19h 59m	413	1
86	Qwen3.8-Max	0.0000	$33.37	86.6M	1.8M	19h 52m	301	2
87	GLM-5.3 Flash	0.0000	$10.99	299.0M	1.2M	12h 18m	713	5
88	Gemini 3.7 Flash	0.0000	$24.96	238.1M	690k	3h 15m	539	2
89	GLM-5.3 Flash	0.0000	$9.30	218.2M	1.0M	15h 25m	600	4
90	Inkling	0.0000	$18.68	19.4M	66k	28m	186	2
91	DeepSeek V4 Flash Vision Exp	0.0000	$6.14	336.4M	959k	16h 29m	853	10
92	Qwen3.8-Max-0902	0.0000	$30.52	39.7M	2.0M	20h 1m	194	1
93	Inkling	0.0000	$22.94	24.0M	46k	17m	274	1
94	DeepSeek V4 Flash Vision Exp	0.0000	$4.17	227.6M	660k	6h 35m	447	3
95	GPT-5.6 Sol	0.0000	$210.32	203.5M	250k	6h 57m	577	6
96	Inkling	0.0000	$8.19	8.5M	35k	16m	142	1
97	Muse Spark 1.2	0.0000	$8.00	39.5M	393k	1h 21m	166	2
98	Qwen3.8-Max-0902	0.0000	$26.48	47.3M	2.3M	20h 5m	231	1
99	Inkling	0.0000	$9.25	9.5M	49k	17m	146	2
100	Qwen3.8-Max-0902	0.0000	$32.89	67.0M	2.0M	20h 5m	285	1
101	DeepSeek V4 Flash Vision Exp	0.0000	$6.64	359.4M	1.1M	12h 22m	882	5
102	GLM-5.3	0.0000	$103.68	348.9M	1.2M	19h 23m	795	14
103	GLM-5.3 Flash	0.0000	$7.12	187.9M	1.1M	14h 19m	562	4
104	DeepSeek V4 Flash Vision Exp	0.0000	$6.97	389.0M	1.0M	14h 29m	887	5
105	Muse Spark 1.2	0.0000	$5.45	30.5M	131k	29m	160	3
106	Qwen3.8-Max	0.0000	$35.43	95.7M	1.8M	20h 1m	338	1
107	Inkling	0.0000	$9.32	9.6M	44k	15m	138	2
108	Muse Spark 1.2	0.0000	$10.30	54.9M	392k	1h 20m	248	4
109	Qwen3.8-Max-0902	0.0000	$26.30	38.4M	2.1M	20h 8m	222	1
110	Qwen3.8-Max	0.0000	$11.62	13.4M	1.3M	19h 53m	112	1

Qubit Routing

task page →110 runs · indexed 2026-10-08

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Opus 5.5	0.9241	$44.40	160.3M	462k	12h 19m	408	3
2	Claude Opus 5.5	0.9229	$49.89	183.1M	502k	16h 23m	427	10
3	GPT-6 Astra	0.9207	$412.86	201.8M	473k	7h 46m	595	12
4	GPT-6 Astra	0.9186	$1381.68	662.3M	1.4M	17h 52m	1621	26
5	GPT-6 Astra	0.9168	$2130.53	1024.2M	1.9M	19h 50m	2384	45
6	Claude Opus 5.5	0.9138	$40.04	137.9M	476k	14h 14m	374	13
7	Claude Sonnet 5.5	0.9132	$39.23	158.7M	579k	6h 58m	352	15
8	GPT-6 Astra	0.9131	$2239.25	1067.1M	2.1M	19h 57m	2429	54
9	Claude Opus 5.5	0.9052	$63.24	243.8M	546k	14h 55m	528	21
10	Claude Sonnet 5.5	0.9031	$82.69	337.6M	1.2M	18h 35m	761	1
11	Claude Sonnet 5.5	0.9003	$69.19	287.2M	925k	18h 59m	671	10
12	Claude Sonnet 5.5	0.8951	$50.53	209.6M	671k	12h 55m	412	7
13	Claude Sonnet 5.5	0.8940	$63.68	259.4M	928k	17h 26m	635	4
14	Claude Opus 5	0.8917	$57.70	85.6M	405k	15h 40m	277	6
15	Claude Fable 5.1	0.8857	$37.59	60.9M	295k	9h 18m	239	6
16	Claude Opus 5	0.8845	$64.52	99.6M	401k	17h 6m	313	3
17	GPT-6 Astra	0.8800	$604.79	292.5M	576k	8h 29m	715	11
18	Claude Opus 5.5	0.8783	$46.14	171.7M	451k	17h 18m	447	9
19	Claude Fable 5.1	0.8752	$40.65	67.2M	290k	7h 39m	274	9
20	Claude Haiku 5.5	0.8732	$11.20	186.6M	660k	13h 13m	398	1
21	Claude Fable 5.1	0.8680	$40.94	67.5M	295k	10h 23m	279	9
22	Claude Opus 5	0.8644	$47.78	71.9M	322k	17h 28m	282	6
23	Gemini 4 Argon	0.8617	$30.08	62.2M	527k	4h 53m	176	4
24	Claude Fable 5.1	0.8599	$41.23	67.7M	303k	9h 29m	256	9
25	Claude Haiku 5.5	0.8544	$5.81	88.2M	507k	6h 15m	252	3
26	Claude Fable 5	0.8542	$114.20	86.2M	385k	16h 7m	288	16
27	Claude Fable 5.1	0.8541	$59.53	101.1M	368k	11h 5m	353	7
28	Claude Opus 5	0.8538	$47.81	68.4M	372k	17h 11m	247	7
29	Claude Haiku 5.5	0.8532	$8.21	132.2M	569k	9h 50m	313	2
30	Gemini 4 Argon	0.8488	$43.62	67.5M	441k	6h 4m	194	5
31	Claude Opus 5	0.8444	$51.06	76.5M	351k	16h 37m	268	3
32	Claude Fable 5	0.8371	$94.87	69.6M	350k	17h 59m	274	7
33	Claude Haiku 5.5	0.8214	$7.68	123.8M	539k	9h 29m	317	2
34	Gemini 4 Argon	0.8155	$105.29	73.3M	505k	9h 43m	177	8
35	Gemini 4 Argon	0.7986	$85.40	56.4M	543k	6h 53m	150	4
36	Claude Haiku 5.5	0.7982	$10.12	165.9M	636k	11h 29m	354	3
37	Gemini 4 Argon	0.7982	$74.01	64.4M	466k	6h 51m	181	7
38	Claude Fable 5	0.7952	$83.84	60.4M	319k	16h 2m	237	9
39	GPT-5.6 Sol	0.7946	$58.77	77.9M	203k	6h 45m	423	8
40	Qwen3.8-Max-0902	0.7845	$41.21	136.5M	995k	20h 0m	480	2
41	GPT-5.6 Sol	0.7781	$75.11	86.9M	202k	10h 41m	410	11
42	Claude Fable 5	0.7664	$98.60	72.9M	355k	17h 31m	271	9
43	Qwen3.8-Max-0902	0.7658	$47.17	161.4M	937k	19h 56m	531	1
44	Claude Fable 5	0.7638	$95.31	70.8M	336k	16h 17m	265	7
45	Gemini 3.7 Flash	0.7591	$2.33	9.2M	139k	1h 43m	99	2
46	GLM-5.3	0.7535	$23.46	64.7M	277k	19h 3m	267	6
47	Kimi K3	0.7526	$40.44	107.3M	401k	19h 37m	387	17
48	Grok 4.6	0.7350	$123.13	116.9M	502k	9h 53m	414	28
49	Grok 4.7	0.7346	$28.62	27.0M	233k	2h 28m	119	7
50	GLM-5.3	0.7343	$45.48	90.2M	303k	16h 27m	365	4
51	GPT-5.6 Sol	0.7314	$114.07	131.6M	228k	8h 56m	569	14
52	Qwen3.8-Max-0902	0.7203	$52.67	179.1M	1.1M	19h 2m	520	2
53	GPT-5.6 Sol	0.7190	$154.11	169.8M	283k	6h 40m	688	13
54	Kimi K3	0.7151	$55.15	146.5M	497k	20h 0m	484	14
55	GPT-5.6 Sol	0.7125	$69.33	92.1M	185k	8h 7m	475	5
56	GLM-5.3 Flash	0.7029	$6.19	134.2M	441k	12h 1m	387	3
57	Kimi K3	0.6987	$79.07	224.2M	641k	20h 1m	556	12
58	GLM-5.3 Flash	0.6977	$3.78	63.8M	292k	10h 53m	273	5
59	GLM-5.3 Flash	0.6919	$5.75	129.8M	427k	15h 0m	361	4
60	Qwen3.8-Max-0902	0.6915	$35.54	115.2M	955k	20h 1m	444	1
61	GLM-5.3	0.6907	$44.40	135.6M	383k	15h 8m	392	12
62	Gemini 3.8 Flash	0.6886	$9.16	61.0M	246k	3h 25m	267	8
63	Grok 4.6	0.6883	$68.03	60.3M	544k	3h 28m	249	6
64	Qwen3.8-Max	0.6875	$16.94	52.6M	563k	18h 44m	340	2
65	Grok 4.7	0.6849	$35.29	35.5M	255k	1h 35m	157	10
66	Muse Spark 1.3	0.6821	$28.97	85.7M	430k	5h 33m	396	21
67	Muse Spark 1.3	0.6752	$165.04	162.2M	432k	18h 15m	706	25
68	Gemini 3.8 Flash	0.6750	$4.96	25.6M	195k	1h 47m	172	5
69	Grok 4.7	0.6735	$227.27	226.0M	738k	19h 38m	925	75
70	Grok 4.6	0.6689	$82.90	75.9M	477k	3h 6m	283	17
71	Gemini 3.7 Flash	0.6672	$3.04	15.3M	126k	1h 37m	140	1
72	Grok 4.6	0.6619	$46.47	48.9M	429k	4h 33m	274	7
73	Grok 4.7	0.6585	$21.77	22.6M	209k	1h 19m	122	7
74	Muse Spark 1.3	0.6538	$51.51	57.6M	507k	5h 41m	286	10
75	Gemini 3.7 Flash	0.6446	$3.44	16.9M	214k	2h 28m	151	2
76	Grok 4.7	0.6417	$24.01	24.2M	210k	1h 51m	131	4
77	Qwen3.8-Max-0902	0.6374	$51.28	174.8M	1.1M	20h 1m	568	2
78	GLM-5.3	0.6370	$18.59	55.0M	231k	19h 13m	287	3
79	Kimi K3	0.6270	$30.79	76.7M	394k	12h 20m	318	20
80	Kimi K3	0.6254	$97.66	258.3M	819k	19h 58m	576	17
81	Muse Spark 1.3	0.6245	$65.15	69.0M	497k	6h 34m	381	13
82	DeepSeek V4 Flash Vision Exp	0.6244	$2.53	133.2M	457k	10h 50m	402	7
83	Muse Spark 1.3	0.6205	$141.97	139.6M	348k	19h 5m	580	13
84	Gemini 3.8 Flash	0.6169	$10.88	84.0M	323k	4h 38m	300	2
85	Muse Spark 1.2	0.6114	$9.23	47.6M	407k	3h 16m	262	4
86	Gemini 3.7 Flash	0.6074	$16.26	72.9M	429k	6h 43m	280	5
87	GLM-5.3	0.6071	$32.20	106.0M	372k	12h 23m	375	5
88	Gemini 3.8 Flash	0.6021	$4.98	34.7M	202k	2h 25m	218	4
89	Muse Spark 1.2	0.5947	$9.36	45.3M	339k	3h 54m	244	2
90	Gemini 3.8 Flash	0.5930	$6.20	47.0M	194k	4h 21m	224	4
91	Grok 4.6	0.5924	$84.01	82.0M	496k	4h 20m	336	10
92	Muse Spark 1.2	0.5914	$9.21	41.8M	337k	3h 53m	230	2
93	Gemini 3.7 Flash	0.5912	$5.63	44.5M	188k	5h 26m	270	1
94	Muse Spark 1.2	0.5787	$10.64	52.8M	389k	2h 30m	272	8
95	Qwen3.8-Max	0.5728	$25.90	85.8M	648k	18h 2m	442	6
96	GLM-5.3 Flash	0.5673	$4.20	81.9M	369k	16h 1m	327	2
97	Muse Spark 1.2	0.5545	$7.70	42.7M	226k	2h 46m	257	5
98	DeepSeek V4 Flash Vision Exp	0.5485	$2.27	120.4M	400k	9h 24m	405	3
99	DeepSeek V4 Flash Vision Exp	0.4883	$3.26	177.3M	519k	9h 8m	426	7
100	DeepSeek V4 Flash Vision Exp	0.4768	$2.33	127.1M	362k	12h 24m	407	4
101	GLM-5.3 Flash	0.4693	$5.24	113.1M	401k	10h 10m	361	9
102	Qwen3.8-Max	0.4459	$25.19	78.5M	819k	19h 40m	374	2
103	DeepSeek V4 Flash Vision Exp	0.4348	$2.35	127.0M	382k	9h 0m	395	7
104	Inkling	0.4268	$4.27	4.3M	39k	1h 28m	101	2
105	Qwen3.8-Max	0.4074	$19.97	63.7M	596k	17h 56m	390	2
106	Qwen3.8-Max	0.2765	$43.05	138.5M	1.3M	20h 1m	512	1
107	Inkling	0.2712	$3.61	3.7M	31k	1h 49m	90	2
108	Inkling	0.2062	$7.66	7.9M	44k	1h 46m	158	2
109	Inkling	0.1413	$4.71	4.8M	39k	39m	98	2
110	Inkling	0.0057	$3.81	3.8M	60k	1h 38m	95	1

Reconnaissance Blind Chess Recovery

task page →110 runs · indexed 2026-10-08

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Opus 5.5	0.9929	$30.52	100.2M	393k	11h 17m	272	6
2	Claude Sonnet 5.5	0.9857	$55.39	234.7M	652k	13h 43m	437	4
3	GPT-6 Astra	0.9857	$244.69	120.7M	324k	13h 51m	408	10
4	Claude Opus 5.5	0.9786	$45.44	160.9M	500k	13h 11m	377	11
5	Claude Opus 5.5	0.9786	$34.33	116.2M	417k	11h 0m	318	8
6	Claude Sonnet 5.5	0.9750	$40.76	168.9M	538k	13h 54m	374	7
7	Claude Opus 5.5	0.9750	$38.61	139.5M	396k	12h 25m	373	6
8	Claude Sonnet 5.5	0.9643	$41.04	170.2M	534k	12h 48m	351	2
9	Claude Opus 5.5	0.9571	$39.47	142.7M	408k	13h 40m	381	7
10	Claude Fable 5.1	0.9536	$56.70	98.3M	311k	9h 23m	314	7
11	Claude Sonnet 5.5	0.9536	$50.70	214.9M	589k	14h 7m	425	5
12	Claude Fable 5.1	0.9536	$46.08	78.9M	273k	10h 59m	276	9
13	Claude Fable 5	0.9500	$67.06	43.5M	324k	14h 13m	175	3
14	Claude Sonnet 5.5	0.9464	$44.69	183.2M	625k	13h 17m	361	3
15	Claude Opus 5	0.9429	$45.45	65.6M	338k	12h 24m	221	3
16	Claude Fable 5.1	0.9429	$31.38	51.1M	239k	11h 53m	233	6
17	GPT-6 Astra	0.9429	$571.36	274.8M	479k	10h 52m	637	7
18	Claude Fable 5	0.9357	$59.50	39.9M	266k	13h 11m	181	6
19	Gemini 4 Argon	0.9286	$102.13	88.3M	440k	7h 8m	224	7
20	Gemini 4 Argon	0.9214	$93.51	158.0M	759k	9h 17m	426	13
21	Gemini 4 Argon	0.8964	$116.54	131.7M	765k	9h 49m	373	8
22	Gemini 4 Argon	0.8893	$85.61	94.9M	445k	9h 1m	199	6
23	GPT-5.6 Sol	0.8893	$28.42	45.1M	149k	6h 33m	291	7
24	Claude Fable 5	0.8821	$103.72	80.8M	300k	11h 27m	277	6
25	Claude Opus 5	0.8750	$46.27	65.7M	362k	13h 32m	222	5
26	Gemini 4 Argon	0.8679	$74.43	68.7M	419k	7h 35m	192	4
27	GPT-5.6 Sol	0.8607	$35.99	57.3M	138k	6h 13m	336	13
28	Grok 4.7	0.8571	$107.31	95.1M	427k	11h 56m	400	26
29	Claude Opus 5	0.8500	$61.18	94.0M	386k	14h 7m	294	4
30	Claude Opus 5	0.8429	$60.29	91.4M	395k	14h 7m	252	7
31	GPT-6 Astra	0.8429	$330.64	163.8M	407k	12h 13m	526	9
32	Claude Fable 5.1	0.8357	$41.84	71.9M	245k	9h 32m	252	11
33	GPT-6 Astra	0.8357	$771.41	367.5M	716k	14h 57m	903	12
34	Claude Haiku 5.5	0.8071	$9.96	166.2M	583k	11h 28m	342	6
35	GPT-5.6 Sol	0.8071	$20.62	32.5M	99k	10h 50m	196	10
36	GPT-5.6 Sol	0.8000	$20.53	30.5M	123k	2h 52m	197	3
37	Claude Fable 5.1	0.7929	$38.22	64.0M	257k	7h 37m	245	4
38	Grok 4.7	0.7821	$110.91	86.8M	288k	10h 45m	306	21
39	Claude Opus 5	0.7643	$37.81	52.1M	323k	13h 43m	213	4
40	Claude Fable 5	0.7607	$96.29	69.5M	362k	13h 19m	226	7
41	Kimi K3	0.7571	$29.23	77.1M	277k	14h 20m	314	2
42	GPT-6 Astra	0.7536	$12.03	4.8M	92k	34m	69	3
43	GPT-5.6 Sol	0.7357	$15.51	21.1M	106k	11h 38m	168	9
44	Grok 4.7	0.7250	$43.26	41.6M	230k	3h 58m	188	3
45	Claude Haiku 5.5	0.6857	$6.75	107.9M	485k	12h 11m	273	4
46	Claude Fable 5	0.6750	$77.86	54.6M	319k	13h 7m	214	4
47	Claude Haiku 5.5	0.6321	$6.72	103.6M	558k	12h 23m	236	4
48	Grok 4.6	0.6036	$87.29	70.2M	604k	8h 33m	259	9
49	Grok 4.7	0.5893	$172.69	157.2M	497k	15h 0m	627	13
50	Grok 4.6	0.5857	$104.51	99.3M	819k	14h 17m	453	24
51	Grok 4.7	0.5607	$45.18	44.4M	217k	3h 31m	197	8
52	Gemini 3.8 Flash	0.5536	$30.25	260.8M	509k	6h 38m	614	3
53	GLM-5.3	0.5357	$48.03	138.0M	410k	14h 33m	388	9
54	GLM-5.3 Flash	0.5250	$5.04	80.8M	333k	14h 2m	284	2
55	Kimi K3	0.5071	$31.51	82.4M	325k	14h 46m	311	4
56	Gemini 3.8 Flash	0.4714	$27.99	242.3M	700k	9h 7m	669	6
57	Muse Spark 1.3	0.4500	$83.30	135.3M	1.0M	14h 48m	459	4
58	Gemini 3.8 Flash	0.4500	$19.92	177.3M	430k	4h 7m	478	3
59	Kimi K3	0.4286	$27.43	64.3M	286k	14h 50m	256	11
60	Gemini 3.7 Flash	0.4250	$12.09	51.0M	293k	4h 11m	266	3
61	GLM-5.3	0.4214	$29.52	79.9M	302k	14h 26m	305	6
62	Grok 4.6	0.4071	$107.37	99.7M	852k	13h 36m	443	17
63	Muse Spark 1.2	0.4036	$5.41	24.7M	331k	4h 44m	150	2
64	Muse Spark 1.2	0.4036	$6.73	30.6M	391k	4h 23m	132	4
65	Muse Spark 1.2	0.3750	$11.18	57.5M	431k	6h 54m	254	6
66	GLM-5.3	0.3607	$36.05	73.5M	339k	13h 30m	259	4
67	Claude Haiku 5.5	0.3571	$5.00	75.9M	441k	13h 8m	222	3
68	Muse Spark 1.2	0.3429	$9.01	40.4M	451k	7h 47m	197	4
69	Claude Haiku 5.5	0.3357	$5.64	87.4M	456k	13h 10m	240	2
70	Kimi K3	0.3143	$28.91	69.9M	341k	12h 50m	253	2
71	Gemini 3.8 Flash	0.3107	$28.23	250.7M	518k	4h 50m	551	4
72	Gemini 3.7 Flash	0.3000	$23.11	214.3M	578k	5h 22m	524	1
73	GLM-5.3	0.2893	$25.26	83.8M	284k	14h 38m	339	4
74	Grok 4.6	0.2786	$78.34	65.1M	979k	10h 29m	301	8
75	GLM-5.3	0.2571	$27.02	69.8M	315k	14h 43m	253	6
76	Gemini 3.7 Flash	0.2393	$20.54	111.4M	562k	4h 43m	327	1
77	Qwen3.8-Max-0902	0.2321	$22.10	36.7M	1.2M	15h 6m	239	1
78	GLM-5.3 Flash	0.2214	$3.70	65.2M	342k	14h 4m	250	2
79	Muse Spark 1.2	0.2071	$9.69	39.5M	748k	4h 22m	138	7
80	Inkling	0.2000	$1.23	1.2M	15k	1h 37m	46	1
81	DeepSeek V4 Flash Vision Exp	0.1786	$2.02	107.0M	337k	14h 34m	328	2
82	Kimi K3	0.1429	$43.61	116.1M	397k	14h 40m	395	2
83	Qwen3.8-Max	0.1357	$20.20	59.6M	794k	13h 51m	345	3
84	Gemini 3.8 Flash	0.1214	$31.55	253.4M	764k	12h 9m	734	2
85	DeepSeek V4 Flash Vision Exp	0.1214	$2.99	164.7M	448k	8h 44m	423	5
86	GLM-5.3 Flash	0.1214	$5.52	97.5M	412k	13h 40m	313	9
87	Muse Spark 1.3	0.1107	$38.26	178.6M	566k	18h 5m	654	36
88	Grok 4.6	0.0929	$172.71	159.1M	1.1M	14h 32m	609	14
89	Qwen3.8-Max-0902	0.0929	$21.34	36.0M	1.2M	15h 8m	263	1
90	Inkling	0.0786	$1.37	1.4M	14k	43m	62	1
91	Qwen3.8-Max	0.0750	$16.70	49.8M	638k	13h 27m	335	3
92	Qwen3.8-Max-0902	0.0607	$20.03	44.0M	1.0M	15h 2m	256	1
93	Qwen3.8-Max	0.0536	$12.92	35.1M	623k	13h 47m	238	3
94	Gemini 3.7 Flash	0.0536	$30.51	186.7M	1.3M	5h 45m	513	2
95	Qwen3.8-Max	0.0536	$22.79	70.5M	780k	13h 7m	415	3
96	GLM-5.3 Flash	0.0500	$3.94	72.4M	357k	14h 22m	251	2
97	Gemini 3.7 Flash	0.0429	$30.88	192.0M	568k	8h 45m	506	2
98	Qwen3.8-Max-0902	0.0429	$15.23	34.3M	1.0M	15h 2m	220	1
99	Inkling	0.0393	$5.27	5.4M	45k	1h 49m	127	2
100	DeepSeek V4 Flash Vision Exp	0.0357	$2.71	145.6M	448k	14h 17m	377	7
101	Muse Spark 1.3	0.0286	$195.94	191.4M	452k	19h 7m	684	7
102	GLM-5.3 Flash	0.0179	$4.71	110.5M	429k	14h 10m	330	7
103	DeepSeek V4 Flash Vision Exp	0.0143	$2.49	130.6M	440k	11h 4m	400	10
104	Inkling	0.0143	$2.10	2.1M	21k	28m	83	1
105	Qwen3.8-Max-0902	0.0143	$12.30	21.2M	813k	15h 8m	188	1
106	Muse Spark 1.3	0.0107	$34.91	148.6M	508k	14h 14m	577	15
107	Qwen3.8-Max	0.0071	$26.05	82.8M	805k	13h 22m	436	3
108	DeepSeek V4 Flash Vision Exp	0.0036	$2.74	145.9M	463k	9h 28m	372	6
109	Muse Spark 1.3	0.0036	$74.56	160.5M	728k	11h 20m	628	10
110	Inkling	0.0000	$3.91	4.0M	36k	2h 12m	98	2

SGLang Inference System Optimization

task page →110 runs · indexed 2026-10-08

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	GPT-6 Astra	0.5349	$1954.03	921.6M	1.6M	19h 56m	1932	31
2	Gemini 4 Argon	0.4359	$139.21	309.9M	955k	8h 57m	746	6
3	Claude Sonnet 5.5	0.3720	$144.95	566.2M	2.4M	18h 50m	1343	11
4	Claude Haiku 5.5	0.1173	$15.92	254.5M	1.2M	10h 40m	625	2
5	Muse Spark 1.3	0.1159	$180.28	204.8M	347k	10h 27m	670	14
6	GPT-5.6 Sol	0.1092	$291.23	284.1M	251k	5h 12m	616	12
7	GPT-5.6 Sol	0.0675	$443.00	437.3M	281k	6h 23m	877	14
8	Grok 4.7	0.0552	$802.99	801.5M	2.3M	19h 22m	3183	24
9	GLM-5.3	0.0534	$68.77	241.2M	448k	15h 36m	650	8
10	Muse Spark 1.2	0.0532	$132.47	498.4M	1.4M	17h 13m	1444	3
11	Claude Fable 5	0.0442	$246.50	205.1M	519k	8h 17m	481	10
12	Grok 4.6	0.0322	$488.14	454.2M	2.3M	19h 33m	1703	83
13	Claude Opus 5	0.0249	$124.87	207.2M	534k	12h 23m	460	7
14	Qwen3.8-Max-0902	0.0242	$30.19	91.7M	767k	20h 6m	434	1
15	Claude Opus 5	0.0223	$101.43	161.0M	523k	12h 35m	390	8
16	Qwen3.8-Max	0.0214	$45.80	153.5M	1.1M	15h 59m	715	7
17	GLM-5.3	0.0211	$67.37	244.9M	383k	14h 17m	698	20
18	Gemini 3.8 Flash	0.0190	$30.46	330.6M	252k	3h 24m	709	11
19	Gemini 3.7 Flash	0.0169	$19.15	195.9M	289k	3h 33m	505	4
20	GPT-5.6 Sol	0.0168	$283.36	279.0M	247k	6h 47m	655	21
21	Kimi K3	0.0103	$110.14	311.1M	507k	20h 1m	740	11
22	Muse Spark 1.2	0.0099	$100.58	403.5M	1.7M	11h 46m	1193	8
23	Muse Spark 1.2	0.0081	$15.03	90.2M	152k	1h 36m	211	7
24	Qwen3.8-Max	0.0081	$33.42	99.7M	1.3M	15h 1m	574	8
25	Gemini 3.8 Flash	0.0058	$23.48	261.8M	192k	2h 8m	563	4
26	Claude Opus 5	0.0048	$110.39	179.3M	515k	12h 40m	431	7
27	Muse Spark 1.2	0.0047	$67.55	167.8M	657k	12h 12m	541	6
28	Grok 4.6	0.0025	$162.38	158.0M	937k	10h 27m	649	14
29	Gemini 3.8 Flash	0.0021	$39.53	434.9M	429k	7h 2m	1119	13
30	Grok 4.6	0.0012	$313.02	296.5M	1.9M	18h 41m	1120	19
31	Qwen3.8-Max-0902	0.0009	$28.62	81.5M	1.2M	20h 5m	391	1
32	Qwen3.8-Max-0902	0.0009	$26.00	76.8M	998k	20h 6m	387	2
33	Grok 4.6	0.0007	$96.00	91.3M	481k	3h 53m	329	3
34	Qwen3.8-Max	0.0004	$47.32	149.7M	1.5M	17h 59m	651	3
35	GLM-5.3	0.0004	$46.58	170.2M	369k	10h 37m	591	7
36	Claude Haiku 5.5	0.0000	$14.49	235.4M	958k	9h 43m	594	3
37	Claude Haiku 5.5	0.0000	$14.92	236.6M	1.1M	7h 37m	597	2
38	Claude Opus 5.5	0.0000	$159.94	559.4M	1.8M	19h 25m	1329	26
39	Claude Fable 5	0.0000	$202.86	169.0M	429k	14h 47m	436	14
40	GLM-5.3	0.0000	$50.83	180.8M	386k	12h 52m	576	4
41	Claude Fable 5.1	0.0000	$219.62	392.8M	963k	7h 58m	844	9
42	Muse Spark 1.2	0.0000	$32.81	65.0M	127k	4h 50m	276	3
43	Claude Fable 5.1	0.0000	$193.70	342.2M	936k	9h 59m	820	11
44	DeepSeek V4 Flash Vision Exp	0.0000	$2.44	138.4M	299k	9h 52m	531	4
45	Claude Haiku 5.5	0.0000	$13.92	215.0M	1.1M	9h 1m	513	3
46	DeepSeek V4 Flash Vision Exp	0.0000	$5.46	332.4M	492k	10h 53m	782	15
47	Grok 4.7	0.0000	$211.95	219.1M	541k	7h 14m	876	5
48	DeepSeek V4 Flash Vision Exp	0.0000	$3.59	206.9M	443k	9h 31m	593	16
49	Claude Opus 5.5	0.0000	$183.76	644.5M	2.0M	19h 7m	1476	23
50	Gemini 4 Argon	0.0000	$275.58	621.4M	1.5M	15h 16m	1391	22
51	Kimi K3	0.0000	$185.83	517.0M	1.5M	20h 0m	1188	13
52	GPT-6 Astra	0.0000	$2665.44	1249.7M	2.4M	19h 33m	2699	42
53	GPT-5.6 Sol	0.0000	$109.76	114.5M	139k	5h 57m	401	7
54	Claude Sonnet 5.5	0.0000	$131.29	505.6M	2.3M	19h 12m	1248	7
55	Grok 4.7	0.0000	$51.59	50.1M	157k	1h 43m	212	5
56	Gemini 4 Argon	0.0000	$100.53	344.7M	936k	8h 35m	828	8
57	Gemini 3.8 Flash	0.0000	$24.27	258.5M	198k	4h 49m	607	3
58	GLM-5.3 Flash	0.0000	$3.26	101.7M	249k	9h 20m	478	7
59	Claude Opus 5.5	0.0000	$185.38	644.4M	2.1M	18h 30m	1447	29
60	Claude Opus 5	0.0000	$89.33	141.7M	467k	15h 55m	376	11
61	Inkling	0.0000	$4.87	5.1M	12k	1h 40m	92	3
62	Grok 4.7	0.0000	$100.20	104.5M	326k	3h 50m	470	5
63	Claude Fable 5.1	0.0000	$177.26	316.2M	794k	11h 11m	833	9
64	Gemini 4 Argon	0.0000	$197.19	445.3M	1.3M	12h 41m	1022	9
65	Kimi K3	0.0000	$246.45	710.4M	1.8M	20h 0m	1736	14
66	Gemini 3.8 Flash	0.0000	$21.47	225.2M	181k	2h 13m	571	5
67	Muse Spark 1.3	0.0000	$429.35	454.7M	706k	19h 28m	1165	8
68	Claude Haiku 5.5	0.0000	$24.57	393.7M	1.7M	13h 34m	905	6
69	Grok 4.7	0.0000	$565.89	566.8M	1.5M	11h 16m	2189	10
70	GLM-5.3	0.0000	$68.85	206.5M	390k	14h 23m	607	17
71	Claude Sonnet 5.5	0.0000	$90.56	360.5M	1.4M	18h 46m	879	8
72	Inkling	0.0000	$7.60	8.0M	10k	1h 46m	120	2
73	Gemini 3.7 Flash	0.0000	$9.47	100.2M	149k	1h 43m	316	4
74	Muse Spark 1.3	0.0000	$58.36	60.2M	197k	8h 23m	386	3
75	GPT-5.6 Sol	0.0000	$182.18	175.0M	189k	6h 4m	485	8
76	Inkling	0.0000	$5.18	5.4M	12k	1h 28m	91	2
77	DeepSeek V4 Flash Vision Exp	0.0000	$3.41	198.5M	360k	8h 1m	562	4
78	Claude Opus 5.5	0.0000	$134.68	471.5M	1.5M	18h 23m	1059	21
79	Grok 4.6	0.0000	$419.76	405.9M	2.2M	19h 4m	1586	60
80	GPT-6 Astra	0.0000	$2441.52	1153.7M	2.1M	19h 6m	2553	41
81	Claude Fable 5.1	0.0000	$182.53	329.2M	746k	9h 31m	841	21
82	GPT-6 Astra	0.0000	$2574.70	1208.7M	2.3M	17h 31m	2622	65
83	GLM-5.3 Flash	0.0000	$5.76	124.8M	287k	14h 48m	473	3
84	Claude Opus 5.5	0.0000	$150.29	514.2M	1.7M	17h 0m	1206	15
85	Inkling	0.0000	$6.42	6.7M	16k	1h 12m	100	1
86	DeepSeek V4 Flash Vision Exp	0.0000	$3.54	209.2M	364k	8h 32m	632	9
87	GPT-6 Astra	0.0000	$1722.33	819.7M	1.5M	19h 18m	1844	42
88	Qwen3.8-Max	0.0000	$42.20	143.0M	952k	17h 51m	604	1
89	Kimi K3	0.0000	$49.64	141.5M	328k	6h 19m	529	7
90	Inkling	0.0000	$14.76	15.4M	36k	34m	194	2
91	Claude Sonnet 5.5	0.0000	$128.12	516.0M	1.9M	18h 29m	1195	8
92	Gemini 3.7 Flash	0.0000	$9.02	85.7M	143k	1h 52m	306	1
93	Gemini 4 Argon	0.0000	$160.21	500.1M	1.5M	13h 56m	1169	20
94	Claude Fable 5	0.0000	$289.82	249.1M	515k	7h 2m	543	12
95	GLM-5.3 Flash	0.0000	$3.52	102.4M	346k	8h 20m	418	2
96	Qwen3.8-Max	0.0000	$47.25	152.8M	1.4M	17h 10m	642	3
97	GLM-5.3 Flash	0.0000	$5.38	133.7M	325k	9h 34m	541	2
98	Gemini 3.7 Flash	0.0000	$12.58	90.8M	182k	4h 36m	317	1
99	Kimi K3	0.0000	$124.72	355.5M	911k	15h 41m	948	24
100	GLM-5.3 Flash	0.0000	$5.24	138.3M	282k	10h 25m	542	3
101	Qwen3.8-Max-0902	0.0000	$29.45	86.5M	1.2M	20h 1m	413	1
102	Claude Fable 5	0.0000	$175.12	145.0M	372k	12h 9m	427	14
103	Claude Sonnet 5.5	0.0000	$140.54	573.5M	2.0M	18h 42m	1322	5
104	Claude Fable 5	0.0000	$186.99	154.4M	396k	7h 35m	429	9
105	Muse Spark 1.3	0.0000	$52.36	191.1M	288k	7h 43m	680	5
106	Muse Spark 1.3	0.0000	$73.59	99.0M	263k	5h 24m	459	3
107	Claude Opus 5	0.0000	$158.96	248.3M	890k	16h 5m	693	15
108	Claude Fable 5.1	0.0000	$226.14	406.3M	953k	11h 14m	902	15
109	Gemini 3.7 Flash	0.0000	$6.58	51.9M	91k	1h 41m	296	4
110	Qwen3.8-Max-0902	0.0000	$22.37	62.1M	1.0M	20h 3m	330	1

Snooker Prediction

task page →110 runs · indexed 2026-10-08

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	GPT-6 Astra	0.6319	$2653.82	1245.9M	2.6M	19h 52m	2769	39
2	GPT-6 Astra	0.6316	$2364.83	1112.0M	2.4M	19h 31m	2539	51
3	GPT-6 Astra	0.6308	$1877.69	889.9M	1.7M	16h 9m	1997	31
4	GPT-6 Astra	0.6304	$348.12	162.7M	418k	3h 11m	442	4
5	Claude Opus 5.5	0.6243	$92.07	333.9M	907k	9h 5m	810	28
6	Gemini 4 Argon	0.6242	$58.63	165.8M	890k	6h 8m	418	2
7	GPT-6 Astra	0.6223	$321.27	149.5M	395k	3h 5m	401	4
8	Claude Opus 5.5	0.6219	$106.45	401.8M	928k	15h 53m	977	39
9	Claude Sonnet 5.5	0.6210	$169.39	673.2M	2.7M	19h 1m	1510	2
10	Claude Opus 5.5	0.6162	$99.16	365.9M	934k	14h 38m	876	24
11	Claude Sonnet 5.5	0.6129	$163.92	638.8M	2.8M	17h 56m	1433	20
12	Claude Sonnet 5.5	0.6127	$143.11	580.3M	2.1M	18h 52m	1315	29
13	Claude Sonnet 5.5	0.6036	$144.28	579.6M	2.2M	18h 20m	1325	8
14	Gemini 4 Argon	0.5920	$39.04	128.3M	687k	6h 49m	357	3
15	Claude Sonnet 5.5	0.5773	$121.18	495.6M	1.7M	19h 1m	1108	10
16	Claude Opus 5.5	0.5538	$119.85	454.9M	1.0M	14h 11m	1016	41
17	Gemini 4 Argon	0.5407	$63.97	191.0M	855k	6h 41m	464	3
18	Claude Opus 5.5	0.5128	$94.50	346.2M	909k	11h 7m	864	12
19	Claude Fable 5.1	0.5126	$87.17	154.8M	404k	6h 52m	408	12
20	Claude Opus 5	0.5001	$197.63	316.4M	1.0M	15h 45m	730	24
21	Claude Fable 5.1	0.4984	$96.24	173.0M	403k	7h 34m	486	21
22	GPT-5.6 Sol	0.4857	$119.18	111.7M	242k	2h 5m	299	11
23	GPT-5.6 Sol	0.4839	$170.16	165.0M	275k	4h 12m	432	9
24	Claude Fable 5.1	0.4822	$86.06	152.3M	409k	8h 44m	402	23
25	Claude Fable 5.1	0.4767	$88.31	156.7M	413k	3h 39m	397	10
26	Claude Opus 5	0.4607	$210.30	340.0M	1.1M	18h 37m	821	19
27	Gemini 4 Argon	0.4552	$134.67	137.5M	744k	5h 5m	372	3
28	GPT-5.6 Sol	0.4428	$157.87	150.8M	208k	2h 34m	354	9
29	Claude Fable 5	0.4405	$329.75	265.7M	842k	18h 32m	627	21
30	Claude Opus 5	0.4307	$172.30	276.6M	886k	18h 12m	665	10
31	GPT-5.6 Sol	0.4295	$132.32	126.4M	240k	4h 1m	369	6
32	Claude Haiku 5.5	0.4124	$12.19	184.2M	1.1M	9h 13m	452	3
33	Muse Spark 1.3	0.3826	$317.27	411.4M	767k	19h 50m	1032	4
34	Grok 4.6	0.3789	$466.11	391.2M	2.0M	19h 43m	1488	73
35	Claude Fable 5.1	0.3738	$86.02	153.2M	388k	5h 28m	424	9
36	Claude Fable 5	0.3600	$264.94	218.2M	625k	12h 41m	435	18
37	Muse Spark 1.3	0.3439	$236.90	672.0M	957k	19h 49m	1569	5
38	Claude Fable 5	0.3089	$197.41	157.5M	532k	18h 2m	369	13
39	Claude Haiku 5.5	0.3050	$9.22	147.7M	647k	7h 56m	307	2
40	Grok 4.6	0.3048	$106.12	86.0M	725k	5h 58m	378	5
41	GPT-5.6 Sol	0.2890	$179.27	154.6M	213k	9h 23m	357	5
42	Claude Haiku 5.5	0.2612	$6.70	101.8M	570k	4h 14m	252	2
43	Claude Opus 5	0.2573	$170.72	274.5M	864k	18h 25m	685	12
44	Claude Haiku 5.5	0.2379	$13.72	212.9M	1.2M	10h 26m	529	24
45	Gemini 4 Argon	0.2214	$64.59	161.0M	705k	6h 32m	420	3
46	GLM-5.3	0.1976	$71.12	251.5M	765k	18h 56m	654	15
47	Grok 4.6	0.1779	$108.45	87.9M	851k	6h 23m	344	15
48	Claude Fable 5	0.1348	$215.61	174.0M	561k	17h 21m	402	12
49	Claude Haiku 5.5	0.1195	$6.94	107.1M	566k	4h 29m	260	2
50	Grok 4.6	0.1175	$125.27	104.6M	768k	11h 27m	446	9
51	Gemini 3.8 Flash	0.1105	$14.99	133.6M	331k	1h 42m	403	3
52	GLM-5.3	0.1013	$57.78	201.7M	793k	14h 16m	531	13
53	Qwen3.8-Max	0.0882	$56.82	135.8M	1.2M	18h 54m	506	1
54	Gemini 3.8 Flash	0.0820	$13.08	117.4M	334k	1h 20m	368	3
55	Claude Opus 5	0.0762	$115.37	188.1M	559k	19h 9m	407	11
56	GLM-5.3	0.0590	$53.37	163.6M	430k	17h 18m	373	6
57	Muse Spark 1.3	0.0370	$445.62	445.7M	615k	19h 19m	857	8
58	Qwen3.8-Max-0902	0.0341	$31.78	57.0M	1.5M	20h 7m	298	1
59	Inkling	0.0335	$2.61	2.6M	26k	1h 16m	72	2
60	GLM-5.3	0.0311	$89.97	246.3M	519k	19h 15m	495	13
61	Claude Fable 5	0.0276	$244.97	201.6M	586k	17h 26m	457	16
62	Qwen3.8-Max-0902	0.0275	$32.69	59.4M	1.3M	20h 2m	303	1
63	Grok 4.6	0.0252	$77.16	60.3M	816k	4h 29m	271	13
64	Inkling	0.0203	$2.91	2.9M	46k	1h 53m	87	2
65	Grok 4.7	0.0086	$81.17	79.8M	486k	3h 8m	332	5
66	Kimi K3	0.0000	$126.53	346.0M	738k	19h 31m	778	8
67	Muse Spark 1.3	0.0000	$518.18	574.1M	1.1M	19h 44m	1290	6
68	Qwen3.8-Max	0.0000	$80.65	199.9M	2.1M	18h 39m	673	1
69	Gemini 3.7 Flash	0.0000	$7.82	66.5M	374k	1h 5m	279	1
70	Grok 4.7	0.0000	$342.95	338.0M	1.2M	19h 47m	1370	39
71	Qwen3.8-Max-0902	0.0000	$38.36	77.8M	1.3M	20h 2m	345	1
72	Gemini 3.7 Flash	0.0000	$8.33	59.1M	343k	2h 40m	282	3
73	Inkling	0.0000	$3.24	3.2M	40k	1h 45m	103	1
74	Gemini 3.7 Flash	0.0000	$5.61	40.2M	288k	59m	224	2
75	Gemini 3.8 Flash	0.0000	$8.11	76.9M	253k	1h 6m	299	5
76	Qwen3.8-Max-0902	0.0000	$30.67	66.2M	1.6M	20h 2m	268	1
77	Muse Spark 1.2	0.0000	$26.92	100.5M	603k	1h 48m	377	5
78	DeepSeek V4 Flash Vision Exp	0.0000	$4.83	277.6M	638k	5h 11m	644	13
79	Muse Spark 1.2	0.0000	$12.08	49.7M	275k	1h 6m	250	8
80	DeepSeek V4 Flash Vision Exp	0.0000	$3.23	179.6M	479k	19h 58m	466	1
81	GLM-5.3 Flash	0.0000	$8.45	217.9M	542k	13h 37m	520	3
82	DeepSeek V4 Flash Vision Exp	0.0000	$3.55	189.8M	629k	6h 0m	485	3
83	Qwen3.8-Max	0.0000	$115.97	289.9M	2.1M	19h 35m	811	1
84	Qwen3.8-Max	0.0000	$102.79	307.3M	1.7M	19h 10m	794	2
85	Gemini 3.7 Flash	0.0000	$6.64	48.6M	323k	1h 55m	247	1
86	Grok 4.7	0.0000	$282.03	285.1M	1.2M	19h 51m	1191	49
87	Qwen3.8-Max-0902	0.0000	$28.07	61.6M	1.3M	20h 0m	287	1
88	Kimi K3	0.0000	$140.38	372.9M	1.4M	19h 51m	925	26
89	Gemini 3.8 Flash	0.0000	$8.70	83.2M	282k	1h 15m	312	1
90	Inkling	0.0000	$2.84	2.7M	69k	45m	82	2
91	Muse Spark 1.2	0.0000	$16.11	78.5M	517k	1h 19m	309	4
92	Qwen3.8-Max	0.0000	$61.40	180.5M	1.6M	20h 1m	551	1
93	GLM-5.3 Flash	0.0000	$10.25	278.8M	597k	10h 31m	596	8
94	Inkling	0.0000	$1.80	1.8M	26k	14m	79	2
95	Muse Spark 1.2	0.0000	$13.70	75.9M	319k	1h 35m	305	3
96	GLM-5.3 Flash	0.0000	$13.39	288.1M	552k	17h 55m	640	19
97	Grok 4.7	0.0000	$61.25	64.2M	387k	4h 4m	335	5
98	Kimi K3	0.0000	$122.14	346.5M	798k	19h 47m	793	8
99	Gemini 3.7 Flash	0.0000	$7.40	56.9M	413k	1h 21m	266	2
100	DeepSeek V4 Flash Vision Exp	0.0000	$4.28	248.4M	549k	14h 16m	593	8
101	Kimi K3	0.0000	$117.31	316.7M	1.0M	19h 47m	806	6
102	Kimi K3	0.0000	$105.21	273.3M	660k	20h 0m	548	3
103	Grok 4.7	0.0000	$79.86	85.0M	390k	6h 4m	384	13
104	Gemini 3.8 Flash	0.0000	$17.30	177.6M	409k	1h 51m	471	5
105	Muse Spark 1.3	0.0000	$449.88	491.2M	832k	19h 52m	1176	2
106	GLM-5.3 Flash	0.0000	$7.94	248.0M	455k	19h 56m	595	7
107	GLM-5.3	0.0000	$94.15	290.7M	465k	18h 52m	627	11
108	DeepSeek V4 Flash Vision Exp	0.0000	$5.35	314.7M	642k	14h 37m	709	3
109	GLM-5.3 Flash	0.0000	$12.92	278.9M	847k	19h 47m	687	1
110	Muse Spark 1.2	0.0000	$16.58	87.1M	508k	2h 14m	321	6

SPICE Circuit Simulator in Rust

task page →110 runs · indexed 2026-10-08

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Gemini 4 Argon	0.9394	$221.62	329.8M	1.9M	11h 7m	731	3
2	Claude Opus 5.5	0.9293	$97.56	280.4M	1.6M	6h 22m	663	14
3	Claude Sonnet 5.5	0.9293	$109.70	401.0M	2.3M	10h 14m	905	22
4	Claude Sonnet 5.5	0.9293	$116.79	422.3M	2.6M	15h 25m	947	16
5	Claude Sonnet 5.5	0.9192	$157.47	595.8M	3.0M	11h 25m	1266	4
6	Claude Sonnet 5.5	0.9192	$114.22	425.5M	2.3M	14h 29m	922	18
7	Gemini 4 Argon	0.9091	$147.35	234.0M	1.0M	8h 39m	503	6
8	GPT-6 Astra	0.9091	$2887.40	1385.4M	2.5M	19h 50m	3121	180
9	Claude Opus 5.5	0.8990	$72.19	195.2M	1.3M	4h 28m	434	13
10	GPT-6 Astra	0.8990	$347.32	164.1M	405k	2h 56m	414	7
11	Claude Sonnet 5.5	0.8990	$108.09	389.8M	2.4M	13h 20m	873	12
12	GPT-6 Astra	0.8990	$599.18	284.8M	531k	4h 2m	616	7
13	GPT-6 Astra	0.8889	$677.88	324.1M	558k	4h 32m	694	7
14	Claude Fable 5	0.8889	$545.26	410.4M	1.8M	15h 21m	932	13
15	Gemini 4 Argon	0.8889	$270.20	399.0M	2.4M	13h 19m	883	6
16	Gemini 4 Argon	0.8788	$203.70	185.4M	882k	6h 7m	426	3
17	Claude Opus 5.5	0.8687	$65.56	170.6M	1.2M	3h 47m	399	6
18	Claude Opus 5.5	0.8687	$104.48	301.0M	1.7M	5h 14m	731	11
19	GPT-6 Astra	0.8687	$172.72	83.5M	308k	2h 47m	288	4
20	Claude Opus 5.5	0.8687	$69.51	187.1M	1.2M	4h 12m	427	15
21	Claude Fable 5.1	0.8586	$166.41	276.3M	1.2M	9h 57m	632	17
22	Claude Opus 5	0.8485	$441.42	701.9M	2.4M	13h 28m	1524	38
23	Claude Fable 5	0.8384	$451.15	353.8M	1.3M	8h 53m	733	11
24	Gemini 4 Argon	0.8283	$51.57	147.9M	741k	4h 35m	403	5
25	Claude Fable 5	0.8081	$505.54	407.1M	1.3M	17h 12m	857	16
26	Claude Opus 5	0.7879	$564.99	947.9M	2.4M	15h 22m	1980	33
27	Claude Fable 5	0.7778	$491.30	392.4M	1.3M	15h 56m	851	13
28	Claude Fable 5	0.7778	$474.28	378.9M	1.3M	17h 20m	800	12
29	Claude Opus 5	0.7677	$321.00	503.6M	1.9M	12h 46m	1092	31
30	Claude Fable 5.1	0.7677	$93.05	145.4M	837k	3h 54m	417	4
31	Grok 4.7	0.7576	$721.50	746.3M	2.8M	15h 40m	3094	16
32	Claude Opus 5	0.7475	$357.80	576.4M	1.9M	14h 22m	1233	23
33	Claude Opus 5	0.7475	$479.75	767.1M	2.6M	12h 42m	1557	22
34	Grok 4.7	0.7172	$780.09	794.5M	2.5M	20h 0m	3280	18
35	Claude Fable 5.1	0.7071	$96.79	153.4M	827k	4h 2m	413	8
36	Grok 4.7	0.7071	$747.81	772.9M	3.1M	17h 23m	3267	17
37	Claude Fable 5.1	0.6970	$69.08	107.8M	623k	2h 37m	227	2
38	Grok 4.7	0.6970	$807.64	821.4M	3.2M	17h 39m	3364	44
39	Claude Fable 5.1	0.6667	$96.05	148.5M	898k	4h 16m	406	6
40	Gemini 3.7 Flash	0.6263	$99.31	1037.7M	2.0M	6h 16m	2577	4
41	Grok 4.7	0.6061	$843.32	872.9M	3.2M	19h 50m	3636	16
42	Muse Spark 1.3	0.5758	$268.72	1376.2M	2.2M	19h 51m	3328	2
43	Grok 4.6	0.5657	$717.25	714.4M	2.7M	18h 22m	2483	266
44	Muse Spark 1.3	0.5657	$460.65	531.6M	1.1M	17h 48m	1347	4
45	Claude Haiku 5.5	0.5556	$14.47	224.0M	1.1M	2h 40m	557	2
46	Claude Haiku 5.5	0.5556	$12.80	192.7M	1.2M	2h 43m	510	4
47	Muse Spark 1.3	0.5455	$559.10	664.1M	1.1M	19h 47m	1540	5
48	Gemini 3.8 Flash	0.5354	$35.82	384.7M	639k	3h 11m	965	4
49	Claude Haiku 5.5	0.5354	$11.83	178.7M	1.1M	2h 15m	437	2
50	GLM-5.3	0.5354	$171.96	618.6M	1.7M	18h 15m	1441	4
51	Gemini 3.7 Flash	0.5253	$121.42	1230.5M	2.0M	12h 5m	2949	3
52	Grok 4.6	0.5253	$835.65	836.5M	3.1M	20h 0m	3128	205
53	GLM-5.3 Flash	0.5051	$24.75	719.3M	1.5M	17h 38m	1725	1
54	GLM-5.3	0.4949	$235.96	867.9M	1.6M	17h 11m	1905	18
55	Grok 4.6	0.4848	$586.10	582.0M	3.0M	17h 34m	2322	89
56	Gemini 3.7 Flash	0.4747	$76.85	790.2M	1.6M	4h 46m	2017	2
57	GLM-5.3 Flash	0.4646	$19.29	592.9M	1.1M	15h 32m	1400	4
58	Grok 4.6	0.4646	$1191.20	1187.2M	2.5M	19h 58m	3879	96
59	Muse Spark 1.3	0.4646	$547.17	608.4M	1.3M	19h 44m	1331	3
60	Grok 4.6	0.4545	$165.93	154.1M	860k	5h 37m	563	37
61	Gemini 3.7 Flash	0.4444	$80.51	766.1M	1.3M	6h 23m	1793	7
62	GPT-5.6 Sol	0.4444	$366.07	350.2M	455k	6h 25m	797	22
63	GLM-5.3	0.4444	$158.88	578.0M	1.3M	10h 53m	1155	4
64	Qwen3.8-Max-0902	0.4242	$87.16	281.5M	2.5M	19h 59m	601	1
65	Gemini 3.8 Flash	0.4242	$24.11	264.9M	400k	1h 50m	610	2
66	GLM-5.3	0.4141	$190.24	693.2M	1.5M	19h 41m	1705	30
67	GPT-5.6 Sol	0.4141	$307.32	296.1M	386k	4h 30m	783	15
68	Qwen3.8-Max-0902	0.4040	$64.29	188.8M	2.6M	20h 1m	516	1
69	GPT-5.6 Sol	0.3939	$538.95	528.2M	617k	6h 30m	1280	21
70	Muse Spark 1.3	0.3939	$557.80	621.9M	1.5M	19h 58m	1391	1
71	GLM-5.3 Flash	0.3737	$22.66	655.9M	1.7M	20h 0m	1644	1
72	Gemini 3.7 Flash	0.3737	$126.28	1246.3M	2.0M	12h 6m	2921	2
73	Qwen3.8-Max	0.3737	$145.98	524.1M	2.1M	19h 26m	1338	1
74	GLM-5.3 Flash	0.3737	$15.05	404.9M	975k	14h 18m	975	2
75	Qwen3.8-Max-0902	0.3737	$124.78	385.3M	4.3M	20h 1m	989	2
76	Claude Haiku 5.5	0.3636	$10.69	164.8M	919k	2h 31m	405	1
77	Claude Haiku 5.5	0.3636	$14.75	230.3M	1.1M	2h 56m	539	1
78	Qwen3.8-Max	0.3434	$127.04	463.9M	1.6M	19h 13m	987	4
79	GLM-5.3 Flash	0.3434	$27.30	749.1M	1.3M	19h 17m	1599	3
80	GPT-5.6 Sol	0.3333	$187.27	186.7M	262k	2h 15m	515	8
81	Gemini 3.8 Flash	0.3232	$15.32	160.0M	291k	1h 17m	426	3
82	GLM-5.3	0.3131	$152.26	545.3M	1.3M	16h 57m	1164	23
83	Qwen3.8-Max-0902	0.3030	$73.95	234.0M	2.3M	20h 1m	470	1
84	Qwen3.8-Max	0.3030	$96.60	342.7M	1.6M	18h 43m	760	2
85	Qwen3.8-Max	0.3030	$110.14	386.5M	2.0M	19h 56m	947	1
86	Qwen3.8-Max	0.2727	$126.58	455.7M	1.9M	19h 26m	951	7
87	GPT-5.6 Sol	0.2626	$226.84	221.2M	327k	3h 7m	592	13
88	Qwen3.8-Max-0902	0.2020	$65.44	194.8M	2.5M	19h 35m	444	2
89	Muse Spark 1.2	0.1313	$32.05	183.6M	731k	2h 8m	465	1
90	Gemini 3.8 Flash	0.0808	$2.16	15.8M	108k	23m	142	1
91	Muse Spark 1.2	0.0707	$10.81	53.3M	351k	1h 35m	223	2
92	Muse Spark 1.2	0.0707	$32.82	175.7M	815k	3h 21m	461	1
93	Muse Spark 1.2	0.0505	$8.77	48.8M	242k	1h 16m	196	2
94	Muse Spark 1.2	0.0404	$5.34	27.9M	188k	26m	143	2
95	Gemini 3.8 Flash	0.0101†	$2.26	17.9M	93k	20m	140	2
96	DeepSeek V4 Flash Vision Exp	0.0000	$11.65	704.6M	1.2M	10h 40m	1549	15
97	Kimi K3	0.0000	$179.57	474.0M	1.9M	19h 42m	1215	9
98	Kimi K3	0.0000	$215.15	588.2M	1.9M	19h 56m	1405	3
99	DeepSeek V4 Flash Vision Exp	0.0000	$19.65	1192.1M	2.0M	18h 3m	2398	10
100	Kimi K3	0.0000	$199.08	565.4M	1.4M	19h 58m	1271	1
101	Kimi K3	0.0000	$204.94	570.3M	1.7M	19h 34m	1399	3
102	Kimi K3	0.0000	$193.03	529.1M	1.8M	19h 58m	1263	1
103	DeepSeek V4 Flash Vision Exp	0.0000	$20.48	1199.6M	2.4M	11h 32m	2563	9
104	Inkling	0.0000	$4.42	4.6M	23k	45m	83	1
105	DeepSeek V4 Flash Vision Exp	0.0000	$21.51	1287.0M	2.2M	19h 10m	2904	13
106	Inkling	0.0000	$19.30	20.1M	40k	1h 20m	170	2
107	Inkling	0.0000	$1.04	1.1M	10k	27m	52	2
108	DeepSeek V4 Flash Vision Exp	0.0000	$19.68	1182.5M	2.0M	19h 8m	2617	23
109	Inkling	0.0000	$5.07	5.3M	18k	1h 13m	110	2
110	Inkling	0.0000	$5.21	5.3M	32k	15m	124	2

† score adjusted by manual review; the trace's Judge tab carries the measured score and the rationale.

Stepper Music Sequencer GBA

task page →110 runs · indexed 2026-10-08

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	GPT-6 Astra	0.9974	$3129.48	1515.4M	2.4M	19h 55m	3376	49
2	Claude Opus 5.5	0.9917	$247.64	942.3M	2.2M	18h 17m	2063	31
3	GPT-6 Astra	0.9897	$2759.22	1308.1M	2.5M	19h 59m	2942	38
4	Claude Opus 5.5	0.9883	$203.46	765.3M	1.9M	19h 3m	1757	33
5	Claude Opus 5.5	0.9874	$283.79	1107.1M	2.3M	19h 19m	2356	8
6	Claude Opus 5.5	0.9866	$313.81	1197.7M	2.7M	18h 20m	2708	44
7	GPT-6 Astra	0.9861	$2721.06	1296.9M	2.6M	19h 56m	2978	26
8	Claude Opus 5.5	0.9844	$258.85	972.6M	2.4M	18h 43m	2211	22
9	Claude Sonnet 5.5	0.9834	$194.76	804.6M	2.6M	19h 42m	1864	5
10	GPT-6 Astra	0.9793	$2755.43	1303.4M	2.5M	19h 59m	2916	37
11	Claude Sonnet 5.5	0.9789	$233.93	974.2M	3.0M	19h 16m	2141	2
12	Claude Sonnet 5.5	0.9751	$203.75	838.4M	2.8M	19h 30m	1864	1
13	Claude Sonnet 5.5	0.9732	$218.44	904.4M	2.9M	19h 31m	2018	2
14	Claude Sonnet 5.5	0.9677	$205.23	838.3M	3.0M	19h 27m	1947	6
15	Claude Fable 5.1	0.9662	$443.40	798.0M	1.8M	18h 14m	1802	27
16	GPT-6 Astra	0.9655	$2773.42	1319.2M	2.5M	19h 57m	2941	30
17	Gemini 4 Argon	0.9654	$544.95	521.5M	2.1M	21h 0m	1162	3
18	Claude Fable 5.1	0.9569	$440.73	799.7M	1.7M	15h 10m	1825	14
19	Claude Opus 5	0.9533	$570.71	935.1M	2.7M	19h 25m	2060	23
20	Gemini 4 Argon	0.9520	$272.84	638.9M	3.5M	20h 55m	1382	4
21	Claude Fable 5.1	0.9473	$365.73	654.8M	1.6M	15h 4m	1521	20
22	Claude Fable 5.1	0.9459	$457.39	831.4M	1.7M	14h 37m	1849	32
23	Gemini 4 Argon	0.9258	$445.36	558.0M	2.3M	20h 59m	1265	1
24	Gemini 4 Argon	0.9237	$409.68	593.6M	2.6M	20h 55m	1445	7
25	Gemini 4 Argon	0.9165	$187.49	667.9M	3.3M	20h 59m	1507	2
26	Claude Haiku 5.5	0.9129	$66.38	1065.9M	4.7M	16h 56m	2384	11
27	Claude Fable 5.1	0.9053	$549.23	1015.6M	1.7M	16h 2m	2234	18
28	Claude Opus 5	0.8947	$669.61	1119.9M	2.9M	19h 5m	2462	32
29	Claude Fable 5	0.8760	$652.52	521.9M	1.8M	18h 59m	1216	15
30	Claude Opus 5	0.8643	$526.73	857.9M	2.6M	14h 11m	1791	32
31	Claude Fable 5	0.8612	$916.45	747.8M	2.3M	18h 51m	1719	16
32	Claude Opus 5	0.8500	$580.43	975.9M	2.5M	18h 59m	2103	31
33	Claude Opus 5	0.8367	$450.82	756.2M	1.9M	15h 49m	1560	11
34	GPT-5.6 Sol	0.7701	$944.54	934.7M	1.1M	15h 1m	2411	21
35	Grok 4.7	0.7472	$628.06	649.9M	2.0M	11h 13m	2618	63
36	Claude Haiku 5.5	0.7427	$60.68	993.0M	4.0M	15h 28m	2269	13
37	GPT-5.6 Sol	0.7406	$409.56	413.9M	542k	8h 34m	1251	15
38	Claude Fable 5	0.7325	$725.08	587.9M	1.9M	18h 21m	1299	12
39	GPT-5.6 Sol	0.7219	$253.31	245.2M	326k	3h 19m	679	6
40	Claude Haiku 5.5	0.7208	$38.44	612.0M	2.8M	8h 23m	1325	2
41	Claude Fable 5	0.7151	$484.68	385.7M	1.3M	18h 44m	837	11
42	Claude Haiku 5.5	0.6803	$24.48	387.9M	1.9M	5h 8m	935	6
43	Claude Haiku 5.5	0.6608	$61.93	983.3M	4.6M	18h 17m	2156	15
44	Claude Fable 5	0.6534	$728.09	583.8M	2.0M	17h 53m	1264	6
45	GLM-5.3	0.6108	$356.80	1301.6M	3.0M	19h 1m	2671	19
46	GPT-5.6 Sol	0.6048	$566.59	550.5M	752k	14h 19m	1513	11
47	Grok 4.7	0.6012	$1385.34	1395.4M	3.2M	19h 29m	5064	67
48	Grok 4.6	0.5671	$305.84	301.9M	2.2M	11h 0m	1227	45
49	Muse Spark 1.3	0.5524	$214.49	1195.2M	2.0M	12h 19m	2846	1
50	Kimi K3	0.5098	$230.68	643.2M	1.9M	20h 0m	1545	3
51	GPT-5.6 Sol	0.4981	$490.17	481.7M	531k	5h 24m	1274	32
52	Grok 4.7	0.4771	$380.66	393.3M	1.3M	7h 10m	1597	32
53	GLM-5.3	0.4715	$281.50	1008.4M	2.3M	16h 6m	2264	16
54	Grok 4.7	0.4712	$1073.48	1101.7M	3.6M	19h 4m	4513	110
55	Grok 4.7	0.4415	$440.85	450.4M	1.7M	8h 48m	1863	19
56	GLM-5.3	0.4046	$276.19	1010.3M	1.8M	15h 31m	2070	21
57	Gemini 3.8 Flash	0.4044	$62.09	665.3M	979k	9h 34m	1663	2
58	Muse Spark 1.3	0.3727	$330.62	1484.1M	2.3M	19h 59m	3305	30
59	Kimi K3	0.3706	$227.60	629.2M	1.9M	19h 47m	1504	2
60	Gemini 3.8 Flash	0.3444	$76.61	826.4M	1.1M	12h 14m	1952	4
61	GLM-5.3	0.3427	$275.26	1018.0M	1.7M	17h 39m	2090	39
62	Grok 4.6	0.3420	$868.26	872.5M	3.4M	16h 30m	3114	373
63	Kimi K3	0.3170	$227.44	622.4M	1.9M	19h 52m	1439	1
64	Muse Spark 1.3	0.3146	$507.64	559.8M	1.1M	19h 23m	1354	2
65	Gemini 3.8 Flash	0.3068	$59.22	604.8M	923k	9h 13m	1507	4
66	Gemini 3.8 Flash	0.2640	$52.50	504.3M	779k	8h 9m	1188	4
67	Grok 4.6	0.2607	$223.96	223.8M	1.5M	7h 20m	882	18
68	Grok 4.6	0.2416	$205.23	195.4M	1.9M	7h 34m	792	28
69	Gemini 3.7 Flash	0.2415	$49.29	499.9M	1.0M	4h 48m	1302	2
70	Grok 4.6	0.2197	$129.50	120.8M	2.0M	7h 52m	540	19
71	GLM-5.3 Flash	0.2060	$16.51	502.4M	1.2M	10h 2m	1235	6
72	Gemini 3.7 Flash	0.1895	$40.63	436.9M	982k	3h 1m	1134	2
73	Gemini 3.7 Flash	0.1724	$88.83	943.5M	2.1M	6h 27m	2449	1
74	Gemini 3.7 Flash	0.1698	$29.14	318.3M	646k	2h 19m	856	2
75	Gemini 3.8 Flash	0.1660	$58.00	614.2M	934k	9h 4m	1564	2
76	GLM-5.3	0.1440	$63.57	232.5M	498k	3h 14m	487	6
77	DeepSeek V4 Flash Vision Exp	0.1348	$19.48	1192.0M	1.8M	11h 51m	2670	7
78	Kimi K3	0.1336	$196.38	508.8M	1.8M	19h 51m	1209	4
79	Gemini 3.7 Flash	0.1224	$55.17	592.3M	1.3M	5h 23m	1507	5
80	DeepSeek V4 Flash Vision Exp	0.1138	$28.55	1772.4M	2.4M	15h 1m	3833	23
81	GLM-5.3 Flash	0.1080	$14.12	436.5M	1.1M	11h 4m	982	7
82	Kimi K3	0.0909	$184.74	495.2M	1.5M	19h 58m	1154	2
83	GLM-5.3 Flash	0.0899	$16.14	483.9M	1.1M	17h 24m	1112	3
84	Muse Spark 1.3	0.0797	$528.52	575.6M	1.2M	19h 59m	1313	1
85	DeepSeek V4 Flash Vision Exp	0.0534	$18.46	1143.5M	1.6M	13h 42m	2700	16
86	Muse Spark 1.3	0.0480	$556.48	567.3M	735k	19h 51m	1107	5
87	DeepSeek V4 Flash Vision Exp	0.0419	$18.93	1164.8M	1.7M	11h 44m	2764	29
88	DeepSeek V4 Flash Vision Exp	0.0319	$17.22	1033.6M	1.8M	11h 35m	2306	13
89	GLM-5.3 Flash	0.0244	$12.30	358.8M	925k	17h 17m	894	1
90	Qwen3.8-Max	0.0240	$114.04	338.0M	2.8M	19h 40m	884	6
91	Qwen3.8-Max	0.0224	$119.80	401.3M	2.9M	20h 1m	934	—
92	GLM-5.3 Flash	0.0188	$8.33	220.9M	946k	17h 57m	605	1
93	Qwen3.8-Max	0.0183	$113.87	373.6M	3.0M	19h 42m	990	—
94	Muse Spark 1.2	0.0175	$37.47	155.3M	1.3M	5h 29m	460	7
95	Qwen3.8-Max	0.0159	$129.76	393.1M	2.8M	19h 53m	1102	—
96	Qwen3.8-Max	0.0002	$139.67	464.7M	2.7M	19h 56m	1203	—
97	Qwen3.8-Max-0902	0.0000	$43.55	102.3M	1.5M	20h 4m	428	1
98	Qwen3.8-Max-0902	0.0000	$38.27	81.2M	1.4M	20h 9m	421	1
99	Qwen3.8-Max-0902	0.0000	$38.10	91.8M	1.7M	20h 3m	413	1
100	Qwen3.8-Max-0902	0.0000	$31.88	60.8M	1.6M	20h 2m	390	1
101	Inkling	0.0000	$3.82	3.9M	21k	15m	103	2
102	Qwen3.8-Max-0902	0.0000	$41.37	105.1M	1.6M	20h 8m	436	1
103	Inkling	0.0000	$6.62	6.9M	28k	22m	168	1
104	Inkling	0.0000	$8.48	8.8M	35k	24m	171	2
105	Muse Spark 1.2	0.0000	$44.79	140.1M	936k	4h 21m	543	4
106	Inkling	0.0000	$10.64	11.1M	32k	19m	193	1
107	Muse Spark 1.2	0.0000†	$4.68	18.3M	123k	58m	156	2
108	Muse Spark 1.2	0.0000†	$53.19	95.9M	776k	4h 50m	351	2
109	Inkling	0.0000	$5.72	5.9M	31k	22m	140	2
110	Muse Spark 1.2	0.0000†	$10.59	43.3M	223k	2h 38m	259	3

† score adjusted by manual review; the trace's Judge tab carries the measured score and the rationale.

Synthetic Music Diarization

task page →110 runs · indexed 2026-10-08

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	GPT-6 Astra	0.5899	$503.76	243.1M	455k	15h 19m	591	11
2	GPT-6 Astra	0.5838	$329.84	157.1M	371k	18h 6m	426	13
3	GPT-6 Astra	0.5821	$102.57	50.4M	226k	17h 2m	214	8
4	GPT-6 Astra	0.5755	$313.33	152.4M	361k	17h 42m	459	12
5	Gemini 4 Argon	0.5622	$95.75	86.8M	749k	11h 34m	276	7
6	Gemini 4 Argon	0.5493	$59.76	63.4M	476k	9h 43m	201	5
7	Claude Sonnet 5.5	0.5485	$40.25	163.7M	585k	16h 59m	377	6
8	Claude Sonnet 5.5	0.5468	$52.30	219.6M	652k	16h 9m	456	6
9	Claude Opus 5.5	0.5460	$17.72	55.1M	249k	18h 32m	256	12
10	Claude Fable 5.1	0.5457	$12.11	18.5M	118k	17h 49m	156	8
11	Claude Sonnet 5.5	0.5456	$46.15	192.4M	590k	18h 34m	420	4
12	Gemini 4 Argon	0.5456	$46.26	59.1M	483k	7h 58m	169	5
13	Claude Sonnet 5.5	0.5444	$47.19	194.7M	641k	18h 8m	414	1
14	Claude Sonnet 5.5	0.5410	$38.75	155.4M	594k	16h 3m	363	8
15	Claude Opus 5.5	0.5369	$24.84	83.3M	302k	17h 50m	310	7
16	Claude Opus 5.5	0.5369	$16.97	51.1M	254k	18h 25m	241	10
17	Claude Opus 5	0.5368	$28.63	38.3M	258k	18h 34m	202	8
18	Claude Fable 5.1	0.5346	$11.41	17.0M	120k	17h 44m	158	7
19	Claude Opus 5	0.5328	$33.95	47.5M	267k	18h 38m	217	7
20	Gemini 4 Argon	0.5249	$103.39	100.4M	544k	9h 12m	246	8
21	GPT-5.6 Sol	0.5247	$44.19	53.3M	154k	7h 34m	267	3
22	Claude Opus 5	0.5241	$22.96	29.3M	223k	18h 54m	175	5
23	Claude Opus 5.5	0.5219	$17.38	51.6M	265k	17h 34m	243	8
24	Claude Haiku 5.5	0.5197	$5.78	94.3M	402k	14h 17m	306	2
25	Claude Fable 5	0.5170	$40.54	26.3M	193k	18h 24m	179	6
26	Claude Fable 5.1	0.5104	$17.60	28.7M	134k	17h 45m	230	11
27	Claude Fable 5	0.5096	$41.08	27.2M	188k	19h 7m	184	6
28	Claude Fable 5	0.5079	$49.45	33.2M	218k	18h 55m	196	8
29	GPT-5.6 Sol	0.5065	$80.65	78.5M	178k	15h 57m	339	11
30	GPT-6 Astra	0.5062	$228.20	112.0M	318k	15h 43m	379	15
31	GPT-5.6 Sol	0.4952	$59.62	69.7M	147k	15h 37m	359	3
32	Claude Opus 5.5	0.4919	$19.72	61.9M	269k	17h 35m	246	9
33	Claude Fable 5	0.4917	$51.32	35.9M	209k	18h 41m	228	6
34	Claude Fable 5.1	0.4905	$10.29	15.1M	114k	17h 40m	140	7
35	Claude Fable 5.1	0.4889	$10.86	15.6M	125k	15h 34m	138	6
36	Qwen3.8-Max-0902	0.4844	$67.12	233.9M	1.2M	19h 8m	645	4
37	Qwen3.8-Max-0902	0.4840	$91.60	324.7M	1.4M	19h 50m	793	1
38	Claude Haiku 5.5	0.4839	$3.57	55.3M	334k	17h 2m	236	5
39	Claude Opus 5	0.4833	$26.82	34.5M	255k	17h 35m	181	3
40	GLM-5.3	0.4816	$35.34	125.4M	361k	17h 30m	462	11
41	Qwen3.8-Max-0902	0.4814	$101.81	367.2M	1.4M	19h 10m	818	2
42	Claude Haiku 5.5	0.4805	$3.70	58.7M	322k	17h 54m	249	3
43	GLM-5.3	0.4802	$31.62	79.0M	303k	18h 9m	361	1
44	Gemini 3.7 Flash	0.4776	$32.86	129.1M	265k	12h 34m	438	2
45	GPT-5.6 Sol	0.4737	$78.78	76.5M	133k	14h 47m	286	4
46	GPT-5.6 Sol	0.4727	$68.59	82.8M	153k	14h 29m	425	7
47	Gemini 3.8 Flash	0.4726	$11.84	72.7M	176k	4h 47m	292	6
48	GLM-5.3 Flash	0.4723	$1.67	48.7M	181k	16h 18m	294	5
49	Gemini 3.7 Flash	0.4664	$11.27	42.4M	155k	7h 29m	254	2
50	Grok 4.6	0.4661	$133.75	133.0M	655k	19h 52m	578	42
51	Qwen3.8-Max	0.4605	$29.62	103.3M	529k	18h 43m	509	11
52	GLM-5.3	0.4602	$19.77	41.7M	196k	19h 9m	253	4
53	Grok 4.7	0.4597	$72.67	61.2M	249k	8h 6m	249	4
54	Muse Spark 1.3	0.4569	$29.57	105.7M	282k	17h 55m	519	8
55	Grok 4.7	0.4458	$66.85	49.2M	265k	14h 13m	205	10
56	Qwen3.8-Max	0.4425	$26.66	93.3M	455k	18h 13m	478	8
57	Gemini 3.7 Flash	0.4422	$14.36	133.2M	250k	5h 49m	389	2
58	Claude Haiku 5.5	0.4413	$4.58	75.8M	323k	15h 34m	288	3
59	DeepSeek V4 Flash Vision Exp	0.4412	$1.72	93.1M	282k	19h 4m	430	2
60	Grok 4.7	0.4406	$208.39	177.1M	399k	19h 30m	612	4
61	Gemini 3.8 Flash	0.4403	$11.00	95.0M	185k	4h 2m	334	5
62	Claude Haiku 5.5	0.4371	$5.40	88.7M	390k	16h 43m	316	2
63	Muse Spark 1.3	0.4340	$126.99	121.2M	301k	15h 7m	437	9
64	Grok 4.6	0.4323	$188.19	192.5M	1.1M	19h 45m	871	56
65	Kimi K3	0.4318	$13.36	33.2M	179k	18h 52m	244	5
66	Qwen3.8-Max	0.4301	$34.00	119.2M	572k	19h 17m	508	10
67	GLM-5.3	0.4285	$28.03	77.4M	258k	18h 22m	366	9
68	Grok 4.7	0.4284	$43.87	32.2M	197k	17h 17m	180	1
69	GLM-5.3	0.4265	$27.27	78.5M	240k	19h 37m	374	6
70	Gemini 3.7 Flash	0.4264	$42.51	256.4M	394k	13h 10m	681	9
71	DeepSeek V4 Flash Vision Exp	0.4247	$2.96	173.2M	352k	17h 55m	617	4
72	Muse Spark 1.3	0.4243	$152.31	160.5M	376k	14h 14m	500	7
73	Gemini 4 Argon	0.4215	$32.54	49.3M	491k	6h 13m	159	5
74	Muse Spark 1.3	0.4206	$121.45	118.8M	279k	16h 52m	510	21
75	Kimi K3	0.4157	$22.81	60.7M	202k	19h 10m	347	7
76	Grok 4.6	0.4120	$228.84	232.3M	993k	19h 44m	918	63
77	Grok 4.7	0.4074	$96.18	79.9M	265k	13h 36m	307	8
78	DeepSeek V4 Flash Vision Exp	0.4065	$1.73	95.5M	262k	17h 34m	448	5
79	Gemini 3.8 Flash	0.4043	$7.61	63.4M	182k	2h 25m	287	4
80	Qwen3.8-Max	0.4043	$33.94	117.1M	655k	18h 57m	501	5
81	Kimi K3	0.4034	$13.34	31.5M	191k	18h 19m	247	4
82	GLM-5.3 Flash	0.4027	$2.55	72.6M	225k	19h 1m	391	8
83	Qwen3.8-Max-0902	0.4015	$71.54	249.6M	1.3M	19h 38m	666	2
84	Gemini 3.7 Flash	0.4000	$15.67	83.4M	201k	4h 25m	322	1
85	Grok 4.6	0.3985	$299.53	304.5M	1.6M	19h 27m	1281	54
86	Muse Spark 1.3	0.3962	$92.96	91.1M	258k	17h 50m	454	10
87	Qwen3.8-Max	0.3951	$17.02	56.1M	424k	19h 10m	391	4
88	Gemini 3.8 Flash	0.3938	$11.34	62.3M	164k	3h 35m	300	12
89	GLM-5.3 Flash	0.3907	$3.73	72.3M	226k	17h 49m	356	3
90	Gemini 3.8 Flash	0.3868	$17.13	91.7M	227k	6h 49m	360	8
91	Kimi K3	0.3852	$16.48	39.9M	237k	19h 5m	234	2
92	Muse Spark 1.2	0.3829	$88.55	356.6M	541k	7h 12m	754	3
93	GLM-5.3 Flash	0.3822	$3.47	56.4M	213k	17h 57m	340	3
94	DeepSeek V4 Flash Vision Exp	0.3803	$1.85	99.9M	306k	18h 37m	455	3
95	Muse Spark 1.2	0.3711	$10.87	42.0M	256k	3h 30m	227	4
96	Qwen3.8-Max-0902	0.3707	$81.42	285.5M	1.4M	19h 28m	710	2
97	Kimi K3	0.3635	$16.00	39.1M	227k	18h 59m	234	6
98	Muse Spark 1.2	0.3611	$12.50	32.3M	167k	4h 4m	213	5
99	GLM-5.3 Flash	0.3570	$5.12	63.4M	220k	18h 51m	342	3
100	Muse Spark 1.2	0.3544	$13.53	50.0M	335k	4h 26m	248	4
101	Muse Spark 1.2	0.3434	$13.40	47.8M	217k	5h 21m	241	5
102	DeepSeek V4 Flash Vision Exp	0.2979	$2.02	115.0M	266k	19h 32m	508	4
103	Claude Fable 5	0.2496	$50.58	32.8M	242k	18h 51m	183	5
104	Claude Opus 5	0.2240	$27.38	35.8M	258k	19h 9m	186	9
105	Inkling	0.2089	$6.22	6.4M	44k	35m	131	2
106	Inkling	0.0477	$6.40	6.6M	38k	28m	123	1
107	Inkling	0.0074	$12.08	12.5M	45k	59m	160	1
108	Inkling	0.0053	$5.67	5.8M	48k	1h 31m	99	2
109	Inkling	0.0012	$9.51	9.9M	36k	38m	170	1
110	Grok 4.6	0.0000	$501.54	509.4M	1.9M	18h 17m	1937	96

Verilog Simulator in Swift

task page →110 runs · indexed 2026-10-08

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Opus 5.5	0.9521	$121.24	461.6M	1.1M	7h 20m	989	14
2	GPT-6 Astra	0.9503	$104.53	49.0M	213k	1h 48m	187	5
3	Claude Opus 5.5	0.9486	$185.48	723.5M	1.5M	11h 40m	1695	26
4	GPT-6 Astra	0.9469	$329.41	155.6M	366k	2h 26m	409	26
5	GPT-6 Astra	0.9452	$47.73	26.0M	162k	1h 5m	130	3
6	Claude Opus 5.5	0.9452	$119.27	459.4M	1.0M	7h 41m	1025	14
7	Claude Opus 5.5	0.9435	$112.90	428.3M	1.0M	8h 8m	989	11
8	Claude Opus 5.5	0.9401	$117.75	445.5M	1.1M	6h 29m	1024	13
9	Claude Sonnet 5.5	0.9384	$170.55	724.9M	2.0M	18h 31m	1739	12
10	Gemini 4 Argon	0.9349	$95.93	83.0M	397k	2h 56m	199	2
11	GPT-6 Astra	0.9349	$94.85	44.6M	216k	1h 33m	174	6
12	Claude Sonnet 5.5	0.9332	$135.20	567.6M	1.7M	12h 14m	1244	6
13	Claude Fable 5.1	0.9332	$262.30	478.6M	947k	9h 23m	1148	17
14	Claude Fable 5.1	0.9315	$216.99	391.4M	878k	7h 55m	966	6
15	Claude Sonnet 5.5	0.9315	$217.23	922.9M	2.5M	15h 59m	2176	16
16	Claude Fable 5	0.9298	$976.76	847.7M	1.7M	19h 7m	2003	24
17	Gemini 4 Argon	0.9281	$95.89	72.5M	365k	2h 52m	178	3
18	Gemini 4 Argon	0.9281	$83.88	72.7M	366k	2h 46m	166	3
19	Claude Sonnet 5.5	0.9281	$215.60	910.8M	2.6M	18h 12m	2122	20
20	Claude Fable 5	0.9264	$317.64	267.5M	691k	8h 11m	499	15
21	GPT-6 Astra	0.9264	$71.95	36.3M	192k	1h 35m	162	6
22	Claude Sonnet 5.5	0.9264	$186.02	788.1M	2.2M	13h 56m	1692	13
23	Claude Opus 5	0.9212	$529.35	935.4M	1.6M	17h 43m	2148	24
24	Gemini 4 Argon	0.9212	$27.80	68.8M	403k	2h 11m	165	4
25	Claude Fable 5.1	0.9195	$211.66	382.5M	840k	9h 40m	989	17
26	Claude Fable 5	0.9178	$1055.99	927.6M	1.7M	18h 46m	2232	22
27	Claude Opus 5	0.9178	$356.58	629.0M	1.1M	18h 31m	1391	16
28	Gemini 4 Argon	0.9178	$56.50	79.3M	431k	2h 53m	191	2
29	Claude Fable 5.1	0.9161	$266.36	483.7M	1.0M	10h 14m	1167	34
30	Claude Fable 5	0.9144	$317.29	267.0M	700k	14h 26m	483	17
31	Claude Opus 5	0.9127	$588.34	1052.2M	1.6M	19h 25m	2364	25
32	Claude Opus 5	0.9110	$505.22	890.6M	1.6M	16h 8m	2006	21
33	Claude Fable 5.1	0.9092	$168.96	308.4M	606k	6h 47m	586	10
34	Claude Fable 5	0.8973	$914.45	785.4M	1.7M	18h 23m	1843	35
35	Claude Haiku 5.5	0.8938	$55.29	968.3M	2.6M	12h 1m	2056	13
36	Claude Opus 5	0.8921	$377.04	660.7M	1.2M	15h 32m	1403	21
37	Grok 4.7	0.8853	$200.79	205.0M	994k	5h 44m	853	1
38	GPT-5.6 Sol	0.8853	$427.16	420.2M	394k	14h 37m	921	9
39	Claude Haiku 5.5	0.8818	$45.82	813.0M	1.9M	7h 26m	1721	4
40	Claude Haiku 5.5	0.8784	$81.94	1434.4M	3.8M	19h 26m	3162	45
41	Grok 4.7	0.8784	$244.06	246.8M	1.1M	15h 21m	1005	27
42	Claude Haiku 5.5	0.8767	$71.12	1257.3M	3.0M	15h 14m	2739	5
43	Claude Haiku 5.5	0.8733	$40.39	708.0M	1.8M	7h 30m	1550	2
44	GPT-5.6 Sol	0.8596	$271.89	260.9M	315k	4h 56m	650	5
45	Grok 4.7	0.8596	$839.06	861.8M	2.3M	19h 58m	3507	37
46	Grok 4.7	0.8510	$180.74	176.1M	878k	5h 58m	658	4
47	GPT-5.6 Sol	0.8493	$335.11	326.9M	349k	4h 23m	746	9
48	Grok 4.7	0.8476	$334.11	341.6M	1.2M	11h 19m	1457	47
49	Kimi K3	0.8373	$248.18	704.4M	1.6M	20h 0m	1636	11
50	GPT-5.6 Sol	0.8339	$184.09	180.1M	273k	4h 24m	480	6
51	Gemini 3.7 Flash	0.8305	$103.65	1018.6M	2.0M	19h 58m	2418	1
52	Gemini 3.8 Flash	0.8305	$76.00	821.2M	1.2M	6h 11m	1908	22
53	Gemini 3.8 Flash	0.8288	$123.53	1333.0M	1.9M	13h 23m	3053	62
54	Gemini 3.7 Flash	0.8288	$74.39	745.3M	2.1M	17h 44m	1828	1
55	Gemini 3.8 Flash	0.8253	$99.98	1070.4M	1.6M	7h 37m	2417	47
56	Grok 4.6	0.8253	$223.65	211.2M	1.9M	12h 53m	884	31
57	GLM-5.3	0.8236	$236.70	879.4M	1.3M	20h 1m	1924	11
58	GPT-5.6 Sol	0.8236	$181.11	176.3M	277k	4h 52m	476	3
59	Qwen3.8-Max-0902	0.8219	$182.82	643.3M	3.3M	19h 35m	1529	3
60	Gemini 3.7 Flash	0.8202	$75.29	725.4M	1.4M	19h 4m	1651	2
61	Gemini 3.8 Flash	0.8151	$108.32	1181.2M	1.8M	8h 5m	2671	41
62	GLM-5.3	0.8151	$195.25	722.0M	1.1M	19h 28m	1706	10
63	Grok 4.6	0.8116	$347.28	325.2M	2.2M	19h 55m	1222	56
64	Qwen3.8-Max-0902	0.8099	$199.79	724.6M	2.6M	19h 4m	1830	12
65	Gemini 3.8 Flash	0.8099	$133.85	1223.1M	1.9M	19h 59m	2834	11
66	GLM-5.3	0.7757	$229.93	817.6M	1.1M	19h 47m	1800	19
67	GLM-5.3	0.7723	$247.89	875.5M	1.2M	19h 55m	1804	17
68	Kimi K3	0.7551	$141.93	387.6M	1.1M	20h 1m	1009	1
69	Qwen3.8-Max	0.7551	$162.17	597.6M	1.8M	19h 37m	1491	1
70	GLM-5.3	0.7551	$285.37	1061.6M	1.4M	19h 46m	2367	23
71	Grok 4.6	0.7517	$518.10	508.9M	2.5M	19h 53m	1842	50
72	Kimi K3	0.7175	$256.44	747.8M	1.5M	20h 1m	1669	2
73	Grok 4.6	0.6901	$168.65	160.9M	1.6M	9h 35m	691	25
74	Kimi K3	0.6849	$265.41	774.0M	1.6M	19h 54m	1682	2
75	Qwen3.8-Max-0902	0.6473	$162.62	575.2M	2.7M	20h 1m	1239	2
76	Kimi K3	0.6250	$235.66	685.5M	1.4M	19h 49m	1521	1
77	GLM-5.3 Flash	0.6216	$20.18	641.6M	943k	19h 41m	1545	5
78	GLM-5.3 Flash	0.6062	$25.33	768.4M	1.0M	18h 54m	1712	18
79	Gemini 3.7 Flash	0.5976	$41.43	380.1M	776k	20h 1m	944	4
80	Muse Spark 1.3	0.5582	$533.21	614.5M	1.1M	19h 59m	1577	1
81	DeepSeek V4 Flash Vision Exp	0.5582	$23.11	1453.0M	1.8M	19h 40m	3118	3
82	Muse Spark 1.3	0.5497	$512.38	619.2M	1.1M	19h 59m	1598	1
83	Qwen3.8-Max-0902	0.5325	$172.11	623.9M	2.3M	19h 44m	1556	15
84	Grok 4.6	0.5188	$107.03	99.0M	943k	6h 25m	402	19
85	Qwen3.8-Max-0902	0.4966	$53.74	187.4M	939k	20h 0m	551	1
86	Muse Spark 1.3	0.4932	$275.58	1496.0M	1.9M	19h 49m	3361	3
87	Muse Spark 1.3	0.4932	$515.26	603.1M	1.3M	19h 58m	1473	2
88	GLM-5.3 Flash	0.4777	$15.24	482.1M	673k	19h 46m	899	5
89	GLM-5.3 Flash	0.4521	$17.29	524.8M	813k	19h 25m	1148	5
90	Muse Spark 1.3	0.4384	$522.03	583.4M	1.2M	19h 58m	1409	1
91	Qwen3.8-Max	0.4384	$128.73	471.1M	1.6M	19h 41m	965	2
92	DeepSeek V4 Flash Vision Exp	0.4212	$16.08	998.7M	1.4M	20h 0m	2128	4
93	DeepSeek V4 Flash Vision Exp	0.3784	$22.06	1380.9M	1.8M	19h 50m	2999	12
94	Gemini 3.7 Flash	0.3545	$53.22	180.3M	689k	19h 59m	574	1
95	Qwen3.8-Max	0.3527	$147.05	546.0M	1.5M	19h 49m	1109	2
96	DeepSeek V4 Flash Vision Exp	0.3253	$27.96	1735.1M	2.4M	17h 46m	3798	22
97	DeepSeek V4 Flash Vision Exp	0.3065	$23.64	1484.7M	1.9M	18h 25m	3157	30
98	Qwen3.8-Max	0.2894	$161.74	608.8M	1.4M	19h 56m	1406	2
99	GLM-5.3 Flash	0.2860	$25.19	814.7M	1.1M	18h 25m	1907	9
100	Qwen3.8-Max	0.2414	$54.19	188.0M	1.1M	20h 0m	597	1
101	Muse Spark 1.2	0.0445	$10.85	58.8M	321k	1h 31m	229	2
102	Muse Spark 1.2	0.0325	$16.81	84.3M	312k	1h 17m	329	2
103	Muse Spark 1.2	0.0120	$23.14	130.5M	679k	1h 19m	395	3
104	Muse Spark 1.2	0.0051	$4.27	21.5M	175k	51m	138	1
105	Inkling	0.0017	$19.93	20.8M	47k	1h 36m	283	1
106	Inkling	0.0000	$3.27	3.4M	20k	1h 3m	98	2
107	Inkling	0.0000	$13.36	13.9M	43k	1h 28m	227	1
108	Muse Spark 1.2	0.0000	$10.69	54.0M	468k	1h 50m	208	1
109	Inkling	0.0000	$2.17	2.2M	13k	1h 11m	80	1
110	Inkling	0.0000	$9.64	10.0M	34k	1h 28m	169	2

Vision-only TORCS Racing Bot

task page →110 runs · indexed 2026-10-08

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Opus 5.5	0.5284	$75.38	280.1M	713k	17h 50m	688	10
2	Claude Opus 5.5	0.5076	$73.95	265.4M	548k	18h 0m	556	10
3	GPT-6 Astra	0.4570	$770.42	373.2M	731k	19h 37m	1013	12
4	Claude Sonnet 5.5	0.4341	$69.62	285.7M	942k	19h 8m	690	6
5	GPT-6 Astra	0.3866	$1781.91	865.7M	1.5M	19h 55m	2133	25
6	Claude Sonnet 5.5	0.3811	$72.87	292.1M	964k	18h 50m	741	7
7	Claude Fable 5.1	0.3604	$65.32	119.8M	246k	14h 32m	399	16
8	Claude Fable 5.1	0.3524	$77.13	147.0M	311k	14h 51m	472	5
9	Claude Opus 5	0.3059	$79.30	127.1M	413k	18h 52m	388	8
10	Claude Sonnet 5.5	0.3045	$106.43	367.7M	1.1M	19h 5m	793	2
11	GPT-6 Astra	0.2803	$1564.06	755.2M	1.2M	19h 40m	1857	16
12	Claude Fable 5.1	0.2715	$77.56	137.7M	370k	10h 32m	426	7
13	Claude Opus 5.5	0.2616	$101.33	304.1M	515k	18h 58m	609	10
14	Claude Fable 5.1	0.2491	$104.18	191.5M	350k	15h 56m	562	8
15	Claude Sonnet 5.5	0.2220	$57.01	213.8M	846k	19h 22m	546	3
16	Claude Sonnet 5.5	0.2007	$76.43	316.3M	994k	19h 1m	751	7
17	Claude Fable 5	0.1944	$174.39	145.2M	388k	18h 8m	447	8
18	Claude Fable 5.1	0.1812	$69.87	126.5M	304k	15h 13m	418	9
19	Claude Opus 5	0.1513	$87.00	142.2M	403k	18h 34m	374	4
20	Claude Haiku 5.5	0.1276	$10.82	179.6M	761k	18h 58m	410	1
21	GPT-5.6 Sol	0.1207	$165.68	160.5M	163k	9h 38m	416	7
22	Grok 4.6	0.1206	$114.81	93.3M	882k	19h 41m	426	10
23	Qwen3.8-Max	0.1078	$31.56	67.3M	1.0M	19h 45m	456	2
24	Claude Opus 5	0.1073	$79.88	127.0M	429k	18h 13m	353	6
25	GPT-6 Astra	0.1048	$823.35	394.4M	766k	19h 59m	1018	14
26	GPT-6 Astra	0.0879	$1737.29	826.0M	1.5M	19h 53m	1937	16
27	Gemini 3.8 Flash	0.0797	$10.30	67.6M	209k	3h 53m	315	3
28	Claude Opus 5	0.0711	$105.95	145.9M	442k	17h 1m	380	4
29	DeepSeek V4 Flash Vision Exp	0.0671	$3.79	218.1M	451k	16h 2m	591	4
30	Kimi K3	0.0612	$51.38	138.7M	388k	19h 16m	456	9
31	Inkling	0.0559	$112.11	117.1M	218k	1h 53m	811	1
32	Claude Opus 5	0.0519	$94.80	144.0M	475k	17h 47m	401	7
33	Claude Haiku 5.5	0.0509	$4.70	70.8M	425k	18h 17m	231	2
34	GLM-5.3	0.0490	$52.08	118.9M	330k	18h 59m	354	2
35	Grok 4.7	0.0456	$436.54	434.7M	834k	19h 58m	1732	29
36	Claude Haiku 5.5	0.0451	$7.98	105.3M	596k	12h 11m	305	4
37	DeepSeek V4 Flash Vision Exp	0.0421	$4.04	235.7M	470k	19h 34m	605	5
38	GLM-5.3	0.0406	$79.08	158.6M	318k	18h 36m	406	8
39	Gemini 3.8 Flash	0.0405	$9.72	57.7M	157k	3h 40m	286	3
40	Muse Spark 1.3	0.0388	$56.56	261.0M	256k	19h 53m	846	5
41	Qwen3.8-Max	0.0385	$33.83	72.9M	764k	18h 52m	422	2
42	Claude Haiku 5.5	0.0383	$8.88	146.9M	548k	11h 16m	356	4
43	Kimi K3	0.0382	$61.72	163.7M	377k	19h 53m	534	3
44	GLM-5.3	0.0363	$101.89	234.7M	397k	19h 55m	487	1
45	Gemini 3.8 Flash	0.0345	$10.37	85.8M	196k	4h 25m	340	2
46	Grok 4.6	0.0335	$207.50	166.7M	1.1M	18h 40m	692	23
47	GLM-5.3	0.0278	$99.55	248.0M	367k	18h 54m	563	14
48	Gemini 3.7 Flash	0.0235	$5.06	24.8M	145k	2h 40m	197	1
49	Qwen3.8-Max	0.0216	$32.92	71.7M	721k	19h 16m	425	2
50	Kimi K3	0.0211	$44.44	114.5M	382k	20h 0m	386	1
51	Inkling	0.0186	$4.32	4.4M	39k	1h 1m	122	1
52	GLM-5.3	0.0180	$78.52	153.1M	353k	18h 42m	396	9
53	DeepSeek V4 Flash Vision Exp	0.0178	$3.85	221.0M	459k	11h 59m	555	7
54	Claude Haiku 5.5	0.0166	$6.71	95.4M	457k	13h 37m	272	2
55	Muse Spark 1.3	0.0144	$191.47	315.7M	497k	17h 52m	814	9
56	Kimi K3	0.0143	$59.95	154.4M	389k	20h 1m	507	2
57	Gemini 3.8 Flash	0.0136	$14.12	129.4M	330k	4h 0m	401	3
58	Muse Spark 1.2	0.0135	$59.82	110.7M	431k	11h 54m	331	6
59	GLM-5.3 Flash	0.0132	$3.71	111.1M	281k	17h 16m	405	3
60	Inkling	0.0129	$1.44	1.4M	20k	2h 0m	84	1
61	GLM-5.3 Flash	0.0128	$10.01	163.4M	383k	19h 11m	520	8
62	Muse Spark 1.2	0.0128	$52.14	70.4M	490k	14h 49m	289	4
63	Gemini 3.7 Flash	0.0127	$18.66	92.0M	417k	5h 34m	318	2
64	Grok 4.6	0.0122	$159.04	128.5M	975k	18h 46m	569	20
65	Gemini 3.7 Flash	0.0121	$11.51	63.9M	262k	4h 49m	310	2
66	Gemini 3.7 Flash	0.0118	$6.81	26.5M	177k	3h 38m	183	2
67	DeepSeek V4 Flash Vision Exp	0.0114	$4.73	279.3M	524k	19h 31m	681	4
68	GLM-5.3 Flash	0.0065	$8.04	140.3M	327k	17h 4m	460	6
69	Qwen3.8-Max	0.0061	$53.75	151.0M	965k	20h 0m	695	2
70	GLM-5.3 Flash	0.0056	$7.97	164.9M	369k	18h 14m	550	6
71	Muse Spark 1.2	0.0051	$32.90	57.9M	354k	9h 41m	372	3
72	GLM-5.3 Flash	0.0050	$8.45	168.8M	413k	16h 34m	519	5
73	Grok 4.6	0.0043	$218.28	183.9M	1.3M	14h 37m	752	15
74	Gemini 3.7 Flash	0.0041	$7.39	32.0M	185k	3h 17m	213	1
75	Inkling	0.0031	$5.44	5.4M	67k	1h 58m	120	2
76	Muse Spark 1.3	0.0029	$53.07	279.6M	311k	19h 59m	985	11
77	Inkling	0.0025	$2.42	2.4M	30k	2h 1m	89	2
78	Qwen3.8-Max-0902	0.0000†	$21.92	49.4M	903k	20h 6m	303	2
79	Muse Spark 1.3	0.0000†	$28.04	94.3M	152k	18h 59m	469	5
80	Gemini 4 Argon	0.0000†	$81.71	58.0M	243k	8h 45m	180	3
81	DeepSeek V4 Flash Vision Exp	0.0000†	$2.30	127.9M	331k	19h 43m	454	3
82	Grok 4.7	0.0000†	$129.68	133.6M	655k	14h 50m	603	11
83	Claude Opus 5.5	0.0000†	$60.58	240.2M	450k	18h 1m	571	11
84	Gemini 4 Argon	0.0000†	$54.62	48.6M	203k	6h 19m	169	2
85	Qwen3.8-Max-0902	0.0000†	$25.56	62.5M	975k	20h 8m	424	1
86	Qwen3.8-Max-0902	0.0000†	$48.14	156.8M	904k	19h 44m	531	1
87	Gemini 4 Argon	0.0000†	$123.80	83.2M	286k	8h 31m	204	3
88	Claude Fable 5	0.0000†	$81.94	61.7M	264k	17h 37m	293	2
89	GPT-5.6 Sol	0.0000†	$58.14	63.0M	111k	13h 6m	288	4
90	GPT-5.6 Sol	0.0000†	$25.75	38.4M	104k	16h 16m	218	4
91	Gemini 4 Argon	0.0000†	$78.71	63.3M	288k	10h 39m	188	5
92	Grok 4.7	0.0000†	$87.58	84.0M	222k	5h 40m	303	8
93	Grok 4.6	0.0000†	$57.50	52.5M	468k	7h 59m	252	3
94	Claude Fable 5	0.0000†	$92.42	70.7M	286k	18h 32m	313	5
95	Qwen3.8-Max	0.0000†	$18.02	51.8M	431k	20h 1m	433	2
96	Qwen3.8-Max-0902	0.0000†	$26.82	62.8M	787k	20h 4m	353	1
97	GPT-5.6 Sol	0.0000†	$51.99	57.9M	124k	4h 17m	277	6
98	Gemini 3.8 Flash	0.0000†	$7.30	51.8M	173k	3h 58m	288	4
99	GPT-5.6 Sol	0.0000†	$79.43	80.8M	113k	10h 50m	285	4
100	Qwen3.8-Max-0902	0.0000†	$28.34	75.5M	595k	20h 1m	372	1
101	Kimi K3	0.0000†	$56.92	159.4M	376k	20h 0m	521	1
102	Grok 4.7	0.0000†	$51.73	43.4M	216k	3h 56m	182	5
103	Muse Spark 1.3	0.0000†	$90.60	82.5M	232k	18h 53m	410	3
104	Claude Fable 5	0.0000†	$85.34	65.1M	256k	19h 10m	286	3
105	Gemini 4 Argon	0.0000†	$96.39	87.4M	314k	10h 9m	198	3
106	Muse Spark 1.2	0.0000†	$35.10	65.9M	265k	9h 20m	229	3
107	Grok 4.7	0.0000†	$353.91	323.1M	770k	19h 55m	1185	13
108	Claude Opus 5.5	0.0000†	$51.68	197.1M	439k	16h 52m	474	7
109	Muse Spark 1.2	0.0000†	$27.43	124.8M	316k	10h 1m	385	4
110	Claude Fable 5	0.0000†	$105.01	80.0M	325k	19h 12m	309	2

† score adjusted by manual review; the trace's Judge tab carries the measured score and the rationale.

Wan 2.1 on MAX/Mojo

task page →110 runs · indexed 2026-10-08

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Opus 5.5	0.9802	$29.40	86.6M	447k	5h 37m	213	11
2	Claude Opus 5.5	0.9629	$25.12	76.6M	348k	3h 15m	207	9
3	GPT-6 Astra	0.9623	$128.98	66.9M	202k	2h 32m	252	3
4	Claude Opus 5.5	0.9582	$32.70	105.1M	419k	5h 38m	261	4
5	Claude Sonnet 5.5	0.9507	$48.17	184.2M	875k	10h 49m	465	8
6	Claude Opus 5.5	0.9506	$43.10	137.3M	584k	6h 47m	287	8
7	Claude Fable 5	0.9491	$104.43	77.8M	325k	9h 34m	228	9
8	Claude Haiku 5.5	0.9458	$12.55	176.8M	1.3M	10h 31m	417	2
9	Claude Opus 5.5	0.9418	$33.25	107.7M	418k	4h 32m	251	4
10	Claude Sonnet 5.5	0.9377	$25.94	98.3M	468k	4h 54m	246	4
11	Claude Haiku 5.5	0.9373	$12.10	180.8M	1.1M	11h 0m	434	2
12	Claude Sonnet 5.5	0.9359	$40.87	157.9M	720k	8h 10m	372	6
13	Claude Sonnet 5.5	0.9357	$44.27	175.5M	715k	8h 40m	410	1
14	Claude Opus 5	0.9354	$108.05	175.6M	512k	12h 46m	386	16
15	Muse Spark 1.3	0.9348	$88.24	91.7M	303k	11h 17m	409	2
16	Claude Fable 5.1	0.9331	$62.26	106.8M	367k	8h 14m	284	8
17	Claude Fable 5	0.9315	$97.01	70.7M	318k	7h 13m	197	8
18	Claude Fable 5.1	0.9314	$52.87	90.3M	319k	5h 49m	274	9
19	Claude Opus 5	0.9312	$80.95	128.6M	410k	11h 23m	326	7
20	Kimi K3	0.9307	$34.42	90.2M	306k	9h 26m	306	5
21	Claude Fable 5	0.9285	$88.74	65.2M	284k	7h 6m	208	10
22	Claude Opus 5	0.9268	$66.75	102.7M	380k	7h 37m	277	3
23	Claude Fable 5	0.9266	$114.36	85.0M	368k	10h 16m	234	6
24	GPT-5.6 Sol	0.9241	$103.11	107.4M	168k	5h 28m	369	8
25	Grok 4.7	0.9226	$177.41	173.6M	635k	11h 45m	724	2
26	GPT-6 Astra	0.9217	$103.16	56.8M	189k	2h 7m	236	3
27	Claude Haiku 5.5	0.9211	$6.92	102.7M	574k	5h 59m	240	2
28	Claude Fable 5.1	0.9205	$68.80	118.8M	390k	9h 16m	323	6
29	Claude Sonnet 5.5	0.9205	$42.18	157.9M	816k	8h 38m	414	4
30	Kimi K3	0.9196	$29.20	75.3M	345k	8h 0m	286	8
31	Claude Fable 5.1	0.9192	$42.87	72.2M	281k	5h 13m	226	5
32	Claude Fable 5.1	0.9186	$55.53	95.2M	329k	8h 32m	279	10
33	Gemini 4 Argon	0.9182	$55.03	65.4M	246k	4h 0m	171	2
34	Claude Opus 5	0.9178	$120.16	201.7M	487k	16h 10m	440	12
35	GPT-6 Astra	0.9173	$183.34	92.0M	226k	1h 45m	305	2
36	Claude Fable 5	0.9155	$108.63	80.2M	355k	9h 49m	237	8
37	GPT-6 Astra	0.9153	$76.89	42.2M	156k	1h 3m	187	3
38	GLM-5.3	0.9144	$51.16	175.7M	465k	14h 14m	461	2
39	GPT-6 Astra	0.9128	$82.94	47.1M	162k	1h 37m	205	3
40	GLM-5.3 Flash	0.9128	$8.72	173.1M	406k	16h 39m	489	2
41	Claude Haiku 5.5	0.9123	$8.89	128.7M	893k	9h 43m	376	2
42	Gemini 4 Argon	0.9115	$102.53	78.3M	305k	5h 26m	181	2
43	Claude Haiku 5.5	0.9110	$9.22	144.9M	750k	8h 8m	333	6
44	GLM-5.3	0.9094	$42.10	140.4M	401k	16h 39m	404	2
45	Grok 4.7	0.9093	$209.09	202.2M	818k	13h 1m	844	2
46	Kimi K3	0.9090	$15.06	37.9M	188k	7h 2m	204	7
47	GLM-5.3 Flash	0.9090	$7.68	170.3M	428k	17h 42m	476	6
48	GLM-5.3	0.9087	$43.47	155.3M	374k	16h 47m	478	2
49	GPT-5.6 Sol	0.9087	$34.04	44.6M	108k	4h 5m	199	9
50	Grok 4.6	0.9085	$149.23	138.7M	1.3M	12h 2m	586	2
51	Claude Opus 5	0.9082	$82.70	130.5M	438k	8h 38m	327	9
52	Gemini 4 Argon	0.9073	$129.59	105.1M	283k	6h 53m	216	2
53	Gemini 4 Argon	0.9069	$23.60	61.0M	217k	3h 19m	175	2
54	GPT-5.6 Sol	0.9062	$69.52	78.1M	124k	3h 50m	295	3
55	Grok 4.7	0.9053	$77.01	77.8M	250k	3h 37m	315	1
56	Grok 4.6	0.9047	$29.23	30.9M	270k	3h 12m	158	1
57	Qwen3.8-Max-0902	0.9041	$56.78	149.1M	2.5M	19h 20m	497	2
58	GPT-5.6 Sol	0.9014	$57.11	66.4M	129k	4h 8m	267	5
59	Grok 4.6	0.9010	$76.51	66.4M	639k	9h 42m	273	2
60	Muse Spark 1.3	0.9005	$242.82	232.9M	452k	19h 26m	628	1
61	Qwen3.8-Max-0902	0.8953	$47.14	117.0M	1.9M	19h 7m	442	3
62	GLM-5.3 Flash	0.8916	$10.09	169.1M	376k	15h 54m	473	4
63	Kimi K3	0.8899	$31.51	84.6M	291k	12h 22m	348	3
64	GPT-5.6 Sol	0.8681	$66.06	72.8M	105k	4h 8m	267	4
65	Muse Spark 1.3	0.4425	$115.32	111.6M	449k	19h 59m	385	1
66	Muse Spark 1.3	0.4417	$126.25	155.2M	591k	19h 59m	465	1
67	Qwen3.8-Max	0.4407	$34.51	98.7M	1.3M	19h 41m	423	2
68	Qwen3.8-Max	0.4404	$41.19	95.2M	1.6M	20h 1m	427	1
69	DeepSeek V4 Flash Vision Exp	0.4396	$4.92	297.3M	464k	19h 11m	681	11
70	Muse Spark 1.3	0.4389	$215.78	199.3M	678k	15h 8m	502	24
71	DeepSeek V4 Flash Vision Exp	0.4384	$3.53	204.1M	419k	17h 55m	562	3
72	DeepSeek V4 Flash Vision Exp	0.3550	$2.99	168.7M	386k	19h 32m	453	3
73	Gemini 3.8 Flash	0.3530	$39.56	333.6M	688k	9h 57m	942	3
74	Qwen3.8-Max-0902	0.3268	$45.66	130.9M	1.8M	19h 28m	461	3
75	Qwen3.8-Max-0902	0.3247	$58.55	156.5M	2.2M	19h 45m	512	3
76	Gemini 3.8 Flash	0.3004	$49.60	408.7M	919k	17h 49m	1099	1
77	Gemini 3.8 Flash	0.2989	$50.71	448.6M	743k	11h 49m	1104	1
78	Qwen3.8-Max	0.2981	$45.25	145.8M	997k	19h 53m	526	1
79	GLM-5.3 Flash	0.2971	$8.91	144.0M	373k	19h 10m	444	3
80	Kimi K3	0.2838	$85.29	208.6M	954k	19h 56m	558	3
81	DeepSeek V4 Flash Vision Exp	0.2720	$3.61	208.7M	436k	14h 28m	562	11
82	Grok 4.6	0.2716	$127.41	108.5M	1.7M	19h 30m	464	4
83	DeepSeek V4 Flash Vision Exp	0.2506	$3.27	186.8M	413k	17h 5m	493	5
84	Grok 4.6	0.2296	$214.90	198.0M	1.7M	19h 41m	759	13
85	Gemini 4 Argon	0.1758	$81.73	98.5M	410k	6h 12m	259	1
86	GLM-5.3	0.0945	$60.08	134.7M	351k	20h 1m	432	1
87	Qwen3.8-Max	0.0664	$30.34	85.4M	1.2M	19h 11m	360	2
88	Qwen3.8-Max-0902	0.0446	$62.40	166.9M	2.3M	19h 54m	538	1
89	Inkling	0.0429	$17.12	17.9M	30k	2h 11m	192	1
90	Inkling	0.0404	$13.85	14.5M	23k	1h 11m	142	1
91	Gemini 3.7 Flash	0.0349	$23.63	178.6M	776k	6h 26m	569	1
92	Gemini 3.7 Flash	0.0347	$27.48	166.6M	453k	6h 47m	438	1
93	Gemini 3.7 Flash	0.0331	$42.63	248.9M	797k	11h 49m	716	1
94	Muse Spark 1.2	0.0325	$10.93	49.7M	405k	2h 44m	182	3
95	Gemini 3.8 Flash	0.0317	$65.98	570.2M	1.4M	17h 58m	1492	1
96	Gemini 3.7 Flash	0.0317	$28.92	149.4M	462k	8h 34m	380	1
97	Gemini 3.7 Flash	0.0310	$36.97	183.1M	560k	8h 49m	483	1
98	Gemini 3.8 Flash	0.0309	$52.78	424.0M	773k	11h 32m	1103	2
99	Inkling	0.0305	$12.93	13.5M	35k	14m	141	1
100	Muse Spark 1.2	0.0252	$38.37	181.5M	771k	3h 21m	481	13
101	GLM-5.3	0.0239	$77.39	173.5M	511k	19h 46m	464	11
102	Qwen3.8-Max	0.0000	$50.91	123.3M	1.9M	19h 56m	467	1
103	Muse Spark 1.2	0.0000	$29.55	111.8M	325k	3h 53m	331	2
104	Muse Spark 1.2	0.0000†	$16.08	77.6M	385k	3h 9m	254	2
105	Inkling	0.0000	$16.05	16.8M	29k	1h 23m	176	2
106	Grok 4.7	0.0000	$218.86	185.5M	568k	19h 57m	791	8
107	Grok 4.7	0.0000	$85.96	87.7M	287k	4h 37m	388	1
108	Muse Spark 1.2	0.0000	$10.80	49.7M	240k	3h 10m	226	3
109	Inkling	0.0000	$11.47	11.9M	30k	1h 18m	145	2
110	GLM-5.3 Flash	0.0000	$11.04	225.6M	397k	17h 7m	578	3

† score adjusted by manual review; the trace's Judge tab carries the measured score and the rationale.