Tasks Traces V1 Blog Changelog Contribute GitHub

Traces

Every run: the agent's full trajectory, its submissions over the 20-hour budget, and the final score.

Multi-GPU Efficient Finetuning

task page →110 runs · indexed 2026-10-08

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Inkling	0.5238	$11.78	12.1M	59k	1h 35m	166	1
2	Claude Opus 5.5	0.4286	$18.55	51.6M	316k	17h 25m	201	4
3	Muse Spark 1.2	0.3810	$64.94	126.7M	662k	16h 40m	386	27
4	Claude Sonnet 5.5	0.3571	$29.39	111.1M	559k	18h 36m	264	2
5	DeepSeek V4 Flash Vision Exp	0.3333	$0.82	36.7M	207k	19h 58m	238	1
6	Kimi K3	0.3333	$19.15	18.0M	131k	19h 9m	167	6
7	Claude Opus 5.5	0.3333	$13.03	35.0M	224k	18h 31m	179	5
8	Claude Opus 5.5	0.3095	$18.49	50.1M	325k	18h 23m	192	5
9	Muse Spark 1.2	0.3095	$11.95	41.8M	358k	12h 44m	199	24
10	Muse Spark 1.3	0.3095	$25.50	21.9M	121k	19h 28m	197	2
11	Kimi K3	0.3095	$16.56	15.3M	148k	19h 10m	134	3
12	Gemini 4 Argon	0.3095	$59.08	60.8M	237k	16h 56m	198	5
13	Claude Fable 5	0.2857	$23.66	14.1M	128k	18h 18m	119	2
14	GLM-5.3	0.2857	$12.88	26.8M	144k	19h 0m	234	2
15	Claude Fable 5.1	0.2857	$5.65	7.6M	75k	17h 4m	98	2
16	Claude Opus 5.5	0.2857	$21.94	61.0M	375k	16h 29m	207	3
17	Qwen3.8-Max	0.2857	$6.42	17.1M	316k	18h 44m	213	5
18	Claude Fable 5.1	0.2857	$5.38	7.9M	60k	16h 16m	118	3
19	Claude Haiku 5.5	0.2857	$3.49	48.9M	402k	16h 53m	163	3
20	GLM-5.3 Flash	0.2857	$0.68	10.0M	99k	15h 21m	124	3
21	GLM-5.3	0.2857	$10.42	18.4M	127k	19h 10m	175	6
22	Claude Opus 5	0.2857	$20.67	27.0M	193k	18h 45m	177	5
23	Claude Haiku 5.5	0.2619	$1.78	24.5M	255k	17h 19m	135	2
24	Claude Opus 5.5	0.2619	$17.33	45.0M	317k	18h 13m	178	4
25	GLM-5.3 Flash	0.2619	$1.17	15.8M	143k	16h 53m	134	3
26	Claude Fable 5	0.2619	$17.72	9.8M	108k	18h 5m	106	4
27	Muse Spark 1.2	0.2619	$355.31	2070.9M	1.9M	15h 46m	5920	22
28	GLM-5.3 Flash	0.2619	$0.97	12.5M	122k	17h 37m	127	2
29	Claude Sonnet 5.5	0.2619	$34.13	135.6M	549k	19h 19m	338	2
30	Claude Fable 5	0.2619	$30.03	18.9M	149k	19h 3m	152	5
31	Claude Fable 5.1	0.2619	$6.12	8.6M	75k	18h 44m	112	4
32	Qwen3.8-Max	0.2619	$6.15	16.6M	296k	17h 38m	220	10
33	Claude Opus 5	0.2381	$22.41	28.9M	215k	18h 43m	177	2
34	GPT-5.6 Sol	0.2381	$19.33	23.5M	110k	15h 47m	167	3
35	Claude Fable 5	0.2381	$19.07	11.3M	104k	19h 17m	120	8
36	GLM-5.3 Flash	0.2381	$1.36	29.9M	170k	18h 53m	219	2
37	Claude Opus 5	0.2381	$21.91	28.5M	201k	18h 44m	180	5
38	Claude Fable 5	0.2381	$17.66	9.8M	106k	18h 38m	106	6
39	Claude Haiku 5.5	0.2143	$2.46	34.8M	304k	17h 30m	158	3
40	Gemini 4 Argon	0.2143	$71.77	65.5M	249k	18h 37m	200	8
41	Claude Fable 5.1	0.2143	$5.26	7.1M	71k	18h 19m	92	3
42	Gemini 3.8 Flash	0.2143	$4.91	23.2M	78k	4h 58m	186	6
43	GPT-6 Astra	0.2143	$155.59	78.1M	233k	19h 27m	299	6
44	Claude Opus 5	0.2143	$16.33	19.4M	178k	17h 33m	142	3
45	Gemini 3.7 Flash	0.2143	$10.24	29.6M	72k	9h 49m	181	31
46	Claude Haiku 5.5	0.1905	$3.24	45.3M	385k	19h 9m	157	1
47	GPT-6 Astra	0.1905	$229.70	113.1M	288k	19h 53m	364	6
48	Muse Spark 1.2	0.1905	$5.52	20.2M	214k	2h 15m	135	3
49	Gemini 4 Argon	0.1905	$119.59	80.8M	260k	19h 19m	230	3
50	GPT-6 Astra	0.1905	$125.10	64.1M	212k	19h 30m	261	6
51	Kimi K3	0.1905	$16.13	11.8M	128k	18h 47m	116	1
52	Gemini 3.7 Flash	0.1667	$11.85	31.1M	102k	19h 53m	202	10
53	Claude Sonnet 5.5	0.1667	$34.90	136.3M	593k	18h 28m	314	3
54	Claude Sonnet 5.5	0.1667	$37.45	150.3M	578k	14h 47m	343	3
55	Claude Fable 5.1	0.1667	$6.92	9.6M	88k	17h 1m	101	4
56	DeepSeek V4 Flash Vision Exp	0.1429	$0.92	41.7M	226k	19h 50m	251	2
57	Muse Spark 1.3	0.1429	$16.18	14.5M	230k	20h 0m	159	1
58	Gemini 4 Argon	0.1429	$59.35	68.3M	219k	18h 18m	207	7
59	Gemini 4 Argon	0.1190	$99.16	76.3M	304k	18h 34m	233	4
60	Gemini 3.8 Flash	0.0952	$10.19	37.4M	109k	8h 17m	227	11
61	GPT-6 Astra	0.0714	$291.48	141.3M	320k	19h 38m	402	6
62	Claude Haiku 5.5	0.0714	$4.82	71.2M	466k	18h 3m	219	1
63	GLM-5.3	0.0714	$17.61	33.2M	221k	18h 56m	211	2
64	Qwen3.8-Max-0902	0.0476	$21.69	51.3M	683k	18h 46m	334	3
65	Muse Spark 1.2	0.0476	$35.64	132.6M	581k	10h 53m	451	13
66	Qwen3.8-Max-0902	0.0476	$21.74	63.3M	874k	19h 49m	397	3
67	Kimi K3	0.0476	$25.13	35.1M	233k	19h 37m	236	1
68	Inkling	0.0476	$5.35	5.5M	24k	1h 26m	132	2
69	GPT-5.6 Sol	0.0238	$19.65	25.0M	71k	12h 53m	133	3
70	Inkling	0.0238	$14.22	14.8M	45k	2h 19m	219	1
71	DeepSeek V4 Flash Vision Exp	0.0238	$0.66	29.1M	168k	19h 41m	226	2
72	Grok 4.7	0.0000	$60.21	36.3M	189k	18h 34m	184	2
73	Muse Spark 1.3	0.0000	$22.96	20.6M	169k	19h 58m	157	5
74	Qwen3.8-Max	0.0000	$4.65	11.0M	287k	19h 49m	183	5
75	Gemini 3.7 Flash	0.0000	$22.18	47.8M	107k	18h 31m	299	2
76	Grok 4.7	0.0000	$37.89	24.1M	164k	18h 10m	140	2
77	GPT-5.6 Sol	0.0000	$16.45	25.9M	78k	16h 50m	180	2
78	GLM-5.3	0.0000	$20.26	34.9M	208k	19h 31m	211	4
79	Claude Opus 5	0.0000	$20.70	27.2M	184k	19h 13m	173	3
80	Inkling	0.0000	$6.66	6.8M	46k	2h 1m	135	1
81	GPT-5.6 Sol	0.0000	$24.13	34.7M	95k	16h 10m	181	2
82	GPT-5.6 Sol	0.0000	$30.41	39.1M	90k	14h 8m	199	3
83	Grok 4.6	0.0000	$47.03	50.4M	373k	19h 22m	261	13
84	Qwen3.8-Max	0.0000	$3.67	9.9M	171k	19h 13m	153	3
85	Qwen3.8-Max-0902	0.0000	$6.94	8.9M	369k	5h 21m	120	1
86	GLM-5.3	0.0000	$17.67	32.5M	199k	19h 29m	215	2
87	Muse Spark 1.3	0.0000	$12.91	12.2M	103k	19h 52m	142	1
88	DeepSeek V4 Flash Vision Exp	0.0000	$0.76	37.0M	151k	19h 59m	275	1
89	Gemini 3.8 Flash	0.0000	$11.92	58.4M	116k	9h 56m	301	7
90	Qwen3.8-Max	0.0000	$7.44	21.5M	284k	19h 29m	253	2
91	Grok 4.6	0.0000	$18.53	24.3M	327k	17h 14m	194	7
92	DeepSeek V4 Flash Vision Exp	0.0000	$1.23	60.8M	248k	19h 49m	323	1
93	Qwen3.8-Max-0902	0.0000	$21.73	60.3M	968k	19h 32m	366	3
94	Grok 4.6	0.0000	$32.88	28.8M	249k	18h 12m	196	5
95	Muse Spark 1.3	0.0000	$30.08	27.4M	216k	19h 49m	204	1
96	Gemini 3.7 Flash	0.0000	$8.30	27.1M	102k	9h 38m	188	10
97	GPT-6 Astra	0.0000	$142.95	74.8M	253k	19h 46m	311	7
98	Grok 4.6	0.0000	$23.95	18.0M	271k	19h 15m	145	4
99	Inkling	0.0000	$4.16	4.3M	15k	34m	107	3
100	Kimi K3	0.0000	$10.94	8.7M	101k	18h 40m	121	1
101	Gemini 3.8 Flash	0.0000	$9.46	51.9M	115k	6h 1m	268	6
102	Claude Sonnet 5.5	0.0000	$32.34	129.9M	492k	18h 23m	326	2
103	Grok 4.7	0.0000	$29.88	18.4M	159k	14h 59m	119	3
104	Gemini 3.7 Flash	0.0000	$30.31	73.0M	132k	18h 49m	310	13
105	GLM-5.3 Flash	0.0000	$2.31	42.4M	198k	17h 43m	261	2
106	Qwen3.8-Max-0902	0.0000	$37.59	115.3M	1.0M	19h 35m	541	3
107	Grok 4.7	0.0000	$36.18	19.8M	164k	20h 0m	126	5
108	Gemini 3.8 Flash	0.0000	$14.98	62.0M	118k	8h 40m	295	7
109	Grok 4.7	0.0000	$59.65	35.5M	185k	17h 44m	170	3
110	Grok 4.6	0.0000	$20.07	18.4M	217k	18h 24m	156	8