Tasks Traces V1 Blog Changelog Contribute GitHub

Traces

Every run: the agent's full trajectory, its submissions over the 20-hour budget, and the final score.

Optimizer Design

task page →110 runs · indexed 2026-10-08

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Opus 5.5	0.3908	$28.07	87.7M	400k	17h 51m	286	12
2	Gemini 4 Argon	0.3700	$138.18	88.1M	358k	17h 59m	225	7
3	GPT-6 Astra	0.3513	$198.61	101.4M	290k	17h 55m	381	5
4	GPT-5.6 Sol	0.3487	$65.53	80.8M	133k	10h 22m	433	4
5	Claude Sonnet 5.5	0.3328	$24.92	96.2M	443k	18h 17m	313	2
6	Gemini 4 Argon	0.3302	$144.66	126.0M	552k	18h 30m	327	6
7	GPT-5.6 Sol	0.3194	$196.66	203.2M	202k	11h 49m	678	23
8	Kimi K3	0.3057	$15.06	37.2M	181k	19h 11m	275	4
9	Grok 4.7	0.2867	$182.37	175.0M	483k	20h 0m	744	15
10	GPT-5.6 Sol	0.2846	$55.29	72.7M	133k	15h 42m	423	5
11	Gemini 4 Argon	0.2842	$147.52	93.7M	408k	17h 44m	246	6
12	Claude Opus 5.5	0.2824	$26.98	82.6M	395k	14h 4m	282	6
13	GPT-5.6 Sol	0.2809	$71.65	93.2M	177k	14h 22m	523	21
14	Gemini 4 Argon	0.2729	$106.83	62.5M	361k	17h 2m	200	6
15	Claude Opus 5.5	0.2715	$26.87	83.2M	387k	12h 26m	281	8
16	Claude Sonnet 5.5	0.2671	$40.11	160.9M	619k	15h 53m	356	3
17	Gemini 4 Argon	0.2637	$128.39	105.4M	394k	12h 7m	261	6
18	Claude Opus 5.5	0.2621	$24.28	68.6M	403k	13h 10m	230	4
19	Claude Sonnet 5.5	0.2578	$28.93	110.0M	545k	14h 19m	282	2
20	Grok 4.6	0.2557	$99.14	98.0M	624k	19h 20m	424	18
21	GLM-5.3 Flash	0.2521	$2.91	48.3M	286k	16h 23m	241	3
22	Claude Opus 5	0.2499	$36.84	48.8M	334k	17h 28m	203	2
23	Claude Fable 5	0.2478	$59.89	43.2M	223k	15h 59m	229	10
24	Claude Sonnet 5.5	0.2467	$39.74	163.6M	545k	19h 1m	391	3
25	GLM-5.3	0.2349	$19.98	35.0M	224k	17h 41m	192	3
26	Qwen3.8-Max-0902	0.2342	$15.36	42.2M	714k	19h 22m	336	2
27	Claude Fable 5.1	0.2272	$15.45	22.4M	175k	11h 13m	164	5
28	GPT-5.6 Sol	0.2234	$31.29	52.9M	100k	13h 17m	373	2
29	GLM-5.3	0.2192	$17.37	36.9M	195k	19h 6m	241	2
30	Claude Sonnet 5.5	0.2147	$34.56	138.5M	535k	18h 9m	365	2
31	GLM-5.3	0.2082	$16.48	26.8M	198k	19h 23m	184	3
32	GLM-5.3 Flash	0.2047	$1.35	17.1M	199k	16h 48m	131	2
33	Grok 4.7	0.2033	$279.55	260.9M	639k	20h 0m	949	17
34	Grok 4.6	0.1957	$53.90	66.7M	503k	19h 24m	395	24
35	Muse Spark 1.3	0.1949	$17.67	14.5M	143k	19h 59m	150	1
36	GPT-6 Astra	0.1940	$334.17	159.8M	312k	18h 23m	412	7
37	Claude Haiku 5.5	0.1858	$3.96	58.4M	400k	17h 26m	217	2
38	Qwen3.8-Max-0902	0.1854	$14.00	33.7M	859k	19h 56m	315	1
39	GPT-6 Astra	0.1777	$89.95	48.3M	203k	19h 38m	244	10
40	Claude Opus 5	0.1760	$28.52	36.5M	275k	16h 52m	182	3
41	Claude Fable 5.1	0.1750	$17.15	26.3M	165k	12h 46m	182	6
42	Claude Opus 5.5	0.1731	$23.28	67.5M	372k	18h 2m	238	7
43	Claude Opus 5	0.1714	$43.94	57.7M	410k	17h 5m	200	4
44	Claude Haiku 5.5	0.1698	$7.09	114.6M	506k	14h 15m	325	3
45	Grok 4.6	0.1570	$59.53	57.5M	295k	19h 26m	258	11
46	Kimi K3	0.1475	$41.51	100.0M	508k	20h 1m	354	4
47	Claude Fable 5.1	0.1459	$14.90	22.4M	152k	15h 9m	171	4
48	Claude Haiku 5.5	0.1416	$5.30	82.3M	440k	13h 57m	256	1
49	Kimi K3	0.1370	$16.05	38.0M	228k	19h 25m	222	4
50	Grok 4.6	0.1368	$54.49	49.3M	462k	19h 16m	240	10
51	Qwen3.8-Max-0902	0.1351	$14.61	42.4M	583k	19h 28m	334	1
52	Claude Fable 5.1	0.1343	$11.08	15.3M	141k	13h 28m	134	3
53	Claude Opus 5	0.1316	$29.92	37.9M	299k	16h 54m	179	2
54	Gemini 3.7 Flash	0.1194	$22.13	241.0M	134k	10h 36m	834	2
55	Claude Fable 5	0.1170	$54.58	37.6M	230k	16h 38m	214	5
56	Claude Haiku 5.5	0.1133	$4.76	75.7M	379k	15h 43m	263	1
57	DeepSeek V4 Flash Vision Exp	0.1123	$1.12	52.3M	255k	18h 23m	256	2
58	Muse Spark 1.3	0.1110	$29.21	25.7M	102k	19h 53m	307	2
59	Claude Haiku 5.5	0.1082	$4.35	64.8M	433k	15h 17m	231	2
60	Muse Spark 1.3	0.1023	$23.80	20.6M	126k	19h 41m	260	2
61	Kimi K3	0.0990	$22.58	49.0M	297k	17h 53m	241	6
62	GLM-5.3	0.0984	$17.01	55.5M	261k	19h 4m	277	5
63	GLM-5.3	0.0965	$13.47	23.0M	159k	18h 59m	180	2
64	GLM-5.3 Flash	0.0964	$2.93	36.3M	234k	16h 39m	223	3
65	Qwen3.8-Max-0902	0.0958	$20.91	59.3M	909k	19h 56m	375	2
66	GPT-6 Astra	0.0929	$278.29	132.5M	303k	16h 52m	378	7
67	Claude Fable 5	0.0927	$57.48	40.3M	232k	16h 57m	210	4
68	GLM-5.3 Flash	0.0879	$2.67	37.1M	228k	16h 58m	207	2
69	Gemini 3.8 Flash	0.0861	$8.14	32.5M	139k	6h 7m	220	2
70	Gemini 3.8 Flash	0.0851	$12.84	46.8M	190k	10h 59m	216	5
71	Grok 4.6	0.0839	$80.45	75.7M	333k	19h 42m	279	3
72	Qwen3.8-Max	0.0827	$7.68	17.4M	516k	19h 49m	226	3
73	DeepSeek V4 Flash Vision Exp	0.0770	$1.22	63.5M	222k	19h 1m	334	2
74	Gemini 3.8 Flash	0.0755	$12.83	41.1M	192k	11h 46m	220	4
75	Qwen3.8-Max	0.0741	$7.61	18.8M	448k	19h 35m	241	3
76	Claude Fable 5	0.0728	$58.69	40.0M	246k	15h 56m	203	9
77	Qwen3.8-Max	0.0655	$6.87	15.9M	443k	19h 59m	203	3
78	Claude Fable 5	0.0619	$49.51	34.2M	205k	16h 37m	199	3
79	Gemini 3.7 Flash	0.0537	$10.37	85.1M	122k	13h 22m	435	2
80	GPT-6 Astra	0.0523	$53.35	35.6M	191k	19h 46m	218	6
81	Gemini 3.8 Flash	0.0506	$9.34	45.5M	175k	10h 6m	233	2
82	Gemini 3.7 Flash	0.0405	$25.34	217.9M	182k	10h 29m	744	2
83	DeepSeek V4 Flash Vision Exp	0.0348	$1.44	75.8M	242k	19h 19m	349	2
84	GLM-5.3 Flash	0.0292	$1.93	36.7M	215k	18h 0m	223	3
85	Claude Fable 5.1	0.0226	$15.74	21.7M	202k	14h 5m	141	4
86	Qwen3.8-Max	0.0143	$7.32	18.3M	421k	19h 31m	238	1
87	Grok 4.7	0.0119	$55.61	43.7M	222k	7h 51m	195	7
88	Gemini 3.8 Flash	0.0094	$13.99	75.0M	146k	11h 41m	384	5
89	Grok 4.7	0.0000	$308.24	304.6M	520k	20h 0m	1215	1
90	Muse Spark 1.2	0.0000	$39.25	85.0M	351k	13h 51m	275	7
91	Muse Spark 1.2	0.0000	$46.70	116.8M	475k	9h 50m	364	8
92	DeepSeek V4 Flash Vision Exp	0.0000	$1.51	79.4M	262k	19h 14m	406	2
93	Muse Spark 1.2	0.0000	$55.59	163.2M	672k	19h 37m	567	1
94	Gemini 3.7 Flash	0.0000	$28.88	259.5M	185k	18h 35m	908	1
95	Gemini 3.7 Flash	0.0000	$12.41	47.0M	144k	13h 48m	297	2
96	DeepSeek V4 Flash Vision Exp	0.0000	$1.33	65.8M	275k	19h 51m	313	1
97	Muse Spark 1.2	0.0000	$27.04	48.5M	345k	6h 22m	256	3
98	Grok 4.7	0.0000	$72.65	66.7M	251k	9h 7m	265	9
99	Muse Spark 1.3	0.0000	$14.04	11.9M	125k	19h 29m	179	1
100	Muse Spark 1.2	0.0000	$16.21	19.8M	148k	19h 57m	188	1
101	Inkling	0.0000	$1.08	1.1M	14k	1h 4m	58	1
102	Kimi K3	0.0000	$19.05	42.1M	297k	14h 45m	219	3
103	Inkling	0.0000	$1.53	1.5M	16k	1h 13m	73	2
104	Qwen3.8-Max-0902	0.0000	$24.43	66.6M	1.2M	19h 37m	381	2
105	Qwen3.8-Max	0.0000	$12.42	33.0M	636k	19h 59m	303	1
106	Inkling	0.0000	$3.84	3.9M	25k	5h 18m	124	1
107	Muse Spark 1.3	0.0000	$22.03	18.0M	123k	19h 58m	201	2
108	Inkling	0.0000	$2.54	2.6M	28k	2h 1m	88	1
109	Inkling	0.0000	$1.63	1.6M	27k	1h 46m	73	1
110	Claude Opus 5	0.0000	$34.43	42.6M	358k	17h 38m	173	3