Traces

Every run: the agent's full trajectory, its submissions over the 20-hour budget, and the final score.

Optimizer Design

task page →50 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	GPT-5.6	0.3487	$65.53	80.8M	133k	10h 22m	433	4
2	GPT-5.6	0.3194	$196.66	203.2M	202k	11h 49m	678	23
3	Kimi K3	0.3057	$15.06	37.2M	181k	19h 11m	275	4
4	GPT-5.6	0.2846	$55.29	72.7M	133k	15h 42m	423	5
5	GPT-5.6	0.2809	$71.65	93.2M	177k	14h 22m	523	21
6	Grok 4.6	0.2557	$99.14	98.0M	624k	19h 20m	424	18
7	GLM-5.3	0.2349	$19.98	35.0M	224k	17h 41m	192	3
8	Claude Fable 5.1	0.2272	$15.45	22.4M	175k	11h 13m	164	5
9	GPT-5.6	0.2234	$31.29	52.9M	100k	13h 17m	373	2
10	GLM-5.3	0.2192	$17.37	36.9M	195k	19h 6m	241	2
11	GLM-5.3	0.2082	$16.48	26.8M	198k	19h 23m	184	3
12	Grok 4.6	0.1957	$53.90	66.7M	503k	19h 24m	395	24
13	Claude Fable 5.1	0.1750	$17.15	26.3M	165k	12h 46m	182	6
14	Grok 4.6	0.1570	$59.53	57.5M	295k	19h 26m	258	11
15	Kimi K3	0.1475	$41.51	100.0M	508k	20h 1m	354	4
16	Claude Fable 5.1	0.1459	$14.90	22.4M	152k	15h 9m	171	4
17	Kimi K3	0.1370	$16.05	38.0M	228k	19h 25m	222	4
18	Grok 4.6	0.1368	$54.49	49.3M	462k	19h 16m	240	10
19	Claude Fable 5.1	0.1343	$11.08	15.3M	141k	13h 28m	134	3
20	Gemini 3.7 Flash	0.1194	$22.13	241.0M	134k	10h 36m	834	2
21	DeepSeek V4 Flash Vision Exp	0.1123	$1.12	52.3M	255k	18h 23m	256	2
22	Kimi K3	0.0990	$22.58	49.0M	297k	17h 53m	241	6
23	GLM-5.3	0.0984	$17.01	55.5M	261k	19h 4m	277	5
24	GLM-5.3	0.0965	$13.47	23.0M	159k	18h 59m	180	2
25	Grok 4.6	0.0839	$80.45	75.7M	333k	19h 42m	279	3
26	Qwen3.8-Max	0.0827	$7.68	17.4M	516k	19h 49m	226	3
27	DeepSeek V4 Flash Vision Exp	0.0770	$1.22	63.5M	222k	19h 1m	334	2
28	Qwen3.8-Max	0.0741	$7.61	18.8M	448k	19h 35m	241	3
29	Qwen3.8-Max	0.0655	$6.87	15.9M	443k	19h 59m	203	3
30	Gemini 3.7 Flash	0.0537	$10.37	85.1M	122k	13h 22m	435	2
31	Gemini 3.7 Flash	0.0405	$25.34	217.9M	182k	10h 29m	744	2
32	DeepSeek V4 Flash Vision Exp	0.0348	$1.44	75.8M	242k	19h 19m	349	2
33	Claude Fable 5.1	0.0226	$15.74	21.7M	202k	14h 5m	141	4
34	Qwen3.8-Max	0.0143	$7.32	18.3M	421k	19h 31m	238	1
35	Muse Spark 1.2	0.0000	$39.25	85.0M	351k	13h 51m	275	7
36	Muse Spark 1.2	0.0000	$46.70	116.8M	475k	9h 50m	364	8
37	DeepSeek V4 Flash Vision Exp	0.0000	$1.51	79.4M	262k	19h 14m	406	2
38	Muse Spark 1.2	0.0000	$55.59	163.2M	672k	19h 37m	567	1
39	Gemini 3.7 Flash	0.0000	$28.88	259.5M	185k	18h 35m	908	1
40	Gemini 3.7 Flash	0.0000	$12.41	47.0M	144k	13h 48m	297	2
41	DeepSeek V4 Flash Vision Exp	0.0000	$1.33	65.8M	275k	19h 51m	313	1
42	Muse Spark 1.2	0.0000	$27.04	48.5M	345k	6h 22m	256	3
43	Muse Spark 1.2	0.0000	$16.21	19.8M	148k	19h 57m	188	1
44	Inkling	0.0000	$1.08	1.1M	14k	1h 4m	58	1
45	Kimi K3	0.0000	$19.05	42.1M	297k	14h 45m	219	3
46	Inkling	0.0000	$1.53	1.5M	16k	1h 13m	73	2
47	Qwen3.8-Max	0.0000	$12.42	33.0M	636k	19h 59m	303	1
48	Inkling	0.0000	$3.84	3.9M	25k	5h 18m	124	1
49	Inkling	0.0000	$2.54	2.6M	28k	2h 1m	88	1
50	Inkling	0.0000	$1.63	1.6M	27k	1h 46m	73	1