Traces

Every run: the agent's full trajectory, its submissions over the 20-hour budget, and the final score.

libexpat Optimization

task page →50 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Fable 5.1	0.5674	$350.25	628.4M	1.5M	16h 3m	1456	82
2	Claude Fable 5.1	0.5395	$313.80	554.2M	1.5M	11h 26m	1289	34
3	Claude Fable 5.1	0.4885	$375.68	676.2M	1.5M	17h 3m	1505	50
4	Claude Fable 5.1	0.4469	$289.07	503.7M	1.5M	13h 41m	1156	50
5	GPT-5.6	0.3847	$146.13	140.6M	311k	2h 25m	439	7
6	GPT-5.6	0.3225	$67.06	75.3M	223k	1h 44m	326	9
7	GPT-5.6	0.3025	$18.35	26.1M	127k	53m	144	4
8	Claude Fable 5.1	0.2880	$179.04	308.9M	1.0M	10h 29m	755	12
9	GPT-5.6	0.2868	$49.11	53.9M	193k	1h 32m	238	5
10	GPT-5.6	0.2751	$110.05	104.7M	243k	2h 24m	350	7
11	Gemini 3.7 Flash	0.2399	$41.35	394.7M	960k	4h 30m	1009	6
12	GLM-5.3	0.2372	$181.79	673.8M	1.1M	8h 14m	1501	3
13	GLM-5.3	0.2303	$183.58	670.1M	1.1M	9h 50m	1460	18
14	GLM-5.3	0.1823	$327.58	1209.3M	1.8M	18h 23m	2635	46
15	Kimi K3	0.1778	$238.01	679.3M	1.8M	19h 52m	1588	83
16	GLM-5.3	0.1583	$236.28	865.5M	1.6M	18h 53m	1991	34
17	Gemini 3.7 Flash	0.1572	$41.30	358.7M	785k	6h 1m	899	5
18	Gemini 3.7 Flash	0.1555	$24.69	216.5M	877k	2h 56m	624	2
19	Kimi K3	0.1513	$169.56	484.2M	1.2M	19h 34m	1155	16
20	Grok 4.6	0.1479	$150.76	148.7M	804k	4h 11m	580	26
21	Grok 4.6	0.1393	$108.10	108.1M	736k	4h 5m	435	11
22	Kimi K3	0.1288	$173.14	509.0M	1.1M	13h 23m	1149	28
23	Grok 4.6	0.1022	$79.42	75.8M	516k	2h 56m	273	7
24	DeepSeek V4 Flash Vision Exp	0.0891	$17.04	1034.3M	1.6M	6h 42m	2273	6
25	DeepSeek V4 Flash Vision Exp	0.0545	$16.46	992.0M	1.7M	6h 43m	2222	11
26	Muse Spark 1.2	0.0000	$40.00	150.5M	651k	3h 23m	388	3
27	Muse Spark 1.2	0.0000	$41.66	132.7M	956k	3h 49m	473	3
28	Inkling	0.0000	$23.47	24.5M	46k	39m	359	2
29	Muse Spark 1.2	0.0000	$16.54	87.9M	378k	1h 23m	309	3
30	Qwen3.8-Max	0.0000	$21.15	36.8M	1.9M	20h 0m	260	—
31	Inkling	0.0000	$3.70	3.7M	39k	18m	85	2
32	Qwen3.8-Max	0.0000	$16.28	22.3M	1.7M	19h 46m	202	—
33	Gemini 3.7 Flash	0.0000	$33.92	328.7M	960k	2h 58m	810	20
34	DeepSeek V4 Flash Vision Exp	0.0000	$13.51	827.8M	1.3M	5h 43m	1720	13
35	DeepSeek V4 Flash Vision Exp	0.0000	$21.93	1323.6M	2.2M	8h 50m	3095	7
36	Gemini 3.7 Flash	0.0000	$29.56	325.1M	599k	2h 10m	830	2
37	DeepSeek V4 Flash Vision Exp	0.0000	$33.36	1965.0M	3.8M	19h 31m	4102	48
38	Kimi K3	0.0000	$214.44	584.8M	1.8M	20h 0m	1459	1
39	Inkling	0.0000	$18.36	19.0M	75k	47m	251	4
40	Qwen3.8-Max	0.0000	$19.23	33.9M	1.7M	19h 53m	233	1
41	Grok 4.6	0.0000	$93.77	99.5M	677k	3h 26m	435	10
42	Muse Spark 1.2	0.0000	$39.83	172.4M	582k	3h 18m	559	6
43	Inkling	0.0000	$3.58	3.6M	31k	16m	74	2
44	Qwen3.8-Max	0.0000	$25.12	46.8M	2.2M	19h 55m	301	—
45	Muse Spark 1.2	0.0000	$32.92	157.9M	806k	3h 52m	344	3
46	Grok 4.6	0.0000	$71.78	70.3M	399k	1h 58m	277	5
47	Qwen3.8-Max	0.0000	$13.06	14.1M	1.6M	19h 35m	156	—
48	Inkling	0.0000	$3.11	3.2M	24k	13m	82	3
49	Kimi K3	0.0000	$256.75	739.1M	1.4M	20h 0m	1576	1
50	GLM-5.3	0.0000	$216.67	774.3M	1.6M	16h 7m	1754	19