Tasks Traces V1 Blog Changelog Contribute GitHub

Traces

Every run: the agent's full trajectory, its submissions over the 20-hour budget, and the final score.

libexpat Optimization

task page →110 runs · indexed 2026-10-08

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Opus 5.5	0.8047	$233.16	850.8M	2.4M	15h 14m	1758	71
2	Claude Fable 5	0.7799	$660.65	527.2M	1.8M	19h 20m	1251	46
3	Claude Sonnet 5.5	0.7156	$145.32	582.0M	2.3M	15h 31m	1253	17
4	Claude Opus 5.5	0.7133	$179.28	606.1M	2.3M	14h 58m	1384	20
5	Claude Sonnet 5.5	0.7089	$104.14	409.0M	1.8M	9h 58m	905	8
6	Claude Opus 5.5	0.6801	$255.81	936.5M	2.6M	17h 42m	2064	87
7	Claude Sonnet 5.5	0.6732	$123.96	482.0M	2.2M	17h 39m	1064	21
8	Claude Sonnet 5.5	0.6432	$111.15	436.3M	1.9M	12h 32m	911	14
9	Gemini 4 Argon	0.6081	$51.02	180.3M	915k	5h 20m	388	8
10	Gemini 4 Argon	0.5881	$18.38	71.0M	446k	2h 16m	160	9
11	Gemini 4 Argon	0.5870	$86.35	125.1M	819k	4h 40m	340	4
12	Claude Fable 5.1	0.5674	$350.25	628.4M	1.5M	16h 3m	1456	82
13	Claude Sonnet 5.5	0.5673	$154.38	620.0M	2.4M	16h 46m	1326	6
14	Gemini 4 Argon	0.5630	$26.24	74.9M	691k	2h 55m	198	9
15	Claude Fable 5.1	0.5395	$313.80	554.2M	1.5M	11h 26m	1289	34
16	Claude Opus 5	0.5365	$422.53	667.4M	2.4M	15h 22m	1491	47
17	Claude Opus 5	0.5236	$756.50	1269.1M	3.2M	19h 11m	2743	94
18	Claude Opus 5	0.5183	$561.01	908.7M	2.8M	16h 42m	2090	66
19	Claude Opus 5	0.5083	$502.07	801.8M	2.6M	18h 34m	1852	60
20	Claude Opus 5	0.5030	$470.72	762.4M	2.3M	16h 43m	1656	45
21	Claude Fable 5.1	0.4885	$375.68	676.2M	1.5M	17h 3m	1505	50
22	Claude Opus 5.5	0.4775	$212.18	685.7M	2.9M	15h 26m	1659	63
23	Claude Opus 5.5	0.4760	$260.62	918.4M	2.9M	17h 55m	2043	64
24	Gemini 4 Argon	0.4643	$17.19	66.1M	478k	2h 17m	158	4
25	Claude Fable 5.1	0.4469	$289.07	503.7M	1.5M	13h 41m	1156	50
26	Claude Fable 5	0.4435	$891.44	742.6M	2.0M	15h 20m	1639	82
27	GPT-5.6 Sol	0.3847	$146.13	140.6M	311k	2h 25m	439	7
28	Claude Fable 5	0.3547	$517.42	421.8M	1.3M	18h 48m	902	36
29	Grok 4.7	0.3527	$861.01	886.1M	3.3M	19h 22m	3633	135
30	Claude Fable 5	0.3358	$480.59	387.1M	1.3M	13h 47m	845	26
31	Claude Haiku 5.5	0.3300	$46.90	739.9M	3.7M	9h 50m	1627	7
32	GPT-5.6 Sol	0.3225	$67.06	75.3M	223k	1h 44m	326	9
33	GPT-5.6 Sol	0.3025	$18.35	26.1M	127k	53m	144	4
34	GPT-6 Astra	0.2941	$510.01	240.5M	518k	4h 1m	542	7
35	Claude Fable 5.1	0.2880	$179.04	308.9M	1.0M	10h 29m	755	12
36	GPT-5.6 Sol	0.2868	$49.11	53.9M	193k	1h 32m	238	5
37	GPT-6 Astra	0.2776	$415.35	191.6M	517k	3h 32m	444	6
38	GPT-5.6 Sol	0.2751	$110.05	104.7M	243k	2h 24m	350	7
39	GPT-6 Astra	0.2744	$177.86	83.5M	336k	2h 10m	283	5
40	Claude Fable 5	0.2667	$556.53	463.3M	1.2M	18h 41m	912	25
41	Gemini 3.7 Flash	0.2399	$41.35	394.7M	960k	4h 30m	1009	6
42	GLM-5.3	0.2372	$181.79	673.8M	1.1M	8h 14m	1501	3
43	GLM-5.3	0.2303	$183.58	670.1M	1.1M	9h 50m	1460	18
44	Claude Haiku 5.5	0.2257	$53.36	858.1M	3.9M	16h 29m	1827	36
45	GLM-5.3	0.1823	$327.58	1209.3M	1.8M	18h 23m	2635	46
46	Kimi K3	0.1778	$238.01	679.3M	1.8M	19h 52m	1588	83
47	GLM-5.3	0.1583	$236.28	865.5M	1.6M	18h 53m	1991	34
48	Grok 4.7	0.1576	$386.43	383.9M	880k	5h 16m	1277	13
49	Gemini 3.7 Flash	0.1572	$41.30	358.7M	785k	6h 1m	899	5
50	Claude Haiku 5.5	0.1556	$46.19	751.2M	3.1M	15h 48m	1610	16
51	Gemini 3.7 Flash	0.1555	$24.69	216.5M	877k	2h 56m	624	2
52	GPT-6 Astra	0.1534	$13.19	6.7M	92k	53m	62	4
53	GPT-6 Astra	0.1516	$19.33	10.3M	134k	38m	78	3
54	Kimi K3	0.1513	$169.56	484.2M	1.2M	19h 34m	1155	16
55	Grok 4.6	0.1479	$150.76	148.7M	804k	4h 11m	580	26
56	Grok 4.6	0.1393	$108.10	108.1M	736k	4h 5m	435	11
57	Claude Haiku 5.5	0.1387	$35.67	574.8M	2.5M	9h 45m	1282	8
58	Kimi K3	0.1288	$173.14	509.0M	1.1M	13h 23m	1149	28
59	Gemini 3.8 Flash	0.1280	$22.80	229.1M	416k	2h 32m	541	2
60	Gemini 3.8 Flash	0.1047	$29.07	295.3M	541k	5h 49m	791	3
61	Grok 4.6	0.1022	$79.42	75.8M	516k	2h 56m	273	7
62	Muse Spark 1.3	0.0996	$565.41	641.8M	1.1M	19h 58m	1422	16
63	Gemini 3.8 Flash	0.0991	$27.80	282.7M	663k	2h 44m	771	17
64	DeepSeek V4 Flash Vision Exp	0.0891	$17.04	1034.3M	1.6M	6h 42m	2273	6
65	Claude Haiku 5.5	0.0877	$18.92	306.6M	1.3M	3h 20m	649	8
66	Muse Spark 1.3	0.0752	$635.38	703.6M	636k	19h 52m	1342	3
67	Gemini 3.8 Flash	0.0565	$31.04	299.7M	611k	3h 45m	818	4
68	DeepSeek V4 Flash Vision Exp	0.0545	$16.46	992.0M	1.7M	6h 43m	2222	11
69	Muse Spark 1.3	0.0491	$376.25	2119.7M	1.7M	19h 47m	4639	144
70	Muse Spark 1.3	0.0092	$609.32	704.0M	1.1M	19h 57m	1497	2
71	Qwen3.8-Max-0902	0.0000	$53.45	131.0M	3.2M	20h 1m	378	1
72	Muse Spark 1.2	0.0000	$40.00	150.5M	651k	3h 23m	388	3
73	Muse Spark 1.2	0.0000	$41.66	132.7M	956k	3h 49m	473	3
74	Inkling	0.0000	$23.47	24.5M	46k	39m	359	2
75	Muse Spark 1.2	0.0000	$16.54	87.9M	378k	1h 23m	309	3
76	Qwen3.8-Max	0.0000	$21.15	36.8M	1.9M	20h 0m	260	—
77	GLM-5.3 Flash	0.0000	$22.01	646.4M	1.1M	19h 18m	1514	1
78	GLM-5.3 Flash	0.0000	$23.47	725.1M	1.2M	15h 11m	1632	14
79	Inkling	0.0000	$3.70	3.7M	39k	18m	85	2
80	Qwen3.8-Max	0.0000	$16.28	22.3M	1.7M	19h 46m	202	—
81	Gemini 3.7 Flash	0.0000	$33.92	328.7M	960k	2h 58m	810	20
82	Qwen3.8-Max-0902	0.0000	$42.48	90.3M	3.1M	19h 53m	348	4
83	DeepSeek V4 Flash Vision Exp	0.0000	$13.51	827.8M	1.3M	5h 43m	1720	13
84	DeepSeek V4 Flash Vision Exp	0.0000	$21.93	1323.6M	2.2M	8h 50m	3095	7
85	Gemini 3.7 Flash	0.0000	$29.56	325.1M	599k	2h 10m	830	2
86	GLM-5.3 Flash	0.0000	$28.19	886.8M	1.2M	17h 12m	2083	25
87	DeepSeek V4 Flash Vision Exp	0.0000	$33.36	1965.0M	3.8M	19h 31m	4102	48
88	Kimi K3	0.0000	$214.44	584.8M	1.8M	20h 0m	1459	1
89	Inkling	0.0000	$18.36	19.0M	75k	47m	251	4
90	GLM-5.3 Flash	0.0000	$25.76	811.2M	1.4M	20h 0m	2421	7
91	Grok 4.7	0.0000	$77.71	79.6M	493k	2h 16m	325	4
92	Grok 4.7	0.0000	$35.69	33.2M	254k	1h 17m	139	5
93	Qwen3.8-Max	0.0000	$19.23	33.9M	1.7M	19h 53m	233	1
94	Qwen3.8-Max-0902	0.0000	$36.15	60.0M	3.4M	20h 0m	280	3
95	Grok 4.6	0.0000	$93.77	99.5M	677k	3h 26m	435	10
96	Muse Spark 1.2	0.0000	$39.83	172.4M	582k	3h 18m	559	6
97	Qwen3.8-Max-0902	0.0000	$95.18	281.1M	3.9M	19h 0m	632	3
98	Inkling	0.0000	$3.58	3.6M	31k	16m	74	2
99	Qwen3.8-Max	0.0000	$25.12	46.8M	2.2M	19h 55m	301	—
100	Gemini 3.8 Flash	0.0000	$35.42	341.5M	629k	6h 5m	907	2
101	Qwen3.8-Max-0902	0.0000	$38.42	73.2M	3.2M	20h 0m	340	2
102	Muse Spark 1.2	0.0000	$32.92	157.9M	806k	3h 52m	344	3
103	Grok 4.6	0.0000	$71.78	70.3M	399k	1h 58m	277	5
104	GLM-5.3 Flash	0.0000	$21.51	630.1M	971k	19h 59m	1458	2
105	Qwen3.8-Max	0.0000	$13.06	14.1M	1.6M	19h 35m	156	—
106	Inkling	0.0000	$3.11	3.2M	24k	13m	82	3
107	Kimi K3	0.0000	$256.75	739.1M	1.4M	20h 0m	1576	1
108	Muse Spark 1.3	0.0000	$571.21	716.3M	974k	19h 59m	1683	2
109	GLM-5.3	0.0000	$216.67	774.3M	1.6M	16h 7m	1754	19
110	Grok 4.7	0.0000†	$83.03	79.6M	516k	2h 36m	314	13

† score adjusted by manual review; the trace's Judge tab carries the measured score and the rationale.