Tasks Traces V1 Blog Changelog Contribute GitHub

Traces

Every run: the agent's full trajectory, its submissions over the 20-hour budget, and the final score.

Notebook Compression

task page →110 runs · indexed 2026-10-08

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Sonnet 5.5	0.7077	$121.38	505.5M	1.6M	19h 29m	1140	23
2	Claude Opus 5.5	0.6404	$76.14	300.9M	607k	18h 32m	620	33
3	Claude Opus 5.5	0.6166	$96.89	380.6M	802k	18h 37m	872	33
4	GPT-6 Astra	0.6122	$2035.40	958.4M	1.9M	17h 45m	2132	53
5	GPT-6 Astra	0.6063	$2124.53	1006.7M	2.1M	18h 49m	2308	42
6	Claude Sonnet 5.5	0.6038	$113.09	459.8M	1.7M	19h 12m	1037	19
7	Claude Fable 5.1	0.6002	$67.85	117.2M	379k	14h 26m	401	17
8	Claude Sonnet 5.5	0.5947	$105.86	436.0M	1.5M	19h 24m	994	2
9	Claude Sonnet 5.5	0.5930	$98.59	408.5M	1.3M	17h 41m	827	1
10	Claude Opus 5.5	0.5926	$80.88	317.0M	668k	19h 35m	606	29
11	Claude Fable 5.1	0.5736	$77.68	134.5M	428k	13h 49m	450	16
12	Claude Sonnet 5.5	0.5678	$87.19	362.7M	1.2M	18h 49m	808	8
13	Claude Opus 5.5	0.5584	$79.55	318.4M	608k	18h 25m	649	27
14	Claude Opus 5.5	0.5566	$69.33	267.9M	597k	18h 1m	557	28
15	Claude Fable 5.1	0.5457	$68.00	118.2M	365k	13h 15m	419	21
16	GPT-6 Astra	0.5422	$1208.04	578.6M	1.3M	19h 17m	1389	24
17	Claude Haiku 5.5	0.5323	$15.39	247.7M	1.1M	15h 23m	618	3
18	Claude Fable 5.1	0.5279	$64.70	113.1M	334k	17h 24m	419	17
19	Claude Opus 5	0.5259	$112.64	181.6M	606k	17h 57m	454	17
20	Claude Haiku 5.5	0.5238	$18.15	307.8M	1.1M	18h 41m	707	23
21	Claude Fable 5	0.4971	$137.67	106.8M	434k	17h 17m	390	15
22	Claude Fable 5	0.4861	$123.21	90.8M	455k	18h 4m	318	16
23	Claude Fable 5.1	0.4824	$61.77	107.1M	338k	15h 49m	418	16
24	Gemini 4 Argon	0.4592	$350.30	290.6M	1.5M	20h 36m	690	25
25	Claude Opus 5	0.4532	$73.01	109.0M	516k	14h 40m	352	12
26	Claude Haiku 5.5	0.4519	$14.24	235.9M	949k	18h 16m	561	21
27	Gemini 4 Argon	0.4467	$232.09	321.0M	1.6M	20h 40m	706	28
28	Claude Opus 5	0.4439	$118.20	194.6M	584k	18h 55m	507	14
29	Claude Opus 5	0.4351	$112.61	183.8M	577k	18h 34m	471	19
30	GPT-6 Astra	0.4322	$274.97	132.6M	360k	5h 47m	407	7
31	Claude Opus 5	0.4296	$84.72	133.1M	506k	18h 0m	383	12
32	Claude Haiku 5.5	0.4204	$18.03	295.9M	1.2M	15h 35m	688	7
33	GPT-6 Astra	0.4121	$327.78	158.4M	411k	6h 16m	482	10
34	Gemini 4 Argon	0.4112	$223.42	270.8M	1.5M	21h 0m	634	23
35	Gemini 4 Argon	0.3971	$258.69	226.7M	1.1M	20h 39m	498	16
36	Gemini 4 Argon	0.3901	$194.88	235.0M	1.3M	15h 31m	566	13
37	Claude Fable 5	0.3654	$123.52	92.2M	440k	14h 28m	330	13
38	Claude Fable 5	0.3578	$103.79	73.7M	421k	17h 44m	273	13
39	Claude Haiku 5.5	0.3415	$9.66	154.9M	685k	7h 45m	327	9
40	Claude Fable 5	0.3346	$106.67	78.6M	394k	17h 8m	293	15
41	GLM-5.3	0.2770	$25.86	86.5M	327k	19h 49m	370	3
42	GLM-5.3	0.2744	$34.96	116.4M	391k	19h 2m	435	2
43	Muse Spark 1.3	0.2729	$60.32	65.3M	317k	6h 41m	357	11
44	Kimi K3	0.2647	$37.66	96.4M	363k	19h 31m	409	17
45	GPT-5.6 Sol	0.2623	$123.00	167.4M	279k	11h 19m	628	8
46	Qwen3.8-Max-0902	0.2600	$30.23	89.6M	1.1M	20h 1m	386	1
47	GPT-5.6 Sol	0.2599	$99.01	115.0M	257k	15h 3m	528	10
48	Grok 4.7	0.2584	$211.20	205.8M	904k	13h 1m	859	13
49	Qwen3.8-Max-0902	0.2513	$27.92	79.6M	1.2M	19h 13m	407	3
50	Grok 4.7	0.2464	$38.17	37.7M	250k	3h 43m	192	8
51	GPT-5.6 Sol	0.2438	$58.48	76.1M	190k	17h 55m	412	14
52	Grok 4.7	0.2422	$40.60	41.8M	253k	4h 6m	221	11
53	Qwen3.8-Max-0902	0.2416	$28.69	81.5M	1.3M	19h 35m	423	1
54	Kimi K3	0.2338	$43.76	111.9M	405k	17h 35m	403	12
55	GPT-5.6 Sol	0.2326	$88.19	110.8M	211k	11h 35m	518	6
56	GLM-5.3 Flash	0.2312	$4.45	118.7M	338k	13h 55m	477	3
57	Grok 4.6	0.2286	$85.31	84.1M	498k	7h 48m	331	14
58	Muse Spark 1.3	0.2285	$27.01	79.4M	314k	19h 7m	407	8
59	Kimi K3	0.2242	$36.27	87.9M	412k	11h 52m	382	12
60	GLM-5.3 Flash	0.2231	$4.83	113.6M	398k	17h 19m	455	18
61	Grok 4.7	0.2221	$20.96	23.3M	213k	3h 13m	149	5
62	Qwen3.8-Max-0902	0.2219	$31.23	88.3M	1.4M	19h 46m	478	1
63	Qwen3.8-Max	0.2190	$26.32	83.3M	818k	19h 14m	454	2
64	Qwen3.8-Max	0.2153	$10.82	27.5M	610k	19h 5m	314	2
65	Kimi K3	0.2151	$24.45	60.7M	274k	11h 45m	302	10
66	Qwen3.8-Max-0902	0.2150	$30.55	85.1M	1.4M	19h 31m	403	3
67	GLM-5.3 Flash	0.2148	$5.19	105.1M	375k	18h 46m	410	2
68	Grok 4.6	0.2146	$36.98	34.6M	412k	13h 20m	220	5
69	DeepSeek V4 Flash Vision Exp	0.2139	$1.97	101.8M	387k	13h 26m	420	1
70	GLM-5.3 Flash	0.2132	$3.44	59.6M	295k	17h 49m	300	2
71	Qwen3.8-Max	0.2125	$16.15	47.4M	646k	17h 46m	362	2
72	Gemini 3.8 Flash	0.2118	$6.55	47.0M	149k	3h 4m	286	6
73	Muse Spark 1.3	0.2112	$78.14	76.4M	260k	11h 8m	358	3
74	Grok 4.6	0.2098	$96.38	88.3M	636k	11h 39m	377	12
75	GLM-5.3	0.2089	$39.28	89.8M	362k	20h 0m	369	1
76	Grok 4.6	0.2060	$71.54	75.2M	388k	6h 34m	381	27
77	GLM-5.3	0.2058	$59.65	209.0M	565k	15h 11m	548	8
78	DeepSeek V4 Flash Vision Exp	0.2028	$2.58	144.1M	374k	19h 46m	508	2
79	Grok 4.6	0.2019	$87.92	83.6M	470k	8h 37m	361	10
80	Grok 4.7	0.2015	$41.06	38.3M	268k	3h 17m	188	5
81	Qwen3.8-Max	0.2005	$15.43	43.7M	693k	18h 44m	414	1
82	Kimi K3	0.1961	$13.59	21.1M	194k	9h 53m	179	9
83	DeepSeek V4 Flash Vision Exp	0.1948	$3.01	169.4M	449k	11h 0m	551	4
84	DeepSeek V4 Flash Vision Exp	0.1939	$2.93	164.0M	435k	14h 31m	526	4
85	Gemini 3.8 Flash	0.1896	$6.34	45.8M	152k	4h 56m	308	6
86	Qwen3.8-Max	0.1831	$20.18	58.6M	851k	19h 8m	443	2
87	GLM-5.3 Flash	0.1829	$5.24	90.6M	377k	19h 26m	352	2
88	DeepSeek V4 Flash Vision Exp	0.1818	$2.40	126.7M	415k	10h 56m	421	6
89	Gemini 3.8 Flash	0.1802	$4.63	34.5M	148k	3h 0m	225	6
90	Gemini 3.7 Flash	0.1775	$2.95	13.6M	108k	3h 17m	129	2
91	Gemini 3.7 Flash	0.1714	$6.92	27.7M	160k	6h 28m	236	5
92	Muse Spark 1.2	0.1646	$89.68	227.7M	1.2M	20h 2m	728	7
93	Gemini 3.8 Flash	0.1634	$11.15	62.0M	214k	5h 27m	315	7
94	GPT-5.6 Sol	0.1417	$64.27	80.6M	214k	10h 2m	389	3
95	Muse Spark 1.2	0.1353	$32.24	170.7M	487k	10h 37m	551	5
96	Muse Spark 1.3	0.1309	$65.04	67.7M	249k	11h 11m	371	6
97	GLM-5.3	0.1261	$28.93	86.9M	409k	13h 16m	328	2
98	Muse Spark 1.3	0.1170	$63.50	77.9M	313k	5h 49m	326	3
99	Muse Spark 1.2	0.0816	$32.40	192.3M	534k	7h 46m	570	4
100	Gemini 3.7 Flash	0.0417	$2.99	16.0M	112k	3h 8m	165	6
101	Gemini 3.7 Flash	0.0412	$3.05	19.8M	150k	3h 10m	194	10
102	Gemini 3.7 Flash	0.0403	$4.56	17.9M	128k	4h 20m	145	8
103	Muse Spark 1.2	0.0387	$183.05	621.0M	1.3M	19h 7m	1497	7
104	Muse Spark 1.2	0.0356	$37.95	196.7M	1.2M	19h 2m	669	3
105	Gemini 3.8 Flash	0.0292	$4.10	23.6M	91k	4h 7m	225	5
106	Inkling	0.0214	$5.30	5.4M	31k	3h 11m	184	2
107	Inkling	0.0209	$6.76	6.9M	44k	2h 24m	193	2
108	Inkling	0.0205	$8.12	8.4M	30k	3h 20m	185	1
109	Inkling	0.0203	$5.44	5.6M	27k	1h 39m	121	3
110	Inkling	0.0179	$5.86	6.0M	36k	2h 37m	166	2