Traces

Every run: the agent's full trajectory, its submissions over the 20-hour budget, and the final score.

PostgreSQL 18 on SQLite

task page →49 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	GPT-5.6	0.5275	$209.42	198.9M	277k	4h 5m	589	12
2	Claude Fable 5.1	0.4011	$588.09	1056.3M	2.5M	17h 46m	2515	13
3	Claude Fable 5.1	0.3901	$721.01	1319.3M	2.5M	17h 44m	2872	14
4	GPT-5.6	0.3901	$654.39	640.1M	687k	19h 58m	1472	28
5	Claude Fable 5.1	0.3516	$384.99	680.6M	1.8M	18h 59m	1688	9
6	Claude Fable 5.1	0.3352	$475.86	850.2M	2.1M	18h 4m	1894	7
7	Gemini 3.7 Flash	0.2582	$124.97	1333.8M	2.5M	11h 51m	3014	37
8	Gemini 3.7 Flash	0.2418	$144.25	1592.6M	2.6M	19h 46m	3450	39
9	Claude Fable 5.1	0.2308	$392.90	697.7M	1.8M	19h 16m	1679	4
10	Gemini 3.7 Flash	0.2253	$193.89	2112.9M	3.0M	19h 55m	4680	16
11	Gemini 3.7 Flash	0.1758	$159.99	1689.8M	2.8M	19h 49m	3751	13
12	Grok 4.6	0.1593	$509.89	496.5M	1.6M	19h 56m	1696	70
13	GLM-5.3	0.1593	$191.86	711.5M	862k	19h 53m	1746	17
14	Grok 4.6	0.1538	$461.37	465.7M	1.9M	17h 59m	1842	49
15	DeepSeek V4 Flash Vision Exp	0.1484	$22.50	1379.3M	2.0M	19h 53m	3146	1
16	GLM-5.3	0.1484	$202.20	744.7M	1.0M	19h 42m	1789	7
17	GLM-5.3	0.1374	$201.89	706.7M	843k	19h 32m	1585	9
18	GLM-5.3	0.1374	$155.77	570.5M	772k	19h 52m	1384	2
19	Qwen3.8-Max	0.1374	$186.86	700.0M	1.6M	19h 30m	1822	2
20	Qwen3.8-Max	0.1319	$178.70	667.7M	1.7M	19h 47m	1549	4
21	DeepSeek V4 Flash Vision Exp	0.1264	$22.61	1383.3M	2.0M	19h 51m	3272	3
22	DeepSeek V4 Flash Vision Exp	0.1044	$19.66	1214.6M	1.7M	19h 25m	2688	9
23	Grok 4.6	0.0989	$257.56	255.7M	1.5M	8h 44m	952	17
24	GLM-5.3	0.0989	$219.76	822.5M	966k	19h 50m	1903	11
25	Qwen3.8-Max	0.0769	$170.86	638.4M	1.5M	19h 53m	1587	2
26	Grok 4.6	0.0714	$398.74	383.4M	2.7M	19h 51m	1457	34
27	Kimi K3	0.0714	$227.56	673.8M	1.2M	20h 0m	1528	1
28	DeepSeek V4 Flash Vision Exp	0.0659	$23.23	1428.3M	2.0M	19h 44m	3348	1
29	Qwen3.8-Max	0.0659	$174.14	653.5M	1.5M	19h 26m	1536	1
30	Kimi K3	0.0659	$265.37	772.2M	1.4M	20h 1m	1427	1
31	Kimi K3	0.0604	$238.20	702.8M	1.4M	20h 1m	1513	1
32	Qwen3.8-Max	0.0549	$158.21	583.9M	1.7M	19h 23m	1405	3
33	Kimi K3	0.0495	$197.52	561.9M	1.3M	20h 0m	1168	1
34	Grok 4.6	0.0495	$148.44	141.0M	985k	6h 17m	559	10
35	Gemini 3.7 Flash	0.0495	$17.96	182.1M	620k	1h 34m	402	2
36	DeepSeek V4 Flash Vision Exp	0.0165	$8.18	489.5M	856k	20h 0m	1194	2
37	Muse Spark 1.2	0.0055	$37.82	217.7M	976k	3h 48m	496	10
38	Muse Spark 1.2	0.0055	$40.72	146.5M	561k	3h 14m	420	4
39	GPT-5.6	0.0055†	$293.05	285.8M	311k	4h 20m	671	6
40	Muse Spark 1.2	0.0000	$32.40	176.5M	473k	2h 53m	436	3
41	Inkling	0.0000	$22.34	23.3M	56k	1h 53m	340	3
42	Muse Spark 1.2	0.0000	$14.08	83.6M	240k	1h 8m	314	2
43	GPT-5.6	0.0000	$137.59	141.5M	243k	3h 27m	480	2
44	GPT-5.6	0.0000	$142.29	142.1M	259k	4h 34m	462	11
45	Inkling	0.0000	$19.06	19.8M	53k	1h 41m	302	2
46	Inkling	0.0000	$19.04	19.9M	46k	23m	252	1
47	Inkling	0.0000	$16.56	17.2M	48k	32m	214	2
48	Inkling	0.0000	$16.62	17.3M	42k	34m	287	1
49	Muse Spark 1.2	0.0000	$33.19	162.4M	603k	2h 4m	458	1

† score adjusted by manual review; the trace's Judge tab carries the measured score and the rationale.