Tasks Traces V1 Blog Changelog Contribute GitHub

Traces

Every run: the agent's full trajectory, its submissions over the 20-hour budget, and the final score.

PostgreSQL 18 on SQLite

task page →109 runs · indexed 2026-10-08

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	GPT-6 Astra	0.7308	$2982.00	1383.4M	2.9M	19h 56m	3073	67
2	GPT-6 Astra	0.7143	$3132.52	1453.4M	2.8M	19h 51m	3140	44
3	Gemini 4 Argon	0.6923	$41.63	62.6M	389k	2h 48m	192	2
4	Gemini 4 Argon	0.6923	$43.92	65.5M	430k	2h 31m	192	2
5	Gemini 4 Argon	0.6923	$26.32	82.0M	301k	2h 32m	224	2
6	Gemini 4 Argon	0.6923	$25.43	64.1M	365k	2h 39m	186	2
7	GPT-6 Astra	0.6593	$3139.16	1477.1M	2.8M	19h 52m	3280	73
8	Gemini 4 Argon	0.6484	$34.67	93.0M	375k	2h 50m	250	2
9	GPT-6 Astra	0.6484	$3119.08	1452.8M	2.8M	19h 56m	3108	62
10	GPT-6 Astra	0.6209	$3312.10	1548.3M	2.9M	20h 0m	3255	61
11	Claude Opus 5.5	0.5934	$433.39	1590.6M	4.3M	19h 2m	3640	18
12	Claude Opus 5.5	0.5659	$363.10	1344.0M	3.5M	19h 34m	3174	20
13	Claude Sonnet 5.5	0.5549	$310.17	1253.5M	4.5M	19h 16m	2929	12
14	GPT-5.6 Sol	0.5275	$209.42	198.9M	277k	4h 5m	589	12
15	Claude Sonnet 5.5	0.5000	$308.88	1278.7M	4.0M	19h 11m	2863	6
16	Claude Sonnet 5.5	0.5000	$284.21	1159.1M	4.0M	19h 17m	2709	1
17	Claude Sonnet 5.5	0.4780	$247.76	1000.6M	3.6M	19h 38m	2348	2
18	Claude Opus 5.5	0.4615	$283.28	1029.0M	2.9M	18h 59m	2312	13
19	Claude Sonnet 5.5	0.4451	$309.32	1277.3M	4.1M	19h 4m	2927	3
20	Claude Fable 5.1	0.4011	$588.09	1056.3M	2.5M	17h 46m	2515	13
21	Gemini 3.8 Flash	0.4011	$163.91	1747.0M	1.9M	19h 43m	3941	86
22	Claude Fable 5.1	0.3901	$721.01	1319.3M	2.5M	17h 44m	2872	14
23	GPT-5.6 Sol	0.3901	$654.39	640.1M	687k	19h 58m	1472	28
24	Gemini 3.8 Flash	0.3846	$244.90	2763.6M	3.1M	19h 16m	6084	124
25	Claude Fable 5.1	0.3516	$384.99	680.6M	1.8M	18h 59m	1688	9
26	Claude Opus 5	0.3462	$503.66	880.6M	1.6M	19h 10m	1883	12
27	Claude Fable 5	0.3407	$1287.66	1111.6M	2.3M	19h 25m	2503	12
28	Muse Spark 1.3	0.3407	$551.80	666.0M	898k	19h 58m	1714	4
29	Claude Fable 5.1	0.3352	$475.86	850.2M	2.1M	18h 4m	1894	7
30	Claude Opus 5.5	0.3352	$189.85	651.3M	2.3M	19h 20m	1478	8
31	Claude Opus 5	0.3352	$381.03	662.1M	1.3M	19h 37m	1475	7
32	Claude Opus 5	0.3352	$353.77	610.8M	1.3M	19h 43m	1274	5
33	Gemini 3.8 Flash	0.3352	$208.72	2281.9M	3.1M	19h 12m	4971	91
34	Gemini 3.8 Flash	0.3297	$180.31	1833.2M	2.5M	19h 54m	4050	100
35	Claude Opus 5	0.3242	$340.68	578.1M	1.3M	19h 18m	1332	6
36	Claude Opus 5.5	0.2967	$184.14	630.5M	2.2M	19h 31m	1487	10
37	Claude Opus 5	0.2912	$434.78	748.3M	1.6M	18h 39m	1774	5
38	Claude Haiku 5.5	0.2857	$70.44	1203.7M	4.0M	19h 19m	2653	34
39	Claude Haiku 5.5	0.2857	$79.48	1383.8M	3.8M	17h 44m	2900	5
40	Grok 4.7	0.2747	$697.58	718.7M	2.2M	19h 51m	2979	27
41	Grok 4.7	0.2637	$767.04	780.4M	2.4M	18h 58m	3117	30
42	Claude Fable 5	0.2637	$836.08	698.8M	1.8M	19h 25m	1580	13
43	Claude Haiku 5.5	0.2637	$80.98	1401.6M	4.1M	17h 22m	3095	1
44	Claude Fable 5	0.2637	$945.64	794.6M	2.0M	19h 56m	1718	6
45	Gemini 3.8 Flash	0.2637	$135.19	1385.8M	1.8M	16h 40m	3053	61
46	Gemini 3.7 Flash	0.2582	$124.97	1333.8M	2.5M	11h 51m	3014	37
47	Grok 4.7	0.2527	$961.79	986.1M	3.0M	20h 0m	3909	26
48	Gemini 3.7 Flash	0.2418	$144.25	1592.6M	2.6M	19h 46m	3450	39
49	Claude Fable 5.1	0.2308	$392.90	697.7M	1.8M	19h 16m	1679	4
50	Claude Haiku 5.5	0.2308	$58.07	999.5M	3.0M	18h 19m	2320	6
51	Gemini 3.7 Flash	0.2253	$193.89	2112.9M	3.0M	19h 55m	4680	16
52	Claude Haiku 5.5	0.2253	$57.46	996.8M	2.8M	17h 55m	2275	3
53	Grok 4.7	0.1978	$1004.11	1036.8M	3.4M	20h 0m	4217	22
54	Gemini 3.7 Flash	0.1758	$159.99	1689.8M	2.8M	19h 49m	3751	13
55	Grok 4.6	0.1593	$509.89	496.5M	1.6M	19h 56m	1696	70
56	GLM-5.3	0.1593	$191.86	711.5M	862k	19h 53m	1746	17
57	Claude Fable 5	0.1538	$641.49	517.5M	1.6M	19h 39m	1292	4
58	Grok 4.6	0.1538	$461.37	465.7M	1.9M	17h 59m	1842	49
59	DeepSeek V4 Flash Vision Exp	0.1484	$22.50	1379.3M	2.0M	19h 53m	3146	1
60	GLM-5.3	0.1484	$202.20	744.7M	1.0M	19h 42m	1789	7
61	GLM-5.3	0.1374	$201.89	706.7M	843k	19h 32m	1585	9
62	GLM-5.3	0.1374	$155.77	570.5M	772k	19h 52m	1384	2
63	Qwen3.8-Max	0.1374	$186.86	700.0M	1.6M	19h 30m	1822	2
64	Qwen3.8-Max	0.1319	$178.70	667.7M	1.7M	19h 47m	1549	4
65	DeepSeek V4 Flash Vision Exp	0.1264	$22.61	1383.3M	2.0M	19h 51m	3272	3
66	Muse Spark 1.3	0.1099	$305.43	1604.1M	2.2M	20h 0m	3690	2
67	DeepSeek V4 Flash Vision Exp	0.1044	$19.66	1214.6M	1.7M	19h 25m	2688	9
68	Grok 4.6	0.0989	$257.56	255.7M	1.5M	8h 44m	952	17
69	GLM-5.3	0.0989	$219.76	822.5M	966k	19h 50m	1903	11
70	GLM-5.3 Flash	0.0769	$23.42	737.2M	1.0M	19h 23m	1578	4
71	Qwen3.8-Max	0.0769	$170.86	638.4M	1.5M	19h 53m	1587	2
72	Grok 4.6	0.0714	$398.74	383.4M	2.7M	19h 51m	1457	34
73	Kimi K3	0.0714	$227.56	673.8M	1.2M	20h 0m	1528	1
74	DeepSeek V4 Flash Vision Exp	0.0659	$23.23	1428.3M	2.0M	19h 44m	3348	1
75	Qwen3.8-Max	0.0659	$174.14	653.5M	1.5M	19h 26m	1536	1
76	GLM-5.3 Flash	0.0659	$26.47	848.3M	1.3M	18h 3m	1663	11
77	Kimi K3	0.0659	$265.37	772.2M	1.4M	20h 1m	1427	1
78	Kimi K3	0.0604	$238.20	702.8M	1.4M	20h 1m	1513	1
79	Muse Spark 1.3	0.0549	$522.84	598.7M	1.2M	20h 0m	1525	1
80	Muse Spark 1.3	0.0549	$518.76	603.9M	1.1M	19h 59m	1572	1
81	Qwen3.8-Max	0.0549	$158.21	583.9M	1.7M	19h 23m	1405	3
82	Kimi K3	0.0495	$197.52	561.9M	1.3M	20h 0m	1168	1
83	GLM-5.3 Flash	0.0495	$25.01	730.3M	929k	19h 49m	1709	1
84	Grok 4.6	0.0495	$148.44	141.0M	985k	6h 17m	559	10
85	Gemini 3.7 Flash	0.0495	$17.96	182.1M	620k	1h 34m	402	2
86	Muse Spark 1.3	0.0495	$525.95	562.0M	986k	20h 0m	1387	1
87	Qwen3.8-Max-0902	0.0330	$183.61	654.3M	2.6M	19h 34m	1737	2
88	GLM-5.3 Flash	0.0330	$27.79	699.2M	1.1M	19h 38m	1496	10
89	Qwen3.8-Max-0902	0.0330	$116.07	407.7M	1.9M	20h 1m	1122	2
90	Qwen3.8-Max-0902	0.0220	$150.66	544.1M	1.9M	20h 0m	1434	1
91	GLM-5.3 Flash	0.0220	$24.50	755.8M	998k	19h 51m	1899	14
92	Qwen3.8-Max-0902	0.0165	$112.13	383.6M	1.8M	19h 52m	1132	5
93	DeepSeek V4 Flash Vision Exp	0.0165	$8.18	489.5M	856k	20h 0m	1194	2
94	Qwen3.8-Max-0902	0.0110	$104.91	366.4M	1.7M	20h 1m	1093	1
95	Muse Spark 1.2	0.0055	$37.82	217.7M	976k	3h 48m	496	10
96	Muse Spark 1.2	0.0055	$40.72	146.5M	561k	3h 14m	420	4
97	GPT-5.6 Sol	0.0055†	$293.05	285.8M	311k	4h 20m	671	6
98	Muse Spark 1.2	0.0000	$32.40	176.5M	473k	2h 53m	436	3
99	Inkling	0.0000	$22.34	23.3M	56k	1h 53m	340	3
100	Claude Fable 5	0.0000†	$385.63	316.2M	920k	5h 26m	738	3
101	Muse Spark 1.2	0.0000	$14.08	83.6M	240k	1h 8m	314	2
102	GPT-5.6 Sol	0.0000	$137.59	141.5M	243k	3h 27m	480	2
103	GPT-5.6 Sol	0.0000	$142.29	142.1M	259k	4h 34m	462	11
104	Grok 4.7	0.0000	$890.38	903.6M	3.1M	20h 0m	3660	12
105	Inkling	0.0000	$19.06	19.8M	53k	1h 41m	302	2
106	Inkling	0.0000	$19.04	19.9M	46k	23m	252	1
107	Inkling	0.0000	$16.56	17.2M	48k	32m	214	2
108	Inkling	0.0000	$16.62	17.3M	42k	34m	287	1
109	Muse Spark 1.2	0.0000	$33.19	162.4M	603k	2h 4m	458	1

† score adjusted by manual review; the trace's Judge tab carries the measured score and the rationale.