Tasks Traces V1 Blog Changelog Contribute GitHub

Traces

Every run: the agent's full trajectory, its submissions over the 20-hour budget, and the final score.

Synthetic Music Diarization

task page →110 runs · indexed 2026-10-08

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	GPT-6 Astra	0.5899	$503.76	243.1M	455k	15h 19m	591	11
2	GPT-6 Astra	0.5838	$329.84	157.1M	371k	18h 6m	426	13
3	GPT-6 Astra	0.5821	$102.57	50.4M	226k	17h 2m	214	8
4	GPT-6 Astra	0.5755	$313.33	152.4M	361k	17h 42m	459	12
5	Gemini 4 Argon	0.5622	$95.75	86.8M	749k	11h 34m	276	7
6	Gemini 4 Argon	0.5493	$59.76	63.4M	476k	9h 43m	201	5
7	Claude Sonnet 5.5	0.5485	$40.25	163.7M	585k	16h 59m	377	6
8	Claude Sonnet 5.5	0.5468	$52.30	219.6M	652k	16h 9m	456	6
9	Claude Opus 5.5	0.5460	$17.72	55.1M	249k	18h 32m	256	12
10	Claude Fable 5.1	0.5457	$12.11	18.5M	118k	17h 49m	156	8
11	Claude Sonnet 5.5	0.5456	$46.15	192.4M	590k	18h 34m	420	4
12	Gemini 4 Argon	0.5456	$46.26	59.1M	483k	7h 58m	169	5
13	Claude Sonnet 5.5	0.5444	$47.19	194.7M	641k	18h 8m	414	1
14	Claude Sonnet 5.5	0.5410	$38.75	155.4M	594k	16h 3m	363	8
15	Claude Opus 5.5	0.5369	$24.84	83.3M	302k	17h 50m	310	7
16	Claude Opus 5.5	0.5369	$16.97	51.1M	254k	18h 25m	241	10
17	Claude Opus 5	0.5368	$28.63	38.3M	258k	18h 34m	202	8
18	Claude Fable 5.1	0.5346	$11.41	17.0M	120k	17h 44m	158	7
19	Claude Opus 5	0.5328	$33.95	47.5M	267k	18h 38m	217	7
20	Gemini 4 Argon	0.5249	$103.39	100.4M	544k	9h 12m	246	8
21	GPT-5.6 Sol	0.5247	$44.19	53.3M	154k	7h 34m	267	3
22	Claude Opus 5	0.5241	$22.96	29.3M	223k	18h 54m	175	5
23	Claude Opus 5.5	0.5219	$17.38	51.6M	265k	17h 34m	243	8
24	Claude Haiku 5.5	0.5197	$5.78	94.3M	402k	14h 17m	306	2
25	Claude Fable 5	0.5170	$40.54	26.3M	193k	18h 24m	179	6
26	Claude Fable 5.1	0.5104	$17.60	28.7M	134k	17h 45m	230	11
27	Claude Fable 5	0.5096	$41.08	27.2M	188k	19h 7m	184	6
28	Claude Fable 5	0.5079	$49.45	33.2M	218k	18h 55m	196	8
29	GPT-5.6 Sol	0.5065	$80.65	78.5M	178k	15h 57m	339	11
30	GPT-6 Astra	0.5062	$228.20	112.0M	318k	15h 43m	379	15
31	GPT-5.6 Sol	0.4952	$59.62	69.7M	147k	15h 37m	359	3
32	Claude Opus 5.5	0.4919	$19.72	61.9M	269k	17h 35m	246	9
33	Claude Fable 5	0.4917	$51.32	35.9M	209k	18h 41m	228	6
34	Claude Fable 5.1	0.4905	$10.29	15.1M	114k	17h 40m	140	7
35	Claude Fable 5.1	0.4889	$10.86	15.6M	125k	15h 34m	138	6
36	Qwen3.8-Max-0902	0.4844	$67.12	233.9M	1.2M	19h 8m	645	4
37	Qwen3.8-Max-0902	0.4840	$91.60	324.7M	1.4M	19h 50m	793	1
38	Claude Haiku 5.5	0.4839	$3.57	55.3M	334k	17h 2m	236	5
39	Claude Opus 5	0.4833	$26.82	34.5M	255k	17h 35m	181	3
40	GLM-5.3	0.4816	$35.34	125.4M	361k	17h 30m	462	11
41	Qwen3.8-Max-0902	0.4814	$101.81	367.2M	1.4M	19h 10m	818	2
42	Claude Haiku 5.5	0.4805	$3.70	58.7M	322k	17h 54m	249	3
43	GLM-5.3	0.4802	$31.62	79.0M	303k	18h 9m	361	1
44	Gemini 3.7 Flash	0.4776	$32.86	129.1M	265k	12h 34m	438	2
45	GPT-5.6 Sol	0.4737	$78.78	76.5M	133k	14h 47m	286	4
46	GPT-5.6 Sol	0.4727	$68.59	82.8M	153k	14h 29m	425	7
47	Gemini 3.8 Flash	0.4726	$11.84	72.7M	176k	4h 47m	292	6
48	GLM-5.3 Flash	0.4723	$1.67	48.7M	181k	16h 18m	294	5
49	Gemini 3.7 Flash	0.4664	$11.27	42.4M	155k	7h 29m	254	2
50	Grok 4.6	0.4661	$133.75	133.0M	655k	19h 52m	578	42
51	Qwen3.8-Max	0.4605	$29.62	103.3M	529k	18h 43m	509	11
52	GLM-5.3	0.4602	$19.77	41.7M	196k	19h 9m	253	4
53	Grok 4.7	0.4597	$72.67	61.2M	249k	8h 6m	249	4
54	Muse Spark 1.3	0.4569	$29.57	105.7M	282k	17h 55m	519	8
55	Grok 4.7	0.4458	$66.85	49.2M	265k	14h 13m	205	10
56	Qwen3.8-Max	0.4425	$26.66	93.3M	455k	18h 13m	478	8
57	Gemini 3.7 Flash	0.4422	$14.36	133.2M	250k	5h 49m	389	2
58	Claude Haiku 5.5	0.4413	$4.58	75.8M	323k	15h 34m	288	3
59	DeepSeek V4 Flash Vision Exp	0.4412	$1.72	93.1M	282k	19h 4m	430	2
60	Grok 4.7	0.4406	$208.39	177.1M	399k	19h 30m	612	4
61	Gemini 3.8 Flash	0.4403	$11.00	95.0M	185k	4h 2m	334	5
62	Claude Haiku 5.5	0.4371	$5.40	88.7M	390k	16h 43m	316	2
63	Muse Spark 1.3	0.4340	$126.99	121.2M	301k	15h 7m	437	9
64	Grok 4.6	0.4323	$188.19	192.5M	1.1M	19h 45m	871	56
65	Kimi K3	0.4318	$13.36	33.2M	179k	18h 52m	244	5
66	Qwen3.8-Max	0.4301	$34.00	119.2M	572k	19h 17m	508	10
67	GLM-5.3	0.4285	$28.03	77.4M	258k	18h 22m	366	9
68	Grok 4.7	0.4284	$43.87	32.2M	197k	17h 17m	180	1
69	GLM-5.3	0.4265	$27.27	78.5M	240k	19h 37m	374	6
70	Gemini 3.7 Flash	0.4264	$42.51	256.4M	394k	13h 10m	681	9
71	DeepSeek V4 Flash Vision Exp	0.4247	$2.96	173.2M	352k	17h 55m	617	4
72	Muse Spark 1.3	0.4243	$152.31	160.5M	376k	14h 14m	500	7
73	Gemini 4 Argon	0.4215	$32.54	49.3M	491k	6h 13m	159	5
74	Muse Spark 1.3	0.4206	$121.45	118.8M	279k	16h 52m	510	21
75	Kimi K3	0.4157	$22.81	60.7M	202k	19h 10m	347	7
76	Grok 4.6	0.4120	$228.84	232.3M	993k	19h 44m	918	63
77	Grok 4.7	0.4074	$96.18	79.9M	265k	13h 36m	307	8
78	DeepSeek V4 Flash Vision Exp	0.4065	$1.73	95.5M	262k	17h 34m	448	5
79	Gemini 3.8 Flash	0.4043	$7.61	63.4M	182k	2h 25m	287	4
80	Qwen3.8-Max	0.4043	$33.94	117.1M	655k	18h 57m	501	5
81	Kimi K3	0.4034	$13.34	31.5M	191k	18h 19m	247	4
82	GLM-5.3 Flash	0.4027	$2.55	72.6M	225k	19h 1m	391	8
83	Qwen3.8-Max-0902	0.4015	$71.54	249.6M	1.3M	19h 38m	666	2
84	Gemini 3.7 Flash	0.4000	$15.67	83.4M	201k	4h 25m	322	1
85	Grok 4.6	0.3985	$299.53	304.5M	1.6M	19h 27m	1281	54
86	Muse Spark 1.3	0.3962	$92.96	91.1M	258k	17h 50m	454	10
87	Qwen3.8-Max	0.3951	$17.02	56.1M	424k	19h 10m	391	4
88	Gemini 3.8 Flash	0.3938	$11.34	62.3M	164k	3h 35m	300	12
89	GLM-5.3 Flash	0.3907	$3.73	72.3M	226k	17h 49m	356	3
90	Gemini 3.8 Flash	0.3868	$17.13	91.7M	227k	6h 49m	360	8
91	Kimi K3	0.3852	$16.48	39.9M	237k	19h 5m	234	2
92	Muse Spark 1.2	0.3829	$88.55	356.6M	541k	7h 12m	754	3
93	GLM-5.3 Flash	0.3822	$3.47	56.4M	213k	17h 57m	340	3
94	DeepSeek V4 Flash Vision Exp	0.3803	$1.85	99.9M	306k	18h 37m	455	3
95	Muse Spark 1.2	0.3711	$10.87	42.0M	256k	3h 30m	227	4
96	Qwen3.8-Max-0902	0.3707	$81.42	285.5M	1.4M	19h 28m	710	2
97	Kimi K3	0.3635	$16.00	39.1M	227k	18h 59m	234	6
98	Muse Spark 1.2	0.3611	$12.50	32.3M	167k	4h 4m	213	5
99	GLM-5.3 Flash	0.3570	$5.12	63.4M	220k	18h 51m	342	3
100	Muse Spark 1.2	0.3544	$13.53	50.0M	335k	4h 26m	248	4
101	Muse Spark 1.2	0.3434	$13.40	47.8M	217k	5h 21m	241	5
102	DeepSeek V4 Flash Vision Exp	0.2979	$2.02	115.0M	266k	19h 32m	508	4
103	Claude Fable 5	0.2496	$50.58	32.8M	242k	18h 51m	183	5
104	Claude Opus 5	0.2240	$27.38	35.8M	258k	19h 9m	186	9
105	Inkling	0.2089	$6.22	6.4M	44k	35m	131	2
106	Inkling	0.0477	$6.40	6.6M	38k	28m	123	1
107	Inkling	0.0074	$12.08	12.5M	45k	59m	160	1
108	Inkling	0.0053	$5.67	5.8M	48k	1h 31m	99	2
109	Inkling	0.0012	$9.51	9.9M	36k	38m	170	1
110	Grok 4.6	0.0000	$501.54	509.4M	1.9M	18h 17m	1937	96