Traces

Every run: the agent's full trajectory, its submissions over the 20-hour budget, and the final score.

Synthetic Music Diarization

task page →50 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Fable 5.1	0.5457	$12.11	18.5M	118k	17h 49m	156	8
2	Claude Fable 5.1	0.5346	$11.41	17.0M	120k	17h 44m	158	7
3	GPT-5.6	0.5247	$44.19	53.3M	154k	7h 34m	267	3
4	Claude Fable 5.1	0.5104	$17.60	28.7M	134k	17h 45m	230	11
5	GPT-5.6	0.5065	$80.65	78.5M	178k	15h 57m	339	11
6	GPT-5.6	0.4952	$59.62	69.7M	147k	15h 37m	359	3
7	Claude Fable 5.1	0.4905	$10.29	15.1M	114k	17h 40m	140	7
8	Claude Fable 5.1	0.4889	$10.86	15.6M	125k	15h 34m	138	6
9	GLM-5.3	0.4816	$35.34	125.4M	361k	17h 30m	462	11
10	GLM-5.3	0.4802	$31.62	79.0M	303k	18h 9m	361	1
11	Gemini 3.7 Flash	0.4776	$32.86	129.1M	265k	12h 34m	438	2
12	GPT-5.6	0.4737	$78.78	76.5M	133k	14h 47m	286	4
13	GPT-5.6	0.4727	$68.59	82.8M	153k	14h 29m	425	7
14	Gemini 3.7 Flash	0.4664	$11.27	42.4M	155k	7h 29m	254	2
15	Grok 4.6	0.4661	$133.75	133.0M	655k	19h 52m	578	42
16	Qwen3.8-Max	0.4605	$29.62	103.3M	529k	18h 43m	509	11
17	GLM-5.3	0.4602	$19.77	41.7M	196k	19h 9m	253	4
18	Qwen3.8-Max	0.4425	$26.66	93.3M	455k	18h 13m	478	8
19	Gemini 3.7 Flash	0.4422	$14.36	133.2M	250k	5h 49m	389	2
20	DeepSeek V4 Flash Vision Exp	0.4412	$1.72	93.1M	282k	19h 4m	430	2
21	Grok 4.6	0.4323	$188.19	192.5M	1.1M	19h 45m	871	56
22	Kimi K3	0.4318	$13.36	33.2M	179k	18h 52m	244	5
23	Qwen3.8-Max	0.4301	$34.00	119.2M	572k	19h 17m	508	10
24	GLM-5.3	0.4285	$28.03	77.4M	258k	18h 22m	366	9
25	GLM-5.3	0.4265	$27.27	78.5M	240k	19h 37m	374	6
26	Gemini 3.7 Flash	0.4264	$42.51	256.4M	394k	13h 10m	681	9
27	DeepSeek V4 Flash Vision Exp	0.4247	$2.96	173.2M	352k	17h 55m	617	4
28	Kimi K3	0.4157	$22.81	60.7M	202k	19h 10m	347	7
29	Grok 4.6	0.4120	$228.84	232.3M	993k	19h 44m	918	63
30	DeepSeek V4 Flash Vision Exp	0.4065	$1.73	95.5M	262k	17h 34m	448	5
31	Qwen3.8-Max	0.4043	$33.94	117.1M	655k	18h 57m	501	5
32	Kimi K3	0.4034	$13.34	31.5M	191k	18h 19m	247	4
33	Gemini 3.7 Flash	0.4000	$15.67	83.4M	201k	4h 25m	322	1
34	Grok 4.6	0.3985	$299.53	304.5M	1.6M	19h 27m	1281	54
35	Qwen3.8-Max	0.3951	$17.02	56.1M	424k	19h 10m	391	4
36	Kimi K3	0.3852	$16.48	39.9M	237k	19h 5m	234	2
37	Muse Spark 1.2	0.3829	$88.55	356.6M	541k	7h 12m	754	3
38	DeepSeek V4 Flash Vision Exp	0.3803	$1.85	99.9M	306k	18h 37m	455	3
39	Muse Spark 1.2	0.3711	$10.87	42.0M	256k	3h 30m	227	4
40	Kimi K3	0.3635	$16.00	39.1M	227k	18h 59m	234	6
41	Muse Spark 1.2	0.3611	$12.50	32.3M	167k	4h 4m	213	5
42	Muse Spark 1.2	0.3544	$13.53	50.0M	335k	4h 26m	248	4
43	Muse Spark 1.2	0.3434	$13.40	47.8M	217k	5h 21m	241	5
44	DeepSeek V4 Flash Vision Exp	0.2979	$2.02	115.0M	266k	19h 32m	508	4
45	Inkling	0.2089	$6.22	6.4M	44k	35m	131	2
46	Inkling	0.0477	$6.40	6.6M	38k	28m	123	1
47	Inkling	0.0074	$12.08	12.5M	45k	59m	160	1
48	Inkling	0.0053	$5.67	5.8M	48k	1h 31m	99	2
49	Inkling	0.0012	$9.51	9.9M	36k	38m	170	1
50	Grok 4.6	0.0000	$501.54	509.4M	1.9M	18h 17m	1937	96