Tasks Traces V1 Blog Changelog Contribute GitHub

Traces

Every run: the agent's full trajectory, its submissions over the 20-hour budget, and the final score.

MEG Speech Decoding

task page →103 runs · indexed 2026-10-08

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Sonnet 5.5	0.3980	$50.51	215.5M	568k	17h 35m	475	11
2	GPT-6 Astra	0.3919	$899.22	430.4M	993k	17h 31m	1064	15
3	Gemini 4 Argon	0.3815	$127.65	83.3M	493k	10h 58m	219	7
4	GPT-6 Astra	0.3729	$274.60	132.9M	344k	8h 16m	411	13
5	Claude Opus 5.5	0.3727	$24.06	81.6M	282k	17h 26m	303	14
6	GPT-6 Astra	0.3704	$154.28	76.5M	270k	4h 15m	297	9
7	Claude Opus 5.5	0.3678	$32.59	115.1M	354k	17h 32m	353	13
8	Claude Opus 5.5	0.3676	$34.48	123.3M	365k	14h 56m	393	16
9	Muse Spark 1.3	0.3657	$29.63	28.5M	123k	19h 14m	290	12
10	Claude Haiku 5.5	0.3592	$3.57	53.4M	356k	17h 25m	205	1
11	Claude Opus 5	0.3591	$41.25	57.4M	336k	16h 38m	244	11
12	Claude Haiku 5.5	0.3589	$6.16	99.7M	434k	15h 47m	312	1
13	GPT-6 Astra	0.3557	$452.78	217.0M	444k	15h 46m	527	14
14	Gemini 4 Argon	0.3541	$66.65	65.7M	370k	10h 16m	208	6
15	Gemini 4 Argon	0.3496	$73.61	73.2M	523k	11h 46m	215	9
16	GPT-6 Astra	0.3431	$526.36	253.0M	673k	17h 55m	694	20
17	Claude Sonnet 5.5	0.3387	$39.17	161.4M	527k	11h 58m	374	2
18	Claude Sonnet 5.5	0.3362	$42.30	176.9M	528k	15h 36m	407	2
19	Claude Sonnet 5.5	0.3295	$44.28	184.1M	576k	15h 25m	433	1
20	Gemini 4 Argon	0.3253	$89.40	63.7M	382k	14h 54m	195	7
21	Claude Haiku 5.5	0.3229	$5.52	86.2M	451k	9h 19m	265	3
22	Claude Opus 5	0.3182	$39.66	57.3M	295k	17h 37m	258	3
23	Claude Fable 5.1	0.3166	$19.44	30.2M	179k	16h 28m	214	8
24	Claude Fable 5.1	0.3123	$26.35	42.8M	204k	16h 51m	240	12
25	GPT-5.6 Sol	0.3119	$41.21	43.5M	133k	5h 23m	204	2
26	Claude Fable 5.1	0.3116	$13.99	21.5M	133k	17h 11m	192	11
27	Claude Sonnet 5.5	0.3091	$43.33	180.7M	552k	16h 17m	436	14
28	Claude Fable 5.1	0.3057	$16.57	26.2M	144k	13h 23m	202	11
29	Grok 4.6	0.3041	$128.46	122.4M	984k	15h 5m	545	23
30	GPT-5.6 Sol	0.3039	$14.10	19.5M	85k	2h 25m	137	2
31	Claude Opus 5	0.3001	$47.82	68.3M	371k	18h 4m	251	14
32	Claude Opus 5.5	0.2944	$25.47	87.0M	298k	19h 1m	298	14
33	Claude Fable 5	0.2869	$40.93	27.8M	172k	18h 49m	197	7
34	Claude Fable 5	0.2738	$32.57	21.0M	153k	17h 28m	176	7
35	Claude Haiku 5.5	0.2649	$5.00	78.8M	385k	13h 42m	250	1
36	Claude Haiku 5.5	0.2545	$5.81	88.4M	483k	16h 59m	251	6
37	Claude Fable 5	0.2462	$69.13	51.5M	236k	18h 13m	278	10
38	GPT-5.6 Sol	0.2311	$153.75	128.1M	180k	10h 45m	298	17
39	Grok 4.6	0.2265	$217.69	204.9M	1.2M	19h 25m	878	23
40	GLM-5.3	0.2244	$22.68	51.0M	188k	18h 6m	315	2
41	Claude Fable 5.1	0.2155	$25.98	41.7M	212k	14h 56m	237	10
42	Grok 4.6	0.2107	$132.86	129.3M	708k	19h 25m	571	33
43	GLM-5.3	0.2036	$34.96	74.2M	244k	14h 55m	372	2
44	GLM-5.3 Flash	0.2035	$2.39	39.2M	192k	18h 35m	253	3
45	Gemini 4 Argon	0.1941	$62.22	52.9M	318k	10h 43m	182	5
46	Claude Opus 5.5	0.1928	$24.23	78.6M	314k	16h 53m	281	9
47	GPT-5.6 Sol	0.1902	$64.35	67.7M	163k	7h 2m	279	4
48	Qwen3.8-Max-0902	0.1889	$35.75	119.7M	797k	19h 10m	470	2
49	Qwen3.8-Max-0902	0.1871	$50.73	177.9M	853k	19h 26m	592	1
50	GPT-5.6 Sol	0.1862	$104.80	98.3M	165k	7h 53m	318	8
51	Grok 4.6	0.1844	$95.16	83.3M	284k	18h 58m	362	11
52	DeepSeek V4 Flash Vision Exp	0.1749	$1.32	68.7M	240k	18h 36m	359	6
53	Grok 4.7	0.1697	$27.46	25.8M	199k	3h 25m	133	5
54	Muse Spark 1.2	0.1659	$17.10	36.3M	144k	3h 7m	130	6
55	GLM-5.3	0.1492	$27.34	90.5M	318k	15h 42m	371	3
56	Qwen3.8-Max	0.1460	$13.18	43.8M	302k	19h 37m	333	7
57	Muse Spark 1.3	0.1430	$33.98	30.2M	124k	18h 44m	278	4
58	Claude Fable 5	0.1418	$35.13	22.9M	161k	18h 17m	176	7
59	GLM-5.3 Flash	0.1356	$1.97	44.0M	221k	9h 31m	277	4
60	Muse Spark 1.3	0.1334	$8.50	30.2M	106k	18h 10m	317	7
61	Claude Opus 5	0.1306	$32.65	45.8M	261k	17h 4m	232	9
62	Claude Fable 5	0.1281	$60.54	43.7M	223k	17h 15m	249	11
63	Qwen3.8-Max-0902	0.1258	$56.59	193.9M	1.1M	18h 55m	561	4
64	Grok 4.6	0.1245	$71.38	67.2M	365k	9h 53m	293	10
65	GLM-5.3 Flash	0.1202	$2.02	38.5M	189k	18h 41m	272	4
66	Muse Spark 1.3	0.1187	$59.14	56.9M	180k	14h 50m	367	3
67	Muse Spark 1.2	0.1175	$9.52	47.7M	272k	3h 44m	183	8
68	Gemini 3.7 Flash	0.1130	$4.87	33.1M	132k	3h 51m	226	1
69	Gemini 3.7 Flash	0.1121	$12.12	51.3M	122k	7h 24m	312	4
70	Kimi K3	0.1102	$14.90	32.2M	173k	18h 22m	254	3
71	GLM-5.3 Flash	0.1039	$3.76	97.3M	271k	17h 33m	431	2
72	GLM-5.3	0.1032	$33.80	77.7M	262k	19h 7m	353	2
73	Gemini 3.8 Flash	0.1012	$4.22	32.1M	134k	3h 25m	220	3
74	Muse Spark 1.2	0.1011	$11.96	61.9M	415k	4h 37m	241	3
75	Qwen3.8-Max	0.0985	$20.47	70.2M	412k	18h 53m	478	2
76	Gemini 3.8 Flash	0.0958	$3.90	29.7M	119k	2h 37m	202	3
77	Gemini 3.7 Flash	0.0948	$9.93	38.7M	111k	7h 46m	266	5
78	Qwen3.8-Max-0902	0.0904	$20.15	52.8M	623k	19h 46m	279	2
79	Gemini 3.7 Flash	0.0854	$10.11	35.8M	110k	7h 53m	303	1
80	DeepSeek V4 Flash Vision Exp	0.0826	$1.73	93.5M	273k	12h 25m	457	2
81	Gemini 3.8 Flash	0.0714	$6.78	37.1M	131k	5h 53m	235	4
82	GLM-5.3	0.0000	$49.90	134.3M	340k	19h 4m	469	21
83	GLM-5.3 Flash	0.0000	$1.17	35.7M	150k	19h 11m	277	4
84	Muse Spark 1.3	0.0000	$32.86	29.1M	146k	18h 59m	269	6
85	Qwen3.8-Max	0.0000	$13.84	45.7M	331k	19h 21m	336	2
86	Inkling	0.0000	$5.29	5.5M	26k	1h 28m	151	1
87	Grok 4.7	0.0000	$17.01	16.9M	171k	3h 5m	110	2
88	Grok 4.7	0.0000	$33.74	29.5M	210k	5h 34m	157	5
89	Qwen3.8-Max	0.0000	$27.96	99.7M	412k	19h 32m	527	4
90	Muse Spark 1.2	0.0000	$13.11	77.0M	195k	5h 0m	202	3
91	Inkling	0.0000	$5.01	5.2M	28k	1h 29m	128	1
92	Qwen3.8-Max-0902	0.0000	$24.18	67.0M	860k	20h 1m	341	1
93	Gemini 3.7 Flash	0.0000	$12.20	48.7M	173k	7h 50m	285	2
94	Grok 4.7	0.0000	$9.82	11.5M	148k	1h 49m	85	2
95	Muse Spark 1.2	0.0000	$16.36	73.7M	275k	2h 42m	244	7
96	Gemini 3.8 Flash	0.0000	$6.01	39.8M	130k	3h 31m	254	7
97	Inkling	0.0000	$6.23	6.3M	53k	2h 6m	139	1
98	Gemini 3.8 Flash	0.0000	$6.66	41.6M	150k	3h 57m	251	6
99	Qwen3.8-Max	0.0000	$8.55	26.2M	281k	19h 56m	242	1
100	Grok 4.7	0.0000	$32.99	32.5M	199k	5h 31m	181	4
101	Inkling	0.0000	$5.31	5.5M	23k	2h 3m	134	2
102	Claude Opus 5	0.0000	$41.18	59.8M	304k	17h 59m	267	9
103	Inkling	0.0000	$5.42	5.6M	24k	1h 30m	116	1