Tasks Traces V1 Blog Changelog Contribute GitHub

Traces

Every run: the agent's full trajectory, its submissions over the 20-hour budget, and the final score.

Granite Mamba2 Inference Optimization

task page →110 runs · indexed 2026-10-08

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Sonnet 5.5	0.9284	$98.15	363.7M	2.0M	12h 14m	758	2
2	Gemini 4 Argon	0.9104	$105.47	115.9M	895k	5h 40m	315	6
3	Qwen3.8-Max-0902	0.9095	$72.15	209.6M	2.3M	19h 47m	563	2
4	Gemini 4 Argon	0.9046	$83.33	130.1M	792k	5h 37m	359	3
5	Gemini 4 Argon	0.8993	$145.24	118.2M	769k	6h 54m	331	8
6	GLM-5.3	0.8927	$85.79	297.0M	636k	6h 7m	634	7
7	Claude Haiku 5.5	0.8911	$15.33	230.6M	1.3M	10h 52m	500	2
8	Gemini 4 Argon	0.8884	$91.03	70.1M	500k	3h 12m	195	3
9	Claude Opus 5.5	0.8846	$214.03	690.9M	2.9M	15h 36m	1506	34
10	Gemini 4 Argon	0.8832	$36.18	98.9M	556k	3h 28m	236	3
11	Claude Opus 5.5	0.8817	$132.91	426.3M	1.8M	6h 35m	922	25
12	Claude Opus 5.5	0.8769	$159.17	476.8M	2.5M	11h 3m	1094	29
13	GLM-5.3	0.8719	$51.39	174.8M	618k	4h 8m	382	8
14	DeepSeek V4 Flash Vision Exp	0.8708	$2.68	149.8M	376k	16h 23m	473	5
15	GPT-6 Astra	0.8707	$2626.81	1259.5M	2.3M	18h 25m	2839	58
16	GPT-6 Astra	0.8658	$272.49	128.6M	369k	2h 40m	367	7
17	Claude Sonnet 5.5	0.8655	$166.81	620.5M	3.4M	17h 36m	1346	14
18	Claude Opus 5	0.8619	$158.22	232.0M	1.1M	7h 45m	543	10
19	Grok 4.7	0.8555	$664.66	670.3M	2.4M	20h 0m	2515	55
20	Claude Fable 5.1	0.8458	$138.31	228.0M	1000k	9h 1m	580	15
21	GLM-5.3	0.8401	$45.93	158.7M	655k	5h 28m	343	6
22	Claude Opus 5	0.8400	$159.98	236.7M	1.1M	11h 10m	566	15
23	Claude Fable 5.1	0.8393	$189.96	320.0M	1.2M	8h 30m	718	28
24	Claude Haiku 5.5	0.8379	$8.93	123.1M	1.0M	4h 22m	301	3
25	Claude Fable 5	0.8372	$204.33	157.4M	642k	12h 58m	310	20
26	GPT-6 Astra	0.8334	$251.26	124.6M	331k	2h 35m	409	4
27	Claude Haiku 5.5	0.8309	$11.48	157.4M	1.3M	4h 35m	386	4
28	Grok 4.7	0.8303	$663.28	652.6M	2.1M	14h 34m	2345	24
29	GPT-6 Astra	0.8266	$344.95	163.8M	406k	3h 8m	459	8
30	GPT-6 Astra	0.8198	$254.82	122.3M	354k	2h 28m	374	7
31	Claude Fable 5.1	0.8171	$168.52	281.0M	1.2M	8h 8m	667	16
32	Muse Spark 1.2	0.8168	$5.16	25.8M	238k	1h 23m	169	3
33	Qwen3.8-Max-0902	0.8140	$43.20	91.8M	3.2M	19h 47m	354	1
34	Muse Spark 1.3	0.8136	$31.62	38.2M	374k	3h 49m	186	5
35	Qwen3.8-Max	0.8086	$63.86	200.9M	2.1M	18h 31m	572	2
36	Claude Fable 5.1	0.8047	$204.26	349.5M	1.2M	8h 27m	784	31
37	Muse Spark 1.3	0.8036	$155.17	181.2M	487k	18h 42m	545	11
38	Claude Haiku 5.5	0.8023	$12.99	188.9M	1.3M	8h 53m	451	2
39	GLM-5.3 Flash	0.7990	$11.45	322.1M	969k	12h 30m	791	4
40	Qwen3.8-Max-0902	0.7969	$68.75	217.9M	2.1M	19h 23m	595	2
41	Inkling	0.7968	$5.41	5.6M	24k	1h 10m	98	3
42	Claude Fable 5	0.7947	$155.09	112.6M	581k	5h 11m	247	9
43	Inkling	0.7910	$2.64	2.7M	12k	47m	61	2
44	DeepSeek V4 Flash Vision Exp	0.7876	$3.53	195.8M	540k	19h 32m	519	8
45	Qwen3.8-Max	0.7876	$54.77	177.9M	1.6M	16h 1m	520	5
46	Kimi K3	0.7872	$122.78	337.3M	986k	19h 10m	771	36
47	Claude Fable 5.1	0.7867	$157.78	258.9M	1.2M	7h 40m	582	11
48	GLM-5.3 Flash	0.7842	$10.09	304.8M	714k	7h 37m	595	8
49	Gemini 3.7 Flash	0.7839	$8.97	84.4M	442k	56m	281	3
50	Muse Spark 1.3	0.7809	$183.67	203.2M	648k	10h 29m	557	11
51	Kimi K3	0.7805	$103.11	286.2M	907k	14h 49m	651	10
52	Grok 4.7	0.7778	$748.71	746.1M	3.0M	20h 0m	2763	23
53	Claude Opus 5	0.7718	$80.48	113.3M	644k	3h 51m	243	5
54	Inkling	0.7670	$5.68	5.8M	36k	1h 22m	101	2
55	GPT-5.6 Sol	0.7649	$74.74	74.2M	207k	2h 36m	270	5
56	DeepSeek V4 Flash Vision Exp	0.7578	$8.78	501.2M	1.2M	13h 5m	1157	3
57	Kimi K3	0.7571	$91.87	248.8M	620k	20h 0m	531	50
58	GLM-5.3	0.7528	$38.94	126.7M	486k	4h 14m	351	5
59	Claude Opus 5.5	0.7510	$78.62	226.6M	1.3M	6h 44m	478	12
60	Kimi K3	0.7477	$74.48	205.5M	639k	8h 3m	456	20
61	Grok 4.7	0.7473	$45.19	45.2M	263k	1h 21m	181	3
62	DeepSeek V4 Flash Vision Exp	0.7467	$8.27	472.9M	1.1M	11h 58m	1140	23
63	GLM-5.3 Flash	0.7460	$13.06	377.4M	1.1M	12h 40m	970	2
64	Claude Haiku 5.5	0.7446	$23.17	349.9M	2.1M	10h 30m	763	4
65	GLM-5.3 Flash	0.7421	$9.37	261.1M	646k	9h 26m	535	2
66	GLM-5.3	0.7402	$85.05	310.6M	673k	18h 16m	645	8
67	Claude Fable 5	0.7331	$198.58	154.1M	599k	9h 40m	302	16
68	Grok 4.6	0.7305	$58.74	52.8M	626k	4h 20m	206	13
69	Grok 4.6	0.7245	$46.67	39.7M	692k	3h 57m	169	10
70	Qwen3.8-Max	0.7188	$63.39	207.4M	1.7M	17h 14m	557	9
71	Muse Spark 1.2	0.7107	$36.53	207.7M	800k	2h 20m	527	1
72	Qwen3.8-Max	0.7023	$29.64	70.4M	1.9M	19h 40m	330	1
73	Claude Opus 5	0.6979	$228.77	352.0M	1.4M	11h 8m	751	18
74	Claude Sonnet 5.5	0.6871	$127.04	479.4M	2.5M	17h 41m	1063	10
75	Qwen3.8-Max	0.6658	$53.16	162.1M	1.8M	18h 29m	495	2
76	Grok 4.6	0.6586	$55.70	47.5M	700k	4h 6m	194	10
77	Grok 4.6	0.6473	$57.88	51.1M	725k	4h 13m	210	5
78	Muse Spark 1.3	0.6457	$320.20	329.1M	590k	13h 47m	773	12
79	GPT-5.6 Sol	0.6303	$111.93	116.6M	261k	11h 2m	425	3
80	Muse Spark 1.3	0.6302	$29.92	88.8M	460k	3h 13m	325	9
81	Kimi K3	0.6169	$112.73	307.9M	980k	13h 16m	708	31
82	Qwen3.8-Max-0902	0.6109	$38.46	77.0M	3.0M	18h 44m	298	2
83	GPT-5.6 Sol	0.6093	$23.53	32.7M	162k	1h 28m	207	3
84	Claude Fable 5	0.5974	$289.56	218.1M	971k	11h 20m	529	29
85	Claude Opus 5.5	0.5826	$111.15	316.0M	1.9M	10h 19m	731	29
86	Claude Sonnet 5.5	0.5555	$156.83	594.5M	3.0M	18h 16m	1275	4
87	Claude Fable 5	0.5364	$304.87	227.0M	1.1M	13h 5m	532	22
88	Claude Opus 5	0.5059	$98.60	149.0M	645k	5h 53m	287	7
89	Gemini 3.8 Flash	0.5042	$12.98	108.1M	394k	2h 18m	329	5
90	Claude Sonnet 5.5	0.4810	$155.71	584.6M	3.1M	16h 20m	1290	8
91	DeepSeek V4 Flash Vision Exp	0.4680	$3.00	159.1M	520k	6h 30m	442	5
92	GPT-5.6 Sol	0.4658	$119.15	122.5M	308k	3h 17m	463	6
93	GLM-5.3 Flash	0.4647	$8.11	210.3M	541k	9h 13m	473	3
94	Grok 4.7	0.4410	$49.65	48.3M	283k	2h 1m	184	13
95	Gemini 3.8 Flash	0.4012	$12.80	133.1M	348k	1h 19m	360	2
96	Gemini 3.7 Flash	0.3912	$15.89	144.8M	798k	2h 17m	479	5
97	Gemini 3.8 Flash	0.3668	$11.65	115.4M	351k	1h 8m	353	6
98	GPT-5.6 Sol	0.3383	$180.46	175.3M	266k	4h 7m	474	10
99	Gemini 3.7 Flash	0.3149	$10.74	97.9M	514k	1h 12m	278	2
100	Gemini 3.8 Flash	0.2792	$7.71	71.8M	268k	1h 9m	283	2
101	Grok 4.6	0.2777	$27.64	23.0M	551k	3h 18m	127	2
102	Gemini 3.7 Flash	0.2425	$12.51	121.4M	559k	2h 10m	326	2
103	Gemini 3.7 Flash	0.2101	$6.70	49.8M	379k	47m	212	3
104	Inkling	0.0000	$7.50	7.7M	51k	1h 46m	135	1
105	Gemini 3.8 Flash	0.0000	$15.57	160.4M	440k	1h 35m	409	1
106	Qwen3.8-Max-0902	0.0000	$60.04	188.3M	1.9M	19h 52m	497	3
107	Muse Spark 1.2	0.0000	$19.40	92.0M	1.0M	4h 3m	307	3
108	Muse Spark 1.2	0.0000	$9.93	53.1M	354k	2h 11m	258	6
109	Muse Spark 1.2	0.0000	$8.05	39.0M	223k	1h 17m	201	4
110	Inkling	0.0000	$4.66	4.8M	18k	1h 13m	91	2