Tasks Traces V1 Blog Changelog Contribute GitHub

Traces

Every run: the agent's full trajectory, its submissions over the 20-hour budget, and the final score.

SGLang Inference System Optimization

task page →110 runs · indexed 2026-10-08

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	GPT-6 Astra	0.5349	$1954.03	921.6M	1.6M	19h 56m	1932	31
2	Gemini 4 Argon	0.4359	$139.21	309.9M	955k	8h 57m	746	6
3	Claude Sonnet 5.5	0.3720	$144.95	566.2M	2.4M	18h 50m	1343	11
4	Claude Haiku 5.5	0.1173	$15.92	254.5M	1.2M	10h 40m	625	2
5	Muse Spark 1.3	0.1159	$180.28	204.8M	347k	10h 27m	670	14
6	GPT-5.6 Sol	0.1092	$291.23	284.1M	251k	5h 12m	616	12
7	GPT-5.6 Sol	0.0675	$443.00	437.3M	281k	6h 23m	877	14
8	Grok 4.7	0.0552	$802.99	801.5M	2.3M	19h 22m	3183	24
9	GLM-5.3	0.0534	$68.77	241.2M	448k	15h 36m	650	8
10	Muse Spark 1.2	0.0532	$132.47	498.4M	1.4M	17h 13m	1444	3
11	Claude Fable 5	0.0442	$246.50	205.1M	519k	8h 17m	481	10
12	Grok 4.6	0.0322	$488.14	454.2M	2.3M	19h 33m	1703	83
13	Claude Opus 5	0.0249	$124.87	207.2M	534k	12h 23m	460	7
14	Qwen3.8-Max-0902	0.0242	$30.19	91.7M	767k	20h 6m	434	1
15	Claude Opus 5	0.0223	$101.43	161.0M	523k	12h 35m	390	8
16	Qwen3.8-Max	0.0214	$45.80	153.5M	1.1M	15h 59m	715	7
17	GLM-5.3	0.0211	$67.37	244.9M	383k	14h 17m	698	20
18	Gemini 3.8 Flash	0.0190	$30.46	330.6M	252k	3h 24m	709	11
19	Gemini 3.7 Flash	0.0169	$19.15	195.9M	289k	3h 33m	505	4
20	GPT-5.6 Sol	0.0168	$283.36	279.0M	247k	6h 47m	655	21
21	Kimi K3	0.0103	$110.14	311.1M	507k	20h 1m	740	11
22	Muse Spark 1.2	0.0099	$100.58	403.5M	1.7M	11h 46m	1193	8
23	Muse Spark 1.2	0.0081	$15.03	90.2M	152k	1h 36m	211	7
24	Qwen3.8-Max	0.0081	$33.42	99.7M	1.3M	15h 1m	574	8
25	Gemini 3.8 Flash	0.0058	$23.48	261.8M	192k	2h 8m	563	4
26	Claude Opus 5	0.0048	$110.39	179.3M	515k	12h 40m	431	7
27	Muse Spark 1.2	0.0047	$67.55	167.8M	657k	12h 12m	541	6
28	Grok 4.6	0.0025	$162.38	158.0M	937k	10h 27m	649	14
29	Gemini 3.8 Flash	0.0021	$39.53	434.9M	429k	7h 2m	1119	13
30	Grok 4.6	0.0012	$313.02	296.5M	1.9M	18h 41m	1120	19
31	Qwen3.8-Max-0902	0.0009	$28.62	81.5M	1.2M	20h 5m	391	1
32	Qwen3.8-Max-0902	0.0009	$26.00	76.8M	998k	20h 6m	387	2
33	Grok 4.6	0.0007	$96.00	91.3M	481k	3h 53m	329	3
34	Qwen3.8-Max	0.0004	$47.32	149.7M	1.5M	17h 59m	651	3
35	GLM-5.3	0.0004	$46.58	170.2M	369k	10h 37m	591	7
36	Claude Haiku 5.5	0.0000	$14.49	235.4M	958k	9h 43m	594	3
37	Claude Haiku 5.5	0.0000	$14.92	236.6M	1.1M	7h 37m	597	2
38	Claude Opus 5.5	0.0000	$159.94	559.4M	1.8M	19h 25m	1329	26
39	Claude Fable 5	0.0000	$202.86	169.0M	429k	14h 47m	436	14
40	GLM-5.3	0.0000	$50.83	180.8M	386k	12h 52m	576	4
41	Claude Fable 5.1	0.0000	$219.62	392.8M	963k	7h 58m	844	9
42	Muse Spark 1.2	0.0000	$32.81	65.0M	127k	4h 50m	276	3
43	Claude Fable 5.1	0.0000	$193.70	342.2M	936k	9h 59m	820	11
44	DeepSeek V4 Flash Vision Exp	0.0000	$2.44	138.4M	299k	9h 52m	531	4
45	Claude Haiku 5.5	0.0000	$13.92	215.0M	1.1M	9h 1m	513	3
46	DeepSeek V4 Flash Vision Exp	0.0000	$5.46	332.4M	492k	10h 53m	782	15
47	Grok 4.7	0.0000	$211.95	219.1M	541k	7h 14m	876	5
48	DeepSeek V4 Flash Vision Exp	0.0000	$3.59	206.9M	443k	9h 31m	593	16
49	Claude Opus 5.5	0.0000	$183.76	644.5M	2.0M	19h 7m	1476	23
50	Gemini 4 Argon	0.0000	$275.58	621.4M	1.5M	15h 16m	1391	22
51	Kimi K3	0.0000	$185.83	517.0M	1.5M	20h 0m	1188	13
52	GPT-6 Astra	0.0000	$2665.44	1249.7M	2.4M	19h 33m	2699	42
53	GPT-5.6 Sol	0.0000	$109.76	114.5M	139k	5h 57m	401	7
54	Claude Sonnet 5.5	0.0000	$131.29	505.6M	2.3M	19h 12m	1248	7
55	Grok 4.7	0.0000	$51.59	50.1M	157k	1h 43m	212	5
56	Gemini 4 Argon	0.0000	$100.53	344.7M	936k	8h 35m	828	8
57	Gemini 3.8 Flash	0.0000	$24.27	258.5M	198k	4h 49m	607	3
58	GLM-5.3 Flash	0.0000	$3.26	101.7M	249k	9h 20m	478	7
59	Claude Opus 5.5	0.0000	$185.38	644.4M	2.1M	18h 30m	1447	29
60	Claude Opus 5	0.0000	$89.33	141.7M	467k	15h 55m	376	11
61	Inkling	0.0000	$4.87	5.1M	12k	1h 40m	92	3
62	Grok 4.7	0.0000	$100.20	104.5M	326k	3h 50m	470	5
63	Claude Fable 5.1	0.0000	$177.26	316.2M	794k	11h 11m	833	9
64	Gemini 4 Argon	0.0000	$197.19	445.3M	1.3M	12h 41m	1022	9
65	Kimi K3	0.0000	$246.45	710.4M	1.8M	20h 0m	1736	14
66	Gemini 3.8 Flash	0.0000	$21.47	225.2M	181k	2h 13m	571	5
67	Muse Spark 1.3	0.0000	$429.35	454.7M	706k	19h 28m	1165	8
68	Claude Haiku 5.5	0.0000	$24.57	393.7M	1.7M	13h 34m	905	6
69	Grok 4.7	0.0000	$565.89	566.8M	1.5M	11h 16m	2189	10
70	GLM-5.3	0.0000	$68.85	206.5M	390k	14h 23m	607	17
71	Claude Sonnet 5.5	0.0000	$90.56	360.5M	1.4M	18h 46m	879	8
72	Inkling	0.0000	$7.60	8.0M	10k	1h 46m	120	2
73	Gemini 3.7 Flash	0.0000	$9.47	100.2M	149k	1h 43m	316	4
74	Muse Spark 1.3	0.0000	$58.36	60.2M	197k	8h 23m	386	3
75	GPT-5.6 Sol	0.0000	$182.18	175.0M	189k	6h 4m	485	8
76	Inkling	0.0000	$5.18	5.4M	12k	1h 28m	91	2
77	DeepSeek V4 Flash Vision Exp	0.0000	$3.41	198.5M	360k	8h 1m	562	4
78	Claude Opus 5.5	0.0000	$134.68	471.5M	1.5M	18h 23m	1059	21
79	Grok 4.6	0.0000	$419.76	405.9M	2.2M	19h 4m	1586	60
80	GPT-6 Astra	0.0000	$2441.52	1153.7M	2.1M	19h 6m	2553	41
81	Claude Fable 5.1	0.0000	$182.53	329.2M	746k	9h 31m	841	21
82	GPT-6 Astra	0.0000	$2574.70	1208.7M	2.3M	17h 31m	2622	65
83	GLM-5.3 Flash	0.0000	$5.76	124.8M	287k	14h 48m	473	3
84	Claude Opus 5.5	0.0000	$150.29	514.2M	1.7M	17h 0m	1206	15
85	Inkling	0.0000	$6.42	6.7M	16k	1h 12m	100	1
86	DeepSeek V4 Flash Vision Exp	0.0000	$3.54	209.2M	364k	8h 32m	632	9
87	GPT-6 Astra	0.0000	$1722.33	819.7M	1.5M	19h 18m	1844	42
88	Qwen3.8-Max	0.0000	$42.20	143.0M	952k	17h 51m	604	1
89	Kimi K3	0.0000	$49.64	141.5M	328k	6h 19m	529	7
90	Inkling	0.0000	$14.76	15.4M	36k	34m	194	2
91	Claude Sonnet 5.5	0.0000	$128.12	516.0M	1.9M	18h 29m	1195	8
92	Gemini 3.7 Flash	0.0000	$9.02	85.7M	143k	1h 52m	306	1
93	Gemini 4 Argon	0.0000	$160.21	500.1M	1.5M	13h 56m	1169	20
94	Claude Fable 5	0.0000	$289.82	249.1M	515k	7h 2m	543	12
95	GLM-5.3 Flash	0.0000	$3.52	102.4M	346k	8h 20m	418	2
96	Qwen3.8-Max	0.0000	$47.25	152.8M	1.4M	17h 10m	642	3
97	GLM-5.3 Flash	0.0000	$5.38	133.7M	325k	9h 34m	541	2
98	Gemini 3.7 Flash	0.0000	$12.58	90.8M	182k	4h 36m	317	1
99	Kimi K3	0.0000	$124.72	355.5M	911k	15h 41m	948	24
100	GLM-5.3 Flash	0.0000	$5.24	138.3M	282k	10h 25m	542	3
101	Qwen3.8-Max-0902	0.0000	$29.45	86.5M	1.2M	20h 1m	413	1
102	Claude Fable 5	0.0000	$175.12	145.0M	372k	12h 9m	427	14
103	Claude Sonnet 5.5	0.0000	$140.54	573.5M	2.0M	18h 42m	1322	5
104	Claude Fable 5	0.0000	$186.99	154.4M	396k	7h 35m	429	9
105	Muse Spark 1.3	0.0000	$52.36	191.1M	288k	7h 43m	680	5
106	Muse Spark 1.3	0.0000	$73.59	99.0M	263k	5h 24m	459	3
107	Claude Opus 5	0.0000	$158.96	248.3M	890k	16h 5m	693	15
108	Claude Fable 5.1	0.0000	$226.14	406.3M	953k	11h 14m	902	15
109	Gemini 3.7 Flash	0.0000	$6.58	51.9M	91k	1h 41m	296	4
110	Qwen3.8-Max-0902	0.0000	$22.37	62.1M	1.0M	20h 3m	330	1