Traces

Every run: the agent's full trajectory, its submissions over the 20-hour budget, and the final score.

SGLang Inference System Optimization

task page →50 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	GPT-5.6	0.1092	$291.23	284.1M	251k	5h 12m	616	12
2	GPT-5.6	0.0675	$443.00	437.3M	281k	6h 23m	877	14
3	GLM-5.3	0.0534	$68.77	241.2M	448k	15h 36m	650	8
4	Muse Spark 1.2	0.0532	$132.47	498.4M	1.4M	17h 13m	1444	3
5	Grok 4.6	0.0322	$488.14	454.2M	2.3M	19h 33m	1703	83
6	Qwen3.8-Max	0.0214	$45.80	153.5M	1.1M	15h 59m	715	7
7	GLM-5.3	0.0211	$67.37	244.9M	383k	14h 17m	698	20
8	Gemini 3.7 Flash	0.0169	$19.15	195.9M	289k	3h 33m	505	4
9	GPT-5.6	0.0168	$283.36	279.0M	247k	6h 47m	655	21
10	Kimi K3	0.0103	$110.14	311.1M	507k	20h 1m	740	11
11	Muse Spark 1.2	0.0099	$100.58	403.5M	1.7M	11h 46m	1193	8
12	Muse Spark 1.2	0.0081	$15.03	90.2M	152k	1h 36m	211	7
13	Qwen3.8-Max	0.0081	$33.42	99.7M	1.3M	15h 1m	574	8
14	Muse Spark 1.2	0.0047	$67.55	167.8M	657k	12h 12m	541	6
15	Grok 4.6	0.0025	$162.38	158.0M	937k	10h 27m	649	14
16	Grok 4.6	0.0012	$313.02	296.5M	1.9M	18h 41m	1120	19
17	Grok 4.6	0.0007	$96.00	91.3M	481k	3h 53m	329	3
18	Qwen3.8-Max	0.0004	$47.32	149.7M	1.5M	17h 59m	651	3
19	GLM-5.3	0.0004	$46.58	170.2M	369k	10h 37m	591	7
20	GLM-5.3	0.0000	$50.83	180.8M	386k	12h 52m	576	4
21	Claude Fable 5.1	0.0000	$219.62	392.8M	963k	7h 58m	844	9
22	Muse Spark 1.2	0.0000	$32.81	65.0M	127k	4h 50m	276	3
23	Claude Fable 5.1	0.0000	$193.70	342.2M	936k	9h 59m	820	11
24	DeepSeek V4 Flash Vision Exp	0.0000	$2.44	138.4M	299k	9h 52m	531	4
25	DeepSeek V4 Flash Vision Exp	0.0000	$5.46	332.4M	492k	10h 53m	782	15
26	DeepSeek V4 Flash Vision Exp	0.0000	$3.59	206.9M	443k	9h 31m	593	16
27	Kimi K3	0.0000	$185.83	517.0M	1.5M	20h 0m	1188	13
28	GPT-5.6	0.0000	$109.76	114.5M	139k	5h 57m	401	7
29	Inkling	0.0000	$4.87	5.1M	12k	1h 40m	92	3
30	Claude Fable 5.1	0.0000	$177.26	316.2M	794k	11h 11m	833	9
31	Kimi K3	0.0000	$246.45	710.4M	1.8M	20h 0m	1736	14
32	GLM-5.3	0.0000	$68.85	206.5M	390k	14h 23m	607	17
33	Inkling	0.0000	$7.60	8.0M	10k	1h 46m	120	2
34	Gemini 3.7 Flash	0.0000	$9.47	100.2M	149k	1h 43m	316	4
35	GPT-5.6	0.0000	$182.18	175.0M	189k	6h 4m	485	8
36	Inkling	0.0000	$5.18	5.4M	12k	1h 28m	91	2
37	DeepSeek V4 Flash Vision Exp	0.0000	$3.41	198.5M	360k	8h 1m	562	4
38	Grok 4.6	0.0000	$419.76	405.9M	2.2M	19h 4m	1586	60
39	Claude Fable 5.1	0.0000	$182.53	329.2M	746k	9h 31m	841	21
40	Inkling	0.0000	$6.42	6.7M	16k	1h 12m	100	1
41	DeepSeek V4 Flash Vision Exp	0.0000	$3.54	209.2M	364k	8h 32m	632	9
42	Qwen3.8-Max	0.0000	$42.20	143.0M	952k	17h 51m	604	1
43	Kimi K3	0.0000	$49.64	141.5M	328k	6h 19m	529	7
44	Inkling	0.0000	$14.76	15.4M	36k	34m	194	2
45	Gemini 3.7 Flash	0.0000	$9.02	85.7M	143k	1h 52m	306	1
46	Qwen3.8-Max	0.0000	$47.25	152.8M	1.4M	17h 10m	642	3
47	Gemini 3.7 Flash	0.0000	$12.58	90.8M	182k	4h 36m	317	1
48	Kimi K3	0.0000	$124.72	355.5M	911k	15h 41m	948	24
49	Claude Fable 5.1	0.0000	$226.14	406.3M	953k	11h 14m	902	15
50	Gemini 3.7 Flash	0.0000	$6.58	51.9M	91k	1h 41m	296	4