Traces

Every run: the agent's full trajectory, its submissions over the 20-hour budget, and the final score.

Flight-Sim Renderer in OpenGL

task page →50 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Fable 5.1	0.4716	$208.91	376.5M	866k	17h 50m	1017	10
2	Claude Fable 5.1	0.3532	$168.41	314.0M	539k	12h 53m	608	4
3	Claude Fable 5.1	0.3481	$106.70	193.6M	429k	9h 8m	454	13
4	Claude Fable 5.1	0.3402	$125.75	229.9M	473k	10h 25m	482	6
5	GPT-5.6	0.2171	$270.86	274.0M	362k	12h 30m	806	4
6	Claude Fable 5.1	0.1985	$113.40	205.2M	463k	5h 28m	430	3
7	GPT-5.6	0.1552	$336.67	340.6M	347k	11h 28m	906	14
8	GPT-5.6	0.1222	$253.61	257.1M	305k	8h 28m	760	3
9	GPT-5.6	0.1119	$309.74	312.3M	342k	9h 22m	857	12
10	GPT-5.6	0.1032	$420.32	406.2M	354k	11h 35m	880	13
11	Grok 4.6	0.0526	$317.62	274.4M	1.8M	19h 55m	1078	110
12	Grok 4.6	0.0382	$286.38	260.8M	1.4M	19h 57m	984	103
13	Grok 4.6	0.0273	$488.84	437.7M	2.0M	19h 56m	1536	307
14	Grok 4.6	0.0266	$454.46	403.4M	2.0M	19h 58m	1435	75
15	Kimi K3	0.0249	$126.07	350.4M	1.1M	19h 56m	936	7
16	Kimi K3	0.0214	$168.60	476.4M	1.3M	20h 1m	1079	5
17	GLM-5.3	0.0166	$91.30	316.9M	852k	19h 35m	891	7
18	Grok 4.6	0.0147	$328.36	283.8M	1.8M	19h 57m	1084	57
19	Gemini 3.7 Flash	0.0116	$11.28	68.7M	237k	4h 7m	304	2
20	Kimi K3	0.0104	$167.00	460.9M	1.4M	20h 1m	1039	1
21	Kimi K3	0.0079	$184.18	516.4M	1.3M	19h 37m	1094	2
22	DeepSeek V4 Flash Vision Exp	0.0069	$10.96	663.9M	1.1M	19h 2m	1527	2
23	DeepSeek V4 Flash Vision Exp	0.0065	$10.60	633.4M	1.1M	15h 56m	1516	8
24	Gemini 3.7 Flash	0.0064	$35.75	195.3M	360k	8h 28m	555	1
25	GLM-5.3	0.0054	$111.06	362.5M	755k	19h 29m	893	10
26	Gemini 3.7 Flash	0.0054	$6.52	51.3M	224k	2h 40m	274	2
27	GLM-5.3	0.0038	$97.66	297.0M	573k	19h 4m	633	5
28	DeepSeek V4 Flash Vision Exp	0.0028	$12.40	756.6M	1.2M	19h 8m	1756	39
29	DeepSeek V4 Flash Vision Exp	0.0027	$6.77	415.2M	638k	18h 28m	837	7
30	Gemini 3.7 Flash	0.0020	$17.05	105.9M	300k	5h 32m	381	1
31	Qwen3.8-Max	0.0018	$183.02	558.1M	1.7M	19h 56m	1375	1
32	Qwen3.8-Max	0.0016	$153.43	448.8M	1.7M	19h 59m	1202	1
33	Qwen3.8-Max	0.0010	$149.46	419.0M	1.2M	19h 25m	993	2
34	Gemini 3.7 Flash	0.0009	$11.28	77.4M	230k	5h 1m	369	3
35	GLM-5.3	0.0006	$94.18	300.9M	621k	18h 57m	592	10
36	Qwen3.8-Max	0.0001	$169.64	519.3M	1.6M	20h 0m	1239	1
37	Muse Spark 1.2	0.0000	$7.77	40.8M	287k	2h 9m	182	5
38	Inkling	0.0000	$6.72	7.0M	25k	22m	138	2
39	Qwen3.8-Max	0.0000	$189.91	559.5M	1.9M	19h 51m	1464	3
40	Kimi K3	0.0000	$182.81	498.0M	1.4M	20h 1m	1063	2
41	Muse Spark 1.2	0.0000	$8.07	38.9M	412k	3h 7m	202	4
42	DeepSeek V4 Flash Vision Exp	0.0000	$11.01	666.7M	1.1M	19h 11m	1467	11
43	Muse Spark 1.2	0.0000	$6.30	24.2M	135k	1h 37m	159	4
44	GLM-5.3	0.0000	$103.09	375.1M	554k	19h 46m	790	12
45	Inkling	0.0000	$6.59	6.8M	31k	2h 10m	146	2
46	Muse Spark 1.2	0.0000†	$2.64	12.4M	42k	2h 25m	90	2
47	Inkling	0.0000	$6.57	6.8M	31k	35m	156	1
48	Inkling	0.0000	$4.24	4.4M	17k	41m	109	1
49	Inkling	0.0000	$8.87	9.2M	43k	1h 49m	170	2
50	Muse Spark 1.2	0.0000†	$1.72	8.1M	39k	1h 17m	88	1

† score adjusted by manual review; the trace's Judge tab carries the measured score and the rationale.