Tasks Traces V1 Blog Changelog Contribute GitHub

Traces

Every run: the agent's full trajectory, its submissions over the 20-hour budget, and the final score.

Flight-Sim Renderer in OpenGL

task page →110 runs · indexed 2026-10-08

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	GPT-6 Astra	0.9965	$2541.33	1210.9M	2.5M	19h 50m	2830	27
2	GPT-6 Astra	0.9845	$2685.88	1278.3M	2.6M	20h 0m	2961	38
3	GPT-6 Astra	0.9844	$2735.71	1297.4M	2.6M	19h 42m	2884	29
4	GPT-6 Astra	0.9841	$2589.72	1228.2M	2.6M	19h 59m	2801	34
5	GPT-6 Astra	0.9841	$2728.19	1288.8M	2.7M	19h 57m	2941	47
6	Gemini 4 Argon	0.9334	$206.78	382.0M	1.3M	15h 47m	877	5
7	Claude Opus 5.5	0.9097	$156.04	545.9M	1.3M	19h 52m	1288	19
8	Claude Sonnet 5.5	0.9083	$181.73	740.0M	2.3M	18h 47m	1546	6
9	Gemini 4 Argon	0.9080	$207.90	379.4M	1.5M	15h 34m	854	10
10	Claude Sonnet 5.5	0.8981	$169.21	677.4M	2.4M	19h 40m	1549	4
11	Claude Opus 5.5	0.8821	$144.22	511.3M	1.0M	18h 11m	1074	26
12	Claude Opus 5.5	0.8618	$140.79	553.3M	1.1M	19h 44m	1191	4
13	Claude Opus 5.5	0.8540	$182.36	671.0M	1.3M	19h 5m	1637	23
14	Gemini 4 Argon	0.8458	$261.90	442.3M	1.8M	19h 2m	1056	6
15	Claude Sonnet 5.5	0.7941	$169.89	694.5M	2.3M	19h 11m	1580	2
16	Gemini 4 Argon	0.7148	$130.35	185.9M	816k	8h 32m	449	3
17	Gemini 4 Argon	0.7072	$170.81	258.5M	1.2M	8h 45m	673	3
18	Claude Sonnet 5.5	0.7057	$164.69	656.9M	2.0M	19h 44m	1566	6
19	Claude Sonnet 5.5	0.6877	$201.48	823.9M	2.7M	19h 38m	1897	8
20	Claude Opus 5.5	0.6632	$158.77	603.5M	1.1M	18h 8m	1203	7
21	Claude Fable 5.1	0.4716	$208.91	376.5M	866k	17h 50m	1017	10
22	Claude Fable 5.1	0.3532	$168.41	314.0M	539k	12h 53m	608	4
23	Claude Fable 5.1	0.3481	$106.70	193.6M	429k	9h 8m	454	13
24	Claude Fable 5.1	0.3402	$125.75	229.9M	473k	10h 25m	482	6
25	Claude Opus 5	0.2471	$159.25	261.3M	595k	19h 14m	530	3
26	GPT-5.6 Sol	0.2171	$270.86	274.0M	362k	12h 30m	806	4
27	Claude Haiku 5.5	0.2045	$31.05	511.9M	1.8M	18h 49m	1138	3
28	Claude Opus 5	0.2039	$238.32	344.9M	699k	19h 25m	735	12
29	Claude Fable 5.1	0.1985	$113.40	205.2M	463k	5h 28m	430	3
30	Claude Fable 5	0.1934	$427.55	353.3M	996k	18h 35m	924	3
31	Claude Haiku 5.5	0.1872	$31.43	521.7M	1.9M	17h 21m	1152	5
32	Claude Opus 5	0.1567	$190.60	308.2M	776k	19h 30m	742	5
33	GPT-5.6 Sol	0.1552	$336.67	340.6M	347k	11h 28m	906	14
34	Claude Fable 5	0.1550	$396.50	323.2M	982k	19h 5m	848	7
35	Grok 4.7	0.1336	$487.10	480.1M	1.3M	20h 1m	1837	77
36	Claude Fable 5	0.1319	$351.84	291.9M	807k	19h 38m	711	8
37	Claude Opus 5	0.1302	$226.32	342.2M	722k	19h 35m	760	14
38	GPT-5.6 Sol	0.1222	$253.61	257.1M	305k	8h 28m	760	3
39	Grok 4.7	0.1182	$509.72	506.8M	1.5M	18h 52m	2068	85
40	Grok 4.7	0.1138	$588.60	597.8M	1.8M	19h 55m	2409	106
41	GPT-5.6 Sol	0.1119	$309.74	312.3M	342k	9h 22m	857	12
42	Muse Spark 1.3	0.1055	$206.91	689.2M	886k	10h 35m	1681	1
43	GPT-5.6 Sol	0.1032	$420.32	406.2M	354k	11h 35m	880	13
44	Claude Fable 5	0.0950	$314.88	250.5M	673k	18h 55m	484	7
45	Claude Haiku 5.5	0.0884	$18.09	294.8M	1.2M	8h 15m	682	4
46	Claude Haiku 5.5	0.0726	$19.59	319.7M	1.3M	12h 4m	710	5
47	Grok 4.7	0.0713	$540.56	547.5M	1.5M	19h 14m	2247	102
48	Grok 4.6	0.0526	$317.62	274.4M	1.8M	19h 55m	1078	110
49	Muse Spark 1.3	0.0410	$299.15	649.2M	932k	19h 37m	1588	2
50	Muse Spark 1.3	0.0386	$233.75	628.1M	517k	18h 2m	1282	5
51	Grok 4.6	0.0382	$286.38	260.8M	1.4M	19h 57m	984	103
52	Gemini 3.8 Flash	0.0349	$41.18	386.0M	562k	10h 22m	970	4
53	Grok 4.6	0.0273	$488.84	437.7M	2.0M	19h 56m	1536	307
54	Grok 4.6	0.0266	$454.46	403.4M	2.0M	19h 58m	1435	75
55	Muse Spark 1.3	0.0254	$147.68	673.6M	1.0M	19h 47m	1561	8
56	Muse Spark 1.3	0.0250	$351.72	511.0M	807k	19h 59m	1300	1
57	Kimi K3	0.0249	$126.07	350.4M	1.1M	19h 56m	936	7
58	Claude Opus 5	0.0249	$181.57	297.4M	666k	19h 31m	602	6
59	Kimi K3	0.0214	$168.60	476.4M	1.3M	20h 1m	1079	5
60	Grok 4.7	0.0196	$518.13	515.0M	1.7M	19h 40m	2079	58
61	GLM-5.3	0.0166	$91.30	316.9M	852k	19h 35m	891	7
62	Grok 4.6	0.0147	$328.36	283.8M	1.8M	19h 57m	1084	57
63	Claude Haiku 5.5	0.0140	$10.65	174.9M	638k	2h 59m	351	5
64	Gemini 3.8 Flash	0.0130	$23.89	196.2M	401k	6h 33m	507	3
65	Gemini 3.7 Flash	0.0116	$11.28	68.7M	237k	4h 7m	304	2
66	Kimi K3	0.0104	$167.00	460.9M	1.4M	20h 1m	1039	1
67	Gemini 3.8 Flash	0.0089	$32.30	275.0M	453k	6h 8m	610	4
68	Kimi K3	0.0079	$184.18	516.4M	1.3M	19h 37m	1094	2
69	DeepSeek V4 Flash Vision Exp	0.0069	$10.96	663.9M	1.1M	19h 2m	1527	2
70	DeepSeek V4 Flash Vision Exp	0.0065	$10.60	633.4M	1.1M	15h 56m	1516	8
71	Gemini 3.7 Flash	0.0064	$35.75	195.3M	360k	8h 28m	555	1
72	GLM-5.3	0.0054	$111.06	362.5M	755k	19h 29m	893	10
73	Gemini 3.7 Flash	0.0054	$6.52	51.3M	224k	2h 40m	274	2
74	GLM-5.3	0.0038	$97.66	297.0M	573k	19h 4m	633	5
75	DeepSeek V4 Flash Vision Exp	0.0028	$12.40	756.6M	1.2M	19h 8m	1756	39
76	DeepSeek V4 Flash Vision Exp	0.0027	$6.77	415.2M	638k	18h 28m	837	7
77	Gemini 3.7 Flash	0.0020	$17.05	105.9M	300k	5h 32m	381	1
78	Qwen3.8-Max	0.0018	$183.02	558.1M	1.7M	19h 56m	1375	1
79	Qwen3.8-Max	0.0016	$153.43	448.8M	1.7M	19h 59m	1202	1
80	GLM-5.3 Flash	0.0015	$14.67	306.8M	449k	20h 1m	746	1
81	Qwen3.8-Max	0.0010	$149.46	419.0M	1.2M	19h 25m	993	2
82	GLM-5.3 Flash	0.0010	$16.55	339.1M	548k	17h 44m	740	5
83	Gemini 3.7 Flash	0.0009	$11.28	77.4M	230k	5h 1m	369	3
84	Gemini 3.8 Flash	0.0007	$23.00	212.3M	376k	5h 7m	526	3
85	GLM-5.3	0.0006	$94.18	300.9M	621k	18h 57m	592	10
86	GLM-5.3 Flash	0.0002	$12.16	275.0M	487k	19h 36m	628	3
87	GLM-5.3 Flash	0.0001	$17.12	399.9M	616k	15h 49m	828	5
88	Qwen3.8-Max	0.0001	$169.64	519.3M	1.6M	20h 0m	1239	1
89	Muse Spark 1.2	0.0000	$7.77	40.8M	287k	2h 9m	182	5
90	Inkling	0.0000	$6.72	7.0M	25k	22m	138	2
91	Qwen3.8-Max	0.0000	$189.91	559.5M	1.9M	19h 51m	1464	3
92	Kimi K3	0.0000	$182.81	498.0M	1.4M	20h 1m	1063	2
93	Muse Spark 1.2	0.0000	$8.07	38.9M	412k	3h 7m	202	4
94	DeepSeek V4 Flash Vision Exp	0.0000	$11.01	666.7M	1.1M	19h 11m	1467	11
95	Qwen3.8-Max-0902	0.0000	$58.14	147.3M	957k	20h 6m	434	1
96	Qwen3.8-Max-0902	0.0000	$33.94	103.0M	952k	20h 7m	367	1
97	Muse Spark 1.2	0.0000	$6.30	24.2M	135k	1h 37m	159	4
98	Qwen3.8-Max-0902	0.0000	$41.85	96.9M	886k	20h 4m	414	1
99	Claude Fable 5	0.0000	$411.65	293.0M	731k	19h 45m	698	3
100	GLM-5.3	0.0000	$103.09	375.1M	554k	19h 46m	790	12
101	Gemini 3.8 Flash	0.0000	$53.67	394.0M	628k	13h 26m	1068	4
102	Inkling	0.0000	$6.59	6.8M	31k	2h 10m	146	2
103	Muse Spark 1.2	0.0000†	$2.64	12.4M	42k	2h 25m	90	2
104	Qwen3.8-Max-0902	0.0000	$50.29	127.1M	914k	20h 7m	479	1
105	Inkling	0.0000	$6.57	6.8M	31k	35m	156	1
106	GLM-5.3 Flash	0.0000	$17.57	389.3M	753k	20h 0m	891	8
107	Qwen3.8-Max-0902	0.0000	$56.38	146.6M	1.2M	20h 0m	468	1
108	Inkling	0.0000	$4.24	4.4M	17k	41m	109	1
109	Inkling	0.0000	$8.87	9.2M	43k	1h 49m	170	2
110	Muse Spark 1.2	0.0000†	$1.72	8.1M	39k	1h 17m	88	1

† score adjusted by manual review; the trace's Judge tab carries the measured score and the rationale.