Tasks Traces V1 Blog Changelog Contribute GitHub

Traces

Every run: the agent's full trajectory, its submissions over the 20-hour budget, and the final score.

Lua Native Compiler

task page →110 runs · indexed 2026-10-08

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Gemini 4 Argon	1.0000	$73.29	90.2M	317k	3h 26m	212	2
2	Claude Sonnet 5.5	1.0000	$63.17	240.2M	1.2M	8h 33m	574	4
3	Claude Fable 5.1	1.0000	$44.19	69.8M	381k	3h 24m	215	8
4	Claude Fable 5.1	1.0000	$51.03	82.5M	402k	3h 56m	245	12
5	Claude Sonnet 5.5	1.0000	$62.32	242.2M	1.1M	9h 42m	620	10
6	GPT-6 Astra	1.0000	$46.19	26.4M	213k	1h 23m	134	7
7	Claude Sonnet 5.5	1.0000	$65.89	255.0M	1.2M	8h 21m	630	3
8	Claude Fable 5	1.0000	$196.16	153.6M	590k	6h 11m	351	6
9	Claude Haiku 5.5	1.0000	$21.88	349.5M	1.7M	12h 38m	805	16
10	Claude Opus 5	1.0000	$209.27	343.2M	992k	13h 38m	826	10
11	Claude Opus 5	1.0000	$125.36	207.5M	580k	6h 45m	426	9
12	Claude Fable 5.1	1.0000	$55.84	88.8M	470k	3h 14m	235	6
13	Claude Haiku 5.5	1.0000	$14.18	224.8M	1.1M	5h 49m	554	3
14	Claude Opus 5	1.0000	$155.19	264.1M	628k	9h 49m	494	11
15	Claude Sonnet 5.5	1.0000	$84.18	344.6M	1.2M	16h 46m	746	14
16	GPT-6 Astra	1.0000	$35.97	23.6M	186k	1h 21m	133	4
17	GPT-6 Astra	1.0000	$131.76	64.1M	279k	2h 36m	232	7
18	Claude Opus 5	0.9992	$107.50	172.4M	579k	5h 52m	375	4
19	Claude Opus 5	0.9992	$128.06	213.3M	585k	7h 58m	454	2
20	Gemini 4 Argon	0.9976	$24.66	60.9M	275k	2h 36m	137	2
21	Gemini 4 Argon	0.9976	$34.72	50.2M	259k	2h 22m	136	2
22	Gemini 4 Argon	0.9976	$36.93	59.8M	371k	3h 58m	154	2
23	GLM-5.3	0.9976	$201.34	735.2M	1.4M	11h 37m	1660	3
24	GPT-6 Astra	0.9976	$79.59	42.6M	240k	2h 8m	195	9
25	Claude Haiku 5.5	0.9976	$23.67	398.7M	1.3M	8h 23m	819	7
26	Gemini 4 Argon	0.9976	$21.71	52.8M	265k	2h 10m	142	2
27	Muse Spark 1.3	0.9976	$451.94	515.6M	941k	16h 46m	1261	2
28	GPT-6 Astra	0.9976	$93.69	48.6M	237k	1h 56m	206	6
29	GLM-5.3	0.9953	$142.13	511.6M	1.2M	9h 22m	1178	2
30	DeepSeek V4 Flash Vision Exp	0.9788	$23.06	1383.4M	2.4M	13h 19m	2911	21
31	GLM-5.3	0.9599	$274.76	1000.1M	1.8M	19h 32m	2273	5
32	Kimi K3	0.9575	$183.51	524.3M	1.3M	19h 59m	1203	20
33	GLM-5.3	0.9527	$236.30	883.7M	1.1M	17h 20m	1835	7
34	Claude Haiku 5.5	0.9292	$10.74	177.0M	661k	2h 38m	328	1
35	Kimi K3	0.8750	$185.22	530.7M	1.3M	20h 0m	1207	1
36	Claude Fable 5.1	0.8463	$43.78	69.0M	381k	3h 39m	231	1
37	Claude Opus 5.5	0.7885	$68.38	213.8M	1.0M	9h 38m	482	11
38	Gemini 3.8 Flash	0.7382	$43.93	463.1M	839k	6h 53m	1136	1
39	Gemini 3.7 Flash	0.6998	$21.69	183.4M	580k	2h 58m	400	1
40	Claude Fable 5	0.6667	$216.90	168.1M	672k	7h 17m	337	5
41	Claude Fable 5	0.6667	$173.23	133.0M	557k	7h 50m	292	8
42	Claude Fable 5.1	0.6667	$57.38	92.1M	466k	4h 40m	248	7
43	Claude Haiku 5.5	0.6667	$17.99	286.4M	1.3M	10h 1m	661	4
44	Claude Opus 5.5	0.6667	$78.97	234.5M	1.3M	9h 24m	649	15
45	Gemini 3.7 Flash	0.6541	$62.39	519.3M	1.2M	7h 8m	1260	1
46	Gemini 3.7 Flash	0.6517	$26.08	255.3M	824k	1h 58m	670	1
47	Claude Opus 5.5	0.6447	$81.59	228.2M	1.4M	10h 19m	607	8
48	Gemini 3.7 Flash	0.6392	$29.13	278.2M	822k	3h 5m	751	1
49	Grok 4.6	0.6384	$176.45	167.1M	1.5M	7h 26m	653	8
50	Kimi K3	0.5063	$220.93	645.6M	1.2M	20h 2m	1461	1
51	Muse Spark 1.3	0.4969	$541.57	625.6M	1.2M	20h 0m	1447	1
52	Muse Spark 1.3	0.4002	$540.38	602.2M	792k	20h 0m	1400	1
53	GLM-5.3 Flash	0.3625	$33.26	1014.3M	1.2M	20h 1m	2002	1
54	Gemini 3.7 Flash	0.3270	$79.34	743.6M	1.7M	8h 42m	1815	1
55	Kimi K3	0.2862	$159.88	431.8M	1.4M	19h 11m	1009	1
56	Claude Fable 5	0.2123	$203.21	167.2M	496k	10h 45m	402	11
57	DeepSeek V4 Flash Vision Exp	0.2005	$21.27	1250.2M	2.4M	15h 16m	2676	17
58	GLM-5.3 Flash	0.1899	$23.36	672.0M	1.1M	19h 59m	1530	3
59	GLM-5.3 Flash	0.0339	$26.65	828.1M	1.4M	19h 57m	1984	2
60	Inkling	0.0000	$27.74	28.9M	72k	42m	300	1
61	Grok 4.7	0.0000	$74.24	76.4M	456k	2h 25m	373	1
62	Inkling	0.0000	$16.92	17.5M	67k	44m	221	1
63	Claude Sonnet 5.5	0.0000	$70.29	272.9M	1.3M	13h 50m	649	5
64	Grok 4.7	0.0000	$23.46	23.7M	161k	55m	121	1
65	Qwen3.8-Max	0.0000	$64.51	206.8M	2.0M	19h 8m	639	2
66	GLM-5.3 Flash	0.0000	$24.51	738.8M	1.3M	18h 5m	1545	29
67	DeepSeek V4 Flash Vision Exp	0.0000	$11.39	676.4M	1.3M	7h 45m	1507	15
68	Claude Fable 5	0.0000	$255.17	207.7M	659k	6h 43m	426	8
69	Grok 4.6	0.0000	$110.36	107.0M	986k	5h 11m	447	8
70	Muse Spark 1.3	0.0000†	$75.32	109.8M	406k	5h 42m	461	6
71	Inkling	0.0000	$48.62	50.6M	132k	1h 1m	367	2
72	GPT-5.6 Sol	0.0000	$35.07	44.0M	172k	2h 41m	211	10
73	Muse Spark 1.2	0.0000	$44.95	118.1M	616k	6h 24m	354	2
74	Qwen3.8-Max-0902	0.0000	$32.39	39.9M	3.6M	20h 0m	206	1
75	Kimi K3	0.0000	$228.55	670.7M	1.4M	19h 8m	1408	3
76	Qwen3.8-Max-0902	0.0000	$40.91	73.7M	3.6M	19h 57m	272	1
77	Inkling	0.0000	$14.81	15.3M	75k	53m	223	1
78	Claude Opus 5.5	0.0000	$124.07	397.7M	1.8M	12h 35m	893	28
79	GPT-5.6 Sol	0.0000†	$7.45	8.9M	65k	25m	65	3
80	GLM-5.3	0.0000	$251.10	909.6M	1.3M	19h 55m	1918	6
81	Gemini 3.8 Flash	0.0000†	$7.66	74.9M	188k	44m	304	2
82	Qwen3.8-Max	0.0000	$121.10	426.7M	2.2M	19h 59m	984	1
83	Claude Opus 5.5	0.0000	$93.26	296.3M	1.3M	8h 6m	673	12
84	Grok 4.6	0.0000	$271.55	264.4M	2.0M	9h 49m	1061	31
85	Muse Spark 1.2	0.0000	$10.77	39.0M	388k	1h 41m	164	4
86	DeepSeek V4 Flash Vision Exp	0.0000	$22.43	1385.4M	2.0M	11h 31m	2860	2
87	GLM-5.3 Flash	0.0000	$31.03	962.8M	1.5M	19h 47m	2160	10
88	Gemini 3.8 Flash	0.0000†	$9.10	88.8M	213k	1h 38m	328	2
89	Qwen3.8-Max	0.0000	$135.25	480.8M	2.3M	19h 7m	1081	2
90	Qwen3.8-Max	0.0000	$60.45	189.1M	2.1M	19h 48m	599	3
91	Grok 4.7	0.0000	$84.78	85.6M	413k	2h 11m	330	3
92	Qwen3.8-Max	0.0000	$31.00	87.6M	1.4M	13h 21m	385	6
93	Qwen3.8-Max-0902	0.0000	$31.88	33.4M	3.8M	20h 13m	209	1
94	Inkling	0.0000	$7.91	8.2M	23k	26m	165	2
95	Grok 4.6	0.0000	$676.54	663.0M	4.1M	19h 59m	2429	277
96	Gemini 3.8 Flash	0.0000†	$23.38	246.7M	422k	1h 50m	682	1
97	GPT-5.6 Sol	0.0000	$127.72	124.4M	217k	2h 5m	382	24
98	Muse Spark 1.2	0.0000	$4.46	22.6M	149k	52m	145	2
99	Muse Spark 1.3	0.0000	$637.70	724.8M	869k	20h 0m	1631	1
100	DeepSeek V4 Flash Vision Exp	0.0000	$18.34	1059.5M	2.2M	17h 1m	2383	9
101	Qwen3.8-Max-0902	0.0000	$34.76	50.3M	3.5M	20h 2m	251	1
102	GPT-5.6 Sol	0.0000	$24.84	37.1M	163k	1h 9m	219	13
103	Grok 4.6	0.0000	$250.76	240.6M	2.1M	11h 13m	930	16
104	Grok 4.7	0.0000	$10.18	12.5M	117k	39m	81	1
105	Muse Spark 1.2	0.0000	$22.61	124.4M	368k	1h 49m	323	2
106	Muse Spark 1.2	0.0000	$12.00	53.2M	328k	1h 52m	189	2
107	Grok 4.7	0.0000	$15.52	17.3M	144k	46m	103	1
108	Gemini 3.8 Flash	0.0000†	$32.38	312.3M	482k	4h 4m	832	1
109	Qwen3.8-Max-0902	0.0000	$91.51	260.0M	4.2M	19h 42m	578	1
110	GPT-5.6 Sol	0.0000	$20.50	39.9M	168k	1h 9m	231	7

† score adjusted by manual review; the trace's Judge tab carries the measured score and the rationale.