Traces

Every run: the agent's full trajectory, its submissions over the 20-hour budget, and the final score.

Cranelift Codegen Optimization

task page →50 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Fable 5.1	0.3484	$166.03	303.2M	604k	16h 29m	825	13
2	Claude Fable 5.1	0.3356	$233.87	425.4M	876k	16h 37m	1021	18
3	Claude Fable 5.1	0.2852	$199.91	365.0M	723k	15h 17m	974	15
4	Claude Fable 5.1	0.2009	$170.71	311.4M	625k	17h 11m	813	17
5	Gemini 3.7 Flash	0.0802	$35.86	231.6M	242k	6h 15m	574	20
6	Kimi K3	0.0603	$148.59	441.7M	786k	16h 22m	1028	15
7	Kimi K3	0.0478	$126.89	362.2M	801k	17h 53m	936	9
8	DeepSeek V4 Flash Vision Exp	0.0237	$8.27	504.6M	723k	9h 18m	1260	11
9	Gemini 3.7 Flash	0.0189	$20.57	157.5M	204k	2h 49m	432	17
10	GPT-5.6	0.0176	$225.59	218.6M	216k	6h 56m	506	6
11	GLM-5.3	0.0162	$93.99	346.3M	474k	11h 28m	766	2
12	Kimi K3	0.0132	$145.36	415.9M	1.0M	20h 3m	1079	7
13	GLM-5.3	0.0129	$109.37	311.0M	484k	17h 22m	770	9
14	Qwen3.8-Max	0.0117	$41.08	108.0M	2.2M	18h 38m	484	6
15	Gemini 3.7 Flash	0.0110	$23.00	164.8M	188k	4h 11m	495	19
16	DeepSeek V4 Flash Vision Exp	0.0107	$5.01	303.8M	439k	7h 50m	723	8
17	GLM-5.3	0.0102	$60.68	223.7M	392k	7h 14m	615	4
18	DeepSeek V4 Flash Vision Exp	0.0030	$5.08	311.7M	419k	8h 3m	781	6
19	Grok 4.6	0.0000	$113.60	103.4M	918k	9h 35m	426	6
20	Inkling	0.0000	$8.38	8.8M	12k	55m	94	1
21	GPT-5.6	0.0000	$297.21	276.6M	238k	7h 17m	549	5
22	Muse Spark 1.2	0.0000	$33.79	127.0M	205k	10h 23m	356	5
23	Muse Spark 1.2	0.0000	$18.72	115.3M	194k	3h 23m	350	4
24	Qwen3.8-Max	0.0000	$34.88	81.5M	2.3M	20h 0m	413	1
25	Kimi K3	0.0000	$136.80	389.4M	1.1M	20h 0m	1006	19
26	Gemini 3.7 Flash	0.0000	$30.61	210.1M	211k	5h 15m	593	21
27	GPT-5.6	0.0000	$234.61	222.5M	240k	7h 38m	461	5
28	Grok 4.6	0.0000	$240.25	234.7M	1.3M	16h 47m	895	24
29	Inkling	0.0000	$13.75	14.4M	22k	1h 40m	135	2
30	Kimi K3	0.0000	$136.72	403.0M	752k	19h 8m	1147	11
31	Inkling	0.0000	$11.79	12.4M	15k	1h 2m	121	1
32	Muse Spark 1.2	0.0000	$87.58	303.3M	711k	10h 13m	562	5
33	GPT-5.6	0.0000	$212.19	205.4M	208k	5h 52m	461	5
34	Inkling	0.0000	$8.24	8.6M	21k	30m	104	2
35	DeepSeek V4 Flash Vision Exp	0.0000	$3.68	220.8M	344k	8h 44m	670	8
36	Inkling	0.0000	$13.82	14.5M	21k	59m	137	2
37	Grok 4.6	0.0000	$138.74	131.9M	1.2M	8h 47m	587	6
38	Claude Fable 5.1	0.0000	$232.11	427.5M	765k	16h 0m	1023	35
39	Grok 4.6	0.0000	$138.58	127.2M	927k	7h 29m	488	9
40	Muse Spark 1.2	0.0000	$76.77	223.4M	655k	10h 10m	450	7
41	Muse Spark 1.2	0.0000	$49.18	242.6M	532k	8h 28m	495	9
42	Qwen3.8-Max	0.0000	$43.78	119.5M	2.2M	18h 46m	496	2
43	Qwen3.8-Max	0.0000	$50.74	159.1M	1.7M	16h 16m	624	4
44	GLM-5.3	0.0000	$110.08	353.7M	515k	18h 43m	761	18
45	Gemini 3.7 Flash	0.0000	$47.46	341.2M	378k	8h 44m	828	22
46	Grok 4.6	0.0000	$108.30	103.9M	740k	6h 37m	447	6
47	DeepSeek V4 Flash Vision Exp	0.0000	$4.87	296.6M	422k	6h 43m	730	3
48	GLM-5.3	0.0000	$69.79	258.7M	392k	7h 49m	703	7
49	GPT-5.6	0.0000	$355.12	344.3M	254k	10h 32m	627	3
50	Qwen3.8-Max	0.0000	$42.54	115.4M	2.2M	20h 0m	491	1