Traces

Every run: the agent's full trajectory, its submissions over the 20-hour budget, and the final score.

Verilog Simulator in Swift

task page →50 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Fable 5.1	0.9332	$262.30	478.6M	947k	9h 23m	1148	17
2	Claude Fable 5.1	0.9315	$216.99	391.4M	878k	7h 55m	966	6
3	Claude Fable 5.1	0.9195	$211.66	382.5M	840k	9h 40m	989	17
4	Claude Fable 5.1	0.9161	$266.36	483.7M	1.0M	10h 14m	1167	34
5	Claude Fable 5.1	0.9092	$168.96	308.4M	606k	6h 47m	586	10
6	GPT-5.6	0.8853	$427.16	420.2M	394k	14h 37m	921	9
7	GPT-5.6	0.8596	$271.89	260.9M	315k	4h 56m	650	5
8	GPT-5.6	0.8493	$335.11	326.9M	349k	4h 23m	746	9
9	Kimi K3	0.8373	$248.18	704.4M	1.6M	20h 0m	1636	11
10	GPT-5.6	0.8339	$184.09	180.1M	273k	4h 24m	480	6
11	Gemini 3.7 Flash	0.8305	$103.65	1018.6M	2.0M	19h 58m	2418	1
12	Gemini 3.7 Flash	0.8288	$74.39	745.3M	2.1M	17h 44m	1828	1
13	Grok 4.6	0.8253	$223.65	211.2M	1.9M	12h 53m	884	31
14	GLM-5.3	0.8236	$236.70	879.4M	1.3M	20h 1m	1924	11
15	GPT-5.6	0.8236	$181.11	176.3M	277k	4h 52m	476	3
16	Gemini 3.7 Flash	0.8202	$75.29	725.4M	1.4M	19h 4m	1651	2
17	GLM-5.3	0.8151	$195.25	722.0M	1.1M	19h 28m	1706	10
18	Grok 4.6	0.8116	$347.28	325.2M	2.2M	19h 55m	1222	56
19	GLM-5.3	0.7757	$229.93	817.6M	1.1M	19h 47m	1800	19
20	GLM-5.3	0.7723	$247.89	875.5M	1.2M	19h 55m	1804	17
21	Kimi K3	0.7551	$141.93	387.6M	1.1M	20h 1m	1009	1
22	Qwen3.8-Max	0.7551	$162.17	597.6M	1.8M	19h 37m	1491	1
23	GLM-5.3	0.7551	$285.37	1061.6M	1.4M	19h 46m	2367	23
24	Grok 4.6	0.7517	$518.10	508.9M	2.5M	19h 53m	1842	50
25	Kimi K3	0.7175	$256.44	747.8M	1.5M	20h 1m	1669	2
26	Grok 4.6	0.6901	$168.65	160.9M	1.6M	9h 35m	691	25
27	Kimi K3	0.6849	$265.41	774.0M	1.6M	19h 54m	1682	2
28	Kimi K3	0.6250	$235.66	685.5M	1.4M	19h 49m	1521	1
29	Gemini 3.7 Flash	0.5976	$41.43	380.1M	776k	20h 1m	944	4
30	DeepSeek V4 Flash Vision Exp	0.5582	$23.11	1453.0M	1.8M	19h 40m	3118	3
31	Grok 4.6	0.5188	$107.03	99.0M	943k	6h 25m	402	19
32	Qwen3.8-Max	0.4384	$128.73	471.1M	1.6M	19h 41m	965	2
33	DeepSeek V4 Flash Vision Exp	0.4212	$16.08	998.7M	1.4M	20h 0m	2128	4
34	DeepSeek V4 Flash Vision Exp	0.3784	$22.06	1380.9M	1.8M	19h 50m	2999	12
35	Gemini 3.7 Flash	0.3545	$53.22	180.3M	689k	19h 59m	574	1
36	Qwen3.8-Max	0.3527	$147.05	546.0M	1.5M	19h 49m	1109	2
37	DeepSeek V4 Flash Vision Exp	0.3253	$27.96	1735.1M	2.4M	17h 46m	3798	22
38	DeepSeek V4 Flash Vision Exp	0.3065	$23.64	1484.7M	1.9M	18h 25m	3157	30
39	Qwen3.8-Max	0.2894	$161.74	608.8M	1.4M	19h 56m	1406	2
40	Qwen3.8-Max	0.2414	$54.19	188.0M	1.1M	20h 0m	597	1
41	Muse Spark 1.2	0.0445	$10.85	58.8M	321k	1h 31m	229	2
42	Muse Spark 1.2	0.0325	$16.81	84.3M	312k	1h 17m	329	2
43	Muse Spark 1.2	0.0120	$23.14	130.5M	679k	1h 19m	395	3
44	Muse Spark 1.2	0.0051	$4.27	21.5M	175k	51m	138	1
45	Inkling	0.0017	$19.93	20.8M	47k	1h 36m	283	1
46	Inkling	0.0000	$3.27	3.4M	20k	1h 3m	98	2
47	Inkling	0.0000	$13.36	13.9M	43k	1h 28m	227	1
48	Muse Spark 1.2	0.0000	$10.69	54.0M	468k	1h 50m	208	1
49	Inkling	0.0000	$2.17	2.2M	13k	1h 11m	80	1
50	Inkling	0.0000	$9.64	10.0M	34k	1h 28m	169	2