Tasks Traces V1 Blog Changelog Contribute GitHub

Traces

Every run: the agent's full trajectory, its submissions over the 20-hour budget, and the final score.

Verilog Simulator in Swift

task page →110 runs · indexed 2026-10-08

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Opus 5.5	0.9521	$121.24	461.6M	1.1M	7h 20m	989	14
2	GPT-6 Astra	0.9503	$104.53	49.0M	213k	1h 48m	187	5
3	Claude Opus 5.5	0.9486	$185.48	723.5M	1.5M	11h 40m	1695	26
4	GPT-6 Astra	0.9469	$329.41	155.6M	366k	2h 26m	409	26
5	GPT-6 Astra	0.9452	$47.73	26.0M	162k	1h 5m	130	3
6	Claude Opus 5.5	0.9452	$119.27	459.4M	1.0M	7h 41m	1025	14
7	Claude Opus 5.5	0.9435	$112.90	428.3M	1.0M	8h 8m	989	11
8	Claude Opus 5.5	0.9401	$117.75	445.5M	1.1M	6h 29m	1024	13
9	Claude Sonnet 5.5	0.9384	$170.55	724.9M	2.0M	18h 31m	1739	12
10	Gemini 4 Argon	0.9349	$95.93	83.0M	397k	2h 56m	199	2
11	GPT-6 Astra	0.9349	$94.85	44.6M	216k	1h 33m	174	6
12	Claude Sonnet 5.5	0.9332	$135.20	567.6M	1.7M	12h 14m	1244	6
13	Claude Fable 5.1	0.9332	$262.30	478.6M	947k	9h 23m	1148	17
14	Claude Fable 5.1	0.9315	$216.99	391.4M	878k	7h 55m	966	6
15	Claude Sonnet 5.5	0.9315	$217.23	922.9M	2.5M	15h 59m	2176	16
16	Claude Fable 5	0.9298	$976.76	847.7M	1.7M	19h 7m	2003	24
17	Gemini 4 Argon	0.9281	$95.89	72.5M	365k	2h 52m	178	3
18	Gemini 4 Argon	0.9281	$83.88	72.7M	366k	2h 46m	166	3
19	Claude Sonnet 5.5	0.9281	$215.60	910.8M	2.6M	18h 12m	2122	20
20	Claude Fable 5	0.9264	$317.64	267.5M	691k	8h 11m	499	15
21	GPT-6 Astra	0.9264	$71.95	36.3M	192k	1h 35m	162	6
22	Claude Sonnet 5.5	0.9264	$186.02	788.1M	2.2M	13h 56m	1692	13
23	Claude Opus 5	0.9212	$529.35	935.4M	1.6M	17h 43m	2148	24
24	Gemini 4 Argon	0.9212	$27.80	68.8M	403k	2h 11m	165	4
25	Claude Fable 5.1	0.9195	$211.66	382.5M	840k	9h 40m	989	17
26	Claude Fable 5	0.9178	$1055.99	927.6M	1.7M	18h 46m	2232	22
27	Claude Opus 5	0.9178	$356.58	629.0M	1.1M	18h 31m	1391	16
28	Gemini 4 Argon	0.9178	$56.50	79.3M	431k	2h 53m	191	2
29	Claude Fable 5.1	0.9161	$266.36	483.7M	1.0M	10h 14m	1167	34
30	Claude Fable 5	0.9144	$317.29	267.0M	700k	14h 26m	483	17
31	Claude Opus 5	0.9127	$588.34	1052.2M	1.6M	19h 25m	2364	25
32	Claude Opus 5	0.9110	$505.22	890.6M	1.6M	16h 8m	2006	21
33	Claude Fable 5.1	0.9092	$168.96	308.4M	606k	6h 47m	586	10
34	Claude Fable 5	0.8973	$914.45	785.4M	1.7M	18h 23m	1843	35
35	Claude Haiku 5.5	0.8938	$55.29	968.3M	2.6M	12h 1m	2056	13
36	Claude Opus 5	0.8921	$377.04	660.7M	1.2M	15h 32m	1403	21
37	Grok 4.7	0.8853	$200.79	205.0M	994k	5h 44m	853	1
38	GPT-5.6 Sol	0.8853	$427.16	420.2M	394k	14h 37m	921	9
39	Claude Haiku 5.5	0.8818	$45.82	813.0M	1.9M	7h 26m	1721	4
40	Claude Haiku 5.5	0.8784	$81.94	1434.4M	3.8M	19h 26m	3162	45
41	Grok 4.7	0.8784	$244.06	246.8M	1.1M	15h 21m	1005	27
42	Claude Haiku 5.5	0.8767	$71.12	1257.3M	3.0M	15h 14m	2739	5
43	Claude Haiku 5.5	0.8733	$40.39	708.0M	1.8M	7h 30m	1550	2
44	GPT-5.6 Sol	0.8596	$271.89	260.9M	315k	4h 56m	650	5
45	Grok 4.7	0.8596	$839.06	861.8M	2.3M	19h 58m	3507	37
46	Grok 4.7	0.8510	$180.74	176.1M	878k	5h 58m	658	4
47	GPT-5.6 Sol	0.8493	$335.11	326.9M	349k	4h 23m	746	9
48	Grok 4.7	0.8476	$334.11	341.6M	1.2M	11h 19m	1457	47
49	Kimi K3	0.8373	$248.18	704.4M	1.6M	20h 0m	1636	11
50	GPT-5.6 Sol	0.8339	$184.09	180.1M	273k	4h 24m	480	6
51	Gemini 3.7 Flash	0.8305	$103.65	1018.6M	2.0M	19h 58m	2418	1
52	Gemini 3.8 Flash	0.8305	$76.00	821.2M	1.2M	6h 11m	1908	22
53	Gemini 3.8 Flash	0.8288	$123.53	1333.0M	1.9M	13h 23m	3053	62
54	Gemini 3.7 Flash	0.8288	$74.39	745.3M	2.1M	17h 44m	1828	1
55	Gemini 3.8 Flash	0.8253	$99.98	1070.4M	1.6M	7h 37m	2417	47
56	Grok 4.6	0.8253	$223.65	211.2M	1.9M	12h 53m	884	31
57	GLM-5.3	0.8236	$236.70	879.4M	1.3M	20h 1m	1924	11
58	GPT-5.6 Sol	0.8236	$181.11	176.3M	277k	4h 52m	476	3
59	Qwen3.8-Max-0902	0.8219	$182.82	643.3M	3.3M	19h 35m	1529	3
60	Gemini 3.7 Flash	0.8202	$75.29	725.4M	1.4M	19h 4m	1651	2
61	Gemini 3.8 Flash	0.8151	$108.32	1181.2M	1.8M	8h 5m	2671	41
62	GLM-5.3	0.8151	$195.25	722.0M	1.1M	19h 28m	1706	10
63	Grok 4.6	0.8116	$347.28	325.2M	2.2M	19h 55m	1222	56
64	Qwen3.8-Max-0902	0.8099	$199.79	724.6M	2.6M	19h 4m	1830	12
65	Gemini 3.8 Flash	0.8099	$133.85	1223.1M	1.9M	19h 59m	2834	11
66	GLM-5.3	0.7757	$229.93	817.6M	1.1M	19h 47m	1800	19
67	GLM-5.3	0.7723	$247.89	875.5M	1.2M	19h 55m	1804	17
68	Kimi K3	0.7551	$141.93	387.6M	1.1M	20h 1m	1009	1
69	Qwen3.8-Max	0.7551	$162.17	597.6M	1.8M	19h 37m	1491	1
70	GLM-5.3	0.7551	$285.37	1061.6M	1.4M	19h 46m	2367	23
71	Grok 4.6	0.7517	$518.10	508.9M	2.5M	19h 53m	1842	50
72	Kimi K3	0.7175	$256.44	747.8M	1.5M	20h 1m	1669	2
73	Grok 4.6	0.6901	$168.65	160.9M	1.6M	9h 35m	691	25
74	Kimi K3	0.6849	$265.41	774.0M	1.6M	19h 54m	1682	2
75	Qwen3.8-Max-0902	0.6473	$162.62	575.2M	2.7M	20h 1m	1239	2
76	Kimi K3	0.6250	$235.66	685.5M	1.4M	19h 49m	1521	1
77	GLM-5.3 Flash	0.6216	$20.18	641.6M	943k	19h 41m	1545	5
78	GLM-5.3 Flash	0.6062	$25.33	768.4M	1.0M	18h 54m	1712	18
79	Gemini 3.7 Flash	0.5976	$41.43	380.1M	776k	20h 1m	944	4
80	Muse Spark 1.3	0.5582	$533.21	614.5M	1.1M	19h 59m	1577	1
81	DeepSeek V4 Flash Vision Exp	0.5582	$23.11	1453.0M	1.8M	19h 40m	3118	3
82	Muse Spark 1.3	0.5497	$512.38	619.2M	1.1M	19h 59m	1598	1
83	Qwen3.8-Max-0902	0.5325	$172.11	623.9M	2.3M	19h 44m	1556	15
84	Grok 4.6	0.5188	$107.03	99.0M	943k	6h 25m	402	19
85	Qwen3.8-Max-0902	0.4966	$53.74	187.4M	939k	20h 0m	551	1
86	Muse Spark 1.3	0.4932	$275.58	1496.0M	1.9M	19h 49m	3361	3
87	Muse Spark 1.3	0.4932	$515.26	603.1M	1.3M	19h 58m	1473	2
88	GLM-5.3 Flash	0.4777	$15.24	482.1M	673k	19h 46m	899	5
89	GLM-5.3 Flash	0.4521	$17.29	524.8M	813k	19h 25m	1148	5
90	Muse Spark 1.3	0.4384	$522.03	583.4M	1.2M	19h 58m	1409	1
91	Qwen3.8-Max	0.4384	$128.73	471.1M	1.6M	19h 41m	965	2
92	DeepSeek V4 Flash Vision Exp	0.4212	$16.08	998.7M	1.4M	20h 0m	2128	4
93	DeepSeek V4 Flash Vision Exp	0.3784	$22.06	1380.9M	1.8M	19h 50m	2999	12
94	Gemini 3.7 Flash	0.3545	$53.22	180.3M	689k	19h 59m	574	1
95	Qwen3.8-Max	0.3527	$147.05	546.0M	1.5M	19h 49m	1109	2
96	DeepSeek V4 Flash Vision Exp	0.3253	$27.96	1735.1M	2.4M	17h 46m	3798	22
97	DeepSeek V4 Flash Vision Exp	0.3065	$23.64	1484.7M	1.9M	18h 25m	3157	30
98	Qwen3.8-Max	0.2894	$161.74	608.8M	1.4M	19h 56m	1406	2
99	GLM-5.3 Flash	0.2860	$25.19	814.7M	1.1M	18h 25m	1907	9
100	Qwen3.8-Max	0.2414	$54.19	188.0M	1.1M	20h 0m	597	1
101	Muse Spark 1.2	0.0445	$10.85	58.8M	321k	1h 31m	229	2
102	Muse Spark 1.2	0.0325	$16.81	84.3M	312k	1h 17m	329	2
103	Muse Spark 1.2	0.0120	$23.14	130.5M	679k	1h 19m	395	3
104	Muse Spark 1.2	0.0051	$4.27	21.5M	175k	51m	138	1
105	Inkling	0.0017	$19.93	20.8M	47k	1h 36m	283	1
106	Inkling	0.0000	$3.27	3.4M	20k	1h 3m	98	2
107	Inkling	0.0000	$13.36	13.9M	43k	1h 28m	227	1
108	Muse Spark 1.2	0.0000	$10.69	54.0M	468k	1h 50m	208	1
109	Inkling	0.0000	$2.17	2.2M	13k	1h 11m	80	1
110	Inkling	0.0000	$9.64	10.0M	34k	1h 28m	169	2