Tasks Traces V1 Blog Changelog Contribute GitHub

Traces

Every run: the agent's full trajectory, its submissions over the 20-hour budget, and the final score.

SPICE Circuit Simulator in Rust

task page →110 runs · indexed 2026-10-08

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Gemini 4 Argon	0.9394	$221.62	329.8M	1.9M	11h 7m	731	3
2	Claude Opus 5.5	0.9293	$97.56	280.4M	1.6M	6h 22m	663	14
3	Claude Sonnet 5.5	0.9293	$109.70	401.0M	2.3M	10h 14m	905	22
4	Claude Sonnet 5.5	0.9293	$116.79	422.3M	2.6M	15h 25m	947	16
5	Claude Sonnet 5.5	0.9192	$157.47	595.8M	3.0M	11h 25m	1266	4
6	Claude Sonnet 5.5	0.9192	$114.22	425.5M	2.3M	14h 29m	922	18
7	Gemini 4 Argon	0.9091	$147.35	234.0M	1.0M	8h 39m	503	6
8	GPT-6 Astra	0.9091	$2887.40	1385.4M	2.5M	19h 50m	3121	180
9	Claude Opus 5.5	0.8990	$72.19	195.2M	1.3M	4h 28m	434	13
10	GPT-6 Astra	0.8990	$347.32	164.1M	405k	2h 56m	414	7
11	Claude Sonnet 5.5	0.8990	$108.09	389.8M	2.4M	13h 20m	873	12
12	GPT-6 Astra	0.8990	$599.18	284.8M	531k	4h 2m	616	7
13	GPT-6 Astra	0.8889	$677.88	324.1M	558k	4h 32m	694	7
14	Claude Fable 5	0.8889	$545.26	410.4M	1.8M	15h 21m	932	13
15	Gemini 4 Argon	0.8889	$270.20	399.0M	2.4M	13h 19m	883	6
16	Gemini 4 Argon	0.8788	$203.70	185.4M	882k	6h 7m	426	3
17	Claude Opus 5.5	0.8687	$65.56	170.6M	1.2M	3h 47m	399	6
18	Claude Opus 5.5	0.8687	$104.48	301.0M	1.7M	5h 14m	731	11
19	GPT-6 Astra	0.8687	$172.72	83.5M	308k	2h 47m	288	4
20	Claude Opus 5.5	0.8687	$69.51	187.1M	1.2M	4h 12m	427	15
21	Claude Fable 5.1	0.8586	$166.41	276.3M	1.2M	9h 57m	632	17
22	Claude Opus 5	0.8485	$441.42	701.9M	2.4M	13h 28m	1524	38
23	Claude Fable 5	0.8384	$451.15	353.8M	1.3M	8h 53m	733	11
24	Gemini 4 Argon	0.8283	$51.57	147.9M	741k	4h 35m	403	5
25	Claude Fable 5	0.8081	$505.54	407.1M	1.3M	17h 12m	857	16
26	Claude Opus 5	0.7879	$564.99	947.9M	2.4M	15h 22m	1980	33
27	Claude Fable 5	0.7778	$491.30	392.4M	1.3M	15h 56m	851	13
28	Claude Fable 5	0.7778	$474.28	378.9M	1.3M	17h 20m	800	12
29	Claude Opus 5	0.7677	$321.00	503.6M	1.9M	12h 46m	1092	31
30	Claude Fable 5.1	0.7677	$93.05	145.4M	837k	3h 54m	417	4
31	Grok 4.7	0.7576	$721.50	746.3M	2.8M	15h 40m	3094	16
32	Claude Opus 5	0.7475	$357.80	576.4M	1.9M	14h 22m	1233	23
33	Claude Opus 5	0.7475	$479.75	767.1M	2.6M	12h 42m	1557	22
34	Grok 4.7	0.7172	$780.09	794.5M	2.5M	20h 0m	3280	18
35	Claude Fable 5.1	0.7071	$96.79	153.4M	827k	4h 2m	413	8
36	Grok 4.7	0.7071	$747.81	772.9M	3.1M	17h 23m	3267	17
37	Claude Fable 5.1	0.6970	$69.08	107.8M	623k	2h 37m	227	2
38	Grok 4.7	0.6970	$807.64	821.4M	3.2M	17h 39m	3364	44
39	Claude Fable 5.1	0.6667	$96.05	148.5M	898k	4h 16m	406	6
40	Gemini 3.7 Flash	0.6263	$99.31	1037.7M	2.0M	6h 16m	2577	4
41	Grok 4.7	0.6061	$843.32	872.9M	3.2M	19h 50m	3636	16
42	Muse Spark 1.3	0.5758	$268.72	1376.2M	2.2M	19h 51m	3328	2
43	Grok 4.6	0.5657	$717.25	714.4M	2.7M	18h 22m	2483	266
44	Muse Spark 1.3	0.5657	$460.65	531.6M	1.1M	17h 48m	1347	4
45	Claude Haiku 5.5	0.5556	$14.47	224.0M	1.1M	2h 40m	557	2
46	Claude Haiku 5.5	0.5556	$12.80	192.7M	1.2M	2h 43m	510	4
47	Muse Spark 1.3	0.5455	$559.10	664.1M	1.1M	19h 47m	1540	5
48	Gemini 3.8 Flash	0.5354	$35.82	384.7M	639k	3h 11m	965	4
49	Claude Haiku 5.5	0.5354	$11.83	178.7M	1.1M	2h 15m	437	2
50	GLM-5.3	0.5354	$171.96	618.6M	1.7M	18h 15m	1441	4
51	Gemini 3.7 Flash	0.5253	$121.42	1230.5M	2.0M	12h 5m	2949	3
52	Grok 4.6	0.5253	$835.65	836.5M	3.1M	20h 0m	3128	205
53	GLM-5.3 Flash	0.5051	$24.75	719.3M	1.5M	17h 38m	1725	1
54	GLM-5.3	0.4949	$235.96	867.9M	1.6M	17h 11m	1905	18
55	Grok 4.6	0.4848	$586.10	582.0M	3.0M	17h 34m	2322	89
56	Gemini 3.7 Flash	0.4747	$76.85	790.2M	1.6M	4h 46m	2017	2
57	GLM-5.3 Flash	0.4646	$19.29	592.9M	1.1M	15h 32m	1400	4
58	Grok 4.6	0.4646	$1191.20	1187.2M	2.5M	19h 58m	3879	96
59	Muse Spark 1.3	0.4646	$547.17	608.4M	1.3M	19h 44m	1331	3
60	Grok 4.6	0.4545	$165.93	154.1M	860k	5h 37m	563	37
61	Gemini 3.7 Flash	0.4444	$80.51	766.1M	1.3M	6h 23m	1793	7
62	GPT-5.6 Sol	0.4444	$366.07	350.2M	455k	6h 25m	797	22
63	GLM-5.3	0.4444	$158.88	578.0M	1.3M	10h 53m	1155	4
64	Qwen3.8-Max-0902	0.4242	$87.16	281.5M	2.5M	19h 59m	601	1
65	Gemini 3.8 Flash	0.4242	$24.11	264.9M	400k	1h 50m	610	2
66	GLM-5.3	0.4141	$190.24	693.2M	1.5M	19h 41m	1705	30
67	GPT-5.6 Sol	0.4141	$307.32	296.1M	386k	4h 30m	783	15
68	Qwen3.8-Max-0902	0.4040	$64.29	188.8M	2.6M	20h 1m	516	1
69	GPT-5.6 Sol	0.3939	$538.95	528.2M	617k	6h 30m	1280	21
70	Muse Spark 1.3	0.3939	$557.80	621.9M	1.5M	19h 58m	1391	1
71	GLM-5.3 Flash	0.3737	$22.66	655.9M	1.7M	20h 0m	1644	1
72	Gemini 3.7 Flash	0.3737	$126.28	1246.3M	2.0M	12h 6m	2921	2
73	Qwen3.8-Max	0.3737	$145.98	524.1M	2.1M	19h 26m	1338	1
74	GLM-5.3 Flash	0.3737	$15.05	404.9M	975k	14h 18m	975	2
75	Qwen3.8-Max-0902	0.3737	$124.78	385.3M	4.3M	20h 1m	989	2
76	Claude Haiku 5.5	0.3636	$10.69	164.8M	919k	2h 31m	405	1
77	Claude Haiku 5.5	0.3636	$14.75	230.3M	1.1M	2h 56m	539	1
78	Qwen3.8-Max	0.3434	$127.04	463.9M	1.6M	19h 13m	987	4
79	GLM-5.3 Flash	0.3434	$27.30	749.1M	1.3M	19h 17m	1599	3
80	GPT-5.6 Sol	0.3333	$187.27	186.7M	262k	2h 15m	515	8
81	Gemini 3.8 Flash	0.3232	$15.32	160.0M	291k	1h 17m	426	3
82	GLM-5.3	0.3131	$152.26	545.3M	1.3M	16h 57m	1164	23
83	Qwen3.8-Max-0902	0.3030	$73.95	234.0M	2.3M	20h 1m	470	1
84	Qwen3.8-Max	0.3030	$96.60	342.7M	1.6M	18h 43m	760	2
85	Qwen3.8-Max	0.3030	$110.14	386.5M	2.0M	19h 56m	947	1
86	Qwen3.8-Max	0.2727	$126.58	455.7M	1.9M	19h 26m	951	7
87	GPT-5.6 Sol	0.2626	$226.84	221.2M	327k	3h 7m	592	13
88	Qwen3.8-Max-0902	0.2020	$65.44	194.8M	2.5M	19h 35m	444	2
89	Muse Spark 1.2	0.1313	$32.05	183.6M	731k	2h 8m	465	1
90	Gemini 3.8 Flash	0.0808	$2.16	15.8M	108k	23m	142	1
91	Muse Spark 1.2	0.0707	$10.81	53.3M	351k	1h 35m	223	2
92	Muse Spark 1.2	0.0707	$32.82	175.7M	815k	3h 21m	461	1
93	Muse Spark 1.2	0.0505	$8.77	48.8M	242k	1h 16m	196	2
94	Muse Spark 1.2	0.0404	$5.34	27.9M	188k	26m	143	2
95	Gemini 3.8 Flash	0.0101†	$2.26	17.9M	93k	20m	140	2
96	DeepSeek V4 Flash Vision Exp	0.0000	$11.65	704.6M	1.2M	10h 40m	1549	15
97	Kimi K3	0.0000	$179.57	474.0M	1.9M	19h 42m	1215	9
98	Kimi K3	0.0000	$215.15	588.2M	1.9M	19h 56m	1405	3
99	DeepSeek V4 Flash Vision Exp	0.0000	$19.65	1192.1M	2.0M	18h 3m	2398	10
100	Kimi K3	0.0000	$199.08	565.4M	1.4M	19h 58m	1271	1
101	Kimi K3	0.0000	$204.94	570.3M	1.7M	19h 34m	1399	3
102	Kimi K3	0.0000	$193.03	529.1M	1.8M	19h 58m	1263	1
103	DeepSeek V4 Flash Vision Exp	0.0000	$20.48	1199.6M	2.4M	11h 32m	2563	9
104	Inkling	0.0000	$4.42	4.6M	23k	45m	83	1
105	DeepSeek V4 Flash Vision Exp	0.0000	$21.51	1287.0M	2.2M	19h 10m	2904	13
106	Inkling	0.0000	$19.30	20.1M	40k	1h 20m	170	2
107	Inkling	0.0000	$1.04	1.1M	10k	27m	52	2
108	DeepSeek V4 Flash Vision Exp	0.0000	$19.68	1182.5M	2.0M	19h 8m	2617	23
109	Inkling	0.0000	$5.07	5.3M	18k	1h 13m	110	2
110	Inkling	0.0000	$5.21	5.3M	32k	15m	124	2

† score adjusted by manual review; the trace's Judge tab carries the measured score and the rationale.