Traces

Every run: the agent's full trajectory, its submissions over the 20-hour budget, and the final score.

Astronomy Toolkit

task page →50 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Fable 5.1	0.8953	$64.06	109.1M	391k	4h 49m	305	17
2	Claude Fable 5.1	0.8935	$57.22	97.5M	350k	6h 25m	298	24
3	Claude Fable 5.1	0.8000	$36.76	59.7M	285k	3h 53m	220	11
4	Claude Fable 5.1	0.7999	$32.01	51.0M	267k	3h 57m	202	12
5	Claude Fable 5.1	0.7955	$42.10	68.8M	317k	4h 11m	232	12
6	Gemini 3.7 Flash	0.7294	$8.39	49.1M	408k	2h 12m	218	1
7	GLM-5.3	0.6985	$72.52	246.9M	707k	19h 16m	599	6
8	Kimi K3	0.6250	$60.98	152.4M	478k	16h 31m	414	12
9	Grok 4.6	0.5299	$15.57	15.6M	323k	1h 52m	106	4
10	GPT-5.6	0.4849	$43.38	52.6M	186k	3h 16m	263	5
11	GLM-5.3	0.4028	$57.26	195.8M	552k	12h 7m	426	8
12	GLM-5.3	0.4017	$72.89	226.5M	514k	17h 29m	476	6
13	Gemini 3.7 Flash	0.4014	$3.09	23.0M	225k	38m	153	3
14	Grok 4.6	0.3993	$48.11	46.2M	395k	3h 31m	208	6
15	Kimi K3	0.3983	$90.41	238.0M	505k	20h 1m	587	20
16	Kimi K3	0.3959	$20.08	46.3M	279k	6h 2m	205	5
17	GLM-5.3	0.3930	$77.97	220.3M	559k	15h 22m	481	2
18	Kimi K3	0.3804	$70.49	167.8M	452k	19h 35m	468	8
19	Kimi K3	0.2995	$53.30	137.6M	418k	11h 54m	430	7
20	GLM-5.3	0.2893	$64.65	196.5M	549k	19h 1m	419	8
21	Grok 4.6	0.2000	$34.68	34.5M	354k	2h 42m	175	8
22	Grok 4.6	0.1971	$39.66	35.3M	400k	3h 43m	165	11
23	GPT-5.6	0.1823	$39.28	48.9M	195k	2h 7m	255	4
24	Muse Spark 1.2	0.1013	$11.07	59.7M	393k	2h 7m	259	1
25	Muse Spark 1.2	0.1000	$12.16	66.0M	402k	2h 34m	231	4
26	Muse Spark 1.2	0.0563	$26.33	90.8M	427k	5h 8m	297	2
27	Qwen3.8-Max	0.0000	$42.82	145.6M	925k	17h 28m	524	2
28	Inkling	0.0000	$4.59	4.7M	24k	22m	105	2
29	DeepSeek V4 Flash Vision Exp	0.0000	$4.02	226.3M	587k	18h 16m	503	5
30	Inkling	0.0000	$5.98	6.1M	33k	29m	144	1
31	DeepSeek V4 Flash Vision Exp	0.0000	$4.14	229.6M	637k	20h 0m	483	1
32	GPT-5.6	0.0000	$29.74	37.3M	170k	2h 5m	211	8
33	Qwen3.8-Max	0.0000	$48.24	157.1M	1.3M	19h 42m	518	2
34	Gemini 3.7 Flash	0.0000	$5.44	38.0M	299k	1h 47m	197	3
35	Inkling	0.0000	$2.97	2.9M	45k	43m	109	2
36	Muse Spark 1.2	0.0000	$53.10	223.4M	907k	9h 16m	535	3
37	DeepSeek V4 Flash Vision Exp	0.0000	$3.11	165.3M	552k	9h 15m	409	6
38	Muse Spark 1.2	0.0000	$11.05	58.0M	328k	3h 11m	241	2
39	Qwen3.8-Max	0.0000	$42.25	133.5M	1.3M	19h 46m	412	2
40	GPT-5.6	0.0000	$23.28	33.7M	166k	2h 33m	207	5
41	DeepSeek V4 Flash Vision Exp	0.0000	$3.44	189.2M	548k	10h 29m	453	4
42	GPT-5.6	0.0000	$47.11	54.0M	192k	2h 34m	262	4
43	Inkling	0.0000	$6.49	6.6M	47k	1h 53m	142	1
44	Qwen3.8-Max	0.0000	$56.35	191.3M	1.2M	19h 55m	589	1
45	Grok 4.6	0.0000	$23.09	23.9M	317k	4h 4m	152	3
46	Gemini 3.7 Flash	0.0000	$3.91	24.5M	212k	1h 28m	155	2
47	Gemini 3.7 Flash	0.0000	$3.82	23.6M	220k	46m	162	2
48	Inkling	0.0000	$6.06	6.2M	36k	1h 0m	118	2
49	Qwen3.8-Max	0.0000	$28.32	86.1M	1.0M	14h 43m	386	3
50	DeepSeek V4 Flash Vision Exp	0.0000	$3.67	205.7M	547k	18h 16m	510	3