Tasks Traces V1 Blog Changelog Contribute GitHub

Traces

Every run: the agent's full trajectory, its submissions over the 20-hour budget, and the final score.

Astronomy Toolkit

task page →110 runs · indexed 2026-10-08

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Opus 5.5	0.9999	$74.48	265.3M	811k	19h 2m	653	24
2	Gemini 4 Argon	0.9999	$11.62	30.2M	300k	2h 33m	131	3
3	Claude Opus 5.5	0.9931	$69.52	233.4M	859k	17h 56m	587	25
4	Claude Opus 5.5	0.9907	$77.01	274.8M	827k	18h 8m	669	37
5	Claude Fable 5	0.9249	$134.09	102.7M	413k	9h 12m	289	20
6	Claude Opus 5	0.9214	$95.87	150.8M	548k	9h 35m	337	5
7	Claude Opus 5	0.9179	$89.22	139.0M	522k	13h 10m	320	6
8	Claude Sonnet 5.5	0.9039	$64.46	256.3M	1.0M	13h 39m	599	11
9	Claude Sonnet 5.5	0.9012	$70.07	279.4M	1.1M	11h 52m	618	11
10	Gemini 4 Argon	0.8999	$8.32	27.3M	281k	1h 26m	117	2
11	Claude Fable 5	0.8978	$116.44	85.6M	417k	5h 53m	271	12
12	Claude Fable 5.1	0.8953	$64.06	109.1M	391k	4h 49m	305	17
13	Claude Fable 5.1	0.8935	$57.22	97.5M	350k	6h 25m	298	24
14	Claude Opus 5	0.8866	$79.79	122.4M	498k	14h 37m	303	4
15	Claude Fable 5	0.8818	$104.89	77.6M	364k	13h 47m	261	9
16	Claude Opus 5.5	0.8205	$49.70	176.5M	532k	9h 15m	362	10
17	GPT-6 Astra	0.8045	$135.65	67.7M	278k	2h 40m	277	5
18	Claude Fable 5.1	0.8000	$36.76	59.7M	285k	3h 53m	220	11
19	Claude Fable 5.1	0.7999	$32.01	51.0M	267k	3h 57m	202	12
20	Claude Fable 5	0.7998	$113.89	82.7M	426k	7h 1m	260	9
21	GPT-6 Astra	0.7969	$2365.47	1121.6M	2.3M	19h 43m	2548	49
22	Claude Fable 5.1	0.7955	$42.10	68.8M	317k	4h 11m	232	12
23	Claude Opus 5	0.7648	$96.45	153.7M	524k	15h 7m	358	5
24	Claude Fable 5	0.7616	$120.57	89.4M	424k	12h 14m	280	10
25	Gemini 3.7 Flash	0.7294	$8.39	49.1M	408k	2h 12m	218	1
26	GPT-6 Astra	0.7058	$233.15	111.6M	353k	2h 41m	356	10
27	Gemini 3.8 Flash	0.7044	$5.05	41.3M	227k	1h 51m	218	2
28	Claude Opus 5.5	0.7024	$55.16	202.1M	545k	11h 37m	413	16
29	GPT-6 Astra	0.7013	$86.82	45.8M	224k	2h 2m	214	5
30	GLM-5.3	0.6985	$72.52	246.9M	707k	19h 16m	599	6
31	GPT-6 Astra	0.6917	$365.75	173.3M	402k	3h 17m	457	7
32	Claude Sonnet 5.5	0.6877	$35.84	138.7M	621k	5h 52m	269	5
33	Claude Opus 5	0.6837	$80.75	120.3M	550k	6h 30m	272	7
34	Kimi K3	0.6250	$60.98	152.4M	478k	16h 31m	414	12
35	Claude Sonnet 5.5	0.5306	$67.58	269.9M	1.1M	17h 27m	630	9
36	Grok 4.6	0.5299	$15.57	15.6M	323k	1h 52m	106	4
37	Gemini 4 Argon	0.5135	$11.83	39.0M	318k	1h 48m	154	2
38	GPT-5.6 Sol	0.4849	$43.38	52.6M	186k	3h 16m	263	5
39	Claude Haiku 5.5	0.4030	$8.08	127.7M	586k	6h 4m	275	2
40	Claude Sonnet 5.5	0.4029	$56.04	216.5M	989k	10h 27m	546	8
41	GLM-5.3	0.4028	$57.26	195.8M	552k	12h 7m	426	8
42	Qwen3.8-Max-0902	0.4022	$62.34	193.6M	1.6M	19h 35m	663	3
43	GLM-5.3	0.4017	$72.89	226.5M	514k	17h 29m	476	6
44	Gemini 3.7 Flash	0.4014	$3.09	23.0M	225k	38m	153	3
45	Grok 4.6	0.3993	$48.11	46.2M	395k	3h 31m	208	6
46	Kimi K3	0.3983	$90.41	238.0M	505k	20h 1m	587	20
47	Kimi K3	0.3959	$20.08	46.3M	279k	6h 2m	205	5
48	GLM-5.3	0.3930	$77.97	220.3M	559k	15h 22m	481	2
49	Kimi K3	0.3804	$70.49	167.8M	452k	19h 35m	468	8
50	Claude Haiku 5.5	0.3093	$8.98	142.5M	643k	5h 2m	290	4
51	Grok 4.7	0.3076	$28.45	28.6M	218k	1h 15m	142	5
52	Claude Haiku 5.5	0.3019	$6.55	100.4M	539k	4h 3m	248	1
53	Gemini 3.8 Flash	0.2999	$4.78	40.8M	198k	1h 11m	225	4
54	Grok 4.7	0.2999	$23.31	23.1M	224k	1h 29m	118	3
55	Kimi K3	0.2995	$53.30	137.6M	418k	11h 54m	430	7
56	GLM-5.3 Flash	0.2975	$5.82	179.7M	476k	11h 27m	453	4
57	Qwen3.8-Max-0902	0.2897	$62.30	208.2M	1.4M	20h 2m	506	1
58	GLM-5.3	0.2893	$64.65	196.5M	549k	19h 1m	419	8
59	Muse Spark 1.3	0.2056	$84.96	112.8M	364k	5h 13m	428	19
60	Grok 4.6	0.2000	$34.68	34.5M	354k	2h 42m	175	8
61	Grok 4.6	0.1971	$39.66	35.3M	400k	3h 43m	165	11
62	GPT-5.6 Sol	0.1823	$39.28	48.9M	195k	2h 7m	255	4
63	GLM-5.3 Flash	0.1158	$3.08	93.0M	371k	3h 57m	320	3
64	Claude Haiku 5.5	0.1077	$9.75	147.2M	915k	6h 7m	380	7
65	Gemini 3.8 Flash	0.1037	$5.88	54.2M	233k	1h 9m	255	3
66	Muse Spark 1.3	0.1015	$57.05	70.7M	278k	5h 16m	361	11
67	Muse Spark 1.2	0.1013	$11.07	59.7M	393k	2h 7m	259	1
68	Muse Spark 1.2	0.1000	$12.16	66.0M	402k	2h 34m	231	4
69	Muse Spark 1.3	0.0976	$81.13	81.9M	258k	12h 38m	397	2
70	Muse Spark 1.2	0.0563	$26.33	90.8M	427k	5h 8m	297	2
71	Claude Haiku 5.5	0.0349	$11.89	182.6M	1.1M	7h 13m	458	3
72	Muse Spark 1.3	0.0279	$94.74	98.7M	274k	19h 49m	426	38
73	Grok 4.7	0.0108	$18.18	18.2M	206k	1h 1m	105	3
74	Grok 4.7	0.0075	$224.04	218.5M	550k	3h 42m	699	15
75	Qwen3.8-Max-0902	0.0063	$53.54	172.9M	1.4M	19h 54m	408	1
76	Grok 4.7	0.0032	$13.33	14.3M	178k	52m	90	2
77	Muse Spark 1.3	0.0028	$18.89	27.3M	239k	2h 57m	175	4
78	Qwen3.8-Max-0902	0.0016	$62.25	179.2M	2.0M	19h 45m	513	2
79	GLM-5.3 Flash	0.0006	$4.33	118.3M	395k	10h 5m	388	3
80	Qwen3.8-Max-0902	0.0003	$33.82	90.4M	1.2M	20h 0m	309	1
81	Qwen3.8-Max	0.0000	$42.82	145.6M	925k	17h 28m	524	2
82	GLM-5.3 Flash	0.0000	$10.24	291.8M	558k	13h 32m	631	2
83	Inkling	0.0000	$4.59	4.7M	24k	22m	105	2
84	DeepSeek V4 Flash Vision Exp	0.0000	$4.02	226.3M	587k	18h 16m	503	5
85	Inkling	0.0000	$5.98	6.1M	33k	29m	144	1
86	DeepSeek V4 Flash Vision Exp	0.0000	$4.14	229.6M	637k	20h 0m	483	1
87	GPT-5.6 Sol	0.0000	$29.74	37.3M	170k	2h 5m	211	8
88	Gemini 3.8 Flash	0.0000	$5.19	43.5M	254k	50m	231	3
89	Qwen3.8-Max	0.0000	$48.24	157.1M	1.3M	19h 42m	518	2
90	Gemini 4 Argon	0.0000	$8.85	35.0M	322k	1h 26m	137	2
91	Gemini 3.7 Flash	0.0000	$5.44	38.0M	299k	1h 47m	197	3
92	Inkling	0.0000	$2.97	2.9M	45k	43m	109	2
93	Muse Spark 1.2	0.0000	$53.10	223.4M	907k	9h 16m	535	3
94	DeepSeek V4 Flash Vision Exp	0.0000	$3.11	165.3M	552k	9h 15m	409	6
95	GLM-5.3 Flash	0.0000	$6.17	161.9M	462k	9h 59m	452	3
96	Muse Spark 1.2	0.0000	$11.05	58.0M	328k	3h 11m	241	2
97	Gemini 4 Argon	0.0000	$27.00	17.6M	236k	2h 6m	93	2
98	Qwen3.8-Max	0.0000	$42.25	133.5M	1.3M	19h 46m	412	2
99	GPT-5.6 Sol	0.0000	$23.28	33.7M	166k	2h 33m	207	5
100	DeepSeek V4 Flash Vision Exp	0.0000	$3.44	189.2M	548k	10h 29m	453	4
101	GPT-5.6 Sol	0.0000	$47.11	54.0M	192k	2h 34m	262	4
102	Gemini 3.8 Flash	0.0000	$13.11	116.9M	410k	2h 14m	355	6
103	Inkling	0.0000	$6.49	6.6M	47k	1h 53m	142	1
104	Qwen3.8-Max	0.0000	$56.35	191.3M	1.2M	19h 55m	589	1
105	Grok 4.6	0.0000	$23.09	23.9M	317k	4h 4m	152	3
106	Gemini 3.7 Flash	0.0000	$3.91	24.5M	212k	1h 28m	155	2
107	Gemini 3.7 Flash	0.0000	$3.82	23.6M	220k	46m	162	2
108	Inkling	0.0000	$6.06	6.2M	36k	1h 0m	118	2
109	Qwen3.8-Max	0.0000	$28.32	86.1M	1.0M	14h 43m	386	3
110	DeepSeek V4 Flash Vision Exp	0.0000	$3.67	205.7M	547k	18h 16m	510	3