Tasks Traces V1 Blog Changelog Contribute GitHub

Traces

Every run: the agent's full trajectory, its submissions over the 20-hour budget, and the final score.

Medium-Range Weather Forecast

task page →110 runs · indexed 2026-10-08

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Opus 5.5	0.6553	$21.31	71.4M	261k	19h 12m	313	10
2	Claude Sonnet 5.5	0.6529	$36.04	148.8M	488k	17h 33m	401	1
3	GPT-6 Astra	0.6474	$297.67	145.6M	335k	19h 47m	441	14
4	Claude Sonnet 5.5	0.6443	$32.66	133.0M	473k	13h 39m	382	9
5	Claude Opus 5.5	0.6405	$16.76	50.5M	249k	15h 23m	250	11
6	GPT-6 Astra	0.6387	$27.55	17.3M	152k	18h 9m	135	12
7	Claude Opus 5.5	0.6368	$21.64	71.5M	273k	18h 3m	288	10
8	GPT-6 Astra	0.6362	$98.77	53.1M	216k	17h 56m	257	14
9	GPT-6 Astra	0.6334	$244.67	123.1M	350k	18h 49m	445	13
10	Claude Opus 5	0.6321	$37.39	53.5M	286k	19h 19m	238	11
11	Claude Opus 5.5	0.6263	$14.11	42.0M	214k	18h 26m	225	7
12	Claude Fable 5.1	0.6214	$15.11	24.2M	125k	16h 58m	204	11
13	Grok 4.7	0.6200	$170.45	150.1M	788k	19h 40m	619	21
14	Claude Sonnet 5.5	0.6188	$32.99	135.6M	450k	18h 0m	394	14
15	Claude Fable 5.1	0.6178	$12.69	20.4M	104k	18h 1m	175	12
16	Claude Sonnet 5.5	0.6171	$43.64	183.7M	536k	18h 29m	447	5
17	Claude Opus 5.5	0.6141	$21.37	70.5M	271k	17h 20m	298	12
18	Claude Sonnet 5.5	0.6027	$41.54	172.7M	541k	18h 15m	423	9
19	Claude Opus 5	0.6013	$44.28	66.4M	300k	18h 19m	297	8
20	Claude Fable 5.1	0.6011	$16.81	27.4M	129k	16h 5m	208	16
21	Claude Fable 5.1	0.5960	$16.06	25.6M	135k	16h 36m	193	12
22	GPT-5.6 Sol	0.5936	$28.82	47.8M	113k	15h 13m	294	4
23	Grok 4.7	0.5877	$164.64	141.5M	437k	18h 53m	596	29
24	GLM-5.3	0.5873	$22.61	71.9M	228k	18h 20m	314	2
25	Claude Fable 5	0.5825	$47.76	32.3M	210k	18h 28m	199	10
26	Claude Opus 5	0.5817	$68.20	109.3M	364k	18h 25m	369	15
27	Grok 4.6	0.5804	$120.40	104.0M	444k	19h 13m	464	37
28	Gemini 3.8 Flash	0.5769	$6.44	56.5M	193k	2h 43m	275	11
29	Gemini 3.7 Flash	0.5747	$22.40	49.9M	145k	13h 34m	256	7
30	Gemini 4 Argon	0.5653	$75.42	53.6M	408k	7h 44m	162	9
31	Claude Haiku 5.5	0.5631	$7.09	116.4M	474k	17h 27m	332	9
32	GLM-5.3 Flash	0.5584	$3.61	51.8M	199k	18h 29m	284	2
33	Claude Haiku 5.5	0.5581	$5.34	84.1M	425k	18h 46m	272	10
34	Gemini 3.7 Flash	0.5566	$16.56	54.7M	160k	9h 58m	285	2
35	Grok 4.6	0.5522	$208.02	184.0M	911k	19h 55m	826	53
36	Grok 4.6	0.5495	$238.57	230.6M	626k	20h 0m	861	62
37	Claude Opus 5	0.5405	$40.40	59.3M	291k	18h 27m	261	16
38	Gemini 3.7 Flash	0.5365	$13.11	37.7M	147k	6h 21m	241	3
39	Gemini 3.7 Flash	0.5356	$19.36	48.1M	172k	8h 5m	252	4
40	Gemini 4 Argon	0.5347	$151.23	88.8M	463k	15h 22m	239	10
41	Grok 4.7	0.5339	$192.06	177.7M	824k	19h 10m	758	17
42	Gemini 3.7 Flash	0.5332	$4.99	21.4M	136k	4h 20m	171	1
43	Gemini 3.8 Flash	0.5330	$4.12	32.2M	140k	2h 31m	197	5
44	GLM-5.3 Flash	0.5292	$1.71	30.3M	154k	17h 56m	222	5
45	Gemini 3.8 Flash	0.5214	$13.54	89.8M	215k	6h 8m	390	14
46	Claude Fable 5	0.5192	$49.62	35.7M	184k	19h 9m	225	8
47	Claude Haiku 5.5	0.5171	$4.72	73.7M	409k	16h 51m	254	1
48	Qwen3.8-Max-0902	0.5127	$30.58	103.4M	640k	19h 23m	449	1
49	Claude Fable 5	0.5126	$51.69	34.8M	233k	18h 40m	219	14
50	Gemini 3.8 Flash	0.4927	$3.71	30.1M	141k	2h 9m	183	2
51	Claude Haiku 5.5	0.4915	$6.89	111.9M	491k	13h 38m	328	3
52	Claude Fable 5	0.4845	$59.30	44.0M	208k	17h 47m	259	6
53	Claude Fable 5	0.4839	$57.20	42.6M	198k	18h 53m	258	9
54	Gemini 4 Argon	0.4825	$53.28	50.9M	579k	8h 27m	159	4
55	GLM-5.3	0.4818	$19.75	47.5M	201k	18h 1m	273	5
56	GLM-5.3	0.4745	$28.03	96.3M	327k	19h 20m	363	7
57	GLM-5.3	0.4723	$21.03	47.8M	206k	18h 36m	293	2
58	Claude Haiku 5.5	0.4720	$4.54	74.1M	339k	17h 59m	276	1
59	Gemini 4 Argon	0.4704	$122.30	82.4M	652k	20h 41m	193	10
60	GLM-5.3	0.4679	$26.30	77.9M	252k	19h 29m	364	2
61	Muse Spark 1.3	0.4646	$17.40	32.8M	150k	19h 29m	250	11
62	Qwen3.8-Max-0902	0.4642	$26.04	87.1M	568k	18h 55m	401	2
63	DeepSeek V4 Flash Vision Exp	0.4639	$1.55	83.0M	253k	19h 5m	403	5
64	GLM-5.3 Flash	0.4629	$3.93	71.7M	248k	18h 47m	338	2
65	Grok 4.6	0.4530	$43.07	35.9M	245k	19h 35m	225	1
66	Qwen3.8-Max-0902	0.4423	$42.33	144.4M	856k	19h 32m	548	1
67	Grok 4.7	0.4365	$22.86	23.1M	216k	4h 0m	130	3
68	Muse Spark 1.3	0.4252	$39.97	63.7M	234k	18h 47m	401	10
69	Grok 4.7	0.4219	$34.89	31.9M	250k	3h 58m	135	11
70	Grok 4.6	0.4154	$92.48	66.7M	263k	19h 27m	303	6
71	Muse Spark 1.3	0.4057	$47.03	41.9M	228k	13h 41m	311	11
72	Qwen3.8-Max	0.4005	$8.94	29.1M	230k	19h 22m	281	1
73	Qwen3.8-Max	0.3989	$14.91	48.7M	387k	19h 25m	357	1
74	Muse Spark 1.3	0.3909	$44.60	41.6M	175k	17h 28m	283	7
75	Gemini 3.8 Flash	0.3886	$4.37	33.2M	181k	2h 4m	200	6
76	GLM-5.3 Flash	0.3763	$2.83	73.4M	262k	16h 13m	343	13
77	Qwen3.8-Max	0.3606	$13.12	44.7M	268k	19h 18m	385	3
78	GLM-5.3 Flash	0.3496	$2.98	85.3M	281k	19h 32m	394	4
79	Muse Spark 1.2	0.3456	$4.38	18.4M	145k	37m	104	2
80	Qwen3.8-Max-0902	0.3435	$56.59	197.4M	966k	18h 25m	518	6
81	Qwen3.8-Max	0.3410	$4.91	14.6M	172k	18h 25m	189	4
82	Qwen3.8-Max	0.3360	$8.01	25.7M	213k	18h 56m	239	8
83	Muse Spark 1.2	0.3227	$38.37	235.4M	385k	2h 26m	592	5
84	Muse Spark 1.3	0.3160	$38.63	81.9M	260k	18h 49m	434	13
85	Muse Spark 1.2	0.3018	$11.62	45.2M	202k	1h 9m	181	7
86	Muse Spark 1.2	0.2944	$2.12	8.7M	99k	50m	86	1
87	Muse Spark 1.2	0.2758	$11.17	51.5M	378k	2h 40m	211	6
88	Inkling	0.2568	$3.54	3.6M	37k	1h 43m	97	1
89	Inkling	0.2237	$3.91	3.9M	47k	1h 43m	120	1
90	Inkling	0.1542	$5.24	5.4M	39k	1h 50m	132	2
91	Inkling	0.1482	$3.49	3.6M	19k	1h 32m	102	1
92	GPT-5.6 Sol	0.0000	$122.35	137.1M	209k	17h 41m	576	7
93	GPT-6 Astra	0.0000	$488.12	238.0M	474k	17h 46m	610	16
94	Kimi K3	0.0000	$20.25	49.9M	257k	19h 31m	296	8
95	Kimi K3	0.0000	$16.03	42.3M	172k	19h 35m	282	1
96	Kimi K3	0.0000	$13.29	32.3M	193k	19h 22m	245	2
97	Gemini 4 Argon	0.0000	$44.59	35.1M	340k	6h 53m	144	5
98	Inkling	0.0000	$6.20	6.2M	72k	38m	132	1
99	DeepSeek V4 Flash Vision Exp	0.0000	$1.66	90.1M	258k	19h 29m	392	3
100	Claude Opus 5	0.0000	$33.36	46.8M	269k	19h 5m	229	5
101	DeepSeek V4 Flash Vision Exp	0.0000	$1.99	110.4M	289k	18h 13m	487	4
102	DeepSeek V4 Flash Vision Exp	0.0000	$1.55	84.2M	243k	18h 42m	398	5
103	GPT-5.6 Sol	0.0000	$63.62	70.4M	130k	17h 56m	290	3
104	Kimi K3	0.0000	$11.04	28.1M	135k	19h 31m	261	6
105	GPT-5.6 Sol	0.0000	$72.57	94.1M	189k	15h 53m	478	14
106	DeepSeek V4 Flash Vision Exp	0.0000	$1.68	93.8M	242k	19h 27m	452	2
107	Claude Fable 5.1	0.0000	$14.60	23.4M	121k	14h 28m	186	8
108	Qwen3.8-Max-0902	0.0000	$27.25	86.2M	783k	19h 31m	389	3
109	GPT-5.6 Sol	0.0000	$88.96	100.2M	138k	14h 52m	416	8
110	Kimi K3	0.0000	$19.09	48.5M	196k	18h 52m	326	6