Tasks Traces V1 Blog Changelog Contribute GitHub

Traces

Every run: the agent's full trajectory, its submissions over the 20-hour budget, and the final score.

FFmpeg libswscale Optimization

task page →110 runs · indexed 2026-10-08

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Opus 5.5	0.8444	$169.75	555.3M	2.2M	12h 14m	1230	42
2	Claude Fable 5.1	0.8080	$283.04	487.0M	1.6M	13h 9m	1070	46
3	Claude Sonnet 5.5	0.5593	$210.44	829.8M	3.5M	18h 18m	1662	32
4	Claude Fable 5.1	0.5509	$202.06	343.4M	1.3M	9h 13m	819	28
5	Claude Sonnet 5.5	0.5128	$191.15	731.8M	3.5M	16h 45m	1549	18
6	Claude Fable 5.1	0.5106	$148.93	254.0M	906k	6h 11m	641	18
7	Claude Fable 5	0.3296	$651.27	511.9M	1.9M	17h 29m	1099	44
8	GPT-6 Astra	0.2755	$169.85	82.4M	307k	2h 7m	290	7
9	Claude Opus 5	0.2730	$386.74	584.2M	2.5M	17h 26m	1236	48
10	Claude Opus 5	0.2685	$323.38	505.1M	1.9M	18h 50m	1055	36
11	Claude Haiku 5.5	0.2461	$32.33	512.1M	2.4M	8h 34m	1174	9
12	Claude Fable 5	0.2400	$694.93	559.7M	1.8M	18h 56m	1226	65
13	Claude Opus 5	0.2348	$359.20	577.0M	1.9M	15h 26m	1190	24
14	Claude Fable 5	0.2293	$457.04	364.2M	1.2M	18h 33m	758	37
15	Claude Haiku 5.5	0.2204	$29.73	450.8M	2.6M	7h 6m	996	2
16	Claude Opus 5	0.2127	$514.92	809.2M	2.9M	17h 33m	1760	45
17	GPT-6 Astra	0.2021	$313.12	153.5M	377k	2h 54m	458	9
18	GPT-6 Astra	0.1983	$306.45	149.1M	351k	2h 48m	444	8
19	Claude Opus 5	0.1899	$481.68	774.7M	2.5M	14h 53m	1592	54
20	GLM-5.3	0.1752	$226.40	818.4M	2.0M	19h 33m	1884	18
21	GPT-6 Astra	0.1741	$262.36	125.2M	349k	2h 37m	379	6
22	Claude Fable 5	0.1712	$501.72	411.4M	1.2M	18h 49m	877	42
23	Claude Haiku 5.5	0.1587	$30.27	466.6M	2.5M	11h 4m	1072	17
24	Gemini 4 Argon	0.1572	$160.32	346.6M	1.9M	11h 31m	765	13
25	Claude Fable 5	0.1436	$530.26	432.3M	1.3M	19h 40m	864	28
26	GLM-5.3	0.1410	$148.35	524.2M	1.7M	16h 26m	1303	8
27	GLM-5.3	0.1369	$139.76	505.8M	1.2M	11h 14m	1097	9
28	Claude Fable 5.1	0.1368	$219.11	385.2M	1.1M	10h 27m	835	22
29	Muse Spark 1.3	0.1277	$499.70	617.7M	951k	19h 17m	1401	24
30	GLM-5.3 Flash	0.1268	$16.00	489.1M	1.2M	13h 53m	1179	110
31	Kimi K3	0.1260	$127.79	342.6M	1.2M	13h 59m	827	13
32	Kimi K3	0.1252	$211.22	606.6M	1.4M	19h 43m	1382	3
33	GLM-5.3	0.1231	$175.55	619.5M	1.7M	16h 47m	1509	8
34	Claude Opus 5.5	0.1224	$83.47	264.7M	1.1M	5h 11m	549	14
35	Claude Opus 5.5	0.1185	$175.12	575.5M	2.3M	11h 51m	1203	46
36	GPT-6 Astra	0.1179	$275.12	132.2M	355k	2h 35m	396	6
37	Muse Spark 1.3	0.1028	$438.57	557.6M	1.0M	16h 30m	1340	23
38	Claude Sonnet 5.5	0.1012	$189.51	720.9M	3.5M	18h 17m	1538	35
39	Muse Spark 1.3	0.1003	$186.76	239.3M	502k	7h 59m	559	14
40	Muse Spark 1.3	0.0996	$157.06	200.2M	465k	6h 34m	504	12
41	GLM-5.3 Flash	0.0819	$11.44	312.2M	1.0M	11h 47m	786	2
42	Kimi K3	0.0799	$127.97	356.2M	1.1M	15h 4m	914	11
43	GLM-5.3	0.0783	$177.95	625.3M	1.7M	17h 1m	1420	8
44	Muse Spark 1.3	0.0779	$279.58	336.6M	595k	9h 26m	688	17
45	Claude Haiku 5.5	0.0742	$11.46	167.4M	1.1M	3h 2m	421	4
46	GLM-5.3 Flash	0.0721	$12.54	348.2M	1.1M	9h 45m	838	4
47	Grok 4.7	0.0717	$958.45	972.7M	3.5M	19h 0m	3810	51
48	GLM-5.3 Flash	0.0710	$14.74	418.0M	1.0M	13h 2m	963	6
49	Claude Sonnet 5.5	0.0703	$164.01	627.4M	3.0M	15h 19m	1268	23
50	Grok 4.7	0.0692	$75.94	76.6M	393k	2h 13m	349	5
51	DeepSeek V4 Flash Vision Exp	0.0668	$8.52	493.3M	1.0M	6h 44m	1095	8
52	Kimi K3	0.0639	$138.03	382.3M	1.1M	17h 32m	902	2
53	DeepSeek V4 Flash Vision Exp	0.0636	$9.11	528.7M	1.1M	6h 10m	1172	5
54	DeepSeek V4 Flash Vision Exp	0.0602	$13.77	800.3M	1.6M	9h 14m	1831	19
55	GLM-5.3 Flash	0.0591	$10.52	294.6M	1.1M	15h 39m	768	1
56	Grok 4.7	0.0562	$84.27	86.0M	423k	2h 12m	353	5
57	DeepSeek V4 Flash Vision Exp	0.0483	$9.39	554.1M	1.0M	5h 54m	1197	16
58	Grok 4.7	0.0482	$1089.41	1092.6M	3.0M	15h 42m	3929	49
59	Claude Sonnet 5.5	0.0471	$76.69	272.4M	1.7M	6h 51m	612	9
60	Claude Opus 5.5	0.0441	$106.98	317.9M	1.6M	6h 52m	693	18
61	Gemini 3.8 Flash	0.0421	$22.33	232.8M	629k	2h 36m	656	8
62	Qwen3.8-Max	0.0373	$86.44	286.2M	2.3M	17h 43m	706	6
63	Grok 4.6	0.0360	$419.25	404.2M	2.4M	19h 56m	1517	114
64	Qwen3.8-Max	0.0181	$64.05	193.9M	2.4M	18h 48m	555	1
65	Gemini 3.7 Flash	0.0125	$20.83	184.0M	617k	2h 35m	524	2
66	Qwen3.8-Max	0.0053	$46.03	124.6M	2.3M	19h 58m	424	1
67	Gemini 3.7 Flash	0.0032	$19.94	167.8M	795k	2h 4m	496	7
68	Grok 4.7	0.0022	$90.46	89.2M	451k	3h 23m	379	9
69	Grok 4.6	0.0000	$1121.42	1126.0M	2.4M	18h 11m	3693	434
70	Muse Spark 1.2	0.0000	$15.42	90.0M	327k	41m	286	2
71	Grok 4.6	0.0000	$68.40	59.6M	603k	3h 50m	229	4
72	Muse Spark 1.2	0.0000	$16.97	85.6M	384k	1h 38m	227	4
73	Gemini 3.8 Flash	0.0000	$23.73	247.4M	548k	2h 37m	674	3
74	Claude Fable 5.1	0.0000	$217.24	363.8M	1.5M	10h 22m	893	27
75	Gemini 4 Argon	0.0000	$156.84	296.0M	1.6M	9h 55m	668	11
76	Gemini 4 Argon	0.0000	$106.17	273.4M	1.3M	8h 28m	605	17
77	Qwen3.8-Max	0.0000	$54.69	158.2M	2.4M	18h 58m	475	1
78	Grok 4.6	0.0000	$132.56	129.2M	778k	5h 28m	562	27
79	Inkling	0.0000	$11.24	11.7M	27k	1h 20m	165	2
80	Inkling	0.0000	$15.45	16.1M	35k	22m	188	2
81	Gemini 3.8 Flash	0.0000	$19.00	192.6M	496k	2h 5m	536	4
82	Qwen3.8-Max-0902	0.0000	$39.61	68.0M	3.6M	20h 1m	275	1
83	Qwen3.8-Max	0.0000	$30.12	66.3M	2.1M	19h 55m	298	1
84	Inkling	0.0000	$39.66	41.4M	77k	41m	354	2
85	Gemini 3.7 Flash	0.0000	$25.27	228.2M	1.1M	2h 14m	655	5
86	GPT-5.6 Sol	0.0000	$66.82	75.1M	179k	1h 59m	318	8
87	Gemini 3.8 Flash	0.0000	$31.46	339.3M	663k	5h 43m	881	17
88	Qwen3.8-Max-0902	0.0000	$44.84	72.8M	3.4M	20h 2m	345	1
89	Inkling	0.0000	$15.51	16.2M	36k	32m	190	1
90	Muse Spark 1.2	0.0000	$11.90	64.7M	340k	1h 14m	223	2
91	GPT-5.6 Sol	0.0000	$78.43	84.8M	190k	2h 4m	348	5
92	Grok 4.6	0.0000	$380.75	368.6M	1.8M	11h 52m	1370	67
93	Muse Spark 1.2	0.0000	$35.70	203.8M	978k	3h 29m	468	8
94	Claude Haiku 5.5	0.0000	$13.45	205.4M	1.1M	2h 38m	488	3
95	GPT-5.6 Sol	0.0000	$123.31	124.9M	233k	2h 12m	422	3
96	Kimi K3	0.0000	$171.14	500.1M	1.1M	19h 41m	1175	47
97	Qwen3.8-Max-0902	0.0000	$51.17	115.3M	3.5M	20h 1m	358	1
98	Claude Opus 5.5	0.0000	$168.91	529.9M	2.4M	15h 41m	1150	37
99	GPT-5.6 Sol	0.0000	$76.67	86.4M	186k	3h 5m	369	10
100	Gemini 4 Argon	0.0000	$58.88	167.7M	1.1M	5h 59m	405	7
101	Qwen3.8-Max-0902	0.0000	$43.43	82.1M	3.6M	20h 1m	306	1
102	DeepSeek V4 Flash Vision Exp	0.0000	$12.34	718.0M	1.4M	8h 32m	1678	16
103	GPT-5.6 Sol	0.0000	$117.96	120.8M	233k	3h 42m	405	8
104	Inkling	0.0000	$11.17	11.6M	27k	16m	167	1
105	Qwen3.8-Max-0902	0.0000	$72.15	209.7M	3.0M	19h 38m	543	1
106	Gemini 4 Argon	0.0000	$190.93	224.2M	1.2M	7h 43m	531	8
107	Gemini 3.8 Flash	0.0000	$22.37	231.7M	581k	2h 9m	683	12
108	Gemini 3.7 Flash	0.0000	$16.99	131.7M	937k	2h 45m	448	6
109	Muse Spark 1.2	0.0000	$27.98	159.1M	568k	1h 40m	345	2
110	Gemini 3.7 Flash	0.0000	$44.74	251.9M	922k	8h 1m	758	6