Traces

Every run: the agent's full trajectory, its submissions over the 20-hour budget, and the final score.

Kolmogorov Audio Compression

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Gemini 3.7 Flash	0.0240	$13.44	120.0M	296k	7h 42m	465	7
2	Gemini 3.7 Flash	0.0232	$21.26	112.5M	588k	9h 45m	340	20
3	Gemini 3.7 Flash	0.0189	$26.85	83.8M	686k	9h 43m	267	16
4	Gemini 3.7 Flash	0.0110	$6.63	19.2M	229k	5h 4m	166	5
5	Gemini 3.7 Flash	0.0097	$3.47	13.3M	97k	5h 22m	153	4

Astronomy Toolkit

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Gemini 3.7 Flash	0.7294	$8.39	49.1M	408k	2h 12m	218	1
2	Gemini 3.7 Flash	0.4014	$3.09	23.0M	225k	38m	153	3
3	Gemini 3.7 Flash	0.0000	$5.44	38.0M	299k	1h 47m	197	3
4	Gemini 3.7 Flash	0.0000	$3.91	24.5M	212k	1h 28m	155	2
5	Gemini 3.7 Flash	0.0000	$3.82	23.6M	220k	46m	162	2

Cranelift Codegen Optimization

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Gemini 3.7 Flash	0.0802	$35.86	231.6M	242k	6h 15m	574	20
2	Gemini 3.7 Flash	0.0189	$20.57	157.5M	204k	2h 49m	432	17
3	Gemini 3.7 Flash	0.0110	$23.00	164.8M	188k	4h 11m	495	19
4	Gemini 3.7 Flash	0.0000	$30.61	210.1M	211k	5h 15m	593	21
5	Gemini 3.7 Flash	0.0000	$47.46	341.2M	378k	8h 44m	828	22

Dart Style in Haskell

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Gemini 3.7 Flash	0.6517	$199.28	2123.1M	2.6M	19h 55m	4917	62
2	Gemini 3.7 Flash	0.5376	$121.24	1110.9M	1.5M	19h 27m	2524	37
3	Gemini 3.7 Flash	0.4365	$75.56	629.3M	934k	19h 55m	1331	3
4	Gemini 3.7 Flash	0.4242	$33.92	324.0M	891k	3h 25m	780	8
5	Gemini 3.7 Flash	0.3137	$25.24	282.5M	578k	1h 28m	590	2

FFmpeg libswscale Optimization

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Gemini 3.7 Flash	0.0125	$20.83	184.0M	617k	2h 35m	524	2
2	Gemini 3.7 Flash	0.0032	$19.94	167.8M	795k	2h 4m	496	7
3	Gemini 3.7 Flash	0.0000	$25.27	228.2M	1.1M	2h 14m	655	5
4	Gemini 3.7 Flash	0.0000	$16.99	131.7M	937k	2h 45m	448	6
5	Gemini 3.7 Flash	0.0000	$44.74	251.9M	922k	8h 1m	758	6

Crash-Proof Flash Filesystem

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Gemini 3.7 Flash	0.0217	$93.80	813.5M	2.3M	19h 56m	2141	1
2	Gemini 3.7 Flash	0.0112	$89.95	756.9M	1.7M	20h 0m	1907	1
3	Gemini 3.7 Flash	0.0049	$62.15	472.1M	1.5M	20h 0m	1139	1
4	Gemini 3.7 Flash	0.0024	$124.70	1220.5M	2.9M	20h 0m	2795	1
5	Gemini 3.7 Flash	0.0010	$64.73	520.0M	1.4M	20h 0m	1356	1

FrogsGame Post-Training

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Gemini 3.7 Flash	0.0620	$5.03	14.3M	76k	5h 27m	128	1
2	Gemini 3.7 Flash	0.0420	$30.72	72.9M	130k	16h 34m	301	1
3	Gemini 3.7 Flash	0.0320	$45.33	138.6M	173k	19h 58m	396	1
4	Gemini 3.7 Flash	0.0220	$3.80	24.6M	101k	2h 41m	142	2
5	Gemini 3.7 Flash	0.0200	$37.07	96.2M	140k	17h 26m	321	1

Stepper Music Sequencer GBA

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Gemini 3.7 Flash	0.2415	$49.29	499.9M	1.0M	4h 48m	1302	2
2	Gemini 3.7 Flash	0.1895	$40.63	436.9M	982k	3h 1m	1134	2
3	Gemini 3.7 Flash	0.1724	$88.83	943.5M	2.1M	6h 27m	2449	1
4	Gemini 3.7 Flash	0.1698	$29.14	318.3M	646k	2h 19m	856	2
5	Gemini 3.7 Flash	0.1224	$55.17	592.3M	1.3M	5h 23m	1507	5

Git to Zig

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Gemini 3.7 Flash	0.1808	$157.01	1517.8M	2.4M	19h 43m	3256	1
2	Gemini 3.7 Flash	0.1495	$79.59	838.0M	1.2M	20h 0m	1855	2
3	Gemini 3.7 Flash	0.1433	$101.57	1069.8M	1.5M	19h 58m	2372	1
4	Gemini 3.7 Flash	0.1237	$71.55	746.3M	1.3M	19h 57m	1690	4
5	Gemini 3.7 Flash	0.0000†	$76.41	819.6M	1.5M	19h 43m	1734	1

† score adjusted by manual review; the trace's Judge tab carries the measured score and the rationale.

Granite Mamba2 Inference Optimization

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Gemini 3.7 Flash	0.7839	$8.97	84.4M	442k	56m	281	3
2	Gemini 3.7 Flash	0.3912	$15.89	144.8M	798k	2h 17m	479	5
3	Gemini 3.7 Flash	0.3149	$10.74	97.9M	514k	1h 12m	278	2
4	Gemini 3.7 Flash	0.2425	$12.51	121.4M	559k	2h 10m	326	2
5	Gemini 3.7 Flash	0.2101	$6.70	49.8M	379k	47m	212	3

Higgs Uncertainty Inference

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Gemini 3.7 Flash	0.1681	$1.72	9.3M	135k	1h 3m	102	1
2	Gemini 3.7 Flash	0.1295	$1.98	10.2M	141k	50m	105	3
3	Gemini 3.7 Flash	0.1075	$1.62	9.2M	146k	1h 7m	102	1
4	Gemini 3.7 Flash	0.0798	$2.53	10.3M	161k	1h 26m	110	1
5	Gemini 3.7 Flash	0.0647	$3.79	20.7M	194k	1h 41m	139	2

SGLang Inference System Optimization

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Gemini 3.7 Flash	0.0169	$19.15	195.9M	289k	3h 33m	505	4
2	Gemini 3.7 Flash	0.0000	$9.47	100.2M	149k	1h 43m	316	4
3	Gemini 3.7 Flash	0.0000	$9.02	85.7M	143k	1h 52m	306	1
4	Gemini 3.7 Flash	0.0000	$12.58	90.8M	182k	4h 36m	317	1
5	Gemini 3.7 Flash	0.0000	$6.58	51.9M	91k	1h 41m	296	4

Lean 4 Kernel Type Checker in Pascal

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Gemini 3.7 Flash	0.9448	$95.08	678.0M	1.4M	19h 49m	1660	1
2	Gemini 3.7 Flash	0.9361	$54.83	453.2M	1.3M	12h 43m	1224	1
3	Gemini 3.7 Flash	0.1248	$74.81	557.5M	1.1M	14h 22m	1217	1
4	Gemini 3.7 Flash	0.0087	$48.94	384.1M	939k	13h 20m	1118	1
5	Gemini 3.7 Flash	0.0000	$101.50	861.0M	1.8M	19h 58m	2092	1

libexpat Optimization

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Gemini 3.7 Flash	0.2399	$41.35	394.7M	960k	4h 30m	1009	6
2	Gemini 3.7 Flash	0.1572	$41.30	358.7M	785k	6h 1m	899	5
3	Gemini 3.7 Flash	0.1555	$24.69	216.5M	877k	2h 56m	624	2
4	Gemini 3.7 Flash	0.0000	$33.92	328.7M	960k	2h 58m	810	20
5	Gemini 3.7 Flash	0.0000	$29.56	325.1M	599k	2h 10m	830	2

Lua Native Compiler

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Gemini 3.7 Flash	0.6998	$21.69	183.4M	580k	2h 58m	400	1
2	Gemini 3.7 Flash	0.6541	$62.39	519.3M	1.2M	7h 8m	1260	1
3	Gemini 3.7 Flash	0.6517	$26.08	255.3M	824k	1h 58m	670	1
4	Gemini 3.7 Flash	0.6392	$29.13	278.2M	822k	3h 5m	751	1
5	Gemini 3.7 Flash	0.3270	$79.34	743.6M	1.7M	8h 42m	1815	1

Machine-Learned Interatomic Potential

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Gemini 3.7 Flash	0.8422	$11.45	30.2M	159k	8h 5m	208	3
2	Gemini 3.7 Flash	0.7285	$10.95	29.5M	154k	8h 16m	213	7
3	Gemini 3.7 Flash	0.5113	$8.21	27.8M	178k	5h 29m	184	2
4	Gemini 3.7 Flash	0.4203	$5.25	21.8M	116k	4h 45m	178	1
5	Gemini 3.7 Flash	0.0010	$6.01	20.4M	174k	4h 35m	154	2

Medium-Range Weather Forecast

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Gemini 3.7 Flash	0.5747	$22.40	49.9M	145k	13h 34m	256	7
2	Gemini 3.7 Flash	0.5566	$16.56	54.7M	160k	9h 58m	285	2
3	Gemini 3.7 Flash	0.5365	$13.11	37.7M	147k	6h 21m	241	3
4	Gemini 3.7 Flash	0.5356	$19.36	48.1M	172k	8h 5m	252	4
5	Gemini 3.7 Flash	0.5332	$4.99	21.4M	136k	4h 20m	171	1

MEG Speech Decoding

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Gemini 3.7 Flash	0.1130	$4.87	33.1M	132k	3h 51m	226	1
2	Gemini 3.7 Flash	0.1121	$12.12	51.3M	122k	7h 24m	312	4
3	Gemini 3.7 Flash	0.0948	$9.93	38.7M	111k	7h 46m	266	5
4	Gemini 3.7 Flash	0.0854	$10.11	35.8M	110k	7h 53m	303	1
5	Gemini 3.7 Flash	0.0000	$12.20	48.7M	173k	7h 50m	285	2

Wan 2.1 on MAX/Mojo

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Gemini 3.7 Flash	0.0349	$23.63	178.6M	776k	6h 26m	569	1
2	Gemini 3.7 Flash	0.0347	$27.48	166.6M	453k	6h 47m	438	1
3	Gemini 3.7 Flash	0.0331	$42.63	248.9M	797k	11h 49m	716	1
4	Gemini 3.7 Flash	0.0317	$28.92	149.4M	462k	8h 34m	380	1
5	Gemini 3.7 Flash	0.0310	$36.97	183.1M	560k	8h 49m	483	1

MS/MS De Novo Generation

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Gemini 3.7 Flash	0.1064	$8.88	40.2M	172k	5h 7m	265	2
2	Gemini 3.7 Flash	0.0865	$5.25	20.8M	119k	3h 41m	172	1
3	Gemini 3.7 Flash	0.0846	$10.26	48.6M	211k	7h 0m	264	3
4	Gemini 3.7 Flash	0.0000	$6.93	28.3M	150k	3h 32m	203	4
5	Gemini 3.7 Flash	0.0000	$15.58	62.6M	304k	13h 22m	285	2

Multi-GPU Efficient Finetuning

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Gemini 3.7 Flash	0.2143	$10.24	29.6M	72k	9h 49m	181	31
2	Gemini 3.7 Flash	0.1667	$11.85	31.1M	102k	19h 53m	202	10
3	Gemini 3.7 Flash	0.0000	$22.18	47.8M	107k	18h 31m	299	2
4	Gemini 3.7 Flash	0.0000	$8.30	27.1M	102k	9h 38m	188	10
5	Gemini 3.7 Flash	0.0000	$30.31	73.0M	132k	18h 49m	310	13

Notebook Compression

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Gemini 3.7 Flash	0.1775	$2.95	13.6M	108k	3h 17m	129	2
2	Gemini 3.7 Flash	0.1714	$6.92	27.7M	160k	6h 28m	236	5
3	Gemini 3.7 Flash	0.0417	$2.99	16.0M	112k	3h 8m	165	6
4	Gemini 3.7 Flash	0.0412	$3.05	19.8M	150k	3h 10m	194	10
5	Gemini 3.7 Flash	0.0403	$4.56	17.9M	128k	4h 20m	145	8

Flight-Sim Renderer in OpenGL

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Gemini 3.7 Flash	0.0116	$11.28	68.7M	237k	4h 7m	304	2
2	Gemini 3.7 Flash	0.0064	$35.75	195.3M	360k	8h 28m	555	1
3	Gemini 3.7 Flash	0.0054	$6.52	51.3M	224k	2h 40m	274	2
4	Gemini 3.7 Flash	0.0020	$17.05	105.9M	300k	5h 32m	381	1
5	Gemini 3.7 Flash	0.0009	$11.28	77.4M	230k	5h 1m	369	3

Optimizer Design

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Gemini 3.7 Flash	0.1194	$22.13	241.0M	134k	10h 36m	834	2
2	Gemini 3.7 Flash	0.0537	$10.37	85.1M	122k	13h 22m	435	2
3	Gemini 3.7 Flash	0.0405	$25.34	217.9M	182k	10h 29m	744	2
4	Gemini 3.7 Flash	0.0000	$28.88	259.5M	185k	18h 35m	908	1
5	Gemini 3.7 Flash	0.0000	$12.41	47.0M	144k	13h 48m	297	2

PostgreSQL 18 on SQLite

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Gemini 3.7 Flash	0.2582	$124.97	1333.8M	2.5M	11h 51m	3014	37
2	Gemini 3.7 Flash	0.2418	$144.25	1592.6M	2.6M	19h 46m	3450	39
3	Gemini 3.7 Flash	0.2253	$193.89	2112.9M	3.0M	19h 55m	4680	16
4	Gemini 3.7 Flash	0.1758	$159.99	1689.8M	2.8M	19h 49m	3751	13
5	Gemini 3.7 Flash	0.0495	$17.96	182.1M	620k	1h 34m	402	2

Quantum ESPRESSO pw.x in Rust

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Gemini 3.7 Flash	0.1545	$46.71	314.3M	747k	10h 28m	740	2
2	Gemini 3.7 Flash	0.0976	$71.79	423.8M	777k	17h 50m	948	3
3	Gemini 3.7 Flash	0.0650	$36.49	288.8M	712k	6h 10m	725	2
4	Gemini 3.7 Flash	0.0000	$51.00	393.0M	804k	11h 3m	1002	4
5	Gemini 3.7 Flash	0.0000	$24.96	238.1M	690k	3h 15m	539	2

Qubit Routing

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Gemini 3.7 Flash	0.7591	$2.33	9.2M	139k	1h 43m	99	2
2	Gemini 3.7 Flash	0.6672	$3.04	15.3M	126k	1h 37m	140	1
3	Gemini 3.7 Flash	0.6446	$3.44	16.9M	214k	2h 28m	151	2
4	Gemini 3.7 Flash	0.6074	$16.26	72.9M	429k	6h 43m	280	5
5	Gemini 3.7 Flash	0.5912	$5.63	44.5M	188k	5h 26m	270	1

Reconnaissance Blind Chess Recovery

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Gemini 3.7 Flash	0.4250	$12.09	51.0M	293k	4h 11m	266	3
2	Gemini 3.7 Flash	0.3000	$23.11	214.3M	578k	5h 22m	524	1
3	Gemini 3.7 Flash	0.2393	$20.54	111.4M	562k	4h 43m	327	1
4	Gemini 3.7 Flash	0.0536	$30.51	186.7M	1.3M	5h 45m	513	2
5	Gemini 3.7 Flash	0.0429	$30.88	192.0M	568k	8h 45m	506	2

Fitness-Recap Video in Remotion

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Gemini 3.7 Flash	0.0129	$13.97	114.7M	211k	4h 23m	382	2
2	Gemini 3.7 Flash	0.0091	$2.94	21.2M	86k	1h 20m	169	2
3	Gemini 3.7 Flash	0.0089	$10.44	67.7M	164k	2h 48m	328	4
4	Gemini 3.7 Flash	0.0080	$7.68	61.7M	148k	2h 7m	300	1
5	Gemini 3.7 Flash	0.0060	$4.67	30.1M	112k	2h 7m	222	1

Snooker Prediction

task page →5 runs · indexed 2026-09-02

#	Model	Cost	In tok	Out tok	Time	Steps	Submits
1	Gemini 3.7 Flash	$7.82	66.5M	374k	1h 5m	279	1
2	Gemini 3.7 Flash	$8.33	59.1M	343k	2h 40m	282	3
3	Gemini 3.7 Flash	$5.61	40.2M	288k	59m	224	2
4	Gemini 3.7 Flash	$6.64	48.6M	323k	1h 55m	247	1
5	Gemini 3.7 Flash	$7.40	56.9M	413k	1h 21m	266	2

SPICE Circuit Simulator in Rust

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Gemini 3.7 Flash	0.6263	$99.31	1037.7M	2.0M	6h 16m	2577	4
2	Gemini 3.7 Flash	0.5253	$121.42	1230.5M	2.0M	12h 5m	2949	3
3	Gemini 3.7 Flash	0.4747	$76.85	790.2M	1.6M	4h 46m	2017	2
4	Gemini 3.7 Flash	0.4444	$80.51	766.1M	1.3M	6h 23m	1793	7
5	Gemini 3.7 Flash	0.3737	$126.28	1246.3M	2.0M	12h 6m	2921	2

Synthetic Music Diarization

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Gemini 3.7 Flash	0.4776	$32.86	129.1M	265k	12h 34m	438	2
2	Gemini 3.7 Flash	0.4664	$11.27	42.4M	155k	7h 29m	254	2
3	Gemini 3.7 Flash	0.4422	$14.36	133.2M	250k	5h 49m	389	2
4	Gemini 3.7 Flash	0.4264	$42.51	256.4M	394k	13h 10m	681	9
5	Gemini 3.7 Flash	0.4000	$15.67	83.4M	201k	4h 25m	322	1

Vision-only TORCS Racing Bot

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Gemini 3.7 Flash	0.0235	$5.06	24.8M	145k	2h 40m	197	1
2	Gemini 3.7 Flash	0.0127	$18.66	92.0M	417k	5h 34m	318	2
3	Gemini 3.7 Flash	0.0121	$11.51	63.9M	262k	4h 49m	310	2
4	Gemini 3.7 Flash	0.0118	$6.81	26.5M	177k	3h 38m	183	2
5	Gemini 3.7 Flash	0.0041	$7.39	32.0M	185k	3h 17m	213	1

Verilog Simulator in Swift

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Gemini 3.7 Flash	0.8305	$103.65	1018.6M	2.0M	19h 58m	2418	1
2	Gemini 3.7 Flash	0.8288	$74.39	745.3M	2.1M	17h 44m	1828	1
3	Gemini 3.7 Flash	0.8202	$75.29	725.4M	1.4M	19h 4m	1651	2
4	Gemini 3.7 Flash	0.5976	$41.43	380.1M	776k	20h 1m	944	4
5	Gemini 3.7 Flash	0.3545	$53.22	180.3M	689k	19h 59m	574	1