Tasks Traces V1 Blog Changelog Contribute GitHub

Traces

Every run: the agent's full trajectory, its submissions over the 20-hour budget, and the final score.

Kolmogorov Audio Compression

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Opus 5.5	0.0856	$64.20	240.0M	620k	16h 10m	530	20
2	Claude Opus 5.5	0.0830	$83.18	303.7M	858k	18h 27m	759	19
3	Claude Opus 5.5	0.0789	$67.50	264.9M	554k	16h 28m	614	17
4	Claude Opus 5.5	0.0783	$41.46	140.9M	515k	18h 1m	396	17
5	Claude Opus 5.5	0.0694	$57.79	216.0M	560k	18h 11m	500	14

Astronomy Toolkit

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Opus 5.5	0.9999	$74.48	265.3M	811k	19h 2m	653	24
2	Claude Opus 5.5	0.9931	$69.52	233.4M	859k	17h 56m	587	25
3	Claude Opus 5.5	0.9907	$77.01	274.8M	827k	18h 8m	669	37
4	Claude Opus 5.5	0.8205	$49.70	176.5M	532k	9h 15m	362	10
5	Claude Opus 5.5	0.7024	$55.16	202.1M	545k	11h 37m	413	16

Cranelift Codegen Optimization

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Opus 5.5	0.3651	$135.88	506.9M	1.2M	19h 1m	1217	34
2	Claude Opus 5.5	0.2984	$145.49	543.9M	1.3M	18h 43m	1227	39
3	Claude Opus 5.5	0.2828	$138.56	513.8M	1.3M	19h 19m	1211	30
4	Claude Opus 5.5	0.2500	$129.78	475.1M	1.3M	18h 48m	1162	27
5	Claude Opus 5.5	0.2063	$105.12	402.9M	850k	17h 7m	905	27

Dart Style in Haskell

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Opus 5.5	0.9894	$168.41	622.4M	1.6M	10h 59m	1389	39
2	Claude Opus 5.5	0.9885	$192.52	720.9M	1.8M	13h 2m	1670	53
3	Claude Opus 5.5	0.9853	$159.11	577.5M	1.6M	9h 35m	1338	18
4	Claude Opus 5.5	0.9829	$190.35	691.9M	2.0M	9h 34m	1602	24
5	Claude Opus 5.5	0.9733	$318.71	1272.8M	2.4M	16h 51m	2868	99

FFmpeg libswscale Optimization

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Opus 5.5	0.8444	$169.75	555.3M	2.2M	12h 14m	1230	42
2	Claude Opus 5.5	0.1224	$83.47	264.7M	1.1M	5h 11m	549	14
3	Claude Opus 5.5	0.1185	$175.12	575.5M	2.3M	11h 51m	1203	46
4	Claude Opus 5.5	0.0441	$106.98	317.9M	1.6M	6h 52m	693	18
5	Claude Opus 5.5	0.0000	$168.91	529.9M	2.4M	15h 41m	1150	37

Crash-Proof Flash Filesystem

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Opus 5.5	0.9981	$27.48	78.1M	449k	2h 59m	213	5
2	Claude Opus 5.5	0.9981	$26.95	76.7M	440k	2h 11m	202	7
3	Claude Opus 5.5	0.9978	$40.02	124.8M	561k	3h 28m	245	7
4	Claude Opus 5.5	0.9978	$56.18	167.0M	853k	5h 25m	375	8
5	Claude Opus 5.5	0.9978	$31.57	92.4M	498k	2h 55m	217	4

FrogsGame Post-Training

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Opus 5.5	0.9380	$18.10	54.0M	277k	17h 56m	203	3
2	Claude Opus 5.5	0.9340	$25.67	82.0M	341k	18h 34m	248	6
3	Claude Opus 5.5	0.9320	$21.18	64.6M	311k	18h 52m	228	5
4	Claude Opus 5.5	0.9060	$27.72	93.7M	337k	17h 43m	292	4
5	Claude Opus 5.5	0.8240	$28.19	95.1M	341k	18h 8m	296	5

Stepper Music Sequencer GBA

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Opus 5.5	0.9917	$247.64	942.3M	2.2M	18h 17m	2063	31
2	Claude Opus 5.5	0.9883	$203.46	765.3M	1.9M	19h 3m	1757	33
3	Claude Opus 5.5	0.9874	$283.79	1107.1M	2.3M	19h 19m	2356	8
4	Claude Opus 5.5	0.9866	$313.81	1197.7M	2.7M	18h 20m	2708	44
5	Claude Opus 5.5	0.9844	$258.85	972.6M	2.4M	18h 43m	2211	22

Git to Zig

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Opus 5.5	0.6812	$313.47	1048.3M	4.0M	19h 32m	2436	25
2	Claude Opus 5.5	0.6644	$241.59	773.6M	3.3M	19h 40m	1843	33
3	Claude Opus 5.5	0.6378	$222.91	717.3M	3.1M	19h 51m	1607	19
4	Claude Opus 5.5	0.6246	$242.07	814.4M	3.0M	19h 50m	1773	32
5	Claude Opus 5.5	0.5824	$188.65	606.3M	2.6M	19h 43m	1437	34

Granite Mamba2 Inference Optimization

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Opus 5.5	0.8846	$214.03	690.9M	2.9M	15h 36m	1506	34
2	Claude Opus 5.5	0.8817	$132.91	426.3M	1.8M	6h 35m	922	25
3	Claude Opus 5.5	0.8769	$159.17	476.8M	2.5M	11h 3m	1094	29
4	Claude Opus 5.5	0.7510	$78.62	226.6M	1.3M	6h 44m	478	12
5	Claude Opus 5.5	0.5826	$111.15	316.0M	1.9M	10h 19m	731	29

Higgs Uncertainty Inference

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Opus 5.5	1.0000	$25.57	70.3M	442k	6h 17m	211	11
2	Claude Opus 5.5	0.9897	$30.03	90.4M	459k	7h 8m	258	13
3	Claude Opus 5.5	0.9370	$28.53	84.2M	453k	7h 52m	260	12
4	Claude Opus 5.5	0.2834	$29.08	88.4M	439k	9h 41m	265	10
5	Claude Opus 5.5	0.2805	$36.69	117.7M	504k	7h 47m	310	12

SGLang Inference System Optimization

task page →5 runs · indexed 2026-09-22

#	Model	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Opus 5.5	$159.94	559.4M	1.8M	19h 25m	1329	26
2	Claude Opus 5.5	$183.76	644.5M	2.0M	19h 7m	1476	23
3	Claude Opus 5.5	$185.38	644.4M	2.1M	18h 30m	1447	29
4	Claude Opus 5.5	$134.68	471.5M	1.5M	18h 23m	1059	21
5	Claude Opus 5.5	$150.29	514.2M	1.7M	17h 0m	1206	15

Lean 4 Kernel Type Checker in Pascal

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Opus 5.5	1.0000	$41.28	133.2M	560k	3h 31m	263	13
2	Claude Opus 5.5	1.0000	$71.71	241.5M	889k	6h 55m	573	21
3	Claude Opus 5.5	1.0000	$35.94	109.8M	528k	2h 59m	232	8
4	Claude Opus 5.5	1.0000	$31.11	88.5M	510k	2h 27m	202	4
5	Claude Opus 5.5	1.0000	$37.66	116.3M	544k	3h 20m	243	8

libexpat Optimization

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Opus 5.5	0.8047	$233.16	850.8M	2.4M	15h 14m	1758	71
2	Claude Opus 5.5	0.7133	$179.28	606.1M	2.3M	14h 58m	1384	20
3	Claude Opus 5.5	0.6801	$255.81	936.5M	2.6M	17h 42m	2064	87
4	Claude Opus 5.5	0.4775	$212.18	685.7M	2.9M	15h 26m	1659	63
5	Claude Opus 5.5	0.4760	$260.62	918.4M	2.9M	17h 55m	2043	64

Lua Native Compiler

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Opus 5.5	0.7885	$68.38	213.8M	1.0M	9h 38m	482	11
2	Claude Opus 5.5	0.6667	$78.97	234.5M	1.3M	9h 24m	649	15
3	Claude Opus 5.5	0.6447	$81.59	228.2M	1.4M	10h 19m	607	8
4	Claude Opus 5.5	0.0000	$124.07	397.7M	1.8M	12h 35m	893	28
5	Claude Opus 5.5	0.0000	$93.26	296.3M	1.3M	8h 6m	673	12

Machine-Learned Interatomic Potential

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Opus 5.5	0.9986	$19.92	56.8M	329k	17h 53m	217	9
2	Claude Opus 5.5	0.9657	$21.32	60.2M	359k	15h 52m	219	7
3	Claude Opus 5.5	0.9563	$17.15	48.3M	286k	18h 37m	196	8
4	Claude Opus 5.5	0.9209	$19.00	56.2M	297k	18h 34m	231	12
5	Claude Opus 5.5	0.4697	$23.43	66.1M	395k	18h 40m	232	8

Medium-Range Weather Forecast

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Opus 5.5	0.6553	$21.31	71.4M	261k	19h 12m	313	10
2	Claude Opus 5.5	0.6405	$16.76	50.5M	249k	15h 23m	250	11
3	Claude Opus 5.5	0.6368	$21.64	71.5M	273k	18h 3m	288	10
4	Claude Opus 5.5	0.6263	$14.11	42.0M	214k	18h 26m	225	7
5	Claude Opus 5.5	0.6141	$21.37	70.5M	271k	17h 20m	298	12

MEG Speech Decoding

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Opus 5.5	0.3727	$24.06	81.6M	282k	17h 26m	303	14
2	Claude Opus 5.5	0.3678	$32.59	115.1M	354k	17h 32m	353	13
3	Claude Opus 5.5	0.3676	$34.48	123.3M	365k	14h 56m	393	16
4	Claude Opus 5.5	0.2944	$25.47	87.0M	298k	19h 1m	298	14
5	Claude Opus 5.5	0.1928	$24.23	78.6M	314k	16h 53m	281	9

Wan 2.1 on MAX/Mojo

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Opus 5.5	0.9802	$29.40	86.6M	447k	5h 37m	213	11
2	Claude Opus 5.5	0.9629	$25.12	76.6M	348k	3h 15m	207	9
3	Claude Opus 5.5	0.9582	$32.70	105.1M	419k	5h 38m	261	4
4	Claude Opus 5.5	0.9506	$43.10	137.3M	584k	6h 47m	287	8
5	Claude Opus 5.5	0.9418	$33.25	107.7M	418k	4h 32m	251	4

MS/MS De Novo Generation

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Opus 5.5	0.1412	$32.15	111.5M	374k	16h 42m	367	12
2	Claude Opus 5.5	0.1378	$36.38	124.0M	438k	17h 32m	359	11
3	Claude Opus 5.5	0.0000	$41.05	143.9M	474k	16h 43m	419	14
4	Claude Opus 5.5	0.0000	$25.87	82.9M	352k	17h 21m	296	8
5	Claude Opus 5.5	0.0000	$33.84	113.6M	428k	17h 31m	346	9

Multi-GPU Efficient Finetuning

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Opus 5.5	0.4286	$18.55	51.6M	316k	17h 25m	201	4
2	Claude Opus 5.5	0.3333	$13.03	35.0M	224k	18h 31m	179	5
3	Claude Opus 5.5	0.3095	$18.49	50.1M	325k	18h 23m	192	5
4	Claude Opus 5.5	0.2857	$21.94	61.0M	375k	16h 29m	207	3
5	Claude Opus 5.5	0.2619	$17.33	45.0M	317k	18h 13m	178	4

Notebook Compression

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Opus 5.5	0.6404	$76.14	300.9M	607k	18h 32m	620	33
2	Claude Opus 5.5	0.6166	$96.89	380.6M	802k	18h 37m	872	33
3	Claude Opus 5.5	0.5926	$80.88	317.0M	668k	19h 35m	606	29
4	Claude Opus 5.5	0.5584	$79.55	318.4M	608k	18h 25m	649	27
5	Claude Opus 5.5	0.5566	$69.33	267.9M	597k	18h 1m	557	28

Flight-Sim Renderer in OpenGL

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Opus 5.5	0.9097	$156.04	545.9M	1.3M	19h 52m	1288	19
2	Claude Opus 5.5	0.8821	$144.22	511.3M	1.0M	18h 11m	1074	26
3	Claude Opus 5.5	0.8618	$140.79	553.3M	1.1M	19h 44m	1191	4
4	Claude Opus 5.5	0.8540	$182.36	671.0M	1.3M	19h 5m	1637	23
5	Claude Opus 5.5	0.6632	$158.77	603.5M	1.1M	18h 8m	1203	7

Optimizer Design

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Opus 5.5	0.3908	$28.07	87.7M	400k	17h 51m	286	12
2	Claude Opus 5.5	0.2824	$26.98	82.6M	395k	14h 4m	282	6
3	Claude Opus 5.5	0.2715	$26.87	83.2M	387k	12h 26m	281	8
4	Claude Opus 5.5	0.2621	$24.28	68.6M	403k	13h 10m	230	4
5	Claude Opus 5.5	0.1731	$23.28	67.5M	372k	18h 2m	238	7

PostgreSQL 18 on SQLite

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Opus 5.5	0.5934	$433.39	1590.6M	4.3M	19h 2m	3640	18
2	Claude Opus 5.5	0.5659	$363.10	1344.0M	3.5M	19h 34m	3174	20
3	Claude Opus 5.5	0.4615	$283.28	1029.0M	2.9M	18h 59m	2312	13
4	Claude Opus 5.5	0.3352	$189.85	651.3M	2.3M	19h 20m	1478	8
5	Claude Opus 5.5	0.2967	$184.14	630.5M	2.2M	19h 31m	1487	10

Quantum ESPRESSO pw.x in Rust

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Opus 5.5	0.9756	$115.01	354.9M	1.7M	9h 7m	784	13
2	Claude Opus 5.5	0.9756	$120.52	382.4M	1.7M	11h 39m	867	25
3	Claude Opus 5.5	0.9594	$109.78	339.6M	1.6M	7h 33m	786	12
4	Claude Opus 5.5	0.9512	$58.52	152.7M	1.1M	4h 44m	350	6
5	Claude Opus 5.5	0.9350	$70.36	204.4M	1.1M	5h 26m	460	11

Qubit Routing

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Opus 5.5	0.9241	$44.40	160.3M	462k	12h 19m	408	3
2	Claude Opus 5.5	0.9229	$49.89	183.1M	502k	16h 23m	427	10
3	Claude Opus 5.5	0.9138	$40.04	137.9M	476k	14h 14m	374	13
4	Claude Opus 5.5	0.9052	$63.24	243.8M	546k	14h 55m	528	21
5	Claude Opus 5.5	0.8783	$46.14	171.7M	451k	17h 18m	447	9

Reconnaissance Blind Chess Recovery

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Opus 5.5	0.9929	$30.52	100.2M	393k	11h 17m	272	6
2	Claude Opus 5.5	0.9786	$45.44	160.9M	500k	13h 11m	377	11
3	Claude Opus 5.5	0.9786	$34.33	116.2M	417k	11h 0m	318	8
4	Claude Opus 5.5	0.9750	$38.61	139.5M	396k	12h 25m	373	6
5	Claude Opus 5.5	0.9571	$39.47	142.7M	408k	13h 40m	381	7

Fitness-Recap Video in Remotion

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Opus 5.5	0.3605	$270.94	1051.2M	2.1M	19h 10m	2403	12
2	Claude Opus 5.5	0.2926	$307.93	1148.9M	1.9M	19h 20m	2553	28
3	Claude Opus 5.5	0.2905	$179.59	686.0M	1.3M	19h 22m	1645	21
4	Claude Opus 5.5	0.2637	$181.24	683.8M	1.4M	19h 15m	1667	19
5	Claude Opus 5.5	0.2443	$84.28	312.7M	521k	9h 32m	660	10

Snooker Prediction

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Opus 5.5	0.6243	$92.07	333.9M	907k	9h 5m	810	28
2	Claude Opus 5.5	0.6219	$106.45	401.8M	928k	15h 53m	977	39
3	Claude Opus 5.5	0.6162	$99.16	365.9M	934k	14h 38m	876	24
4	Claude Opus 5.5	0.5538	$119.85	454.9M	1.0M	14h 11m	1016	41
5	Claude Opus 5.5	0.5128	$94.50	346.2M	909k	11h 7m	864	12

SPICE Circuit Simulator in Rust

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Opus 5.5	0.9293	$97.56	280.4M	1.6M	6h 22m	663	14
2	Claude Opus 5.5	0.8990	$72.19	195.2M	1.3M	4h 28m	434	13
3	Claude Opus 5.5	0.8687	$65.56	170.6M	1.2M	3h 47m	399	6
4	Claude Opus 5.5	0.8687	$104.48	301.0M	1.7M	5h 14m	731	11
5	Claude Opus 5.5	0.8687	$69.51	187.1M	1.2M	4h 12m	427	15

Synthetic Music Diarization

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Opus 5.5	0.5460	$17.72	55.1M	249k	18h 32m	256	12
2	Claude Opus 5.5	0.5369	$24.84	83.3M	302k	17h 50m	310	7
3	Claude Opus 5.5	0.5369	$16.97	51.1M	254k	18h 25m	241	10
4	Claude Opus 5.5	0.5219	$17.38	51.6M	265k	17h 34m	243	8
5	Claude Opus 5.5	0.4919	$19.72	61.9M	269k	17h 35m	246	9

Vision-only TORCS Racing Bot

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Opus 5.5	0.5284	$75.38	280.1M	713k	17h 50m	688	10
2	Claude Opus 5.5	0.5076	$73.95	265.4M	548k	18h 0m	556	10
3	Claude Opus 5.5	0.2616	$101.33	304.1M	515k	18h 58m	609	10
4	Claude Opus 5.5	0.0000†	$60.58	240.2M	450k	18h 1m	571	11
5	Claude Opus 5.5	0.0000†	$51.68	197.1M	439k	16h 52m	474	7

† score adjusted by manual review; the trace's Judge tab carries the measured score and the rationale.

Verilog Simulator in Swift

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Opus 5.5	0.9521	$121.24	461.6M	1.1M	7h 20m	989	14
2	Claude Opus 5.5	0.9486	$185.48	723.5M	1.5M	11h 40m	1695	26
3	Claude Opus 5.5	0.9452	$119.27	459.4M	1.0M	7h 41m	1025	14
4	Claude Opus 5.5	0.9435	$112.90	428.3M	1.0M	8h 8m	989	11
5	Claude Opus 5.5	0.9401	$117.75	445.5M	1.1M	6h 29m	1024	13