Tasks Traces V1 Blog Changelog Contribute GitHub

Traces

Every run: the agent's full trajectory, its submissions over the 20-hour budget, and the final score.

Astronomy Toolkit

task page →5 runs · indexed 2026-11-12

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Fable 5.2	0.9998	$142.16	310.5M	1.3M	19h 26m	709	29
2	Claude Fable 5.2	0.9943	$111.66	244.9M	979k	18h 3m	541	13
3	Claude Fable 5.2	0.9929	$98.04	200.9M	920k	14h 32m	477	12
4	Claude Fable 5.2	0.9897	$146.76	333.0M	1.2M	18h 53m	739	23
5	Claude Fable 5.2	0.8999	$147.63	317.4M	1.3M	18h 33m	708	24

Cranelift Codegen Optimization

task page →5 runs · indexed 2026-11-12

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Fable 5.2	0.7217	$180.42	425.9M	1.4M	19h 13m	949	24
2	Claude Fable 5.2	0.6592	$232.35	544.2M	1.9M	18h 43m	1235	19
3	Claude Fable 5.2	0.5449	$206.00	481.4M	1.7M	19h 20m	1145	32
4	Claude Fable 5.2	0.4858	$225.46	503.9M	1.9M	18h 56m	1168	47
5	Claude Fable 5.2	0.4696	$217.08	485.4M	1.8M	18h 47m	1186	23

Crash-Proof Flash Filesystem

task page →5 runs · indexed 2026-11-12

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Fable 5.2	1.0000	$102.03	180.4M	1.1M	9h 41m	403	8
2	Claude Fable 5.2	0.9997	$143.04	261.4M	1.5M	12h 31m	578	7
3	Claude Fable 5.2	0.9997	$97.50	151.2M	1.1M	6h 30m	347	7
4	Claude Fable 5.2	0.9981	$204.11	374.7M	2.1M	18h 36m	825	19
5	Claude Fable 5.2	0.9978	$122.61	241.6M	1.2M	11h 7m	455	10

Dart Style in Haskell

task page →5 runs · indexed 2026-11-12

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Fable 5.2	0.9966	$144.53	219.0M	1.7M	8h 12m	547	11
2	Claude Fable 5.2	0.9957	$220.99	435.0M	2.2M	13h 21m	971	25
3	Claude Fable 5.2	0.9954	$190.90	402.0M	1.8M	13h 19m	851	21
4	Claude Fable 5.2	0.9945	$115.39	201.6M	1.3M	6h 26m	410	12
5	Claude Fable 5.2	0.9940	$215.82	400.4M	2.3M	16h 13m	965	20

FFmpeg libswscale Optimization

task page →5 runs · indexed 2026-11-12

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Fable 5.2	0.2852	$305.94	529.6M	3.3M	18h 43m	1135	31
2	Claude Fable 5.2	0.2270	$288.25	422.3M	3.4M	16h 7m	903	27
3	Claude Fable 5.2	0.1716	$310.15	442.3M	3.7M	17h 31m	915	34
4	Claude Fable 5.2	0.1657	$347.06	530.1M	4.0M	18h 49m	1087	36
5	Claude Fable 5.2	0.0762	$280.38	458.2M	3.1M	15h 20m	961	31

Fitness-Recap Video in Remotion

task page →5 runs · indexed 2026-11-12

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Fable 5.2	0.7416	$309.24	739.9M	2.4M	19h 32m	1674	13
2	Claude Fable 5.2	0.6276	$306.52	725.5M	2.4M	19h 20m	1583	14
3	Claude Fable 5.2	0.5288	$288.29	735.8M	2.1M	19h 28m	1571	12
4	Claude Fable 5.2	0.4662	$259.70	566.4M	1.8M	19h 38m	1198	19
5	Claude Fable 5.2	0.4590	$216.36	553.2M	1.6M	19h 17m	1305	9

Flight-Sim Renderer in OpenGL

task page →5 runs · indexed 2026-11-12

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Fable 5.2	0.9819	$265.37	640.2M	2.1M	19h 34m	1368	26
2	Claude Fable 5.2	0.9814	$281.06	633.3M	2.2M	19h 32m	1357	20
3	Claude Fable 5.2	0.9807	$275.12	628.6M	2.2M	19h 44m	1453	28
4	Claude Fable 5.2	0.9799	$283.79	585.5M	2.6M	19h 7m	1354	24
5	Claude Fable 5.2	0.9763	$312.67	715.7M	2.6M	18h 55m	1605	13

FrogsGame Post-Training

task page →5 runs · indexed 2026-11-12

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Fable 5.2	0.9820	$57.65	128.5M	519k	19h 3m	275	6
2	Claude Fable 5.2	0.9560	$46.32	88.3M	456k	18h 11m	200	5
3	Claude Fable 5.2	0.9400	$71.35	182.2M	560k	18h 31m	352	6
4	Claude Fable 5.2	0.5780	$80.26	171.7M	762k	19h 3m	393	7
5	Claude Fable 5.2	0.3460	$66.01	152.4M	584k	19h 16m	330	6

Git to Zig

task page →5 runs · indexed 2026-11-12

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Fable 5.2	0.8516	$404.66	771.5M	4.1M	19h 47m	1802	33
2	Claude Fable 5.2	0.8312	$363.85	629.2M	4.0M	19h 53m	1543	21
3	Claude Fable 5.2	0.8296	$377.19	630.2M	4.3M	19h 59m	1646	27
4	Claude Fable 5.2	0.8166	$351.38	574.4M	4.0M	19h 52m	1291	25
5	Claude Fable 5.2	0.8155	$358.00	583.1M	4.1M	19h 46m	1394	29

Granite Mamba2 Inference Optimization

task page →5 runs · indexed 2026-11-12

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Fable 5.2	0.8412	$230.69	401.9M	2.5M	16h 39m	876	24
2	Claude Fable 5.2	0.7934	$261.26	441.0M	2.9M	15h 5m	963	30
3	Claude Fable 5.2	0.7869	$218.28	379.0M	2.4M	15h 35m	849	25
4	Claude Fable 5.2	0.5421	$262.19	463.3M	2.9M	19h 13m	1010	34
5	Claude Fable 5.2	0.2543	$258.70	432.2M	2.9M	14h 5m	965	22

Higgs Uncertainty Inference

task page →5 runs · indexed 2026-11-12

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Fable 5.2	1.0000	$48.42	94.2M	487k	9h 35m	236	13
2	Claude Fable 5.2	1.0000	$53.15	100.2M	543k	8h 51m	227	13
3	Claude Fable 5.2	1.0000	$50.38	89.3M	539k	7h 38m	216	10
4	Claude Fable 5.2	0.9897	$39.57	58.2M	466k	4h 56m	169	11
5	Claude Fable 5.2	0.9122	$49.39	88.4M	516k	6h 50m	207	14

Kolmogorov Audio Compression

task page →5 runs · indexed 2026-11-12

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Fable 5.2	1.0000	$213.18	535.5M	1.7M	19h 2m	1123	16
2	Claude Fable 5.2	0.2914	$182.43	392.4M	1.7M	19h 29m	857	13
3	Claude Fable 5.2	0.1861	$226.44	474.0M	2.1M	19h 33m	1042	10
4	Claude Fable 5.2	0.1139	$172.73	346.6M	1.6M	19h 28m	723	12
5	Claude Fable 5.2	0.0797	$175.36	368.8M	1.6M	18h 52m	810	10

Lean 4 Kernel Type Checker in Pascal

task page →5 runs · indexed 2026-11-12

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Fable 5.2	1.0000	$127.41	268.5M	1.2M	7h 53m	507	18
2	Claude Fable 5.2	1.0000	$109.57	216.4M	1.1M	9h 33m	476	12
3	Claude Fable 5.2	1.0000	$221.66	487.8M	2.0M	18h 40m	1067	18
4	Claude Fable 5.2	1.0000	$123.89	256.5M	1.2M	7h 30m	528	17
5	Claude Fable 5.2	1.0000	$107.08	201.3M	1.1M	7h 0m	430	14

libexpat Optimization

task page →5 runs · indexed 2026-11-12

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Fable 5.2	0.9271	$171.43	330.2M	1.8M	14h 41m	715	17
2	Claude Fable 5.2	0.9261	$241.34	483.6M	2.4M	18h 51m	1012	34
3	Claude Fable 5.2	0.8955	$238.54	490.8M	2.3M	19h 13m	1051	15
4	Claude Fable 5.2	0.8806	$170.09	313.5M	1.8M	17h 49m	715	14
5	Claude Fable 5.2	0.8747	$121.34	211.4M	1.3M	8h 12m	393	9

Lua Native Compiler

task page →5 runs · indexed 2026-11-12

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Fable 5.2	1.0000	$101.57	195.1M	1.1M	8h 49m	467	12
2	Claude Fable 5.2	1.0000	$130.60	279.8M	1.3M	16h 23m	637	12
3	Claude Fable 5.2	0.0000	$111.24	221.4M	1.1M	11h 35m	517	13
4	Claude Fable 5.2	0.0000	$115.34	226.5M	1.2M	10h 6m	521	14
5	Claude Fable 5.2	0.0000	$125.65	239.6M	1.3M	12h 44m	534	11

Machine-Learned Interatomic Potential

task page →5 runs · indexed 2026-11-12

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Fable 5.2	0.9938	$65.53	152.2M	577k	15h 54m	346	11
2	Claude Fable 5.2	0.9788	$58.75	130.4M	532k	18h 35m	299	9
3	Claude Fable 5.2	0.9768	$69.53	165.8M	595k	18h 16m	329	13
4	Claude Fable 5.2	0.9767	$49.11	104.4M	465k	17h 37m	276	15
5	Claude Fable 5.2	0.9656	$50.38	97.5M	507k	18h 6m	257	9

Medium-Range Weather Forecast

task page →5 runs · indexed 2026-11-12

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Fable 5.2	0.6561	$53.93	129.3M	460k	17h 50m	346	12
2	Claude Fable 5.2	0.6560	$54.21	134.8M	440k	18h 34m	356	14
3	Claude Fable 5.2	0.6468	$49.98	113.7M	449k	18h 16m	312	10
4	Claude Fable 5.2	0.6334	$45.49	103.3M	401k	18h 34m	307	10
5	Claude Fable 5.2	0.6256	$49.78	105.9M	473k	15h 23m	304	9

MEG Speech Decoding

task page →5 runs · indexed 2026-11-12

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Fable 5.2	0.5302	$53.79	128.1M	454k	17h 35m	327	14
2	Claude Fable 5.2	0.4783	$52.33	117.3M	461k	17h 4m	298	9
3	Claude Fable 5.2	0.4777	$51.29	120.8M	428k	17h 10m	306	9
4	Claude Fable 5.2	0.4661	$56.98	137.0M	480k	17h 54m	343	10
5	Claude Fable 5.2	0.4547	$54.23	137.4M	427k	18h 21m	350	11

MS/MS De Novo Generation

task page →5 runs · indexed 2026-11-12

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Fable 5.2	0.1784	$57.74	122.2M	559k	18h 22m	284	13
2	Claude Fable 5.2	0.1405	$57.56	87.1M	522k	15h 55m	286	6
3	Claude Fable 5.2	0.1402	$56.62	87.8M	512k	18h 29m	299	9
4	Claude Fable 5.2	0.1323	$53.44	94.1M	500k	15h 56m	322	7
5	Claude Fable 5.2	0.0000	$76.59	123.0M	775k	18h 35m	393	9

Multi-GPU Efficient Finetuning

task page →5 runs · indexed 2026-11-12

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Fable 5.2	0.3095	$35.22	64.9M	361k	18h 19m	207	7
2	Claude Fable 5.2	0.2857	$34.18	59.5M	356k	18h 11m	193	5
3	Claude Fable 5.2	0.2857	$39.65	73.9M	400k	18h 44m	206	4
4	Claude Fable 5.2	0.2619	$33.11	64.1M	309k	12h 58m	195	5
5	Claude Fable 5.2	0.2381	$43.43	92.7M	406k	17h 26m	266	4

Notebook Compression

task page →5 runs · indexed 2026-11-12

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Fable 5.2	0.7775	$141.07	359.8M	1.1M	18h 8m	749	33
2	Claude Fable 5.2	0.7528	$124.14	288.1M	1.1M	18h 56m	627	21
3	Claude Fable 5.2	0.7161	$117.77	257.1M	1.1M	19h 6m	592	18
4	Claude Fable 5.2	0.6991	$137.78	324.4M	1.2M	19h 0m	665	24
5	Claude Fable 5.2	0.6658	$148.00	353.2M	1.3M	17h 15m	694	29

Optimizer Design

task page →5 runs · indexed 2026-11-12

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Fable 5.2	0.3709	$52.44	112.4M	486k	18h 31m	288	9
2	Claude Fable 5.2	0.3657	$52.74	108.5M	513k	17h 32m	272	11
3	Claude Fable 5.2	0.3584	$47.23	89.6M	475k	16h 51m	250	8
4	Claude Fable 5.2	0.3582	$64.80	134.7M	624k	17h 6m	285	9
5	Claude Fable 5.2	0.2610	$53.90	114.8M	495k	17h 28m	272	8

PostgreSQL 18 on SQLite

task page →5 runs · indexed 2026-11-12

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Fable 5.2	0.6758	$448.97	976.8M	4.0M	19h 14m	2287	32
2	Claude Fable 5.2	0.6484	$451.28	1038.5M	3.8M	18h 54m	2378	22
3	Claude Fable 5.2	0.6429	$421.90	880.3M	3.9M	19h 37m	2009	33
4	Claude Fable 5.2	0.6319	$428.33	939.4M	3.8M	19h 8m	2144	19
5	Claude Fable 5.2	0.6209	$417.65	865.6M	3.9M	18h 29m	1983	18

Quantum ESPRESSO pw.x in Rust

task page →5 runs · indexed 2026-11-12

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Fable 5.2	1.0000	$326.30	564.7M	3.5M	17h 40m	1275	29
2	Claude Fable 5.2	1.0000	$304.76	515.8M	3.3M	15h 33m	1121	30
3	Claude Fable 5.2	1.0000	$279.93	469.3M	3.1M	16h 38m	1016	28
4	Claude Fable 5.2	1.0000	$309.21	548.3M	3.3M	19h 10m	1215	31
5	Claude Fable 5.2	0.9919	$279.82	453.4M	3.1M	15h 1m	986	20

Qubit Routing

task page →5 runs · indexed 2026-11-12

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Fable 5.2	0.9250	$75.74	197.1M	585k	17h 37m	378	20
2	Claude Fable 5.2	0.9216	$78.38	195.8M	648k	15h 3m	370	8
3	Claude Fable 5.2	0.9145	$102.10	235.9M	881k	18h 17m	549	22
4	Claude Fable 5.2	0.9102	$69.12	173.4M	559k	16h 37m	359	14
5	Claude Fable 5.2	0.9092	$96.00	230.3M	812k	18h 45m	528	21

Reconnaissance Blind Chess Recovery

task page →5 runs · indexed 2026-11-12

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Fable 5.2	1.0000	$66.57	159.5M	558k	18h 26m	304	10
2	Claude Fable 5.2	0.9929	$77.55	170.6M	692k	18h 57m	387	8
3	Claude Fable 5.2	0.9821	$106.24	238.6M	930k	18h 4m	548	10
4	Claude Fable 5.2	0.9786	$92.65	215.0M	790k	18h 37m	499	12
5	Claude Fable 5.2	0.9714	$74.36	189.5M	583k	17h 4m	333	6

SGLang Inference System Optimization

task page →5 runs · indexed 2026-11-12

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Fable 5.2	0.5490	$221.84	459.2M	2.0M	16h 27m	1037	21
2	Claude Fable 5.2	0.0000	$212.79	433.2M	1.9M	16h 55m	938	22
3	Claude Fable 5.2	0.0000	$249.25	518.8M	2.2M	19h 8m	1188	22
4	Claude Fable 5.2	0.0000	$161.67	351.3M	1.4M	17h 54m	799	14
5	Claude Fable 5.2	0.0000	$160.33	338.6M	1.4M	16h 36m	783	12

Snooker Prediction

task page →5 runs · indexed 2026-11-12

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Fable 5.2	0.6322	$165.65	325.9M	1.5M	13h 20m	696	18
2	Claude Fable 5.2	0.6317	$241.89	459.7M	2.3M	18h 45m	1026	18
3	Claude Fable 5.2	0.6317	$155.06	278.7M	1.5M	11h 40m	647	14
4	Claude Fable 5.2	0.6312	$186.62	353.7M	1.6M	11h 56m	747	22
5	Claude Fable 5.2	0.6252	$272.73	439.2M	3.0M	18h 14m	892	23

SPICE Circuit Simulator in Rust

task page →5 runs · indexed 2026-11-12

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Fable 5.2	1.0000	$310.66	447.6M	3.7M	13h 35m	996	29
2	Claude Fable 5.2	1.0000	$213.81	316.3M	2.5M	8h 47m	677	22
3	Claude Fable 5.2	0.9798	$145.45	179.0M	1.8M	6h 8m	410	17
4	Claude Fable 5.2	0.9697	$322.39	487.7M	3.7M	15h 35m	1068	36
5	Claude Fable 5.2	0.9596	$155.89	181.0M	2.0M	6h 30m	395	10

Stepper Music Sequencer GBA

task page →5 runs · indexed 2026-11-12

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Fable 5.2	0.9999	$226.72	530.2M	1.9M	19h 44m	1233	22
2	Claude Fable 5.2	0.9994	$287.56	701.1M	2.4M	19h 28m	1469	31
3	Claude Fable 5.2	0.9988	$236.22	554.1M	2.0M	19h 43m	1284	28
4	Claude Fable 5.2	0.9973	$312.93	709.4M	2.7M	19h 18m	1588	20
5	Claude Fable 5.2	0.9821	$297.78	700.2M	2.5M	19h 36m	1563	35

Synthetic Music Diarization

task page →5 runs · indexed 2026-11-12

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Fable 5.2	0.5932	$55.55	130.2M	480k	18h 32m	316	8
2	Claude Fable 5.2	0.5606	$50.95	107.1M	482k	18h 34m	282	10
3	Claude Fable 5.2	0.5585	$44.69	94.9M	414k	18h 38m	270	7
4	Claude Fable 5.2	0.5457	$60.57	137.9M	536k	18h 54m	327	12
5	Claude Fable 5.2	0.5184	$61.68	128.4M	592k	19h 16m	286	12

Verilog Simulator in Swift

task page →5 runs · indexed 2026-11-12

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Fable 5.2	0.9760	$365.69	920.3M	3.0M	18h 34m	1971	40
2	Claude Fable 5.2	0.9726	$275.40	639.9M	2.4M	16h 2m	1372	27
3	Claude Fable 5.2	0.9658	$262.75	620.3M	2.3M	14h 33m	1382	19
4	Claude Fable 5.2	0.9658	$353.61	847.0M	3.0M	17h 40m	1842	40
5	Claude Fable 5.2	0.9623	$340.66	788.8M	3.0M	15h 57m	1695	26

Vision-only TORCS Racing Bot

task page →5 runs · indexed 2026-11-12

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Fable 5.2	0.4414	$98.73	215.9M	821k	18h 12m	545	6
2	Claude Fable 5.2	0.4360	$100.04	83.6M	465k	12h 58m	254	9
3	Claude Fable 5.2	0.3185	$84.04	67.5M	411k	8h 32m	244	5
4	Claude Fable 5.2	0.2925	$159.95	224.8M	773k	18h 1m	458	10
5	Claude Fable 5.2	0.2615	$186.77	128.2M	492k	15h 31m	368	16

Wan 2.1 on MAX/Mojo

task page →5 runs · indexed 2026-11-12

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Claude Fable 5.2	0.9856	$83.37	140.6M	883k	12h 26m	359	13
2	Claude Fable 5.2	0.9707	$119.90	239.2M	1.2M	16h 2m	504	12
3	Claude Fable 5.2	0.9604	$109.47	207.4M	1.1M	12h 59m	454	7
4	Claude Fable 5.2	0.9468	$87.22	157.0M	876k	16h 20m	415	18
5	Claude Fable 5.2	0.9270	$74.72	121.4M	803k	9h 44m	292	5