Traces

Every run: the agent's full trajectory, its submissions over the 20-hour budget, and the final score.

Kolmogorov Audio Compression

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	GPT-5.6	0.0416	$183.22	165.9M	262k	15h 47m	471	12
2	GPT-5.6	0.0408	$43.32	47.2M	136k	19h 34m	249	10
3	GPT-5.6	0.0224	$49.42	77.2M	169k	19h 53m	422	11
4	GPT-5.6	0.0163	$47.54	55.3M	185k	19h 43m	291	10
5	GPT-5.6	0.0159	$41.60	61.4M	164k	18h 50m	368	11

Astronomy Toolkit

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	GPT-5.6	0.4849	$43.38	52.6M	186k	3h 16m	263	5
2	GPT-5.6	0.1823	$39.28	48.9M	195k	2h 7m	255	4
3	GPT-5.6	0.0000	$29.74	37.3M	170k	2h 5m	211	8
4	GPT-5.6	0.0000	$23.28	33.7M	166k	2h 33m	207	5
5	GPT-5.6	0.0000	$47.11	54.0M	192k	2h 34m	262	4

Cranelift Codegen Optimization

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	GPT-5.6	0.0176	$225.59	218.6M	216k	6h 56m	506	6
2	GPT-5.6	0.0000	$297.21	276.6M	238k	7h 17m	549	5
3	GPT-5.6	0.0000	$234.61	222.5M	240k	7h 38m	461	5
4	GPT-5.6	0.0000	$212.19	205.4M	208k	5h 52m	461	5
5	GPT-5.6	0.0000	$355.12	344.3M	254k	10h 32m	627	3

Dart Style in Haskell

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	GPT-5.6	0.7949	$998.14	972.7M	983k	19h 18m	2146	16
2	GPT-5.6	0.6779	$433.00	423.6M	391k	6h 1m	844	4
3	GPT-5.6	0.6344	$89.80	91.5M	201k	1h 55m	362	5
4	GPT-5.6	0.6068	$48.47	52.9M	204k	1h 35m	274	3
5	GPT-5.6	0.4547	$71.79	84.5M	237k	2h 25m	406	7

FFmpeg libswscale Optimization

task page →5 runs · indexed 2026-09-02

#	Model	Cost	In tok	Out tok	Time	Steps	Submits
1	GPT-5.6	$66.82	75.1M	179k	1h 59m	318	8
2	GPT-5.6	$78.43	84.8M	190k	2h 4m	348	5
3	GPT-5.6	$123.31	124.9M	233k	2h 12m	422	3
4	GPT-5.6	$76.67	86.4M	186k	3h 5m	369	10
5	GPT-5.6	$117.96	120.8M	233k	3h 42m	405	8

Crash-Proof Flash Filesystem

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	GPT-5.6	0.0225	$66.23	68.5M	180k	1h 47m	274	8
2	GPT-5.6	0.0225	$81.19	79.4M	199k	1h 55m	291	7
3	GPT-5.6	0.0224	$73.40	72.5M	171k	1h 41m	273	8
4	GPT-5.6	0.0173	$159.04	153.2M	259k	4h 31m	440	16
5	GPT-5.6	0.0173	$92.72	87.0M	212k	1h 59m	268	7

FrogsGame Post-Training

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	GPT-5.6	0.0900	$19.46	28.7M	100k	17h 8m	206	2
2	GPT-5.6	0.0540	$20.87	32.1M	115k	12h 56m	241	3
3	GPT-5.6	0.0380	$22.11	34.6M	103k	18h 7m	223	2
4	GPT-5.6	0.0360	$61.31	64.5M	145k	15h 2m	277	2
5	GPT-5.6	0.0120	$91.18	51.7M	86k	17h 41m	228	4

Stepper Music Sequencer GBA

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	GPT-5.6	0.7701	$944.54	934.7M	1.1M	15h 1m	2411	21
2	GPT-5.6	0.7406	$409.56	413.9M	542k	8h 34m	1251	15
3	GPT-5.6	0.7219	$253.31	245.2M	326k	3h 19m	679	6
4	GPT-5.6	0.6048	$566.59	550.5M	752k	14h 19m	1513	11
5	GPT-5.6	0.4981	$490.17	481.7M	531k	5h 24m	1274	32

Git to Zig

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	GPT-5.6	0.3440	$1238.08	1225.0M	944k	19h 8m	2627	21
2	GPT-5.6	0.2804	$1082.72	1080.1M	850k	19h 36m	2314	42
3	GPT-5.6	0.2801	$1175.97	1171.4M	884k	19h 37m	2602	29
4	GPT-5.6	0.2369	$955.56	945.7M	722k	19h 4m	1982	13
5	GPT-5.6	0.1874	$770.31	763.5M	629k	19h 0m	1668	6

Granite Mamba2 Inference Optimization

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	GPT-5.6	0.7649	$74.74	74.2M	207k	2h 36m	270	5
2	GPT-5.6	0.6303	$111.93	116.6M	261k	11h 2m	425	3
3	GPT-5.6	0.6093	$23.53	32.7M	162k	1h 28m	207	3
4	GPT-5.6	0.4658	$119.15	122.5M	308k	3h 17m	463	6
5	GPT-5.6	0.3383	$180.46	175.3M	266k	4h 7m	474	10

Higgs Uncertainty Inference

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	GPT-5.6	0.7863	$99.90	97.4M	228k	4h 26m	327	20
2	GPT-5.6	0.2084	$52.26	56.6M	152k	2h 42m	244	5
3	GPT-5.6	0.0777	$44.33	50.5M	161k	1h 56m	229	6
4	GPT-5.6	0.0080	$56.44	60.4M	164k	6h 2m	259	12
5	GPT-5.6	0.0049	$70.63	65.2M	124k	2h 29m	194	6

SGLang Inference System Optimization

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	GPT-5.6	0.1092	$291.23	284.1M	251k	5h 12m	616	12
2	GPT-5.6	0.0675	$443.00	437.3M	281k	6h 23m	877	14
3	GPT-5.6	0.0168	$283.36	279.0M	247k	6h 47m	655	21
4	GPT-5.6	0.0000	$109.76	114.5M	139k	5h 57m	401	7
5	GPT-5.6	0.0000	$182.18	175.0M	189k	6h 4m	485	8

Lean 4 Kernel Type Checker in Pascal

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	GPT-5.6	0.9807	$221.16	222.7M	429k	6h 13m	691	19
2	GPT-5.6	0.9791	$78.01	86.8M	272k	4h 32m	395	2
3	GPT-5.6	0.9757	$207.31	199.1M	381k	7h 23m	620	10
4	GPT-5.6	0.9339	$340.15	337.0M	382k	8h 35m	789	11
5	GPT-5.6	0.7874	$397.16	404.0M	396k	8h 54m	1134	7

libexpat Optimization

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	GPT-5.6	0.3847	$146.13	140.6M	311k	2h 25m	439	7
2	GPT-5.6	0.3225	$67.06	75.3M	223k	1h 44m	326	9
3	GPT-5.6	0.3025	$18.35	26.1M	127k	53m	144	4
4	GPT-5.6	0.2868	$49.11	53.9M	193k	1h 32m	238	5
5	GPT-5.6	0.2751	$110.05	104.7M	243k	2h 24m	350	7

Lua Native Compiler

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	GPT-5.6	0.0000	$35.07	44.0M	172k	2h 41m	211	10
2	GPT-5.6	0.0000†	$7.45	8.9M	65k	25m	65	3
3	GPT-5.6	0.0000	$127.72	124.4M	217k	2h 5m	382	24
4	GPT-5.6	0.0000	$24.84	37.1M	163k	1h 9m	219	13
5	GPT-5.6	0.0000	$20.50	39.9M	168k	1h 9m	231	7

† score adjusted by manual review; the trace's Judge tab carries the measured score and the rationale.

Machine-Learned Interatomic Potential

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	GPT-5.6	0.9925	$108.64	109.2M	151k	11h 16m	348	8
2	GPT-5.6	0.9918	$109.29	114.5M	152k	15h 47m	404	16
3	GPT-5.6	0.9437	$26.20	41.3M	108k	14h 19m	237	3
4	GPT-5.6	0.0076	$48.99	56.5M	122k	14h 29m	293	5
5	GPT-5.6	0.0010	$36.55	49.8M	109k	7h 20m	272	3

Medium-Range Weather Forecast

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	GPT-5.6	0.5936	$28.82	47.8M	113k	15h 13m	294	4
2	GPT-5.6	0.0000	$122.35	137.1M	209k	17h 41m	576	7
3	GPT-5.6	0.0000	$63.62	70.4M	130k	17h 56m	290	3
4	GPT-5.6	0.0000	$72.57	94.1M	189k	15h 53m	478	14
5	GPT-5.6	0.0000	$88.96	100.2M	138k	14h 52m	416	8

MEG Speech Decoding

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	GPT-5.6	0.3119	$41.21	43.5M	133k	5h 23m	204	2
2	GPT-5.6	0.3039	$14.10	19.5M	85k	2h 25m	137	2
3	GPT-5.6	0.2311	$153.75	128.1M	180k	10h 45m	298	17
4	GPT-5.6	0.1902	$64.35	67.7M	163k	7h 2m	279	4
5	GPT-5.6	0.1862	$104.80	98.3M	165k	7h 53m	318	8

Wan 2.1 on MAX/Mojo

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	GPT-5.6	0.9241	$103.11	107.4M	168k	5h 28m	369	8
2	GPT-5.6	0.9087	$34.04	44.6M	108k	4h 5m	199	9
3	GPT-5.6	0.9062	$69.52	78.1M	124k	3h 50m	295	3
4	GPT-5.6	0.9014	$57.11	66.4M	129k	4h 8m	267	5
5	GPT-5.6	0.8681	$66.06	72.8M	105k	4h 8m	267	4

MS/MS De Novo Generation

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	GPT-5.6	0.1316	$55.65	69.9M	170k	12h 29m	332	2
2	GPT-5.6	0.1312	$87.71	101.7M	199k	15h 19m	453	6
3	GPT-5.6	0.0000	$29.44	43.7M	172k	11h 11m	291	4
4	GPT-5.6	0.0000	$115.28	137.3M	233k	11h 7m	603	10
5	GPT-5.6	0.0000	$33.43	41.9M	177k	7h 37m	257	4

Multi-GPU Efficient Finetuning

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	GPT-5.6	0.2381	$19.33	23.5M	110k	15h 47m	167	3
2	GPT-5.6	0.0238	$19.65	25.0M	71k	12h 53m	133	3
3	GPT-5.6	0.0000	$16.45	25.9M	78k	16h 50m	180	2
4	GPT-5.6	0.0000	$24.13	34.7M	95k	16h 10m	181	2
5	GPT-5.6	0.0000	$30.41	39.1M	90k	14h 8m	199	3

Notebook Compression

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	GPT-5.6	0.2623	$123.00	167.4M	279k	11h 19m	628	8
2	GPT-5.6	0.2599	$99.01	115.0M	257k	15h 3m	528	10
3	GPT-5.6	0.2438	$58.48	76.1M	190k	17h 55m	412	14
4	GPT-5.6	0.2326	$88.19	110.8M	211k	11h 35m	518	6
5	GPT-5.6	0.1417	$64.27	80.6M	214k	10h 2m	389	3

Flight-Sim Renderer in OpenGL

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	GPT-5.6	0.2171	$270.86	274.0M	362k	12h 30m	806	4
2	GPT-5.6	0.1552	$336.67	340.6M	347k	11h 28m	906	14
3	GPT-5.6	0.1222	$253.61	257.1M	305k	8h 28m	760	3
4	GPT-5.6	0.1119	$309.74	312.3M	342k	9h 22m	857	12
5	GPT-5.6	0.1032	$420.32	406.2M	354k	11h 35m	880	13

Optimizer Design

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	GPT-5.6	0.3487	$65.53	80.8M	133k	10h 22m	433	4
2	GPT-5.6	0.3194	$196.66	203.2M	202k	11h 49m	678	23
3	GPT-5.6	0.2846	$55.29	72.7M	133k	15h 42m	423	5
4	GPT-5.6	0.2809	$71.65	93.2M	177k	14h 22m	523	21
5	GPT-5.6	0.2234	$31.29	52.9M	100k	13h 17m	373	2

PostgreSQL 18 on SQLite

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	GPT-5.6	0.5275	$209.42	198.9M	277k	4h 5m	589	12
2	GPT-5.6	0.3901	$654.39	640.1M	687k	19h 58m	1472	28
3	GPT-5.6	0.0055†	$293.05	285.8M	311k	4h 20m	671	6
4	GPT-5.6	0.0000	$137.59	141.5M	243k	3h 27m	480	2
5	GPT-5.6	0.0000	$142.29	142.1M	259k	4h 34m	462	11

† score adjusted by manual review; the trace's Judge tab carries the measured score and the rationale.

Quantum ESPRESSO pw.x in Rust

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	GPT-5.6	0.5203	$304.21	296.3M	260k	4h 50m	606	9
2	GPT-5.6	0.4634	$57.42	64.9M	189k	2h 47m	279	13
3	GPT-5.6	0.4553	$120.99	116.9M	214k	5h 5m	404	8
4	GPT-5.6	0.4065	$104.97	107.5M	204k	3h 16m	391	10
5	GPT-5.6	0.0000	$210.32	203.5M	250k	6h 57m	577	6

Qubit Routing

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	GPT-5.6	0.7946	$58.77	77.9M	203k	6h 45m	423	8
2	GPT-5.6	0.7781	$75.11	86.9M	202k	10h 41m	410	11
3	GPT-5.6	0.7314	$114.07	131.6M	228k	8h 56m	569	14
4	GPT-5.6	0.7190	$154.11	169.8M	283k	6h 40m	688	13
5	GPT-5.6	0.7125	$69.33	92.1M	185k	8h 7m	475	5

Reconnaissance Blind Chess Recovery

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	GPT-5.6	0.8893	$28.42	45.1M	149k	6h 33m	291	7
2	GPT-5.6	0.8607	$35.99	57.3M	138k	6h 13m	336	13
3	GPT-5.6	0.8071	$20.62	32.5M	99k	10h 50m	196	10
4	GPT-5.6	0.8000	$20.53	30.5M	123k	2h 52m	197	3
5	GPT-5.6	0.7357	$15.51	21.1M	106k	11h 38m	168	9

Fitness-Recap Video in Remotion

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	GPT-5.6	0.2152	$950.51	938.4M	1.0M	19h 3m	2343	45
2	GPT-5.6	0.1930	$656.93	636.3M	650k	19h 12m	1478	11
3	GPT-5.6	0.1913	$298.28	288.7M	261k	7h 23m	651	12
4	GPT-5.6	0.1895	$283.99	275.5M	274k	5h 43m	687	6
5	GPT-5.6	0.1374	$190.12	189.9M	230k	5h 21m	589	2

Snooker Prediction

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	GPT-5.6	0.4857	$119.18	111.7M	242k	2h 5m	299	11
2	GPT-5.6	0.4839	$170.16	165.0M	275k	4h 12m	432	9
3	GPT-5.6	0.4428	$157.87	150.8M	208k	2h 34m	354	9
4	GPT-5.6	0.4295	$132.32	126.4M	240k	4h 1m	369	6
5	GPT-5.6	0.2890	$179.27	154.6M	213k	9h 23m	357	5

SPICE Circuit Simulator in Rust

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	GPT-5.6	0.4444	$366.07	350.2M	455k	6h 25m	797	22
2	GPT-5.6	0.4141	$307.32	296.1M	386k	4h 30m	783	15
3	GPT-5.6	0.3939	$538.95	528.2M	617k	6h 30m	1280	21
4	GPT-5.6	0.3333	$187.27	186.7M	262k	2h 15m	515	8
5	GPT-5.6	0.2626	$226.84	221.2M	327k	3h 7m	592	13

Synthetic Music Diarization

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	GPT-5.6	0.5247	$44.19	53.3M	154k	7h 34m	267	3
2	GPT-5.6	0.5065	$80.65	78.5M	178k	15h 57m	339	11
3	GPT-5.6	0.4952	$59.62	69.7M	147k	15h 37m	359	3
4	GPT-5.6	0.4737	$78.78	76.5M	133k	14h 47m	286	4
5	GPT-5.6	0.4727	$68.59	82.8M	153k	14h 29m	425	7

Vision-only TORCS Racing Bot

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	GPT-5.6	0.1207	$165.68	160.5M	163k	9h 38m	416	7
2	GPT-5.6	0.0000†	$58.14	63.0M	111k	13h 6m	288	4
3	GPT-5.6	0.0000†	$25.75	38.4M	104k	16h 16m	218	4
4	GPT-5.6	0.0000†	$51.99	57.9M	124k	4h 17m	277	6
5	GPT-5.6	0.0000†	$79.43	80.8M	113k	10h 50m	285	4

† score adjusted by manual review; the trace's Judge tab carries the measured score and the rationale.

Verilog Simulator in Swift

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	GPT-5.6	0.8853	$427.16	420.2M	394k	14h 37m	921	9
2	GPT-5.6	0.8596	$271.89	260.9M	315k	4h 56m	650	5
3	GPT-5.6	0.8493	$335.11	326.9M	349k	4h 23m	746	9
4	GPT-5.6	0.8339	$184.09	180.1M	273k	4h 24m	480	6
5	GPT-5.6	0.8236	$181.11	176.3M	277k	4h 52m	476	3