Traces

Every run: the agent's full trajectory, its submissions over the 20-hour budget, and the final score.

Kolmogorov Audio Compression

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Qwen3.8-Max	0.0171	$21.55	69.7M	606k	19h 26m	446	15
2	Qwen3.8-Max	0.0155	$18.80	60.7M	524k	19h 36m	400	4
3	Qwen3.8-Max	0.0118	$16.99	53.1M	556k	19h 0m	415	2
4	Qwen3.8-Max	0.0098	$20.15	54.4M	994k	19h 8m	321	6
5	Qwen3.8-Max	0.0089	$23.54	77.9M	588k	18h 20m	403	2

Astronomy Toolkit

task page →5 runs · indexed 2026-09-02

#	Model	Cost	In tok	Out tok	Time	Steps	Submits
1	Qwen3.8-Max	$42.82	145.6M	925k	17h 28m	524	2
2	Qwen3.8-Max	$48.24	157.1M	1.3M	19h 42m	518	2
3	Qwen3.8-Max	$42.25	133.5M	1.3M	19h 46m	412	2
4	Qwen3.8-Max	$56.35	191.3M	1.2M	19h 55m	589	1
5	Qwen3.8-Max	$28.32	86.1M	1.0M	14h 43m	386	3

Cranelift Codegen Optimization

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Qwen3.8-Max	0.0117	$41.08	108.0M	2.2M	18h 38m	484	6
2	Qwen3.8-Max	0.0000	$34.88	81.5M	2.3M	20h 0m	413	1
3	Qwen3.8-Max	0.0000	$43.78	119.5M	2.2M	18h 46m	496	2
4	Qwen3.8-Max	0.0000	$50.74	159.1M	1.7M	16h 16m	624	4
5	Qwen3.8-Max	0.0000	$42.54	115.4M	2.2M	20h 0m	491	1

Dart Style in Haskell

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Qwen3.8-Max	0.4828	$83.97	303.2M	1.2M	16h 38m	893	2
2	Qwen3.8-Max	0.3212	$156.83	579.3M	1.7M	20h 1m	1506	1
3	Qwen3.8-Max	0.2726	$83.47	304.6M	1.0M	17h 28m	843	2
4	Qwen3.8-Max	0.2674	$50.86	177.6M	941k	18h 25m	662	12
5	Qwen3.8-Max	0.0853	$104.77	373.0M	1.7M	19h 28m	957	1

FFmpeg libswscale Optimization

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Qwen3.8-Max	0.0373	$86.44	286.2M	2.3M	17h 43m	706	6
2	Qwen3.8-Max	0.0181	$64.05	193.9M	2.4M	18h 48m	555	1
3	Qwen3.8-Max	0.0053	$46.03	124.6M	2.3M	19h 58m	424	1
4	Qwen3.8-Max	0.0000	$54.69	158.2M	2.4M	18h 58m	475	1
5	Qwen3.8-Max	0.0000	$30.12	66.3M	2.1M	19h 55m	298	1

Crash-Proof Flash Filesystem

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Qwen3.8-Max	0.0013	$12.18	16.0M	1.3M	19h 33m	130	1
2	Qwen3.8-Max	0.0001	$13.81	19.3M	1.5M	19h 35m	164	3
3	Qwen3.8-Max	0.0000	$14.39	19.8M	1.5M	20h 1m	160	1
4	Qwen3.8-Max	0.0000	$9.18	10.7M	1.0M	19h 50m	100	1
5	Qwen3.8-Max	0.0000	$13.72	18.7M	1.5M	18h 42m	146	3

FrogsGame Post-Training

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Qwen3.8-Max	0.2160	$11.97	38.7M	277k	19h 34m	297	3
2	Qwen3.8-Max	0.0980	$10.69	32.2M	339k	19h 46m	272	1
3	Qwen3.8-Max	0.0880	$5.85	16.5M	229k	19h 7m	195	1
4	Qwen3.8-Max	0.0360	$35.74	120.9M	650k	19h 55m	546	2
5	Qwen3.8-Max	0.0300	$11.98	37.6M	292k	18h 44m	323	4

Stepper Music Sequencer GBA

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Qwen3.8-Max	0.0240	$114.04	338.0M	2.8M	19h 40m	884	6
2	Qwen3.8-Max	0.0224	$119.80	401.3M	2.9M	20h 1m	934	—
3	Qwen3.8-Max	0.0183	$113.87	373.6M	3.0M	19h 42m	990	—
4	Qwen3.8-Max	0.0159	$129.76	393.1M	2.8M	19h 53m	1102	—
5	Qwen3.8-Max	0.0002	$139.67	464.7M	2.7M	19h 56m	1203	—

Git to Zig

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Qwen3.8-Max	0.1212	$184.24	696.5M	1.3M	19h 33m	1805	12
2	Qwen3.8-Max	0.1038	$196.74	742.9M	1.4M	19h 38m	1831	26
3	Qwen3.8-Max	0.1023	$180.44	685.7M	1.2M	19h 14m	1776	2
4	Qwen3.8-Max	0.0987	$178.20	662.0M	1.7M	19h 50m	1710	6
5	Qwen3.8-Max	0.0927	$190.64	725.2M	1.3M	19h 52m	1620	35

Granite Mamba2 Inference Optimization

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Qwen3.8-Max	0.8086	$63.86	200.9M	2.1M	18h 31m	572	2
2	Qwen3.8-Max	0.7876	$54.77	177.9M	1.6M	16h 1m	520	5
3	Qwen3.8-Max	0.7188	$63.39	207.4M	1.7M	17h 14m	557	9
4	Qwen3.8-Max	0.7023	$29.64	70.4M	1.9M	19h 40m	330	1
5	Qwen3.8-Max	0.6658	$53.16	162.1M	1.8M	18h 29m	495	2

Higgs Uncertainty Inference

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Qwen3.8-Max	0.9778	$23.74	79.2M	556k	12h 13m	428	14
2	Qwen3.8-Max	0.7685	$16.66	51.1M	571k	13h 28m	342	5
3	Qwen3.8-Max	0.4289	$16.21	49.4M	555k	8h 58m	285	8
4	Qwen3.8-Max	0.3449	$15.61	47.3M	543k	7h 37m	278	8
5	Qwen3.8-Max	0.3078	$18.56	57.9M	578k	8h 28m	322	6

SGLang Inference System Optimization

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Qwen3.8-Max	0.0214	$45.80	153.5M	1.1M	15h 59m	715	7
2	Qwen3.8-Max	0.0081	$33.42	99.7M	1.3M	15h 1m	574	8
3	Qwen3.8-Max	0.0004	$47.32	149.7M	1.5M	17h 59m	651	3
4	Qwen3.8-Max	0.0000	$42.20	143.0M	952k	17h 51m	604	1
5	Qwen3.8-Max	0.0000	$47.25	152.8M	1.4M	17h 10m	642	3

Lean 4 Kernel Type Checker in Pascal

task page →5 runs · indexed 2026-09-02

#	Model	Cost	In tok	Out tok	Time	Steps	Submits
1	Qwen3.8-Max	$14.14	28.2M	1.1M	19h 59m	169	1
2	Qwen3.8-Max	$8.31	10.6M	914k	19h 59m	109	2
3	Qwen3.8-Max	$13.92	25.5M	1.2M	19h 56m	167	2
4	Qwen3.8-Max	$19.68	45.8M	1.3M	20h 0m	211	1
5	Qwen3.8-Max	$10.35	13.6M	1.1M	19h 57m	115	1

libexpat Optimization

task page →5 runs · indexed 2026-09-02

#	Model	Cost	In tok	Out tok	Time	Steps	Submits
1	Qwen3.8-Max	$21.15	36.8M	1.9M	20h 0m	260	—
2	Qwen3.8-Max	$16.28	22.3M	1.7M	19h 46m	202	—
3	Qwen3.8-Max	$19.23	33.9M	1.7M	19h 53m	233	1
4	Qwen3.8-Max	$25.12	46.8M	2.2M	19h 55m	301	—
5	Qwen3.8-Max	$13.06	14.1M	1.6M	19h 35m	156	—

Lua Native Compiler

task page →5 runs · indexed 2026-09-02

#	Model	Cost	In tok	Out tok	Time	Steps	Submits
1	Qwen3.8-Max	$64.51	206.8M	2.0M	19h 8m	639	2
2	Qwen3.8-Max	$121.10	426.7M	2.2M	19h 59m	984	1
3	Qwen3.8-Max	$135.25	480.8M	2.3M	19h 7m	1081	2
4	Qwen3.8-Max	$60.45	189.1M	2.1M	19h 48m	599	3
5	Qwen3.8-Max	$31.00	87.6M	1.4M	13h 21m	385	6

Machine-Learned Interatomic Potential

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Qwen3.8-Max	0.9731	$4.10	11.4M	180k	17h 54m	185	3
2	Qwen3.8-Max	0.9220	$5.13	15.3M	180k	19h 14m	209	2
3	Qwen3.8-Max	0.8093	$6.94	20.8M	251k	18h 48m	231	2
4	Qwen3.8-Max	0.2071	$6.23	16.9M	295k	18h 26m	218	2
5	Qwen3.8-Max	0.0010	$4.41	12.0M	206k	19h 18m	179	1

Medium-Range Weather Forecast

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Qwen3.8-Max	0.4005	$8.94	29.1M	230k	19h 22m	281	1
2	Qwen3.8-Max	0.3989	$14.91	48.7M	387k	19h 25m	357	1
3	Qwen3.8-Max	0.3606	$13.12	44.7M	268k	19h 18m	385	3
4	Qwen3.8-Max	0.3410	$4.91	14.6M	172k	18h 25m	189	4
5	Qwen3.8-Max	0.3360	$8.01	25.7M	213k	18h 56m	239	8

MEG Speech Decoding

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Qwen3.8-Max	0.1460	$13.18	43.8M	302k	19h 37m	333	7
2	Qwen3.8-Max	0.0985	$20.47	70.2M	412k	18h 53m	478	2
3	Qwen3.8-Max	0.0000	$13.84	45.7M	331k	19h 21m	336	2
4	Qwen3.8-Max	0.0000	$27.96	99.7M	412k	19h 32m	527	4
5	Qwen3.8-Max	0.0000	$8.55	26.2M	281k	19h 56m	242	1

Wan 2.1 on MAX/Mojo

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Qwen3.8-Max	0.4407	$34.51	98.7M	1.3M	19h 41m	423	2
2	Qwen3.8-Max	0.4404	$41.19	95.2M	1.6M	20h 1m	427	1
3	Qwen3.8-Max	0.2981	$45.25	145.8M	997k	19h 53m	526	1
4	Qwen3.8-Max	0.0664	$30.34	85.4M	1.2M	19h 11m	360	2
5	Qwen3.8-Max	0.0000	$50.91	123.3M	1.9M	19h 56m	467	1

MS/MS De Novo Generation

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Qwen3.8-Max	0.0879	$12.07	38.5M	344k	19h 44m	290	3
2	Qwen3.8-Max	0.0652	$23.10	77.1M	550k	18h 46m	436	2
3	Qwen3.8-Max	0.0000	$18.80	59.9M	558k	19h 4m	366	1
4	Qwen3.8-Max	0.0000	$22.77	75.2M	580k	19h 13m	492	3
5	Qwen3.8-Max	0.0000	$31.11	106.6M	635k	17h 29m	500	3

Multi-GPU Efficient Finetuning

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Qwen3.8-Max	0.2857	$6.42	17.1M	316k	18h 44m	213	5
2	Qwen3.8-Max	0.2619	$6.15	16.6M	296k	17h 38m	220	10
3	Qwen3.8-Max	0.0000	$4.65	11.0M	287k	19h 49m	183	5
4	Qwen3.8-Max	0.0000	$3.67	9.9M	171k	19h 13m	153	3
5	Qwen3.8-Max	0.0000	$7.44	21.5M	284k	19h 29m	253	2

Notebook Compression

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Qwen3.8-Max	0.2190	$26.32	83.3M	818k	19h 14m	454	2
2	Qwen3.8-Max	0.2153	$10.82	27.5M	610k	19h 5m	314	2
3	Qwen3.8-Max	0.2125	$16.15	47.4M	646k	17h 46m	362	2
4	Qwen3.8-Max	0.2005	$15.43	43.7M	693k	18h 44m	414	1
5	Qwen3.8-Max	0.1831	$20.18	58.6M	851k	19h 8m	443	2

Flight-Sim Renderer in OpenGL

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Qwen3.8-Max	0.0018	$183.02	558.1M	1.7M	19h 56m	1375	1
2	Qwen3.8-Max	0.0016	$153.43	448.8M	1.7M	19h 59m	1202	1
3	Qwen3.8-Max	0.0010	$149.46	419.0M	1.2M	19h 25m	993	2
4	Qwen3.8-Max	0.0001	$169.64	519.3M	1.6M	20h 0m	1239	1
5	Qwen3.8-Max	0.0000	$189.91	559.5M	1.9M	19h 51m	1464	3

Optimizer Design

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Qwen3.8-Max	0.0827	$7.68	17.4M	516k	19h 49m	226	3
2	Qwen3.8-Max	0.0741	$7.61	18.8M	448k	19h 35m	241	3
3	Qwen3.8-Max	0.0655	$6.87	15.9M	443k	19h 59m	203	3
4	Qwen3.8-Max	0.0143	$7.32	18.3M	421k	19h 31m	238	1
5	Qwen3.8-Max	0.0000	$12.42	33.0M	636k	19h 59m	303	1

PostgreSQL 18 on SQLite

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Qwen3.8-Max	0.1374	$186.86	700.0M	1.6M	19h 30m	1822	2
2	Qwen3.8-Max	0.1319	$178.70	667.7M	1.7M	19h 47m	1549	4
3	Qwen3.8-Max	0.0769	$170.86	638.4M	1.5M	19h 53m	1587	2
4	Qwen3.8-Max	0.0659	$174.14	653.5M	1.5M	19h 26m	1536	1
5	Qwen3.8-Max	0.0549	$158.21	583.9M	1.7M	19h 23m	1405	3

Quantum ESPRESSO pw.x in Rust

task page →5 runs · indexed 2026-09-02

#	Model	Cost	In tok	Out tok	Time	Steps	Submits
1	Qwen3.8-Max	$48.53	146.5M	1.8M	19h 49m	431	1
2	Qwen3.8-Max	$54.35	159.6M	2.3M	19h 59m	413	1
3	Qwen3.8-Max	$33.37	86.6M	1.8M	19h 52m	301	2
4	Qwen3.8-Max	$35.43	95.7M	1.8M	20h 1m	338	1
5	Qwen3.8-Max	$11.62	13.4M	1.3M	19h 53m	112	1

Qubit Routing

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Qwen3.8-Max	0.6875	$16.94	52.6M	563k	18h 44m	340	2
2	Qwen3.8-Max	0.5728	$25.90	85.8M	648k	18h 2m	442	6
3	Qwen3.8-Max	0.4459	$25.19	78.5M	819k	19h 40m	374	2
4	Qwen3.8-Max	0.4074	$19.97	63.7M	596k	17h 56m	390	2
5	Qwen3.8-Max	0.2765	$43.05	138.5M	1.3M	20h 1m	512	1

Reconnaissance Blind Chess Recovery

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Qwen3.8-Max	0.1357	$20.20	59.6M	794k	13h 51m	345	3
2	Qwen3.8-Max	0.0750	$16.70	49.8M	638k	13h 27m	335	3
3	Qwen3.8-Max	0.0536	$12.92	35.1M	623k	13h 47m	238	3
4	Qwen3.8-Max	0.0536	$22.79	70.5M	780k	13h 7m	415	3
5	Qwen3.8-Max	0.0071	$26.05	82.8M	805k	13h 22m	436	3

Fitness-Recap Video in Remotion

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Qwen3.8-Max	0.1474	$77.77	271.1M	994k	13h 7m	800	4
2	Qwen3.8-Max	0.0733	$163.48	458.4M	1.7M	19h 54m	1249	1
3	Qwen3.8-Max	0.0127	$196.38	482.4M	1.8M	19h 51m	1048	1
4	Qwen3.8-Max	0.0062	$52.83	157.6M	675k	9h 28m	578	4
5	Qwen3.8-Max	0.0000†	$11.91	31.2M	238k	4h 2m	292	2

† score adjusted by manual review; the trace's Judge tab carries the measured score and the rationale.

Snooker Prediction

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Qwen3.8-Max	0.0882	$56.82	135.8M	1.2M	18h 54m	506	1
2	Qwen3.8-Max	0.0000	$80.65	199.9M	2.1M	18h 39m	673	1
3	Qwen3.8-Max	0.0000	$115.97	289.9M	2.1M	19h 35m	811	1
4	Qwen3.8-Max	0.0000	$102.79	307.3M	1.7M	19h 10m	794	2
5	Qwen3.8-Max	0.0000	$61.40	180.5M	1.6M	20h 1m	551	1

SPICE Circuit Simulator in Rust

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Qwen3.8-Max	0.3737	$145.98	524.1M	2.1M	19h 26m	1338	1
2	Qwen3.8-Max	0.3434	$127.04	463.9M	1.6M	19h 13m	987	4
3	Qwen3.8-Max	0.3030	$96.60	342.7M	1.6M	18h 43m	760	2
4	Qwen3.8-Max	0.3030	$110.14	386.5M	2.0M	19h 56m	947	1
5	Qwen3.8-Max	0.2727	$126.58	455.7M	1.9M	19h 26m	951	7

Synthetic Music Diarization

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Qwen3.8-Max	0.4605	$29.62	103.3M	529k	18h 43m	509	11
2	Qwen3.8-Max	0.4425	$26.66	93.3M	455k	18h 13m	478	8
3	Qwen3.8-Max	0.4301	$34.00	119.2M	572k	19h 17m	508	10
4	Qwen3.8-Max	0.4043	$33.94	117.1M	655k	18h 57m	501	5
5	Qwen3.8-Max	0.3951	$17.02	56.1M	424k	19h 10m	391	4

Vision-only TORCS Racing Bot

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Qwen3.8-Max	0.1078	$31.56	67.3M	1.0M	19h 45m	456	2
2	Qwen3.8-Max	0.0385	$33.83	72.9M	764k	18h 52m	422	2
3	Qwen3.8-Max	0.0216	$32.92	71.7M	721k	19h 16m	425	2
4	Qwen3.8-Max	0.0061	$53.75	151.0M	965k	20h 0m	695	2
5	Qwen3.8-Max	0.0000†	$18.02	51.8M	431k	20h 1m	433	2

† score adjusted by manual review; the trace's Judge tab carries the measured score and the rationale.

Verilog Simulator in Swift

task page →5 runs · indexed 2026-09-02

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Qwen3.8-Max	0.7551	$162.17	597.6M	1.8M	19h 37m	1491	1
2	Qwen3.8-Max	0.4384	$128.73	471.1M	1.6M	19h 41m	965	2
3	Qwen3.8-Max	0.3527	$147.05	546.0M	1.5M	19h 49m	1109	2
4	Qwen3.8-Max	0.2894	$161.74	608.8M	1.4M	19h 56m	1406	2
5	Qwen3.8-Max	0.2414	$54.19	188.0M	1.1M	20h 0m	597	1