Tasks Traces V1 Blog Changelog Contribute GitHub

Traces

Every run: the agent's full trajectory, its submissions over the 20-hour budget, and the final score.

Kolmogorov Audio Compression

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Grok 4.7	0.0462	$180.50	154.3M	515k	19h 45m	631	5
2	Grok 4.7	0.0329	$52.47	42.4M	229k	5h 56m	210	12
3	Grok 4.7	0.0313	$93.78	78.9M	256k	12h 4m	313	14
4	Grok 4.7	0.0218	$33.81	32.7M	228k	2h 52m	171	8
5	Grok 4.7	0.0128	$49.77	44.3M	205k	7h 40m	231	6

Astronomy Toolkit

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Grok 4.7	0.3076	$28.45	28.6M	218k	1h 15m	142	5
2	Grok 4.7	0.2999	$23.31	23.1M	224k	1h 29m	118	3
3	Grok 4.7	0.0108	$18.18	18.2M	206k	1h 1m	105	3
4	Grok 4.7	0.0075	$224.04	218.5M	550k	3h 42m	699	15
5	Grok 4.7	0.0032	$13.33	14.3M	178k	52m	90	2

Cranelift Codegen Optimization

task page →5 runs · indexed 2026-09-22

#	Model	Cost	In tok	Out tok	Time	Steps	Submits
1	Grok 4.7	$645.99	652.2M	2.1M	20h 0m	2589	15
2	Grok 4.7	$228.14	230.9M	694k	6h 44m	948	7
3	Grok 4.7	$477.48	483.7M	1.5M	16h 55m	1975	20
4	Grok 4.7	$635.61	634.0M	1.8M	19h 42m	2545	20
5	Grok 4.7	$640.87	669.7M	1.8M	19h 25m	2877	27

Dart Style in Haskell

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Grok 4.7	0.7672	$810.72	832.8M	2.4M	19h 51m	3315	144
2	Grok 4.7	0.7101	$934.33	959.7M	2.7M	20h 0m	3786	76
3	Grok 4.7	0.6935	$819.81	827.3M	2.4M	19h 30m	3292	79
4	Grok 4.7	0.6906	$879.91	896.0M	2.8M	19h 59m	3561	82
5	Grok 4.7	0.4324	$263.14	264.6M	900k	6h 3m	996	20

FFmpeg libswscale Optimization

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Grok 4.7	0.0717	$958.45	972.7M	3.5M	19h 0m	3810	51
2	Grok 4.7	0.0692	$75.94	76.6M	393k	2h 13m	349	5
3	Grok 4.7	0.0562	$84.27	86.0M	423k	2h 12m	353	5
4	Grok 4.7	0.0482	$1089.41	1092.6M	3.0M	15h 42m	3929	49
5	Grok 4.7	0.0022	$90.46	89.2M	451k	3h 23m	379	9

Crash-Proof Flash Filesystem

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Grok 4.7	0.0806	$696.28	696.9M	2.0M	12h 34m	2584	17
2	Grok 4.7	0.0184	$312.58	313.6M	1.2M	9h 0m	1280	10
3	Grok 4.7	0.0184	$793.49	801.3M	2.8M	20h 0m	3084	23
4	Grok 4.7	0.0173	$406.03	410.3M	1.5M	9h 50m	1624	19
5	Grok 4.7	0.0164	$349.50	343.9M	1.4M	11h 51m	1349	8

FrogsGame Post-Training

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Grok 4.7	0.4940	$60.79	36.4M	219k	19h 41m	169	1
2	Grok 4.7	0.3640	$259.38	236.3M	353k	19h 57m	795	2
3	Grok 4.7	0.2980	$90.99	63.2M	209k	18h 43m	277	2
4	Grok 4.7	0.2920	$118.94	66.1M	194k	19h 26m	282	2
5	Grok 4.7	0.0460	$50.68	36.5M	192k	18h 50m	196	2

Stepper Music Sequencer GBA

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Grok 4.7	0.7472	$628.06	649.9M	2.0M	11h 13m	2618	63
2	Grok 4.7	0.6012	$1385.34	1395.4M	3.2M	19h 29m	5064	67
3	Grok 4.7	0.4771	$380.66	393.3M	1.3M	7h 10m	1597	32
4	Grok 4.7	0.4712	$1073.48	1101.7M	3.6M	19h 4m	4513	110
5	Grok 4.7	0.4415	$440.85	450.4M	1.7M	8h 48m	1863	19

Git to Zig

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Grok 4.7	0.2530	$1242.68	1266.7M	4.0M	20h 0m	5068	70
2	Grok 4.7	0.2469	$910.55	936.9M	2.7M	19h 58m	3768	53
3	Grok 4.7	0.2342	$1107.04	1144.5M	3.0M	20h 0m	4638	66
4	Grok 4.7	0.2156	$741.65	767.0M	2.0M	19h 54m	3099	73
5	Grok 4.7	0.1876	$623.38	637.1M	2.0M	13h 32m	2600	18

Granite Mamba2 Inference Optimization

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Grok 4.7	0.8555	$664.66	670.3M	2.4M	20h 0m	2515	55
2	Grok 4.7	0.8303	$663.28	652.6M	2.1M	14h 34m	2345	24
3	Grok 4.7	0.7778	$748.71	746.1M	3.0M	20h 0m	2763	23
4	Grok 4.7	0.7473	$45.19	45.2M	263k	1h 21m	181	3
5	Grok 4.7	0.4410	$49.65	48.3M	283k	2h 1m	184	13

Higgs Uncertainty Inference

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Grok 4.7	0.7706	$11.93	13.0M	204k	1h 11m	90	5
2	Grok 4.7	0.0000	$420.44	408.4M	739k	19h 20m	1293	18
3	Grok 4.7	0.0000	$7.84	8.3M	179k	58m	63	3
4	Grok 4.7	0.0000	$17.21	17.6M	220k	1h 13m	99	5
5	Grok 4.7	0.0000	$9.29	10.1M	177k	1h 2m	76	2

SGLang Inference System Optimization

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Grok 4.7	0.0552	$802.99	801.5M	2.3M	19h 22m	3183	24
2	Grok 4.7	0.0000	$211.95	219.1M	541k	7h 14m	876	5
3	Grok 4.7	0.0000	$51.59	50.1M	157k	1h 43m	212	5
4	Grok 4.7	0.0000	$100.20	104.5M	326k	3h 50m	470	5
5	Grok 4.7	0.0000	$565.89	566.8M	1.5M	11h 16m	2189	10

Lean 4 Kernel Type Checker in Pascal

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Grok 4.7	0.9589	$192.69	192.9M	802k	7h 19m	790	8
2	Grok 4.7	0.9461	$478.73	472.8M	1.7M	19h 19m	1887	5
3	Grok 4.7	0.9103	$351.92	346.3M	1.1M	20h 0m	1425	6
4	Grok 4.7	0.8682	$294.16	294.8M	1.1M	13h 7m	1229	6
5	Grok 4.7	0.0000	$418.16	407.2M	1.6M	18h 42m	1640	7

libexpat Optimization

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Grok 4.7	0.3527	$861.01	886.1M	3.3M	19h 22m	3633	135
2	Grok 4.7	0.1576	$386.43	383.9M	880k	5h 16m	1277	13
3	Grok 4.7	0.0000	$77.71	79.6M	493k	2h 16m	325	4
4	Grok 4.7	0.0000	$35.69	33.2M	254k	1h 17m	139	5
5	Grok 4.7	0.0000†	$83.03	79.6M	516k	2h 36m	314	13

† score adjusted by manual review; the trace's Judge tab carries the measured score and the rationale.

Lua Native Compiler

task page →5 runs · indexed 2026-09-22

#	Model	Cost	In tok	Out tok	Time	Steps	Submits
1	Grok 4.7	$74.24	76.4M	456k	2h 25m	373	1
2	Grok 4.7	$23.46	23.7M	161k	55m	121	1
3	Grok 4.7	$84.78	85.6M	413k	2h 11m	330	3
4	Grok 4.7	$10.18	12.5M	117k	39m	81	1
5	Grok 4.7	$15.52	17.3M	144k	46m	103	1

Machine-Learned Interatomic Potential

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Grok 4.7	0.9608	$60.45	36.2M	244k	15h 59m	187	2
2	Grok 4.7	0.9588	$76.24	57.2M	274k	16h 2m	238	12
3	Grok 4.7	0.9549	$72.03	45.6M	246k	19h 9m	215	7
4	Grok 4.7	0.9454	$102.72	67.6M	289k	18h 50m	254	8
5	Grok 4.7	0.0000	$87.65	66.0M	273k	12h 6m	260	15

Medium-Range Weather Forecast

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Grok 4.7	0.6200	$170.45	150.1M	788k	19h 40m	619	21
2	Grok 4.7	0.5877	$164.64	141.5M	437k	18h 53m	596	29
3	Grok 4.7	0.5339	$192.06	177.7M	824k	19h 10m	758	17
4	Grok 4.7	0.4365	$22.86	23.1M	216k	4h 0m	130	3
5	Grok 4.7	0.4219	$34.89	31.9M	250k	3h 58m	135	11

MEG Speech Decoding

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Grok 4.7	0.1697	$27.46	25.8M	199k	3h 25m	133	5
2	Grok 4.7	0.0000	$17.01	16.9M	171k	3h 5m	110	2
3	Grok 4.7	0.0000	$33.74	29.5M	210k	5h 34m	157	5
4	Grok 4.7	0.0000	$9.82	11.5M	148k	1h 49m	85	2
5	Grok 4.7	0.0000	$32.99	32.5M	199k	5h 31m	181	4

Wan 2.1 on MAX/Mojo

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Grok 4.7	0.9226	$177.41	173.6M	635k	11h 45m	724	2
2	Grok 4.7	0.9093	$209.09	202.2M	818k	13h 1m	844	2
3	Grok 4.7	0.9053	$77.01	77.8M	250k	3h 37m	315	1
4	Grok 4.7	0.0000	$218.86	185.5M	568k	19h 57m	791	8
5	Grok 4.7	0.0000	$85.96	87.7M	287k	4h 37m	388	1

MS/MS De Novo Generation

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Grok 4.7	0.1109	$12.56	12.3M	170k	1h 49m	85	2
2	Grok 4.7	0.1098	$130.86	121.5M	481k	15h 47m	498	4
3	Grok 4.7	0.0968	$55.01	43.9M	294k	8h 31m	190	6
4	Grok 4.7	0.0896	$57.29	42.5M	256k	5h 56m	203	6
5	Grok 4.7	0.0000	$47.01	35.3M	253k	6h 31m	169	4

Multi-GPU Efficient Finetuning

task page →5 runs · indexed 2026-09-22

#	Model	Cost	In tok	Out tok	Time	Steps	Submits
1	Grok 4.7	$60.21	36.3M	189k	18h 34m	184	2
2	Grok 4.7	$37.89	24.1M	164k	18h 10m	140	2
3	Grok 4.7	$29.88	18.4M	159k	14h 59m	119	3
4	Grok 4.7	$36.18	19.8M	164k	20h 0m	126	5
5	Grok 4.7	$59.65	35.5M	185k	17h 44m	170	3

Notebook Compression

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Grok 4.7	0.2584	$211.20	205.8M	904k	13h 1m	859	13
2	Grok 4.7	0.2464	$38.17	37.7M	250k	3h 43m	192	8
3	Grok 4.7	0.2422	$40.60	41.8M	253k	4h 6m	221	11
4	Grok 4.7	0.2221	$20.96	23.3M	213k	3h 13m	149	5
5	Grok 4.7	0.2015	$41.06	38.3M	268k	3h 17m	188	5

Flight-Sim Renderer in OpenGL

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Grok 4.7	0.1336	$487.10	480.1M	1.3M	20h 1m	1837	77
2	Grok 4.7	0.1182	$509.72	506.8M	1.5M	18h 52m	2068	85
3	Grok 4.7	0.1138	$588.60	597.8M	1.8M	19h 55m	2409	106
4	Grok 4.7	0.0713	$540.56	547.5M	1.5M	19h 14m	2247	102
5	Grok 4.7	0.0196	$518.13	515.0M	1.7M	19h 40m	2079	58

Optimizer Design

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Grok 4.7	0.2867	$182.37	175.0M	483k	20h 0m	744	15
2	Grok 4.7	0.2033	$279.55	260.9M	639k	20h 0m	949	17
3	Grok 4.7	0.0119	$55.61	43.7M	222k	7h 51m	195	7
4	Grok 4.7	0.0000	$308.24	304.6M	520k	20h 0m	1215	1
5	Grok 4.7	0.0000	$72.65	66.7M	251k	9h 7m	265	9

PostgreSQL 18 on SQLite

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Grok 4.7	0.2747	$697.58	718.7M	2.2M	19h 51m	2979	27
2	Grok 4.7	0.2637	$767.04	780.4M	2.4M	18h 58m	3117	30
3	Grok 4.7	0.2527	$961.79	986.1M	3.0M	20h 0m	3909	26
4	Grok 4.7	0.1978	$1004.11	1036.8M	3.4M	20h 0m	4217	22
5	Grok 4.7	0.0000	$890.38	903.6M	3.1M	20h 0m	3660	12

Quantum ESPRESSO pw.x in Rust

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Grok 4.7	0.6179	$248.05	251.2M	1.1M	11h 32m	1086	16
2	Grok 4.7	0.4553	$566.99	570.1M	2.1M	20h 0m	2281	1
3	Grok 4.7	0.2764	$412.96	420.2M	1.8M	14h 39m	1779	17
4	Grok 4.7	0.1138	$593.38	588.6M	2.1M	20h 0m	2376	1
5	Grok 4.7	0.0650	$449.65	416.6M	1.1M	20h 0m	1517	11

Qubit Routing

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Grok 4.7	0.7346	$28.62	27.0M	233k	2h 28m	119	7
2	Grok 4.7	0.6849	$35.29	35.5M	255k	1h 35m	157	10
3	Grok 4.7	0.6735	$227.27	226.0M	738k	19h 38m	925	75
4	Grok 4.7	0.6585	$21.77	22.6M	209k	1h 19m	122	7
5	Grok 4.7	0.6417	$24.01	24.2M	210k	1h 51m	131	4

Reconnaissance Blind Chess Recovery

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Grok 4.7	0.8571	$107.31	95.1M	427k	11h 56m	400	26
2	Grok 4.7	0.7821	$110.91	86.8M	288k	10h 45m	306	21
3	Grok 4.7	0.7250	$43.26	41.6M	230k	3h 58m	188	3
4	Grok 4.7	0.5893	$172.69	157.2M	497k	15h 0m	627	13
5	Grok 4.7	0.5607	$45.18	44.4M	217k	3h 31m	197	8

Fitness-Recap Video in Remotion

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Grok 4.7	0.0961	$439.29	445.6M	1.3M	9h 16m	1738	103
2	Grok 4.7	0.0840	$675.93	699.4M	1.7M	12h 52m	2790	89
3	Grok 4.7	0.0491	$750.66	786.8M	2.0M	15h 22m	3238	130
4	Grok 4.7	0.0481	$814.19	833.8M	1.7M	19h 58m	3225	109
5	Grok 4.7	0.0422	$931.62	946.8M	1.9M	20h 0m	3560	153

Snooker Prediction

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Grok 4.7	0.0086	$81.17	79.8M	486k	3h 8m	332	5
2	Grok 4.7	0.0000	$342.95	338.0M	1.2M	19h 47m	1370	39
3	Grok 4.7	0.0000	$282.03	285.1M	1.2M	19h 51m	1191	49
4	Grok 4.7	0.0000	$61.25	64.2M	387k	4h 4m	335	5
5	Grok 4.7	0.0000	$79.86	85.0M	390k	6h 4m	384	13

SPICE Circuit Simulator in Rust

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Grok 4.7	0.7576	$721.50	746.3M	2.8M	15h 40m	3094	16
2	Grok 4.7	0.7172	$780.09	794.5M	2.5M	20h 0m	3280	18
3	Grok 4.7	0.7071	$747.81	772.9M	3.1M	17h 23m	3267	17
4	Grok 4.7	0.6970	$807.64	821.4M	3.2M	17h 39m	3364	44
5	Grok 4.7	0.6061	$843.32	872.9M	3.2M	19h 50m	3636	16

Synthetic Music Diarization

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Grok 4.7	0.4597	$72.67	61.2M	249k	8h 6m	249	4
2	Grok 4.7	0.4458	$66.85	49.2M	265k	14h 13m	205	10
3	Grok 4.7	0.4406	$208.39	177.1M	399k	19h 30m	612	4
4	Grok 4.7	0.4284	$43.87	32.2M	197k	17h 17m	180	1
5	Grok 4.7	0.4074	$96.18	79.9M	265k	13h 36m	307	8

Vision-only TORCS Racing Bot

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Grok 4.7	0.0456	$436.54	434.7M	834k	19h 58m	1732	29
2	Grok 4.7	0.0000†	$129.68	133.6M	655k	14h 50m	603	11
3	Grok 4.7	0.0000†	$87.58	84.0M	222k	5h 40m	303	8
4	Grok 4.7	0.0000†	$51.73	43.4M	216k	3h 56m	182	5
5	Grok 4.7	0.0000†	$353.91	323.1M	770k	19h 55m	1185	13

† score adjusted by manual review; the trace's Judge tab carries the measured score and the rationale.

Verilog Simulator in Swift

task page →5 runs · indexed 2026-09-22

#	Model	Reward	Cost	In tok	Out tok	Time	Steps	Submits
1	Grok 4.7	0.8853	$200.79	205.0M	994k	5h 44m	853	1
2	Grok 4.7	0.8784	$244.06	246.8M	1.1M	15h 21m	1005	27
3	Grok 4.7	0.8596	$839.06	861.8M	2.3M	19h 58m	3507	37
4	Grok 4.7	0.8510	$180.74	176.1M	878k	5h 58m	658	4
5	Grok 4.7	0.8476	$334.11	341.6M	1.2M	11h 19m	1457	47