Traces

Every run: the agent's full trajectory, its submissions over the 20-hour budget, and the final score.

Kolmogorov Audio Compression

task page →5 runs · indexed 2026-09-02
#ModelRewardCostIn tokOut tokTimeStepsSubmits
1DeepSeek V4 Flash Vision Exp0.0268$2.15112.2M397k7h 43m3893
2DeepSeek V4 Flash Vision Exp0.0228$5.13304.7M583k13h 30m7127
3DeepSeek V4 Flash Vision Exp0.0094$6.17373.9M627k11h 53m7989
4DeepSeek V4 Flash Vision Exp0.0081$3.17180.3M434k7h 52m5542
5DeepSeek V4 Flash Vision Exp0.0056$3.93222.7M553k14h 38m5307

Astronomy Toolkit

task page →5 runs · indexed 2026-09-02
#ModelRewardCostIn tokOut tokTimeStepsSubmits
1DeepSeek V4 Flash Vision Exp0.0000$4.02226.3M587k18h 16m5035
2DeepSeek V4 Flash Vision Exp0.0000$4.14229.6M637k20h 0m4831
3DeepSeek V4 Flash Vision Exp0.0000$3.11165.3M552k9h 15m4096
4DeepSeek V4 Flash Vision Exp0.0000$3.44189.2M548k10h 29m4534
5DeepSeek V4 Flash Vision Exp0.0000$3.67205.7M547k18h 16m5103

Cranelift Codegen Optimization

task page →5 runs · indexed 2026-09-02
#ModelRewardCostIn tokOut tokTimeStepsSubmits
1DeepSeek V4 Flash Vision Exp0.0237$8.27504.6M723k9h 18m126011
2DeepSeek V4 Flash Vision Exp0.0107$5.01303.8M439k7h 50m7238
3DeepSeek V4 Flash Vision Exp0.0030$5.08311.7M419k8h 3m7816
4DeepSeek V4 Flash Vision Exp0.0000$3.68220.8M344k8h 44m6708
5DeepSeek V4 Flash Vision Exp0.0000$4.87296.6M422k6h 43m7303

Dart Style in Haskell

task page →5 runs · indexed 2026-09-02
#ModelRewardCostIn tokOut tokTimeStepsSubmits
1DeepSeek V4 Flash Vision Exp0.4014$18.121122.6M1.6M19h 51m260223
2DeepSeek V4 Flash Vision Exp0.2342$7.03436.2M602k8h 13m9143
3DeepSeek V4 Flash Vision Exp0.2234$21.231304.8M2.0M15h 38m28367
4DeepSeek V4 Flash Vision Exp0.1638$15.31956.5M1.3M19h 4m20233
5DeepSeek V4 Flash Vision Exp0.0766$15.69953.5M1.5M16h 48m21995

FFmpeg libswscale Optimization

task page →5 runs · indexed 2026-09-02
#ModelRewardCostIn tokOut tokTimeStepsSubmits
1DeepSeek V4 Flash Vision Exp0.0668$8.52493.3M1.0M6h 44m10958
2DeepSeek V4 Flash Vision Exp0.0636$9.11528.7M1.1M6h 10m11725
3DeepSeek V4 Flash Vision Exp0.0602$13.77800.3M1.6M9h 14m183119
4DeepSeek V4 Flash Vision Exp0.0483$9.39554.1M1.0M5h 54m119716
5DeepSeek V4 Flash Vision Exp0.0000$12.34718.0M1.4M8h 32m167816

Crash-Proof Flash Filesystem

task page →5 runs · indexed 2026-09-02
#ModelRewardCostIn tokOut tokTimeStepsSubmits
1DeepSeek V4 Flash Vision Exp0.0110$29.121761.2M2.9M19h 19m391124
2DeepSeek V4 Flash Vision Exp0.0053$22.611343.5M2.5M19h 20m286726
3DeepSeek V4 Flash Vision Exp0.0049$20.651205.0M2.5M19h 56m249819
4DeepSeek V4 Flash Vision Exp0.0049$17.941097.8M1.6M18h 58m234814
5DeepSeek V4 Flash Vision Exp0.0049$20.781251.0M2.1M19h 4m280415

FrogsGame Post-Training

task page →5 runs · indexed 2026-09-02
#ModelRewardCostIn tokOut tokTimeStepsSubmits
1DeepSeek V4 Flash Vision Exp0.1100$1.2765.0M240k20h 4m3061
2DeepSeek V4 Flash Vision Exp0.0860$1.4779.6M227k19h 17m3902
3DeepSeek V4 Flash Vision Exp0.0160$1.3368.9M247k19h 56m3452
4DeepSeek V4 Flash Vision Exp0.0160$1.1355.9M221k19h 37m2642
5DeepSeek V4 Flash Vision Exp0.0000$1.6187.5M249k20h 2m3951

Stepper Music Sequencer GBA

task page →5 runs · indexed 2026-09-02
#ModelRewardCostIn tokOut tokTimeStepsSubmits
1DeepSeek V4 Flash Vision Exp0.1348$19.481192.0M1.8M11h 51m26707
2DeepSeek V4 Flash Vision Exp0.1138$28.551772.4M2.4M15h 1m383323
3DeepSeek V4 Flash Vision Exp0.0534$18.461143.5M1.6M13h 42m270016
4DeepSeek V4 Flash Vision Exp0.0419$18.931164.8M1.7M11h 44m276429
5DeepSeek V4 Flash Vision Exp0.0319$17.221033.6M1.8M11h 35m230613

Git to Zig

task page →5 runs · indexed 2026-09-02
#ModelRewardCostIn tokOut tokTimeStepsSubmits
1DeepSeek V4 Flash Vision Exp0.1304$28.181759.3M2.2M19h 24m372315
2DeepSeek V4 Flash Vision Exp0.1213$28.061756.1M2.2M19h 43m379311
3DeepSeek V4 Flash Vision Exp0.1193$26.161630.6M2.1M19h 37m35848
4DeepSeek V4 Flash Vision Exp0.1079$21.831364.0M1.7M17h 17m29778
5DeepSeek V4 Flash Vision Exp0.0988$20.501272.8M1.7M19h 17m273211

Granite Mamba2 Inference Optimization

task page →5 runs · indexed 2026-09-02
#ModelRewardCostIn tokOut tokTimeStepsSubmits
1DeepSeek V4 Flash Vision Exp0.8708$2.68149.8M376k16h 23m4735
2DeepSeek V4 Flash Vision Exp0.7876$3.53195.8M540k19h 32m5198
3DeepSeek V4 Flash Vision Exp0.7578$8.78501.2M1.2M13h 5m11573
4DeepSeek V4 Flash Vision Exp0.7467$8.27472.9M1.1M11h 58m114023
5DeepSeek V4 Flash Vision Exp0.4680$3.00159.1M520k6h 30m4425

Higgs Uncertainty Inference

task page →5 runs · indexed 2026-09-02
#ModelRewardCostIn tokOut tokTimeStepsSubmits
1DeepSeek V4 Flash Vision Exp0.9597$1.5170.8M347k3h 23m2834
2DeepSeek V4 Flash Vision Exp0.7308$2.17110.7M423k3h 32m3749
3DeepSeek V4 Flash Vision Exp0.0718$2.02102.8M397k15h 29m3464
4DeepSeek V4 Flash Vision Exp0.0700$1.5070.3M352k6h 13m2906
5DeepSeek V4 Flash Vision Exp0.0559$1.3262.9M305k6h 32m2784

SGLang Inference System Optimization

task page →5 runs · indexed 2026-09-02
#ModelRewardCostIn tokOut tokTimeStepsSubmits
1DeepSeek V4 Flash Vision Exp0.0000$2.44138.4M299k9h 52m5314
2DeepSeek V4 Flash Vision Exp0.0000$5.46332.4M492k10h 53m78215
3DeepSeek V4 Flash Vision Exp0.0000$3.59206.9M443k9h 31m59316
4DeepSeek V4 Flash Vision Exp0.0000$3.41198.5M360k8h 1m5624
5DeepSeek V4 Flash Vision Exp0.0000$3.54209.2M364k8h 32m6329

Lean 4 Kernel Type Checker in Pascal

task page →5 runs · indexed 2026-09-02
#ModelRewardCostIn tokOut tokTimeStepsSubmits
1DeepSeek V4 Flash Vision Exp0.3599$12.71725.7M1.6M10h 38m16144
2DeepSeek V4 Flash Vision Exp0.3164$9.48551.4M1.2M7h 58m12288
3DeepSeek V4 Flash Vision Exp0.2513$13.44768.2M1.8M13h 43m17607
4DeepSeek V4 Flash Vision Exp0.0143$12.56761.2M1.2M12h 33m162735
5DeepSeek V4 Flash Vision Exp0.0000$11.64694.6M1.3M11h 22m141333

libexpat Optimization

task page →5 runs · indexed 2026-09-02
#ModelRewardCostIn tokOut tokTimeStepsSubmits
1DeepSeek V4 Flash Vision Exp0.0891$17.041034.3M1.6M6h 42m22736
2DeepSeek V4 Flash Vision Exp0.0545$16.46992.0M1.7M6h 43m222211
3DeepSeek V4 Flash Vision Exp0.0000$13.51827.8M1.3M5h 43m172013
4DeepSeek V4 Flash Vision Exp0.0000$21.931323.6M2.2M8h 50m30957
5DeepSeek V4 Flash Vision Exp0.0000$33.361965.0M3.8M19h 31m410248

Lua Native Compiler

task page →5 runs · indexed 2026-09-02
#ModelRewardCostIn tokOut tokTimeStepsSubmits
1DeepSeek V4 Flash Vision Exp0.9788$23.061383.4M2.4M13h 19m291121
2DeepSeek V4 Flash Vision Exp0.2005$21.271250.2M2.4M15h 16m267617
3DeepSeek V4 Flash Vision Exp0.0000$11.39676.4M1.3M7h 45m150715
4DeepSeek V4 Flash Vision Exp0.0000$22.431385.4M2.0M11h 31m28602
5DeepSeek V4 Flash Vision Exp0.0000$18.341059.5M2.2M17h 1m23839

Machine-Learned Interatomic Potential

task page →5 runs · indexed 2026-09-02
#ModelRewardCostIn tokOut tokTimeStepsSubmits
1DeepSeek V4 Flash Vision Exp0.9173$3.12179.4M413k18h 25m5586
2DeepSeek V4 Flash Vision Exp0.6906$1.8093.9M332k18h 0m3786
3DeepSeek V4 Flash Vision Exp0.4712$1.0554.4M192k19h 29m3422
4DeepSeek V4 Flash Vision Exp0.2438$1.5984.5M269k19h 1m3895
5DeepSeek V4 Flash Vision Exp0.0562$2.23120.5M374k18h 41m4363

Medium-Range Weather Forecast

task page →5 runs · indexed 2026-09-02
#ModelRewardCostIn tokOut tokTimeStepsSubmits
1DeepSeek V4 Flash Vision Exp0.4639$1.5583.0M253k19h 5m4035
2DeepSeek V4 Flash Vision Exp0.0000$1.6690.1M258k19h 29m3923
3DeepSeek V4 Flash Vision Exp0.0000$1.99110.4M289k18h 13m4874
4DeepSeek V4 Flash Vision Exp0.0000$1.5584.2M243k18h 42m3985
5DeepSeek V4 Flash Vision Exp0.0000$1.6893.8M242k19h 27m4522

MEG Speech Decoding

task page →2 runs · indexed 2026-09-02
#ModelRewardCostIn tokOut tokTimeStepsSubmits
1DeepSeek V4 Flash Vision Exp0.1749$1.3268.7M240k18h 36m3596
2DeepSeek V4 Flash Vision Exp0.0826$1.7393.5M273k12h 25m4572

Wan 2.1 on MAX/Mojo

task page →5 runs · indexed 2026-09-02
#ModelRewardCostIn tokOut tokTimeStepsSubmits
1DeepSeek V4 Flash Vision Exp0.4396$4.92297.3M464k19h 11m68111
2DeepSeek V4 Flash Vision Exp0.4384$3.53204.1M419k17h 55m5623
3DeepSeek V4 Flash Vision Exp0.3550$2.99168.7M386k19h 32m4533
4DeepSeek V4 Flash Vision Exp0.2720$3.61208.7M436k14h 28m56211
5DeepSeek V4 Flash Vision Exp0.2506$3.27186.8M413k17h 5m4935

MS/MS De Novo Generation

task page →5 runs · indexed 2026-09-02
#ModelRewardCostIn tokOut tokTimeStepsSubmits
1DeepSeek V4 Flash Vision Exp0.0000$1.89103.3M298k19h 46m4353
2DeepSeek V4 Flash Vision Exp0.0000$2.64142.6M438k12h 9m4745
3DeepSeek V4 Flash Vision Exp0.0000$2.54140.9M373k17h 33m4735
4DeepSeek V4 Flash Vision Exp0.0000$2.76151.1M447k15h 16m5036
5DeepSeek V4 Flash Vision Exp0.0000$2.75153.9M409k18h 4m5102

Multi-GPU Efficient Finetuning

task page →5 runs · indexed 2026-09-02
#ModelRewardCostIn tokOut tokTimeStepsSubmits
1DeepSeek V4 Flash Vision Exp0.3333$0.8236.7M207k19h 58m2381
2DeepSeek V4 Flash Vision Exp0.1429$0.9241.7M226k19h 50m2512
3DeepSeek V4 Flash Vision Exp0.0238$0.6629.1M168k19h 41m2262
4DeepSeek V4 Flash Vision Exp0.0000$0.7637.0M151k19h 59m2751
5DeepSeek V4 Flash Vision Exp0.0000$1.2360.8M248k19h 49m3231

Notebook Compression

task page →5 runs · indexed 2026-09-02
#ModelRewardCostIn tokOut tokTimeStepsSubmits
1DeepSeek V4 Flash Vision Exp0.2139$1.97101.8M387k13h 26m4201
2DeepSeek V4 Flash Vision Exp0.2028$2.58144.1M374k19h 46m5082
3DeepSeek V4 Flash Vision Exp0.1948$3.01169.4M449k11h 0m5514
4DeepSeek V4 Flash Vision Exp0.1939$2.93164.0M435k14h 31m5264
5DeepSeek V4 Flash Vision Exp0.1818$2.40126.7M415k10h 56m4216

Flight-Sim Renderer in OpenGL

task page →5 runs · indexed 2026-09-02
#ModelRewardCostIn tokOut tokTimeStepsSubmits
1DeepSeek V4 Flash Vision Exp0.0069$10.96663.9M1.1M19h 2m15272
2DeepSeek V4 Flash Vision Exp0.0065$10.60633.4M1.1M15h 56m15168
3DeepSeek V4 Flash Vision Exp0.0028$12.40756.6M1.2M19h 8m175639
4DeepSeek V4 Flash Vision Exp0.0027$6.77415.2M638k18h 28m8377
5DeepSeek V4 Flash Vision Exp0.0000$11.01666.7M1.1M19h 11m146711

Optimizer Design

task page →5 runs · indexed 2026-09-02
#ModelRewardCostIn tokOut tokTimeStepsSubmits
1DeepSeek V4 Flash Vision Exp0.1123$1.1252.3M255k18h 23m2562
2DeepSeek V4 Flash Vision Exp0.0770$1.2263.5M222k19h 1m3342
3DeepSeek V4 Flash Vision Exp0.0348$1.4475.8M242k19h 19m3492
4DeepSeek V4 Flash Vision Exp0.0000$1.5179.4M262k19h 14m4062
5DeepSeek V4 Flash Vision Exp0.0000$1.3365.8M275k19h 51m3131

PostgreSQL 18 on SQLite

task page →5 runs · indexed 2026-09-02
#ModelRewardCostIn tokOut tokTimeStepsSubmits
1DeepSeek V4 Flash Vision Exp0.1484$22.501379.3M2.0M19h 53m31461
2DeepSeek V4 Flash Vision Exp0.1264$22.611383.3M2.0M19h 51m32723
3DeepSeek V4 Flash Vision Exp0.1044$19.661214.6M1.7M19h 25m26889
4DeepSeek V4 Flash Vision Exp0.0659$23.231428.3M2.0M19h 44m33481
5DeepSeek V4 Flash Vision Exp0.0165$8.18489.5M856k20h 0m11942

Quantum ESPRESSO pw.x in Rust

task page →5 runs · indexed 2026-09-02
#ModelRewardCostIn tokOut tokTimeStepsSubmits
1DeepSeek V4 Flash Vision Exp0.0000$7.62422.1M1.1M12h 5m9126
2DeepSeek V4 Flash Vision Exp0.0000$6.14336.4M959k16h 29m85310
3DeepSeek V4 Flash Vision Exp0.0000$4.17227.6M660k6h 35m4473
4DeepSeek V4 Flash Vision Exp0.0000$6.64359.4M1.1M12h 22m8825
5DeepSeek V4 Flash Vision Exp0.0000$6.97389.0M1.0M14h 29m8875

Qubit Routing

task page →5 runs · indexed 2026-09-02
#ModelRewardCostIn tokOut tokTimeStepsSubmits
1DeepSeek V4 Flash Vision Exp0.6244$2.53133.2M457k10h 50m4027
2DeepSeek V4 Flash Vision Exp0.5485$2.27120.4M400k9h 24m4053
3DeepSeek V4 Flash Vision Exp0.4883$3.26177.3M519k9h 8m4267
4DeepSeek V4 Flash Vision Exp0.4768$2.33127.1M362k12h 24m4074
5DeepSeek V4 Flash Vision Exp0.4348$2.35127.0M382k9h 0m3957

Reconnaissance Blind Chess Recovery

task page →5 runs · indexed 2026-09-02
#ModelRewardCostIn tokOut tokTimeStepsSubmits
1DeepSeek V4 Flash Vision Exp0.1786$2.02107.0M337k14h 34m3282
2DeepSeek V4 Flash Vision Exp0.1214$2.99164.7M448k8h 44m4235
3DeepSeek V4 Flash Vision Exp0.0357$2.71145.6M448k14h 17m3777
4DeepSeek V4 Flash Vision Exp0.0143$2.49130.6M440k11h 4m40010
5DeepSeek V4 Flash Vision Exp0.0036$2.74145.9M463k9h 28m3726

Fitness-Recap Video in Remotion

task page →5 runs · indexed 2026-09-02
#ModelRewardCostIn tokOut tokTimeStepsSubmits
1DeepSeek V4 Flash Vision Exp0.1621$10.73658.6M994k19h 13m165118
2DeepSeek V4 Flash Vision Exp0.0738$6.53405.2M565k19h 26m99410
3DeepSeek V4 Flash Vision Exp0.0483$6.64411.0M525k16h 0m98321
4DeepSeek V4 Flash Vision Exp0.0324$12.46767.8M1.1M18h 4m178924
5DeepSeek V4 Flash Vision Exp0.0121$4.01242.4M395k12h 19m77716

Snooker Prediction

task page →5 runs · indexed 2026-09-02
#ModelRewardCostIn tokOut tokTimeStepsSubmits
1DeepSeek V4 Flash Vision Exp0.0000$4.83277.6M638k5h 11m64413
2DeepSeek V4 Flash Vision Exp0.0000$3.23179.6M479k19h 58m4661
3DeepSeek V4 Flash Vision Exp0.0000$3.55189.8M629k6h 0m4853
4DeepSeek V4 Flash Vision Exp0.0000$4.28248.4M549k14h 16m5938
5DeepSeek V4 Flash Vision Exp0.0000$5.35314.7M642k14h 37m7093

SPICE Circuit Simulator in Rust

task page →5 runs · indexed 2026-09-02
#ModelRewardCostIn tokOut tokTimeStepsSubmits
1DeepSeek V4 Flash Vision Exp0.0000$11.65704.6M1.2M10h 40m154915
2DeepSeek V4 Flash Vision Exp0.0000$19.651192.1M2.0M18h 3m239810
3DeepSeek V4 Flash Vision Exp0.0000$20.481199.6M2.4M11h 32m25639
4DeepSeek V4 Flash Vision Exp0.0000$21.511287.0M2.2M19h 10m290413
5DeepSeek V4 Flash Vision Exp0.0000$19.681182.5M2.0M19h 8m261723

Synthetic Music Diarization

task page →5 runs · indexed 2026-09-02
#ModelRewardCostIn tokOut tokTimeStepsSubmits
1DeepSeek V4 Flash Vision Exp0.4412$1.7293.1M282k19h 4m4302
2DeepSeek V4 Flash Vision Exp0.4247$2.96173.2M352k17h 55m6174
3DeepSeek V4 Flash Vision Exp0.4065$1.7395.5M262k17h 34m4485
4DeepSeek V4 Flash Vision Exp0.3803$1.8599.9M306k18h 37m4553
5DeepSeek V4 Flash Vision Exp0.2979$2.02115.0M266k19h 32m5084

Vision-only TORCS Racing Bot

task page →5 runs · indexed 2026-09-02
#ModelRewardCostIn tokOut tokTimeStepsSubmits
1DeepSeek V4 Flash Vision Exp0.0671$3.79218.1M451k16h 2m5914
2DeepSeek V4 Flash Vision Exp0.0421$4.04235.7M470k19h 34m6055
3DeepSeek V4 Flash Vision Exp0.0178$3.85221.0M459k11h 59m5557
4DeepSeek V4 Flash Vision Exp0.0114$4.73279.3M524k19h 31m6814
5DeepSeek V4 Flash Vision Exp0.0000$2.30127.9M331k19h 43m4543

† score adjusted by manual review; the trace's Judge tab carries the measured score and the rationale.

Verilog Simulator in Swift

task page →5 runs · indexed 2026-09-02
#ModelRewardCostIn tokOut tokTimeStepsSubmits
1DeepSeek V4 Flash Vision Exp0.5582$23.111453.0M1.8M19h 40m31183
2DeepSeek V4 Flash Vision Exp0.4212$16.08998.7M1.4M20h 0m21284
3DeepSeek V4 Flash Vision Exp0.3784$22.061380.9M1.8M19h 50m299912
4DeepSeek V4 Flash Vision Exp0.3253$27.961735.1M2.4M17h 46m379822
5DeepSeek V4 Flash Vision Exp0.3065$23.641484.7M1.9M18h 25m315730