| # | Model | Score | Steps | In tok | Out tok | Avg cost | Avg time |
|---|---|---|---|---|---|---|---|
| 1 | Claude Fable 5.1 | 0.3423 ±0.097 | 601 | 263.8M | 554k | $144.63 | 11.1h |
| 2 | GPT-5.6 | 0.1419 ±0.046 | 844 | 318.0M | 342k | $318.24 | 10.7h |
| 3 | Grok 4.6 | 0.0319 ±0.014 | 1,232 | 332.0M | 1.8M | $375.13 | 19.9h |
| 4 | Kimi K3 | 0.0129 ±0.010 | 1,046 | 460.4M | 1.3M | $165.73 | 19.9h |
| 5 | GLM-5.3 | 0.0053 ±0.007 | 763 | 330.5M | 671k | $99.46 | 19.4h |
| 6 | Gemini 3.7 Flash | 0.0052 ±0.004 | 379 | 99.7M | 270k | $16.38 | 5.2h |
| 7 | DeepSeek V4 Flash Exp | 0.0038 ±0.003 | 1,424 | 627.2M | 1.0M | $10.35 | 18.4h |
| 8 | Qwen3.8-Max | 0.0009 ±0.001 | 1,258 | 500.9M | 1.6M | $169.09 | 19.8h |
| 9 | Muse Spark 1.2 | 0.0000 ±0.000 | 146 | 24.9M | 183k | $5.30 | 2.1h |
| 10 | Inkling | 0.0000 ±0.000 | 146 | 6.8M | 29k | $6.60 | 1.1h |
A real-time 3D engine is a tower of interlocking systems: mesh and texture loading, camera projection, physically-based shading, shadows, reflections, particles, and, in a flight simulator, a physics integrator whose state feeds every frame. Reproducing one pixel for pixel means getting every layer exactly right at once: a slightly wrong lift coefficient or gamma curve diverges within frames, and the error compounds over a 40-second sortie.
The task fixes one world (a hillside airfield) and defines the target purely by behavior: a reference renderer turns keystroke input logs into videos, and the agent must build an engine whose output is indistinguishable from it. The reference is runnable on any log the agent writes and dumps its per-tick state, so the physics coefficients and shading math are identifiable by experiment, not given.
The agent must build its own renderer plus flight simulator from scratch in /app/src so that make -C /app produces /app/render, and its frames match the reference's output pixel for pixel on any input log over the declared event set.
/app/world.json, with all content in /app/assets/; a scene in /app/scenes/ is a plain-text input log where keys are pressed and released at 240 Hz ticks and every 8th tick is a video frame.reference-renderer world.json <scene.txt> <out_dir>, and /app/run_tests.py [name ...] renders both sides scene by scene and reports the difference.A workspace at /app with the fixed world blueprint (world.json), the full asset pack (meshes, PBR texture sets, an HDR environment), sample scene logs, a Makefile, the run_tests.py comparison harness with an MSE tool, and /app/README.md, the contract documenting the file formats, CLI, input-log grammar, and the event/camera/world vocabulary, but not the physics coefficients or shading math. The runnable reference renderer dumps its per-tick state for any log the agent writes. The image preinstalls a C++ toolchain (g++, make, cmake), the software OpenGL stack (OSMesa/llvmpipe), ffmpeg, and NumPy. The machine is offline.
A clean-room verifier in a separate container rebuilds the submission with make -C /appas the non-root agent user, restores a pristine copy of the assets and world blueprint from a root-only mirror, and renders a hidden scored corpus of 12 seeded scenes spanning 10 weighted families (missions, aerobatics, landings, night takeoffs, payload drops, a ray-traced pond pass, camera tours, stills, and effects). Each scene's score is exp(-0.004 × MSE) over the per-pixel error across every frame; the reward averages within each family, then weights across families.
| Base image | debian:bookworm-slim |
| Tools available | FFmpeg, OSMesa/llvmpipe, CMake, C/C++ toolchain, Python 3 |
| Compute | 4 CPUs · 8 GB RAM |
| Time limit | 20h |