07Visual ReasoningImplementation

Flight-Sim Renderer in OpenGL

#ModelScoreStepsIn tokOut tokAvg costAvg time
1Claude Fable 5.1
0.3423
±0.097
601263.8M554k$144.6311.1h
2GPT-5.6
0.1419
±0.046
844318.0M342k$318.2410.7h
3Grok 4.6
0.0319
±0.014
1,232332.0M1.8M$375.1319.9h
4Kimi K3
0.0129
±0.010
1,046460.4M1.3M$165.7319.9h
5GLM-5.3
0.0053
±0.007
763330.5M671k$99.4619.4h
6Gemini 3.7 Flash
0.0052
±0.004
37999.7M270k$16.385.2h
7DeepSeek V4 Flash Exp
0.0038
±0.003
1,424627.2M1.0M$10.3518.4h
8Qwen3.8-Max
0.0009
±0.001
1,258500.9M1.6M$169.0919.8h
9Muse Spark 1.2
0.0000
±0.000
14624.9M183k$5.302.1h
10Inkling
0.0000
±0.000
1466.8M29k$6.601.1h

Background

A real-time 3D engine is a tower of interlocking systems: mesh and texture loading, camera projection, physically-based shading, shadows, reflections, particles, and, in a flight simulator, a physics integrator whose state feeds every frame. Reproducing one pixel for pixel means getting every layer exactly right at once: a slightly wrong lift coefficient or gamma curve diverges within frames, and the error compounds over a 40-second sortie.

The task fixes one world (a hillside airfield) and defines the target purely by behavior: a reference renderer turns keystroke input logs into videos, and the agent must build an engine whose output is indistinguishable from it. The reference is runnable on any log the agent writes and dumps its per-tick state, so the physics coefficients and shading math are identifiable by experiment, not given.

Task

The agent must build its own renderer plus flight simulator from scratch in /app/src so that make -C /app produces /app/render, and its frames match the reference's output pixel for pixel on any input log over the declared event set.

  • The world is fixed and fully declared in /app/world.json, with all content in /app/assets/; a scene in /app/scenes/ is a plain-text input log where keys are pressed and released at 240 Hz ticks and every 8th tick is a video frame.
  • A deterministic flight model integrates throttle, ailerons, elevator, rudder, brakes, and flaps from a named spawn pose; the simulation must be reproduced along with the rendering.
  • Instant events switch cameras, drop payloads, fade daylight, trail smoke, and toggle a ray-traced pipeline; all of them are in scope.
  • The reference is runnable as reference-renderer world.json <scene.txt> <out_dir>, and /app/run_tests.py [name ...] renders both sides scene by scene and reports the difference.

What The Agent Gets

A workspace at /app with the fixed world blueprint (world.json), the full asset pack (meshes, PBR texture sets, an HDR environment), sample scene logs, a Makefile, the run_tests.py comparison harness with an MSE tool, and /app/README.md, the contract documenting the file formats, CLI, input-log grammar, and the event/camera/world vocabulary, but not the physics coefficients or shading math. The runnable reference renderer dumps its per-tick state for any log the agent writes. The image preinstalls a C++ toolchain (g++, make, cmake), the software OpenGL stack (OSMesa/llvmpipe), ffmpeg, and NumPy. The machine is offline.

Verification

A clean-room verifier in a separate container rebuilds the submission with make -C /appas the non-root agent user, restores a pristine copy of the assets and world blueprint from a root-only mirror, and renders a hidden scored corpus of 12 seeded scenes spanning 10 weighted families (missions, aerobatics, landings, night takeoffs, payload drops, a ray-traced pond pass, camera tours, stills, and effects). Each scene's score is exp(-0.004 × MSE) over the per-pixel error across every frame; the reward averages within each family, then weights across families.

  • The reference renders live alongside the agent's binary in the same container, into root-only directories; no expected values are hardcoded anywhere.
  • Anti-cheat: a frame is rejected if it is a symlink, resolves outside the agent's own output directory, or has the wrong size; agent processes are killed before any reference frame exists; and submissions are capped at 16 MB of source-like bytes, so baked frame dumps cannot masquerade as code.
  • A failed build zeroes the result, and a missing or short frame zeroes that scene. A failed reference render is treated as a harness defect and skipped, never blamed on the agent.

Environment

Base imagedebian:bookworm-slim
Tools availableFFmpeg, OSMesa/llvmpipe, CMake, C/C++ toolchain, Python 3
Compute4 CPUs · 8 GB RAM
Time limit20h

References

  • Light Aircraft (by weirdybeardyman) (CC0 1.0): The player aircraft mesh and livery textures. Exported to OBJ, rescaled and reoriented, control surfaces and prop split into hinged submeshes, converted to the task's PMESH format, and liveries recolored and repacked per part.
  • Cesium Milk Truck (CC BY 4.0 (with trademark limitations)): Re-themed as the airfield fuel truck. Rescaled and converted to PMESH; the Cesium wordmark band and every roundel in the vendored texture are repainted flat, so no Cesium trademark ships with the task.
  • Kenney City Kit (Industrial) (CC0 1.0): Industrial buildings, tank, and chimney meshes with their colormap texture, converted to PMESH and scaled per instance in the world blueprint.
  • ambientCG PBR texture sets (CC0 1.0): Seven 1K PBR material sets (grass, dirt, asphalt, concrete, steel, wood, metal), renamed to per-set albedo/normal/rough/ao/metal maps in the NormalGL convention.
  • Kloofendal 48d Partly Cloudy (Pure Sky), Poly Haven (CC0 1.0): The HDR sky environment by Greg Zaal (capture) and Jarod Guest (sky-only edit); the cubemap and IBL products are derived deterministically at image build.
  • stb_image.h (Public domain / MIT (dual)): Image decoding (PNG/JPG/HDR) shipped to both the reference engine and the agent workspace.