30Visual ReasoningImplementation

Stepper Music Sequencer GBA

#ModelScoreStepsIn tokOut tokAvg costAvg time
1Claude Fable 5.1
0.9443
±0.023
1,853819.9M1.7M$451.3015.8h
2GPT-5.6
0.6671
±0.113
1,430525.2M644k$532.839.3h
3GLM-5.3
0.3947
±0.172
1,923914.2M1.9M$250.6614.3h
4Grok 4.6
0.3262
±0.142
1,319342.9M2.2M$346.5610.1h
5Kimi K3
0.2844
±0.173
1,376579.8M1.8M$213.3719.9h
6Gemini 3.7 Flash
0.1791
±0.043
1,456558.2M1.2M$52.614.4h
7DeepSeek V4 Flash Exp
0.0751
±0.046
2,8611261.3M1.9M$20.5312.8h
8Qwen3.8-Max
0.0162
±0.010
1,027394.1M2.8M$123.4319.9h
9Muse Spark 1.2
0.0035
±0.008
35690.6M672k$30.143.7h
10Inkling
0.0000
±0.000
1577.3M29k$7.060.3h

Background

The Game Boy Advance generates music through four programmable sound generator (PSG) channels driven directly by hardware registers (square waves, a programmable wavetable, and a noise channel), with every parameter change landing on an exact frame boundary. A step sequencer built on this hardware is a dense knot of state: pattern grids, per-channel parameter editors, sound banks, tempo logic, and a playback engine whose audible output depends on register-level synthesis details and frame-accurate timing.

This task turns a real, shipped GBA homebrew tracker into a clean-room reverse-engineering exercise. The reference ROM is never readable by the agent: it lives root-only inside the sandbox and can only be observed through a probe tool that runs keystroke scripts against it and returns what it drew and played. Everything the clone must reproduce has to be inferred from those observations.

Task

The agent must reimplement the 4-channel PSG step sequencer from scratch as its own GBA ROM: the exact same screens, pixels, and controls, and above all the exact same sound and timing. The source lives in /app (starting from src/main.c) and make -C /app must produce /app/tracker.gba.

  • Observe the reference as a black box with ref-probe <script.txt> <out_dir>, which captures screenshots (shot_*.png), audio traces (listen_*.npy), and frame-sequence recordings with an .mp4 preview for any keystroke script the agent writes.
  • Match the UI pixel for pixel across pattern grids, parameter editors, sound banks, and transport screens.
  • Match the PSG audio synthesis and frame-accurate sequencer timing: the audible output, not just the visuals.
  • Iterate with ./run-tests.sh, which builds the ROM and reports, per script, how close its frames and audio are to the reference.

What The Agent Gets

A ready-to-build GBA project at /app: a Makefile, a starter src/main.c, sample keystroke scripts in /app/scripts/, the ref-probe client, capture-and-compare tooling in /app/tools/ (the same capture code the verifier uses), and /app/README.md documenting the script format and workflow. The reference ROM itself is runnable but never readable; only its captured frames and audio are observable. The image preinstalls the devkitARM GBA toolchain, the mGBA emulator with Python bindings, and an audio-analysis stack (librosa, scipy, soundfile). The machine is offline.

Verification

A clean-room verifier in a separate container rebuilds the clone from captured source (make -C /app, run as the non-root agent user) and captures both the reference and the clone on a hidden suite of keystroke scripts: seeded, full-range variants of the public samples that sweep every scale, parameter, sound bank, tempo, and sustained 30-60 second songs. The reward is 0.25 × visual + 0.75 × audio, deliberately weighted toward the harder dimension.

  • Visual closeness is an exponential of per-pixel MSE on every captured frame; a bit-identical clone scores 1.0. Audio closeness combines log-mel spectral distance, RMS level ratio, and trace length ratio, with symmetric silence gating so a silent clone scores zero on audible events.
  • A self-check captures the reference twice and requires both dimensions to agree above 0.999; a failed self-check marks the trial invalid as a harness fault rather than scoring the agent zero.
  • A failed build or missing ROM zeroes the result, and tracker.gba is rejected if it is a symlink or resolves outside /app; the clone is fed to the emulator purely as data, with no agent process alive during capture.

Environment

Base imagedevkitpro/devkitarm:latest
Tools availableFFmpeg, C/C++ toolchain, Python 3
Compute8 CPUs · 8 GB RAM
Time limit20h

References

  • STEPPER (by Bad Diode) (MIT): The black-box reference ROM (v1.9, 96 KiB). It is baked root-only into the verifier side of the image and never enters the agent workspace; the agent observes it only through ref-probe captures. No STEPPER or Bad Diode branding appears in any agent-visible surface, and no bytes of the ROM reach the graded deliverable.
  • mGBA (MPL-2.0): Version 0.10.5, built from source into the image (headless core plus Python bindings); used by both the agent for probing and the verifier for capture and grading. Fetched at image build, not redistributed in the task repository.
  • devkitPro / devkitARM (GPL/LGPL (libgba: LGPL with static-linking exception)): The GBA homebrew toolchain base image (arm-none-eabi GCC and binutils plus libgba) that both the agent and the verifier build with. Pulled at image build, not redistributed in the task repository.