Fitness-Recap Video in Remotion

Parametric video generators (the engines behind year-in-review recaps) turn structured data into minutes of choreographed animation: scene sequencing, spring-based motion curves, particle effects, data-driven layouts, and a soundtrack whose stingers land on visual beats. Reproducing one exactly means recovering not just the look of every scene but the full mapping from input data to timing, layout, and audio mix.

The reference here is a re-themed derivative of a real production recap video, rebuilt around a wellness "year in motion" dataset. It is observable only as a black box: the agent can render it on any input it likes and inspect the resulting frames and audio, but never its source. Every visual is a deterministic function of the input data, so the behavior is fully inferable from probing.

The agent must build a parametric video generator that turns a JSON file (one person's year of fitness data) into a fully animated 1080×1080, 30 fps recap video with a soundtrack, matching the reference generator's output exactly in both video and audio.

• Probe the reference with reference-generator <input.json> <out_dir>, which writes the video as a PNG frame sequence plus an audio.wav (with --mp4 for a watchable preview and --seconds for a quick trim), on any input the agent constructs.
• Build a Remotion project at /app/generator (sources in src/) so that ./render.sh <input.json> <out_dir> reproduces the reference's frames and audio from the agent's source alone.
• Handle the full input space described by samples/schema.json: video length and scene content depend on the data, so the duration logic is part of the task.
• Compare against the reference with /app/samples/check.sh <sample> [seconds], which renders both sides and reports per-frame and audio differences.

A scaffolded Remotion project at /app/generator with render.sh wired up and the complete static asset pack already in public/ (illustration art, mascot planets, digit sprites, fonts, and the CC0 soundtrack and SFX), plus three example inputs in /app/samples/ and samples/schema.json documenting the input format and its conventions. The runnable-but-not-readable reference generator is on the PATH. The image is built on Node 22 with Remotion 4.0.501 and its renderer preinstalled from a pinned offline dependency store, along with the headless browser runtime it renders through. The machine is offline.

A clean-room verifier in a separate container removes the reference probe tool, then renders six hidden JSON inputs with the agent's ./render.sh as the non-root agent user (each render capped at 1200 s), kills every agent process, and renders the reference bundle on the same inputs into root-only directories. Per input, the score is 0.85 × visual + 0.15 × audio, and the reward is the mean over the six inputs.

• Visual closeness is an exponential of per-pixel MSE computed on every reference frame: a pixel-exact render scores 1.0, a near-miss rebuild lands orders of magnitude lower, and a blank render scores ~0. The mean is divided by whichever side rendered more frames, so missing frames count zero and extra frames dilute the score.
• Audio closeness is an exponential of the reference-normalized waveform MSE times a length ratio: identical audio scores 1, while silence, missing sound effects, or a wrong mix score ~0.
• Anti-cheat: frames and audio are rejected if they are symlinks or resolve outside the agent's own output directory, the scorer only reads files and never executes agent code, and the reward directory is locked before any agent code runs. A failed reference render marks the trial invalid as an infrastructure fault rather than zeroing the agent.