34Implementation

Wan 2.1 on MAX/Mojo

#ModelScoreStepsIn tokOut tokAvg costAvg time
1Claude Fable 5.1
0.9246
±0.007
27996.7M337k$56.477.4h
2GPT-5.6
0.9017
±0.021
28173.9M127k$65.974.3h
3Kimi K3
0.7866
±0.281
34399.3M417k$39.109.2h
4Grok 4.6
0.6431
±0.359
452108.5M1.1M$119.4612.8h
5GLM-5.3
0.5702
±0.467
450155.9M420k$54.8417.5h
6DeepSeek V4 Flash Exp
0.3511
±0.089
552213.1M423k$3.6617.8h
7Qwen3.8-Max
0.2491
±0.207
443109.6M1.4M$40.4419.7h
8Gemini 3.7 Flash
0.0331
±0.002
520185.3M609k$31.938.5h
9Inkling
0.0228
±0.021
16114.9M29k$14.281.3h
10Muse Spark 1.2
0.0115
±0.016
29794.1M425k$21.153.3h

Background

Wan 2.1 T2V-1.3B is a text-to-video diffusion model (a diffusion transformer paired with a 3D VAE and a text encoder) whose inference pipeline spans text encoding, iterative denoising, and video decoding on the GPU. Porting it off PyTorch means re-expressing every model component on a different ML stack: Modular's MAX graph and inference APIs, with custom Mojo kernels where the built-in operations fall short.

The port is measured against the original: the agent's pipeline must reproduce the PyTorch reference's output frames to high pixel fidelity, working entirely offline from the reference source, the pretrained weights, and the bundled MAX API documentation. PyTorch itself is not installed; the target stack is the only stack available.

Task

Implement Wan 2.1 T2V-1.3B text-to-video generation on Modular's MAX/Mojo stack. The deliverable is the Python package at /app/wan21_max/, whose entrypoint module wan_pipeline.py must export generate_video(prompt, height, width, num_frames, num_steps, seed) returning a list of PIL Images, one per frame.

  • The signature stays stable and everything derives from the arguments; sources are .py modules (plus .mojo kernels for custom ops) under /app/wan21_max/.
  • Everything the pipeline needs at generation time must live in the package: /app/reference/ and the .py files under /app/weights/ are removed before scoring.
  • The port must be built on the MAX SDK; torch, transformers, and diffusers are hard library constraints: they are not installed and must not be used.
  • A short generation (5 frames, 4 steps) must complete within 10 minutes including any first-call compilation.

What The Agent Gets

The package scaffold at /app/wan21_max/ and an /app/README.md spelling out the exact API, the workload family, what counts as correct output, and the library constraints. For porting: the Wan 2.1 PyTorch reference source at /app/reference/ (read-only), the pretrained T2V-1.3B weights in diffusers format at /app/weights/, and the Modular MAX API documentation at /app/max_docs/. For self-checking: sample workloads with baked reference frames under /app/examples/ and /app/verify_correctness.py, which generates each sample workload, compares output against the reference frames, and enforces the per-generation time budget. The machine has one H100 GPU, is offline, ships the MAX SDK and Mojo toolchain with warmed compile caches, and exposes a sandbox-timer CLI for the remaining budget.

Verification

A clean-room verifier runs in a fresh container on the captured workspace. It reconstructs the scored package from a pristine scaffold plus only the agent's wan21_max/ files into a root-owned, read-only directory, strips the reference source and stray Python files as disclosed, then runs a gate sequence: a source-level anti-cheat verdict (banned imports, verifier-internals references, shell or dynamic-import reach, and a required MAX SDK import), an import check, and the disclosed smoke gate. Only then does it generate frames for nine hidden workloads as a non-root user. Root then scores the saved frames against root-only reference frames; no candidate code runs during scoring.

  • Per workload, validity gates require the exact frame count and size, no symlinked frames (which would otherwise echo back the references), and no blank frames; credit is then a continuous ramp on mean per-frame PSNR: 0 at 10 dB, 1.0 at the disclosed 25 dB bar.
  • The reward is the geometric mean of per-workload credits, with a small 0.02 per-workload floor so a single hard workload cannot zero an otherwise strong port.
  • Banned-library imports, a missing MAX import, an unimportable entrypoint, or a failed smoke test (crash, timeout, or blank frames) zero the result; workloads that miss the generation stage's start deadline degrade to partial credit rather than erroring the run.

Environment

Base imagenvidia/cuda:12.8.1-devel-ubuntu22.04
Tools availableCUDA, MAX SDK, Mojo, CMake, C/C++ toolchain, Python 3
Compute8 CPUs · 128 GB RAM
GPU1× H100
Time limit20h

Changes From V1

This task is an improved revision of Wan 2.1 on MAX/Mojo from FrontierSWE V1.

  • Scoring changed from a geometric-mean speedup against a PyTorch baseline (gated on an all-or-nothing 25 dB PSNR check where one failed workload zeroed everything) to a pure-correctness reward: a geometric mean of continuous per-workload PSNR credits ramping from 10 dB to the disclosed 25 dB bar, with a small per-workload floor so partial ports earn partial credit.
  • The PyTorch ban became structural: V1 scanned .py files for banned imports; V2 uninstalls torch, transformers, and diffusers from the image entirely (verified in preflight), so the constraint cannot be routed around at runtime.
  • Verification moved to a clean-room separate container with privilege separation: the scored package is reconstructed from a pristine scaffold plus only the agent's wan21_max/ files into a root-owned read-only directory, candidate code runs de-rooted, reference frames stay root-only, and the scorer rejects symlinked frames.
  • The smoke gate grew from 120 seconds to 10 minutes for the short generation, explicitly covering first-call Mojo compilation, and the Mojo compile cache was made agent-writable after measuring that a root-owned cache silently cost 20-30 seconds per cold call; the generation stage now degrades to partial credit at its start deadline instead of failing hard.
  • The agent budget grew from 4 hours to 20 hours, and both the agent and verifier images are pinned by content hash, with the verifier additionally pinned to the same H100 GPU class that the PSNR scoring runs on.
  • The deliverable was restructured from a single /app/submission/candidate_pipeline.py into the /app/wan21_max/ package with the full contract moved into /app/README.md, and the /app/.timer file interface was replaced by the standard sandbox-timer CLI.

References

  • Wan 2.1 (Apache-2.0): The PyTorch reference implementation, cloned at image build to /app/reference/ for the agent to read while porting the pipeline to MAX.
  • Wan 2.1 T2V-1.3B weights (diffusers format) (Apache-2.0): The pretrained weights the candidate pipeline loads, fetched at image build and exposed at /app/weights. The same weights, run through the diffusers WanPipeline, generated the baked reference frames used for scoring and the visible examples.
  • diffusers / transformers / PyTorch (Apache-2.0 (diffusers, transformers); BSD-3-Clause (PyTorch)): The stock PyTorch WanPipeline stack the port is measured against; it generated the baked reference frames out-of-band at pinned versions. The stack is not installed in the task image: the deliverable must not use it, and the ban is structural (absent, with no egress to fetch it).
  • Modular MAX / Mojo SDK (MAX Platform / Community License): The MAX/Mojo toolchain the task targets, pip-installed from public PyPI at image build (not vendored or modified), plus its API documentation fetched from the public docs.modular.com endpoints into /app/max_docs/.