FFmpeg libswscale Optimization

libswscale is FFmpeg's image scaling and pixel-format conversion library: the code that turns YUV into RGB, packs and unpacks planes, and resamples frames, in some of the hottest loops in video processing. Re-implementing it means reproducing the numerics of decades-old scalar C (coefficient tables, rounding, chroma subsampling) closely enough to pass per-plane PSNR bars, across dozens of format pairs and three scaling algorithms, and then making it substantially faster.

The rewrite must be genuinely new work: it is written in Zig behind a small custom C ABI, FFmpeg's own code cannot be wrapped, linked, loaded, or embedded, and inline assembly is banned; the speed has to come from portable SIMD (Zig's @Vector) and better algorithms, not from recalling or repackaging the original.

Build libswscale_candidate.so from the sources in /app/swscale-impl/, exporting the three C-linkage functions in /app/swscale_api.h (swscale_create / swscale_process / swscale_destroy), covering ten pixel formats and three scaling algorithms.

• The correctness contract is 42 same-size conversions plus 16 scaling configurations (58 in all), checked plane by plane against the FFmpeg scalar reference: PSNR of at least 60 dB for same-size conversion and 40 dB for scaling. All 58 must pass; a library fast on 57 of 58 counts the same as an empty scaffold.
• The contract fixes format pairs and algorithms, not frame sizes, so the implementation has to generalize rather than special-case the visible workloads.
• Performance is single-threaded measured work, fully modeled into /app/perf-check; the goal is beating the reference's per-workload numbers.
• Zig 0.14.0 only, no inline assembly, no dlopen/dlsym, and no FFmpeg code in any form; target x86-64-v3 (AVX2, FMA, BMI2) and no higher, since the measurement simulator has no AVX-512.
• Changes are confined to /app/swscale-impl/.

A Zig starter scaffold at /app/scaffold/zig/ that already exports the three functions, the ABI header, and /app/libswscale_public_baseline.so: FFmpeg's scalar swscale (built with --disable-asm) behind the same ABI, the reference the output is compared against and whose work is being beaten. /app/perf-check rebuilds, compares output against the unmodified reference, and measures (--contract for the full 58-conversion sweep, --quick to skip measurement, or a named workload); reference numbers are baked in /app/baseline-work.json. The /app/driver C program that loads and exercises a library is usable under gdb/perf, the measurement stack is readable at /app/performance/, and FFmpeg's libswscale/libavutil C sources are at /reference/ffmpeg-src/ for study. The driver synthesizes deterministic source pixels, so there are no image files and every run sees the same input. The machine is offline.

A clean-room verifier, separate from the agent's environment, first strips every FFmpeg source and archive from the container, rebuilds the submission from source as a non-root user, and runs the full 58-conversion contract plus ten hidden held-out benchmark workloads: the same conversion families and size classes as the public set, at dimensions and format pairs the agent has not seen. Speed is not wall-clock: each workload's work is measured by callgrind-based per-opcode instruction pricing (a deterministic cost model derived from uops.info throughput data), so the score is independent of host speed.

• Reward is gated on build success, provenance, and full correctness, then scored as 2**u - 1 where u scales the geometric-mean work reduction across held-out workloads, reaching full credit at 20x; matching the baseline or regressing scores zero.
• Provenance checks reject FFmpeg delegation in any form: linked library names are restricted to a C-runtime allowlist, RPATH/RUNPATH and dynamic-loading imports are rejected, and the built objects are scanned for FFmpeg symbols, build strings, inline assembly, and large binary blobs.
• The driver installs a seccomp filter that blocks fork/exec before the candidate loads, sources are perturbed between runs to defeat cached outputs, and iteration-identity and work-linearity gates catch skipped work.
• A crash or hang on a measured workload counts as a correctness failure; if fewer than 75% of workloads could be measured the trial is marked invalid rather than scored.