Synthetic Music Diarization

Music diarization asks "who or what is sounding, and when": for instrumental audio that means transcribing every note — instrument label, MIDI pitch, onset, and offset — and for vocal audio it means segmenting which singer is active over time. Polyphonic transcription is a long-standing hard problem: notes overlap across instruments with similar timbres, onsets must be localized to within a fraction of a second, and singer identity must be inferred without any enrollment data.

This task combines both problems in one offline command-line tool evaluated across four source datasets (BabySlakh, URMP, MusicNet, and vocadito) whose manifests are deliberately anonymized — IDs and filenames are opaque, so the diarizer cannot shortcut by recognizing which dataset a clip came from. It must genuinely analyze the audio, on CPU only, within a hard batch time limit.

Build an offline diarizer for short instrumental-music and solo-vocal WAV clips. For clips in the instrument_note family it must emit timestamped instrument-note events with integer MIDI pitches; for singer_segment clips it must emit singer-identity activity segments. All runtime files live in /app/diarizer.

• The entrypoint must support python /app/diarizer/diarize.py --input-dir /path/to/songs --output /path/to/predictions.jsonl, reading a songs.jsonl manifest and writing exactly one JSON object per input ID.
• Instrument events carry a label from a fixed 27-instrument vocabulary plus a MIDI note; singer events use labels singer_a through singer_l.
• Production batches may contain up to 233 clips, must finish within 900 seconds, and must produce at most 8 MiB of JSONL.
• Missing, extra, or duplicate IDs invalidate the output, and labels must not be inferred from IDs or file paths — the manifests are intentionally opaque.

Labeled development data under /app/datain three splits — train (with gzipped labels), val, and public— comprising over 11 hours of WAV audio derived from BabySlakh, URMP, and MusicNet (instrument-note clips) and vocadito (solo-vocal singer-segment clips), all resampled excerpts with per-file hash-validated integrity. The interface and label schemas are documented in /app/README.md, and a starter stub lives at /app/diarizer/diarize.py. The preinstalled Python 3.13 stack is CPU-only: torch and torchaudio 2.8.0, librosa, soundfile, scipy, scikit-learn, and numpy. The machine is offline with no GPU; a sandbox-timer CLI reports the remaining budget.

A root-owned clean-room verifier stages a hidden 233-clip replay set drawn from the same task families, deletes the visible /app/data, and runs the submitted diarizer as the unprivileged agentuser under a 900-second timeout — with stracemonitoring for any attempt to write the verifier's reward artifacts. Before staging, a manifest-privacy check confirms the replay exposes no source metadata.

• An instrument note matches only with equal label and MIDI note, onset error at most 0.12 s, and interval IoU at least 0.35; singer segments match with equal label and IoU at least 0.35.
• Each family is scored by event F1 blended with temporal-boundary quality; instrument-note transcription is weighted 75% and singer segmentation 25%, and a gate scales the instrument contribution down until its component reaches 0.20.
• A contract multiplier penalizes invalid events and zeroes the score for structural errors (missing, extra, or duplicate IDs).
• A crashed or timed-out diarizer, oversized output, or an attempted reward-file write zeroes the result; submission sources are also scanned for suspicious references to verifier paths.