Multi-GPU Efficient Finetuning

Finetuning a 14-billion-parameter model on two T4 GPUs is a memory-bound exercise: the frozen base weights alone dwarf a single card's memory, so any training run has to combine parameter-efficient methods, sharding or offloading, and careful precision choices just to fit, and then still make efficient use of both devices inside a fixed wall-clock budget.

The target capability is exact-answer mathematical reasoning: competition-style problems where only a correctly boxed final answer counts. The agent must decide how to spend a hard 20-hour budget across data selection, training configuration, and validation, and leave behind a loadable adapter no matter when the clock runs out.

The agent must improve the exact-answer mathematical reasoning of the provided frozen Qwen3-14B model by producing an offline-trained PEFT adapter. All work happens under /app/math_adapter, and /app/math_adapter/train.sh must be an executable entrypoint that trains and exports the adapter.

• The exported adapter/adapter_model.safetensors and its native PEFT configuration must load offline against the read-only base model at /models/qwen3-14b.
• Only native methods supported by PEFT 0.9 are allowed; rank-based methods may use rank at most 64, and the serialized adapter is capped at 300,000,000 tensor elements and 1.25 GiB of safetensors weights.
• Frozen-base tensors, saved embeddings or other full modules, modules_to_save, custom auto-mapping, pickle-capable weights, and symlinks are all forbidden in the adapter.
• A run_summary.json with method_name, devices, total_elapsed_seconds, and checkpoint_path must accompany the adapter, and a usable checkpoint must remain if training ends early.
• Hard-coding problem-specific answers, caching canned responses, and modifying /models or /app/data are prohibited.

The frozen Qwen3-14B base model at /models/qwen3-14b (materialized from a pinned Hugging Face revision) and visible training data at /app/data/train.jsonl (math reasoning traces from the DAPO-MATH-17k-oss-reasoning dataset), both read-only. The workspace ships a placeholder train.sh and a README documenting the exact submission contract.

The machine is offline (Hugging Face hub access is disabled at the image level); everything needed is preinstalled, including PyTorch with CUDA 12.4, transformers 4.57.6, PEFT 0.9.0, accelerate, bitsandbytes, datasets, and lm-eval. A sandbox-timer CLI reports the remaining wall-clock budget.

A separate clean-room verifier, running on different hardware than the agent ever sees, first validates the submission against the bounded adapter contract, then loads the adapter onto the integrity-checked frozen base and evaluates it on a sealed, root-only panel of 60 held-out AIME problems from 2025 and 2026, both past the base model's training cutoff.

• Each problem gets exactly three sampled generations under a frozen contract (bfloat16, temperature 0.7, top-p 0.95, fixed seed, 16,384 max tokens) and is scored best-of-3 on exact-answer match, using boxed-answer normalization adapted from the lm-evaluation-harness.
• The score is count-calibrated: the measured frozen-base performance (18 of 60 correct) maps to 0 and a perfect 60 of 60 maps to 1.0, with a constant increment per additional correct answer.
• Model execution happens in unprivileged evaluation workers sharded across GPUs; only the root parent sees the sealed labels, and evidence files are schema-checked before scoring.
• Contract violations zero the result: an oversized or non-native adapter, smuggled base tensors or full modules, pickle-capable weights, a missing run_summary.json, or an adapter that fails to load offline.