FrogsGame Post-Training

The Frog Placement Game is a Queens-style constraint puzzle: an N×N grid of N colors, on which exactly one frog must be placed in every row, every column, and every color, with no two frogs adjacent — including diagonally. Boards range from N=6 (easy) to N=13 (expert), and the game is played through a tool-call interface: the board is not included in the prompt, so the model must call get_state to discover the layout, then place, check, and revise frogs one call at a time within a 200-call budget.

Post-training a small model to do this is hard because nothing is given: no training boards are provided, so the agent must generate its own solvable boards across every difficulty tier, build a multi-turn tool-use dataset in exactly the format the verifier replays, and execute the entire fine-tuning pipeline on a single local GPU within the time budget — while producing a policy that generalizes to board sizes it may have undertrained on.

Post-train Qwen/Qwen3-8B to solve unseen Frog Placement Game boards through iterative tool use, generalizing across board sizes N=6 through N=13. The agent implements the training pipeline in /app/train.py, executes it on the local GPU, and leaves a trained adapter in place.

• The deliverable is an unmerged PEFT LoRA adapter at /app/checkpoint/adapter/ (an adapter_config.json plus adapter_model.safetensors) that targets Qwen/Qwen3-8B and has rank at most 256.
• /app/prepare.py— the immutable game engine, tool schemas, and shared prompt builder — must not be modified; training must use its build_system_prompt() and USER_MESSAGE so the format matches the verifier exactly.
• No training boards are provided: the agent must generate its own boards and verify each one is solvable before using it.
• The trained model must operate without solver or solution access at inference time, and the LoRA must not be merged into the base model.
• The pipeline must actually be executed — the adapter files must exist in the workspace before the sandbox times out.

A self-contained workspace at /app: prepare.py (game engine, tool interface, and eval harness), train.py (the editable training entry point), infer.py(a vLLM smoke test that verifies an adapter loads and generates), and a local copy of the Qwen3-8B tokenizer. The Qwen3-8B base weights are baked into the image's Hugging Face cache, and the pinned stack — Axolotl 0.18.0 for LoRA fine-tuning, vLLM 0.23.0 for local inference, PyTorch with CUDA, and PEFT — is preinstalled. Training and inference run on a dedicated A100-80GB GPU. The machine is offline; a sandbox-timer CLI reports the remaining time budget.

A clean-room verifier, running as root in a separate container, generates its own 500 hidden test boards (125 per difficulty tier), loads a pristine Qwen3-8B plus the submitted LoRA adapter in local vLLM, and replays every board through the same prompt and tool interface defined in prepare.py, with the same 200-tool-call budget. The reward is the solve rate over the fixed 500-board denominator; boards left unevaluated when the scoring deadline expires count as unsolved.

• The visible /app/prepare.pyis checked against a SHA-256 hash pin of the trusted baked copy — any modification is a contract violation that zeroes the result.
• The adapter is strictly audited before evaluation: LoRA-only tensors with matching A/B pairs, safetensors format (no adapter_model.bin), no symlinks, no DoRA, modules_to_save, or per-module rank/alpha patterns, at most 1000 files and 8 GiB total.
• A declared base model other than Qwen/Qwen3-8Bor a LoRA rank outside 1–256 zeroes the result; a missing or invalid adapter cannot be evaluated and scores zero.
• Pipeline-quality checks on train.py, the generated boards, and the checkpoint are reported as diagnostics but are excluded from the reward.