Vision-only TORCS Racing Bot

Driving a race car from pixels alone is a classic hard problem in embodied AI: the policy must infer speed, track position, and upcoming curvature from a single forward camera (no speedometer, no track sensors, no HUD) and turn that into steering, throttle, and brake commands fast enough to hold a racing line. Doing it on tracks the policy has never seen rules out memorizing any one circuit; the solution has to generalize.

The task builds on a patched, headless build of the TORCS racing simulator with a lock-step control loop: the engine blocks for the bot's action each step and runs faster than real time, so the agent can roll out thousands of episodes to hand-tune a controller, collect data, or train a model; the approach is entirely open.

The agent must implement act(obs) in /app/bot.py: a policy that takes the camera observation and returns driving controls, completing laps as fast as it can, including on tracks it has not seen.

• The only input is obs["frame"], a 480×640 RGB forward road camera image; speed, track position, and lap progress must be inferred from the pixels.
• The bot returns steer in [-1, 1] plus accel and brake in [0, 1]; gear and clutch are automatic.
• The camera publishes a new image every second control step (and the first frames are black), so motion estimation from pixels must account for the frame cadence.
• An episode runs up to 21,000 control steps (~430 s of sim time); a crawling car reaches the step limit before finishing a lap. Each control step must return within ~30 seconds, including the first (model loading counts).
• Module-level state and one-time setup (e.g. loading a trained model) are allowed; the policy may be hand-written, learned, or anything in between.

A workspace at /app with the bot.py stub, a Gymnasium-style harness (game_harness) that runs the sim headless with reset/step/restart, a runner (python3 /app/run_bot.py), a minimal demo (harness_demo.py), a sample camera frame, and /app/README.md documenting the observation/control interface and episode rules. Five practice tracks are available in the sandbox (g-track-1, eroad, ruudskogen, aalborg, alpine-1); the evaluation tracks are not among them. The image preinstalls numpy, scipy, scikit-image, Pillow, OpenCV, and torch with torchvision, and the box has an A10G GPU for training. The machine is offline.

A clean-room verifier in a separate container restores the harness from a root-only pristine copy (keeping the agent's own files), unlocks two held-out tracks that never appear in the agent workspace (corkscrew and street-1) and races the bot 3 times on each, de-rooted as the non-root agent user with pixels-only observations. Scoring reads only root-written telemetry from the verifier's own harness, never anything the agent wrote.

• A completed lap maps to a reward by piecewise-linear interpolation over five per-track lap-time anchors: 1.0 for superhuman pace, 0.8 for a skilled human, 0.6 for a published imitation-learning bot, 0.4 for this build's heuristic oracle, and 0.2 for barely finishing within the step budget. Street-1's 0.8 and 0.6 anchors come from Muñoz, Gutiérrez and Sanchis, IEEE CIG/ITW 2010. Corkscrew has no published record: 0.8 is a 90s lap by Akshit, and 0.6 is extrapolated from the paper's human-to-bot gap. The 0.4 rung is measured on this exact build.
• A DNF scores at most 0.2 times the fraction of the track covered, always below the completion floor. The final reward is the mean over runs, then over the two tracks.
• A bot that fails to answer within the 30-second per-step cap is declared dead; the engine and all agent processes are killed before scoring. Harness faults mark the trial invalid rather than scoring the agent zero.