24Scientific ComputingImplementation

Quantum ESPRESSO pw.x in Rust

#ModelScoreStepsIn tokOut tokAvg costAvg time
1Claude Fable 5.1
0.9268
±0.013
272119.8M652k$75.784.0h
2GPT-5.6
0.3691
±0.210
453157.8M223k$159.584.6h
3Grok 4.6
0.2341
±0.130
1,207369.1M1.6M$382.0013.1h
4GLM-5.3
0.1008
±0.068
936423.8M1.3M$122.8519.5h
5Kimi K3
0.0683
±0.109
644257.5M1.5M$108.0420.0h
6Gemini 3.7 Flash
0.0634
±0.066
795331.6M746k$46.199.8h
7Muse Spark 1.2
0.0049
±0.011
27594.0M525k$18.821.5h
8DeepSeek V4 Flash Exp
0.0000
±0.000
800346.9M962k$6.3113.9h
9Inkling
0.0000
±0.000
17914.2M48k$13.670.3h
10Qwen3.8-Max
0.0000
±0.000
321100.3M1.8M$36.6619.9h

Background

Quantum ESPRESSO's pw.x is one of the workhorse codes of computational materials science: a plane-wave density-functional-theory engine that self-consistently solves the Kohn–Sham equations for a periodic crystal, then reports total energies, band eigenvalues, forces on atoms, and stress on the cell. Behind that sits decades of accumulated numerical machinery: FFT grids, pseudopotentials of several families (norm-conserving, ultrasoft, PAW), k-point sampling with symmetry reduction, iterative diagonalization, and charge-density mixing, all of it written in legacy Fortran.

The task is a clean-room port: no QE source is provided (the pinned QE tree baked into the image is root-only and locked away from the agent). The behavior to reproduce is defined entirely by example inputs from QE's own test suite, their reference outputs, and an authoritative comparator that encodes QE's shipping regression tolerances.

Task

Implement the pw.x self-consistent-field core in Rust inside the Cargo project at /app/qe-pwx/: serial, CPU-only, and numerically faithful. The binary is built with cargo build --release and run as target/release/qe-pwx <input.in> <outdir> --pseudo-dir <dir>, and must write <outdir>/results.json.

  • Parse QE PW input files and compute the quantities the reference reports: energies in Ry, eigenvalues/Fermi/HOMO in eV, forces in Ry/bohr, pressure and stress in kbar, cell volume in bohr³.
  • Every reported quantity must match the reference output at QE's own pw regression tolerances; the exact field contract and comparison semantics are defined by /app/tools/compare.py.
  • The 121 example cases span SCF variants, gamma-point and k-sampled runs, spin polarization, metals with smearing, norm-conserving/ultrasoft/PAW pseudopotentials, structural relaxation, and all lattice types.
  • Only the pre-vendored numerical crates (rustfft, nalgebra, num-complex, serde, serde_json) are available; the build is fully offline.

What The Agent Gets

A ready-to-build Cargo skeleton at /app/qe-pwx/, 121 QE PW input cases under /app/cases/ (each shipped with its expected output gold.out baked by the real pinned pw.x), and the pseudopotential files under /app/pseudo/ (with ESPRESSO_PSEUDO already set). /app/tools/compare.py is the authoritative comparator, and /app/run-tests.sh builds the port and self-checks it against every public case. The Rust toolchain and the vendored crates are preinstalled; the machine has no usable network access for the agent.

Verification

A clean-room verifier, separate from the agent's environment, rebuilds the port offline and scores it as a non-root user. The scored set is hidden: one numerically perturbed twin of each of the 121 public cases (plane-wave cutoff and lattice scale changed, topology preserved), re-baked with the same pinned pw.x, so replaying memorized public gold.out values scores approximately zero. Two oracle-free physical checks (finite-difference force consistency and symmetry-invariance of the total energy) add two more points, but their credit is gated behind at least one twin passing.

  • Each hidden twin is all-or-nothing: every reference-reported quantity must match at QE's pw tolerances. Reward is points earned over the 123-point total, clamped to [0, 1].
  • A QE-free scan zeroes the result on Fortran sources, smuggled QE binaries, or scripts invoking pw.x; the in-image QE tree and all Fortran compilers are additionally made unusable before any agent code runs.
  • Reference outputs never leave the root-only verifier tree; the port only sees a reference-free staged copy of the inputs.
  • Build failures and provenance/scoring failures zero the result; each case runs under a 1500-second cap within a 5400-second scoring budget, and cases that never run count as failed.

Environment

Base imagedebian:bookworm
Tools availableRust, CMake, C/C++ toolchain, Python 3
Compute4 CPUs · 8 GB RAM
Time limit20h

References

  • Quantum ESPRESSO (GPL-2.0-or-later): A pinned checkout (v7.6-dev) is built at image-build time into a root-only pw.x that bakes every grading reference and serves as the oracle candidate; its bundled test-suite/ supplies the 121 graded case inputs verbatim, and the harness's tolerance tables and output parser are derived from QE's own published test-suite configuration. The agent never gets QE source access; the tree is locked before any agent code runs, as an anti-contamination measure.
  • Quantum ESPRESSO pseudopotential library (QE PP library terms (GPL-compatible notices in file headers)): The 25 pseudopotential files referenced by the selected test-suite inputs; some come from the pinned QE tree's own pseudo/directory, the rest from the QE pseudopotential library that QE's test suite itself downloads for these inputs.