libexpat Optimization

libexpat is a widely used stream-oriented XML parser: callers register handlers, feed a document whole or in arbitrary chunks, and receive parse events in order. Reproducing it in hand-written assembly means matching not just well-formed parsing but libexpat's exact observable behaviour: the same events in the same order, the same coalescing of character data, and on malformed input the same specific error code, not merely an error. Chunked feeding is where from-scratch parsers first diverge: a construct can split across calls at any byte, so the parser must carry partial state and still emit exactly what a one-shot parse would.

The parser's C implementation is withheld: the agent gets the public API headers as the specification, example documents with their expected event traces, and nothing else to copy from. The performance bar makes assembly the point rather than a gimmick: the deliverable must get through a document doing measurably less work than libexpat's own compiled C.

Ship /app/asm-port/libexpat.so, assembled from the *.s/*.S/*.asm sources in /app/asm-port/: a System V AMD64 C-ABI shared library implementing the interface in /app/tests/expat.h.

• Thirteen entry points are resolved by name, from XML_ParserCreate and the handler setters through XML_Parse, XML_GetErrorCode, and XML_ParserFree; a partial library still loads and runs.
• Every document is parsed four ways (namespace processing off and on, one-shot and chunked), and all four must agree with libexpat: same events, same order, same final status and error code.
• Correctness is all-or-nothing: one wrong document takes the result to zero however fast the rest are.
• libc may be called from assembly (malloc, memcpy, memchr, …) and is the expected way to get memory, but its work is priced like the agent's own.
• Target x86-64-v3 and no higher: the measurement simulator has no AVX-512, and an unmeasurable parse is no better than a wrong one.
• Changes are confined to /app/asm-port/.

The ABI headers (/app/tests/expat.h, expat_external.h), example documents at /app/tests/corpus/ with the expected event traces for all four modes at /app/tests/expected/, and the C source of the workers that produce and measure those traces. /app/build-lib.sh assembles and links the deliverable (nasm/as/ld, no C-compilation step anywhere); /app/run-tests.sh diffs the parser's events against libexpat's on every example document; /app/perf-check measures what one parse costs against the reference numbers baked in /app/baseline-work.json, over the benchmark documents in /app/bench/. The measurement stack itself is readable at /app/performance/, and /app/workloads.py defines the workload space: nine document shapes, any size in the 64–192 KB band, any of the four modes. The machine is offline, with nasm, as, ld, gdb, and valgrind preinstalled.

A clean-room verifier, separate from the agent's environment, assembles the submission from its assembly sources alone and replays it over a root-only scored corpus drawn deterministically from the W3C XML Conformance Test Suite, including mutated twin documents, so a table of memorized traces is not an implementation. Every scored unit (a document in one of the four parse modes) must hash-match the trace of the reference libexpat 2.6.4 build; one wrong unit zeroes the result, with the unit pass rate kept only as a diagnostic.

• Once fully correct, the reward is 2**u - 1 where u scales the geometric mean of reference-to-candidate work ratios, reaching full credit at 7x; doing no better than libexpat scores zero.
• Work is whole-process instruction-priced measurement (callgrind-based per-opcode throughput pricing plus a capped branch-mispredict term, no cache term). It is deterministic, so one run per change is enough and host speed does not matter.
• Anti-cheat gates: no withheld expat C files in the deliverable, hard-failing source scans for dlopen/dlsym, Python/pyexpat delegation, and .incbin; an ELF import check restricted to libc; a preloaded guard that refuses loading a foreign XML library; and a no-exec seccomp filter during measured parses. Large binary files under /app are read as a parser carried in as data.
• A work-linearity gate and a native wall-time audit catch submissions that skip priced iterations; if fewer than 75% of documents could be measured, the trial is marked invalid rather than scored.