We now hold the top record on a public ML speedrun with an 80.8× improvement found by an autonomous agent.
Oneirology-1 is a system of safe, reliable infrastructure, reinforcement-learning data collection, and an agent harness: an autonomous agent system that proposes, runs, and evaluates machine-learning and physics research against verifiable benchmarks. We also use the resulting trajectories to gather training data and roll out newer models against them. Agents now do machine-learning research on public leaderboards: the top two entries on the nanoTabPFN speedrun were both found by agents, and both say so. Oneirology-1 holds record 10 at 0.92 minutes, an 80.8× improvement over the leaderboard's 74.32-minute starting point, reached over 95 days with no human-written code. A follow-on systems patch the same harness found, a further 17.7% speedup, is currently pending review upstream.
Oneirology-1 also mines trajectories from autoresearch across many agents and turns them into a scientific exploration and research gym for agents. We compiled the metrics and data we gathered in this speedrun process at the end of this report.
autoresearch HPO loop plus human intervention; record 10
is ours, found with an autonomous agent harness. Both say so in their own record files. Our point is a prediction
for their hardware, not a measurement on it.The nanoTabPFN speedrun is a public leaderboard in the modded-nanogpt tradition: one L40S, train a tabular foundation model that beats Random Forest on TabArena, go faster. Ten records, 74.32 minutes down to 0.92 over 95 days.
The metric being lowered is wall-clock training time, in minutes, to reach a fixed target: a tabular foundation model that beats Random Forest's accuracy on TabArena, on one pinned L40S. It is verifiable on the leaderboard's own terms: every record is a pinned commit plus a training log, and a submission is not accepted until the maintainers re-run the exact code on the reference machine and confirm the same number. A record is not a claim until someone else's hardware agrees with it.
Three systems changes against a pinned upstream base: batched Newton-Schulz, a producer-thread dataloader, fused SDPA. No hyperparameter change, no algorithm change.
median −17.7% · Wilcoxon p = 0.00018 · epochs-to-target unchanged, MWU p = 0.615
| quantity | value |
|---|---|
| median wall time, 1×L40S, n=15 paired | 1.032 → 0.873 min, -17.7% |
| Wilcoxon signed-rank | p = 0.00018, 14/15 wins |
| second host, 1×5090, n=20 | 0.650 → 0.530 min, -18.5%, 20/20 |
| steady-state seconds per epoch | 0.870 → 0.750, -13.8%, p = 0.00057 |
| epochs to target | unchanged, Mann-Whitney p = 0.615 |
The design point that makes this defensible: the absolute baseline varies 0.86 to 1.01 seconds per epoch across containers, while the within-container delta is +12.8% to +25.7% in every single one. The absolute is a property of the machine. The delta is a property of the patch. An unpaired version of this measurement would be worthless, and most published speedups are unpaired.
We also decline the obvious claim. We do not reproduce their 0.92 minutes - our host is roughly 12% slower per epoch. We publish a prediction of 0.76 to 0.79 minutes on the record node and invite re-timing. The single round where our patch lost is reported in full, with its cause.
oneirology-propose: a 115 kB wheel, zero dependencies for the verifier, 101 items,
with a recorded install transcript. Clean directory, clean virtualenv, wheel installed, and:
No absolute path is read at run time, the scorer produces identical numbers under an empty environment
(env -i), and it reproduces the certifier's numbers to four decimal places while sharing no code
with it. A trivial agent scores argmax 0.4158 end to end with no GPU and no API key.
Not “a task description.” The incumbent train.py verbatim, shipped by sha256 with its
text beside the hash, plus: the K candidate patches as unified diffs, measured val_bpb and
num_steps per arm, the state's own measured noise floor, step_regime,
difficulty_band, pass_band, the measured top-2 gap, the wrong-pick cost, the ledger depth,
and the container identity the measurement came from.
101 items, 302 arms, 55 distinct incumbent scripts, built 2026-08-19T19:50:44Z from
CERT_MINTED.json at sha 89e5d1d5….
difficulty_band is the top-2 gap over that state's own floor:
| band | rule | n |
|---|---|---|
| unresolvable | ≤1× floor | 19 |
| near_floor | 1–2× | 19 |
| resolved | 2–5× | 33 |
| wide_margin | >5× | 30 |
| floor used | items resolvable |
|---|---|
| the item's own measured floor | 81.2% |
| step-adjusted twin floor | 77.2% |
| raw twin floor | 70.3% |
| type | count | usable as |
|---|---|---|
| menus (root + K measured candidates) | 262 | ranking benchmark, RL item |
| measured candidate arms | 785 | label + cost scale |
| mined research states | 3,018 | trajectory corpus |
| scorable decision points (≥2 measured alternatives) | 514 | sequential RL item |
| chains of ≥2 consecutive scorable points | 222 | multi-turn credit assignment |
| scored RL rollouts (plan, ranking, reward) | 2,064 | offline RL data |
| byte-identical twin pairs | 53 | the instrument floor |
Every arm carries hypothesis, expected_delta_bpb and expected_num_steps -
the agent's own forecast, recorded before the measurement. That makes this a calibration corpus and
not only a ranking one.
Concretely: 262 menus (root plus measured candidates) and 785 measured candidate arms form the ranking layer; 3,018 mined research states are the raw trajectory corpus; 514 scorable decision points and the 222 chains of two-or-more consecutive ones turn that corpus into sequential RL items with multi-turn credit assignment; 2,064 scored RL rollouts (plan, ranking, reward) are the offline RL data itself; and 53 byte-identical twin pairs measure the floor of our own instrument. This is what Oneirology-1 has gathered so far, and it is the layer we plan to keep growing across new domains and new agent harnesses.
Next steps: we are releasing v2 rollouts on Qwen 3.8 27B, pushing the frontier of what a single-GPU-sized model can do. If you want autoresearch environments, MLE task samples, or are interested in research collaborations in physics, contact Abhi and Tom.