A Starting Score Is Not a Ceiling: When Accuracy Before Reinforcement Learning Predicts Accuracy After It, Why Each Proposed Replacement Sees a Different Failure, and the Axis Test That Decides Which Proxy to Trust

2026-10-23T12:19:28Z https://lifeofresearch.com/oai2

oai:lifeofresearch.com:129 2026-10-23T08:38:19Z type:preprint openaire

10.5281/zenodo.23581295 https://lifeofresearch.com/preprints/129 Pranay Mahendrakar 0009-0003-7224-029X SONYTECH A Starting Score Is Not a Ceiling: When Accuracy Before Reinforcement Learning Predicts Accuracy After It, Why Each Proposed Replacement Sees a Different Failure, and the Axis Test That Decides Which Proxy to Trust Life of Research 2026 large language models reinforcement learning with verifiable rewards supervised fine-tuning post-training checkpoint selection proxy metrics pass@k evaluation methodology Preprint Creative Commons Attribution 4.0 (c) 2026 Pranay Mahendrakar. Licensed under CC BY 4.0. Life of Research Reasoning models are usually built in two stages, supervised fine-tuning and then reinforcement learning against a verifier, and the first stage is usually judged by the benchmark accuracy of its checkpoint. An August 2026 study reports that the strategy with by far the lowest accuracy before reinforcement learning finished with the highest accuracy after it, and concludes that pre-RL accuracy is not a reliable indicator of post-RL potential. In this analysis the author reviews and synthesises published results to show that the conclusion is right about the instrument and too broad about its failure. Arithmetic on that study's own tables shows that the whole reversal comes from one candidate whose output format had been disrupted: the four format-stable strategies keep the same order before and after reinforcement learning on the nine-benchmark mean, and pass@32, the replacement proxy proposed elsewhere, ranks the disrupted candidate last as well. Across the wider record, starting accuracy ranks candidates correctly when they differ in how much fresh data they saw, and fails when they differ in repetition, homogeneity, behavioural repertoire or output format, axes along which pass@1 and reinforcement-learning readiness come apart. Each proposed replacement, pass@k at large k, held-out validation loss, behaviour and faithfulness probes, early curve fits, sees one of these failure channels and is blind to at least one other, and "post-RL accuracy" is itself indexed by the reinforcement-learning budget and readout. The paper consolidates 22 published results in a table and a figure, states an axis-first decision procedure for choosing a proxy, and specifies five studies that would settle what remains open.