A Steeper Slope Is Not Yet a New Law: What a Claimed Architectural Change to a Scaling Exponent Has to Survive, Why Studies of Looping, Growth, Routing and Optimizers Reach Opposite Verdicts, and the Floor, Recipe and Ladder Controls That Would Decide It

@misc{mahendrakar2026steeper,
  author       = {Mahendrakar, Pranay},
  title        = {{A Steeper Slope Is Not Yet a New Law: What a Claimed Architectural Change to a Scaling Exponent Has to Survive, Why Studies of Looping, Growth, Routing and Optimizers Reach Opposite Verdicts, and the Floor, Recipe and Ladder Controls That Would Decide It}},
  year         = {2026},
  month        = oct,
  date         = {2026-10-28},
  publisher    = {Life of Research},
  howpublished = {Preprint},
  doi          = {10.5281/zenodo.23735381},
  url          = {https://lifeofresearch.com/preprints/163},
  abstract     = {(c) 2026 Pranay Mahendrakar. Licensed under CC BY 4.0. Life of Research A long-standing working rule of language-model scaling holds that architectural changes move the constant of a scaling law and leave its exponent alone, so that a better design buys a fixed factor of compute at every budget rather than a factor that compounds. A September 2026 study reports the opposite for three interventions on looped transformers - a normalise-and-reinject boundary operator, depth growth during training, and their combination - with compute multipliers over a standard transformer that rise across two decades of compute, and a concurrent study reports a steeper frontier for a looped mixture-of-experts model. Another 2026 study finds that recurrence loses to unique depth at matched compute by a margin that does not close, and a third finds that looping pays at a fixed parameter count without showing an advantage once size and data are optimised for compute. In this analysis the author reviews and synthesises published results to show that these verdicts are not yet answers to one question, because a reported exponent is indexed by five choices that the studies make differently: the axis compute is counted on, how the irreducible floor is fitted, whether each arm has its own training recipe, which shape and allocation rule the ladder follows, and how wide the fitted window is. The paper derives that a lower floor and a steeper exponent both produce a multiplier that rises with scale over a finite window, and that fitting every arm against the baseline's floor reads the first as the second by construction. It shows from the anchor study's own ablations that the boundary operator's exponent gain disappears under the baseline's recipe, and that when executed depth is held fixed its gain in the two-pass pair (Untied-2 against Deep Vanilla) becomes a constant, so that the measured exponent belongs to a ladder and a recipe as much as to a block; the corresponding controls for growth are not reported. It finds the same protocol dependence in two precedents, routing and optimizers, where the trend with scale changed between studies, in the routing case from shrinking to widening. The paper consolidates 24 published results in a table, maps eleven designs onto the five indices, states a classification procedure, and specifies the four controls that would separate a new exponent from a new floor.},
  keywords     = {scaling laws, scaling exponents, looped transformers, model growth, compute-optimal training, irreducible loss, hyperparameter transfer, mixture of experts, pretraining},
  language     = {en},
  copyright    = {Creative Commons Attribution 4.0}
}