Which Fine-Tuning Failed? The Reported Failure and Recovery of Fine-Tuning in Model Editing Are Findings About the Edit Protocol, Why the Breadth-First Recovery Changes What the Editor May Remember, and the Matched Comparisons That Would Assign the Credit

@misc{mahendrakar2026which,
  author       = {Mahendrakar, Pranay},
  title        = {{Which Fine-Tuning Failed? The Reported Failure and Recovery of Fine-Tuning in Model Editing Are Findings About the Edit Protocol, Why the Breadth-First Recovery Changes What the Editor May Remember, and the Matched Comparisons That Would Assign the Credit}},
  year         = {2026},
  month        = oct,
  date         = {2026-10-25},
  publisher    = {Life of Research},
  howpublished = {Preprint},
  doi          = {10.5281/zenodo.23642775},
  url          = {https://lifeofresearch.com/preprints/141},
  abstract     = {(c) 2026 Pranay Mahendrakar. Licensed under CC BY 4.0. Life of Research Fine-tuning has been treated as the weak baseline of model editing since the field's first benchmarks. A 2025 study argues that the weakness was an artefact of implementation: when edits are optimised in epochs over mini-batches instead of one at a time to convergence, and when a single, empirically selected module is the only thing tuned (an MLP down-projection for two of the three models, a whole layer for the third), fine-tuning beats every specialised editor it was compared with on editing success, and holds up at 100,000 edits on a 72-billion-parameter model. In this analysis the author reviews and synthesises published results to show that both the failure and the recovery are findings about the edit protocol rather than about fine-tuning as a method. The paper separates five axes on which editing comparisons silently differ: what the editor may keep of earlier edits, how hard each edit is optimised, which parameters move, which auxiliary data are used, and how the edited model is scored. The recovery moves on at least three axes at once. Its breadth-first pipeline revisits every earlier edit in every epoch, which in continual-learning terms is full replay. Its scorer is autoregressive decoding, under which plain one-at-a-time fine-tuning was already the strongest sequential editor in an earlier study from the same group. Its tuning location was chosen on three of the five benchmarks that make up its capability score. Read along these axes, the record since 2020 shows fine-tuning losing under single-pass, teacher-forced, single-example protocols and tying or winning under batch, augmented or replayed ones. The paper consolidates 24 published results in one table, states an attribution procedure, and specifies the matched comparisons, chiefly a streaming test with a bounded replay buffer, that would show what fine-tuning contributes on its own.},
  keywords     = {model editing, knowledge editing, fine-tuning, sequential editing, lifelong editing, continual learning, experience replay, evaluation methodology},
  language     = {en},
  copyright    = {Creative Commons Attribution 4.0}
}