{"id":109,"identifier":"LOR-00109","url":"https://lifeofresearch.com/articles/109","title":"Which Signal Did the Synthesis Import? Information, Re-presentation and Selection in Synthetic Pretraining, Why 'the Verifier Is Doing the Work' Holds Only Where Correctness Is the Target, and the Controls That Would Assign the Credit","abstract":"(c) 2026 Pranay Mahendrakar. Licensed under CC BY 4.0.\nThe expectation that language models will keep improving after the supply of human text runs out rests on a claim about synthetic data: that a model, or a family of models, can produce training text worth more than the text it was trained on. A competing reading holds that no synthetic-data success is self-generated at all, because every pipeline that works draws on something outside the generating model, most visibly a verifier. In this analysis the author reviews and synthesises published results on synthetic data for pretraining, continued pretraining and mid-training, and argues that both readings are posed at the wrong grain. Read as a conservation law, the imported-signal claim is true and uninformative, since every generator's weights were themselves fitted to human text. Read as a claim about what explains a measured gain, it splits. The paper separates three contributions a synthetic corpus can make - new information, re-presentation of information already in an organic seed, and selection within the generator's own distribution - and four places the signal can come from: the seed text, the generator's prior, a selection signal, and the designer's prompts or grammars. Against that ledger, the best-controlled pretraining gains are mostly re-presentation: they beat repetition of the same corpus, they persist when the generator is small, and summarisation of web pages nearly matches teacher-written textbooks. There is no verifier in those pipelines. The verifier does the work where the target is correctness, in geometry, code and arithmetic. The most self-contained bootstrapping results use held-out human text as the scorer. No located pretraining result shows a gain from self-generation with none of the four sources present, and the pipelines that come closest score below the real-data baseline on factuality. The paper consolidates the reported values, states a credit-assignment protocol, and names the five controls that would settle the open part.","authors":[{"name":"Pranay Mahendrakar","affiliation":"SONYTECH","orcid":"0009-0003-7224-029X"}],"type":"article","license":"CC-BY-4.0","keywords":["synthetic data","language model pretraining","model collapse","data-constrained scaling","rephrasing","verification","self-improvement","credit assignment"],"publication_date":"2026-10-14","version":1,"views":7,"downloads":7,"created_at":"2026-10-15 08:25:59","doi":"10.5281/zenodo.23353594","source":"Zenodo","source_url":"https://zenodo.org/records/23353594","files":[{"id":111,"name":"which-signal-did-synthesis-import.pdf","size":394447,"mime":"application/pdf","downloads":7,"download_url":"https://lifeofresearch.com/articles/109/files/111/which-signal-did-synthesis-import.pdf"}]}