{"id":257,"identifier":"LOR-00257","url":"https://lifeofresearch.com/preprints/257","title":"A Second Sample Is Not a Second Signal: What the One-Sample Semantic Entropy Result Measures, Why the Sampling Premium Is Mostly a Disagreement About the Baseline, and the Matched Comparison That Would Settle It","abstract":"(c) 2026 Pranay Mahendrakar. Licensed under CC BY 4.0.\nLife of Research\nSemantic entropy detects confabulations in language models by drawing several answers, grouping them by meaning and measuring how spread the meanings are. Because each extra answer costs a full generation, a line of work asks how few samples will do. A 2025 Bayesian estimator matches the standard estimator's AUROC with 53 percent of its samples and, in its authors' words, is useful \"even with just one sample\", a result they call counterintuitive because with one sample no comparison of meanings takes place. In this analysis the author reviews and synthesises published results to show that the one-sample result is a likelihood result, and that the wider question it raises, how much a second sample adds, has no settled answer because the literature has not fixed what the second sample is compared against. Under one protocol, ten samples beat the log-probability of a low-temperature answer by 3 to 27 AUROC points depending on model and dataset. Under another, the log-probability of the greedy answer outperforms the sampling-based measures in 13 of 18 settings. Laid side by side on TriviaQA and SVAMP, the two papers' mean discrete semantic-entropy scores differ by less than one point, while their mean single-sequence scores differ by about 17 and 27 points, so most of the disagreement about the sampling premium is a disagreement about the baseline. Correctness labels move the comparison further: two 2025 studies show that the choice of grading function can reorder methods, and one finds that a single judge leaves a standard deviation as large as typical differences between methods. The author separates the conditions under which extra samples carry information a single pass lacks (long answers, several valid answers, reasoning chains) from errors that no number of samples can expose, and proposes a matched-budget protocol that would measure the premium directly.","authors":[{"name":"Pranay Mahendrakar","affiliation":"SONYTECH","orcid":"0009-0003-7224-029X"}],"type":"preprint","license":"CC-BY-4.0","keywords":["semantic entropy","hallucination detection","uncertainty quantification","sequence likelihood","sample efficiency","evaluation methodology","confabulation","language models"],"publication_date":"2026-11-20","version":1,"views":2,"downloads":2,"created_at":"2026-11-21 11:16:23","doi":"10.5281/zenodo.24398949","source":"Zenodo","source_url":"https://zenodo.org/records/24398949","files":[{"id":281,"name":"a-second-sample-is-not-a-second-signal.pdf","size":396252,"mime":"application/pdf","downloads":2,"download_url":"https://lifeofresearch.com/preprints/257/files/281/a-second-sample-is-not-a-second-signal.pdf"}]}