Measured Against an Incomplete Answer Key: What Unknown Recall Certifies in Open-World Object Detection, Why Its Two Cost-Side Instruments Disagree, and the Separation That Was Published and Not Adopted

@misc{mahendrakar2026measured,
  author       = {Mahendrakar, Pranay},
  title        = {{Measured Against an Incomplete Answer Key: What Unknown Recall Certifies in Open-World Object Detection, Why Its Two Cost-Side Instruments Disagree, and the Separation That Was Published and Not Adopted}},
  year         = {2026},
  month        = sep,
  date         = {2026-09-15},
  publisher    = {Life of Research},
  doi          = {10.5281/zenodo.22759926},
  url          = {https://lifeofresearch.com/articles/19},
  abstract     = {(c) 2026 Pranay Mahendrakar. Licensed under CC BY 4.0. Open-world object detection asks a detector to put boxes on objects whose classes it was never trained on, and the field's headline number for that ability is unknown recall. This paper reconstructs where that metric came from and what it can certify. The founding paper of the task did not report it: its protocol carried two cost-side instruments, wilderness impact and absolute open-set error, and no recall term for unknown objects at all. Unknown recall entered at the next CVPR, and the reason its introducers gave was that the test sets do not annotate every unknown object, which makes any precision-based unknown metric unsound. That reason is correct and is independently asserted by later work. It is also the reason unknown recall cannot be read as a discovery rate: the same missing annotations that make false positives uncountable remove any cost on over-detection, and the two instruments that would have supplied that cost were moved into appendices by the paper that installed the new metric and by its successor. Those two instruments are not independent - one of the critiques states the identity relating them - and in the one published table that reports all three for the same models, they produce three different orderings, with the model ranked best on one cost-side metric ranked worst on the other. The field's four documented repairs pull in incompatible directions, and two of them are explicit that the other's metric is flawed. The saturation result usually read as deflationary cannot carry that reading, because its authors state that their baselines have seen the unknown classes in pre-training and that the comparison is impossible. What would separate discovery from proposal strength already exists: a localisation-versus-discrimination split published in 2022 by the field's own direct critique. A mechanical check of thirteen papers from 2021 to 2026 found it named in none of the other eleven, and the field's own survey puts its adoption at three of the eighteen methods it tabulates, while unknown recall appears in every method paper from 2022 onward, including both 2026 papers checked. This paper states what follows for reading the literature, states flatly what is not known, and names the comparisons that would settle the open part. The literature search, drafting and citation verification for this paper were carried out with AI assistance under the author's direction. Every citation was machine-verified against the arXiv record or Crossref before inclusion, and every quantitative claim was read back against the cited source's own table or text. The counts reported in Section 10 were produced by fetching each named paper's full text and searching it mechanically; the procedure is stated in Section 2 so that it can be repeated. No experiment was run and no number in this paper was measured by its author. The author is responsible for the final text and for all claims made in it.},
  keywords     = {open-world object detection, evaluation metrics, unknown recall, benchmark design, annotation completeness, class-agnostic proposals, foundation models, negative results},
  language     = {en},
  copyright    = {Creative Commons Attribution 4.0}
}