2026-11-01T18:08:34Z https://lifeofresearch.com/oai2

oai:lifeofresearch.com:174 2026-11-01T10:35:10Z type:preprint openaire

10.5281/zenodo.23849157 https://lifeofresearch.com/preprints/174 Pranay Mahendrakar 0009-0003-7224-029X SONYTECH Life of Research 2026 personally identifiable information de-identification named entity recognition distribution shift evaluation methodology clinical text privacy risk text anonymisation Preprint Creative Commons Attribution 4.0 (c) 2026 Pranay Mahendrakar. Licensed under CC BY 4.0. Life of Research Automatic detection of personally identifiable information (PII) is the step that decides what text may leave an organisation, and its published record appears to contradict itself. Clinical de-identification systems have been reported to match or exceed human annotators, with recall above 0.94 on held-out notes and above 0.99 for one rule-based tool; a 2026 stress test reports that encoder, rule-based and language-model detectors all degrade under noisy and informal input, with location recall falling below 0.5, and a unified benchmark reports span F1 below 0.14 for every system it tried. In this analysis the author reviews and synthesises published results to show that most of the contradiction is in what the scores index rather than in the detectors. A PII detection score carries four indices that headlines omit: how the test text relates to the text the detector was built on, the scoring rule, the taxonomy that defines what counts, and the unit over which misses are aggregated. The same shared-task system misses 9.1 percent of protected items under one official rule and 3.6 percent under another; the lowest unified-benchmark scores come partly from entity types the evaluated detectors do not define; and in the 2026 stress test several of the losses attributed to the language model are spans it covered under another label. A residual shift effect survives every index: under fixed scoring, cross-institution transfer cost about ten points of strict F1, and systems run unmodified on new psychiatric records scored a median of 0.63. None of the indices is the quantity a release decision needs, which is the probability that a released document still carries an identifying item someone could find. The paper consolidates the evidence in a table, derives how per-item recall compounds at the document level, and gives a coverage-audit procedure with the sample sizes a release claim requires.