Preprint
Which Way the Verdict Falls: Noise-Induced Flips in an LLM Bias Judge Run Toward the Label Its Prompt Makes Easiest to Reach, the Harm-Detection Literature Records the Opposite Sign, and Whether Noise Inflates a Bias Score Depends on the Score
(c) 2026 Pranay Mahendrakar. Licensed under CC BY 4.0. Life of Research Language models are now used as judges that read a passage and decide whether it leans toward a stereotype, and the passages they read are often noisy. A 2026 study corrupted 3,822 stereotype-probing responses with typos…