INT4 Is Not One Treatment: Why the Quantisation-Safety Literature Cannot Be Reconciled as Reported, How Six Design Axes Separate Its Studies Before Bit-Width Does, and Why Two 2026 Studies of the Same Model Names Reach Opposite Verdicts

@misc{mahendrakar2026int4,
  author       = {Mahendrakar, Pranay},
  title        = {{INT4 Is Not One Treatment: Why the Quantisation-Safety Literature Cannot Be Reconciled as Reported, How Six Design Axes Separate Its Studies Before Bit-Width Does, and Why Two 2026 Studies of the Same Model Names Reach Opposite Verdicts}},
  year         = {2026},
  month        = oct,
  date         = {2026-10-13},
  publisher    = {Life of Research},
  doi          = {10.5281/zenodo.23328132},
  url          = {https://lifeofresearch.com/articles/102},
  abstract     = {(c) 2026 Pranay Mahendrakar. Licensed under CC BY 4.0. Post-training quantisation to 4 bits is widely treated as nearly free, a judgement resting on perplexity and multiple-choice accuracy. Whether it is also free for safety is now contested: 2025-2026 studies report that 4-bit checkpoints lose refusal while their quality holds, that 3-bit weights make previously unbiased answers stereotyped in a dose-response pattern, and, in the opposite direction, that standard 4-bit quantisation is approximately safety-neutral. In this analysis the author reviews and synthesises published results to show that these findings cannot yet be set against one another, because the studies differ on six design axes before they differ on bit-width: what the treatment is, which toolchain produced the artefact, what outcome is scored, which instrument scores it, how the behaviour is elicited, and how the change is accounted. The sharpest case is a pair of 2026 studies that both quantise models named Llama 3.2 3B and Mistral 7B with AWQ to 4 bits. One reports refusal losses of 22.7 and 15.9 points; the other reports attack-success changes of +1.6 and +0.5 points. They differ on at least four of the six axes and cannot be shown to share the other two. Read along those axes, the literature supports three claims: 8-bit is close to neutral almost everywhere, retained quality cannot certify retained safety, and item-level behaviour changes even when aggregates do not. It does not support a general claim about 4-bit safety in either direction. The paper states a reporting protocol and the matched measurements that would settle the open part.},
  keywords     = {post-training quantization, safety alignment, refusal, social bias, LLM-as-a-judge, evaluation methodology, model compression, confounding},
  language     = {en},
  copyright    = {Creative Commons Attribution 4.0}
}