Evaluation methodology

Journal article

A Rewrite That Keeps the Label Has Already Read It: Why an LLM Paraphrase Can Raise a Sentiment Classifier's Accuracy Without the Classifier Getting Better, Four Accounts of the Gain, and the Re-Annotation and Neutral-Rate Controls That Would Separate Them

Pranay Mahendrakar – 2026

A 2026 study reports that sentiment classifiers score higher on reviews rewritten by small language models than on the human originals, by up to 5.8 accuracy points, and reads the gain as stylistic alignment: the rewrite strips idiosyncrasy and hands the classifier text closer to what it was…

Cite Related research LOR-00110 · 2 views · 1 download

PDF374 KB

Journal article

Lossless at What Length? Near-Lossless KV Cache Compression Is a Claim Indexed by Decode Exposure, Scoring Statistic and Budget Accounting, the Decode-Aware Line Has Answered the First Index Mostly on Final-Answer Accuracy, and Divergence From the Full Cache Is Not by Itself the Defect

Pranay Mahendrakar – 2026

Key-value cache compression is routinely reported as near-lossless at aggressive budgets: eviction, quantisation and merging methods keep a fifth or a tenth of the cache and match the full-cache score. A 2025-2026 line of work argues that the result was manufactured by evaluation, because the…

Cite Related research LOR-00106 · 6 views · 5 downloads

PDF383.7 KB

Journal article

INT4 Is Not One Treatment: Why the Quantisation-Safety Literature Cannot Be Reconciled as Reported, How Six Design Axes Separate Its Studies Before Bit-Width Does, and Why Two 2026 Studies of the Same Model Names Reach Opposite Verdicts

Pranay Mahendrakar – 2026

Post-training quantisation to 4 bits is widely treated as nearly free, a judgement resting on perplexity and multiple-choice accuracy. Whether it is also free for safety is now contested: 2025-2026 studies report that 4-bit checkpoints lose refusal while their quality holds, that 3-bit weights…

Cite Related research LOR-00102 · 6 views · 5 downloads

PDF406 KB

Journal article

Three Hazards Under One Horizon: Constant-Hazard, Degrading-Step and Log-Logistic Accounts of Agent Failure Describe Different Levels of the Same Data, the 50 Percent Trend Barely Depends on Which Is Right, and Every High-Reliability Forecast Does

Pranay Mahendrakar – 2026

Forecasts of autonomous AI capability increasingly rest on one number: the length of task, measured in human working time, that an agent completes half the time. That number has doubled roughly every seven months, and three accounts of why agents fail on longer tasks now sit behind its…

Cite Related research LOR-00094 · 7 views · 3 downloads

PDF416.3 KB

Journal article

Detectors Say 'Different', Not 'What': Score, Deviation and Name Are Three Claims With Three Referees, Only the Name Needs a Referee Outside the Detector, and Published Results Show Novelty Detected Far More Reliably Than It Is Named

Pranay Mahendrakar – 2026

An anomaly or novelty detector reports that an input is unlike the data it was built from. Users want more: they want to know what the unusual thing is. Explainable anomaly detection is often presented as supplying that second answer, but three different claims travel under the one word…

Cite Related research LOR-00085 · 8 views · 14 downloads

PDF388 KB

Journal article

Spend the Budget on the Known or the Unknown? Open-Set Active Learning Scores Two Opposed Objectives Under One Name, Its Filtering Side's Own Results Do Not Treat Unknowns as Waste, and the Winner Is Set by a Relevance Label and a Query Price the Benchmarks Fix in Advance

Pranay Mahendrakar – 2026

Active learning chooses which unlabelled examples a human should label. When the unlabelled pool contains classes the model has never been shown, two lines of work give opposite instructions under the same name. One, usually called open-set active learning or active open-set annotation, counts…

Cite Related research LOR-00082 · 9 views · 12 downloads

PDF409.7 KB

Journal article

Problem Choice Without a Referee: An Automated Novelty Check Certifies an Empty Search, a Rediscovery Benchmark Credits the Hypothesis Humans Found Next, and Every Located Referee of Worth Independent of Proposer and Field Needs the Objective Given

Pranay Mahendrakar – 2026

Pipelines that claim to automate scientific discovery gate their search on a judgement that a proposed idea or problem is novel and worth pursuing. The best-controlled evidence on that judgement points two ways at once: in a blind study involving more than one hundred NLP researchers, expert…

Cite Related research LOR-00075 · 9 views · 13 downloads

PDF530.7 KB

Journal article

How Many Classes Are Out There? In Category Discovery the Class Count Is Granularity Carried Over From the Labelled Set, Supplying It Can Supply the Taxonomy, and Without It the Count Is Set by a Hyperparameter

Pranay Mahendrakar – 2026

Category discovery asks a model to sort an unlabelled image collection into classes, some of which it has never been shown, using a labelled subset of other classes as its guide. Almost every method needs one number before it can produce an answer: how many classes the unlabelled data contains…

Cite Related research LOR-00056 · 10 views · 26 downloads

PDF6.6 MB

Journal article

Transfer Is a Directed Relation: Four Quantities Under One Word in the Cross-Domain RLVR Debate, and Why the Structural-Similarity Reading Does Not Survive Its Own Evidence

Pranay Mahendrakar – 2026

Reinforcement learning with verifiable rewards is the standard route to reasoning-tuned language models, and the field has split over whether its gains leave the training domain. One body of work reports that most models succeeding at mathematics fail to transfer and that single-domain…

Cite Related research LOR-00066 · 8 views · 30 downloads

PDF475.6 KB

Journal article

Transfer Is a Directed Relation: Four Quantities Under One Word in the Cross-Domain RLVR Debate, and Why the Structural-Similarity Reading Does Not Survive Its Own Evidence

Pranay Mahendrakar – 2026

Reinforcement learning with verifiable rewards is the standard route to reasoning-tuned language models, and the field has split over whether its gains leave the training domain. One body of work reports that most models succeeding at mathematics fail to transfer and that single-domain…

Cite Related research LOR-00031 · 5 views · 5 downloads

PDF473.3 KB

Journal article

The Crossover Does Not Carry the Claim: Pass@k Inversion, the Reasoning Boundary, and Five Premises the RLVR Debate Leaves Unstated

Pranay M. Mahendrakar – 2026

A model trained with reinforcement learning from verifiable rewards beats its base model at one sample and loses to it at many. The field has read that crossover as a measurement of the reasoning boundary and has spent two years arguing about which direction the boundary moved. This paper…

Cite Related research LOR-00072 · 7 views · 9 downloads

PDF461 KB

Journal article

The Crossover Does Not Carry the Claim: Pass@k Inversion, the Reasoning Boundary, and Five Premises the RLVR Debate Leaves Unstated

Pranay M. Mahendrakar – 2026

A model trained with reinforcement learning from verifiable rewards beats its base model at one sample and loses to it at many. The field has read that crossover as a measurement of the reasoning boundary and has spent two years arguing about which direction the boundary moved. This paper…

Cite Related research LOR-00034 · 6 views · 29 downloads

PDF457.3 KB

Journal article

The Self-Verification Gap: Why Working Hallucination Detectors Are External, and What Real-Time Correction Would Actually Require

Pranay M. Mahendrakar – 2026

Large language models generate fluent text that is sometimes false, and a substantial literature now proposes to have the model notice and repair those errors while it writes. Semantic entropy detects confabulations across datasets and tasks without prior knowledge of the task (Farquhar et al…

Cite Related research LOR-00041 · 9 views · 37 downloads

PDF541.9 KB