Subject

Benchmark validity

6 open-access journal articles on benchmark validity, published in 2026 in Life of Research. Every record is free to read and download, with a permanent link and a ready-made citation.

6 records

Journal article

Who Spoke Next Is Not Who Should Have: What Zero-Shot Failure on Multi-Party Turn-Taking Benchmarks Measures, Why the Human Reference Sits Beside the Models, and What Would Show That Knowing When to Speak Must Be Trained

Pranay Mahendrakar 2026

An assistant that joins a group conversation has to decide, at every pause, whether to take the floor. Two 2026 benchmarks report that language models prompted zero-shot make this decision badly and that fine-tuning repairs it, and both conclude that knowing when to speak is learned rather than…

Cite Related research LOR-00111 · 1 view · 1 download

PDF420.3 KB

Journal article

A Blind Score Is Not a Reliance Measure: What Answering Without the Audio Shows About Audio-Language Benchmarks, Why the Partial-Input Debate Already Settled Half of It, and the Counterfactual-Audio Test the Other Half Needs

Pranay Mahendrakar 2026

Audio-language models are now routinely tested with the audio taken away, and the results are read in opposite directions. One 2026 audit reports that eight models keep 60 to 72 percent of their accuracy on three benchmarks when the audio is omitted, and concludes that the scores largely reflect…

Cite Related research LOR-00107 · 5 views · 5 downloads

PDF429.3 KB

Journal article

What Counts as a Control? A Scaffold Effect Is a Contrast Conditional on What Was Held Fixed, the 2026 Controlled Studies Each Held Something Different, and Only a Declared Spend Interval Makes a Harness Claim Refutable

Pranay Mahendrakar 2026

A language-model agent is a model wrapped in a harness that builds its context, exposes its tools, decides when to retry and when to stop. Several 2026 studies set out to measure how much the harness matters relative to the model, holding the model and the tasks fixed, and their answers do not…

Cite Related research LOR-00103 · 5 views · 6 downloads

PDF405.2 KB

Journal article

Refusal Is Not a Rate: What a Context-Conditioned Refusal Policy Would Have to Specify, Why the Benchmarks That Score Context Assume It Is Verified, and the Provenance Term No Instrument Prices

Pranay Mahendrakar 2026

A language model that refuses a harmful request and a model that refuses a harmless one produce the same event, and most of the literature on the jailbreak/over-refusal trade-off counts both as one refusal rate. The standard proposal for escaping the trade-off is to make refusal depend on context…

Cite Related research LOR-00055 · 8 views · 20 downloads

PDF6.3 MB

Journal article

The Allowance Is Doing the Work: Constructed Premises in Chain-of-Thought Step Verification, What Formal Validity Still Certifies Once They Are Permitted, and the Dependence Test Argumentation Runs and Step Verification Does Not

Pranay Mahendrakar 2026

A verifier that checks the steps of a chain of thought cannot demand that each step state everything it relies on, because no real step does. Every published step verifier therefore permits a class of premises that the text does not contain: the 2023 system that introduced step-wise deductive…

Cite Related research LOR-00023 · 5 views · 5 downloads

PDF587.4 KB

Journal article

Not Acting Is Not One Decision: Two Abstention Triggers in Tool-Using Agents, Why the Calibration and Permission Readings Cover Different Ones, and the Cost Term Neither Supplies

Pranay Mahendrakar 2026

An agent that declines to send the email has made a decision that looks like the decision a language model makes when it declines to answer a question, and the resemblance has organised the 2026 literature into two camps. One reads not-acting as a calibration problem: the agent should estimate its…

Cite Related research LOR-00018 · 8 views · 22 downloads

PDF537.1 KB

Cite

APA

BibTeX

Import into a reference manager: BibTeX · RIS (Zotero, Mendeley, EndNote)