Calibration: open-access research

Journal article

Errors That Should Compound, and When They Do: The Product Rule Assumes Independent Steps, Conformal Guarantees Assume Exchangeable Instances, and the Unit of Independence Decides What Each Can Claim

Pranay Mahendrakar 2026

Two bodies of work quantify uncertainty in multi-step language-model reasoning, and both rest on an independence assumption placed at some unit. Step-level error models multiply per-step reliabilities, which, at an illustrative 95% per-step accuracy, predicts that a fifty-step chain succeeds about…

Cite Related research LOR-00077 · 14 views · 55 downloads

PDF551.4 KB

Journal article

Not Acting Is Not One Decision: Two Abstention Triggers in Tool-Using Agents, Why the Calibration and Permission Readings Cover Different Ones, and the Cost Term Neither Supplies

Pranay Mahendrakar 2026

An agent that declines to send the email has made a decision that looks like the decision a language model makes when it declines to answer a question, and the resemblance has organised the 2026 literature into two camps. One reads not-acting as a calibration problem: the agent should estimate its…

Cite Related research LOR-00018 · 8 views · 22 downloads

PDF537.1 KB

Journal article

Consistency Is Not Correctness: What Internal Contradiction Detection Certifies, Why the Entailment Is Disjunctive, and Which Decisions It Leaves Unlicensed

Pranay Mahendrakar 2026

Contradiction detection occupies an unusual position among proposals for making a language model check itself. Comparing two of a model's own outputs appears to need no external oracle, which makes it look like the one exception to the observation that working hallucination detectors sit outside…

Cite Related research LOR-00009 · 7 views · 14 downloads

PDF485.9 KB

Journal article

Predicting Your Own Failure Is Not Predicting Another Model's Advantage: What Router Results Establish About Agent Self-Knowledge, and the Incremental-Validity Test Nobody Has Run

Pranay Mahendrakar 2026

A common reading of the LLM routing literature holds that delegation works without self-knowledge: a router allocates queries using external features of the task and observed outcome statistics over a model pool, and never consults the candidate model's opinion of its own competence. This paper…

Cite Related research LOR-00060 · 6 views · 16 downloads

PDF464 KB

Journal article

Predicting Your Own Failure Is Not Predicting Another Model's Advantage: What Router Results Establish About Agent Self-Knowledge, and the Incremental-Validity Test Nobody Has Run

Pranay Mahendrakar 2026

A common reading of the LLM routing literature holds that delegation works without self-knowledge: a router allocates queries using external features of the task and observed outcome statistics over a model pool, and never consults the candidate model's opinion of its own competence. This paper…

Cite Related research LOR-00014 · 5 views · 5 downloads

PDF460.2 KB

Journal article

An Unknown Unknown Is Defined Relative to an Oracle: Four Places External Judgement Enters Blind-Spot Discovery, and Why the Self-Detection Results Do Not Remove It

Pranay Mahendrakar 2026

An unknown unknown is standardly defined as an input on which a model makes a high-confidence mistake. The definition is not incidental to the research problem; it fixes its shape. A quantity defined by the conjunction of high confidence and error cannot be recovered from confidence alone, and the…

Cite Related research LOR-00059 · 7 views · 8 downloads

PDF491.8 KB

Journal article

An Unknown Unknown Is Defined Relative to an Oracle: Four Places External Judgement Enters Blind-Spot Discovery, and Why the Self-Detection Results Do Not Remove It

Pranay Mahendrakar 2026

An unknown unknown is standardly defined as an input on which a model makes a high-confidence mistake. The definition is not incidental to the research problem; it fixes its shape. A quantity defined by the conjunction of high confidence and error cannot be recovered from confidence alone, and the…

Cite Related research LOR-00028 · 6 views · 17 downloads

PDF489.5 KB

7 records

Errors That Should Compound, and When They Do: The Product Rule Assumes Independent Steps, Conformal Guarantees Assume Exchangeable Instances, and the Unit of Independence Decides What Each Can Claim

Not Acting Is Not One Decision: Two Abstention Triggers in Tool-Using Agents, Why the Calibration and Permission Readings Cover Different Ones, and the Cost Term Neither Supplies

Consistency Is Not Correctness: What Internal Contradiction Detection Certifies, Why the Entailment Is Disjunctive, and Which Decisions It Leaves Unlicensed

Predicting Your Own Failure Is Not Predicting Another Model's Advantage: What Router Results Establish About Agent Self-Knowledge, and the Incremental-Validity Test Nobody Has Run

Predicting Your Own Failure Is Not Predicting Another Model's Advantage: What Router Results Establish About Agent Self-Knowledge, and the Incremental-Validity Test Nobody Has Run

An Unknown Unknown Is Defined Relative to an Oracle: Four Places External Judgement Enters Blind-Spot Discovery, and Why the Self-Detection Results Do Not Remove It

An Unknown Unknown Is Defined Relative to an Oracle: Four Places External Judgement Enters Blind-Spot Discovery, and Why the Self-Detection Results Do Not Remove It