Paper
How Robust are Robustness Checks?
Authors
Brenda Prallon
Abstract
Robustness checks are routine in empirical work, but there is no standard statistical procedure to formally measure what one can learn from them. I propose a "robustness radius" measure to quantify the amount by which the robustness checks estimands differ from the main specification estimand. I do so by framing robustness checks as explicitly biased regressions, clarifying what exactly the estimands are when comparing multiple regressions with slightly different samples, and applying a test from the moment inequalities literature. The robustness radius is easily interpretable and adapts to sampling uncertainty and correlation across regressions. An application shows that, although assessing overall robustness is context-specific, the robustness radius guides those judgments and improves transparency.
Metadata
Related papers
Fractal universe and quantum gravity made simple
Fabio Briscese, Gianluca Calcagni • 2026-03-25
POLY-SIM: Polyglot Speaker Identification with Missing Modality Grand Challenge 2026 Evaluation Plan
Marta Moscati, Muhammad Saad Saeed, Marina Zanoni, Mubashir Noman, Rohan Kuma... • 2026-03-25
LensWalk: Agentic Video Understanding by Planning How You See in Videos
Keliang Li, Yansong Li, Hongze Shen, Mengdi Liu, Hong Chang, Shiguang Shan • 2026-03-25
Orientation Reconstruction of Proteins using Coulomb Explosions
Tomas André, Alfredo Bellisario, Nicusor Timneanu, Carl Caleman • 2026-03-25
The role of spatial context and multitask learning in the detection of organic and conventional farming systems based on Sentinel-2 time series
Jan Hemmerling, Marcel Schwieder, Philippe Rufin, Leon-Friedrich Thomas, Mire... • 2026-03-25
Raw Data (Debug)
{
"raw_xml": "<entry>\n <id>http://arxiv.org/abs/2602.19384v1</id>\n <title>How Robust are Robustness Checks?</title>\n <updated>2026-02-22T23:32:23Z</updated>\n <link href='https://arxiv.org/abs/2602.19384v1' rel='alternate' type='text/html'/>\n <link href='https://arxiv.org/pdf/2602.19384v1' rel='related' title='pdf' type='application/pdf'/>\n <summary>Robustness checks are routine in empirical work, but there is no standard statistical procedure to formally measure what one can learn from them. I propose a \"robustness radius\" measure to quantify the amount by which the robustness checks estimands differ from the main specification estimand. I do so by framing robustness checks as explicitly biased regressions, clarifying what exactly the estimands are when comparing multiple regressions with slightly different samples, and applying a test from the moment inequalities literature. The robustness radius is easily interpretable and adapts to sampling uncertainty and correlation across regressions. An application shows that, although assessing overall robustness is context-specific, the robustness radius guides those judgments and improves transparency.</summary>\n <category scheme='http://arxiv.org/schemas/atom' term='econ.EM'/>\n <published>2026-02-22T23:32:23Z</published>\n <arxiv:primary_category term='econ.EM'/>\n <author>\n <name>Brenda Prallon</name>\n </author>\n </entry>"
}