American Economic Review
ISSN 0002-8282 (Print) | ISSN 1944-7981 (Online)
Unpacking P-hacking and Publication Bias
American Economic Review
vol. 113,
no. 11, November 2023
(pp. 2974–3002)
Abstract
We use unique data from journal submissions to identify and unpack publication bias and p-hacking. We find initial submissions display significant bunching, suggesting the distribution among published statistics cannot be fully attributed to a publication bias in peer review. Desk-rejected manuscripts display greater heaping than those sent for review; i.e., marginally significant results are more likely to be desk rejected. Reviewer recommendations, in contrast, are positively associated with statistical significance. Overall, the peer review process has little effect on the distribution of test statistics. Lastly, we track rejected papers and present evidence that the prevalence of publication biases is perhaps not as prominent as feared.Citation
Brodeur, Abel, Scott Carrell, David Figlio, and Lester Lusher. 2023. "Unpacking P-hacking and Publication Bias." American Economic Review, 113 (11): 2974–3002. DOI: 10.1257/aer.20210795Additional Materials
JEL Classification
- A11 Role of Economics; Role of Economists; Market for Economists
- A14 Sociology of Economics
- C13 Estimation: General
- L82 Entertainment; Media