What are your suggestions to improve transparency, reproducibility , and credibility of economic research?

Question

In the most recent issue of JEL there is a paper by Christensen and Miguel that surveys work on these topics. I agree that publication bias and inability to replicate published papers contribute a lot to undermine the credibility of economic research. Furthermore, incentives to work on replications are weak, because they tend to be  hard to publish (especially if they do not discover a "mistake" in a widely cited paper). All this contributes to what is called the replication crisis in economics (and the situation seems to be similar in other fields). What do you think  - would more opportunities to publish replication studies (irrespective of their results) contribute to  solution here? And what do you think of journals that specialize in the publication of replication studies - ideally, open access journals with no submission or publication fee that are sponsored by research foundations (like IREE - International Journal for Re-Views in Empirical Economics)?

Answer 1

A cool XXI century solution would be along the lines of journalism on the blockchain and the beautiful study in Nature by Camerer et al. (2018) (see also "can gambling save science?" on marginalrevolution.com). Long story short: let people bet money on a piece of research being nonsense and not reproducible, and let people support their claim with arguments, and let other people join or bet against them. Example: you do not believe in the currency momentum, and you go to a special platform where you could bet 100 bucks that the paper is not reproducible on a longer setting; I have long supported your view but never had time to check myself, so I put another 100 bucks in your support, but my colleagues would like to bet against you and me with their money; you then conduct the study, and the outcome is resolved somehow, the winning party taking the pile of cash home.

Comments

This would only work with papers that have clear expectations or prediction. Much of the new research in the field is rather wishy-washy as economics is a messy science, even with all the rigor. I like the idea of prediction markets in validating research papers, though!

Answer 2

Hyperlinks!

Research papers are still stuck on old methods of citations. Hyperlinking could improve on all three of the margins. Moreover, a separate database could hold reproducibility numbers, similar to how citations are tracked on the web.

Answer 3

I think its as simple as we need more journals (or editors and referees ) that value replication, or spaces in journals for valuing replication.  

I think the new generation will move in this direction as many younger researchers are sharing their code ahead of time on GitHub.  Integration with Github.  

But even then we need an appetite that embraces the science part of being a social science, which means replication is needed, and valuable.