How should grad students approach writing a referee report?

Question

When asked as a grad student to write a referee report for a journal, how should we approach it? Specifically, what is our core contribution expected to be relative to faculty members and other senior researchers who will be the other referees?

Answer 1

You're expected to review the article.  Assess the following things.  Don't worry about the other referees.  Just try to be the best reviewer you can be.  

1. Is the methodology correct?
2. Is the methodology new?
3. Is the contribution incremental or large?
4. Is it written well?

The most challenging this is 3, because it requires you to have a sense of a whole literature which you are probably trying to learn yourself.  That's ok.  

If in doubt, check in with your advisor.  They'lll have great advice for you.  

The first time I refereed an article I spent WAY too long but I wanted to get it right.
So I had a proof showing the authors approach would be biased.
I also did simulations showing this.  
I probably spent a week on the report.  These days, I spend probably 4-7 hours total reviewing a paper.  Take your time this first time around and try to get good habits established for yourself going forward.   Being a referee used to be exciting for me.  These days....less so.

Comments

This is very helpful. I think (3) feels like the hardest, as a grad student. It also feels very hard to make a recommendation when we don't know either the literature or the normal standard of the journal super well.

---

I like that list, Ben. Items 2 and 3 are good things to think about when you have an idea in mind and are thinking about whether to write it into a paper, and, if it becomes a paper, perhaps answer those questions in the introduction without sounding too much like you're tooting your own horn.

Answer 2

Some common grad student referee errors would be, I suppose:

- Objective errors (so Ben Hansen's approach is probably the right one for a first-time report)
- Trying to micromanage a paper (trying to get the authors to conform exactly to the style your specific advisers have told you is one and only way of doing things)
- Overly aggressive language

Also, read "How to Write an Effective Referee Report and Improve the Scientific Review Process" by Jonathan B. Berk, Campbell R. Harvey, and David Hirshleifer in the Journal of Economic Perspectives

Comments

Very useful - thanks!

Answer 3

The JEP piece and the other comments are great, but I'll also add this oldie-but-a-goodie:

https://are.berkeley.edu/courses/ARE251/2004/assignments/RRGuidelines.pdf

and this nice set of tips from Macartan Humphries (political science, but still helpful):

http://www.macartan.nyc/teaching/discuss/

People might have different views on this, but I find it helpful to sit on it for a day or two then carefully edit the tone, drop the most trivial comments, and edit it back to at most 3 pages.

Comments

This is great, thanks!

Answer 4

A useful source of advice for this and other questions is A Guide for the Young Economist (The MIT Press)  by William Thomson
Here's the Amazon blurb for the second edition (I recall reading the first edition):
"Detailed advice on writing papers, giving presentations, and refereeing, plus an essential guide to the basics of being a graduate student in economics.

This book is an invaluable handbook for young economists working on their dissertations, preparing their first articles for submission to professional journals, getting ready for their first presentations at conferences and job seminars, or undertaking their first refereeing assignments. In clear, concise language―a model in itself―William Thomson describes how to make written and oral presentations both engaging and efficient. Declaring "I would certainly take up arms for clarity, simplicity, and unity," Thomson covers the basics of clear exposition, including such nuts-and-bolts topics as titling papers, writing abstracts, presenting research results, and holding an audience's attention.

This second edition features a substantial new chapter, "Being a Graduate Student in Economics," that offers guidance on such essential topics as the manners and mores of graduate school life, financial support, selecting an advisor, and navigating the job market. The chapter on giving talks has been rewritten to reflect the widespread use of presentation software, and new material has been added to the chapter on writing papers."

Comments

Really helpful - thank you!