Peer review is a painful process both to authors and to referees, and I gather from Eilon’s post that editors are not thrilled with it either. This, I believe, is in part because we game theorists expect the referee not only to judge the paper, but also to discuss the paper more than is necessary to justify the judgment, and, even worse, to suggest ways to improve the paper.
This is why we produce long reports, sometimes of low quality because we force ourselves to write something even when we have nothing valuable to say. Sure, the referee’s feedback can be useful — As Eilon says, some authors believe that it is actually worth the delay in publication even when the paper is bound to be rejected. But even in these cases I think the effort to write the report is not worth the small audience that the report will receive. If the referee has some interesting observation or criticism about the paper, why hide it from the rest of community ?
This is why I suggest that the profession adopt the guidlines of The Annals of Statistics, which to my knowledge are common in math and physics journals
you are not expected to rewrite the paper or to suggest major revisions or avenues for further research. Your role is simply to recommend whether or not the paper should be published.
2 comments
January 19, 2010 at 3:57 am
eilonsolan
Eran, let me disagree with you. Leaving aside your interpretation to my post, I think that a one-word referee report that says “accept” or “reject” is not useful. The referee should explain his or her point of view, and so the paper should be carefully read. Once you read a paper thoroughly, you find many ways to improve it. Why do we find many such ways? Young scholars often do not know how to present their results optimally. Moreover, the authors are often immersed so much in their work, that they gloss over issues that people who see the problem for the first time find challenging. I think that it is the referee’s duty to point at ways to improve a paper. As an editor, I expect that.
Another issue: suppose that a paper who receives a grade higher than 80 should be accepted. Suppose that you believe that the present version is only 75, but the paper can be improved, either by adding explanations or motivation, or by proving additional results. What will you, as a referee, say? To reject the paper? If so, I think you miss your duty as a referee.
So, I think that referees (may) do a great service to authors. It is not a matter of small audience. It is that you can help others improve their papers, and they will do the same unto you in due course.
January 19, 2010 at 1:09 pm
Eran
`you can help others improve their papers, and they will do the same unto you in due course’ — Both have happened to me many times, but not through the journals reviewing machine. The anonymity of this process, which is essential for honest judgment, is obstructive for communication of criticism and also for reciprocity in this service.
Regarding the situation you described in which the paper is just below the threshold, my guess is that most authors would be willing to pay the price of a rejected paper that might have been accepted under a revision for the utility they get from a more efficient process.