Enhancing the Peer Review Process

Part 1 of 2: The Paper Review (Revise and Resubmit)

Peer Review is a research evaluation process to ensure quality of published manuscripts. It is used in today’s technical conferences and journals as a means to rank the submitted manuscripts in order of perceived quality. There is probably one or two rounds of rebuttal in which the authors are able to get some insights of how well they presented their research. At a conference, authors would get fifteen minutes to present their ideas, and a few more minutes for questions/answers. If they are lucky, they might have good hallway discussions later on.

I have held roles on all sides of the paper review process: program chair, reviewer and author. From my experience, I can say that I’ve seen all spectra of arguments, both for and against the current peer review process. From a program chair perspective, there is a desire to meet the deadline while maintaining quality of the reviews. The process is dependent on members of program committee that volunteer their precious time for paper reviews. From a reviewer perspective, there is a challenge to give good feedback in the reviews. It would be easy just to assign the paper to my graduate students, but sometimes, it is best to give my personal attention and read the paper myself. The best reviews I’ve given are those that I can give insights on how to better present the material. That is, I love it when I can offer a certain perspective that would make it a great paper. From an author’s viewpoint, sometimes I find it hard to believe the type of reviews I would get. I often wonder, "did the reviewer not read what I wrote?". In some cases, it is easy to see that this is indeed the case. For the other cases, when I do get very valuable feedback, I would add an acknowledgment in the final paper.

I truly believe that the current peer review process limits the interaction between the authors and the reviewer. There is simply not enough discussions to make good papers great and to weed out bad ones. Great papers can be lost among many bad ones, and sometimes I think it is more important to have fewer papers in lieu of a few seminal ones. It is no surprise then that there is an actual conference about the peer review process itself (The Organizing Committee of the International Symposium on Peer Review: ISPR 2009). That is, there is a growing community that believes the peer review process is not as effective as it should be. Here’s an example quote:

In a survey of members of the Scientific Research Society, "only 8% agreed that 'peer review works well as it is'." (Chubin and Hackett, 1990; p.192).

I would consider this as an uplifting bit of news and I would urge others to think about what they are getting from their current peer review process.

(Continues on the next post)