My first real non-clinical job was as a very junior lecturer in the Anatomy Department of The Middlesex Hospital Medical School. I used to get in to work early and would walk up the stairs to the fourth floor. One day, in the early 1970s, I was walking up the stairs when I noticed a strange smell. It seemed to be coming from the second floor, where the physiologists were located, so I walked along the corridor to see if I could find the source. All of the office and lab doors were closed, except for one that seemed to be the source of the odor. It was the office of a rather taciturn but kindly Senior Lecturer in Physiology. He looked to be asleep with his head on the desk, but he was dead. I looked at his desk and on it were a rejection letter and a set of reviews from the Journal of Physiology. My medical school colleague had taken his own life by swallowing cyanide. The pain of the rejection and the harsh reviews had proved too much for him. As long as I live I will never forget the smell of burned-almonds.
A month or two after that experience, I was flattered to receive a request to review a paper for the American Journal of Physical Anthropology, but I realized that although I had been the recipient of referee’s reports, I did not have the foggiest clue about how to go about reviewing a manuscript. Fortunately I had a wise senior colleague who helped put me on the right track, but some of you may not be so fortunate. So here are his tips, plus some I have picked up during c.45 years of preparing referee’s reports.
- Never submit a referee’s report until you have read it through in the same way the recipient will. How would you feel if you were the recipient?
- A good rule of thumb is never write anything in a referee’s report that you would not be prepared to say to someone’s face.
- Don’t just dwell on the negative. If something impresses you, say so. No matter how dire the manuscript, try and say something encouraging and constructive, especially if you think the author(s) is/are junior.
- I have been the recipient of referee’s reports where it is seemed clear to me that the writer had not understood what I was trying to do, and in some cases it is difficult to believe that they had read the manuscript carefully. So, to establish your credibility as a reviewer use the first paragraph to summarize the manuscript (rather than risk violating any confidentiality agreement with the journals, instead of using an example from the 236 referee’s reports on ms. and grants on my computer I have concocted a paragraph I would hope to read in one of the referee’s reports of a recently submitted ms. of ours)
- Your job as a reviewer is to advise the editor about whether the submitted ms. reports sound science. This does not mean that you necessarily agree with the results, or you would have done the research the way the authors did. Both are irrelevant to the job in hand.
- What the reviewer has to do is scrutinize the premises that underlie the research, judge the authors’ familiarity with the literature, decide whether the research design is sound, make sure the samples are appropriate and adequate, the methods are sound and appropriate, and the conclusions the authors reach are justified by their data.
- Common problems with methods and results are A) not enough attention to reproducibility, and B) unrealistic precision (e.g., results expressed to three places of decimals when the methods cannot justify that level of precision).
- Reviewers also function as the “under-citation” police. Have the authors given appropriate credit to others? Have they not cited others because they are not familiar enough with the literature, or because they are deliberately suppressing information about previous relevant research in order to boost their own contribution?
- Don’t just check the text, also look at the Tables and Figures. Do they match with what is written in the text? Are the legends correct?
- Some conscientious reviewers check that all the references are cited and all the citations appear in the reference list. To my mind, that is the job of the editor, not the reviewer.
- Do not be afraid to suggest that you are not an appropriate reviewer because A) you are not familiar enough with the topic to do 8, or B) you have a conflict of interest (e.g., this is a topic you are working on in much the same way, a student of yours is a ‘competitor’, etc.)
- I have learned a great deal by reviewing ms. for journals that as a matter of policy circulate all of the reviews to all of the reviewers. I wish more journals did the same.
- I am not in favor of anonymous reviewing; it tempts people to be more scathing than they would be if reviews were signed. I decided early on to begin my reviews with “Thank you for asking me to comment on this paper.” I am sure my colleagues rumbled my identity long ago.
Next time – present-day problems with the peer-review process.
Example of a first paragraph of a referee’s report that would convince me that the referee had at the very least taken the trouble to read and comprehend a recently submitted manuscript of ours: -
“This manuscript examines the pattern and degree of sexual dimorphism with respect to the size and shape of the posterior cranial base in extant hominoids. The comparative sample comprised lateral and basal radiographs of 407 individuals from Pan paniscus, Pan troglodytes, Gorilla gorilla, and Pongo pygmaeus and regional samples of Homo sapiens from Africa, China, Europe and Australia. Eleven inter-landmark distances were selected to represent the parts of the posterior cranial base as seen in norma basilaris that are best preserved in the hominin fossil record. Each log-transformed inter-landmark distance was divided by a ‘posterior cranial base geometric mean’ calculated as the nth root of the product of all the variables. The degree of posterior cranial base variation within and among hominoid taxa was then ascertained using a multivariate randomized Levene test; size and shape were considered separately. Patterns of size-related and shape-related basicranial sexual dimorphism among the extant hominoid taxon samples were compared using an adaptation of Euclidean Distance Matrix Analysis. The results show that African apes and modern humans share a pattern of sexual dimorphism that is significantly different from the one seen orangutans. Within modern humans there were significant differences between the pattern seen in the European sample and those of the other three regions.