In the old days one of the “perks” of being medically-qualified in the UK was that if you or your family were sick, colleagues would bend-over-backwards to make sure you or your family were seen as soon as possible. They would see you in their office “out-of-hours,” or they would add you to the end of a busy clinic.
My old boss, a consummately skilled surgeon, would never do that. He insisted that colleagues be given a proper slot in his next National Health clinic, or he would see to it that they had an early appointment in his rooms in Harley Street (but he did not bill them for his time). His reasons were the following. When people are seen “out-of-the-system” they do not get processed using the routines that have been established over many years. Some of these routines seem tiresome, but they are there to make sure mistakes are not made. Consultations that are rushed and preemptory are never quite as thorough as those carried out as part of a well-honed routine, and my boss wanted his colleagues and their families to have his proper consideration and the best level of care.
What, you are probably thinking, has all this to do with peer-review? I would argue that taking a careful history and conducting a thorough physical examination are much like carrying out a proper peer-review [1]. Neither activity can be rushed or modified, subjected to deadlines, or be overly influenced by forceful personalities, without running the risk that quality will be compromised.
If you are an author of scholarly work, the peer-review system, when operated properly by a wise and experienced editor who uses similarly wise and knowledgeable reviewers, is much more of a boon than a bane. It gives you the chance to have your work and writing exposed to scholars who know your field at least as well as you think you do, and, with luck, they may be more knowledgeable and experienced than you are. They are able to stand back and gently suggest that your methods may not be as flawless as you think, or your interpretations may not be the only ones that should be considered.
This scenario used to be a familiar one when I was a young researcher. People like Bill Pollitzer at the AJPA would find experienced reviewers who gave generously of their time to write comments and suggest changes that improved your work. But the success of the system was that it was “operated properly by a wise and experienced editor who uses equally wise and knowledgeable reviewers,” and that it was applied to all-comers, without fear or favor.
In those days editors were almost always senior researchers who did not take kindly to being bullied by colleagues who thought they were so elevated, or the material they were presenting was so important, that they had outgrown the peer-review system. Manuscripts came in one at a time, not in packages with a prominent metaphorical label on the bundle saying, in effect, “take it or leave it.” The implication is that if the editor of the journal will not cut the researchers a favorable deal (i.e., protection from pesky knowledgeable reviewers) they will take their business down the road to the competition.
Effective peer-review is easy enough to skirt around. One way is to submit a much longer list of researchers to whom a manuscript should not be sent for review, than to whom it might be sent. This means that unless these conditions are resisted, the hapless editor has to scrabble around in the intellectual undergrowth to find someone not on the “black list” who has enough familiarity with the material to offer any sort of opinion, let alone an informed one. Another way to avoid effective peer review is to ignore the recommendations of knowledgeable reviewers by huffing and puffing at the editor along the lines of “these people don’t know what they are talking about”. It is the academic equivalent of a bank that is “too big to fail”. The logic goes like this. We have been working so long and so diligently on these fossils that no one can possibly know them as intimately as we do, ergo no one else is qualified to gainsay our interpretations. This strategy is less likely to succeed with the editors of specialist journals, but evidently it can be effective if the editor is not familiar with the field.
Of course, one wishes all this was just a bad dream, but many of you will recognize that it reflects some recent, and some not quite so recent, reality. In the latter case the problems associated with finding experienced “un-black listed” reviewers were compounded by the journal in question using its editorial pages, reporters, and publicity machine to give the impression that it was promoting and endorsing the interpretations of the authors of the papers. On that occasion the boundaries between science and journalism were crossed to the detriment of all concerned.
Good, experienced and courageous editors and knowledgeable and impartial reviewers are the bulwarks of the peer-review system. Without the former you do not get the latter and the latter need the support of the former. But, as we have seen recently, without either or both of these essential components peer-review is not effective. As usual, my old surgical boss was right. You circumvent or weaken the checks and balances in any system at your peril.
1 http://www.nap.edu/openbook/0309059437/gifmid/9.gif
http://www.sciencedirect.com/science/article/pii/S0167779902019856
Acknowledgements
Several friends and colleagues have given me advice, much of which I heeded. Normally I would be pleased to name and thank them, but given the circumstances (and in an effort to protect the innocent) their names have been withheld.