Those subject to peer review of their proposals for research funding or of their scholarly products view it as a significant hurdle to succeeding in their careers. To those outside of academia, peer review can be misinterpreted as cronyism that illegitimately rewards friends and allies. That interpretation is far away from my experience.
The peer review process is founded on the belief that those at the cutting edge of their fields are best suited to judging the value of proposed new work. The rate of success for many federal government research grant proposals is less than 15%. Having served on scores of such review panels, I have vivid memories of the care and critical review that is exercised in the evaluative deliberations. With the success rate so low, the critical review is fierce in most panels.
For all fields, the same peer review process is used to judge the value of completed scholarly products. In those fields producing books, the prestige of publishers is related to the rigor of the review process. Editors jealously compete to attract the best work of the best scholars. Editorial boards give advice to the organization on the value of a given series. The author of a mediocre manuscript submits to a long sequence of presses before an affirmative decision to publish is given, if ever.
For those fields whose scholarship is disseminated through journals, peer review rigor is often reflected in the success rate of submissions, which for some journals hovers in single digit percentages. The decision of a journal is the result of critical reviews by peers in the area the article addresses. The competition is fierce.
So, in sum, most of the attributes of peer review act to reward the very best in scholarship. But there are weaknesses.
I have vivid memories of a scientist friend of mine, now one of the most highly cited in his field, in his early years of work. He was attacking the dominant paradigm in practice in his field and having repeated difficulties getting his work published. The rejections brought criticism, calls for more evidence, and resistance to his approach. He was forced to publish his work in less impactful, second-tier journals. All was not lost, as the value of his work was eventually recognized, albeit much more slowly that, in retrospect, it deserved to be.
Peer review is effective in evaluating the marginal contribution of work fully within the accepted framework of a subfield. Peer review performs less well for work disruptive of the status quo. Its conservatism in evaluating field-changing results is a weakness from one perspective, but a strength, from another, as it asks a higher level of evidence for such challenges to decades of accepted findings.
As the number of scholarly outlets increases with web-based journals and other electronic publications, the opportunities have increased to get one’s work disseminated. One hopes that path-breaking work has higher opportunity to see the light of day. In any case, honest criticism inherent in the peer review process remains a strength of the academic enterprise.
Another very interesting post, thank you.
“Those subject to peer review of their proposals for research funding or of their scholarly products view it as a significant hurdle to succeeding in their careers. … ”
My experience is sometimes yes, but most of the time no, with the caveat that this ratio is affected by where someone might be in their career trajectory and also by the quality of the peer review. Younger academics working hard to proceed through the ranks and towards tenure might see peer review as more of a hurdle, but as anyone proceeds in their career, the positive aspects of *good* peer review become increasingly valuable. In my case at least, cherished. Good peer review (meaning truly objective, thorough, with politics left at the door) is a tremendous asset, a valuable gift. The entire point is to try look for flaws in the methodology or to poke holes in the conclusions. Having a thoughtful expert perform that service on behalf of your work improves the work, and everybody (particularly the author) wants that. Where the process breaks down is when politics rears it’s ugly head (which it almost always does at elitist journals and grant panels starved for appropriate funds) or when experts do a shoddy job. In today’s cynical world it can be difficult to see why doing an excellent job at peer review is a top priority. But everyone that I know that has focussed on doing good peer review for decades will tell you the rewards are many. Interestingly, I find that a fair number of these folks avoid elitist journals and will tell you that finding the “best” work in their field within the pages of an “elite” journal is about as common as finding it anywhere else. Some journals operate on a “for profit” basis and charge submitting authors a small fortune for the privilege of publishing within their pages. Not surprisingly, the relationship between higher publishing cost and more strongly advertised elitist status is very strongly positive. Journals that consider themselves to be elite sometimes successfully make even more dough by cloning themselves to produce additional more specialized versions of the original journal, with every academic library then forced to pay ever higher subscription fees. Other journals are non profit and are published by scientific societies because their mission is to disseminate knowledge in the field. Period. To improve peer review, perhaps we should start by ensuring that every student in every field knows this distinction and understands the consequences of these two publication models. For what it’s worth, when I review grants, I don’t count short and sexy supporting publications that were published in elitist journals as much as I count lengthier pubs in solid “meat and potatoes” journals published by academic societies. I find them more satisfying to read and digest, often because their peer review appears to have been less politicized.
Cheers
Paul