Peer Review and Fake News

While universities have been accused of being slow and out of touch with the modern world, they do have some redeeming features. One of them is the role of peer review in scholarly products.

An analog we see in our daily news feeds is the “fact-checker.” The fact checking features in this “fake news” era have the goal of building a trusted source of assessment. They often focus on a statement of one or two sentences uttered or written by a public figure. Most of the fact-checking reports contain hundreds or thousands of words, presenting evidence justifying a judgment of the truth of the statements made.

There are four features of this approach that deserve note. First, the needed length of the evidentiary discussion of the fact-checker probably limits readership. Second, the fact-checking is always reactive, not proactive. Third, the producers of the fact-checking are publicly known and limited often to a single author. The best of the fact checkers are careful to present multiple viewpoints and introduce the reader to the complexity of an issue. Others are less careful and are subject to criticism about their politics or ideology affecting their judgments versus only factual evidence. Fourth, the fact checkers are not necessarily experts in any of the fields they are checking. To the extent that these three features are common, the impact of the fact-checking is diminished.

In contrast to the practice in popular media, in most academic fields discernment of facts has a different structure.

1. Academic peer review occurs before and after the release, not merely after the release.
The evaluation generally precedes the release of a finding or scholarly product. For example, in grant proposals for research funding foundations, an anonymous set of reviewers read and study the proposed research. Book manuscripts are sent to reviewers who read the draft. Lengthy written reviews are given to the author. Negative reviews stop dissemination. Once the work jumps the hurdle of publication, then reviews by other scholars continue. In some fields, work that has importance is cited by later scholars.

2. The review is conducted by those deeply expert in the field of inquiry of the scholarly product, not by all-purpose “fact-checkers.”
Successful scholars in the same field as that of the author are the reviewers. There are no “all-purpose” reviewers. There are strong cultural norms within most academic fields that review work is an obligation of being an active participant in the field. Hence, the reviews are deeply informed by much prior work in the same area. The reviews probe whether the proposed work is credible and novel enough to merit support and eventual publication.

3. The review is conducted by multiple persons, not a single fact-checker.
The value of the multiple reviewers is greater assurance that diversity of viewpoints is uncovered. It is not unusual for authors reading such reviews to be surprised at some of the comments. Through this process, no one reviewer has the power to determine “truth.”

4. The review process is controlled by the media of dissemination, not by a different medium.
The authors submit their work to these media. Publishers, editorial boards, and editors organize the review of submitted products. They seek to find the novel, the important, and the credible. Only such work is published. Such quality filtering is important for the publisher to maintain their stature in the field.

Each of these differences between the fact-checking culture of modern media and the peer review process of academic fields is important. The academic process slows the release of findings, but it reduces the likelihood that bad work, unreliable findings, and poor scholarship are disseminated.

At times, especially in the internet-dominated world, when I attempt to weed through reams of information, searching for the truth, I admire the academic peer review process even more strongly.