Inequalities within Science

There are two thought-provoking bibliometric articles in the Proceedings of the National Academies of Science (PNAS) describing an increasing concentration of influence of a small number of scientists within their fields. The two articles look at different features of the scientific enterprise, the first one noting the growing inequalities of citation rates and the other examining the relationship between field size and dominance of a small number of influential works.

The first article concludes that the “top 1% most-cited scientists have increased their cumulative citation shares from 14 to 21% between 2000 and 2015.” The second infers that a growing size of a scientific field may “impede the rise of new ideas.” The fears raised by reading both papers are that of a throttling back of the dynamism of a field.

Both of the pieces analyze properties of an unusually large sets of scientific peer reviewed journal articles. One is based on over 90 million articles and 1.8 billion citations, data made possible by the Web of Science documentation of scientific product. As with any observational study, assertions of causal relationships are tenuous, but there is a complex of co-occurring phenomena that prompt causal speculations.

First is the growing prominence of the scientific product from western Europe and Australia, in contrast to that of the United States. This is an observation of change over time. The sheer size of the United States scientific enterprise still leads to pre-eminence in many fields, but the trends favor non-US scientific enterprises. The sharp increase in scientific papers from China is a notable global feature.

Second, the level of collaboration among scientists appears to be increasing. Articles with many authors are common in areas of physics but occurs elsewhere given the need for diverse skills to produce new data relevant to the scientific question at hand. For example, the Higgs boson article had over 5,000 different authors; the major social science data collections are now led by teams of researchers not a single investigator. So the growth of fields and the nature of collaboration is correlated over time. To the extent that large collections of scientists continue their collaborations, one can guess how that leads to heavy citations of the leaders of those collaborations.

Third, one paper concludes that large sizes of subfields, as indicated by many papers being simultaneously produced, tend to ossify accepted “truths” of the field. It argues that such high velocity of findings makes it harder for paradigm-shifting discoveries to achieve the level of attention required for prominence. The paper promotes the notion that large numbers of papers overwhelm a scientist’s capacity to absorb the information. More superficial absorption of new information is necessary (e.g., reading just the abstracts of flagship journals). Papers reporting discoveries challenging the accepted “truths” get overlooked in the churning sea of reports. What cognitive psychologies would call “confirmation bias” predominates as a heuristic to deal with the limited capacity to absorb new information.

These conclusions are troubling.

As with all research, there are obvious weaknesses in the PNAS papers. There are tricky problems of self-citation, disambiguating names, and definitions of subfields that plague the data. There may also be an increasing tendency to include more authors in scientific articles, giving visibility to research technicians formerly cited in acknowledgement text. This, indirectly, would yield proportionately more distinct authors in the data set with fewer citations. Both papers use the number of citations of a given paper or a given author as evidence of influence. These are the data the investigators had at hand. The work is not privy to the context of the citation.

One paper uses the number of citations as an indication that the paper is part of the “canon” of the field – a well-established, consensus of the scientists of fundamental truths that are the basis of further exploration. This ignores other reasons for citing a paper. In some fields large data collections are openly shared among thousands of scientists (e.g., the NSF/NIH sponsored social and health survey data collections). The measurements are the joint work of hundreds of scientists. Most all articles analyzing the data will cite the overall description of the initial data collection, as a way of giving the reader the source of the data analyzed. Thus, the “influence” of the cited authors is not from a scientific finding but the production of a scientific data set. In that sense, considering the construction of a data set as part of the “canon” of a field gives a different impression of the findings.

In short, citations are, at best, proxy indicators of the importance of a piece of scientific work. In fields where they are accepted as meaningful, they can be too easily misused. For example, in tenure and promotion reviews, basing the value of a faculty member’s work solely on a citation index, without peer evaluation through careful reading, would be dangerous.

Ever since Kuhn’s work, we have been sensitized to the inertial power of dominant scientific paradigms. I hope the conclusions of these articles are oversimplifications of the state of science. We should be forever arguing about the most valuable conceptual frameworks within scientific fields. Those arguments are the lifeblood of scientific progress. The activities of fields that even indirectly reduce those arguments are dangerous.