Skip to main content


ICC 650
Box 571014

37th & O St, N.W.
Washington, D.C. 20057

maps & directions

Phone: (202) 687.6400



In Praise of Colleagues

Posted on

I recently experienced the death of a close colleague. We had co-authored a few papers and books over the years, and her passing makes me reflect on how fortunate I was to be able to work with her.

My musings led me to think more broadly about the rich relationships that are possible in academia and how important they are to the common enterprise of education and scholarship we pursue.

The scholarly life of academics is often rather hidden from students, who form too many of their impressions of academic life from classroom performances of instructors. Much new research on the impact of higher education on students has shown that working jointly on research with faculty Is a powerful accelerator to lasting value of learning. Having colleagues who show us all how to do this, and involve multiple faculty in such opportunities make us all better,

Further, faculty in the same field who are giving of their time are precious resources at various points in a research project. At the very beginning when we’re selecting the next question to address or the next creation to make, a common issue is how risky a project should be attempted. Big important breakthrough products usually start out with great uncertainty. Projects that are the next logical small increment generally have higher odds of completion but lower impact. Having a good colleague to help assess risk of success and failure is valuable.

At some point in every research project I ever was involved in, there were unexpected and troubling intermediate outcomes — puzzles that made no sense. At a certain point, thinking through the puzzles more and more was a dead end. However, having a colleague listen to your narration of the issues often illuminates new ways of “getting unstuck.” My experience is that oftentimes the new insight came magically merely from the retelling of the issues. What I needed was a knowledgeable listener, who asked a few questions.

Similar experiences occur at the stage of drafting descriptions of the research. All writers get stuck, and colleagues who are willing to read lousy first drafts are treasures. Such relationships depend on great trust because all the weaknesses of the writer are exposed. A colleague who can provide insightful and supportive suggestions is of priceless value.

One of my best memories of learning from colleagues stems from joint teaching activities, especially when the skill sets of the colleagues were complementary to mine. I never exited such an experience without rethinking my own contribution to the class. Several research ideas from such joint teaching led to wonderful collaborations.

Perhaps most important in these kinds of supportive behaviors is the fact that they generate a virtuous cycle. One colleague helps another in one of the ways above; soon the generosity is reciprocated. Intellectual support generates more intellectual support.

So this is a post in praise of caring colleagues. Those of you who enjoy such benefits, it’s worth thanking those who provide them from time to time. My memories of my colleague who died are filled with such sentiments.

Respondent Burden Among Students

Posted on

Every year a host of requests for surveys of Georgetown students arise. Some are from external researchers and university consortia, studying attitudes and behaviors of interest to them. Others come from within, from student organizations curious about their impact, from faculty with research questions, and from staff seeking “customer” reactions.

Each of these surveys seeks precious time from the students. Each of the surveys requires support from those University offices having contact information for students. With the proliferation of surveys comes “survey fatigue.” And with fatigue comes declining response rates. These, in turn, can produce questionable survey results.

Survey research has scientific bases. Small variations in question wording can have big effects on answers. The order of questions can affect answers. The sponsorship of the survey can affect who is disposed to participate. Although surveys merely appear to be sets of questions, and “everybody knows how to ask questions,” surveys are not for amateurs. Bad surveys often yield useless results.

To reconcile these competing interests, last year the University adopted a student survey policy. This policy aims to balance the benefit of survey information with the burden that surveys impose on our students. To achieve this balance, this policy manages and limits the number of University-sponsored surveys that students are invited to take. It coordinates the timing of surveys to maximize their effectiveness. It also promotes good survey design and sound survey practices. Finally, this policy ensures that student surveys benefit the University as a whole.

The Georgetown University student survey policy mandates the following:

  • Anyone seeking to survey 100 or more Georgetown University students must apply to the Office of Assessment & Decision Support (OADS) for permission to administer the survey. Applicants can include administrators, faculty and students within Georgetown, as well as any organizations outside the University. Applicants are strongly encouraged to submit their applications at least 3 months in advance of the proposed survey launch date. The online application form can be accessed by clicking here.
  • All applications to survey Georgetown students will be referred to the Student Survey Oversight Group, which must review and approve any new student survey. Appointed by the provost, members of this group will consider the new survey in light of existing surveys to minimize any duplication and avoid scheduling conflicts. They will review the survey’s design and methodology (e.g., sampling, incentives) to ensure alignment with best practices. They will also verify that the survey complies with FERPA requirements and meets the IRB’s approval, which must be obtained separately. Finally, this group will consider whether the new survey’s findings are likely to serve the broader interests and priorities of the University.
  • The oversight group may approve the new survey; it may approve the survey with conditions attached; or it may decline to approve the survey. Where approved, the principal investigator works with OADS to construct a schedule for the survey. The student survey calendar is available by clicking here.
  • For approved survey proposals, the Office of Assessment & Decision Support will provide student contact information (usually email addresses) required for the new survey. Unless there are compelling reasons to employ a census, the list of students should be produced using a randomly drawn sample. The principal investigator will work with OADS to determine the target population and the minimal sample size needed to yield statistically significant results. Note that no other office may provide this list of student email addresses without the consent of OADS. The principal investigator must be willing to share the survey data and findings with OADS.

Through this policy we seek to reserve the time of students only to the most important measurements and to maximize the benefits of student surveys to the whole university.

An Update on Building an Institute for Racial Justice

Posted on

In February, 2016, President DeGioia announced a variety of steps that Georgetown was to undertake to address the US society’s legacy of slavery and Georgetown’s history of the sale of enslaved persons.

He named a working group of faculty, staff, and students from all three Georgetown campuses. In addition to building a new department of African-American Studies, located in Georgetown College, the group was charged with assessing the best way forward for a Georgetown Institute for Racial Justice, an institute that was a university-wide entity to coordinate existing scholarship in the area and catalyze new scholarship and related activities. The Institute, ideally, would have strong ties with the Center on Research on the Study of Slavery and its Legacies. The working group submitted its report to the president at the end of the academic year, summarizing months of work by devoted members. He approved the report, permitting the next steps in building the Institute.

The group had met with scores of faculty around the university to inform itself about interests in a possible new institute. In essence, the question was posed, “What attributes of an Institute for Racial Justice would make you excited about working with the Institute?” From these meetings, the group met many colleagues, on all three campuses, whose work is related to racial justice. There was real enthusiasm that Georgetown, at this moment in history, could build a uniquely impactful center of scholarship and community outreach.

From the meetings with faculty throughout the university, the working group identified multiple areas for potential activities of the Institute:

Disparities and Inequality (e.g., health, education, income, housing, employment)
What are the sources and dimensions of enduring racial disparities in areas, such as health, education, income, housing and employment? How should we understand the impact of family structures and environmental conditions on social and political outcomes? What are the long-term trends and future projections that define racial inequality in the United States? The working group met with many faculty who are actively tackling these questions.

Advancing Social Structures (e.g., legal, governmental, education systems, medicine, policy) What legal or political structures perpetuate injustice along racial lines? What proven solutions seem to work in areas such as education and medicine to alleviate the power of race as a determinant to community trajectories? How might areas such as banking and voting regulations be transformed to enable full participation in markets and elections? The working group found much activity in this area, notably, active scholarship, educational programs, and outreach activities on prisoners and justice system inequities.

Diasporic Conversations (e.g., immigration, reparation, inclusion/exclusion, assimilation) To what degree is the U.S. experience of race a national, a continental, or even a global phenomenon? How do cognate ways of perceiving and structuring the world, such as ethnicity and gender, intersect with the past and present? Under what conditions do other social groups and structures around the world, such as religion, become “racialized”? How can we understand and analyze the monumental cultural products of the African American experience and their connections to the struggle for justice?

Finally, the working group came to the conclusion that the Institute for Racial Justice could be a major institution in Georgetown’s outreach to the DC community. Faculty engaged in scholarly work that focuses on D.C., including health equity, arts, and legal work, were hopeful that the center would convene faculty in ways that would inspire new research collaborations that could have real practical impact on Washington, D.C. This could lead to cross-field collaborations for those interested in health, education, housing, and legal justice. Such a focus is completely consistent with the strategic goals of enhancing the service of Georgetown to the betterment of its local community. Further, Georgetown has a tradition of exploring challenging social issues from multiple perspectives and through multiple traditions, including arts, performance, and media. The center could support this work in several ways, including taking advantage of our D.C. location by hosting visiting performers. There seems to be great promise that Georgetown could develop a distinctive program in this area.

Finally, the group formed an alliance with those working on the Center for the Study of Slavery and Its Legacies, created as part of the Working Group on Slavery, Memory, and Reconciliation. Both groups believe that the efforts should be coordinated, and we are working cooperatively in forming the two entities.

In the coming months we will launch a search process (university-wide) for tenure-line faculty who will be joint appointments between the Institute for Racial Justice and an educational unit within the University. Next year a speaker series will host prominent scholars working in areas related to Institute.

This is a very exciting time at Georgetown. We have much work to do, but many of us sense that we can create a uniquely valuable resource for the university, the city, and the country, if we work together toward that end.

Interdisciplinary Review of Interdisciplinary Faculty

Posted on

Any large organization is necessarily segmented into specialty areas. Automobile companies have divisions devoted to single brand names within the larger enterprise. Government agencies have directorates, divisions, branches – all oriented to coalescing talent and responsibilities of sub-processes within the larger mission of the agencies. So too universities have schools, departments, programs, areas, fields, etc., oriented around knowledge in particular domains and/or professions recognized as career clusters.

Increasingly, all the organizations above realize that members who have knowledge across the various subunits can contribute to the overall mission in uniquely powerful ways. Hence, the rise of attempts to flatten business organizational structures and create cross-functional work teams. In universities with missions that include improving global conditions, scholars who combine knowledge from multiple domains can offer unique contributions to that mission.

Interdisciplinary scholarship thrives with reward systems that sustain the effort to bridge disciplines. One reward often comes directly – students are attracted to new ways of combining information and the joy of teaching engaged students is a reward. Further, for those domains that are supported with external funding opportunities, rewards seem to be growing, as many research funders are disproportionately funding interdisciplinary work. However, almost by definition, the publication outlets for interdisciplinary work are less prestigious, within any one discipline contributing to the interdisciplinary content. Further, scholars in one of the fields, especially those deeply embedded in that field, may not give high marks to the quality of interdisciplinary work involving their field.

All of this is relevant at any point that an academic is evaluated on his/her performance. In the academy, in addition to annual merit reviews, there are important mid-course pre-tenure reviews (often in the third or fourth year of an assistant professor’s career), and reviews for tenure and promotion to associate professor, and for promotion to full professor.

After having read many promotion dossiers over the years, there seem to be a few important features of an effective evaluation of an interdisciplinary scholar. First, the quality of the internal review by colleagues is enriched when the colleagues collectively represent the various knowledge domains exhibited by the candidate, ideally when they are engaged in similar interdisciplinary efforts. This produces a real peer review. Second, the quality of the external review is enhanced when there are multiple different types of expertise represented – the individual fields of knowledge being combined by the candidate, as well as the interdisciplinary field of the candidate. It is especially useful for external reviewers working in the same interdisciplinary area to comment on the quality and relative impact of the publication media and products of the candidate. It’s to be expected that those representing a single one of the fields may have trouble evaluating the interdisciplinary work, but a good review also needs to determine whether the candidate’s contributions to the single field are useful to the field, if only to test theories of the field.

Interdisciplinary scholarship is best judged, therefore, by a combination of perspectives. Our institution’s synthesis of those perspectives needs to recognize that the critical question is whether the work of the scholar has impact on the interdisciplinary field as well as the constituent individual fields.

The Georgetown Humanities Initiative

Posted on

For several months a group of faculty has led discussions regarding efforts to enrich the scholarship and learning environment for the humanities at Georgetown. Georgetown routinely produces large numbers of works from our faculty in theology, philosophy, history, classics, literature, languages and linguistics, anthropology, and cultural studies. Having faculty at the cutting edge of their fields enhances the learning of our students. While the faculty who collectively constitute the humanities at the university are one of its great strengths, there currently is no university-level organization that continuously nurtures this strength.

We wondered whether we could enhance the scholarship of our faculty by becoming more intentional in designing an environment supportive of these activities. We saw this as an opportunity to unify more fully the scholarly lives of faculty and the learning lives of students. While students in the humanities are exposed to critical thinking, creative writing, and construction of critical argumentation, these activities most often occur within classrooms and offices of faculty mentors. In contrast to the natural sciences, where students can work hand-in-hand with faculty in laboratories doing science, humanities students get fewer opportunities to witness the doing of the humanities – the presentation by scholars of their work to other scholars, the live critique of text-based scholarship, the explanation of how the work was crafted by the author, the description of technique employed to create an object by an artist, the story of how a play evolved over its writing. We could use more structured opportunities for such gatherings of students and the scholars who are their instructors and mentors.

As I have written in the past, the future of liberal education depends on a renewed sense of the contribution of the humanities to the world. As a Jesuit university, understanding of alternative approaches to the human experience is key to building a better world through service to others. Identifying solutions to the remaining world problems needs values, creative skills, and life principles honed in the disciplines of the humanities. “Out-of-the-box” thinking propels the humanities forward; debate about alternative interpretations is essential to all the fields.

Working over several months the faculty group produced a draft document that describes their ideas. It’s being used both to seek more input and to inquire whether there might be external financial support to achieve some of its vision. For example, this past academic year we’ve invited many leaders of humanities centers throughout the country to visit us, give us advice, and forward suggestions on how a university-based humanities organization could be uniquely valuable in the nation’s capital.

The report sketches some of the ideas that might be part of a humanities center — faculty and student fellowships, research and curricular support, interdisciplinary workshops and public symposiums, as well as exhibitions, performances, and visual culture programming. Given our location in Washington, D.C., partnerships between such a center and other cultural organizations seem especially promising. The report notes the opportunity for Georgetown to take a global and interdisciplinary approach to the humanities that situates the traditions and histories of human expression in an international and connected context, valuing the humanities contribution to complex problems facing society. It could develop leadership in the digital humanities, using digital media and computing to advance the ways that we research, teach and understand human expression. Finally, it could play a signature role in the public humanities that seeks new ways to preserve and advance the value of humanities and the arts through cultural institutions and creative public expression and education.

Through the work of the faculty group, we are convinced that Georgetown needs a Humanities Center to achieve its rightful place among the great universities of the world. For that reason, the Office of the Provost has accepted the proposal and during the next academic year will form the Georgetown Humanities Initiative. The purpose of the Initiative is to construct the vision, concept, structure and early activities for a humanities center. The goal will be to maintain the great momentum of the faculty group, as we simultaneously seek philanthropy and research funding support for the center. We will seek to fund collaborations across the humanities and between humanities and other fields in order to create proofs of concept on the work that a humanities center could do at Georgetown.

So, What Difference Do the Social Sciences Really Make?

Posted on

These days we’re all spending a lot of time rethinking how funding for basic and applied research will be sustained. This affects the humanities and basic sciences in almost all fields. It’s thus ever-more important to learn how best to communicate why research in diverse fields is important to the future of the world.

In that regard, it was interesting to see a recent report from the National Academies of Sciences, “The Value of Social, Behavioral, and Economic Sciences to National Priorities.”

The report concluded that the social, behavioral, and economic sciences are key to our understanding of the “human aspects of the natural world,” that their diverse methods of inquiry are of ubiquitous use, and that the education of future social scientists should prepare them to be more data-intensive, interdisciplinary, and team oriented.

The report contained some good examples of social science findings that truly have changed the world. Developments in game theory decades ago now facilitate the matching of kidney donors to those who need them, vastly improving the efficiency of the process and saving more lives. There is clear evidence that social isolation and stress are precursors to hypertension and diabetes; ignoring that knowledge portends poor outcomes. Experiments in the relevant cognitive burdens of “opt-in” versus “opt-out” decisions have led to large increases in savings for retirement among workers, improving the welfare of millions. Cognitive psychological research in visual perception and memory has informed courtroom practices in eyewitness testimony, consistent with findings that prior beliefs and lawyers’ wording of questions can affect reported memories. SBE research has shown that young children simultaneously learning multiple languages can do so easily, leading to bilingual adults with more cognitive flexibility and less likely to experience cognitive decline in aging. Demographic models of life expectancy are key everyday tools in the insurance sector.

One of the issues in communicating the impact is that the findings themselves have so fully been incorporated into common knowledge that their source has been lost. In contrast to other fields of inquiry (e.g., astrophysics), all of us are embedded in social systems, which confront us with many of the behaviors and thoughts that are themselves the objects of study of the social sciences. To support our self-assessment of our own abilities to navigate this reality, we must believe we understand the mechanisms underlying these behaviors. In that sense, we are all social scientists, amateur, though we are. When new basic research in the social sciences becomes known, we can quickly integrate them into our own belief systems. The findings then become obvious, but only in retrospect.

But there are also counterintuitive findings that have trouble bucking conventional belief systems. While it’s often thought that successful people achieve their state because of superior intelligence, repeated social science findings show that the wealth of parents, quality of schooling, and place of birth has big impacts regardless of intelligence. When groups make decisions, it’s better to have them based on all members’ input than to rely on the single best decision maker in the group. Much of the current biomedical research focus on race and gender differences was motivated by decades of social science findings of health outcome differences in these groups, despite those fields continuing to assume the homogeneity of humans. Such counterintuitive findings from the social sciences generally take longer to have their impact on the society.

So many of the remaining unsolved world problems require deep understanding of human thought, beliefs, and behavior. The Academies’ report and others like it are valuable reminders to the importance of basic research in the social sciences to any hope we have of building a better world.

Protecting Human Subjects of Research

Posted on

Over many years the Federal government has constructed a system for peer and community review of research involving human subjects, in order to assure objective assessments of the risks and benefits of the research to the subjects and the larger scientific enterprise. The notion of “risk” includes both the probability of an adverse event and the harm that would occur to a subject with such an adverse event. The application of these procedures began with biomedical research but was also applied to social and behavioral sciences. These procedures are implemented through Institutional Review Boards (IRBs).

As several National Academy of Sciences panels have noted, both the probability of harm and the nature of harm to individual subjects vary between the biomedical fields and others. Hence, the application of human subjects’ protections needs to take into account those differences. Over the past several years these differences have increasingly been recognized by research universities. For example, at Georgetown we have a separate IRB devoted to social and behavioral research.

We want to continuously improve our performance as a research university, and this includes improving human subject protection review procedures. To improve on services, we commissioned an external review of the social and behavioral research IRB was conducted by several members of AAHRPP (Association for Accreditation Human Research Protection Programs), as they are responsible for assuring the highest standards in protection of human subjects. The review team met with the IRB (faculty committee that reviews applications), the IRB staff, the Director of Regulatory Affairs, Dean of Research, Vice Provost for Research, the Provost, and faculty members from Main Campus, the Law Center and the Medical Center.

We’ve received the report and had meetings to discuss how we can implement some of the recommendations:

  • To reduce the burden on investigators completing IRB packages, a new software platform will be introduced within the next year.
  • We will eliminate the requirement that chairs or unit heads on Main Campus approve an IRB application before it is submitted to the IRB.
  • To expedite IRB review of protocols, the IRB staff themselves will determine whether an application is non-human subject research, exempt from review, or eligible for expedited review. These steps allow the IRB members to focus on more important matters.
  • Beyond what is required by federal regulations, scientific review will not take place on the social and behavioral IRB. Further, it is assumed that grant funding and peer-review in the publication process are sufficient scientific review. Undergraduate students must gain input from their mentor(s) to meet the standards of rigorous research.
  • To reduce delays, the IRB will process applications during the summer.

Finally, we learned from our benchmarking that we should encourage more Main Campus faculty active in research to serve a term as IRB members. Such membership will assure that the IRB can execute effective peer review processes intended in the regulations.

Our hope in making these changes is that rigorous human subject protection review can also be expeditious, and that the efforts of the IRB members are disproportionately focused on higher-risk research.

We congratulate all those providing input and evaluative comments to this review process and thank the IRB staff and members who will execute these innovations.

Emotion and Rationality

Posted on

One of the interesting differences among academic disciplines is the relative role of emotion versus rationality in the culture of the field.

The power of poetry lies in its compact generation of emotions — so too, the words and movements of a play, the text-based creation of a character in literature, the brushstrokes of a painting, and the movements of dance. But mathematics, philosophy, physics, and many of the empirically based fields have a structure that values a very rigid protocol of evidence. Indeed, the presentation of scientific findings in many fields is limited to strictly factual description. There aren’t too many adjectives and adverbs in scientific papers relative to writing in other fields.

On the other hand, it is not unusual for mathematicians to talk of the beauty of a proof or for physicists, of the elegance of a theoretical result. Further, scientists can be deeply passionate about their work, which is a source of inspiration to the students they mentor. Emotion is often part of doing science but rarely part of its product.

While different disciplines value differently the role of emotion and the role of rational thought and logic, it seems increasingly clear that many day-to-day decisions are complicated mixes of emotion and rationality. For example, several scientists have noted the weakness of arguing policy changes consistent with scientific findings without appealing to emotions connected to the policy change. More and more, narrative construction (and who captures the narrative first) seems to predominate the news cycles more powerfully than the facts connected to them.

It’s interesting in this regard to note the increasing attention of scientists to the power of storytelling. Stories often have their power because of the emotions they generate. This also makes them more memorable, as the cognitive psychologists have taught us. Hence, when stories that pack emotional punch can exemplify a scientific finding, the memory of the scientific finding can be enhanced. Memory of the findings is key to any chance that the findings might shape opinions about any issue relevant to the findings.

Of course, the weakness of stories is that they often don’t capture all the subtleties of complicated scientific findings. They are powerful partly because they are simple and accessible, so as scientists embrace this new communication genre, they grapple with critiques of oversimplification.

The stories that may be most powerful are those that relate directly to a person’s welfare, where the emotions generated within the listener are ones of fear, joy, or sadness via empathy. In communicating research findings, such stories are much more easily generated for biomedical research than for basic research. Biomedical research findings often translate directly into saved lives. Basic research often lacks such direct input into the human day-to-day. So emotion-filled narratives on basic research are more difficult to construct. In that regard, emotion-generating stories for basic research seem to have two lines: a) a story about the original discovery, retelling the hundreds of failed attempts at a finding, the resilience of the researcher in bouncing back from repeated failures, the suffering of a career stalled, and then the eventual glory of the findings (e.g., Thomas Edison creating a sustainable electric light), b) the joy of a labor-saving device (e.g., a smart phone), the thrill of a collective accomplishment (e.g., human space flight), or the solution of a long-lasting problem. The story starts at the end – the wonderful new state of the world – and traces that state back to the original basic research findings.

However, both of these ways to pack emotional memory cues into basic science require more of the listener – the first, the trials of the researcher, requires some understanding of how research works; the second, the source of a wonderful new state launched by basic research, requires the patience of the listener and understanding of the causal chain.

I applaud real attempts at researchers communicating their work to lay audiences through stories that evoke emotional reactions. We’re all relatively new at this, but public support of research requires their connecting their lives to the work of researchers. We can’t stop trying.


Posted on

Provosts go to many commencement ceremonies. It’s an attribute not shared by most faculty members or most students. Because the role is one of giving a short welcoming to each set of guests and students, there’s plenty of time to observe and reflect.

The events are filled with joy. It’s heartwarming to see parents and grandparents shouting and cheering when their graduate is announced. It’s fascinating to see the various emotions of the graduates as they walked across the stage – some just beaming with full smiles, others approaching a dean or president with clear trepidation.

The metaphor that stuck in my head over the many ceremonies I witnessed was an agricultural one. Academics are a little like farmers. They plant their seeds early in the scholarly life of a student; for many programs it takes years for the seeds to sprout, be fertilized and pruned. The harvest is the graduation. (The metaphor doesn’t really work because we plant seeds with the first year students, but later that year harvest a different set of students.) The cycle of new student orientation and graduation provides bookends to the year and a predictable sense of accomplishment.

Of course, it’s also a treat to listen to commencement speeches and observe how the speaker handles the temptation to drift into clichés about the ending that is a beginning. This year I observed a surprising uniformity in the themes of the speakers. I could count maybe six different honorary degree recipients and alumni speakers who offered very similar comments:

First, we need to search for people who are different from us or, said by some, find people who disagree with us. We should engage them, actively trying to understand how they view the world.

Second, groups that contain diverse people are more productive. Even if you’re the best decision maker in a unit, better decisions come from you with a group, instead of you alone. Homogeneous groups tend to be unusually swayed by small biases. Freely given input from many folks with different perspectives yields better decisions.

Third, cooperation is a multiplier. When conflicts among different groups can be overcome, and they begin to cooperate, they can achieve much more than each group singly. Cooperation often requires compromise.

It was notable to me that different speakers completely independently chose the same messages to the graduates.

We are all reminded almost daily of the growing wealth inequality in our country, the extreme language inhabiting public discourse, the segmentation of media communities, and the fear of others different from us. Even then, I didn’t expect so many of our commencement speakers to focus on inter-group dialogue as a key challenge to the graduates.

Evidence and Talking Past One Another

Posted on

We all find ourselves talking often these days about what constitutes credible evidence relevant to any decision. This is especially common in discussions about the evaluation of scientific findings.

One feature of the scientific method that isn’t widely understood are the roles of reproducibility, replicability, and falsifiability in science. These three features are part of the disconnect between lay understandings of a “finding” or “evidence” and a scientist’s meaning of those words.

“Reproducibility” is sometimes meant that consistent results are obtained when a similar study is conducted. Some make a distinction between the notion of reproducibility and “replicability,” by which they mean, usually in an experimental setting, that identical results are obtained when exactly the same conditions occur again. In some sense, replicability requires more full allegiance to the original study’s conditions.

Clearly, scientists gain evidence of great value from events that are unique (e.g., naturally occurring but unique astronomical events). So some evidence can never really be duplicated. But in experimental and even in some observational settings, a conclusion is stronger when it can be reproduced in similar but slightly different settings. If there is a failure to achieve that, then science returns to the original finding to see if it can be replicated.

Part of the peer review process in scientific findings is attempting to judge whether the results are reproducible. Hence, scientists are required to describe their methods as fully as is feasible in order to judge the reproducibility of the results. Bad research can be winnowed out through this process.

Scientists are comfortable with the fact also that hypotheses can rarely be confirmed, but can be judged to be false. As many have noted, just because we have never seen a swan of a different color than white, doesn’t mean that all swans are white. Seeing one’s first black swan, however, immediately allows us to judge false the hypothesis that all swans are white.

Scientists are quite comfortable with the fact that the current evidence in any field is a cumulation of plausible hypotheses that have received some support. Careful scientists are quick to note the limitations of their knowledge (anticipating a falsification at some future point). They are comfortable when traditional findings are not reproduced. Often such failure to reproduce a result is the beginning of a breakthrough in the understanding of some phenomena. Falsification is big news that spurs on new work.

For the larger public, not embedded in the scientific method as a perspective, failure to reproduce a result or failure to replicate is troubling. Some viewed the original result as a fact, something that is immutable and fully true. When new scientific results show that the original finding is not true, or true under only a limited set of circumstances, they lose trust with the field that produced the result.

The growing concern with the lack of replicability and of reproducibility is related partially to this lack of understanding that a current scientific understanding is merely that – an understanding of the particular moment in the evolution of findings. At the same time, a field that rewards attempts to replicate and reproduce is one that can more efficiently focus on successful cumulative knowledge building.

I suspect that scientists can do better at communicating the nature of the scientific method. It’s key to maintaining the trust of the public.

Office of the ProvostBox 571014 650 ICC37th and O Streets, N.W., Washington D.C. 20057Phone: (202) 687.6400Fax: (202)

Connect with us via: