Skip to main content


ICC 650
Box 571014

37th & O St, N.W.
Washington, D.C. 20057

maps & directions

Phone: (202) 687.6400



The Limits of Speed in Human Connection

Posted on

It is common, almost trite, to observe that modern life has reduced the amount of time to do many day-to-day tasks. It takes less time to communicate to another because we have instant messaging, texting, email and other near real-time communication devices. We are recipients of news of events around the world within minutes of their occurrence. With the ubiquity of quick electronic communication we can greatly enlarge the number of persons whom we contact at any one time.

We also are increasingly assisted by devices that permit and encourage multi-tasking. We can sit in a meeting listening to a speaker while we simultaneously communicate with someone far away on a different topic, read a passage on an unrelated document, or make a purchase on an e-commerce site. We can capture notes on one issue while other processors in our laptop are scanning for information on some other issue.

These facts lead to, at least in appearance, greater productivity.

Of course, there are costs to this. Sometimes we all feel that we are indeed processing more transactions of one sort or another than we accomplished formerly, but at the same time we’re losing important texture in our lives.

One of the great contributions of the humanities as a collection of fields of scholarship is the use of stories and narratives to convey complex messages effectively. Stories convey fundamental and ubiquitous features of human existence. They can effectively evoke the emotions connected to experiences, which lead to long-lasting memories and rationales for future behavior. Stories defy 140 character limits. Stories told face-to-face seem to add rich texture to our lives.

There are some old social psychological studies of communication that varied the channels of communication from written, to aural-only, to full aural and video (2-dimensional), to face-to-face (audio and 3-dimensional visual). The outcome variables were sometimes effectiveness of collaboration, accuracy of message reception, and interpersonal trust. Face-to-face interactions showed superior performance, especially in matters of emotional content and interpersonal trust building.

So, what’s the connection? One of the great gaps in society today is lack of understanding across groups who do not routinely interact with one another.
Understanding one another requires some time. Personal stories need to be exchanged. The meaning and tone of words need to be understood. The nonverbal behavior needs to be interpreted.

In that context, much electronic communication that allows all of us to “process” large numbers of transactions each day relies on rather “narrow” channels. They are limited to a small number of words or symbols. They’re fast, but they’re lean. The amount of information is often limited. With limited content, the emotional impact is restricted (or worse ambiguous). With restricted channels, interpersonal trust is difficult to build.

It seems likely that jumps in inter-group understanding needs sufficient time, exchange of personal stories, and communication of emotional states that can be really achieved only slowly and in a face-to-face setting.

If true, it feels old fashioned in 2017 to be concluding this. We actually might have to slow down, engage in extended dialogue, to achieve the interpersonal understanding that seems so rare these days. It’s pretty counterculture. But it’s actually something that we can do pretty well at Georgetown.

The Notion of a Sabbatical

Posted on

Hanging out with relatives over the holiday reminded me of how little understood is the role of the sabbatical for academics. December was also the time for review of applications for sabbaticals at Georgetown, and the office finds itself discussing the issue. There aren’t too many work organizations that sponsor sabbaticals, and it’s tempting for the uninformed to think of a sabbatical as an extended vacation.

First, maybe we should document the “why” of a sabbatical. The knowledge that is transmitted within universities is ever-changing. Faculty routinely change their syllabi semester-by-semester to reflect ongoing advances in their fields, to incorporate new research findings, and to describe the latest controversies in the area. With the evolution of global communication, and the fact that much scholarship is taking place outside the US, keeping up with developments is challenging.

An individual scholar’s work must be original to have impact. The second scholar to discover or produce something has little impact relative to that of the first. Sabbaticals are a time to solidify one’s scholarly contributions at the forefront of the field.

Some universities call sabbaticals “research leaves.” I like that nomenclature as one way to dispel the notion that relaxation and recreation are the goals of the time. However, even that moniker misses the varied options of work during a sabbatical.

At Georgetown, applications for sabbatical are really a research proposal. Sometimes they describe high-energy focused efforts to synthesize work into a book manuscript; sometimes it’s the collection of new observations to be analyzed fully upon return to Georgetown. Sometimes the proposal involves a retooling of knowledge and skills in order for the faculty member to develop a new line of research. In that sense, the sabbatical propels a new era in their careers. Sometimes the proposal involves activities within a different country, a different research environment, a business, or a nongovernmental organization. In these, there are usually reciprocal benefits, both to the visited organization and to the faculty member.

Based on faculty surveys a few years ago, we realized that more flexibility in sabbatical use could improve Georgetown. Some scholars’ cycles of work needed something other than the every six-year, one semester research leave option. So we now permit the use of partial sabbaticals of more frequent but shorter duration (see here).

The outcomes sought from a sabbatical are new peer-reviewed research products and books, but also new research collaborations outside of Georgetown. An additional outcome is a scholar returning re-energized with new paths of scholarship identified that can be exploited over several future years.

Of course, Georgetown students are often the most direct and immediate beneficiaries of sabbaticals. Sabbaticals can enrich the quality of instruction they experience from the returning faculty member, resulting in new examples, new project foci, new problem sets, and sometimes new internship opportunities.

Georgetown’s future needs faculty at the cutting edge of their fields; sabbaticals are one of the ways the faculty continue their cutting edge contributions.

Universities and Fake News

Posted on

It appears that we are entering a new phase of social communication, in which falsehoods will be intentionally disseminated via various media. When nation-states or governments do this, we often label this as propaganda. It’s known to have powerful influence on public opinion when the falsehoods can be simply communicated and are repeated over and over. When one-off false news releases occur on diverse topics, saturating the media space, it yields a skepticism about all information. Both of these are undesirable in democracies.

It occurs to me that much of scholarly inquiry uses skills that are well-equipped to help us survive in an era of fake news. Indeed, one feature of scholarly inquiry that spans all the disciplines is its dispassionate search for truth, often in the midst of irrelevant and/or contradictory information. The ability of scholars to discard the untruths and noise in a set of observations and identify the true attributes is a key criterion for their success. All scholars are skeptical of new information until it has proven itself to be true.

So, in this age of fake news, what questions can we apply to each bit of news we encounter? First, and maybe most importantly, we need to ask whether we, independently, can acquire information that confirms or disproves the news item. If we have the ability to observe the same circumstance or make the same observations as the news report, would we conclude that the news item is correct or incorrect?

Second, does the news item provide some transparency to the method of achieving the knowledge that produced the item? Do we know the source of the knowledge — is this a first-hand report from an original observer or a second/third-hand report? If there is an explicit description of how the information was obtained, could anyone replicate this method? Is it a technique that could generate any other outcome or does the method necessarily produce the outcome reported? If there is no description of how the information was obtained, disbelief is prudent, for the time being.

Third, does the news item comport with other known facts related to the item? If the item is true, what events must have preceded it? What events, if any, must follow it, if it indeed happened? If the item is true, what other things must be true, and do we have any evidence that those things are true? If this logic leads to contradictions between known facts and the news time, suspending belief is prudent.

Fourth, do we find the source of the news credible? Does it have a track record or is it a completely new source? If it has a track record, what do we know about its veracity in the past? If it has no track record, was the same content also disseminated by sources known to have their own vetting? If these questions yield no support, disbelief is prudent.

Fifth and last, how long have we known this news? Has sufficient time passed since the announcement that independent sources might have vetted the news? Truth sometimes stubbornly resists immediate revelation. That’s why scientific ethics require replication and peer review before acceptance of a new finding. A suspension of belief, so common among scholars, seem a useful skill in this age.

These are the processes that are common to critical review of new information within all academic fields. It may be time to promote them in our everyday lives.

Reflections at the End of Fall, 2016

Posted on

The life of an academic has more scheduled cycles than most professions. Each fall is a clean slate. New students flock to a class that begins with a fresh start by the instructor. Semesters offer a framework for a beginning, a middle, and an end, all in a short four months. Examinations and final papers are delivered and graded, and grades are submitted to the registrar’s office. The semester is over. A break occurs. The next cycle then begins.

Fall semesters are doubly filled with ends of cycles. The calendar year ends roughly when the semester ends. Such co-occurrence breeds a type of reflection, a looking back on recent events and attempts at synthesis of meaning.

This was a rough year, in our country and on our campuses. We live in a world of widespread conflicts, clashes of beliefs and of economic statuses. Ethnic and racial tensions seem ever-visible. Many people, even those not directly involved in these tensions, seem unsettled. Inequalities on every dimension abound. Across all groups, feelings of disenfranchisement seem more ubiquitous.

Of course, technological change powered by and aiding globalization appear correlated with these issues. It seems clearer now how these two forces, so important to wonderful new benefits to the world, also bring new problems. For example, we now know it is easier to expose ourselves only to information that is compatible with our prior beliefs, given the segmentation of information media.

We at Georgetown are part of this world. It affects us, even while we are attempting to affect it.

But as we reflect on the semester and the year, I think we also possess a set of values that can help us see a way forward. Here at Georgetown we acknowledge that we have been privileged. That privilege brings with it a responsibility.

Many of the world’s problems, complex and large as they seem, are rooted in inter-personal and inter-group conflicts. Georgetown can draw on a 450 year tradition of a search for empathy, understanding of differences, and inculturation of our work within environments quite different from ours. These values seek engagement with the world, especially when aspects of it frighten us. Honesty about our ignorance of others and humility in our approaches serve us well in this engagement.

Reflecting on the year of 2016, it seems urgent for all of us to attempt to understand those who are not in our own group. We need to reidentify what we all share in our worldview. We need dialogue to understand how very different perspectives than ours can be held.

In short, being “women and men for others” cannot succeed when we know too little of the “others.” And so, before we change the world, we might give a little attention to changing ourselves and how we relate to the “others” among us. Indeed, starting with those around us might be a great training ground for our work in the wider world.

What Do the Faculty Think About It?

Posted on

In January 2016, the Harvard School of Education sent out a web survey invitation to both tenure-line and non-tenure-line faculty at the main campus, seeking their evaluations of various parts of their jobs. This was a replication of a survey conducted in 2014 and was a Provost’s Office attempt to gauge whether it was serving the needs of the faculty. The survey was attractive to us because it permitted comparisons of Georgetown faculty evaluations with those of faculty from other universities. The data are kept confidential and the office receives statistical summaries.

The findings of the 2014 survey were used to identify strategic initiatives of the Provost’s Office, including: building a career framework for the non-tenure-line faculty; increasing mentoring and other support for associate professors; increasing protections for interdisciplinary activities; improving the research administration process; and inventing more equitable and flexible retirement plans.

Repeating the survey gave us all a chance to evaluate whether these initiatives were achieving their goals and also whether new concerns had arisen since 2014.

This year about 59% of the faculty completed the survey, much higher than the 42% of our peer universities. The lowest response rates came from non-tenure-line faculty and faculty of color; women had higher response rates than men.

Using the new results, there are two questions that can be answered:

  1. What issues underlie the most prevalent dissatisfactions; which are the source of greatest satisfaction?
  2. What issues are less important or more important in 2016 than they were in 2014?

For the first question, we can compare Georgetown with five peer institutions (listed in the January 13 post). The top two issues on which Georgetown faculty are less satisfied than their peers are clarity regarding the promotion from associate to full professor and lack of support for interdisciplinary work. The top two issues on which we are more satisfied than our peers are health and retirement benefits and the quality of departmental leadership.

In terms of what changed between 2014 and 2016, there is one item that received lower ratings: the effectiveness of communication of priorities by me. I’ll be seeking help from faculty on how I might improve my performance on this. The more notable increases between 2014 and 2016 in faculty satisfaction include tenure policies and the clarity of expectations concerning them, as well as faculty mentoring.

The survey asked some specific questions about the initiatives of the Provost’s Office that were motivated by the 2014 survey. Faculty were asked whether they thought a given area had substantially worsened, worsened, stayed the same, improved, or substantially improved since 2014. The issues on which faculty tended to see improvement or substantial improvement were the full-time non-tenure-line framework and the curricular experimentation (e.g., ITEL and Designing the Future(s)). There was somewhat lower recognition of progress for the initiatives to benefit associate professors. Finally, the least notice of improvement was for the phased retirement and buyout initiatives and research administration. We still have work to do on the latter two.

We are examining differences of opinions, reactions, and concerns of different faculty groups now (e.g., men/women, assistant/associate/full professors, non-tenure-line/tenure-line). We can report findings on those over the coming days.

I thank the entire faculty that took the time to complete the web survey. Just as with the 2014 version, we will use this as a report card on performance by Provost’s Office, and will attempt to identify ways to improve Georgetown on the key issues highlighted by the survey. We’ll seek guidance from faculty in small group discussions about how best to achieve such improvements.

One Final Post on Antarctica

Posted on

A few last reflections on the southernmost continent.

I did indeed meet up with our colleagues Sarah Johnson and senior Angela Bai on my last day in Antarctica. They showed me their miniaturized sequencer that they plan to use to help unlock the mysteries of how microbes survive in the driest and coldest area of the world. There are other scientists interested in such questions because of the belief that the environment of Antarctica best resembles the environment of Mars, the next planet that humans aspire to visit. Doing science out in the field in Antarctica is not for those with casual interests. Helicoptering out to a site takes time; setting up a campsite is tricky. Winds are strong and sometimes prevent air travel back to the base station. Every bit of trash that humans produce must be packaged and later shipped back to the United States. There is no electricity in the field, and solar-powered devices are the norm.

So one tends to find unusually passionate researchers in Antarctica. Further, the mix of ongoing research permits new collaborations that wouldn’t normally form.

For example, one scientist interested in the geotectonic evolution of continental crust sat at breakfast with another never-before-met scientist who studies the climate of hundreds of thousands of years ago. The first was interested in drilling techniques that could relatively quickly get through thousands of feet of ice into the land, rock and soil below the ice. The second was interested in examining ice cores from drilling in multiple locations of interest at a much more rapid speed. (The typical ice core drill might spend 3 years working at one site.) Their new scientific drilling approach is designed to penetrate up to 3,300 meters of ice (nearly 11,000 ft.) and take sample cores in less than 200 hours. This rapid performance will allow a drilling crew to operate from start to finish in about 10 days before moving on to the next drilling site. In essence, more sites can be studied in much less time. A chance meeting at breakfast led to the collaboration of many years between the two.

The Georgetown team is now attempting to do their work in a remote site. You can see from their blog that the weather is affecting their ability to get to their camp. Most of their work has been in the laboratory thus far. I wish them well. They’re collecting original specimens and producing unique data in real time. Truly exciting!

Now that I’m back in DC, I’ve been asked several times what was notable in the visit. The first is probably the size, diversity, and harshness of the continent; the second is its importance for our understanding of climate and astrophysical properties of all that exists; the third is the collection of quite unusual people who work there, both those supporting the research and those doing the research; the fourth is how rewarding is the freedom to think across disciplines in order to solve research problems.

As a National Science Board member, I now know much more about the aging physical infrastructure that support US activities on the continent and can more wisely form judgments about proposals for new investments.

A Missive from Antarctica

Posted on

As I mentioned last week, my duties as a member of the National Science Board, overseeing the National Science Foundation, took me to Antarctica this week.

We arrived on the western coast of Antarctica at McMurdo Station on Monday afternoon from Christchurch, New Zealand. We landed in a C-130 on an ice shelf off shore and drove about 45 minutes to the base. The weather was clear and warm – about 20°F. The station resembles a mining town, with unpaved streets and industrial-looking buildings.

Through a series of briefings I came to the conclusion that it was best to think of the station as a small city. It must be self-sufficient because of its remoteness. It generates its own electricity and manages a water treatment plant and a fire station. It imports all food and fuel. It operates large numbers of unusual vehicles, uniquely designed to move scientists and equipment to field sites. There are snow tractors, helicopters, fixed winged airplanes, pickup trucks, and vans. Every bit of the refuse of human activity is packed for shipment off the continent, back to the US. This is the start of the busy season, with about 900 people currently at the station. The ratio of staff to scientists seems to be about 3:1.

There are reminders everywhere of the age of explorers, which launched the study of the continent. Robert F. Scott’s original cabin is nearby, protected by a charitable trust out of New Zealand. A cross on a high hill nearby was placed to remember Ernest Shackleton. A small chapel has memorials to others who died while working in this environment. The current station is led by a set of people passionate about their work and obviously feeling the sense of adventure that guided the explorers of the late 1800’s. But now most are driven by the possibility of scientific discoveries that the continent may permit.

It is the summer season, where the sun is up all 24 hours and moves around a tight circle. In fact, they say the next sunset will be in March 2017! Then a winter of all darkness lasts another six months.

Tuesday, we flew for three hours from McMurdo to the South Pole station at an altitude of about 9,000 feet in the interior of the continent. It was about -25°F when we landed. The station is a beautiful 8-year old facility. Nearby teams are doing ice core extraction to study climate and environmental changes over thousands of years. The method uses the fact that snow falls at the South Pole continuously, packs down and forms ice over time. Another important study uses a one-kilometer cube of sensors to capture the evidence of neutrinos, massless subatomic particles so small that they pass through the earth. There are astrophysics studies using a 10-meter telescope, using the pure atmosphere to peer into the heavens. The facility was supplied only by airplane until a few years ago. As we’re here, there is a convoy of tracked vehicles making their way from McMurdo towing fuel to the site. There were 150 students, postdoctoral fellows, faculty, and operations staff on site conducting the studies from universities all over the world.

We were treated to some South Pole humor: “Which way is North?” I was able to walk around the world…by walking around the South Pole marker. We took some pictures, one with me holding the Georgetown banner!

I find it all a lot to take in. The diversity of the ongoing science work is impressive. There are biologists studying the adaptive attributes of unusual species that populate the waters. There are glaciologists studying the movement and melting of ice onshore and offshore. There are physicists studying cosmic background radiation in an attempt to understand the big bang and inflation afterward. There are geologists studying evidence that at one time the continent was filled with lush forests and dinosaurs. But there are also mechanics trying to keep equipment operating in frigid environments. There are cooks and dishwashers. There are caterpillar operators moving snow. There are lab technicians, IT staff, air traffic controllers, fire fighters, and a health care staff.

The whole continent is a unique environment. There are scores of countries that are part of a treaty to work together on the continent. There are sharing arrangements — the New Zealanders share with the US some wind-generated electricity. The US helps the Italians, and the Italians provide a flight to the US. A NASA weather balloon lands in a remote area, and the Chinese, Australians, and Americans cooperate to retrieve it. It’s a model of inter-nation cooperation that the world doesn’t commonly see. For that reason alone, it makes an American proud to see the active pushing of the frontiers of science.

Sarah Johnson and Angela Bai, my Georgetown colleagues who are working on an NSF research project, arrive on Thursday (after a day-long weather delay) to launch their fieldwork. I’ll get a sense of their research setup Thursday night. They’ll be here for a month; I leave Friday so we won’t overlap much. I’ll do one more report on Antarctica next Wednesday.

A Personal Note – Off to Antarctica

Posted on

The day after Thanksgiving, I’ll be traveling to Antarctica. The trip is part of my duties as a member of the National Science Board, a 25-member group that is part of the governance structure of the National Science Foundation (NSF) and advises the President and Congress on science policy matters. NSF is a key agency in funding basic social and natural sciences, engineering, and science education activities.

One of the important units at NSF is the Division of Polar programs, which funds and oversees research both in the Arctic and Antarctica. Despite impressions that are suggested by most two dimensional map projections of the world, Antarctica is a huge continent (50% larger than North America). Indeed, one ice shelf near the continent, which is a key focus of research on the rate of melt, is the size of Spain!

The US manages three research stations on the continent, and most all the NSF research is based out of these stations. In addition, there are two research vessels for off-continent research and heavy icebreaker ships to help assure a supply of necessary provisions. Small planes and helicopters ferry researchers from the three stations to field sites to do the research. There are approximately 30 other countries that have year-round or seasonal research stations on the continent. Some estimate that there are about 4,000 people during the summer months and 1,000 over the winter months.

Doing research in the hostile environment of the Antarctic is difficult and expensive, with hundreds of millions of dollars spent by the US annually. Hence, one of the roles of the National Science Board is to assure that the taxpayers’ money is spent effectively, funding the most important science that the scientific community can produce. For example, a current issue is that many of the facilities were built decades ago, with concerns about the impact on continuation of key efforts. Three members of the Board will visit McMurdo station on the coast and then the South Pole station.

A special treat for a Georgetown provost going at this time is that Professor Sarah Johnson, with a PhD in planetary science from MIT, will be leading a research effort in Antarctica connected to her interest in the evolutionary course of planets and the question of life on Mars. Sarah has a joint appointment in the Science Technology and International Affairs program in the School of Foreign Service and the Department of Biology in Georgetown College. In true Georgetown fashion, on her Antarctica team will be a Georgetown undergraduate, Angela Bai, who will use the research as part of her thesis. We’re attempting to have a little Hoya gathering during the visit.

Although Professor Johnson’s research regarding Mars may seem unrelated to Antarctica, a common question for both is how life or evidence of former life can exist in extremely harsh conditions. Her research site is in the “Dry Valleys” area of the continent, far from the McMurdo station, examining whether microbial life traces can be identified. Given the geological and climatic conditions of Antarctica, it is likely that few important changes have occurred in the last one to two million years. Sarah and Angela’s time there will be spent both gathering specimens at the site a helicopter ride away and back in a McMurdo laboratory, taking some novel measurements, using real-time DNA sequencing technology, on the specimens.

It’s an honor for me to witness her research in real-time. I’ll try a post next Wednesday from Antarctica, but I’m told the Internet connection is a little dicey. I’ll do my best and forward reflections when I can.

Blue and Gray

Posted on

Last week we had the annual Veterans’ Day ceremony on campus, drawing about 100 or so folks on White-Gravenor lawn. As is true each year, it was a moment to honor those who have served or are serving in the military, their families, and others who support them. Georgetown has an active student veterans’ association, who expertly handled many of the logistics of the ceremony. Many of our illustrious alumni were also military leaders; indeed, the Washington location of Georgetown has allowed it to nurture a supporting culture for those in the military.

One of the speakers was an alumnus, now working in the Department of Defense. He made two important points.

The first was filled with advice about how to connect to one another, especially at a time when only a very small percentage of young persons enter the military. The now trite phrase, “Thank you for your service,” when offered to a veteran, he noted, usually yields no meaningful interaction. Yet both veterans and the non-veterans who offer that thanks would benefit from real interaction. So he counseled both sides to take the time to share the relevant stories – why they chose to enter the military, how it shaped their current thinking about the world, etc. In short, he suggested slowing down and learning about one another. Only then, he asserted, could the true contribution of veterans to the university be realized.

The other set of comments told us about ourselves. He reminded that we sit on hallowed grounds. At one of the most fractious times in US history, Georgetown played an important role. The civil war years were ones where families were split on basic values, brother against brother, cousin against cousin. Vast numbers of the students left Georgetown and fought in the war. More students are said to have fought for the South than for the Union. The school ceased to be what it had been before the war.

At the end of the war, students returned from their military service. One can imagine that some of the students had actually fought one another in the same battles. Imagine the healing that needed to be accomplished! Imagine the inter-group dialogue that was necessary to form real bonds as classmates! Imagine the courage and discipline it took to leave the wounds of past grievances behind them!

Around that time, the blue and gray that are now the school colors were chosen to illustrate the devotion of Georgetown to play a role in healing the large breaches among the nation’s people. The students embraced the mission of moving ahead. It was here that the wounds of war were healed, that harsh words and slurs were forgiven and that a new generation forged a way to work together. Our speaker at the veterans’ ceremony implied this was the history that made the campus hallowed grounds. Knocking down barriers among us; learning to understand another’s perspective; investing in strengthening community – these were the contributions of those years.

Indeed, the legacy of those years can still be felt on campus. We are at our best when we listen across the normal impediments among ourselves and thereby widen our understanding of the world.

Prediction Risks Using Polls

Posted on

Schuman once wrote an article parsing the difference between “survey” and “poll.” He observes the following: “’Poll’ is a four letter word, generally thought to be from the ancient Germanic term referring to ‘head,’ as in counting heads. The two-syllable word ‘survey,’ on the other hand, comes from the French survee, which in turn derives from Latin super (over) and videre (to look).” He goes on to note that the statistical properties of the two practices diverge, and notes a distinct set of higher standards evoked by “survey” than “poll.” Schuman’s etymological hypothesis has come to mind several times over the past few months – first, with the poor results of polls predicting the Brexit vote and more recently, with the polls on the US presidential election.

Much of our knowledge of the US society and economy come from sample surveys, based on probability (“random”) samples of the full population, well-proven standardized measurement, and careful statistical estimation. They uniformly describe the recent past (e.g., last month’s unemployment rate). For convenience, I’ll call them “scientific surveys.”

For the household population, these scientific surveys use “sampling frames,” that cover the whole country, assuring that all eligible persons have a possibility of being measured. Many of the polls used to estimate the sentiment toward candidates use telephone number sampling frames, some only landline numbers. Those without telephone coverage or sometimes only with mobile phones are excluded. Other polls use voter lists, excluding those not yet registered at the time the list was obtained. Still others skip the use of a sampling frame and rely on volunteers who’ve been asked to provide answers to diverse survey questions over time. Some of the poll problems we’re experiencing can arise from people who are excluded from the sampling frames used in the political polling. That is, if the method excludes those with different voting behaviors than those measured, polls can fail.

Scientific surveys very carefully design statistical samples of the full sampling frame so that all persons have a known chance of selection into the sample survey. There are well-accepted theories of statistical inference for such samples allowing accurate estimates of the full sampling frame. Some of the political polling doesn’t use such sampling techniques, but rather assembles collections of people (like the volunteer panel above). There is no theoretically sound reason to believe that such assembly offers accurate statistics of the full sampling frame. Without known chances of selection into the survey, the poll sample has no known prediction value for the election outcome.

Many of the scientific surveys use well-tested measurement techniques that have proven themselves robust to changes over time. The contrast to political polling is not as sharp here, but the polls have a real measurement challenge. The polls attempt to answer questions about a future state: are you registered to vote?, and if no, will you register in time to vote; will you vote?, and, if yes, for whom will you vote? Each of these questions seem simple, in themselves, but there is much research to show a tendency to overreport vote likelihood. Over the decades if the tendency to overreport vote likelihood remains the same, the pollster can adjust for that in estimating the probability that a given respondent will vote. However, if the current election shows different patterns than the past elections, the adjustment can fail. Some of the polls may have suffered from this problem.

Some scientific surveys suffer threats that respondents will underreport stigmatizing or socially undesirable attributes (e.g., receiving welfare payments). Complicated measurement techniques are employed to reduce this reporting error. In pre-election polling some respondents may believe their preferred candidate is not approved by the survey organization. If some social influence is perceived by the respondent in honestly reporting the chosen candidate, there may be a tendency to underreport the vote for a “socially undesirable” choice. Some US polls might have suffered from this.

Scientific surveys attempt to maximize the proportion of the sample cases that are successfully measured. This requires expensive repeated efforts to gain the participation of busy or reluctant sample persons. Many polls do not have the time or money to support those efforts. If the decisions to participate in the poll are themselves related to registration, vote likelihood, or choice of candidate, the polling estimates can be biased. Attempts to remove the bias by differentially weighting cases is common but risky (see Nate Cohn’s piece). Few statisticians believe that the weighting removes the bias when the decision to participate in the poll is itself a function of voter participation. For many of the polls, it’s likely that the proportion of the sample successfully measured is far less than 10%. This means, in essence, that the adjustment procedures for nonresponse is attempting to correct for missing more than 90% of the sample. The risks taken in using such adjusted estimates are high; we should expect that the polls fail as a function of that risk.

Finally, scientific surveys measure the extent of uncertainty in the estimates from being based on a sample of the population not the full population. This is called sampling error or “margins of error” in some popular media. These measure only the uncertainty due to sampling, not all of the errors above. The larger the number of respondents to the survey, other things being equal, the smaller the error due to sampling. However, margins of error in polls are often misinterpreted to cover all uncertainties in the process, leading to overconfidence in differences between candidates’ support.

Scientific surveys are rarely used to predict the future. Polls are explicitly used to predict the future. Hence, polls face another unknown source of problem – the amount of real change occurring between the time of the poll and the election itself. I’ve heard hundreds of pollsters in a post mortem of an election observing that there were large changes in sentiment in the last few days of the campaign, a convenient and untestable hypothesis for failure to predict the outcome.

The pre-election polling world has evolved into using methods that are quite different from those in scientific surveys; many have no statistical theory underlying them. They are rational and reasonable reactions to the need for inexpensive and quick measurement of public attitudes. Indeed, I’m not sure I have better ideas in many cases. However, the risks of prediction errors with such methods can be very large. We should expect them to occur.

Office of the ProvostBox 571014 650 ICC37th and O Streets, N.W., Washington D.C. 20057Phone: (202) 687.6400Fax: (202)

Connect with us via: