Skip to main content

Address

ICC 650
Box 571014

37th & O St, N.W.
Washington, D.C. 20057

maps & directions
Contact

Phone: (202) 687.6400

Email: provost@georgetown.edu

 

A Missive from Antarctica

Posted on

As I mentioned last week, my duties as a member of the National Science Board, overseeing the National Science Foundation, took me to Antarctica this week.

We arrived on the western coast of Antarctica at McMurdo Station on Monday afternoon from Christchurch, New Zealand. We landed in a C-130 on an ice shelf off shore and drove about 45 minutes to the base. The weather was clear and warm – about 20°F. The station resembles a mining town, with unpaved streets and industrial-looking buildings.

Through a series of briefings I came to the conclusion that it was best to think of the station as a small city. It must be self-sufficient because of its remoteness. It generates its own electricity and manages a water treatment plant and a fire station. It imports all food and fuel. It operates large numbers of unusual vehicles, uniquely designed to move scientists and equipment to field sites. There are snow tractors, helicopters, fixed winged airplanes, pickup trucks, and vans. Every bit of the refuse of human activity is packed for shipment off the continent, back to the US. This is the start of the busy season, with about 900 people currently at the station. The ratio of staff to scientists seems to be about 3:1.

There are reminders everywhere of the age of explorers, which launched the study of the continent. Robert F. Scott’s original cabin is nearby, protected by a charitable trust out of New Zealand. A cross on a high hill nearby was placed to remember Ernest Shackleton. A small chapel has memorials to others who died while working in this environment. The current station is led by a set of people passionate about their work and obviously feeling the sense of adventure that guided the explorers of the late 1800’s. But now most are driven by the possibility of scientific discoveries that the continent may permit.

It is the summer season, where the sun is up all 24 hours and moves around a tight circle. In fact, they say the next sunset will be in March 2017! Then a winter of all darkness lasts another six months.

Tuesday, we flew for three hours from McMurdo to the South Pole station at an altitude of about 9,000 feet in the interior of the continent. It was about -25°F when we landed. The station is a beautiful 8-year old facility. Nearby teams are doing ice core extraction to study climate and environmental changes over thousands of years. The method uses the fact that snow falls at the South Pole continuously, packs down and forms ice over time. Another important study uses a one-kilometer cube of sensors to capture the evidence of neutrinos, massless subatomic particles so small that they pass through the earth. There are astrophysics studies using a 10-meter telescope, using the pure atmosphere to peer into the heavens. The facility was supplied only by airplane until a few years ago. As we’re here, there is a convoy of tracked vehicles making their way from McMurdo towing fuel to the site. There were 150 students, postdoctoral fellows, faculty, and operations staff on site conducting the studies from universities all over the world.

We were treated to some South Pole humor: “Which way is North?” I was able to walk around the world…by walking around the South Pole marker. We took some pictures, one with me holding the Georgetown banner!

I find it all a lot to take in. The diversity of the ongoing science work is impressive. There are biologists studying the adaptive attributes of unusual species that populate the waters. There are glaciologists studying the movement and melting of ice onshore and offshore. There are physicists studying cosmic background radiation in an attempt to understand the big bang and inflation afterward. There are geologists studying evidence that at one time the continent was filled with lush forests and dinosaurs. But there are also mechanics trying to keep equipment operating in frigid environments. There are cooks and dishwashers. There are caterpillar operators moving snow. There are lab technicians, IT staff, air traffic controllers, fire fighters, and a health care staff.

The whole continent is a unique environment. There are scores of countries that are part of a treaty to work together on the continent. There are sharing arrangements — the New Zealanders share with the US some wind-generated electricity. The US helps the Italians, and the Italians provide a flight to the US. A NASA weather balloon lands in a remote area, and the Chinese, Australians, and Americans cooperate to retrieve it. It’s a model of inter-nation cooperation that the world doesn’t commonly see. For that reason alone, it makes an American proud to see the active pushing of the frontiers of science.

Sarah Johnson and Angela Bai, my Georgetown colleagues who are working on an NSF research project, arrive on Thursday (after a day-long weather delay) to launch their fieldwork. I’ll get a sense of their research setup Thursday night. They’ll be here for a month; I leave Friday so we won’t overlap much. I’ll do one more report on Antarctica next Wednesday.

A Personal Note – Off to Antarctica

Posted on

The day after Thanksgiving, I’ll be traveling to Antarctica. The trip is part of my duties as a member of the National Science Board, a 25-member group that is part of the governance structure of the National Science Foundation (NSF) and advises the President and Congress on science policy matters. NSF is a key agency in funding basic social and natural sciences, engineering, and science education activities.

One of the important units at NSF is the Division of Polar programs, which funds and oversees research both in the Arctic and Antarctica. Despite impressions that are suggested by most two dimensional map projections of the world, Antarctica is a huge continent (50% larger than North America). Indeed, one ice shelf near the continent, which is a key focus of research on the rate of melt, is the size of Spain!

The US manages three research stations on the continent, and most all the NSF research is based out of these stations. In addition, there are two research vessels for off-continent research and heavy icebreaker ships to help assure a supply of necessary provisions. Small planes and helicopters ferry researchers from the three stations to field sites to do the research. There are approximately 30 other countries that have year-round or seasonal research stations on the continent. Some estimate that there are about 4,000 people during the summer months and 1,000 over the winter months.

Doing research in the hostile environment of the Antarctic is difficult and expensive, with hundreds of millions of dollars spent by the US annually. Hence, one of the roles of the National Science Board is to assure that the taxpayers’ money is spent effectively, funding the most important science that the scientific community can produce. For example, a current issue is that many of the facilities were built decades ago, with concerns about the impact on continuation of key efforts. Three members of the Board will visit McMurdo station on the coast and then the South Pole station.

A special treat for a Georgetown provost going at this time is that Professor Sarah Johnson, with a PhD in planetary science from MIT, will be leading a research effort in Antarctica connected to her interest in the evolutionary course of planets and the question of life on Mars. Sarah has a joint appointment in the Science Technology and International Affairs program in the School of Foreign Service and the Department of Biology in Georgetown College. In true Georgetown fashion, on her Antarctica team will be a Georgetown undergraduate, Angela Bai, who will use the research as part of her thesis. We’re attempting to have a little Hoya gathering during the visit.

Although Professor Johnson’s research regarding Mars may seem unrelated to Antarctica, a common question for both is how life or evidence of former life can exist in extremely harsh conditions. Her research site is in the “Dry Valleys” area of the continent, far from the McMurdo station, examining whether microbial life traces can be identified. Given the geological and climatic conditions of Antarctica, it is likely that few important changes have occurred in the last one to two million years. Sarah and Angela’s time there will be spent both gathering specimens at the site a helicopter ride away and back in a McMurdo laboratory, taking some novel measurements, using real-time DNA sequencing technology, on the specimens.

It’s an honor for me to witness her research in real-time. I’ll try a post next Wednesday from Antarctica, but I’m told the Internet connection is a little dicey. I’ll do my best and forward reflections when I can.

Blue and Gray

Posted on

Last week we had the annual Veterans’ Day ceremony on campus, drawing about 100 or so folks on White-Gravenor lawn. As is true each year, it was a moment to honor those who have served or are serving in the military, their families, and others who support them. Georgetown has an active student veterans’ association, who expertly handled many of the logistics of the ceremony. Many of our illustrious alumni were also military leaders; indeed, the Washington location of Georgetown has allowed it to nurture a supporting culture for those in the military.

One of the speakers was an alumnus, now working in the Department of Defense. He made two important points.

The first was filled with advice about how to connect to one another, especially at a time when only a very small percentage of young persons enter the military. The now trite phrase, “Thank you for your service,” when offered to a veteran, he noted, usually yields no meaningful interaction. Yet both veterans and the non-veterans who offer that thanks would benefit from real interaction. So he counseled both sides to take the time to share the relevant stories – why they chose to enter the military, how it shaped their current thinking about the world, etc. In short, he suggested slowing down and learning about one another. Only then, he asserted, could the true contribution of veterans to the university be realized.

The other set of comments told us about ourselves. He reminded that we sit on hallowed grounds. At one of the most fractious times in US history, Georgetown played an important role. The civil war years were ones where families were split on basic values, brother against brother, cousin against cousin. Vast numbers of the students left Georgetown and fought in the war. More students are said to have fought for the South than for the Union. The school ceased to be what it had been before the war.

At the end of the war, students returned from their military service. One can imagine that some of the students had actually fought one another in the same battles. Imagine the healing that needed to be accomplished! Imagine the inter-group dialogue that was necessary to form real bonds as classmates! Imagine the courage and discipline it took to leave the wounds of past grievances behind them!

Around that time, the blue and gray that are now the school colors were chosen to illustrate the devotion of Georgetown to play a role in healing the large breaches among the nation’s people. The students embraced the mission of moving ahead. It was here that the wounds of war were healed, that harsh words and slurs were forgiven and that a new generation forged a way to work together. Our speaker at the veterans’ ceremony implied this was the history that made the campus hallowed grounds. Knocking down barriers among us; learning to understand another’s perspective; investing in strengthening community – these were the contributions of those years.

Indeed, the legacy of those years can still be felt on campus. We are at our best when we listen across the normal impediments among ourselves and thereby widen our understanding of the world.

Prediction Risks Using Polls

Posted on

Schuman once wrote an article parsing the difference between “survey” and “poll.” He observes the following: “’Poll’ is a four letter word, generally thought to be from the ancient Germanic term referring to ‘head,’ as in counting heads. The two-syllable word ‘survey,’ on the other hand, comes from the French survee, which in turn derives from Latin super (over) and videre (to look).” He goes on to note that the statistical properties of the two practices diverge, and notes a distinct set of higher standards evoked by “survey” than “poll.” Schuman’s etymological hypothesis has come to mind several times over the past few months – first, with the poor results of polls predicting the Brexit vote and more recently, with the polls on the US presidential election.

Much of our knowledge of the US society and economy come from sample surveys, based on probability (“random”) samples of the full population, well-proven standardized measurement, and careful statistical estimation. They uniformly describe the recent past (e.g., last month’s unemployment rate). For convenience, I’ll call them “scientific surveys.”

For the household population, these scientific surveys use “sampling frames,” that cover the whole country, assuring that all eligible persons have a possibility of being measured. Many of the polls used to estimate the sentiment toward candidates use telephone number sampling frames, some only landline numbers. Those without telephone coverage or sometimes only with mobile phones are excluded. Other polls use voter lists, excluding those not yet registered at the time the list was obtained. Still others skip the use of a sampling frame and rely on volunteers who’ve been asked to provide answers to diverse survey questions over time. Some of the poll problems we’re experiencing can arise from people who are excluded from the sampling frames used in the political polling. That is, if the method excludes those with different voting behaviors than those measured, polls can fail.

Scientific surveys very carefully design statistical samples of the full sampling frame so that all persons have a known chance of selection into the sample survey. There are well-accepted theories of statistical inference for such samples allowing accurate estimates of the full sampling frame. Some of the political polling doesn’t use such sampling techniques, but rather assembles collections of people (like the volunteer panel above). There is no theoretically sound reason to believe that such assembly offers accurate statistics of the full sampling frame. Without known chances of selection into the survey, the poll sample has no known prediction value for the election outcome.

Many of the scientific surveys use well-tested measurement techniques that have proven themselves robust to changes over time. The contrast to political polling is not as sharp here, but the polls have a real measurement challenge. The polls attempt to answer questions about a future state: are you registered to vote?, and if no, will you register in time to vote; will you vote?, and, if yes, for whom will you vote? Each of these questions seem simple, in themselves, but there is much research to show a tendency to overreport vote likelihood. Over the decades if the tendency to overreport vote likelihood remains the same, the pollster can adjust for that in estimating the probability that a given respondent will vote. However, if the current election shows different patterns than the past elections, the adjustment can fail. Some of the polls may have suffered from this problem.

Some scientific surveys suffer threats that respondents will underreport stigmatizing or socially undesirable attributes (e.g., receiving welfare payments). Complicated measurement techniques are employed to reduce this reporting error. In pre-election polling some respondents may believe their preferred candidate is not approved by the survey organization. If some social influence is perceived by the respondent in honestly reporting the chosen candidate, there may be a tendency to underreport the vote for a “socially undesirable” choice. Some US polls might have suffered from this.

Scientific surveys attempt to maximize the proportion of the sample cases that are successfully measured. This requires expensive repeated efforts to gain the participation of busy or reluctant sample persons. Many polls do not have the time or money to support those efforts. If the decisions to participate in the poll are themselves related to registration, vote likelihood, or choice of candidate, the polling estimates can be biased. Attempts to remove the bias by differentially weighting cases is common but risky (see Nate Cohn’s piece). Few statisticians believe that the weighting removes the bias when the decision to participate in the poll is itself a function of voter participation. For many of the polls, it’s likely that the proportion of the sample successfully measured is far less than 10%. This means, in essence, that the adjustment procedures for nonresponse is attempting to correct for missing more than 90% of the sample. The risks taken in using such adjusted estimates are high; we should expect that the polls fail as a function of that risk.

Finally, scientific surveys measure the extent of uncertainty in the estimates from being based on a sample of the population not the full population. This is called sampling error or “margins of error” in some popular media. These measure only the uncertainty due to sampling, not all of the errors above. The larger the number of respondents to the survey, other things being equal, the smaller the error due to sampling. However, margins of error in polls are often misinterpreted to cover all uncertainties in the process, leading to overconfidence in differences between candidates’ support.

Scientific surveys are rarely used to predict the future. Polls are explicitly used to predict the future. Hence, polls face another unknown source of problem – the amount of real change occurring between the time of the poll and the election itself. I’ve heard hundreds of pollsters in a post mortem of an election observing that there were large changes in sentiment in the last few days of the campaign, a convenient and untestable hypothesis for failure to predict the outcome.

The pre-election polling world has evolved into using methods that are quite different from those in scientific surveys; many have no statistical theory underlying them. They are rational and reasonable reactions to the need for inexpensive and quick measurement of public attitudes. Indeed, I’m not sure I have better ideas in many cases. However, the risks of prediction errors with such methods can be very large. We should expect them to occur.

Art and Science, Part II

Posted on

My post a couple of weeks ago led to some emails back and forth over the past few days.  The point of that post was that art and science share a key attribute of seeking to understand both ourselves and the world more deeply. Since that post, I’ve been alerted to an interesting other phenomenon linking the two domains.
 
The first observation came from a colleague who noted that our current sharp distinction between the two domains was not always the case.  Indeed, C.P. Snow’s influential lecture at Cambridge that led to the Two Cultures publication was a 1959 event, fairly late in the development of human knowledge.  That discussion is an oft-quoted statement of the difficulty of communication between the humanities and sciences.
 
Before that time, however, there are plenty of examples of those whom we revere as humanists and artists who simultaneously contributed to the sciences.  As Duncan Wu notes in Thirty Great Myths about the Romantics, “Goethe conducted experiments in optics, while Hegel classified forms of scientific endeavor; Friedrich Schelling postulated a Naturphilosophie that would inspire Hans Christian Oersted’s work on electromagnetism and J.W. Ritter’s discovery of ultra-violet light.” (p. 16, Myth 3).  More recently, Wu notes, Vladimir Nabokov was a respected lepidopterist; Michael Crichton was trained as an MD; Kurt Vonnegut trained as a chemist and anthropologist; Norman Mailer’s major was in aeronautical engineering; Conan Doyle trained as a doctor.
 
Further, there are those whom we revere as scientists who are also active in the arts and humanities.  An unusual article (Bernstein et al. (2008) “Arts Foster Scientific Success: Avocations of Nobel, National Academy, Royal Society, and Sigma Xi Members,” Journal of Psychology of Science and Technology, 1:2, pp. 51-63.) examined the arts and humanities activities of Nobel laureates in the sciences.
 
The research compared the general public to members of Sigma Xi, open to most active scientists; to members of the honorific US National Academy of Sciences; and to Nobel Prize recipients.  It measured the percentage in each population reported to have been actively involved in painting, photography, acting, performing music, composing, writing poetry, dancing, producing crafts, etc.  It found that scientist Nobelists were 29 times more likely than the overall public to have participated in such activities, 32 times more likely than scientists in Sigma Xi, and 12 times more likely than members of the National Academy of Sciences. There are many technical problems with the comparisons (e.g., some groups are more international than others), but the results are interesting, if only for the anecdotes reported.  Prominent scientists are disproportionately active in the arts and humanities.
 
For example, the article notes that Einstein played musical instruments, especially in those moments when he was stymied in making progress on scientific work.  Indeed, he noted that the theory of relativity, one of his most important discoveries, was catalyzed during his playing music.  It was an intuition borne of musical perception. The article cites scientists’ comparisons of beauty in art and poetry mirrored in the beauty of mathematical forms and biological systems. In the opposite direction, it references work that finds 20% of the Nobel laureates in literature trained or worked in science and engineering (e.g., founders of kinetic sculpture, modern composers).
 
So, the post a couple of weeks ago (“Art and Science”) noted that both the arts/humanities and the sciences are seeking truth and understanding.  This post notes that there is strong evidence that the arts and humanities help power the sciences and vice versa.  Each domain’s search for insight is aided by knowledge of the other domain.  This fact should bolster our devotion to liberal education.   Exposure to the multiple ways of thinking of the natural sciences, the social sciences, the arts, and humanities appears to propel success in any single field.  They reinforce one another.  There are lessons here for students, their parents, faculty, and even provosts.

Statistics and Elections

Posted on

I’ve lived through enough presidential elections to expect calls from journalists as we approach the voting date. The interviews usually begin asking about whether we should believe the pre-election polls. This year has been a busy one.

The questions are easy to predict. How are these surveys actually conducted? What features make for good practice and what features cause concerns about the credibility of results? Why do the polls obtain different results even though they might be conducted at the same time? How can polls go wrong? Can the failure to predict the outcome of the Brexit vote happen here in the US for the election? What questions do you ask about a survey result to determine whether you believe it?

There are new questions in this election season, however, about surveys and statistical estimates that have nothing to do with predicting the vote outcome. We’ve heard during the campaign that the percentage of the labor force members who are unemployed is much larger than any statistic provided by the Bureau of Labor Statistics. The number of undocumented immigrants is stated to be larger than what demographic analysis suggests. There are claims in social media that government statistics have been manipulated.

In the US, like in many democracies, a set of government statistical agencies, with leadership guided by a strong code of ethics of impartiality and scientific rigor, provide statistical descriptions of key features of the society. These agencies, aided by strong regulatory protections, are cordoned off from interference from political influence. In addition to the regulatory protections, their cultures are ones of staunch independence. They are devoted to letting the data speak, whether they produce good or bad news for the politics of the current leadership.

Protection from political interference is also strengthened by an inquiring press, seeking information directly from the technical staff who produce the statistics. For that reason, another part of the culture of these agencies is to perform their work with transparency in their methods and a self-critical stance to their work. Weaknesses in the statistics are documented explicitly. Questions about procedures are answered openly. Indeed, if political interference were attempted, it’s most likely that the technical staff would let journalists know about the attempts immediately.

As the old saying goes, “every one is entitled to his own opinion, but not his own facts” (usually attributed to Daniel Patrick Moynihan). We should always question statistics, in a continual search to get closer to the truth that we seek, but creating numbers with the purpose of supporting a policy position can’t be tolerated. Claiming that they are manipulated with little evidence, or creating one’s own estimates with undocumented or clearly inferior methods, deserves active critique.

Economic statistics are complicated beasts. They reflect the real lives of families and businesses. They in turn have impact on those lives. The most important statistics are those with the greatest potential impact on those lives.

Over many decades, in democracies throughout the world, we have learned over and over the value of creating and supporting government statistical agencies, insulated from political interference, that produce objective statistics. They are crucial to nurturing an informed citizenry, without which democracies become fragile.

Art and Science

Posted on

We had a wonderful event at Georgetown yesterday — the opening of the new space for the Georgetown Environment Initiative (Regents Hall, Suite 391). An unusual feature of the initiation of the space was a showing of the art of Denise Milan whose creations of mixed media portray links between humanity and nature.

The art transformed the space, enlivening it, and evoking the purpose of the Environment Initiative (GEI) – to convey the import of scientific findings to affect public discourse and beliefs regarding environmental issues.

After the opening, there was a panel discussion sponsored by the Lowey lecture series of the School of Foreign Service (especially the Science, Technology, and International Affairs program), the Global Futures Initiative, and the GEI. A set of experts from science, arts, and the environmental movement discussed the art exhibit. A real focus of the panel was the potential impacts of a collaboration of science and the arts.

One panelist, a Nobel laureate in physics, opined that the arts and sciences share several attributes. One is that they both seek understanding of reality. They are seeking insight and deeper appreciation of how things work and of new meanings of observations around us. They seek new perceptions of individual truths and relationships among them. They value basic foundational principles, but seek innovation in method and insight. They disproportionately value the newness of a piece of work. Hence, they are constantly undergoing change.

But there were also observations about important differences between art and science. Art often evokes emotion in its audience. Science eschews emotions. Science demands very explicit, unambiguous meaning. Art evokes multiple interpretations from the same work. Indeed, a piece of work that evokes uniquely powerful meanings by each viewer is often viewed as especially impactful. Each seek to create something beautiful, but they do so in very different ways.

Scientists have found inspiration for new approaches to their craft from studying art. Artists have found inspiration from scientific perspectives.

The panel agreed on one point: Combining science and art is often needed to influence human behavior. Repeated efforts to communicate the scientific findings about climate change and other environmental changes as the sole influence on action have achieved too little effect. Appealing to basic human emotions – love of family and children, future sustenance of oneself, beauty of nature – may be more impactful.

Hearing scientists and humanists discuss the same topic together, with deep respect for each other’s knowledge was a privilege for me. It would be good for all of us to reach across such boundaries for mutual benefit.

Entrepreneurship as a Way of Thinking

Posted on

Over the past few decades, evidence has grown that one engine of innovation in a society is the business start-up, the proverbial garage-based company. Few days go by without a media celebration of some new, disruptive, entrepreneurial initiative that blossomed into a major enterprise.

Following this trend, many business schools developed co-curricular activities that provide support for students to engage in the creation of new businesses. Some schools even have venture-capital funds supporting those activities.

At Georgetown, the McDonough School of Business has benefited from the success of StartupHoyas, with courses, off-campus activities, and other ways of connecting entrepreneurial alumni with students who aspire to follow in their footsteps.

Entrepreneurship is a way of thinking. Its motivation is solutions, with a premium placed on new approaches. Often these solutions are novel combinations of existing knowledge, previously unthought-of uses of existing technologies, and information-intensive practices

Entrepreneurship is a way of life. A common theme of biographies of entrepreneurs is that they are motivated by building not by money. They are resilient to failure. Indeed, they seek the greater success in bouncing back, in learning from failures of one idea, in order to succeed at an even bolder moves.

Entrepreneurship is an approach to solving problems. It focuses on identifying gaps and unmet needs; it intensely studies human behaviors tied up with the needs; it is devoted to improvement through trial and error, sometimes even randomized controlled trials; it’s conscious of what solutions can scale to larger application.

Indeed, this way of thinking, this way of life, this way of approaching problems is not inherently limited to the business world. We at Georgetown have many examples of students and faculty entrepreneurs in diverse settings. The entrepreneurial spirit is thriving in research centers in which new solutions are being evaluated every day. Each research breakthrough is a bold act of creation, requiring much entrepreneurship. Georgetown is alive with diverse social entrepreneurial thrusts: At the Center for Social Justice, the clinical law activities at the Law Center, the Beeck Center for Social Impact and Innovation, maker spaces at the university, and scores of other units.

In short, entrepreneurial approaches can exist in every human endeavor.

An open question as we think of the future of Georgetown is how can we expose every student to entrepreneurship so that they will be armed with those skills in whatever career they pursue.

Could we enlarge our concept of entrepreneurial activity across the wide spectrum of the university? What learning spaces might be opened to allow students to learn the skills they need? Is learning entrepreneurial thinking compatible with experiential learning initiatives? What can be learned about entrepreneurship in traditional pedagogy? How much of the practice of entrepreneurship in business can translate easily to other fields? How do we construct realistic experiences of failure to help students learn the resilience so powerful among entrepreneurs?

Documenting the Scholarly Product of Academics

Posted on

With the rise of the Internet and digital records of publications, comparisons of quality of universities are increasingly utilizing statistics based on this documentation (e.g., the Times Higher Education university rankings). Many academic fields themselves are comparing the product of scholars by using counts of citations to work (through h-indexes and other statistics). Journals are routinely compared on their impact partially through such citation evidence. Some academic fields have rated their journals into tiers of “quality” based on these numbers. Platforms like Google Scholar and ResearchGate are building repositories of documentation of the work of scholars.

At the same time, we have learned that the practice of evaluating the impact of work is highly variable across fields. Some fields have a scholarly culture in which citations of related work are mandatory for the credibility of the research product. Other fields have no culture supporting citations and view them as irrelevant to the quality of the scholarship in question.

Some fields use the journal article as the basic unit of research output; others use books and monographs as the basic unit. Some fields are built around individual scholarship in which a book-length product might take 6 or more years to produce. Others are built around teams of collaborating researchers in which articles might be produced at the rate of 5-10 per year from each team.

Finally, few would claim that the use of counts of articles/books and citations fully capture the contribution to human knowledge of a scholar’s work. There are too many examples of a single product of a scholar being transformative, sometimes decades after its inception.

While a one-size-fits-all evaluation tool is not appropriate, having tools to compare how universities vary on some standard statistics might be useful. Such was the motivation of Academic Analytics, a faculty-inspired data base that counts articles, books, citations, grants, and professional society awards. The data are available at the count level for many universities across the country, with the ability to compare like departments/units across universities.

Some faculty, given the comments above, see little value in comparing counts of articles and citations and other outputs. Other faculty could find it a useful auxiliary piece of evaluative information, if the counts were accurate.

Over the summer, the Provost Office examined the correspondence between what faculty listed on their curricula vitae (CV’s) and their Academic Analytics records. The categories of scholarly activities analyzed included: academic papers, external grants, scholarly books, and academic awards. The project was conducted in two phases.

In the first phase we examined CV’s of all current faculty members from the McCourt School who were represented in Academic Analytics (AA) for a total of 21 faculty. Two research assistants compared the data from AA with the faculty member’s CV independently and then cross-checked each other’s work for quality assurance. For each scholarly item included in our analysis the item was marked either (1) In AA and on the CV, (2) In AA but not on the CV, or (3) Not in AA but on the CV. During this phase a few analysis parameters were decided:

  1. We limited the analysis to only the years on which AA had data (e.g., AA presents academic papers published only within the last 5 years).
  2. We included data that we knew would not be in AA in order to examine the under-representation of scholarly activities of our faculty (e.g., AA does not collect information on grants from foundations, but we included any such grants found on a CV in our dataset).
  3. For any category 3 items (not in AA but on the CV), we would look at the granular data to determine if there is any obvious pattern that would explain the missing data.

For the McCourt School faculty, Academic Analytics captured scholarly books fairly accurately, missing 5%. The situation worsened for academic papers, where 35% of articles published and listed on CVs were not captured. Surprisingly, several articles in the area of economics, a standard disciplinary area, were not captured because they were in venues that were not covered by the data base. About 30% of the grants obtained by McCourt faculty were missed, mostly because they were private foundation grants, not covered by the database.

In the second phase of our project we collected AA data from 350 randomly chosen faculty members in the College. We were able to find current CVs for 348 of these faculty and conducted the same analysis described above on these CVs. In contrast to the McCourt finding, AA was not as accurate in reflecting scholarly books published by faculty in the College, missing 19%. While the database does not cover books from foreign presses as fully as US presses, not all the missing were published by foreign presses.

Academic papers, including conference proceedings, were poorly represented by AA with only 48% captured. More than a dozen departments had at least one instance of a missed paper. The two departments that faired least well were Computer Science and Psychology. Some of the missing products in Computer Science are conference proceedings. The case for Psychology is less clear, but might be related to publishing in sub-discipline.

As with McCourt faculty, grants awarded by non-federal foundations or institutions were not captured by AA which, therefore, significantly underrepresented faculty grant activity.

Finally, we discovered some articles in the Academic Analytics database assigned to a Georgetown faculty member that do not appear on the faculty member’s CV. We drilled into the case to try to understand the cause. It appears to be a person mismatch, with the database having attached to a Georgetown faculty member’s record a set of articles published by someone else who has a very similar name.
In short, the quality of AA coverage of the scholarly products of those faculty studied are far from perfect. Even with perfect coverage, the data have differential value across fields that vary in book versus article production and in their cultural supports for citations of others’ work. With inadequate coverage, it seems best for us to seek other ways of comparing Georgetown to other universities. For that reason, we will be dropping our subscription to Academic Analytics.

Learning to Understand Others

Posted on

One of the wonderful attributes of Georgetown is its devotion to inter-group dialogue as a tool to deeper understanding and solidarity. We have several organizations and activities that help us understand perspectives of alternative faith systems. We continue “A Different Dialogue” permitting students to develop comfort with, and skill for, discourse on difficult topics in order to foster positive, meaningful, and sustained cross-group relationships (issues like Ability and Disability, Religion, Social Class, Sexual Orientation, and Race and Ethnicity). We have a thriving Center for Social Justice that engages Georgetown students, faculty, and staff in close collaboration with disadvantaged DC community members. The Doyle Engaging Difference Program offers faculty and students opportunities in learning techniques for pedagogical innovation relevant to issues of difference and diversity, identity and inclusion.

All of us are embedded in a larger society struggling to deal with long run effects of racism, fragmented relationships between law enforcement institutions and their communities, and residential segregation by income and race. Research on the recent violent deaths show large differences by race and ethnicity in how those events are understood. Yet many of us seem to struggle with talking to those outside our own group about these important issues.

We are also members of an institution that is grappling with its own history of slavery and the sale of 272 enslaved persons in 1838. It is doing so in a transparent, public way. It is doing so by reaching out to the descendants of those 272 persons and seeking their help in shaping our actions going forward. Through that, it is attempting to help each of us discover our particular role in grappling with this history.

Conversations across groups from different cultures are difficult. We don’t want to offend anyone. We don’t want to embarrass ourselves by using inappropriate words. We don’t want to make someone in the conversation feel guilty about some prior offense. Yet at the same time we seek to understand the world view of others in the conversation.

Such conversations require a set of social skills that all of us can learn, ones that clearly express our sincere interest in another person’s thoughts. None of us come equipped with full knowledge of how different words are perceived in groups with whom we don’t frequently interact. Trained facilitators can help us over these hurdles.

The Provost’s Committee for Diversity met yesterday and discussed a proposal to create discussion fora with such facilitated dialogue across subgroup lines. We will use the deep experiences in mounting such discussions within the Student Affairs staff, CNDLS, and the Center for Social Justice. The committee is actively seeking collaborators across the university.

None of us can understand others as clearly as we understand ourselves, but honest dialogue with others is a necessary step to any increase in the understanding of others. This is a special moment at Georgetown; generations from now deserve that all of us be engaged as part of progress on these matters. Learning to talk and listen to one another is a good first step.

Office of the ProvostBox 571014 650 ICC37th and O Streets, N.W., Washington D.C. 20057Phone: (202) 687.6400Fax: (202) 687.5103provost@georgetown.edu

Connect with us via: