Data Equity

The events over the past few years has refocused attention on members of society who are disadvantaged and less well served by existing institutions. Businesses, nonprofits, and other institutions are attempting to build more accessible and inclusive environments.

This milieu has also heightened the salience of how societies measure their well-being. More attention is being paid to designing and using data for social good. At the nation-state level, almost all central governments produce information on the country’s income, housing, education, labor force participation, consumer behavior, etc. In addition, academics and private sector researchers measure attitudes toward political affairs, pressing problems facing the country, personal and family aspirations. Most of these efforts use sample surveys as a tool of measurement.

There is a beginning of a new wave of self-criticism of those methods in describing those more marginalized in the society. See, for example, here and here. The criticisms reiterate issues that discussed over decades of research.

The first critique involves what parts of populations fail to be covered by the operations of data collection. For example, it is well-documented that country-level censuses tend to miss well-defined subsets of the population. Young children tend not to be reported by adults as members of the household. Transient subpopulations often are missed because they have ambiguous residential locations at any one moment. There is often no effort to enumerate the unhoused population in national censuses.

The measurement techniques used in the surveys sometimes use internet-based platforms. Those populations not well-served by internet connectivity, either because they do not have smart phones or home internet, tend to fall out of the measurement.

The second critique concerns basic concepts applied in surveys, especially those using households as a basic sampling unit. The very concept of “the household” begins to fray at the edges of most societies. The term describes the people who live and eat together in a physical dwelling unit. That simple phrase, “live and eat together,” becomes complicated for certain populations, especially young men, some of whom move among residences over time. They might not consider themselves a member of the household because of the impermanency of their presence. But many surveys use the household as the sampling unit. We tend to fail to measure persons with tenuous ties to any one particular residence.

Another problematic concept affects surveys producing the consumer price index, a basic source of information about inflation of the day-to-day costs of US residents. It measures “consumer units,” a term describing a set of people who share the expenses of housing, food, and other living expenses. Thus, if there is a household with several financially independent people, there could be multiple consumer units within one household unit. As is documented in ethnographic studies, some subcultures in the US pool resources across households. One group of family members at one residence will share income and purchases across them. This possibility is overlooked in price indexes.

All of these are well-documented weaknesses of the survey method. When the use of the survey data describes the full population, these weaknesses may be minor, because the populations mismeasured constitute a small portion of the population. When, however, the survey data are used to extract information about these very subpopulations, large problems can arise. Much of the system of national statistical applied to societies are based on assumptions applicable to the majority culture. The system has challenges when users want to extract information about subcultures whose life experiences diverge from that of the majority.

Global experiences have shown that subcultures that feel disenfranchised from the major societal institutions tend to avoid participating in data collections by those institutions. Addressing some of the mismatches between traditional survey methods and the life experiences of those groups, however, will require new ways of proceeding. Bringing those communities into the measurement design process from the beginning, to seek their input into the methods of the measurement, is a sine qua non of progress in this area.