Explanation versus Prediction

There are increasing contrasts between most scientific and scholarly enterprises and much of the data science, machine learning, and artificial intelligence protocols. The former is often focused on discovering the mechanisms that produce various outcomes that are observable. What causal mechanisms underlie the process that produce metastatic cancers? What social forces produce widespread lack of trust in social institutions? How does the loss of biodiversity affect social inequalities?

These are often “why” or “how” questions versus “what” questions. Many of the tasks of machine learning and artificial intelligence systems are interest is prediction of an outcome. GPT-4 predicts the next word in a sequence of words based on learning from massive quantities of text. Simpler algorithms predict the likelihood of recidivism of an incarcerated person. Others predict a product we might be interest in buying given various expressions of interest in purchases.

Early uses of new models made popular in data science took as their starting point an existing set of data. The perceived job of the data scientist was to use all the data possible to predict a given outcome of interest. But such a task of prediction does not require the analyst to seek understanding of the causal mechanisms of an outcome of interest. The goal of accurate prediction can often be attained merely by sophisticated analytic techniques. That is, this can happen many times successfully, until it doesn’t happen. Changed circumstances, new populations that experience influences the training data did not, can lead to model breakdown.

It is interesting that many of the social sciences that use data to discover causal mechanisms, often do not attempt a prediction step. Instead, they are interested in the relative importance of alternative causal factors. The debate in fields is not about the accuracy of prediction but about whether the scholars have missed important factors that may influence the outcome or whether they have misunderstood the manner of influence on the outcome of some factor.

Of course, some academic efforts move from the identification of causal factors and estimation of their relative impact to prediction. This is common, for example, in simulation of policy options for interventions in some social outcome. How much would the child tax credit change the number of children in poverty? How much would increasing the number of police reduce the number of burglaries in a city?

Sometimes models similar to those that have been built to understand causal mechanisms are used in such predictions. But such models tend to be much more heavily scrutinized with regard to their applicability, because their designs are open and transparent.

The system of models that underlying many of the prediction tools arising in artificial intelligence are much more complex, because the sole goal of prediction accuracy doesn’t require the scrutiny of causal assertions. The application of models with billions of parameters yields the result that even the inventors of the system cannot explain its performance on a particular outcome. Transparency is lost through complexity.

Prediction models formed with disregard for the causal mechanisms of an outcome are doomed to failure at some point. The real challenge is whether humanity is capable of building general purpose artificial intelligence systems that are aware of such causal mechanisms – in the extreme, the identification of causal mechanisms for all possible outcomes.