Why we should not call for experts instead of experiments

By Jan Michael Bauer

◦ 3 min read 

The recent elections in Germany turned out as the historic loss for conservatives that pollsters have predicted a few weeks before. Responding to the declining poll numbers, the conservative party presented a “team for the future” consisting of several field experts that should help the candidate addressing the big challenges of the country. Their slogan was „Experts instead of experiments“. The message was clear: we know how to solve these issues and voting for another party would be an experiment and therefore risky. While this might appeal to a conservative base, I think this slogan sends a wrong if not hypocritical message.  

Framing an experiment as something uncertain and dangerous that should be avoided taints one of sciences most successful methods and obscures the undeniable level of uncertainty associated with policy decisions.

Even after acknowledging the turbulent times during Merkel’s legacy, remarkably little was done to address the challenges of the future. While some inaction might be attributed to a lack of courage or lobbying by special interests, it certainly constitutes the lack of obvious and simple solutions for the many problems the country is facing. 

The challenges we face are new and unpredicted in magnitude. While few would disagree with the need for action, there is disagreement about what needs to be done. Experts argue with each other, often struggle to persuade their colleagues, and remain unconvinced by the evidence that substantiates the oppositions claims. Fierce debates about the right course of action often overshadow a sad truth that in many cases no one is and really shouldn’t be sure that the proposed path will be the most successful one. Even more disheartening might be that even after the implementation of a policy, we often have a hard time qualifying if the intervention did more good than harm or quantifying these benefits. 

A fundamental problem, particularly economists try to resolve through (quasi-) experimental research methods to understand how a specific policy intervention works. Broadly speaking, they become experts because they do experiments. Pioneers in the study of causal relationships were recently awarded the Nobel Price. Among them David Card who is famous for a study on the effects of a higher minimum wage on employment exploiting a so-called natural experiment.  

Experimentation can help us to find out if our ideas and theories work in practice. They should increase our confidence in the people applying them rather than creating a fear of uncertainty.

Our knowledge that COVID vaccines are effective mostly relies on the result of randomized experimental trials. An approach increasingly used to answer questions in the social sciences. For instance, we don’t know how people will respond to universal basic income, which is why a three-year experimental study is currently on its way in Germany.

Pharmaceutical trials are also designed to show that potential drugs have no sever side-effects. While the necessity to ensure the safety of a drug is quite intuitive, the unintended consequences of non-medical products and services are less straight forward. For instance, social media has been suspected to inadvertently contribute to political polarization and erode democratic processes. While many of these claims are based on anecdotes, recent experimental studies from researchers outside Facebook have added hard evidence to the debate and conclude: 

Our results leave little doubt that Facebook provides large benefits for its users [but also] make clear that the downsides are real. We find that four weeks without Facebook improves subjective well-being and substantially reduces post-experiment demand, suggesting that forces such as addiction and projection bias may cause people to use Facebook [..] it also makes them less polarized by at least some measures, consistent with the concern that social media have played some role in the recent rise of polarization in the United States.

– Allcott, Hunt, Luca Braghieri, Sarah Eichmeyer, and Matthew Gentzkow. 2020. “The Welfare Effects of Social Media.” American Economic Review, 110 (3): 629-76.

There are obvious differences between vaccines and a social media platform, and probably nobody would suggest that Facebook should have undergone a randomized safety study in the mid-2000s before going public. Such products and services develop over time and can be used in very different ways. However, despite these differences there is an open question about the potential side-effects and the burden of proof. To ensure a healthy society, it might be worth considering that at least with a reasonable initial suspicion of harm, also non-medical companies should be obligated to proof their products’ safety using a suitable experimental design.

There will remain many problems where experiments are unfeasible, and, as seen with the development of Facebook, even the results of the best experiment today might not be a valid description of tomorrow in an increasingly complex and dynamic world. Such a world, however, should also humble us and our experts but foster an acknowledgement that there are many questions for which we don’t know the answer. Hence, we should make use of the best scientific methods available to reduce this uncertainty, which ultimately means that we need more experiments and not less.  


About the Author

Jan Michael Bauer is Associate Professor at Copenhagen Business School and part of the Consumer & Behavioural Insights Group at CBS Sustainability. His research interests are in the fields of sustainability, consumer behavior and decision-making.


Photo by Scott Webb on Unsplash