Accidental Insight and Forced Experimentation in Uncertain Times

by | 5 May 2020 | Innovation

In early February 2014, a strike saw some stations of the London Underground close for three days. Not all stations were closed, however. Commuters could in many cases still get to their destinations, but were forced to experiment with alternate routes. Academics at Oxford and Cambridge found that around 5% of these commuters stuck with their new route once the strike had ended. The implication here is that a significant number of commuters discovered a more efficient route, and that they did so only because they were forced to experiment by the strike.

In his novel The Plague, Camus’ reflected that “all a man could win in the conflict between plague and life was knowledge and memories.” As disruptions go, a three-day mass-transit strike pales in comparison to a pandemic. It seems likely that the amount of forced experimentation due to COVID-19 will be commensurately larger.

Knowledge and Memories

Already, this crisis seems poised to impart to us a great deal of new knowledge. Researchers are anticipating the availability of useful data on phenomena that do not lend themselves to large-sample randomised controlled experiments. Working from home is an obvious example. Climate change is another. On a more personal level, many of us have been forced to carry out our own experiments: trialling new routes through the digital world to access goods and services we previously accessed in person.

Remote Working

Remote working is now a reality for millions of people in the developed world, but its impact on productivity is poorly understood. Past studies on working from home have tended to involve relatively small sample sizes, or be limited to a single role type or company. In many countries social distancing measures have impacted virtually the entire population. As a result, we will soon have access to sufficient data on remote working to address the multitude of conflating factors inherent in work. The blanket statements that journalists make based on narrow studies and surveys (e.g. “It’s more productive to work from home”, “Working from home is a productivity ‘disaster’”) belie a great deal of complexity. Common sense tells us that factors such as profession, family situation, access to technology, and personality will play some role in determining the efficacy of remote working for an individual. The sheer size of the sample now being created, when combined with modern statistical and analytical techniques such as artificial intelligence and machine learning, will offer us a chance to better understand these relationships between these factors and overall productivity. Moreover, managers looking to understand the way their own teams respond to remote work need not rely solely on studies and surveys. They should soon be able to draw on data from their teams’ actual experience to drive decision-making.

Climate Change

Climate change is another area in which COVID-19 has forced experimentation. Early data suggests that social distancing has caused greenhouse gas emissions to fall steeply. While it stands to reason that an easing of restrictions will see emissions rebound fairly quickly, forced experimentation may result in some lasting changes to our habits and routines, leading to some permanent reductions. If people decide that they like working from home we could see the emissions associated with commuting fall significantly. Likewise, if the business community finds that deals can be made and meetings had over video conference as well as they can in person, business air travel may never return to pre-coronavirus levels. With demand for oil declining enough to see the price literally stray into negative territory, the global community has incidentally demonstrated that it can survive with dramatically less dependence on fossil fuels than it currently does. Determining which behavioural changes have what impact, and which can be made permanent, will be the role of researchers and policy makers.

Consumer Behaviour

Behavioural change is also rife in the commercial world, with consumers embracing e-commerce en masse. Businesses have seen their projections about the gradual adoption of digital business models smashed in a very short space of time. Consumers who would otherwise have been reluctant holdouts have had to make the choice between going without or going online, and a vast number have chosen the latter (Emarsys, a customer engagement platform provider, estimates that 43% of purchases on their platform in March were made by first-time shoppers). The usual suspects have done well— established players like Amazon — but there have also been a number of less conventional attempts to digitise previously analogue experiences. Museums, art galleries, theatre companies, musicians, and even zoos are all attempting to reach their audiences through the internet. Virtual tours of now shuttered museums and galleries seek to offer would-be visitors a substitute to actually being there. Digital concerts are doing the same for music fans and theatre-goers. These forced experiments in the way that we consume goods, services, and indeed experiences, will no doubt have a lasting impact on the way that many companies interact with their customers.

Experimenting with Experimentation

The COVID-19 pandemic has forced experimentation. That many of the experiments that we trial will lead to lasting change suggests two things: that a crisis can provide the impetus for innovation, but also that we are missing out by not innovating under more normal circumstances. Perhaps the key lesson to take from all this will not be the outcome of any of our incidental experiments, but rather a reminder of the value of experimentation itself.


Kevin Fernandez leads the consulting business at Novigi, and is based in the Melbourne office.

For more information about anything you’ve read here, or if you have a more general inquiry, please contact us.

We love sharing our knowledge and insights, and stimulating discussion about data and technology in financial services. 

Browse our most popular articles

Pin It on Pinterest

Share This