There’s more to being data driven than collecting and looking at data. Leaders who seek to use data to drive success must step up and change their own behaviours. The alternative is to end up like Susan and Bob…

A sad story about biscuits

Bob and Susan set up a business selling luxury biscuits online. Susan is CEO, heads up recipe development and the business as a whole, while Bob picks up the tech and data side of things as CTO. They have some great recipes, a solid supply chain, a great website and the biscuit business is booming.

Like all good tech startups, Susan and Bob turn to data to help drive their strategy. They set up web analytics, monitor ad performance, lead times and quality control, stock levels, delivery times, customer satisfaction… There are charts everywhere, and they proudly refer to themselves as a data-driven business.

One January, at a routine business review meeting, Susan and Bob have their attention drawn to a downturn in sales of a particular product line: the Double-Chocolate Super-Bourbon.

Sales have declined rapidly since Christmas and it’s impacting revenue. Staring hard at the downward trend on the chart, Susan suddenly hits on an explaination: it’s January. Sales of the Double-Chocolate Super-Bourbon are down because people are on a New Year health kick.

Great! This is a seasonality thing. Sales will pick up again soon - and we can save some ad spending by bidding less for chocolate bourbon related keywords. Bob and Susan breathe a sigh of relief - and Bob feels quite proud of how they used data to save the day.

Fast forward a couple of months and Susan and Bob’s Biscuit Business is not in such a great place. A major hotel chain has dropped them in favour of a cheaper competitor. The sad thing is that our heroes could easily have lowered their price to stay competitive, if only they’d reacted earlier.

The hotel ran a trial for a few weeks, swapping out their cholcolate bourbons to gauge customer response, before moving all of their biscuit business. Bob and Susan could have grown their organic consumer sales to make up the shortfall in revenue, if they’d advertised more aggressively - but they’d done the opposite.

The dip in sales was nothing to do with Christmas. Susan jumped to the wrong conclusion, and Bob fought hard to support her, because after all, the data showed she was right…

…right?

The missing ingredient is Science!

The obvious problem with Susan and Bob’s response to what they saw in the data was that they immediately jumped to the wrong conclusion about the cause. It was valuable that the data gave them an early warning of a problem, but that value was immediately undermined by misinterpretation of the cause.

Broadly speaking, things should have gone a bit like this:

• Spot the problem using signals from the data
• Establish a hypothesis to explain why it’s happened
• Seek to prove or disprove your hypothesis using data
• Make a plan to respond
• Measure the impact of your response (using data again) and adjust accordingly as you go

This workflow is essentially just standard Scientific Method. You’d do broadly the same things to create a new drug to treat a disease, detect and plan a response to climate change, devise a new investing strategy…
In accademia this is standard practice, and people are kept honest via another critical activity: peer review.

Researchers validate their hypotheses, write up the results and send them to the world at large to spot what they missed. The reviewers look for gaps in logic, missing evidence, further questions to ask… and the process works because everyone holds everyone else to account.

There are things here that any business leader should be very interested in:

• Facts over opinions - a provable logical flow from cause to effect
• Accountability - everyone is held to the same standard and understands what is expected of them
• Objectivity about results and openess to change when presented with new evidence

Better living through Data

While Susan, as CEO, may have made the wrong call in this case, the real problem here was a missing culture of peer review and accountability within the leadership team. Bob, as the senior data person in the room, should have ensured that any hypothesis was appropriately tested and validated before being acted upon (or at least ASAP thereafter): correlation does not imply causation after all!

Whether it’s CTO, CDO or CFO, every leadership team must have at least one person who understands and follows the steps above when working with data. As an aside, those who I have seen do this best have invariably been CFOs. I think it’s a side effect of being focussed on financial impacts and fluent in data manipulation - but it’s something everyone can and should do.

Of course, you don’t have to be data driven. There is nothing wrong with going with your gut. What went particularly wrong for Bob and Susan is that they convinced themselves something was a ground truth, when in reality it was not. Data played an active role in making them worse leaders in this case. Whether this is down to a lack of data literacy, a lack of critical thinking or some cultural factor preventing challenge, I guess we’ll never know.

Data is a powerful tool

Data is a powerful tool for any business - more so for senior leadership than for any other area - but it can cut both ways. If you know how to use it, Data will give you a formidible advantage. If you don’t use it wisely, it can lead you to ruin.

My advice to you is this:

• Use data to spot issues - like an ECG machine for your business, keep your finger on the pulse
• Use KPIs, OKRs and metrics to define and measure success - because objectivity is good!
• Understand the applicaton of scientific method - know what the data is and isn’t telling you
• Build a culture of acountability - empower others to challenge hypotheses and conclusions
• Know the difference between a fact and a hunch - use both to full effect
• Measure your actions as you go - adjust and replan as needed

Here’s a link, in case you’d like to read more about my thoughts on Data Strategy.