Data can reveal – but it also can mislead. Take averages, perhaps the first method we learn to analyze data – and one we cling to. Roger Martin, former dean of the Rotman School of Management, warns it can prevent us from recognizing important insights in our businesses.
In the world of strategy, all analytical guns have long been trained on means, medians and modes, he notes in the Towards Data Science blog. For example: What is the most representative customer behaviour? What is the biggest customer need? What is the average cost of our product? How do most people get information about our product?
Big data has exacerbated that trend. It’s now even easier to look for patterns in what has happened in the past – typically in the middle of the numbers set. And that’s fine, if you want to hone and refine what currently exists. But in strategy, it’s vital to look at what could be. And for that, Prof. Martin argues, you must shift from averages to outliers. Complicating that search can be that outliers might have been scrubbed from the data in an early stage as unimportant (interference). But outliers can be harbingers of change. You need to bring them back to the data set, and focus on them.
“When you think about outliers, ask what about the outlier is totally sensible. Don’t focus on what is silly, stupid or egomaniacal about the outlier,” he says.
“Dylan had a reason for playing an electric guitar. Mutual-fund investors weren’t necessarily unsophisticated. Hackers and geeks just were more willing than the average user to sacrifice computing power for power over their computer. Focus on what could be a not yet evenly distributed future. That will help you be a better and more creative strategist.”
Increased war for talent spurring better benefits offered to contract workers
Five tips for project planning so you don’t go over time or budget
Focusing on data is on the rise. Companies such as Amazon and Airbnb use advanced analytics to drive their business. Even sports teams have been getting into the act – and in some ways leading it, because they are so high-profile – looking to computer analytics for answers on who to draft and when to bunt. Mark Schrutt, a vice-president at the tech research firm IDC, is therefore discouraged that a survey his company conducted in 2019 found barely 30 per cent of business decisions are made primarily on data. Indeed, that dropped to only 20 per cent for developing strategies and operational decisions.
“Gut and intuition may have worked for decades, but no longer cut it when the customer demands a shift with the next tweet or Instagram post, and your industry can be fundamentally changed within a few quarters,” he writes in his book Shifting the Balance.
Leaving aside that the survey result is slippery – how do you give a percentage to data’s role in your recent decisions? – few businesses, of course, depend on the next tweet. And divining when the next disruption may come – and, more importantly, how to profit from it – is not going to flow automatically from data. Yes, data will help. But so will intuition and judgment. They work together, the exact combination varying by the situation.
In some cases, data provides an exact answer. Sensors embedded in machinery can tell us when it needs urgent attention. But mostly, data has to be interpreted, and it can lead us astray – or simply be used by various sides in a workplace argument, each brandishing numbers favourable to their preference, a fondness perhaps developed before they saw (and latched onto) the numbers. In his book The Data Detective, Tim Harford asks what it says about statistics – and us – that the most successful book on the subject, How to Lie with Statistics, “is, from cover to cover, a warning about misinformation?”
His first of 11 rules for handling data comes from Star Wars, specifically Darth Vader’s declaration to his son Luke Skywalker: “Search your feelings – you know it to be true!” You’ll go wrong in handling data if you can’t control your own emotional reactions to the statistical claims you see. “We often find ways to dismiss evidence that we don’t like. And the opposite is true, too: When evidence seems to support our preconceptions, we are less likely to look too closely for flaws,” Mr. Harford points out.
While you want to be wary of your personal feelings, at the same time he advises you not to be too quick to dismiss your personal experiences. He found his personal experience travelling to London on jammed public buses contradicting the official proclaimed average of 12 people per bus. But he was travelling in the busiest part of the day, he realized. “Sometimes the statistics give us a vastly better way to understand the world; sometimes they mislead us,” he says. Sometimes the data is too slow to capture fast-moving trends. Sometimes it’s faulty, as with the data-obsessed Robert McNamara’s stewardship of the U.S. military in the Vietnam War.
Eric Haller, global head of Experian DataLabs, and consultant Greg Satell puckishly point out that it has been said that data is the plural of anecdote. In the Harvard Business Review, they recommend combatting distortion by asking: How was the data sourced, how was it analyzed (the quality of analytical models vary widely), and what doesn’t the data tell us? “If you are managing what you measure, you need to ensure that what you are measuring reflects the real world, not just the data that’s easiest to collect,” they stress.
It goes back to Mr. Harford’s Golden Rule: Be curious.
Look deeper, and ask questions. Separate the averages from the anomalies, the extraneous from the essential, the silly from the scintillating. Your mind is crucial for that – so is your gut.
CANNONBALLS
- “Averages are bad measures,” Amazon CEO Jeff Bezos told his staff in a memo journalist Brad Stone shares in his book Amazon Unbound. “I want to see actuals, highs, lows and why – not an average. An average is just lazy.”
- Who is the better student, asks entrepreneur Seth Godin: The one with a 3.95 grade point average (GPA) or the one with 3.96? They are essentially the same. Just because we can increase the digits doesn’t mean we see more clearly.
- An age of plentiful data can lead to rampant analysis paralysis. To break free, consultant Mike Figliuolo advises being clear up front about what analysis is required versus what analysis is interesting. If you know the analysis is a must-do, pursue it. If it’s just interesting but it won’t change the answer, you shouldn’t be doing that analysis.
Stay ahead in your career. We have a weekly Careers newsletter to give you guidance and tips on career management, leadership, business education and more. Sign up today.
Harvey Schachter