When Data Speaks and Intuition Listens
I’m a big believer in the importance of data and in running an organization based on data-driven decisions.
The scientist in me wants the facts. Good data keeps you from having too rosy - or too dire - a picture in your head as you’re trying to figure out what to do. I think most people would agree that data and metrics are important. Anyone who has tried to run an organization or a department, though, knows that the tricky part is often deciding what should be measured, and how, to actually give you the information you need.
I recently took the Certified Manager of Quality/Organizational Excellence (CMQ-OE) exam through the American Society for Quality (ASQ), which meant spending a lot of time reading the ASQ handbook, and I came across a quote from W. Edwards Deming, one of the founders of the modern quality movement:
“He that would run his company on visible figures alone will in time have neither company nor figures”
That might sound like an odd line to find in an ASQ handbook. But I think experienced leaders understand the point. You shouldn’t ignore data - especially data that disagrees with you. But you also have to understand the limits of the data you have. And if your instincts are telling you that something about the numbers doesn’t make sense, it may be worth listening.
I have one example that sticks out in my mind.
I had been working for years to move a company toward more data-driven decision-making. One of my latest efforts was a spreadsheet to estimate our current full-time staffing and capacity, with projections about three to six months into the future.
We all knew we needed at least one more person. What we wanted to understand was where we needed them and how many people the upcoming workload actually justified. So I gathered the data, built the projections, and got an unexpected answer: according to the spreadsheet, we were one or two people over capacity.
We should have had plenty of people to do the work.
So we listened to the data and held off on hiring for a few weeks.
But at the same time, everyone involved still felt pretty strongly that we needed help. What we were seeing in the day-to-day work simply didn’t match what the spreadsheet was telling us. Eventually, we went back through the model more carefully to figure out what we were missing. It turned out I had forgotten to account for non-direct time: overhead, business development, G&A, and all the other work people do that doesn’t get charged directly to a project. Once I added that time, the picture changed. The data now showed that we were slightly over one person short, which was pretty close to what everyone had been telling us all along.
That didn’t mean the spreadsheet was a bad idea. After that adjustment, it became a useful tool for hiring and capacity decisions, and we kept using it and tweaking it as we learned more. But the experience changed the way I think about “trusting the data.”
Because we followed the numbers too literally, we waited a few weeks to make a hire we probably already knew we needed.
But because we also paid attention to the fact that the numbers didn’t feel right, we found a flaw in the model and made the system better.
There’s a place for both data and intuition in good decision-making. Data can tell us when our instincts are wrong. Sometimes our instincts can also tell us when we need to take a closer look at the data.
Maybe that’s the real value of intuition in a data-driven organization.
When the data speaks, intuition should still be listening.
Not so it can overrule the numbers, but so it can notice when something doesn’t quite add up.