“It is not necessary to change. Survival is not mandatory.”
- W. Edwards Deming
Successful organizations don’t need to change. They can keep doing what they are doing…at least for a period of time. Successful businesses get very complacent. Managers think they are brilliant for keeping the lights on and making money. That is until someone with a better idea comes along and destroys them. It happens to almost all companies eventually. It is those managers that question every single thing they do to see if they can make it better that survive. Far too many managers are complacent. They fear risk. I’ve seen to many managers not wanting changes because it will impact the stock price and their bonus not realizing that their lack of embracing change will destroy the company. Maybe these are harsh words and a bit over the top, but we can all rattle off the names of companies that died because of complacency. And we have all worked at corporations or for managers that fear change.
“Fat, drunk and stupid is no way to go through life, son.”
- Dean Wormer
In 2022 with words and phrases like AI and Machine Learning having been in the vernacular of almost every manager on the planet for over a decade why is it that so many companies just give data science lip service? Why is it that they don’t elevate the function of deeply understanding the business to the CEO level? Most managers at a certain level are highly adept at putting together beautiful graphs and uber complex spreadsheets that they think help them understand the business. And to a degree they are correct. They give everyone a snapshot of what is going on in the organization. Now the vast majority of these works of numerical art all point to the past. And most importantly what they don’t do is tell you why or where you are going. They don’t tease out subtle patterns and they don’t find complex causal relationships between variables. They don’t find those type of deep meaningful connections that can guide a manager in their decision making. Data science is very often finding that proverbial needle in a haystack. That haystack being mountains of data.
Every business no matter how big or small captures a significant amount of data. Maybe you are a small retail shop capturing sales data in your point-of-sale system. Or maybe you are a 10,000 person firm that’s relatively sophisticated but has never used data science before and generates petabytes of data annually. Whatever the size of your organization data science can provide an immense amount of insight into the inner workings of your business in ways that you previously didn’t know existed. Data science is not just for large, sophisticated organizations with large IT divisions. Data science is a mind set that the organization must embrace. It’s a way of thinking that is different than most managers are accustomed. Even those that are accustomed to working with numbers all day. It is human nature to trust our experience when making complex decisions. But we also fill in the gaps in our knowledge when making decisions and that can lead to bad outcomes.
Now I’m not saying that data science can fix those issues, but it can augment your decision making by giving you another perspective. More data points. More ways to look at a problem. More depth and nuance. It can give you those pieces of information that you never knew existed. And quite possibly it is one of those pieces of information that changed your approach to solving an issue.
Now yes you can use data science for more traditional business problems like identifying new customers to send marketing material or trying to figure out which customers are most alike for recommendations. But those are the easy well trodden problems. So well trodden that not doing them is just plain dumb. It’s like breathing. You just do it. Sure, they help your business and add value but there is so much more value data science can provide.
I once worked for the credit trading division of a large investment bank. Easily a billion dollar a year business. It generated massive amounts of data daily. Through the use of analytics, we had a deep understanding of our traders, sales and counter parties. We brought together hundreds of different data sources to give us a highly detailed view of everyone involved in trading. But yes, we still had those that didn’t believe in what we were doing and questioned it. There were those that had been working in the business for over thirty years and thought they knew it all. If you are doing a job for thirty years and getting paid millions a year, you are allowed to think that way. But human intuition if from from flawless and what we think, and reality often don’t reconcile too well. But ask someone if they are certain that they know what’s going on in one of their clients’ heads and they will guarantee you that they are a mind reader. After all they have been working with this person for decades.
But and there is always a ‘but’, they didn’t have perfect information. Very often they had terrible information which resulted in bad trades and unhappy clients. This is where data science came in. We were able through of our analysis of external data and our ability to tie it all together in ways that were never done before giving them something closer to perfect data or a far more nuanced via of a client. Conversations with clients became radically different when you knew what your client was doing with the rest of the street. It was a total game changer and altered how management thought about the use of analytics. From then on forward it was mandatory to use my team for major decisions. Empirical evidence had to support decision making. Now that is just one of many things we did with the use of data and statistics but suffice to say it had a very large impact on the bottom line. And this is at a relatively stodgy investment bank.
So why data science? It makes your decision making better. Far better. It’s the night vision goggles in the dark of night that provide visibility when you can’t see anything else. It’s the microscope that helps you discover new insights. It WILL be part of every business at some point. Because if you are not doing nobody will be speaking about the company because it won’t be around. But organizations have to be open to thinking differently. Taking on new perspectives and new ways of looking at old problems. And more importantly they need to listen to those that they normally don’t interact. The CEO and the lead data scientist don’t normally go golfing together. Perhaps they should.