Companies are investing a whole lotta resource into cobbling data together behind the scenes, in and out of the tools. If you’re lucky, your company has a team of BI data engineers who do this work for the strategic leaders who aren’t data engineers. If you’re unlucky, loads of folks across the organization are doing this work on their own.
But how wise are either of those paths? Why don’t the tools work as advertised, as sold? And yet. Here we are day in and day out, pulling data from one system to another, with all sorts of fail points along the way.
And what does that cost organizations?
Fivetran did an interesting report recently, FiveTran: The State of Data Management, about the cost of poor data systems. I highly recommend it.
One slice Fivetran considered is a company with data engineers doing the data cobbling. The top line below is their data.
I then took their model and did a rough estimate for when senior leaders are doing the data cobbling. Neither line takes into account the multitudes across the business who are cobbling data every damn day.
So we can get close to a human FTE cost associated with bad data. What’s harder is – what’s the cost to your business of all the strategic things these wicked smart (and expensive) folks *aren’t* doing while they’re cobbling together data? So much lost strategic work on behalf of your company. How do we begin to quantify *that*?
From the Fivetran report
- Companies are overpaying—to the tune of over a half a million dollars a year—for data integration solutions, due to their obsolete reliance on the manual building and management of data pipelines.
- Companies are paying huge sums only to achieve bad outcomes. For all the time and money spent on data pipelines, data is still not fresh, leading to old and error-prone information that is costing companies money.
- Time spent manually building (and re-building) data pipelines is time not spent contributing to business decisions—which nearly all data leaders (97%) agree would improve business outcomes if they were able to.
So while the $s are significant, it’s that 3rd bullet that I’d say is an invisible killer.
A couple points from the fab Dorie Clark:
“Almost every leader wants to make more time for strategic thinking. In one survey of 10,000 senior leaders, 97% of them said that being strategic was the leadership behavior most important to their organization’s success.
And yet in another study, a full 96% of the leaders surveyed said they lacked the time for strategic thinking.”
Yowza.
So we can’t entirely blame the lack of strategic thinking time on the state of our data infrastructure, but cobbling data is eating up a significant portion of time across our organizations.
Ok, Maureen, Get Us Off This Hamster Wheel
Gladly.
We need data we can trust and we need that data sooner rather than later. And we need to stop tying up valuable humans with busy work that serves no one.
The two critical paths are 1. Small Data and 2. Structured Human Collaboration
#SmallData
I’ve recently covered the importance of Small Data in a recent newsletter: Small Data, Big Opportunity?
The tl;dr:
- How is the promise of Big Data delivering to most companies? How is the promise keeping us leashed to Legacy Mountain?
- And might #SmallData provide a better path in to customer insights and out of all this productivity debt caused by our current digital infrastructures?
Structured Human Collaboration
Another topic that is key for me is how silo’d our organizations are. Both the humans and the tech. It’s such a critical topic that I wrote a book on it.
The cobbling of data needs to stop.
The market is awash in solutions that claim to solve for this but their customers are clearly struggling. If business wants to not just survive but thrive, we need to unleash our people from this tiresome work to free them up to be more strategic.
How can we get closer to data we can trust?
One way is through our colleagues. Every functional area has insights the others need but we haven’t been great at sharing to date. Combining insights, collaborating with our colleagues is an excellent path to uncovering strategic insights that had previously been invisible right in front of us.
Start by determining where you are in this Maturity Model
Then, together with your colleagues, get a plan in place to continuously improve. Need help? Just let us know.
We Must Free Up Mental Capacity
Otherwise we stay on the unfulfilling hamster wheel. It’s easier said than done largely because it’s hard for everyone to let go of old frames and structures. The pull of #LegacyMountain is strong.
If you can get colleagues and higher ups to agree to some testing with Small Data and Structured, Cross-functional collaboration, you can land some early wins, expand the program and start a more fulfilling workpath for yourself and your company.
