Business Intelligence (BI) software has enjoyed rapid adoption in many companies. These platforms connect to flat files and databases, allowing users to create visualizations of data that can be filtered and with calculations that can be re-grouped with a few clicks.
why does bi exist?
If you’re an IT professional, you’re probably wondering how SQL Server Reporting Services, which has been around forever (and is objectively better than BI tools) has left even a breath of air in the market for companies like SAP Lumira and Tableau.
The short answer is this: BI dashboards can be made by businesses without involving IT. Even Mid-Cap companies have an incredibly difficult time deploying IT effectively in their organizations, clinging to a function-delineated silo model that just doesn’t work. Engaging with decrepit IT departments that are out of touch, out of date, and out of talent is an absolutely mind-numbing experience.
Just how bad is it to deal with these groups? Bad enough that the business world created a $16.9 billion industry to avoid doing so.
The short answer is this: BI dashboards can be made by businesses without involving IT. Even Mid-Cap companies have an incredibly difficult time deploying IT effectively in their organizations, clinging to a function-delineated silo model that just doesn’t work. Engaging with decrepit IT departments that are out of touch, out of date, and out of talent is an absolutely mind-numbing experience.
Just how bad is it to deal with these groups? Bad enough that the business world created a $16.9 billion industry to avoid doing so.
why does bi sell?
Given the amount of pain involved in preparing information for monthly or quarterly business reviews, one would expect anything that helps people leverage work from the month before to be an easy sell. The initial work of building a dashboard – the logic goes – can then be amortized over all the meetings that happen in the future.
Initially, the logic is seductive, but the veneer is thin.
Initially, the logic is seductive, but the veneer is thin.
why will bi (mostly) die?
The Chain of Dependency
While the idea that this kind of tool is democratizing analytics is noble, it comes with some unfortunate baggage: It’s not true. BI tools let you see data that already exists, and usually only after it has been extensively cleansed beforehand.
The reality is that simply visualizing data usually only leads to insights when datasets that have never been seen together can be merged. While some BI platforms have novel modules to help with this task, actually getting the data is almost always a set of manual tasks (someone running reports, which takes time), as is cleansing, merging, and transforming them to the format accepted by your BI tool of preference.
Not to mention actually getting the teams who own the various systems to build the report in the first place, and the days or weeks of correspondence involved before you have anything in your hands at all.
The reality is that simply visualizing data usually only leads to insights when datasets that have never been seen together can be merged. While some BI platforms have novel modules to help with this task, actually getting the data is almost always a set of manual tasks (someone running reports, which takes time), as is cleansing, merging, and transforming them to the format accepted by your BI tool of preference.
Not to mention actually getting the teams who own the various systems to build the report in the first place, and the days or weeks of correspondence involved before you have anything in your hands at all.
Proper Data Visualization Workflow
Looks pretty simple...
Here's the reality: All these data-gathering tasks must be done electronically (part of the data strategy, I've mentioned before). And this isn’t a dream scenario. It’s not something you’ll grow into. Trying a crawl-walk-run game plan to eventually move to automation is like trying to launch a rocket with one booster at a time: You'll spend a ton of effort, and the best case scenario is winding up exactly where you started.
This needs to be square one, and it is the first nail in the coffin for BI. By rolling out a BI tool without an automated way of gathering, cleansing, and merging your data, you’ve also rolled out a dependency on every system that generates the necessary reports, every person in every business unit who runs those reports, and the guy in your business unit who knows how those reports should be gathered, cleansed, and merged inside the BI environment.
This needs to be square one, and it is the first nail in the coffin for BI. By rolling out a BI tool without an automated way of gathering, cleansing, and merging your data, you’ve also rolled out a dependency on every system that generates the necessary reports, every person in every business unit who runs those reports, and the guy in your business unit who knows how those reports should be gathered, cleansed, and merged inside the BI environment.
BI Data Visualization Workflow
Slightly more moving parts...
The long line of dependencies means that, when a dashboard is built, the countdown to failure and re-work has begun.
the market structure
First, a simplifying assumption that is mostly true: Let's say these products are homogeneous. I know someone reading this is about to burst an artery defending one product or another, but stick with me for a moment. No matter which product you choose, you're filling the same need: Turning data into an easily-accessible story.
If one product has a feature that helps with (for instance) data blending, we should remember that the advantage only exists because the task has first been moved from a more dynamic environment (Data Warehouses) better-suited to the tasks. Thus, the small differentiating factors in the products are illusory, and reduce to claims that theirs is the best way to patch together the artwork they want you to break.
If one product has a feature that helps with (for instance) data blending, we should remember that the advantage only exists because the task has first been moved from a more dynamic environment (Data Warehouses) better-suited to the tasks. Thus, the small differentiating factors in the products are illusory, and reduce to claims that theirs is the best way to patch together the artwork they want you to break.
"...the small differentiating factors in the products...reduce to claims that theirs is the best way to patch together the artwork they want you to break." |
|
Now let's put on our Econ 201 hats for a second. As a BI company develops its product, it is better and better able to anticipate client needs and issues that must be handled in their tools. The developers can solve more problems faster and better. The average cost of development goes down. In economics, we call that a natural monopoly, and it presents serious issues for a lot of companies in the BI game.
In a natural monopoly, one person tends to win. Think about it like this: If you start a town and one guy has a pickup truck, he may go around and pick up people's gargbage on some mornings for a fee. Then someone on the other side of town gets a pickup truck to do the same thing for people over there. Now, you come in with a proper garbage truck and collect garbage every day. You can service everyone in town more efficiently than people with multi-use vehicles, and a fellow garbage truck owner couldn't offer cheaper service since his truck would cost the same as yours to run. Ultimately, he wouldn't be able to gain any market share because people only need one garbage man, and would go out of business. Bottom line: You'd be only show in town, and it wouldn't make sense for anyone else to try to compete.
So, who will win in the BI race? SAP Lumira and Microsoft Power BI have tremendous advantages. SAP has managed to church up their databases and RDBMS offerings into something they call ERPs (Enterprise Resource Planning systems) while Microsoft has the most utilized database/RDBMS platform in the business world.
SAP has an easy time crafting a message that sounds like there is some synergy (I'm immediately suspicious of anyone who uses this word) between their products to the uninformed. Microsoft, on the other hand, is actually blending Power BI functionality with SSRS, creating a unified data environment. I believe these two can swim in the same pond for a while, but one will out in the end.
My money is on Microsoft, but a far less risky gamble is that Tableau, Qlik, and others are going to be pushed out of the market. Even their newest software-as-a-service cloud offerings betray their weakness: Pitting Tableau against Power BI might seem fair, but putting Tableau Online against Microsoft Azure feels like watching a terrier fight a tank.
In a natural monopoly, one person tends to win. Think about it like this: If you start a town and one guy has a pickup truck, he may go around and pick up people's gargbage on some mornings for a fee. Then someone on the other side of town gets a pickup truck to do the same thing for people over there. Now, you come in with a proper garbage truck and collect garbage every day. You can service everyone in town more efficiently than people with multi-use vehicles, and a fellow garbage truck owner couldn't offer cheaper service since his truck would cost the same as yours to run. Ultimately, he wouldn't be able to gain any market share because people only need one garbage man, and would go out of business. Bottom line: You'd be only show in town, and it wouldn't make sense for anyone else to try to compete.
So, who will win in the BI race? SAP Lumira and Microsoft Power BI have tremendous advantages. SAP has managed to church up their databases and RDBMS offerings into something they call ERPs (Enterprise Resource Planning systems) while Microsoft has the most utilized database/RDBMS platform in the business world.
SAP has an easy time crafting a message that sounds like there is some synergy (I'm immediately suspicious of anyone who uses this word) between their products to the uninformed. Microsoft, on the other hand, is actually blending Power BI functionality with SSRS, creating a unified data environment. I believe these two can swim in the same pond for a while, but one will out in the end.
My money is on Microsoft, but a far less risky gamble is that Tableau, Qlik, and others are going to be pushed out of the market. Even their newest software-as-a-service cloud offerings betray their weakness: Pitting Tableau against Power BI might seem fair, but putting Tableau Online against Microsoft Azure feels like watching a terrier fight a tank.
Evolving data users
It's no secret that "intuitive" workers (People whose expertise can be swapped out with computational power and a quantitative conception of the problems they solve) are being displaced in droves. As a result, the intersection between people who can intelligently consume data and people with the technical skills to wrangle that data in more powerful platforms than BI is increasing monotonically.
In other words, the people with the ability to gain the most insight from your data are already past the level of proficiency that BI assumes. They need more functionality to answer deeper questions, and they've moved beyond the scope of what BI can provide.
Though the final swing of the hammer may still be years away, this is the final nail in the coffin for BI. An increasingly data- and tech-savvy workforce that blurs the traditional line between analysis and IT is becoming the norm, and change is coming at a quick pace. As this happens, BI's value proposition shrinks to an exclusive focus on delivering low-level analysis at great cost to people who are not particularly good at consuming the results.
It was disruptive while it lasted.
In other words, the people with the ability to gain the most insight from your data are already past the level of proficiency that BI assumes. They need more functionality to answer deeper questions, and they've moved beyond the scope of what BI can provide.
Though the final swing of the hammer may still be years away, this is the final nail in the coffin for BI. An increasingly data- and tech-savvy workforce that blurs the traditional line between analysis and IT is becoming the norm, and change is coming at a quick pace. As this happens, BI's value proposition shrinks to an exclusive focus on delivering low-level analysis at great cost to people who are not particularly good at consuming the results.
It was disruptive while it lasted.