Issue link: https://read.uberflip.com/i/1365400
Every organization has a big data issue that has been masked in different ways. Your first reaction may be this is a business intelligence process and tools challenge that has plagued organizations since time began, not a true big data problem. Without getting into a religious debate about what analytics vs. reporting vs. business intelligence are, the key fact is that every organization has a big data problem. With artificial intelligence and machine learning capabilities starting to come to fruition, it's now more important than ever for enterprises to get a better handle on the data they have, and how to really harness its power to become a data-driven organization. The mental model I have for becoming a data-driven organization is the nervous system of the human body. Nerve endings extend throughout our bodies sending sensory signals to the spinal cord to be processed by our brains and acted upon. It's a model that's being emulated through data architectures that have the ability to receive, process and store data real time from anywhere inside and outside the enterprise. The signals are processed in real time and acted upon by machine learning algorithms. Unfortunately, far too many organizations believe these new capabilities are nice-to-haves, only applicable to specialized data scenarios, or they try to relabel legacy business intelligence platforms as "data lakes." Data dysfunctions Most of us think of big data problems as being about volume. The truth is every organization has data issues beyond volume that have been masked in a number of different ways. Here are a few common data dysfunctions that I've observed: Lonely and discarded data First, many organizations don't realize that a lot of interesting data is thrown away or is just not accessible. Some examples include data like user activity in the application (and how they might use the application in relation to other applications); the telemetry of the infrastructure hosting the application; or older versions of data that are no longer compatible with current table schemas. Second, data is siloed across many applications and data warehouses. While no one application may be "big," together, they are big. So, when the business needs to analyze data across multiple sources, it becomes very challenging. That's because siloed data poses an access problem. Every place where data is stored comes with its own access roles, rules, and ceremonies to be adhered to that can make it challenging to access the data.