IDC Report: Calming the Storm with Cloud Native Data Integration

Issue link:

Contents of this Issue


Page 0 of 6

IDC TECHNOLOGY SPOTLIGHT Sponsored by: Matillion Calming the Storm with Cloud-Native Data Integration March 2021 Written by: Stewart Bond, Research Director, Data Integration and Data Intelligence Software Introduction When the term "big data" emerged, it was defined by three Vs: volume, variety, and velocity. There isn't as much talk about "big" data anymore. Instead, data is now described as the lifeblood of the digital economy, defined by scale of volume, velocity, diversity, distribution, dynamics, and consumption. At a macro level, IDC's Worldwide Global DataSphere Forecast, 2020–2024 estimated that 45 zettabytes of new data were created in 2019; it also noted that the amount of data that will be created in the next three years will be more than the amount of data created over the past 30 years. The study estimated that the enterprise was responsible for over half of the new data that was created in 2019. Of that new data, 88% was replicated, 60% was distributed on endpoints and at the edge, and 19% was real time. The effects of modern data distribution, diversity, and dynamics can be seen at the enterprise level, where 95% of organizations have data pipelines running across hybrid-cloud environments, according to IDC's December 2020 DataOps Survey of 401 DataOps professionals in North America. The survey results illustrate that 95% of respondents have data pipelines running across hybrid cloud environments. These pipelines are handling up to 10 different types of data, which is coming from or going to up to nine different data management technologies, including mainframes, relational databases, streaming, NoSQL, and cloud-native data warehouses. Organizations have been on a steady pace of migrating data to the cloud to take advantage of the scalability and elasticity of cloud computing. However, the use of cloud increases data distribution, allows for more diversity, and permits data to be more dynamic because in some cases, fewer controls exist. The global pandemic accelerated digital transformation and created an insatiable appetite for data, which exposed data availability, quality, and privacy gaps and accelerated the move to the cloud. This situation has created a perfect data storm in the cloud, further increasing the scale, diversity, and distribution of data at a much faster pace and hampering efforts to deliver value with data. In addition to the data being more diverse, distributed, and dynamic, a growing number of organizational roles — from technical to business, from operations to strategy, and from the back office to the front office — work with data daily to complete tasks, make decisions, and affect business outcomes. This tide of users is increasing the demand for data consumption throughout organizations. The data must be accessible from anywhere people are working but controlled to ensure the data is being used by the right resource and for the right reason. Enter the cloud-native data warehouse to meet As companies transform to compete better in the digital economy, cloud is a key part of their disruptive transformation journey. Enter cloud-native data integration to help organizations reclaim control of scattered data in the cloud. KEY TAKEAWAY Top benefits of cloud-native technologies: » Efficient infrastructure use » Higher performance » Reduced processing time » Reduced data latency » Faster time to value » Optimized costs » Better cloud ecosystem interoperability » Fewer SaaS data silos AT A GLANCE

Articles in this issue

view archives of Reports - IDC Report: Calming the Storm with Cloud Native Data Integration