Gaining Time, Savings, and Insights via Cloud-Powered Data Transformation

Issue link: https://read.uberflip.com/i/1309708

Contents of this Issue


Page 0 of 3

Market Pulse Gaining Time, Savings, and Insights via Cloud-Powered Data Transformation It's no secret that modern organizations are increasingly relying on analyzing huge and diverse data sets to improve efficiencies, identify threats and opportunities, and position themselves for long-term success. It's also no secret that finding relevant data, collecting it, and—especially—transforming it into forms that can be readily analyzed have proven tough tasks for many. Indeed, a 2019 IDG survey of 200 organizations with 1,000 or more employees found that they faced several daunting challenges: On average, the organizations' data volumes were growing at 63% each month. The average organization was drawing from 400 unique data sources, with one-fifth tapping 1,000 or more data sources, respectively. More than 90% of those surveyed said that it was challenging to some degree to make data available in a usable format for data analytics. Now a follow-up IDG survey has more fully explored the trends, challenges, and strategies associated with preanalytics data management and transformation. The new survey found that technical challenges such as speeding up data transformations and dealing with massive data volumes are only some of the challenges organizations face in this area. They must also overcome a variety of organizational hurdles, ranging from skills gaps to communication breakdowns between data users. Fortunately, a new generation of cloud data management and transformation services is helping address many technical challenges, as well as organizational ones. Overcoming these obstacles has become a top priority in the data-dependent, real-time world of modern business operations. Evolving cloud data platforms open new opportunities As part of its new research, IDG surveyed 200 IT leaders as well as data engineers, scientists, and analysts working at U.S. organizations with more than 1,000 employees. The survey respondents represented nearly 20 industry sectors. More than one-third (38%) of the organizations surveyed said they are already using cloud data warehouses. Long-term, 43% expected to have all of their data in the cloud, with the remainder planning to pursue hybrid models that leverage both cloud and on-premises data warehouses. It's not just data warehouses in the cloud that are poised for rapid growth, however. So are their semistructured data cousins, data lakes. Cloud data lakes can serve as repositories for huge amounts of raw data drawn from different sources and stored in a variety of formats. Data scientists, who require staggeringly large data sets for tasks such as running machine learning operations and other tasks, routinely build specialized queries and algorithms to extract and manipulate needed data from these lakes. Subsets of data residing in data lakes can also be extracted and transformed into consistent and useful formats to populate structured data warehouses (which can also accept data directly from other sources). Rather than data scientists, the primary users of data warehouses are data and business analysts performing business intelligence and other operationally relevant tasks. 1 Source: IDG Survey, 2016 Prepping massive data sets for analytics engines introduces a host of technical and organizational challenges. Powerful and scalable cloud-native solutions can help optimize this business-critical process. Sponsored Content

Articles in this issue

view archives of Ebooks - Gaining Time, Savings, and Insights via Cloud-Powered Data Transformation