TDWI Pulse Report Infographic: Beyond Lift and Shift

Issue link:

Contents of this Issue


Page 0 of 0

TDWI INFOGRAPHIC SPONSORED BY PROVIDED BY TDWI is the premier provider of in-depth, high-quality education and research in the analytics and data management industry. © 2021 by TDWI, a division of 1105 Media, Inc. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. Email requests or feedback to Product and company names mentioned herein may be trademarks and/or registered trademarks of their respective companies. Matillion is data transformation for cloud data warehouses. Matillion is purpose-built for Snowflake, Google BigQuery, and Amazon Redshift, enabling businesses to achieve new levels of simplicity, speed, scale, and savings. Trusted by companies of all sizes to meet their data integration and transformation needs, Matillion products are highly rated across the AWS, GCP, and Microsoft Azure marketplaces. Dual-headquartered in Manchester, U.K., and Denver, Colorado, Matillion also has offices in New York City and Seattle. Learn more about how you can unlock the potential of your data with Matillion's cloud-based approach to data transformation. Visit us at Cloud migration can accelerate the use of data and analytics to accomplish essential business goals. Although a simple "lift and shift" of existing systems and practices to the cloud can reduce costs and improve performance, data- driven business innovation demands more. Organizations need an architecture that takes full advantage of the benefits of cloud-based BI, analytics, transformation, and data management. Enterprises must evaluate how they can rearchitect key processes such as data extraction, transformation, and loading (ETL) and rethink existing data warehouse and data lake practices to support growth in business-driven, self-service data interaction with the help of the cloud. TDWI PULSE REPORT BEYOND LIFT AND SHIFT: REARCHITECTING BI, ETL, AND DATA WAREHOUSING FOR THE CLOUD BEYOND "LIFT AND SHIFT": REARCHITECTING BI, ETL, AND DATA WAREHOUSING FOR THE CLOUD WHAT'S DRIVING ENTERPRISES TO THE CLOUD? Cloud-based BI, analytics, data transformation, and data management can accelerate business-critical projects that languish with traditional on-premises systems and practices because they can't scale or are slow and expensive to modernize. What else is driving enterprises to the cloud? Leading drivers for cloud-first or cloud-migration strategies include: Source: Page 5 TRADITIONAL ETL IS A CHOKE POINT FOR CLOUD MIGRATION Getting data moved to the cloud can be tricky. ETL processes often require lots of data movement and manual IT work, slowing cloud migration and data availability for BI and analytics. In a recent TDWI survey, 38% of respondents said data loading, movement, and integration is one of their biggest challenges in trying to augment or replace on-premises systems with new cloud-based services. How satisfied is your organization with your current cloud-based services for self-service data loading and movement? Q3 2020 TDWI Best Practices Report: Evolving from Traditional Business Intelligence to Modern Business Analytics, figure 6, page 18 CLOUD MIGRATION CAN EXPAND SELF-SERVICE DATA TRANSFORMATION AND PREPARATION Cloud-based solutions today allow data pipelines that load, transform, and prepare data to be reusable so that "citizen" data professionals who are not in IT can access data sooner. TDWI research finds room for growth; nearly one-third would like to perform self-service data prep, transformation, and pipeline development in the near future. How many enterprises are performing data preparation, transformation, and pipeline development in self-service mode (that is, with little or no IT involvement)? Source: Page 16 (including footnote) USE ELT WITHIN A DATA LAKEHOUSE TO ACCELERATE INNOVATION AND IMPROVE GOVERNANCE Organizations can innovate faster by more tightly integrating their data warehouse and data lake. With this cloud "data lakehouse" combination, organizations can use ELT for both traditional BI data warehouse workloads and AI/ML workloads that need a broader selection of data from the warehouse and lake. Organizations can also govern data and track data lineage more effectively. This is critical; in a recent survey, 43% of respondents say a lack of visibility into data-related activities for governance is one of their biggest challenges in trying to augment or replace existing on-premises systems with cloud-based services— something that the holistic view provided by an integrated cloud data lakehouse would help solve. In your organization, which of the following BI, analytics, and data management systems are currently running on cloud-based (e.g., PaaS) or SaaS platforms? Source: Figure 1, page 6 22% say their enterprise needs a major upgrade 47% are somewhat satisfied but say they need improvement 22% are very satisfied with their current environment 9% don't know or say it's not an important issue Lower up-front investment for expanding data use Need to expand self-service BI and analytics to new types of users and projects Shorter data preparation Better alignment of data storage and management with users' data access patterns 17% are currently doing so 41% Cloud data warehouse 31% would like to in the near future 39% Cloud data lake 47% have no near- term plans to do so 25% Cloud data pipelines 22% Cloud data quality, profiling, and validation processes SWITCH FROM TRADITIONAL ETL TO ELT With ELT, different types of users can interact with the landed data to apply transformation and other preparation routines for their workloads. ELT can use cloud processing platforms to improve speed and efficiency, especially for advanced analytics and machine learning. MODERNIZE DATA TRANSFORMATION FOR FASTER PREPARATION Cloud-based data transformation services help users answer questions with less data prep by IT, provide smarter, more automated monitoring of data pipeline workloads, and highlight anomalies and data quality issues. EMPOWER CITIZEN DATA PROFESSIONALS WITH MODERN AUTOMATION AND USER INTERFACES These can reduce the complexity and inconsistency that often make data transformation and pipeline development tough for citizen data professionals. Leading solutions offer prebuilt connectors that reduce the workload for those who traditionally hand-code connectivity to extract data. RECOMMENDATIONS

Articles in this issue

Links on this page

view archives of Reports - TDWI Pulse Report Infographic: Beyond Lift and Shift