What Is ETL?
ETL stands for Extract, Transform and Load. It refers to a database usage process that pulls data from one or more data sources; transforms the data, or changes it to a format or structure that is accessible for queries and suitable for analysis; and loads it into the final target database, data warehouse or application. Typically, since data extraction can take some time – particularly when it’s consolidating large amounts of information from different sources – the transformation and loading steps may begin before extraction is complete. This means that while data is being pulled, a transformation process can execute, preparing the data for loading; data loading may then begin without waiting for the completion of the other phases.
In this post, we look at the role ETL plays in customer experience and customer communications management.
Advantages, Disadvantages of ETL
ETL tools can be an appropriate choice for organizations that have many source data systems they need to integrate (e.g., CRM, commerce system, line-of-business applications, etc.), when source systems are in different formats, and when a process needs to be run frequently and repeatedly. A limitation of many ETL tools is that they are often single-stream, meaning one source, one transformation, and one destination. Some offer parallel processing, and others are moving toward virtual ETL, both of which are helping to address enterprises’ need to connect with more – and more varied – data sources and more complex transformations. Unlike data virtualization, though, which can perform those processes all in memory, most ETL software writes transformed data as a file that another tool has to pick up and then do something with.
As Gary Angel stated in an article for CustomerThink earlier this year, “ETL has always been an important and largely unsung part of the broader data warehousing world. But there are several reasons why its role is larger and more challenging these days. The centerpiece…is typically a data lake – an Hadoop-based repository for traditional enterprise data, machine generated sources like digital data, and truly unstructured data from social media, call-center notes, user-feedback and other text-oriented systems. When you drop all that data into one place, you create an unprecedented set of ETL challenges.” That “one place” might be your digital experience platform or customer communications management software.
The sheer volume and velocity of data coming in – plus the proliferation of destination points – has created the need for ever-more-complex data pipelines and transformations, slowing systems and creating bottlenecks as software tools compete for query bandwidth.
Despite growing challenges in the ETL landscape, though, many enterprises rely on ETL systems for moving data. Some companies plan to continue to use ETL while others have moved or are in the process of moving to data virtualization. Still others use a hybrid data integration system made up of complementary ETL and data virtualization tools.
ETL in CX and CCM
In customer communications management (CCM) and customer experience (CX) management, we need customer data from a variety of sources to be able to create personalized customer communications that are relevant and accurate for each and every customer. Sometimes the data reside in completely separate, sometimes physically separated data centers, and often it is stored and formatted in very different ways.
We’re used to that, so we at Topdown have built in ETL capability to help our clients join their various data sources for better customer communications management. As we’ve emphasized before, data integration capability is critically important in a CCM solution, and, according for Forrester, not many CCM vendors are working to solve the issue for their clients. We continue to work to streamline data integration capabilities with our tools. We build in robust data handling capabilities, and we work with our clients to help them integrate the data they need for personalized, automated large-batch communications. Our CLIENT LETTER® CCM solution can push or pull data using our data virtualization capability, which is something that sets our software and customer service apart from other CCM solutions currently available.
ETL and other data management and integration systems are just as important for customer experience management as it is for CCM. For more insight into data and CX, download our complimentary e-book covering that topic and others.
