Extract, Transform and Load (ETL) is the traditional approach for moving and transforming data for analysis, especially in large organizations working in regulated industries.
But modern, digital enterprise companies want their data right away. And they see no reason to invest time and money in ETL when a modern, cloud-native integration platform delivers the data they need far faster and more efficiently.
In this post, I’ll walk through the benefits and drawbacks of ETL and how an iPaaS (integration platform as a service) can easily eliminate the challenges of ETL and turbo-charge this integration method.
ETL is a way of extracting data from a source system, manipulating and transforming it as it’s being moved between systems, and then loading it into a data warehouse or some other kind of data repository.
This is a tried and proven model of moving data, particularly for projects that require a data warehouse for storing data and making it available for analysis or enriching data sets in other applications.
ETL has been around for 30-plus years, and it’s something that enterprise architects all learn when we begin our careers in data integration and management.
"In many industries, we’re seeing young, cloud-centric companies skip ETL altogether and go straight to iPaaS. These companies are used to moving quickly, and iPaaS suits their needs for moving data and getting the answers they need on time."
ETL is a fine approach for certain types of data integration and analysis projects. When I think of a typical ETL project, I imagine a financial services organization using an ETL tool to extract data from various core systems, transform it, and load it into a data warehouse for analysis and reporting.
There are a few key characteristics to note:
In this kind of scenario with a financial institution, ETL processes do fine. At Slalom, we are seeing large organizations continue to use ETL for data projects like this for the foreseeable future.
Want to find out more about how Boomi helps customers around the globe transform their organizations? Join Boomi customers, partners, executives and prospects Oct. 1-3 in Washington D.C. for Boomi World 2019.
Now let’s contrast that example with a different kind of use case from another industry, such as consumer packaged goods (CPG) or retail. Compared to financial services, these markets move much more quickly.
For example, in the fashion market, manufacturers can design, create and launch new clothing lines in a matter of weeks.
Retailers in fashion and other fast-moving markets have to be nimble: rapidly analyzing trends, placing orders, designing and running sales campaigns, measuring results, and starting and stopping activities within weeks or even days.
In markets like this, slow, steady integration projects simply won't do. They will fail to deliver the insights that today's digital enterprises need, when they need them.
Developers must be able to integrate new sources quickly, load data into platforms for business intelligence and trend analysis, and get results back to business decision-makers as quickly as possible.
In these markets:
In situations like this, iPaaS shines. Using the Boomi unified platform for integration, data governance, API management and workflow automation, organizations can rapidly tie together their applications and data.
A low-code interface and ready-to-use connectors bring unprecedented speed to unite and transform your data.
At Slalom, we’re hyper-focused on our customers, and we’re dedicated to working in partnership to help them address their key business challenges and achieve their strategic goals.
We see other companies across industries — from education to telecommunications to retail — also increasing their attention on customers. That increased attention requires raw data and data analytics. How are campaigns working? Do customer like new features? How are they using our products? In which regions and in which demographics?
To answer questions like these, businesses need to be agile, curious and open-minded — you might not know exactly what you’re looking for at first. You might want to examine a data set six different ways. You might need to connect to Salesforce, a retail store real-time analytics application, a mobile service, and survey data. And you might need to run different types of analytic algorithms to find the answers you need, and then to follow up with more questions and find even more answers.
To do that, you can’t rely solely on ETL. It’s too cumbersome and inflexible. Instead, you need a unified platform like Boomi that lets you build integrations in days or hours. With Boomi, you get the business insights you need faster than the competition.
In many industries, we’re seeing young, cloud-centric companies skip ETL altogether and go straight to iPaaS. These companies are used to moving quickly, and iPaaS suits their needs for moving data and getting the answers they need on time.
ETL remains an important technology for large companies processing transaction data and other types of structured data as part of repeatable processes.
iPaaS is the right technology for many other types of data operations, where moving and analyzing data requires speed, flexibility and agility. Especially for cloud-centric businesses, we recommend iPaaS as an essential part of an IT architecture.
To learn more about all the ways the Boomi unified platform can help your organization do more, contact our partner, Slalom, or speak with a Boomi integration specialist today.