An illustration of a serverless ETL migration with AWS Glue.

Serverless ETL Migration with AWS Glue

Discover, prepare, and combine data for analytics, machine learning, and application development with ease.

An illustration of a serverless ETL migration with AWS Glue.
Overview

ETL on AWS Glue

Organizations’ data is becoming increasingly complex: it may be diverse, from multiple novel sources, growing rapidly, and/or used by many applications and people with varied skill sets. On-premises ETL solutions are simply not suited to handle the intricacy of workloads today.

AWS Glue is a fully managed data integration service that makes it simple and cost-effective to categorize, clean and enrich your organization’s data, and move it reliably between data stores.

Hot data lake storage with AWS data lake implementaiton.
Success Story

Hot Data Storage through AWS Data Lake Implementation

Our client needed a single data platform solution to integrate customer data so it could be accessed immediately when needed for use cases or analysis.

Adastra implemented a data lake in Amazon S3 and created an enterprise data model on top of it, running on Redshift. We created data marts, using stored procedures and AWS Glue jobs, allowing us to overlay the enterprise data model on top of the data lake, mimicking the client’s on-prem solution, in the cloud.

100+

jobs handled in real time

6x

faster reporting

Less

IT infrastructure costs

Common AWS Glue Use Cases

Building data lakes and lake houses for scalable data analysis
Processing petabytes of data both in batch and real-time using Apache Spark
Migrating from expensive traditional ETL solutions to gain flexibility and reduce costs
Preparing data for machine learning
Cataloging data assets to make them available to AWS analytics services

Adastra measures its success by the success of the organizations they provide support to and that is proved by their actions and work ethics. I was really happy with Adastra’s capabilities of not only delivering the right solution to our needs but also helping us with planning how the Cloud Analytics Platform can evolve in the near future in order to better support our growth and success.

Chris Smith

VP Digital Health, Skylight Health Group

Featured Resources

Move Your ETL to Serverless with AWS Glue

We will review your current ETL jobs and plan a migration to AWS Glue that lowers maintenance and scales with your data.

AWS Glue FAQ

AWS Glue is a fully managed, serverless data integration service from AWS. It lets you discover, categorize, clean and enrich data and move it reliably between data stores. Glue connects to more than 30 data sources, processes petabytes of data in batch and event-driven modes, and prepares data for analytics, machine learning and applications.

On-premises ETL tools struggle with today’s diverse, fast-growing data and the many people who use it. AWS Glue removes infrastructure setup and maintenance: you allocate compute power and run your jobs. Jobs scale with demand, run on a schedule or an event, and are written in open source SparkSQL, PySpark or Scala, so there is no technology lock-in.

You pay only for the resources your jobs consume while they are running. The all-in-one pricing model includes infrastructure and is 55% cheaper than other cloud data integration options. Because there are no servers to maintain, moving away from expensive traditional ETL solutions also reduces running costs. Adastra estimates the expected cost for your workloads before migration.

Yes. AWS Glue offers development environments for different skill sets: visual ETL development for data engineers, notebook-style development for data scientists and no-code development for data analysts. Everyone works on the same managed service and data catalog, so each team builds pipelines in the way it prefers while Adastra sets common standards for quality and deployment.

Re-platforming data pipelines by hand is slow and error-prone. Adastra uses ADELE, our own ETL migration tool, to migrate data pipelines from legacy platforms to any target platform with a single click, while you keep full control over the process. Automation lowers migration costs and removes the coding errors that come with rewriting pipelines manually.

ADELE makes data pipeline re-platforming and cloud migration simple, quick and accurate. It provides data lineage and metadata reporting, so you get more insight into your pipelines, and it prepares technology-agnostic SQL prototypes for data integration processes on any cloud or on-premises platform. You become platform agnostic and can migrate legacy solutions regardless of technology.

For one client, Adastra implemented a data lake in Amazon S3 with an enterprise data model on Redshift, using stored procedures and AWS Glue jobs to build data marts that mimic the client’s on-premises solution in the cloud. The platform handles 100+ jobs in real time, delivers 6x faster reporting and lowers IT infrastructure costs.