Revolutionizing Retail Data: 60% Faster Data Processing Through Teradata to BigQuery Migration
Giant Tiger, a retail giant, faced challenges with their existing Teradata data warehouse and sought a more flexible and scalable cloud environment. Their data ecosystem was intricate, involving several databases, ETL processes, and downstream systems. Adastra helped Giant Tiger migrate from Teradata to Google BigQuery, thus overcoming several existing limitations.
more efficient resource utilization
faster data processing
lower storage costs
reduction in data latency
cost savings in data management
The Challenge
Giant Tiger is a leading retailer in North America, distinguished by its scale and variety of products.
Giant Tiger’s existing data warehouse infrastructure, built on Teradata, had numerous challenges such as elongated ETL processes, performance bottlenecks, and lack of governance. Their Business Intelligence (BI) team and other downstream systems were considerably hampered, making real-time analytics a distant dream.
Giant Tiger needed to move their large-scale Teradata warehouse environment, which included integration with a variety of 3rd party business applications, and sought an experienced partner to help perform the complex migration.
This migration was a cornerstone in their digital transformation journey, aimed at improving data management and real-time analytics.
Solution StoryÂ
Upon assessing the existing on-premises architecture and operational nuances, our team proposed a solution to migrate to Google Cloud Platform’s BigQuery. We kicked off the project with a well-defined roadmap that included setting up the landing zone and Google Cloud Storage, BigQuery setup, leveraging Informatica’s Secure Agent for data ingestion, and using MicroStrategy Platform as a Service (PaaS).
Teradata to BigQuery MigrationÂ
The cloud migration was seamless thanks to a custom migration agent and BigQuery Data Transfer Service. Once it went live on GCP, Adastra noticed a drastic reduction in ETL batch processing times from up to 3 hours to mere minutes, significantly improving query performance and system efficiency. Furthermore, the implementation of data governance policies and user access controls streamlined data management, eliminating unnecessary data, and reducing costs.
We also provided Giant Tiger with robust logging and monitoring solutions that allow for real-time system health checks and proactive maintenance. By migrating to BigQuery, Giant Tiger has not only modernized its data stack but has also taken a giant leap towards achieving data excellence and operational efficiency.
Benefits StoryÂ
The migration from Teradata to BigQuery brought many benefits to Giant Tiger. They experienced a paradigm shift in how they approached data, transitioning from a largely operational standpoint to a more strategic, data-driven focus.
The success of the project didn’t just solve their existing issues but also set our client up with a scalable, future-proof data infrastructure, enabling them to be more agile and data-driven in their decision-making process.
Business decisions are now made twice as fast, and the 30% savings in data storage costs came as an unexpected bonus. The project had both a technological and organizational impact, bringing about a cultural shift toward data-centricity.
The new Google Cloud-based EDW offers Giant Tiger a variety of benefits:
- 3x more efficient resource utilizationÂ
- 60% faster data processingÂ
- 30% lower storage costsÂ
- 28% reduction in data latencyÂ
- 12% cost savings in data managementÂ
- Faster data access – The high-performance EDW model significantly reduces data latency
- Improved reliability – The new architecture eliminates issues in the store-to-EDW process
- Cost efficiency – Migrating to the cloud has resulted in cost savings through operational efficiency
