ComAp: Migrating 1 trillion telemetry records to the cloud as the foundation for modern reporting
Adoki Migrated Up to 15 Years of Telemetry Data from 60,000 Customers to Azure Data Explorer. The Automated Migration Preserved Business Logic and Ensured Scalability.
telemetry records
legacy database instances
Challenge
Legacy Data as a Barrier to Modernization
To modernize its energy equipment monitoring and reporting platform, ComAp needed to securely migrate more than 50 TB of historical telemetry data from its legacy MervisDB system to a new cloud architecture based on Azure Data Explorer. The data spanned 60,000 customers, with up to 15 years of history in some cases, and was spread across more than 50 database instances in Europe and Australia.
ComAp turned to Adastra to design a migration strategy that would minimize data migration risks, maintain data consistency, and ensure a transparent, controlled migration process. Using our proprietary metadata-driven ETL platform, Adoki, we eliminated the need for ComAp to spend time and resources developing a one-off migration tool.
Solution
From Distributed Data to Automated, Metadata-Driven Migration
We designed the migration strategy and end-to-end architecture to maintain full control over the migration process.
- Adoki orchestrated each stage of the migration.
- PostgreSQL tracked migration metadata and progress.
During the first three months, we mapped customer data histories across all source databases, extracted most of the data, and established the correct scope of the migration. We then focused on three key areas:
Historical data processing
We cleaned, transformed, and aggregated the time-series data, ensuring structural and logical consistency with new data.
Transfer to the target data model
We transformed the data to fit the new Azure Data Explorer data model, following established business rules.
Scaling the migration
The actual data volume turned out to be significantly higher than initially estimated, so we gradually increased the solution’s capacity to handle it.
The customer’s team was able to focus entirely on transformation logic, data quality, and validating the results.
Impact
A Data Foundation for Advanced Analytics and Continued Modernization
We established an accessible cloud-based data layer to support the modernization of reporting, monitoring, and customer dashboards, as well as new analytics use cases. This was an essential step before ComAp could move forward with its broader data and AI initiatives.
ComAp particularly valued three outcomes:
Full control over the migration
Adoki gave ComAp real-time visibility into migration progress and results for each customer, ensuring full control throughout the process.
Faster migration with minimal resource requirements
Adastra’s metadata-driven ETL platform eliminated the need to develop a separate migration tool, saving time and resources.
Data ready for broader use
Historical telemetry data is no longer locked in a legacy system and can now support new analytics, data, and AI use cases.
The telemetry data migration established the technical and data foundation for ComAp’s next phase of modernization. Projects like this lay the technical foundations for new analytics and AI initiatives, enabling businesses to move forward.
