“Data is the central component of every application, process, and business decision,” stated Swami Sivasubramanian, VP of Database, Analytics, and Machine Learning at AWS. This sentiment resonates with me completely. Currently, many customers utilize data pipelines to transfer data from Amazon Aurora to Amazon Redshift, enabling them to gain valuable insights that drive sales growth, cost reduction, and business optimization. To alleviate the burden of data preparation for analysis and allow you to focus on extracting value from your data, we introduced Amazon Aurora zero-ETL integration with Amazon Redshift at AWS re:Invent 2022. This integration has been in public preview for Amazon Aurora MySQL-Compatible Edition since June 2023 and is now generally available.
Today, we are excited to announce the general availability of Amazon Aurora MySQL zero-ETL integration with Amazon Redshift. With this fully managed solution, you no longer need to build and maintain complex data pipelines to obtain time-sensitive insights from your transactional data. This integration enables you to make informed business decisions without delay. By connecting Amazon Aurora and Amazon Redshift, you can run near real-time analytics and machine learning (ML) on vast amounts of transactional data stored in Amazon Redshift. When data is written into Aurora, it becomes available in Amazon Redshift within seconds. Additionally, this integration allows you to perform consolidated analytics across multiple Aurora MySQL database clusters in Amazon Redshift, providing comprehensive insights across various applications or partitions.
The Amazon Aurora MySQL zero-ETL integration with Amazon Redshift is capable of processing over 1 million transactions per minute, which is equivalent to 17.5 million insert/update/delete row operations per minute. It ensures that these transactions are replicated and available in Amazon Redshift in less than 15 seconds (p50 latency lag). Moreover, you can leverage the analytics and built-in ML capabilities of Amazon Redshift, such as materialized views, cross-Region data sharing, and federated access to multiple data stores and data lakes.
To get started with this integration, follow these steps using your existing Amazon Aurora MySQL serverless database and Amazon Redshift data warehouse. First, navigate to Amazon RDS and select “Create zero-ETL integration” on the Zero-ETL integrations page. On the configuration page, provide an identifier for your integration and proceed to the next step. Select your source database from the available options, followed by choosing the target Amazon Redshift data warehouse. If necessary, update the resource policy and authorize the integration source in the Amazon Redshift data warehouse. Review the configurations, including tags and encryption settings, and create the zero-ETL integration.
After a few minutes, your zero-ETL integration will be successfully created. Switch to Amazon Redshift, where you can view your newly created integration on the Zero-ETL integrations page. If a target database is not present, create one. The integration configuration is now complete, and you can see that it is active with one replicated table. To test the integration, create a table in your Amazon Aurora database and insert a record. Then, access the Redshift query editor v2 within Amazon Redshift and connect to the integrated database. By running a simple query, you can verify that your data is readily available in Amazon Redshift.
I found this zero-ETL integration incredibly convenient for two reasons. Firstly, it allows me to consolidate data from multiple database clusters and perform aggregate analysis. Secondly, the seamless integration ensures that transactional data written into Amazon Aurora MySQL is immediately accessible in Amazon Redshift. Here are a few important details to note: Amazon Aurora zero-ETL integration with Amazon Redshift is available in various regions, including US East, Asia Pacific, and Europe. Currently, it supports MySQL-compatible editions of Amazon Aurora, with support for Amazon Aurora PostgreSQL-Compatible Edition in progress. This integration is provided at no additional cost, and you will only be charged for the resources used in Amazon Aurora and Amazon Redshift.
We are committed to helping you focus on extracting value from your data rather than spending time on data preparation. To learn more about getting started with Amazon Aurora MySQL zero-ETL integration with Amazon Redshift, please visit the dedicated page. Happy integrating!
– Donnie
Source link
