[Update] AWS Launches Amazon Redshift Integration with Apache Spark#reinvent

Hello Everyone!

AWS announced Amazon Redshift Integration with Apache Spark at the Keynote Session of re:Invent 2022.

Amazon Redshift Integration with Apache Spark

Amazon Redshift Integration with Apache Spark was made GA[General Available] at the Keynote session. With this engineers can run Apache Spark applications on Redshift data, i.e, Spark applications can read and write data from Amazon Redshift cluster.

 

 

Refer to the following blog for the official announcement about Amazon Redshift Integration with Apache Spark.

 

Redshift integration with Apache Spark is available for EMR [starting from EMR 6.9] - EC2, EKS, and Serverless, AWS Glue[starting from Glue 4.0] and Amazon Redshift in all regions. With this integration third-party Apache Spark connector is no longer required for using Spark applications with Amazon EMR, Amazon SageMaker and AWS Glue. And also Apache Spark runs 3x faster on AWS.

 

Refer to the following blog to know how to integrate Apache spark with Redshift.

Summary

Redshift Integration with Apache Spark makes it easier to build and run Spark applications on Redshift.

Reference :

Amazon Redshift Integration for Apache Spark

Using Amazon Redshift integration for Apache Spark with Amazon EMR