Movement of Data From DynamoDB to S3 using DataPipeline

2021.11.28

この記事は公開されてから1年以上経過しています。情報が古い可能性がありますので、ご注意ください。

DataPipeline

It is a service that helps in sort, reformating, analyzing, filtering, reporting data, and deriving an outcome from it. It is a web service that helps you reliably process and move data between different AWS compute and storage services, as well as on-premises data sources, at specified intervals.

S3

Simple and popular AWS Service for storage. Replicates data by default across multiple facilities. It charges per usage. It is deeply integrated with AWS Services. Buckets are logical storage units. Objects are data added in the bucket. S3 has a storage class on object level which can save money by moving less frequently accessed objects to colder storage class.

DynamoDB

An Amazon AWS Web Services database which is a fully managed, serverless, key-value NoSQL database, has high-performance applications at any scale. It has the following benefits such as built-in security, continuous backups, automated multi-region replication, in-memory caching, and data export tools.

Before starting the below demo please do create the following roles by following the below blog

Demo

Creating DynamoDB table Clicking on create table Click on create item Adding data into the table

Now creating an S3 bucket Click on create a bucket

Now creating the DataPipeline Click on activate Now wait for the wait on dependences to change to Running state Checking S3 Bucket and data has transferred successfully