Hello Everyone!

AWS announced AWS Step Functions Distributed Map at the Keynote Session of re:Invent 2022.

Overview

AWS Step Functions Distributed Map was announced and it is Generally Available now. AWS Step Functions Distributed Map supports the orchestration of large-scale parallel workloads in serverless applications. It can iterate over millions of objects like images, logs or .csv files stored in an S3 bucket.

 

 

The processing steps are the same as Step Function's map state but the map state is limited to 40 parallel iterations at a time. With AWS Step Functions Distributed Map you can launch up to ten thousand parallel workflows. It supports up to 10,000 executions in parallel. The data can be processed by any service API supported by Step Functions.

 

 

Refer to the following blog for the know more about AWS Step Functions Distributed Map.

Available Regions

AWS Step Functions Distributed Map is available in the following regions:

• US East (Ohio, N. Virginia)
• US West (Oregon)
• Asia Pacific (Singapore, Sydney, Tokyo)
• Canada (Central)
• Europe (Frankfurt, Ireland, Stockholm)

 

Summary

With this feature of AWS Step Functions, large-scale data can be processed in parallel.

Reference :

Orchestrating large-scale parallel workloads in your state machines