Scheduling Lambda with AWS EventBridge

System Administrators,very often use cron daemon for automating and scheduling task like running backups, patching and upgrading, or running system maintenance tasks. We can make use of the same idea of "scheduling" for AWS Lambda functions to perform a repetitive task using AWS EventBridge.

Glossary

Event

In AWS, An event indicates a change in your AWS environment. AWS resources can generate events when their state changes. For example, Amazon EC2 generates an event when the state of an EC2 instance changes from pending to running, CloudTrail service generates an event when we make API calls [List of services that generates events].

Rules

A rule is needed in order to match a particular event and send them to the target for processing. A single rule can route to multiple targets, all of which are processed in parallel.

Targets

Events are processed by Targets. Targets can include Amazon EC2 instances, AWS Lambda functions, Kinesis streams, Amazon ECS tasks, Step Functions state machines, Amazon SNS topics, Amazon SQS queues, and built-in targets. Events are in JSON format.

Event Bus

An event bus is a channel that allows different components (applications, integrated Software-as-a-Service (SaaS) applications, and AWS services) to talk with each other without knowing about each other. A component can send an event to the event bus without knowing who will pick it up or how many others will pick it up. Components can also listen to events on an event bus, without knowing who sent the Events.  Components can communicate without depending on each other. Also, adding or removing components becomes easy as long as the new component understands the Events that are being sent and received, the other components will never know.

This will make development and splitting up an application into independent parts a lot easier.

Note

The rule and corresponding target should be in the same region.

What is AWS EventBridge?

Amazon EventBridge is a serverless event bus that is used for building event-driven applications. It uses events from sources like your applications, integrated Software-as-a-Service (SaaS) applications (3rd party), and AWS services. EventBridge delivers real-time data from event sources which could be third-party SaaS services or events from the changes in your AWS environment and services.

Eventbridge can be described as an enhanced version of AWS CloudWatch Events, however, EventBridge has been introduced to address the problems of SaaS platform integration with AWS services. 

Issues with distributed architecture (microservices)

We all know the benefits of microservices (loosely coupled) and distributed architecture, however, as a system involves synchronous APIs when services and complexity increases; even microservices tend to become tightly coupled and fairly monolithic. There can be many problems depending on the use cases but I will focus mainly on being tightly coupled, designing an API, and availability of such architecture.

To understand various problems we shall consider the famous example of e-commerce website architecture.

Problem: Tightly Coupled

In synchronous APIs but when services increase the producer has to understand the definition of each consuming API, if there is an update to a consuming API or some incompatibility, updates have to be done on the producer.

When the system gets complex with more microservices, all the consumers are dependent on the producer, hence they are blocked if the producer doesn't send data. The consumers which were developed to operate independently now becomes completely dependent on the producer.

"Decoupling  of services is replaced by tightly coupled."

Problem: Designing an API gets tough

Adding a new service to an already existing complex system needs to refactor the existing components ( producer and consumers) to accommodate the new service. Also, there are situations where a new service is connected to the producer but lacks proper integration with other consumers, in such a case when an error or rollback of an event initiated by another consumer; the new service is not able to handle such an exception.

Microservices interdependence leads to another issue when a new service incorporates a new API or features and other depending services are unaware of it.

Problem: Availability with synchronous API

In a system, all the services don't perform their tasks at the same speed, in the case of synchronous API performance inconsistency and delays are a common scenario when fast service is placed in front of a slower service.

Availability of the entire system is a major risk, the services (producer) interacting with customers cannot perform their task if the underlying backend service is down. In other words, if downstream goes down, it appears the entire system is down because of its interdependence nature.

Why do we use EventBridge?

In a tightly coupled API and previous example, events from producers were hard-coded to consumers it means that each event is issued to specific recipients or consumers.

When we use AWS EventBridge, event Producers do not need to be aware of who is listening to events or other services to consume events without needing to know upstream.

EventBridge has 2 components :-

• event bus -> gives endpoint where producer sends events.
• router and rules ->  filters and direct event to various targets without calling them directly and consumers consume events which they want ( either few or all ), making producers and consumers decoupled all the time.

Even if there is an error, the service raises an error event and it is observed by other services which take actions accordingly, or if a new service is added just update rules for the new service, eliminating the need to refactor the code. 

EventBridge address all the problems we discussed above whether ensuring the system remains decoupled even if it gets complex or new services are being added with time or fostering simplified routing of events with availability and third-party integration.

EventBridge allows the architecture to be simple, asynchronous, and make sure services remain decoupled.

Let's Schedule an AWS Lambda function using EventBridge

• We will be scheduling a lambda function that will send a water-drinking-reminder email to my email address.
• For sending emails, will be using AWS Simple Email Service (SES).
• Note:- To prevent spam, fraud and protect our reputation as a sender, all new accounts are placed in the SES sandbox. Inside the sandbox, we can use all the features of SES but can only send emails to verified emails and domains. When the account is out of the sandbox, we can send an email to any recipient, regardless of whether the recipient's address or domain is verified.
• Verify your identities (the domains or email addresses that you send email from) to confirm that you own them, and to prevent unauthorized use, and for receiving emails also let's verify another email address.

• Create a lambda function in the console, code for lambda resides in this repository.
• Update the lambda role under IAM -> roles ( same name as your lambda function) and add the following permission.

{
    "Effect":"Allow",
    "Action":[
         "ses:SendEmail",
         "ses:SendRawEmail"
    ],
    "Resource":"*"
}

• To schedule, Lambda, move to EventBridge service and create a rule.
• Input a name and description for the rule.

• Under Define pattern, we have the option to either configure the rule for event pattern either from AWS services, service partners, or can schedule.
• The schedule can be set using fixed-rate or cron expression. Here we are using cron expression 30 00-16 * * ? * which invokeslambda at minute 30 past every hour from midnight through 16 hours everyday in GMTbut in IST it will be from 6 AM to 10 PM, crontab guru is a great website for verifying your cron expressions.

• Under Select target choose lambda function as target category and function name of our lambda.

• We can add multiple targets for the same event also by using add target.
• Note:- When the Schedule is selected custom or partner event bus cannot be used.
•  Events should be allowed to trigger (invoke) our Lambda function. All events that will invoke a Lambda function need to have explicit permission to do that.
• Under our lambda function -> configuration -> triggers we can add triggers, but in our case EventBridge automatically does that for us.

• Every hour we can see our reminder in our mailbox.

Conclusion

AWS EventBridge can help us to achieve decoupling, third-party integration, efficient event routing, resilient architecture. Consumers have to listen to events which they care about and producers are only responsible for just creating events. EventBridge has far more advantages than AWS Kinesis, Cloudwatch Events, and SNS with respect to event routing.

Happy learning :)