I Tried Building a FastQC Execution Environment Using AWS Batch with Only the AWS Management Console

Introduction

Hello, this is Horiguchi.

What kind of environment do you usually use for RNA-seq analysis?
I imagine many of you run it on a lab server or your personal PC.

In this article, I'll walk you step by step through how to build an environment for running FastQC on AWS using the AWS Management Console. As a minimal setup, we'll create an S3 bucket to store data and run just one FastQC job with AWS Batch.

I hope this serves as a reference for what the architecture looks like when you run analyses that you would normally run on a lab server or personal PC in the cloud.

Notes

• This article uses public data (GATK Test Data)
• This article does not provide any scientific interpretation of the analysis results

Glossary

RNA-seq
A method for investigating which genes are being used, and how much, inside cells. When a gene is used, a molecule called RNA, which is a copy of its information, is produced. RNA-seq reads the sequences of RNA and measures their amounts to estimate how active genes are.

FASTQ
Sequences read by a device called a sequencer are generally stored in a file format called FASTQ. A FASTQ file records the sequences that were read and the quality of each read.

FastQC
Software for checking, through graphs and tables, whether there are any quality problems in the sequence data stored in FASTQ files.

In short, the goal of this article can be summed up in one sentence:
"Let's check the quality of sequence data read by RNA-seq on AWS."

Thank you for reading.

Architecture We'll Build

Overall architecture built in this article

Since the goal is to complete everything in the Management Console, we won't build the FastQC container image on a local PC. Instead, we'll create it with CodeBuild and store it in ECR.

The main AWS services used this time are as follows.

We use Fargate for the AWS Batch Compute environment.
You don't need to create or manage EC2 instances yourself.

1. Upload the Input FASTQ to S3

Now, let's get started.

First, create an S3 bucket to store the input data and the FastQC output.
Open the Amazon S3 console and choose "Create bucket".

• Traditionally, bucket names had to be unique across all AWS accounts, but by using account regional namespaces, the same base name can be used in different AWS accounts and Regions. However, it must still be unique within the same AWS account and the same Region.

Keep the settings at their defaults and leave "Block all public access" enabled.

Creating the S3 bucket

Next, create the following two prefixes (folders in the console) in the bucket.

• input_data/: stores the FASTQ files to analyze
• output_data/: stores the FastQC results

Creating the input_data and output_data folders in the S3 bucket

For paired-end data, place the input data in a structure like the following, for example.

input_data/
├── sample1_R1.fastq.gz
└── sample1_R2.fastq.gz

For single-end data, on the other hand, there is only one FASTQ file.

Note down the S3 URIs of the uploaded objects, as you'll use them later when creating the AWS Batch job definition.

2. Create an ECR Repository for FastQC

2-1. Background

In AWS Batch, instead of installing analysis software directly on the host environment, jobs run using a container image that bundles the required software and execution steps. Therefore, we first need to create a container image that includes FastQC, Java, the AWS CLI, and so on.

A container image is an execution environment that packages the software and settings required for analysis. By fixing the FastQC version and required libraries inside the image, you don't need to set up the environment every time you run it, and you can run the analysis under the same conditions.

Since the goal this time is to complete everything in the AWS Management Console, we won't operate Docker on a local PC. Instead, we'll use AWS CodeBuild to create the FastQC container image. The created image is stored in Amazon ECR, and Fargate pulls it from ECR when running jobs with AWS Batch.

2-2. Create the ECR Repository

Now, let's create the ECR repository that will store the FastQC container image.

I created a private repository from "Create repository" in Amazon ECR.
The repository name this time is as follows.

rna-seq/fastq-test

Because a repository name can contain /, you can express the project name and purpose hierarchically.

ECR repository creation screen

After creating it, note down the ECR repository URI.

AWSアカウントID.dkr.ecr.us-east-1.amazonaws.com/rna-seq/fastq-test

You'll use this URI later in the AWS Batch job definition.

3. Build the FastQC Container with CodeBuild

Create the CodeBuild project with the following settings.

Basic settings of the CodeBuild project

Environment settings of the CodeBuild project

Enable Privileged Mode

In the CodeBuild environment settings, enable privileged mode.
Expand "Additional configuration" at the bottom of the "Environment" section and check the privileged box.

Privileged setting in CodeBuild

What Is Privileged Mode?

The CodeBuild build environment itself also runs in a container. This time, we run the following commands inside it.

docker build
docker push

Privileged mode must be enabled in order to run commands that operate on Docker images, such as docker build, inside CodeBuild.

Note that privileged mode and IAM permissions are separate things.

Docker build などを実行するために必要
 → 特権モード
ECR に push するために必要
 → CodeBuild サービスロールの IAM ポリシー

Only when both privileged mode and the appropriate IAM policy are in place can the created Docker image be pushed to ECR.

Set Environment Variables

I set the following environment variables in the CodeBuild project.

CodeBuild environment variables

For ECR_REPOSITORY, set only the repository name, not the full ECR URI.
Since we use FastQC 0.12.1 this time, the image tag is set to 0.12.1.

Processing Run When the Container Starts

When started, the container we create this time does the following.

• Downloads the FASTQ from S3
• Runs FastQC
• Uploads the HTML and ZIP to S3

This time, because we enter the contents of the Dockerfile and the shell script on a single line in the Build commands field, we encoded each of them in Base64. At build time, they are decoded with base64 -d and written out as files.

The command set in the Build commands field is as follows.

echo RlJPTSBwdWJsaWMuZWNyLmF3cy9hbWF6b25saW51eC9hbWF6b25saW51eDoyMDIzCkFSRyBGQVNUUUNfVkVSU0lPTj0wLjEyLjEKUlVOIGRuZiBpbnN0YWxsIC15IGphdmEtMTctYW1hem9uLWNvcnJldHRvLWhlYWRsZXNzIHBlcmwgdW56aXAgd2dldCBhd3NjbGkgZ3ppcCB0YXIgJiYgZG5mIGNsZWFuIGFsbApSVU4gd2dldCAtcSAiaHR0cHM6Ly93d3cuYmlvaW5mb3JtYXRpY3MuYmFicmFoYW0uYWMudWsvcHJvamVjdHMvZmFzdHFjL2Zhc3RxY192JHtGQVNUUUNfVkVSU0lPTn0uemlwIiAtTyAvdG1wL2Zhc3RxYy56aXAgJiYgdW56aXAgL3RtcC9mYXN0cWMuemlwIC1kIC9vcHQgJiYgY2htb2QgK3ggL29wdC9GYXN0UUMvZmFzdHFjICYmIGxuIC1zIC9vcHQvRmFzdFFDL2Zhc3RxYyAvdXNyL2xvY2FsL2Jpbi9mYXN0cWMgJiYgcm0gL3RtcC9mYXN0cWMuemlwCkNPUFkgcnVuX2Zhc3RxYy5zaCAvdXNyL2xvY2FsL2Jpbi9ydW5fZmFzdHFjLnNoClJVTiBjaG1vZCAreCAvdXNyL2xvY2FsL2Jpbi9ydW5fZmFzdHFjLnNoCkVOVFJZUE9JTlQgWyIvdXNyL2xvY2FsL2Jpbi9ydW5fZmFzdHFjLnNoIl0K | base64 -d > Dockerfile && echo IyEvdXNyL2Jpbi9lbnYgYmFzaApzZXQgLWV1byBwaXBlZmFpbAoKOiAiJHtJTlBVVF9SMTo/SU5QVVRfUjEgaXMgcmVxdWlyZWR9Igo6ICIke09VVFBVVF9TMzo/T1VUUFVUX1MzIGlzIHJlcXVpcmVkfSIKSU5QVVRfUjI9IiR7SU5QVVRfUjI6LU5PTkV9IgpUSFJFQURTPSIke1RIUkVBRFM6LTJ9IgoKbWtkaXIgLXAgL3dvcmsvaW5wdXQgL3dvcmsvb3V0cHV0CgpSMV9MT0NBTD0iL3dvcmsvaW5wdXQvJChiYXNlbmFtZSAiJElOUFVUX1IxIikiCmF3cyBzMyBjcCAiJElOUFVUX1IxIiAiJFIxX0xPQ0FMIgpJTlBVVFM9KCIkUjFfTE9DQUwiKQoKaWYgW1sgIiRJTlBVVF9SMiIgIT0gIk5PTkUiICYmIC1uICIkSU5QVVRfUjIiIF1dOyB0aGVuCiAgUjJfTE9DQUw9Ii93b3JrL2lucHV0LyQoYmFzZW5hbWUgIiRJTlBVVF9SMiIpIgogIGF3cyBzMyBjcCAiJElOUFVUX1IyIiAiJFIyX0xPQ0FMIgogIElOUFVUUys9KCIkUjJfTE9DQUwiKQpmaQoKZmFzdHFjIC0tdGhyZWFkcyAiJFRIUkVBRFMiIC0tb3V0ZGlyIC93b3JrL291dHB1dCAiJHtJTlBVVFNbQF19Igphd3MgczMgY3AgL3dvcmsvb3V0cHV0LyAiJHtPVVRQVVRfUzMlL30vIiAtLXJlY3Vyc2l2ZQo= | base64 -d > run_fastqc.sh && ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text) && REGION=$AWS_DEFAULT_REGION && ECR_URI=${ACCOUNT_ID}.dkr.ecr.${REGION}.amazonaws.com/${ECR_REPOSITORY} && aws ecr get-login-password --region ${REGION} | docker login --username AWS --password-stdin ${ACCOUNT_ID}.dkr.ecr.${REGION}.amazonaws.com && docker build -t ${ECR_URI}:${IMAGE_TAG} . && docker push ${ECR_URI}:${IMAGE_TAG}

The contents of the Dockerfile before encoding are as follows.

FROM public.ecr.aws/amazonlinux/amazonlinux:2023

ARG FASTQC_VERSION=0.12.1

RUN dnf install -y \
      java-17-amazon-corretto-headless \
      perl \
      unzip \
      wget \
      awscli \
      gzip \
      tar \
    && dnf clean all

RUN wget -q \
      "https://www.bioinformatics.babraham.ac.uk/projects/fastqc/fastqc_v${FASTQC_VERSION}.zip" \
      -O /tmp/fastqc.zip \
    && unzip /tmp/fastqc.zip -d /opt \
    && chmod +x /opt/FastQC/fastqc \
    && ln -s /opt/FastQC/fastqc /usr/local/bin/fastqc \
    && rm /tmp/fastqc.zip

COPY run_fastqc.sh /usr/local/bin/run_fastqc.sh

RUN chmod +x /usr/local/bin/run_fastqc.sh

ENTRYPOINT ["/usr/local/bin/run_fastqc.sh"]

The shell script that runs FastQC, before encoding, is as follows.

#!/usr/bin/env bash
set -euo pipefail

: "${INPUT_R1:?INPUT_R1 is required}"
: "${OUTPUT_S3:?OUTPUT_S3 is required}"

INPUT_R2="${INPUT_R2:-NONE}"
THREADS="${THREADS:-2}"

mkdir -p /work/input /work/output

R1_LOCAL="/work/input/$(basename "$INPUT_R1")"
aws s3 cp "$INPUT_R1" "$R1_LOCAL"
INPUTS=("$R1_LOCAL")

if [[ "$INPUT_R2" != "NONE" && -n "$INPUT_R2" ]]; then
  R2_LOCAL="/work/input/$(basename "$INPUT_R2")"
  aws s3 cp "$INPUT_R2" "$R2_LOCAL"
  INPUTS+=("$R2_LOCAL")
fi

fastqc \
  --threads "$THREADS" \
  --outdir /work/output \
  "${INPUTS[@]}"

aws s3 cp \
  /work/output/ \
  "${OUTPUT_S3%/}/" \
  --recursive

Since we used a CodeBuild project with "No source" this time, we set a command in the build command field that generates the Dockerfile and the shell script, then builds and pushes the Docker image.

CodeBuild build command settings

Supplement: When Building the Container Image in a Local Environment

This time, to complete the work using only the AWS Management Console, we used CodeBuild to create the container image for FastQC.

On the other hand, if you can use Docker on your own PC, you can also build the container image locally and push it directly to ECR. With this method, you don't need to create a CodeBuild project or an IAM role for CodeBuild.

Note that a container image built locally cannot be sent to AWS Batch as is. It needs to be stored in a container registry such as ECR so that AWS Batch can pull it.

What You Need in Advance
Prepare the following tools on your local PC.

• Docker
• AWS CLI
• AWS credentials for use with the AWS CLI

Place the Dockerfile and the Execution Script
Create a working directory and place the following two files in it.

fastqc-batch/
├── Dockerfile
└── run_fastqc.sh

In CodeBuild, we encoded these files in Base64 so we could enter them on a single line in the Build commands field. In a local environment you can create the files directly, so there is no need to use Base64.

In the directory where you placed the files, build the container image using Docker.

Push the Container Image to ECR
Open the target repository in the ECR console and choose "View push commands" at the upper right of the screen. You can then see the commands corresponding to your account ID, Region, and repository name.

Push commands for ECR

The displayed commands include logging in to ECR, building the container image, tagging it, and pushing it to ECR. Basically, you can complete the push by running them in the order shown.

By default, latest is used as the image tag. To match the settings in this article, change latest to 0.12.1 in both the tagging command and the push command.

When the push is complete, open the ECR repository in the AWS Management Console. If the image with the 0.12.1 tag is shown, it succeeded.

In the AWS Batch job definition, specify an image URI in the following format, just as when building with CodeBuild.

AWSアカウントID.dkr.ecr.us-east-1.amazonaws.com/rna-seq/fastq-test:0.12.1

Permissions for Pushing to ECR
The IAM user or IAM role that runs the commands needs permission to push images to ECR.

You mainly use permissions for the following actions. (Reference)

ecr:GetAuthorizationToken
ecr:BatchCheckLayerAvailability
ecr:InitiateLayerUpload
ecr:UploadLayerPart
ecr:CompleteLayerUpload
ecr:PutImage
ecr:BatchGetImage

If permissions are insufficient, an error such as AccessDenied is displayed when running docker push.

Differences from Using CodeBuild

With either method, the FastQC container image ends up stored in ECR. Therefore, the AWS Batch configuration and how to run it after the push to ECR is complete are the same.

If Docker is already installed on your local PC, building locally and pushing to ECR involves fewer steps. On the other hand, if you don't want to change your local environment or want to finish everything using only browser operations, using CodeBuild is more suitable.

4. Grant the CodeBuild Role Permission to Push to ECR

To allow CodeBuild to push images to ECR, add an IAM policy to the CodeBuild service role.
Enter "codebuild" in the search box on the IAM roles dashboard, find the role that was created when you chose "Create a new service role" earlier, and click the role name.

IAM policy settings screen

Click "Add permissions" → "Create inline policy" at the upper right of the Permissions policies section to open the policy editor.
Select the "JSON" tab at the upper right of the policy editor and write the policy contents.

This time, I set the following policy.

{
 "Version": "2012-10-17",
 "Statement": [
   {
     "Sid": "GetEcrAuthorizationToken",
     "Effect": "Allow",
     "Action": [
       "ecr:GetAuthorizationToken"
     ],
     "Resource": "*"
   },
   {
     "Sid": "PushFastqcImage",
     "Effect": "Allow",
     "Action": [
       "ecr:BatchCheckLayerAvailability",
       "ecr:GetDownloadUrlForLayer",
       "ecr:BatchGetImage",
       "ecr:InitiateLayerUpload",
       "ecr:UploadLayerPart",
       "ecr:CompleteLayerUpload",
       "ecr:PutImage"
     ],
     "Resource": "arn:aws:ecr:us-east-1:AWSアカウントID:repository/rna-seq/fastq-test"
   }
 ]
}

This IAM policy directly specifies the ARN of the ECR repository in Resource. Therefore, the Region name, AWS account ID, and repository name must match the repository we created earlier.

5. Create IAM Roles for AWS Batch

AWS Batch Fargate jobs use two kinds of IAM roles.

The names are similar, but their purposes differ.

The Execution role is used by Fargate to pull the container image from ECR and send execution logs to CloudWatch Logs. The Job role, on the other hand, is used when the running container accesses AWS services such as S3.

In this configuration, the Job role is used when the FastQC container retrieves FASTQ files from S3 and saves the analysis results to S3.

Execution role

Attach the following AWS managed policy to the Execution role.

AmazonECSTaskExecutionRolePolicy

For example, name the role as follows.

FastqcBatchExecutionRole

Now, let's actually create the role.
First, enter the role creation screen from the "Create role" button at the upper right of the IAM roles dashboard.

For the trusted entity type, select "AWS service", and for the use case, select "ECS Task".

For the permissions policy, search for and select the AWS managed policy "AmazonECSTaskExecutionRolePolicy".

Creating the Execution role

Job role

I named the Job role as follows.

FastqcBatchJobRole

For this role as well, specify "AWS service" as the trusted entity type and "ECS Task" as the use case.

Set the S3 access permissions not with an AWS managed policy but with an inline policy, as follows.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "ListBucket",
      "Effect": "Allow",
      "Action": [
        "s3:ListBucket",
        "s3:GetBucketLocation"
      ],
      "Resource": "arn:aws:s3:::バケット名-アカウントID-リージョン-an"
    },
    {
      "Sid": "ReadFastqInput",
      "Effect": "Allow",
      "Action": [
        "s3:GetObject"
      ],
      "Resource": "arn:aws:s3:::バケット名-アカウントID-リージョン-an/input_data/*"
    },
    {
      "Sid": "WriteFastqcOutput",
      "Effect": "Allow",
      "Action": [
        "s3:PutObject"
      ],
      "Resource": "arn:aws:s3:::バケット名-アカウントID-リージョン-an/output_data/*"
    }
  ]
}

Object read and write permissions are limited to the prefixes used this time. Only reading is allowed for input_data/, and only writing is allowed for output_data/.

6. Create the AWS Batch Compute Environment

Next, create the AWS Batch Compute environment.

The settings are as follows.

Compute environment creation screen

Maximum vCPUs

The "Maximum vCPUs" of a Compute environment is not the number of vCPUs allocated to a single job, but the upper limit on the number of vCPUs that can be used simultaneously across the whole environment.

The number of vCPUs used by one job, on the other hand, is specified in the Job definition. For example, in an environment with a maximum of 4 vCPUs, if each job uses 2 vCPUs, up to 2 jobs can run at the same time. The third and subsequent jobs wait until running jobs finish and vCPUs become available.

Since this is a test run with one sample, a maximum of 4 vCPUs is sufficient.

Also, even if you set the maximum vCPUs to 256, 256 vCPUs are not running all the time.
However, if you unintentionally submit a large number of jobs, it may scale up to that limit.
From a cost perspective, it is safer to keep it small during the verification stage.

After creation, confirm that the Compute environment is in the following state.

状態：ENABLED
ステータス：VALID

Details of the Compute environment after creation

Don't We Use EC2?

When thinking about running applications on AWS, the first service that comes to mind is probably EC2 (Elastic Compute Cloud). In fact, if you only want to run FastQC once, it is simpler to run it directly on a local PC or on EC2 without using AWS Batch.

The purpose of using AWS Batch in this article is not to make a one-off FastQC run more efficient. The goal is to build a foundation where, as the number of samples grows, you can submit many analysis jobs to a queue in advance and have them run automatically according to the available compute resources.

For that reason, we are building the environment on the premise of using AWS Batch from the start.

7. Create a Job Queue

Next, create a Job queue that accepts jobs.
Select "Job queues" from the left menu of AWS Batch and start creating one with the "Create" button at the upper right.

The settings are as follows.

Job queue creation screen

After creation, confirm the following state.

状態：ENABLED
ステータス：VALID

Details of the Job queue after creation

8. Create the FastQC Job Definition

In the job definition, you configure the container image to run, CPU, memory, IAM roles, environment variables, and so on.
This time, I created a job definition for Fargate.

• Leave the scheduling priority blank.

For the image, specify the image URI pushed to ECR by CodeBuild.

AWSアカウントID.dkr.ecr.us-east-1.amazonaws.com/rna-seq/fastq-test:0.12.1

Container image settings in the job definition

Making the Root File System Read-Only and Adding a Volume

In this configuration, to comply with Security Hub CSPM control ECS.5 (containers should be limited to read-only access to root filesystems), we select "Enable read-only filesystem" in the job definition to prohibit writes to the container's root file system.

However, if we leave it as is with writes prohibited, operations that involve writing, such as downloading FASTQ files and FastQC's temporary output, would fail. So we add a volume to the job definition and mount it at /work in the container. All file downloads and temporary output in the script are performed under this /work.

As a result, no writes occur at the container root, and we can run FastQC while satisfying the Security Hub CSPM control.

In addition, so that the temporary directories implicitly referenced by FastQC itself and surrounding tools also point to /work, we fix the following environment variables on the job definition side.

Environment variable settings

Enabling the read-only filesystem and adding a volume

Environment Variables

The input and output information passed to the container was set as environment variables.
For paired-end data, there are the following three.

For example, the values look like this.

INPUT_R1=s3://バケット名-アカウントID-リージョン名-an/input_data/sample1_R1.fastq.gz
INPUT_R2=s3://バケット名-アカウントID-リージョン名-an/input_data/sample1_R2.fastq.gz
OUTPUT_S3=s3://バケット名-アカウントID-リージョン名-an/output_data/

For single-end data, set INPUT_R2 as follows.

INPUT_R2=NONE

In addition to the four environment variables set earlier, set these S3 URIs.

Adding the sample input/output locations to the environment variables

Since this was a test with only one sample, the S3 URIs are set directly in the job definition.
When processing multiple samples, I think it is easier to use if you don't define the input and output environment variables here, and instead make it possible to override the values when submitting jobs, as described in the next section.

9. Submit the FastQC Job

Now that the job definition and Job queue are ready, select "Submit new job" from the "Jobs" screen of AWS Batch.

The settings are as follows.

Job submission screen

10. Check the Processing in CloudWatch Logs

When you open the log stream from the job details screen, you can check the container's execution log in CloudWatch Logs.

In this log, you can mainly confirm the following processing.

• Downloading R1 from S3
• Downloading R2 from S3
• Running FastQC
• Uploading results to S3

Even if a job fails, looking at CloudWatch Logs first lets you see which step caused the error.

CloudWatch Logs from the FastQC run

In the end, the AWS Batch job status became SUCCEEDED.

Job SUCCEEDED screen

11. Check the FastQC Report Output to S3

Finally, check output_data/ in the S3 bucket.
For paired-end data, an HTML and a ZIP are created for each FASTQ.

output_data/
├── sample1_R1_fastqc.html
├── sample1_R1_fastqc.zip
├── sample1_R2_fastqc.html
└── sample1_R2_fastqc.zip

FastQC files output to S3

If you download the HTML file and open it in a local browser, you can view the usual FastQC report.

FastQC report opened in a browser

Good work!

With this, the quality check of the FASTQ file using AWS Batch from the AWS Management Console is complete
We were able to confirm the whole flow of analyzing a FASTQ on S3 with AWS Batch and returning the results to S3.
It may take time to set up at first, but it will become smoother as you get used to it.

Also, as explained in detail in the next section, many of the resources created this time do not incur charges just by being kept. Some resources, such as the Compute environment, Job queue, and IAM roles, can be reused, so next time you can prepare the environment with fewer steps than this time.

12. Cleanup

After verification, let's delete the resources you no longer need.

• CloudWatch Logs log group
• Input FASTQ
• FastQC results
• CodeBuild project
• ECR repository

No Fargate Charges Unless Jobs Are Running

Fargate is billed for vCPU, memory, and so on only while jobs are actually running.
Therefore, even if you leave the AWS Batch Compute environment in place, no Fargate compute charges are incurred as long as no jobs are running.

Also, the following resources basically do not incur charges just by being left in place.

• AWS Batch
  • Compute environment
  • Job queue
  • Job definition
• IAM roles and policies
• Default VPC
• Subnets
• Security groups

On the other hand, the services where charges may be incurred while data is stored are mainly the following.

• S3
• ECR
• CloudWatch Logs

Be Careful with NAT Gateways

This time, we did not create a NAT Gateway and instead used a configuration that assigns a public IP to the Fargate job.
If you have created a NAT Gateway, you are charged according to the time the NAT Gateway exists, even when there is no traffic. If it is not needed after verification, you need to delete it.

Summary

This time, we used AWS Batch and Fargate to analyze a FASTQ file on S3 with FastQC.
The flow we followed is summarized as follows.

• Upload the FASTQ file to S3
• Create an ECR repository
• Build the FastQC container with CodeBuild
• Push the container image to ECR
• Create IAM roles for AWS Batch
• Create a Fargate Compute environment
• Create a Job queue and Job definition
• Submit the FastQC job
• Check the HTML report output to S3

By first running one sample, we were able to go through the following relationships in AWS Batch.

• Job definition: what to run and with which resources
• Job queue: where to make jobs wait
• Compute environment: on which infrastructure jobs actually run

This time we used a container image that includes FastQC, but if you change the software built into the container and the execution script, the same mechanism can be applied to other bioinformatics analyses. For example, read trimming, mapping, and expression quantification can also be run on AWS Batch by preparing a container image that includes the corresponding software.