この記事は公開されてから1年以上経過しています。情報が古い可能性がありますので、ご注意ください。
おはようございます、もきゅりんです。
先日、Fargateを利用するCodePipelineをCloudFormation(以下CFn)で作成したので、備忘録的なまとめとしてブログ化しておきます。とりあえずどんなもんか雰囲気を知るのが自身の趣旨だったので、手抜かりはいっぱい残っていることをご了承下さいませ。
「Flask+RDS(MySQL5.7)+FargateをCFnで構築してみる」とFargateを使ったCodePipelineをCFnで構築でセットになります。
今回使うサンプルで使うFlaskのファイルはこちらです。
はじめに
まず、「Flask+RDS(MySQL5.7)+FargateをCFnで構築してみる」については、下記の弊社記事に大きく依っています。
こちらの前提条件を含めての準備が必要となります。
- 利用予定のアカウントで、AWS CLIが利用可能
-
VPC、ALB Security Group、ECS Task Security Group(以下SG)が構築済み
-
Fargateを配置するサブネットは、インターネット通信可能であること
-
Fargateのコンテナから接続できるRDSを構築済みであること
-
ECRにファイルをPushすること (上記記事にPushの手順は記載されています)
-
CodeCommitに今回使うファイルをPushしておくこと (次回使います)
CodeCommitは下記を参考にして下さい。
AWS CLI 認証情報ヘルパーを使用する Linux, macOS, or Unix での AWS CodeCommit リポジトリへの HTTPS 接続のセットアップステップ
SGの設定例
対象 | Port | Source |
---|---|---|
ALB | 80 | 0.0.0.0/0 |
ECS | 80 | ALBのSG |
RDS | 3306 | ECSのSG |
なお、RDSですが、MySQL互換DBのみを対象としているのにご注意下さい。
MySQL5.7.25とMariaDB10.3.13は動作確認済みです。
RDS(MySQL)の作成
export SECURITY_GROUP_NAME=mysql-from-default-vpc
export RDS_DATABASE_NAME=demo-flask-mysql
export DB_NAME=flask_db
export USER_NAME=YOUR_NAME
export USER_PASSWORD=YOUR_PASSWORD
# セキュリティグループ(以下sg)の作成
SECURITY_GROUP_ID=`aws ec2 create-security-group \
--description ${SECURITY_GROUP_NAME} \
--group-name ${SECURITY_GROUP_NAME} \
| jq -r '.GroupId'`
# YOUR_VPC_CIDRから3306を開放するルールを作成
aws ec2 authorize-security-group-ingress \
--group-id ${SECURITY_GROUP_ID} \
--protocol tcp \
--port 3306 \
--cidr YOUR_VPC_CIDR
# DBを作成
aws rds create-db-instance \
--db-instance-identifier ${RDS_DATABASE_NAME} \
--db-name ${DB_NAME} \
--vpc-security-group-ids ${SECURITY_GROUP_ID} \
--allocated-storage 20 \
--db-instance-class db.t2.micro \
--engine mysql \
--engine-version 5.7 \
--master-username ${USER_NAME} \
--master-user-password ${USER_PASSWORD}
しばらくしたらエンドポイントを確認しましょう。
表示されたら控えます。
後ほどjsonパラメータに利用します。
aws rds describe-db-instances \
--db-instance-identifier ${RDS_DATABASE_NAME}
Cloudformationテンプレート
上記参考ブログのテンプレートと、今回利用するテンプレートの変更点だけ記載しておきます。
- ヘルスチェックパスを追記 (ログインページにリダイレクトするため)
- ALBのバケットの記載はコメントアウト(検証のみのため)
- 環境変数としてDBの接続情報を追記
DBの接続情報ですが、SSMのパラメータストアから取得したかったのですが、現在(2019/5/6)は出来なそうだったため、外部パラメータとしています。
Amazon Elastic Container Service TaskDefinition KeyValuePair
ECS::TaskDefinition ContainerDefinitions don't support Secrets property?
準備ができたら、下記コマンドでスタック作成します。
aws cloudformation create-stack --stack-name YOUR_STACK_NAME \
--template-body file://`pwd`/sample-fargate.yaml \
--parameters file://`pwd`/fargate.parameter.json \
--capabilities CAPABILITY_NAMED_IAM
※ 今回の例では、ECSClusterName,ECSContainerName,ECSServiceNameを、Defaultから変更しないようお勧めします。 (次回のパイプライン作成時に設定が煩雑になるため。)
sample-fargate.yaml
AWSTemplateFormatVersion: 2010-09-09
Description:
Fargate and ALB Create
Metadata:
"AWS::CloudFormation::Interface":
ParameterGroups:
- Label:
default: "Project Name Prefix"
Parameters:
- ProjectName
- Label:
default: "InternetALB Configuration"
Parameters:
- InternetALBName
- TargetGroupName
- Label:
default: "Fargate for ECS Configuration"
Parameters:
- ECSClusterName
- ECSTaskName
- ECSTaskCPUUnit
- ECSTaskMemory
- ECSContainerName
- ECSImageName
- ECSServiceName
- ECSTaskDesiredCount
- Label:
default: "Netowork Configuration"
Parameters:
- VpcId
- ALBSecurityGroupId
- ALBSubnetId1
- ALBSubnetId2
- ECSSecurityGroupId
- ECSSubnetId1
- ECSSubnetId2
- Label:
default: "Scaling Configuration"
Parameters:
- ServiceScaleEvaluationPeriods
- ServiceCpuScaleOutThreshold
- ServiceCpuScaleInThreshold
- TaskMinContainerCount
- TaskMaxContainerCount
ParameterLabels:
InternetALBName:
default: "InternetALBName"
TargetGroupName:
default: "TargetGroupName"
ECSClusterName:
default: "ECSClusterName"
ECSTaskName:
default: "ECSTaskName"
ECSTaskCPUUnit:
default: "ECSTaskCPUUnit"
ECSTaskMemory:
default: "ECSTaskMemory"
ECSContainerName:
default: "ECSContainerName"
ECSImageName:
default: "ECSImageName"
ECSServiceName:
default: "ECSServiceName"
ECSTaskDesiredCount:
default: "ECSTaskDesiredCount"
# ------------------------------------------------------------#
# Input Parameters
# ------------------------------------------------------------#
Parameters:
ProjectName:
Default: kaji-test
Type: String
# 追記
DBMasterName:
Type: String
DBPass:
Type: String
DBEndPoint:
Type: String
DBNAME:
Type: String
#VPCID
VpcId:
Description : "VPC ID"
Type: AWS::EC2::VPC::Id
#ALBSecurity Group
ALBSecurityGroupId:
Type: AWS::EC2::SecurityGroup::Id
#ALBSubnet1
ALBSubnetId1:
Description : "ALB Subnet 1st"
Type : AWS::EC2::Subnet::Id
#ALBSubnet2
ALBSubnetId2:
Description : "ALB Subnet 2st"
Type : AWS::EC2::Subnet::Id
#ECSSecurity Group
ECSSecurityGroupId:
Type: AWS::EC2::SecurityGroup::Id
#ECSSubnet1
ECSSubnetId1:
Description : "ECS Subnet 1st"
Type : AWS::EC2::Subnet::Id
#ECSSubnet2
ECSSubnetId2:
Description : "ECS Subnet 2st"
Type : AWS::EC2::Subnet::Id
#InternetALB
InternetALBName:
Type: String
Default: "alb"
#TargetGroupName
TargetGroupName:
Type: String
Default: "tg"
#ECSClusterName
ECSClusterName:
Type: String
Default: "cluster"
#ECSTaskName
ECSTaskName:
Type: String
Default: "task"
#ECSTaskCPUUnit
ECSTaskCPUUnit:
AllowedValues: [ 256, 512, 1024, 2048, 4096 ]
Type: String
Default: "256"
#ECSTaskMemory
ECSTaskMemory:
AllowedValues: [ 256, 512, 1024, 2048, 4096 ]
Type: String
Default: "512"
#ECSContainerName
ECSContainerName:
Type: String
Default: "container"
#ECSImageName
ECSImageName:
Type: String
Default: "xxxxxxxxxxxx.dkr.ecr.ap-northeast-1.amazonaws.com/kaji-test-ecr:latest"
#ECSServiceName
ECSServiceName:
Type: String
Default: "service"
#ECSTaskDesiredCount
ECSTaskDesiredCount:
Type: Number
Default: 1
# Scaling params
ServiceScaleEvaluationPeriods:
Description: The number of periods over which data is compared to the specified threshold
Type: Number
Default: 2
MinValue: 2
ServiceCpuScaleOutThreshold:
Type: Number
Description: Average CPU value to trigger auto scaling out
Default: 50
MinValue: 0
MaxValue: 100
ConstraintDescription: Value must be between 0 and 100
ServiceCpuScaleInThreshold:
Type: Number
Description: Average CPU value to trigger auto scaling in
Default: 25
MinValue: 0
MaxValue: 100
ConstraintDescription: Value must be between 0 and 100
TaskMinContainerCount:
Type: Number
Description: Minimum number of containers to run for the service
Default: 1
MinValue: 1
ConstraintDescription: Value must be at least one
TaskMaxContainerCount:
Type: Number
Description: Maximum number of containers to run for the service when auto scaling out
Default: 2
MinValue: 1
ConstraintDescription: Value must be at least one
Resources:
# ------------------------------------------------------------#
# Target Group
# ------------------------------------------------------------#
TargetGroup:
Type: "AWS::ElasticLoadBalancingV2::TargetGroup"
Properties:
# 追記
HealthCheckPath: /auth/login
VpcId: !Ref VpcId
Name: !Sub "${ProjectName}-${TargetGroupName}"
Protocol: HTTP
Port: 80
TargetType: ip
# ------------------------------------------------------------#
# Internet ALB
# ------------------------------------------------------------#
InternetALB:
Type: "AWS::ElasticLoadBalancingV2::LoadBalancer"
Properties:
Name: !Sub "${ProjectName}-${InternetALBName}"
Tags:
- Key: Name
Value: !Sub "${ProjectName}-${InternetALBName}"
Scheme: "internet-facing"
LoadBalancerAttributes:
- Key: "deletion_protection.enabled"
Value: false
- Key: "idle_timeout.timeout_seconds"
Value: 60
# CommentOut
# - Key: "access_logs.s3.enabled"
# Value: true
# - Key: "access_logs.s3.bucket"
# Value: !Sub "alb-log-${AWS::AccountId}"
SecurityGroups:
- !Ref ALBSecurityGroupId
Subnets:
- !Ref ALBSubnetId1
- !Ref ALBSubnetId2
ALBListener:
Type: "AWS::ElasticLoadBalancingV2::Listener"
Properties:
DefaultActions:
- TargetGroupArn: !Ref TargetGroup
Type: forward
LoadBalancerArn: !Ref InternetALB
Port: 80
Protocol: HTTP
# ------------------------------------------------------------#
# ECS Cluster
# ------------------------------------------------------------#
ECSCluster:
Type: "AWS::ECS::Cluster"
Properties:
ClusterName: !Sub "${ProjectName}-${ECSClusterName}"
# ------------------------------------------------------------#
# ECS LogGroup
# ------------------------------------------------------------#
ECSLogGroup:
Type: "AWS::Logs::LogGroup"
Properties:
LogGroupName: !Sub "/ecs/logs/${ProjectName}-ecs-group"
# ------------------------------------------------------------#
# ECS Task Execution Role
# ------------------------------------------------------------#
ECSTaskExecutionRole:
Type: AWS::IAM::Role
Properties:
RoleName: !Sub "${ProjectName}-ECSTaskExecutionRolePolicy"
Path: /
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service: ecs-tasks.amazonaws.com
Action: sts:AssumeRole
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy
# ------------------------------------------------------------#
# ECS TaskDefinition
# ------------------------------------------------------------#
ECSTaskDefinition:
Type: "AWS::ECS::TaskDefinition"
Properties:
Cpu: !Ref ECSTaskCPUUnit
ExecutionRoleArn: !Ref ECSTaskExecutionRole
Family: !Sub "${ProjectName}-${ECSTaskName}"
Memory: !Ref ECSTaskMemory
NetworkMode: awsvpc
RequiresCompatibilities:
- FARGATE
#ContainerDefinitions
ContainerDefinitions:
- Name: !Sub "${ProjectName}-${ECSContainerName}"
Image: !Ref ECSImageName
Environment:
- Name: USER_NAME
Value: !Ref DBMasterName
- Name: USER_PASS
Value: !Ref DBPass
- Name: DB_ENDPOINT
Value: !Ref DBEndPoint
- Name: DB_NAME
Value: !Ref DBNAME
LogConfiguration:
LogDriver: awslogs
Options:
awslogs-group: !Ref ECSLogGroup
awslogs-region: !Ref "AWS::Region"
awslogs-stream-prefix: !Ref ProjectName
MemoryReservation: 128
PortMappings:
- HostPort: 80
Protocol: tcp
ContainerPort: 80
# ------------------------------------------------------------#
# ECS Service
# ------------------------------------------------------------#
ECSService:
Type: AWS::ECS::Service
DependsOn: ALBListener
Properties:
Cluster: !Ref ECSCluster
DesiredCount: !Ref ECSTaskDesiredCount
LaunchType: FARGATE
LoadBalancers:
-
TargetGroupArn: !Ref TargetGroup
ContainerPort: 80
ContainerName: !Sub "${ProjectName}-${ECSContainerName}"
NetworkConfiguration:
AwsvpcConfiguration:
AssignPublicIp: ENABLED
SecurityGroups:
- !Ref ECSSecurityGroupId
Subnets:
- !Ref ECSSubnetId1
- !Ref ECSSubnetId2
ServiceName: !Sub "${ProjectName}-${ECSServiceName}"
TaskDefinition: !Ref ECSTaskDefinition
# ------------------------------------------------------------#
# Auto Scaling Service
# ------------------------------------------------------------#
ServiceAutoScalingRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service: application-autoscaling.amazonaws.com
Action: sts:AssumeRole
Path: /
Policies:
- PolicyName: !Sub "${ProjectName}-${ECSContainerName}-autoscaling"
PolicyDocument:
Statement:
- Effect: Allow
Action:
- application-autoscaling:*
- cloudwatch:DescribeAlarms
- cloudwatch:PutMetricAlarm
- ecs:DescribeServices
- ecs:UpdateService
Resource: '*'
ServiceScalingTarget:
Type: AWS::ApplicationAutoScaling::ScalableTarget
Properties:
MinCapacity: !Ref TaskMinContainerCount
MaxCapacity: !Ref TaskMaxContainerCount
ResourceId: !Sub
- service/${EcsClusterName}/${EcsDefaultServiceName}
- EcsClusterName: !Ref ECSCluster
EcsDefaultServiceName: !Sub "${ProjectName}-${ECSServiceName}"
RoleARN: !GetAtt ServiceAutoScalingRole.Arn
ScalableDimension: ecs:service:DesiredCount
ServiceNamespace: ecs
DependsOn:
- ECSService
- ServiceAutoScalingRole
ServiceScaleOutPolicy:
Type: AWS::ApplicationAutoScaling::ScalingPolicy
Properties:
PolicyName: !Sub "${ProjectName}-${ECSServiceName}-ScaleOutPolicy"
PolicyType: StepScaling
ScalingTargetId: !Ref ServiceScalingTarget
StepScalingPolicyConfiguration:
AdjustmentType: ChangeInCapacity
Cooldown: 60
MetricAggregationType: Average
StepAdjustments:
- ScalingAdjustment: 1
MetricIntervalLowerBound: 0
DependsOn: ServiceScalingTarget
ServiceScaleInPolicy:
Type: AWS::ApplicationAutoScaling::ScalingPolicy
Properties:
PolicyName: !Sub "${ProjectName}-${ECSServiceName}-ScaleInPolicy"
PolicyType: StepScaling
ScalingTargetId: !Ref ServiceScalingTarget
StepScalingPolicyConfiguration:
AdjustmentType: ChangeInCapacity
Cooldown: 60
MetricAggregationType: Average
StepAdjustments:
- ScalingAdjustment: -1
MetricIntervalUpperBound: 0
DependsOn: ServiceScalingTarget
ServiceScaleOutAlarm:
Type: AWS::CloudWatch::Alarm
Properties:
AlarmName: !Sub "${ProjectName}-${ECSServiceName}-ScaleOutAlarm"
EvaluationPeriods: !Ref ServiceScaleEvaluationPeriods
Statistic: Average
TreatMissingData: notBreaching
Threshold: !Ref ServiceCpuScaleOutThreshold
AlarmDescription: Alarm to add capacity if CPU is high
Period: 60
AlarmActions:
- !Ref ServiceScaleOutPolicy
Namespace: AWS/ECS
Dimensions:
- Name: ClusterName
Value: !Ref ECSCluster
- Name: ServiceName
Value: !Sub "${ProjectName}-${ECSServiceName}"
ComparisonOperator: GreaterThanThreshold
MetricName: CPUUtilization
DependsOn:
- ECSService
- ServiceScaleOutPolicy
ServiceScaleInAlarm:
Type: AWS::CloudWatch::Alarm
Properties:
AlarmName: !Sub "${ProjectName}-${ECSServiceName}-ScaleInAlarm"
EvaluationPeriods: !Ref ServiceScaleEvaluationPeriods
Statistic: Average
TreatMissingData: notBreaching
Threshold: !Ref ServiceCpuScaleInThreshold
AlarmDescription: Alarm to reduce capacity if container CPU is low
Period: 300
AlarmActions:
- !Ref ServiceScaleInPolicy
Namespace: AWS/ECS
Dimensions:
- Name: ClusterName
Value: !Ref ECSCluster
- Name: ServiceName
Value: !Sub "${ProjectName}-${ECSServiceName}"
ComparisonOperator: LessThanThreshold
MetricName: CPUUtilization
DependsOn:
- ECSService
- ServiceScaleInPolicy
パラメータ例
fargate.parameter.json
[
{
"ParameterKey": "ProjectName",
"ParameterValue": "demo-flask"
},
{
"ParameterKey": "VpcId",
"ParameterValue": "vpc-xxxxxx"
},
{
"ParameterKey": "ALBSecurityGroupId",
"ParameterValue": "sg-xxxxxx"
},
{
"ParameterKey": "ALBSubnetId1",
"ParameterValue": "subnet-xxxxxx"
},
{
"ParameterKey": "ALBSubnetId2",
"ParameterValue": "subnet-yyyyyy"
},
{
"ParameterKey": "ECSSecurityGroupId",
"ParameterValue": "sg-zzzzzz"
},
{
"ParameterKey": "ECSSubnetId1",
"ParameterValue": "subnet-zzzzzz"
},
{
"ParameterKey": "ECSSubnetId2",
"ParameterValue": "subnet-mmmmmm"
},
{
"ParameterKey": "ECSImageName",
"ParameterValue": "xxxxxx.dkr.ecr.ap-northeast-1.amazonaws.com/YOUR_REPOSITORY_NAME"
},
{
"ParameterKey": "DBMasterName",
"ParameterValue": "DB_USER_NAME"
},
{
"ParameterKey": "DBPass",
"ParameterValue": "DB_PASS"
},
{
"ParameterKey": "DBEndPoint",
"ParameterValue": "RDS_ENDPOINT"
},
{
"ParameterKey": "DBNAME",
"ParameterValue": "DB_NAME"
}
]
確認
作成完了後、ALBのDNS名で外部からアクセスすると、以下のような画面が表示されます。
ユーザー登録しましょう
無事登録できたので、ログインします
無事にログインできたので、記事を書きます
記事が表示されました
では、続きはFargate+CodePipelineをCFnで構築してみるです。