Fargate+RDS(MySQL5.7)+FlaskをCFnで構築してみる

2019.05.08

この記事は公開されてから1年以上経過しています。情報が古い可能性がありますので、ご注意ください。

おはようございます、もきゅりんです。

先日、Fargateを利用するCodePipelineをCloudFormation(以下CFn)で作成したので、備忘録的なまとめとしてブログ化しておきます。とりあえずどんなもんか雰囲気を知るのが自身の趣旨だったので、手抜かりはいっぱい残っていることをご了承下さいませ。

「Flask+RDS(MySQL5.7)+FargateをCFnで構築してみる」とFargateを使ったCodePipelineをCFnで構築でセットになります。

今回使うサンプルで使うFlaskのファイルはこちらです。

はじめに

まず、「Flask+RDS(MySQL5.7)+FargateをCFnで構築してみる」については、下記の弊社記事に大きく依っています。

CloudformationでFargateを構築する

こちらの前提条件を含めての準備が必要となります。

利用予定のアカウントで、AWS CLIが利用可能
VPC、ALB Security Group、ECS Task Security Group(以下SG)が構築済み
Fargateを配置するサブネットは、インターネット通信可能であること
Fargateのコンテナから接続できるRDSを構築済みであること
ECRにファイルをPushすること (上記記事にPushの手順は記載されています)
CodeCommitに今回使うファイルをPushしておくこと (次回使います)

CodeCommitは下記を参考にして下さい。

AWS CLI 認証情報ヘルパーを使用する Linux, macOS, or Unix での AWS CodeCommit リポジトリへの HTTPS 接続のセットアップステップ

SGの設定例

対象	Port	Source
ALB	80	0.0.0.0/0
ECS	80	ALBのSG
RDS	3306	ECSのSG

なお、RDSですが、MySQL互換DBのみを対象としているのにご注意下さい。
MySQL5.7.25とMariaDB10.3.13は動作確認済みです。

RDS(MySQL)の作成

export SECURITY_GROUP_NAME=mysql-from-default-vpc
export RDS_DATABASE_NAME=demo-flask-mysql
export DB_NAME=flask_db
export USER_NAME=YOUR_NAME
export USER_PASSWORD=YOUR_PASSWORD

# セキュリティグループ(以下sg)の作成
SECURITY_GROUP_ID=`aws ec2 create-security-group \
--description ${SECURITY_GROUP_NAME} \
--group-name ${SECURITY_GROUP_NAME} \
| jq -r '.GroupId'`

# YOUR_VPC_CIDRから3306を開放するルールを作成
aws ec2 authorize-security-group-ingress \
--group-id ${SECURITY_GROUP_ID} \
--protocol tcp \
--port 3306 \
--cidr YOUR_VPC_CIDR

# DBを作成
aws rds create-db-instance \
--db-instance-identifier ${RDS_DATABASE_NAME} \
--db-name ${DB_NAME} \
--vpc-security-group-ids ${SECURITY_GROUP_ID} \
--allocated-storage 20 \
--db-instance-class db.t2.micro \
--engine mysql \
--engine-version 5.7 \
--master-username ${USER_NAME} \
--master-user-password ${USER_PASSWORD}

しばらくしたらエンドポイントを確認しましょう。

表示されたら控えます。

後ほどjsonパラメータに利用します。

aws rds describe-db-instances \
--db-instance-identifier ${RDS_DATABASE_NAME}

Cloudformationテンプレート

上記参考ブログのテンプレートと、今回利用するテンプレートの変更点だけ記載しておきます。

ヘルスチェックパスを追記 (ログインページにリダイレクトするため)
ALBのバケットの記載はコメントアウト(検証のみのため)
環境変数としてDBの接続情報を追記

DBの接続情報ですが、SSMのパラメータストアから取得したかったのですが、現在(2019/5/6)は出来なそうだったため、外部パラメータとしています。

Amazon Elastic Container Service TaskDefinition KeyValuePair

ECS::TaskDefinition ContainerDefinitions don't support Secrets property?

準備ができたら、下記コマンドでスタック作成します。

aws cloudformation create-stack --stack-name YOUR_STACK_NAME \
--template-body file://`pwd`/sample-fargate.yaml \
--parameters file://`pwd`/fargate.parameter.json \
--capabilities CAPABILITY_NAMED_IAM

※ 今回の例では、ECSClusterName,ECSContainerName,ECSServiceNameを、Defaultから変更しないようお勧めします。 (次回のパイプライン作成時に設定が煩雑になるため。)

sample-fargate.yaml

AWSTemplateFormatVersion: 2010-09-09
Description:
  Fargate and ALB Create

Metadata:
  "AWS::CloudFormation::Interface":
    ParameterGroups:
      - Label:
          default: "Project Name Prefix"
        Parameters:
          - ProjectName
      - Label:
          default: "InternetALB Configuration"
        Parameters:
          - InternetALBName
          - TargetGroupName
      - Label:
          default: "Fargate for ECS Configuration"
        Parameters:
          - ECSClusterName
          - ECSTaskName
          - ECSTaskCPUUnit
          - ECSTaskMemory
          - ECSContainerName
          - ECSImageName
          - ECSServiceName
          - ECSTaskDesiredCount
      - Label:
          default: "Netowork Configuration"
        Parameters:
          - VpcId
          - ALBSecurityGroupId
          - ALBSubnetId1
          - ALBSubnetId2
          - ECSSecurityGroupId
          - ECSSubnetId1
          - ECSSubnetId2
      - Label:
          default: "Scaling Configuration"
        Parameters:
          - ServiceScaleEvaluationPeriods
          - ServiceCpuScaleOutThreshold
          - ServiceCpuScaleInThreshold
          - TaskMinContainerCount
          - TaskMaxContainerCount

    ParameterLabels:
      InternetALBName:
        default: "InternetALBName"
      TargetGroupName:
        default: "TargetGroupName"
      ECSClusterName:
        default: "ECSClusterName"
      ECSTaskName:
        default: "ECSTaskName"
      ECSTaskCPUUnit:
        default: "ECSTaskCPUUnit"
      ECSTaskMemory:
        default: "ECSTaskMemory"
      ECSContainerName:
        default: "ECSContainerName"
      ECSImageName:
        default: "ECSImageName"
      ECSServiceName:
        default: "ECSServiceName"
      ECSTaskDesiredCount:
        default: "ECSTaskDesiredCount"

# ------------------------------------------------------------#
# Input Parameters
# ------------------------------------------------------------# 
Parameters:
  ProjectName:
    Default: kaji-test
    Type: String

  # 追記
  DBMasterName:
    Type: String
  DBPass:
    Type: String
  DBEndPoint:
    Type: String
  DBNAME:
    Type: String            

#VPCID
  VpcId:
    Description : "VPC ID"
    Type: AWS::EC2::VPC::Id

#ALBSecurity Group
  ALBSecurityGroupId:
    Type: AWS::EC2::SecurityGroup::Id

#ALBSubnet1
  ALBSubnetId1:
    Description : "ALB Subnet 1st"
    Type : AWS::EC2::Subnet::Id

#ALBSubnet2
  ALBSubnetId2:
    Description : "ALB Subnet 2st"
    Type : AWS::EC2::Subnet::Id

#ECSSecurity Group
  ECSSecurityGroupId:
    Type: AWS::EC2::SecurityGroup::Id

#ECSSubnet1
  ECSSubnetId1:
    Description : "ECS Subnet 1st"
    Type : AWS::EC2::Subnet::Id

#ECSSubnet2
  ECSSubnetId2:
    Description : "ECS Subnet 2st"
    Type : AWS::EC2::Subnet::Id

#InternetALB
  InternetALBName:
    Type: String
    Default: "alb"

#TargetGroupName
  TargetGroupName:
    Type: String
    Default: "tg"

#ECSClusterName
  ECSClusterName:
    Type: String
    Default: "cluster"

#ECSTaskName
  ECSTaskName:
    Type: String
    Default: "task"

#ECSTaskCPUUnit
  ECSTaskCPUUnit:
    AllowedValues: [ 256, 512, 1024, 2048, 4096  ]
    Type: String
    Default: "256"

#ECSTaskMemory
  ECSTaskMemory:
    AllowedValues: [ 256, 512, 1024, 2048, 4096  ]
    Type: String
    Default: "512"

#ECSContainerName
  ECSContainerName:
    Type: String
    Default: "container"

#ECSImageName
  ECSImageName:
    Type: String
    Default: "xxxxxxxxxxxx.dkr.ecr.ap-northeast-1.amazonaws.com/kaji-test-ecr:latest"

#ECSServiceName
  ECSServiceName:
    Type: String
    Default: "service"

#ECSTaskDesiredCount
  ECSTaskDesiredCount:
    Type: Number
    Default: 1

# Scaling params
  ServiceScaleEvaluationPeriods:
    Description: The number of periods over which data is compared to the specified threshold
    Type: Number
    Default: 2
    MinValue: 2

  ServiceCpuScaleOutThreshold:
    Type: Number
    Description: Average CPU value to trigger auto scaling out
    Default: 50
    MinValue: 0
    MaxValue: 100
    ConstraintDescription: Value must be between 0 and 100

  ServiceCpuScaleInThreshold:
    Type: Number
    Description: Average CPU value to trigger auto scaling in
    Default: 25
    MinValue: 0
    MaxValue: 100
    ConstraintDescription: Value must be between 0 and 100

  TaskMinContainerCount:
    Type: Number
    Description: Minimum number of containers to run for the service
    Default: 1
    MinValue: 1
    ConstraintDescription: Value must be at least one

  TaskMaxContainerCount:
    Type: Number
    Description: Maximum number of containers to run for the service when auto scaling out
    Default: 2
    MinValue: 1
    ConstraintDescription: Value must be at least one

Resources:

# ------------------------------------------------------------#
#  Target Group
# ------------------------------------------------------------#
  TargetGroup:
    Type: "AWS::ElasticLoadBalancingV2::TargetGroup"
    Properties:
      # 追記
      HealthCheckPath: /auth/login
      VpcId: !Ref VpcId
      Name: !Sub "${ProjectName}-${TargetGroupName}"
      Protocol: HTTP
      Port: 80
      TargetType: ip

# ------------------------------------------------------------#
#  Internet ALB
# ------------------------------------------------------------#
  InternetALB:
    Type: "AWS::ElasticLoadBalancingV2::LoadBalancer"
    Properties:
      Name: !Sub "${ProjectName}-${InternetALBName}"
      Tags:
        - Key: Name
          Value: !Sub "${ProjectName}-${InternetALBName}"
      Scheme: "internet-facing"
      LoadBalancerAttributes:
        - Key: "deletion_protection.enabled"
          Value: false
        - Key: "idle_timeout.timeout_seconds"
          Value: 60
        # CommentOut  
        # - Key: "access_logs.s3.enabled"
        #   Value: true
        # - Key: "access_logs.s3.bucket"
        #   Value: !Sub "alb-log-${AWS::AccountId}"
      SecurityGroups:
        - !Ref ALBSecurityGroupId
      Subnets:
        - !Ref ALBSubnetId1
        - !Ref ALBSubnetId2

  ALBListener:
    Type: "AWS::ElasticLoadBalancingV2::Listener"
    Properties:
      DefaultActions:
        - TargetGroupArn: !Ref TargetGroup
          Type: forward
      LoadBalancerArn: !Ref InternetALB
      Port: 80
      Protocol: HTTP

# ------------------------------------------------------------#
# ECS Cluster
# ------------------------------------------------------------#
  ECSCluster:
    Type: "AWS::ECS::Cluster"
    Properties:
      ClusterName: !Sub "${ProjectName}-${ECSClusterName}"

# ------------------------------------------------------------#
#  ECS LogGroup
# ------------------------------------------------------------#
  ECSLogGroup:
    Type: "AWS::Logs::LogGroup"
    Properties:
      LogGroupName: !Sub "/ecs/logs/${ProjectName}-ecs-group"

# ------------------------------------------------------------#
#  ECS Task Execution Role
# ------------------------------------------------------------#
  ECSTaskExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: !Sub "${ProjectName}-ECSTaskExecutionRolePolicy"
      Path: /
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service: ecs-tasks.amazonaws.com
            Action: sts:AssumeRole
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy

# ------------------------------------------------------------#
#  ECS TaskDefinition
# ------------------------------------------------------------#
  ECSTaskDefinition:
    Type: "AWS::ECS::TaskDefinition"
    Properties:
      Cpu: !Ref ECSTaskCPUUnit
      ExecutionRoleArn: !Ref ECSTaskExecutionRole
      Family: !Sub "${ProjectName}-${ECSTaskName}"
      Memory: !Ref ECSTaskMemory
      NetworkMode: awsvpc
      RequiresCompatibilities:
        - FARGATE

#ContainerDefinitions
      ContainerDefinitions:
        - Name: !Sub "${ProjectName}-${ECSContainerName}"
          Image: !Ref ECSImageName
          Environment:
           - Name: USER_NAME
             Value: !Ref DBMasterName
           - Name: USER_PASS
             Value: !Ref DBPass
           - Name: DB_ENDPOINT
             Value: !Ref DBEndPoint
           - Name: DB_NAME
             Value: !Ref DBNAME                  
          LogConfiguration:
            LogDriver: awslogs
            Options:
              awslogs-group: !Ref ECSLogGroup
              awslogs-region: !Ref "AWS::Region"
              awslogs-stream-prefix: !Ref ProjectName
          MemoryReservation: 128
          PortMappings:
            - HostPort: 80
              Protocol: tcp
              ContainerPort: 80

# ------------------------------------------------------------#
#  ECS Service
# ------------------------------------------------------------#
  ECSService:
    Type: AWS::ECS::Service
    DependsOn: ALBListener
    Properties:
      Cluster: !Ref ECSCluster
      DesiredCount: !Ref ECSTaskDesiredCount
      LaunchType: FARGATE
      LoadBalancers:
        -
          TargetGroupArn: !Ref TargetGroup
          ContainerPort: 80
          ContainerName: !Sub "${ProjectName}-${ECSContainerName}"
      NetworkConfiguration:
       AwsvpcConfiguration:
           AssignPublicIp: ENABLED
           SecurityGroups:
             - !Ref ECSSecurityGroupId
           Subnets:
             - !Ref ECSSubnetId1
             - !Ref ECSSubnetId2
      ServiceName: !Sub "${ProjectName}-${ECSServiceName}"
      TaskDefinition: !Ref ECSTaskDefinition

# ------------------------------------------------------------#
#  Auto Scaling Service
# ------------------------------------------------------------#
  ServiceAutoScalingRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Statement:
          - Effect: Allow
            Principal:
              Service: application-autoscaling.amazonaws.com
            Action: sts:AssumeRole
      Path: /
      Policies:
        - PolicyName: !Sub "${ProjectName}-${ECSContainerName}-autoscaling"
          PolicyDocument:
            Statement:
              - Effect: Allow
                Action:
                  - application-autoscaling:*
                  - cloudwatch:DescribeAlarms
                  - cloudwatch:PutMetricAlarm
                  - ecs:DescribeServices
                  - ecs:UpdateService
                Resource: '*'

  ServiceScalingTarget:
    Type: AWS::ApplicationAutoScaling::ScalableTarget
    Properties:
      MinCapacity: !Ref TaskMinContainerCount
      MaxCapacity: !Ref TaskMaxContainerCount
      ResourceId: !Sub
        - service/${EcsClusterName}/${EcsDefaultServiceName}
        - EcsClusterName: !Ref ECSCluster
          EcsDefaultServiceName: !Sub "${ProjectName}-${ECSServiceName}"
      RoleARN: !GetAtt ServiceAutoScalingRole.Arn
      ScalableDimension: ecs:service:DesiredCount
      ServiceNamespace: ecs
    DependsOn:
      - ECSService
      - ServiceAutoScalingRole

  ServiceScaleOutPolicy:
    Type: AWS::ApplicationAutoScaling::ScalingPolicy
    Properties:
      PolicyName: !Sub "${ProjectName}-${ECSServiceName}-ScaleOutPolicy"
      PolicyType: StepScaling
      ScalingTargetId: !Ref ServiceScalingTarget
      StepScalingPolicyConfiguration:
        AdjustmentType: ChangeInCapacity
        Cooldown: 60
        MetricAggregationType: Average
        StepAdjustments:
          - ScalingAdjustment: 1
            MetricIntervalLowerBound: 0
    DependsOn: ServiceScalingTarget

  ServiceScaleInPolicy:
    Type: AWS::ApplicationAutoScaling::ScalingPolicy
    Properties:
      PolicyName: !Sub "${ProjectName}-${ECSServiceName}-ScaleInPolicy"
      PolicyType: StepScaling
      ScalingTargetId: !Ref ServiceScalingTarget
      StepScalingPolicyConfiguration:
        AdjustmentType: ChangeInCapacity
        Cooldown: 60
        MetricAggregationType: Average
        StepAdjustments:
          - ScalingAdjustment: -1
            MetricIntervalUpperBound: 0
    DependsOn: ServiceScalingTarget

  ServiceScaleOutAlarm:
    Type: AWS::CloudWatch::Alarm
    Properties:
      AlarmName: !Sub "${ProjectName}-${ECSServiceName}-ScaleOutAlarm"
      EvaluationPeriods: !Ref ServiceScaleEvaluationPeriods
      Statistic: Average
      TreatMissingData: notBreaching
      Threshold: !Ref ServiceCpuScaleOutThreshold
      AlarmDescription: Alarm to add capacity if CPU is high
      Period: 60
      AlarmActions:
        - !Ref ServiceScaleOutPolicy
      Namespace: AWS/ECS
      Dimensions:
        - Name: ClusterName
          Value: !Ref ECSCluster
        - Name: ServiceName
          Value: !Sub "${ProjectName}-${ECSServiceName}"
      ComparisonOperator: GreaterThanThreshold
      MetricName: CPUUtilization
    DependsOn:
      - ECSService
      - ServiceScaleOutPolicy

  ServiceScaleInAlarm:
    Type: AWS::CloudWatch::Alarm
    Properties:
      AlarmName: !Sub "${ProjectName}-${ECSServiceName}-ScaleInAlarm"
      EvaluationPeriods: !Ref ServiceScaleEvaluationPeriods
      Statistic: Average
      TreatMissingData: notBreaching
      Threshold: !Ref ServiceCpuScaleInThreshold
      AlarmDescription: Alarm to reduce capacity if container CPU is low
      Period: 300
      AlarmActions:
        - !Ref ServiceScaleInPolicy
      Namespace: AWS/ECS
      Dimensions:
        - Name: ClusterName
          Value: !Ref ECSCluster
        - Name: ServiceName
          Value: !Sub "${ProjectName}-${ECSServiceName}"
      ComparisonOperator: LessThanThreshold
      MetricName: CPUUtilization
    DependsOn:
      - ECSService
      - ServiceScaleInPolicy

パラメータ例

fargate.parameter.json

[
  {
    "ParameterKey": "ProjectName",
    "ParameterValue": "demo-flask"
  },
  {
    "ParameterKey": "VpcId",
    "ParameterValue": "vpc-xxxxxx"
  },
  {
    "ParameterKey": "ALBSecurityGroupId",
    "ParameterValue": "sg-xxxxxx"
  },
  {
    "ParameterKey": "ALBSubnetId1",
    "ParameterValue": "subnet-xxxxxx"
  },
  {
    "ParameterKey": "ALBSubnetId2",
    "ParameterValue": "subnet-yyyyyy"
  },
  {
    "ParameterKey": "ECSSecurityGroupId",
    "ParameterValue": "sg-zzzzzz"
  },
  {
    "ParameterKey": "ECSSubnetId1",
    "ParameterValue": "subnet-zzzzzz"
  },
  {
    "ParameterKey": "ECSSubnetId2",
    "ParameterValue": "subnet-mmmmmm"
  },
  {
    "ParameterKey": "ECSImageName",
    "ParameterValue": "xxxxxx.dkr.ecr.ap-northeast-1.amazonaws.com/YOUR_REPOSITORY_NAME"
  },
  {
    "ParameterKey": "DBMasterName",
    "ParameterValue": "DB_USER_NAME"
  },
  {
    "ParameterKey": "DBPass",
    "ParameterValue": "DB_PASS"
  },
  {
    "ParameterKey": "DBEndPoint",
    "ParameterValue": "RDS_ENDPOINT"
  },
  {
    "ParameterKey": "DBNAME",
    "ParameterValue": "DB_NAME"
  }
]