Fargate+RDS(MySQL5.7)+FlaskをCFnで構築してみる

おはようございます、もきゅりんです。

先日、Fargateを利用するCodePipelineをCloudFormation(以下CFn)で作成したので、備忘録的なまとめとしてブログ化しておきます。とりあえずどんなもんか雰囲気を知るのが自身の趣旨だったので、手抜かりはいっぱい残っていることをご了承下さいませ。

「Flask+RDS(MySQL5.7)+FargateをCFnで構築してみる」とFargateを使ったCodePipelineをCFnで構築でセットになります。

今回使うサンプルで使うFlaskのファイルはこちらです。

はじめに

まず、「Flask+RDS(MySQL5.7)+FargateをCFnで構築してみる」については、下記の弊社記事に大きく依っています。

CloudformationでFargateを構築する

こちらの前提条件を含めての準備が必要となります。

• 利用予定のアカウントで、AWS CLIが利用可能

• VPC、ALB Security Group、ECS Task Security Group(以下SG)が構築済み

• Fargateを配置するサブネットは、インターネット通信可能であること

• Fargateのコンテナから接続できるRDSを構築済みであること

• ECRにファイルをPushすること (上記記事にPushの手順は記載されています)

• CodeCommitに今回使うファイルをPushしておくこと (次回使います)

CodeCommitは下記を参考にして下さい。

AWS CLI 認証情報ヘルパーを使用する Linux, macOS, or Unix での AWS CodeCommit リポジトリへの HTTPS 接続のセットアップステップ

SGの設定例

対象	Port	Source
ALB	80	0.0.0.0/0
ECS	80	ALBのSG
RDS	3306	ECSのSG

なお、RDSですが、MySQL互換DBのみを対象としているのにご注意下さい。
MySQL5.7.25とMariaDB10.3.13は動作確認済みです。

RDS(MySQL)の作成

export SECURITY_GROUP_NAME=mysql-from-default-vpc
export RDS_DATABASE_NAME=demo-flask-mysql
export DB_NAME=flask_db
export USER_NAME=YOUR_NAME
export USER_PASSWORD=YOUR_PASSWORD

# セキュリティグループ(以下sg)の作成
SECURITY_GROUP_ID=`aws ec2 create-security-group \
--description ${SECURITY_GROUP_NAME} \
--group-name ${SECURITY_GROUP_NAME} \
| jq -r '.GroupId'`

# YOUR_VPC_CIDRから3306を開放するルールを作成
aws ec2 authorize-security-group-ingress \
--group-id ${SECURITY_GROUP_ID} \
--protocol tcp \
--port 3306 \
--cidr YOUR_VPC_CIDR

# DBを作成
aws rds create-db-instance \
--db-instance-identifier ${RDS_DATABASE_NAME} \
--db-name ${DB_NAME} \
--vpc-security-group-ids ${SECURITY_GROUP_ID} \
--allocated-storage 20 \
--db-instance-class db.t2.micro \
--engine mysql \
--engine-version 5.7 \
--master-username ${USER_NAME} \
--master-user-password ${USER_PASSWORD}

しばらくしたらエンドポイントを確認しましょう。

表示されたら控えます。

後ほどjsonパラメータに利用します。

aws rds describe-db-instances \
--db-instance-identifier ${RDS_DATABASE_NAME}

Cloudformationテンプレート

上記参考ブログのテンプレートと、今回利用するテンプレートの変更点だけ記載しておきます。

• ヘルスチェックパスを追記 (ログインページにリダイレクトするため)
• ALBのバケットの記載はコメントアウト(検証のみのため)
• 環境変数としてDBの接続情報を追記

DBの接続情報ですが、SSMのパラメータストアから取得したかったのですが、現在(2019/5/6)は出来なそうだったため、外部パラメータとしています。

Amazon Elastic Container Service TaskDefinition KeyValuePair

ECS::TaskDefinition ContainerDefinitions don't support Secrets property?

準備ができたら、下記コマンドでスタック作成します。

aws cloudformation create-stack --stack-name YOUR_STACK_NAME \
--template-body file://`pwd`/sample-fargate.yaml \
--parameters file://`pwd`/fargate.parameter.json \
--capabilities CAPABILITY_NAMED_IAM

※ 今回の例では、ECSClusterName,ECSContainerName,ECSServiceNameを、Defaultから変更しないようお勧めします。 (次回のパイプライン作成時に設定が煩雑になるため。)

sample-fargate.yaml

AWSTemplateFormatVersion: 2010-09-09
Description:
  Fargate and ALB Create

Metadata:
  "AWS::CloudFormation::Interface":
    ParameterGroups:
      - Label:
          default: "Project Name Prefix"
        Parameters:
          - ProjectName
      - Label:
          default: "InternetALB Configuration"
        Parameters:
          - InternetALBName
          - TargetGroupName
      - Label:
          default: "Fargate for ECS Configuration"
        Parameters:
          - ECSClusterName
          - ECSTaskName
          - ECSTaskCPUUnit
          - ECSTaskMemory
          - ECSContainerName
          - ECSImageName
          - ECSServiceName
          - ECSTaskDesiredCount
      - Label:
          default: "Netowork Configuration"
        Parameters:
          - VpcId
          - ALBSecurityGroupId
          - ALBSubnetId1
          - ALBSubnetId2
          - ECSSecurityGroupId
          - ECSSubnetId1
          - ECSSubnetId2
      - Label:
          default: "Scaling Configuration"
        Parameters:
          - ServiceScaleEvaluationPeriods
          - ServiceCpuScaleOutThreshold
          - ServiceCpuScaleInThreshold
          - TaskMinContainerCount
          - TaskMaxContainerCount

    ParameterLabels:
      InternetALBName:
        default: "InternetALBName"
      TargetGroupName:
        default: "TargetGroupName"
      ECSClusterName:
        default: "ECSClusterName"
      ECSTaskName:
        default: "ECSTaskName"
      ECSTaskCPUUnit:
        default: "ECSTaskCPUUnit"
      ECSTaskMemory:
        default: "ECSTaskMemory"
      ECSContainerName:
        default: "ECSContainerName"
      ECSImageName:
        default: "ECSImageName"
      ECSServiceName:
        default: "ECSServiceName"
      ECSTaskDesiredCount:
        default: "ECSTaskDesiredCount"

# ------------------------------------------------------------#
# Input Parameters
# ------------------------------------------------------------# 
Parameters:
  ProjectName:
    Default: kaji-test
    Type: String

  # 追記
  DBMasterName:
    Type: String
  DBPass:
    Type: String
  DBEndPoint:
    Type: String
  DBNAME:
    Type: String            

#VPCID
  VpcId:
    Description : "VPC ID"
    Type: AWS::EC2::VPC::Id

#ALBSecurity Group
  ALBSecurityGroupId:
    Type: AWS::EC2::SecurityGroup::Id

#ALBSubnet1
  ALBSubnetId1:
    Description : "ALB Subnet 1st"
    Type : AWS::EC2::Subnet::Id

#ALBSubnet2
  ALBSubnetId2:
    Description : "ALB Subnet 2st"
    Type : AWS::EC2::Subnet::Id

#ECSSecurity Group
  ECSSecurityGroupId:
    Type: AWS::EC2::SecurityGroup::Id

#ECSSubnet1
  ECSSubnetId1:
    Description : "ECS Subnet 1st"
    Type : AWS::EC2::Subnet::Id

#ECSSubnet2
  ECSSubnetId2:
    Description : "ECS Subnet 2st"
    Type : AWS::EC2::Subnet::Id

#InternetALB
  InternetALBName:
    Type: String
    Default: "alb"

#TargetGroupName
  TargetGroupName:
    Type: String
    Default: "tg"

#ECSClusterName
  ECSClusterName:
    Type: String
    Default: "cluster"

#ECSTaskName
  ECSTaskName:
    Type: String
    Default: "task"

#ECSTaskCPUUnit
  ECSTaskCPUUnit:
    AllowedValues: [ 256, 512, 1024, 2048, 4096  ]
    Type: String
    Default: "256"

#ECSTaskMemory
  ECSTaskMemory:
    AllowedValues: [ 256, 512, 1024, 2048, 4096  ]
    Type: String
    Default: "512"

#ECSContainerName
  ECSContainerName:
    Type: String
    Default: "container"

#ECSImageName
  ECSImageName:
    Type: String
    Default: "xxxxxxxxxxxx.dkr.ecr.ap-northeast-1.amazonaws.com/kaji-test-ecr:latest"

#ECSServiceName
  ECSServiceName:
    Type: String
    Default: "service"

#ECSTaskDesiredCount
  ECSTaskDesiredCount:
    Type: Number
    Default: 1

# Scaling params
  ServiceScaleEvaluationPeriods:
    Description: The number of periods over which data is compared to the specified threshold
    Type: Number
    Default: 2
    MinValue: 2

  ServiceCpuScaleOutThreshold:
    Type: Number
    Description: Average CPU value to trigger auto scaling out
    Default: 50
    MinValue: 0
    MaxValue: 100
    ConstraintDescription: Value must be between 0 and 100

  ServiceCpuScaleInThreshold:
    Type: Number
    Description: Average CPU value to trigger auto scaling in
    Default: 25
    MinValue: 0
    MaxValue: 100
    ConstraintDescription: Value must be between 0 and 100

  TaskMinContainerCount:
    Type: Number
    Description: Minimum number of containers to run for the service
    Default: 1
    MinValue: 1
    ConstraintDescription: Value must be at least one

  TaskMaxContainerCount:
    Type: Number
    Description: Maximum number of containers to run for the service when auto scaling out
    Default: 2
    MinValue: 1
    ConstraintDescription: Value must be at least one

Resources:

# ------------------------------------------------------------#
#  Target Group
# ------------------------------------------------------------#
  TargetGroup:
    Type: "AWS::ElasticLoadBalancingV2::TargetGroup"
    Properties:
      # 追記
      HealthCheckPath: /auth/login
      VpcId: !Ref VpcId
      Name: !Sub "${ProjectName}-${TargetGroupName}"
      Protocol: HTTP
      Port: 80
      TargetType: ip

# ------------------------------------------------------------#
#  Internet ALB
# ------------------------------------------------------------#
  InternetALB:
    Type: "AWS::ElasticLoadBalancingV2::LoadBalancer"
    Properties:
      Name: !Sub "${ProjectName}-${InternetALBName}"
      Tags:
        - Key: Name
          Value: !Sub "${ProjectName}-${InternetALBName}"
      Scheme: "internet-facing"
      LoadBalancerAttributes:
        - Key: "deletion_protection.enabled"
          Value: false
        - Key: "idle_timeout.timeout_seconds"
          Value: 60
        # CommentOut  
        # - Key: "access_logs.s3.enabled"
        #   Value: true
        # - Key: "access_logs.s3.bucket"
        #   Value: !Sub "alb-log-${AWS::AccountId}"
      SecurityGroups:
        - !Ref ALBSecurityGroupId
      Subnets:
        - !Ref ALBSubnetId1
        - !Ref ALBSubnetId2

  ALBListener:
    Type: "AWS::ElasticLoadBalancingV2::Listener"
    Properties:
      DefaultActions:
        - TargetGroupArn: !Ref TargetGroup
          Type: forward
      LoadBalancerArn: !Ref InternetALB
      Port: 80
      Protocol: HTTP

# ------------------------------------------------------------#
# ECS Cluster
# ------------------------------------------------------------#
  ECSCluster:
    Type: "AWS::ECS::Cluster"
    Properties:
      ClusterName: !Sub "${ProjectName}-${ECSClusterName}"

# ------------------------------------------------------------#
#  ECS LogGroup
# ------------------------------------------------------------#
  ECSLogGroup:
    Type: "AWS::Logs::LogGroup"
    Properties:
      LogGroupName: !Sub "/ecs/logs/${ProjectName}-ecs-group"

# ------------------------------------------------------------#
#  ECS Task Execution Role
# ------------------------------------------------------------#
  ECSTaskExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: !Sub "${ProjectName}-ECSTaskExecutionRolePolicy"
      Path: /
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service: ecs-tasks.amazonaws.com
            Action: sts:AssumeRole
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy

# ------------------------------------------------------------#
#  ECS TaskDefinition
# ------------------------------------------------------------#
  ECSTaskDefinition:
    Type: "AWS::ECS::TaskDefinition"
    Properties:
      Cpu: !Ref ECSTaskCPUUnit
      ExecutionRoleArn: !Ref ECSTaskExecutionRole
      Family: !Sub "${ProjectName}-${ECSTaskName}"
      Memory: !Ref ECSTaskMemory
      NetworkMode: awsvpc
      RequiresCompatibilities:
        - FARGATE

#ContainerDefinitions
      ContainerDefinitions:
        - Name: !Sub "${ProjectName}-${ECSContainerName}"
          Image: !Ref ECSImageName
          Environment:
           - Name: USER_NAME
             Value: !Ref DBMasterName
           - Name: USER_PASS
             Value: !Ref DBPass
           - Name: DB_ENDPOINT
             Value: !Ref DBEndPoint
           - Name: DB_NAME
             Value: !Ref DBNAME                  
          LogConfiguration:
            LogDriver: awslogs
            Options:
              awslogs-group: !Ref ECSLogGroup
              awslogs-region: !Ref "AWS::Region"
              awslogs-stream-prefix: !Ref ProjectName
          MemoryReservation: 128
          PortMappings:
            - HostPort: 80
              Protocol: tcp
              ContainerPort: 80

# ------------------------------------------------------------#
#  ECS Service
# ------------------------------------------------------------#
  ECSService:
    Type: AWS::ECS::Service
    DependsOn: ALBListener
    Properties:
      Cluster: !Ref ECSCluster
      DesiredCount: !Ref ECSTaskDesiredCount
      LaunchType: FARGATE
      LoadBalancers:
        -
          TargetGroupArn: !Ref TargetGroup
          ContainerPort: 80
          ContainerName: !Sub "${ProjectName}-${ECSContainerName}"
      NetworkConfiguration:
       AwsvpcConfiguration:
           AssignPublicIp: ENABLED
           SecurityGroups:
             - !Ref ECSSecurityGroupId
           Subnets:
             - !Ref ECSSubnetId1
             - !Ref ECSSubnetId2
      ServiceName: !Sub "${ProjectName}-${ECSServiceName}"
      TaskDefinition: !Ref ECSTaskDefinition

# ------------------------------------------------------------#
#  Auto Scaling Service
# ------------------------------------------------------------#
  ServiceAutoScalingRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Statement:
          - Effect: Allow
            Principal:
              Service: application-autoscaling.amazonaws.com
            Action: sts:AssumeRole
      Path: /
      Policies:
        - PolicyName: !Sub "${ProjectName}-${ECSContainerName}-autoscaling"
          PolicyDocument:
            Statement:
              - Effect: Allow
                Action:
                  - application-autoscaling:*
                  - cloudwatch:DescribeAlarms
                  - cloudwatch:PutMetricAlarm
                  - ecs:DescribeServices
                  - ecs:UpdateService
                Resource: '*'

  ServiceScalingTarget:
    Type: AWS::ApplicationAutoScaling::ScalableTarget
    Properties:
      MinCapacity: !Ref TaskMinContainerCount
      MaxCapacity: !Ref TaskMaxContainerCount
      ResourceId: !Sub
        - service/${EcsClusterName}/${EcsDefaultServiceName}
        - EcsClusterName: !Ref ECSCluster
          EcsDefaultServiceName: !Sub "${ProjectName}-${ECSServiceName}"
      RoleARN: !GetAtt ServiceAutoScalingRole.Arn
      ScalableDimension: ecs:service:DesiredCount
      ServiceNamespace: ecs
    DependsOn:
      - ECSService
      - ServiceAutoScalingRole

  ServiceScaleOutPolicy:
    Type: AWS::ApplicationAutoScaling::ScalingPolicy
    Properties:
      PolicyName: !Sub "${ProjectName}-${ECSServiceName}-ScaleOutPolicy"
      PolicyType: StepScaling
      ScalingTargetId: !Ref ServiceScalingTarget
      StepScalingPolicyConfiguration:
        AdjustmentType: ChangeInCapacity
        Cooldown: 60
        MetricAggregationType: Average
        StepAdjustments:
          - ScalingAdjustment: 1
            MetricIntervalLowerBound: 0
    DependsOn: ServiceScalingTarget

  ServiceScaleInPolicy:
    Type: AWS::ApplicationAutoScaling::ScalingPolicy
    Properties:
      PolicyName: !Sub "${ProjectName}-${ECSServiceName}-ScaleInPolicy"
      PolicyType: StepScaling
      ScalingTargetId: !Ref ServiceScalingTarget
      StepScalingPolicyConfiguration:
        AdjustmentType: ChangeInCapacity
        Cooldown: 60
        MetricAggregationType: Average
        StepAdjustments:
          - ScalingAdjustment: -1
            MetricIntervalUpperBound: 0
    DependsOn: ServiceScalingTarget

  ServiceScaleOutAlarm:
    Type: AWS::CloudWatch::Alarm
    Properties:
      AlarmName: !Sub "${ProjectName}-${ECSServiceName}-ScaleOutAlarm"
      EvaluationPeriods: !Ref ServiceScaleEvaluationPeriods
      Statistic: Average
      TreatMissingData: notBreaching
      Threshold: !Ref ServiceCpuScaleOutThreshold
      AlarmDescription: Alarm to add capacity if CPU is high
      Period: 60
      AlarmActions:
        - !Ref ServiceScaleOutPolicy
      Namespace: AWS/ECS
      Dimensions:
        - Name: ClusterName
          Value: !Ref ECSCluster
        - Name: ServiceName
          Value: !Sub "${ProjectName}-${ECSServiceName}"
      ComparisonOperator: GreaterThanThreshold
      MetricName: CPUUtilization
    DependsOn:
      - ECSService
      - ServiceScaleOutPolicy

  ServiceScaleInAlarm:
    Type: AWS::CloudWatch::Alarm
    Properties:
      AlarmName: !Sub "${ProjectName}-${ECSServiceName}-ScaleInAlarm"
      EvaluationPeriods: !Ref ServiceScaleEvaluationPeriods
      Statistic: Average
      TreatMissingData: notBreaching
      Threshold: !Ref ServiceCpuScaleInThreshold
      AlarmDescription: Alarm to reduce capacity if container CPU is low
      Period: 300
      AlarmActions:
        - !Ref ServiceScaleInPolicy
      Namespace: AWS/ECS
      Dimensions:
        - Name: ClusterName
          Value: !Ref ECSCluster
        - Name: ServiceName
          Value: !Sub "${ProjectName}-${ECSServiceName}"
      ComparisonOperator: LessThanThreshold
      MetricName: CPUUtilization
    DependsOn:
      - ECSService
      - ServiceScaleInPolicy

パラメータ例

fargate.parameter.json

[
  {
    "ParameterKey": "ProjectName",
    "ParameterValue": "demo-flask"
  },
  {
    "ParameterKey": "VpcId",
    "ParameterValue": "vpc-xxxxxx"
  },
  {
    "ParameterKey": "ALBSecurityGroupId",
    "ParameterValue": "sg-xxxxxx"
  },
  {
    "ParameterKey": "ALBSubnetId1",
    "ParameterValue": "subnet-xxxxxx"
  },
  {
    "ParameterKey": "ALBSubnetId2",
    "ParameterValue": "subnet-yyyyyy"
  },
  {
    "ParameterKey": "ECSSecurityGroupId",
    "ParameterValue": "sg-zzzzzz"
  },
  {
    "ParameterKey": "ECSSubnetId1",
    "ParameterValue": "subnet-zzzzzz"
  },
  {
    "ParameterKey": "ECSSubnetId2",
    "ParameterValue": "subnet-mmmmmm"
  },
  {
    "ParameterKey": "ECSImageName",
    "ParameterValue": "xxxxxx.dkr.ecr.ap-northeast-1.amazonaws.com/YOUR_REPOSITORY_NAME"
  },
  {
    "ParameterKey": "DBMasterName",
    "ParameterValue": "DB_USER_NAME"
  },
  {
    "ParameterKey": "DBPass",
    "ParameterValue": "DB_PASS"
  },
  {
    "ParameterKey": "DBEndPoint",
    "ParameterValue": "RDS_ENDPOINT"
  },
  {
    "ParameterKey": "DBNAME",
    "ParameterValue": "DB_NAME"
  }
]

確認

作成完了後、ALBのDNS名で外部からアクセスすると、以下のような画面が表示されます。

ユーザー登録しましょう

無事登録できたので、ログインします

無事にログインできたので、記事を書きます

記事が表示されました

では、続きはFargate+CodePipelineをCFnで構築してみるです。

参考

CloudformationでFargateを構築する