この記事は公開されてから1年以上経過しています。情報が古い可能性がありますので、ご注意ください。
はじめに
おはようございます、もきゅりんです。
皆さん、クロスパイプラインしてますか? (初めて使いました)
本稿では、クロスアカウントの CodePipeline を構築します。
別のAWSアカウントにある CodeCommit Repository へのブランチプッシュをイベントフックに Fargate Blue/Green デプロイを実行します。
別のAWSアカウントにあるCodeCommit RepositoryをソースとするCodePipelineをCloudFormationで構築してみた | DevelopersIO の ECS / Fargate 版になります。
CodePipelineでアカウントをまたいだパイプラインを作成してみる | DevelopersIO も参考としました。
合せ技になります。
元ネタは 他の CodePipeline アカウントのリソースを使用するパイプラインを AWS に作成する - AWS CodePipeline になります。
これらに付け加えて、本稿では、CodeCommit の任意のブランチプッシュをイベントフックとしたデプロイにしたいので、AWS アカウント間のイベントの送受信 - Amazon EventBridge を参考に EventBridge (CloudWatchEvent) の設定を CloudFormation で行っています。
やりたいこと
下図のような構成を CloudFormation で構築します。
今回は、Fargate のB/G デプロイのパターンのうち、プレースホルダを利用をしたもので実行します。
設定の詳細内容は、上に挙げた2つのブログで記載されているため、ここでは割愛しますが (いろいろと設定が出てくるのですが) 、大まかにやっていることを掴むには、下記 3つを意識すると良いと思います。
- パイプラインを持つアカウント側でカスタマーマネージドの CMK が必要なこと
- 両アカウント間のデプロイを通じて使用されるリソースが、上記のキーを利用できるようにすること
- パイプラインが別アカウントに存在する必要なリソースを利用できるようにすること
両アカウント間のデプロイでどのリソースを使うのか?どのリソースがどのリソースにアクセスするのか?を把握して、上の1~3と見比べると掴みやすいかと思います。
今回の例でまとめると、下表のようになります。
アカウント | リソース | 要件 |
---|---|---|
A | S3 (アーティファクトバケット) | CodePipelineと同じアカウント |
A | CMK | CodePipeline と同じリージョンでカスタマーマネージドキーを作成 |
B | CodeCommit | 別アカウントの S3バケット、そのオブジェクトの暗号化に使うKMSの暗号化キーにアクセスできること |
A | CodeBuild | S3バケット、そのオブジェクトの暗号化に使うKMSの暗号化キーにアクセスできること |
A | CodeDeploy | S3バケット、そのオブジェクトの暗号化に使うKMSの暗号化キーにアクセスできること |
A | CodePipeline | 別アカウントの CodeCommit を利用できること |
B | CloudWatchEvent | 別アカウントへ Event に送信すること |
A | CloudWatchEvent | 別アカウントからの Event を受信できてターゲットを実行する こと |
やること
ここは、別のAWSアカウントにあるCodeCommit RepositoryをソースとするCodePipelineをCloudFormationで構築してみた | DevelopersIO の流れとほぼ同様です。
A,Bそれぞれのサフィックスは実行する環境を示します。
- IAMロールやポリシーの事前準備 (A)
- CodeCommit と CloudWatchEvent の設定 (B)
- ECS / Fargate の Blue / Green デプロイを構築 (A)
- CI/CD Pipeline and Environment と CloudWatchEvent の設定 (A)
前提
- CodeCommitが作成済み (B)
- ECRにリポジトリがあること (A)
- 承認ステージで使用するSNSは作成済みであること(A)
必要なパラメータ
何が必要か分からなってくるので、パラメータをまとめておきました。
取得タイミングはテンプレート実行回数を表します。
(下表は別アカウントのリポジトリを利用するコードパイプラインを構築する準備で必要なパラメータです。B/Gデプロイやパイプライン構築では別途さらにVPCやサブネット、コンテナ名などのパラメータが必要になります。。)
取得タイミング | パラメータ |
---|---|
0 | ProductionAccountId |
0 | RepositoryAccountId |
0 | CodeCommitRepositoryArn |
1 | CodeBuildRole |
1 | CodeDeployRoleArn |
1 | CmkArn |
1 | CodePipelineRoleArn |
1 | S3BucketArn (ArtifactBucketName) |
2 | CodeCommitRoleArn |
1. IAMロールやポリシーの事前準備 (A)
パイプライン側のAWSアカウント(A)でパイプラインが利用する IAMロール、アーティファクトS3バケット、CMKなどを作成します。
リポジトリアカウント(B)に対してアーティファクトに保存する権限、保存する際に暗号化するための権限をポリシーに設定しています。
AWSTemplateFormatVersion: 2010-09-09
Description: Step 1, Pre-requirements (in Production Account)
# ------------------------------------------------------------#
# Parameters
# ------------------------------------------------------------#
Parameters:
RepositoryAccountId:
Description: Repository Account ID
MaxLength: 12
MinLength: 12
Type: String
# ------------------------------------------------------------#
# Resources
# ------------------------------------------------------------#
Resources:
PipelineRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- codepipeline.amazonaws.com
Action:
- sts:AssumeRole
Path: /
Policies:
- PolicyName: Pipeline
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action:
- sts:AssumeRole
Resource:
- !Sub arn:aws:iam::${RepositoryAccountId}:role/* # Cross Account Access
- Effect: Allow
Action:
- iam:PassRole
Resource:
- '*'
Condition:
StringEqualsIfExists:
iam:PassedToService:
- cloudformation.amazonaws.com
- elasticbeanstalk.amazonaws.com
- ec2.amazonaws.com
- ecs-tasks.amazonaws.com
- Effect: Allow
Action:
- codedeploy:CreateDeployment
- codedeploy:GetApplication
- codedeploy:GetApplicationRevision
- codedeploy:GetDeployment
- codedeploy:GetDeploymentConfig
- codedeploy:RegisterApplicationRevision
Resource:
- '*'
- Effect: Allow
Action:
- elasticbeanstalk:*
- ec2:*
- elasticloadbalancing:*
- autoscaling:*
- cloudwatch:*
- s3:*
- sns:*
- cloudformation:*
- rds:*
- sqs:*
- ecs:*
Resource:
- '*'
- Effect: Allow
Action:
- lambda:InvokeFunction
- lambda:ListFunctions
Resource:
- '*'
- Effect: Allow
Action:
- cloudformation:CreateStack
- cloudformation:DeleteStack
- cloudformation:DescribeStacks
- cloudformation:UpdateStack
- cloudformation:CreateChangeSet
- cloudformation:DeleteChangeSet
- cloudformation:DescribeChangeSet
- cloudformation:ExecuteChangeSet
- cloudformation:SetStackPolicy
- cloudformation:ValidateTemplate
Resource:
- '*'
- Effect: Allow
Action:
- codebuild:BatchGetBuilds
- codebuild:StartBuild
Resource:
- '*'
- Effect: Allow
Action:
- cloudformation:ValidateTemplate
Resource:
- '*'
- Effect: Allow
Action:
- ecr:DescribeImages
Resource:
- '*'
BuildRole:
Type: AWS::IAM::Role
Properties:
Path: /
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service: codebuild.amazonaws.com
Action: sts:AssumeRole
Policies:
- PolicyName: BuildPolicy
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Resource: '*'
Action:
- logs:CreateLogGroup
- logs:CreateLogStream
- logs:PutLogEvents
- Effect: Allow
Action:
- s3:PutObject
- s3:GetObject
- s3:GetObjectVersion
- s3:GetBucketAcl
- s3:GetBucketLocation
Resource:
- '*'
- Effect: Allow
Action:
- codebuild:CreateReportGroup
- codebuild:CreateReport
- codebuild:UpdateReport
- codebuild:BatchPutTestCases
- codebuild:BatchPutCodeCoverages
Resource: '*'
- Effect: Allow
Action:
- ecr:GetAuthorizationToken
- ecr:BatchCheckLayerAvailability
- ecr:GetDownloadUrlForLayer
- ecr:GetRepositoryPolicy
- ecr:DescribeRepositories
- ecr:ListImages
- ecr:DescribeImages
- ecr:BatchGetImage
- ecr:InitiateLayerUpload
- ecr:UploadLayerPart
- ecr:CompleteLayerUpload
- ecr:PutImage
Resource: '*'
DeployRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- codedeploy.amazonaws.com
Action:
- sts:AssumeRole
Path: /
ManagedPolicyArns:
- arn:aws:iam::aws:policy/AWSCodeDeployRoleForECS
S3Bucket:
Type: AWS::S3::Bucket
# DeletionPolicy: Retain
Properties:
BucketName: !Join
- '-'
- - artifacts
- !Ref AWS::Region
- !Ref AWS::AccountId
- 'pipeline'
S3ArtifactBucketPolicy:
Type: AWS::S3::BucketPolicy
Properties:
Bucket: !Ref S3Bucket
PolicyDocument:
Statement:
- Action:
- s3:PutObject
Effect: Deny
Principal: '*'
Resource:
- !Sub arn:aws:s3:::${S3Bucket}/*
Condition:
StringNotEquals:
s3:x-amz-server-side-encryption: aws:kms
- Action:
- s3:*
Effect: Deny
Principal: '*'
Resource:
- !Sub arn:aws:s3:::${S3Bucket}/*
Condition:
Bool:
aws:SecureTransport: false
- Action:
- s3:Get*
- s3:Put*
Effect: Allow
Principal:
AWS:
- !Sub arn:aws:iam::${RepositoryAccountId}:root
Resource:
- !Sub arn:aws:s3:::${S3Bucket}/*
- Action:
- s3:ListBucket
Effect: Allow
Principal:
AWS:
- !Sub arn:aws:iam::${RepositoryAccountId}:root
Resource:
- !Sub arn:aws:s3:::${S3Bucket}
Version: 2012-10-17
Key:
Type: AWS::KMS::Key
Properties:
Description: An example symmetric CMK
KeyPolicy:
Version: 2012-10-17
Id: key-default-1
Statement:
- Sid: Enable IAM User Permissions
Effect: Allow
Principal:
AWS:
- !Sub arn:aws:iam::${AWS::AccountId}:root
Action: kms:*
Resource: '*'
- Sid: Allow administration of the key
Effect: Allow
Principal:
AWS:
- !Sub arn:aws:iam::${AWS::AccountId}:user/USERNAME
Action:
- kms:Create*
- kms:Describe*
- kms:Enable*
- kms:List*
- kms:Put*
- kms:Update*
- kms:Revoke*
- kms:Disable*
- kms:Get*
- kms:Delete*
- kms:ScheduleKeyDeletion
- kms:CancelKeyDeletion
Resource: '*'
- Sid: Allow use of the key
Effect: Allow
Principal:
AWS:
- !Sub arn:aws:iam::${RepositoryAccountId}:root
- !GetAtt PipelineRole.Arn
- !GetAtt BuildRole.Arn
- !GetAtt DeployRole.Arn
Action:
- kms:DescribeKey
- kms:Encrypt
- kms:Decrypt
- kms:ReEncrypt*
- kms:GenerateDataKey
- kms:GenerateDataKeyWithoutPlaintext
Resource: '*'
- Sid: Allow attachment of persistent resources
Effect: Allow
Principal:
AWS:
- !Sub arn:aws:iam::${RepositoryAccountId}:root
- !GetAtt PipelineRole.Arn
- !GetAtt BuildRole.Arn
- !GetAtt DeployRole.Arn
Action:
- kms:CreateGrant
- kms:ListGrants
- kms:RevokeGrant
Resource: '*'
Condition:
Bool:
kms:GrantIsForAWSResource: true
Alias:
Type: AWS::KMS::Alias
Properties:
AliasName: alias/CodePipelineArtifact
TargetKeyId: !Ref Key
# ------------------------------------------------------------#
# Outputs
# ------------------------------------------------------------#
Outputs:
BuildRole:
Value: !GetAtt BuildRole.Arn
Description: BuildRole Arn
Export:
Name: BuildRoleArn
DeployRole:
Value: !GetAtt DeployRole.Arn
Description: DeployRole Arn
Export:
Name: DeployRoleArn
PipelineRole:
Value: !GetAtt PipelineRole.Arn
Description: PipelineRole Role Arn
Export:
Name: PipelineRoleArn
Key:
Value: !GetAtt Key.Arn
Description: CMK Role Arn
Export:
Name: CMKRoleArn
S3Bucket:
Value: !GetAtt S3Bucket.Arn
Description: S3Bucket Arn
Export:
Name: S3BucketArn
完了したら、パラメータに使用するため、スタックの出力値を取得しましょう。
2. CodeCommit と CloudWatchEvent の設定 (B)
パイプライン側のAWSアカウント(A) による AssumeRole を許可する権限を追加します。
パイプライン(A)の Source Stage の Action Role にアーティファクトへ保存する権限、アーティファクトを暗号化/復号する権限を追加しています。
CloudWatchEvent によって、パイプライン側のAWSアカウント(A) のイベントバスに送信する設定をしています。
AWSTemplateFormatVersion: 2010-09-09
Description: Step 2, CodeCommit (in Repository Account)
# ------------------------------------------------------------#
# Parameters
# ------------------------------------------------------------#
Parameters:
NameTagPrefix:
Type: String
Default: system
Description: Prefix of Name tags.
ProductionAccountId:
Description: Production Account ID
MaxLength: 12
MinLength: 12
Type: String
S3BucketArn:
Description: Production Account S3 Bucket ARN for Artifact (Created by 01-requirement.yml)
Type: String
CmkArn:
Description: Production Account CMK ARN (Created by 01-requirement.yml)
Type: String
CodeCommitRepositoryArn:
Description: CodeCommit Repository Arn
Type: String
CodeCommitRepositoryName:
Description: Repository Account CodeCommit Repository Name (Created by 02-codecommit.yml)
Type: String
BranchName:
Type: String
Default: master
# ------------------------------------------------------------#
# Resources
# ------------------------------------------------------------#
Resources:
# CodeWatchEventを実行できるIAMRole
CloudwatchEventRole:
Type: AWS::IAM::Role
Properties:
RoleName: !Sub ${NameTagPrefix}-CloudWatchEventRole
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- events.amazonaws.com
Action: sts:AssumeRole
Path: /
Policies:
- PolicyName: CloudWatchEventsBus
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action: events:PutEvents
Resource: !Sub arn:aws:events:${AWS::Region}:${ProductionAccountId}:event-bus/default
# CloudWatchEventの実行ルール
AmazonCloudWatchEventRule:
Type: AWS::Events::Rule
Properties:
EventPattern:
source:
- aws.codecommit
detail-type:
- CodeCommit Repository State Change
resources:
- Fn::Join:
- ''
- - 'arn:aws:codecommit:'
- !Ref 'AWS::Region'
- ':'
- !Ref 'AWS::AccountId'
- ':'
- !Ref CodeCommitRepositoryName
detail:
event:
- referenceCreated
- referenceUpdated
referenceType:
- branch
referenceName:
- !Ref BranchName
Targets:
- Arn: !Sub arn:aws:events:${AWS::Region}:${ProductionAccountId}:event-bus/default
RoleArn: !GetAtt CloudwatchEventRole.Arn
Id: codepipeline-AppPipeline
Role:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
AWS:
- !Sub arn:aws:iam::${ProductionAccountId}:root
Action:
- sts:AssumeRole
Path: /
Policies:
- PolicyName: source
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action:
- s3:PutObject
- s3:PutObjectAcl
Resource:
- !Sub ${S3BucketArn}/*
- Effect: Allow
Action:
- kms:DescribeKey
- kms:GenerateDataKey*
- kms:Encrypt
- kms:ReEncrypt*
- kms:Decrypt
Resource:
- !Ref CmkArn
- Effect: Allow
Action:
- codecommit:GetBranch
- codecommit:GetCommit
- codecommit:UploadArchive
- codecommit:GetUploadArchiveStatus
- codecommit:CancelUploadArchive
Resource:
- !Ref CodeCommitRepositoryArn
# ------------------------------------------------------------#
# Outputs
# ------------------------------------------------------------#
Outputs:
Role:
Value: !GetAtt Role.Arn
Description: CodeCommitRole Arn
Export:
Name: CodeCommitRoleArn
完了したら、パラメータに使用するため、CodeCommitRole Arn
の出力値を取得しましょう。
3. ECS / Fargate の Blue / Green デプロイを構築 (A)
ここでは、Fargate のBlue / Green デプロイを1発で構築します。
詳細は、CloudFormation 一撃で Fargate の Blue/Green Deployment 環境を構築する | DevelopersIO を参照下さい。
AWSTemplateFormatVersion: '2010-09-09'
Description: Step 3, Fargate Blue/Green Deployment (in Production Account).
Metadata:
'AWS::CloudFormation::Interface':
ParameterGroups:
- Label:
default: 'Project Name Prefix'
Parameters:
- ProjectName
- Label:
default: 'InternetALB Configuration'
Parameters:
- InternetALBName
- TargetGroupName1
- TargetGroupName2
- Label:
default: 'Fargate for ECS Configuration'
Parameters:
- ECSClusterName
- ECSTaskName
- ECSTaskCPUUnit
- ECSTaskMemory
- ECSContainerName
- ECSImageName
- ECSServiceName
- ECSTaskDesiredCount
- Label:
default: 'Netowork Configuration'
Parameters:
- VpcId
- ALBSecurityGroupId
- ALBSubnetId1
- ALBSubnetId2
- ECSSecurityGroupId
- ECSSubnetId1
- ECSSubnetId2
- Label:
default: 'Scaling Configuration'
Parameters:
- ServiceScaleEvaluationPeriods
- ServiceCpuScaleOutThreshold
- ServiceCpuScaleInThreshold
- TaskMinContainerCount
- TaskMaxContainerCount
- Label:
default: 'CodeDeploy Configuration'
Parameters:
- CodeDeployAppName
- CodeDeployDeploymentGroupName
ParameterLabels:
InternetALBName:
default: 'InternetALBName'
TargetGroupName1:
default: 'TargetGroupName1'
TargetGroupName2:
default: 'TargetGroupName2'
ECSClusterName:
default: 'ECSClusterName'
ECSTaskName:
default: 'ECSTaskName'
ECSTaskCPUUnit:
default: 'ECSTaskCPUUnit'
ECSTaskMemory:
default: 'ECSTaskMemory'
ECSContainerName:
default: 'ECSContainerName'
ECSImageName:
default: 'ECSImageName'
ECSServiceName:
default: 'ECSServiceName'
ECSTaskDesiredCount:
default: 'ECSTaskDesiredCount'
CodeDeployAppName:
default: 'CodeDeployAppName'
CodeDeployDeploymentGroupName:
default: 'CodeDeployDeploymentGroupName'
# ------------------------------------------------------------#
# Input Parameters
# ------------------------------------------------------------#
Parameters:
NameTagPrefix:
Type: String
Default: system
Description: Prefix of Name tags.
ProjectName:
Default: sample-fargate
Type: String
#VPCID
VpcId:
Description: 'VPC ID'
Type: AWS::EC2::VPC::Id
#ALBSecurity Group
ALBSecurityGroupId:
Type: AWS::EC2::SecurityGroup::Id
#ALBSubnet1
ALBSubnetId1:
Description: 'ALB Subnet 1st'
Type: AWS::EC2::Subnet::Id
#ALBSubnet2
ALBSubnetId2:
Description: 'ALB Subnet 2st'
Type: AWS::EC2::Subnet::Id
#ECSSecurity Group
ECSSecurityGroupId:
Type: AWS::EC2::SecurityGroup::Id
#ECSSubnet1
ECSSubnetId1:
Description: 'ECS Subnet 1st'
Type: AWS::EC2::Subnet::Id
#ECSSubnet2
ECSSubnetId2:
Description: 'ECS Subnet 2st'
Type: AWS::EC2::Subnet::Id
#InternetALB
InternetALBName:
Type: String
Default: 'alb'
#TargetGroupName1
TargetGroupName1:
Type: String
Default: 'tg1'
#TargetGroupName2
TargetGroupName2:
Type: String
Default: 'tg2'
#ECSClusterName
ECSClusterName:
Type: String
Default: 'cluster'
#ECSTaskName
ECSTaskName:
Type: String
Default: 'task'
#ECSTaskCPUUnit
ECSTaskCPUUnit:
AllowedValues: [256, 512, 1024, 2048, 4096]
Type: String
Default: 256
#ECSTaskMemory
ECSTaskMemory:
AllowedValues: [256, 512, 1024, 2048, 4096]
Type: String
Default: 512
#ECSContainerName
ECSContainerName:
Type: String
Default: 'container'
#ECSImageName
ECSImageName:
Type: String
Default: 'nginxdemos/hello:latest'
#ECSServiceName
ECSServiceName:
Type: String
Default: 'service'
CodeDeployAppName:
Type: String
Default: 'app'
CodeDeployDeploymentGroupName:
Type: String
Default: 'dg'
#ECSTaskDesiredCount
ECSTaskDesiredCount:
Type: Number
Default: 1
# Scaling params
ServiceScaleEvaluationPeriods:
Description: The number of periods over which data is compared to the specified threshold
Type: Number
Default: 2
MinValue: 2
ServiceCpuScaleOutThreshold:
Type: Number
Description: Average CPU value to trigger auto scaling out
Default: 50
MinValue: 0
MaxValue: 100
ConstraintDescription: Value must be between 0 and 100
ServiceCpuScaleInThreshold:
Type: Number
Description: Average CPU value to trigger auto scaling in
Default: 25
MinValue: 0
MaxValue: 100
ConstraintDescription: Value must be between 0 and 100
TaskMinContainerCount:
Type: Number
Description: Minimum number of containers to run for the service
Default: 1
MinValue: 1
ConstraintDescription: Value must be at least one
TaskMaxContainerCount:
Type: Number
Description: Maximum number of containers to run for the service when auto scaling out
Default: 2
MinValue: 1
ConstraintDescription: Value must be at least one
CodeDeployRoleArn:
Description: Production Account CodeDeploy Service Role ARN (Created by 01-requirement.yml)
Type: String
Resources:
# ------------------------------------------------------------#
# Target Group
# ------------------------------------------------------------#
TargetGroup1:
Type: 'AWS::ElasticLoadBalancingV2::TargetGroup'
Properties:
VpcId: !Ref VpcId
Name: !Sub '${NameTagPrefix}-${ProjectName}-${TargetGroupName1}'
Protocol: HTTP
HealthCheckPath: '/'
Port: 80
TargetType: ip
TargetGroup2:
Type: 'AWS::ElasticLoadBalancingV2::TargetGroup'
Properties:
VpcId: !Ref VpcId
Name: !Sub '${NameTagPrefix}-${ProjectName}-${TargetGroupName2}'
Protocol: HTTP
HealthCheckPath: '/'
Port: 8080
TargetType: ip
# ------------------------------------------------------------#
# Internet ALB
# ------------------------------------------------------------#
InternetALB:
Type: 'AWS::ElasticLoadBalancingV2::LoadBalancer'
Properties:
Name: !Sub '${NameTagPrefix}-${ProjectName}-${InternetALBName}'
Tags:
- Key: Name
Value: !Sub '${NameTagPrefix}-${ProjectName}-${InternetALBName}'
Scheme: 'internet-facing'
LoadBalancerAttributes:
- Key: 'deletion_protection.enabled'
Value: 'false'
- Key: 'idle_timeout.timeout_seconds'
Value: '60'
- Key: 'access_logs.s3.enabled'
Value: 'true'
- Key: 'access_logs.s3.bucket'
Value: !Sub 'alb-log-${AWS::AccountId}'
SecurityGroups:
- !Ref ALBSecurityGroupId
Subnets:
- !Ref ALBSubnetId1
- !Ref ALBSubnetId2
ALBListener1:
Type: 'AWS::ElasticLoadBalancingV2::Listener'
Properties:
DefaultActions:
- TargetGroupArn: !Ref TargetGroup1
Type: forward
LoadBalancerArn: !Ref InternetALB
Port: 80
Protocol: HTTP
ALBListener2:
Type: 'AWS::ElasticLoadBalancingV2::Listener'
Properties:
DefaultActions:
- TargetGroupArn: !Ref TargetGroup2
Type: forward
LoadBalancerArn: !Ref InternetALB
Port: 8080
Protocol: HTTP
# ------------------------------------------------------------#
# ECS Cluster
# ------------------------------------------------------------#
ECSCluster:
Type: 'AWS::ECS::Cluster'
Properties:
ClusterName: !Sub '${NameTagPrefix}-${ProjectName}-${ECSClusterName}'
# ------------------------------------------------------------#
# ECS LogGroup
# ------------------------------------------------------------#
ECSLogGroup:
Type: 'AWS::Logs::LogGroup'
Properties:
LogGroupName: !Sub '/ecs/logs/${NameTagPrefix}-${ProjectName}-ecs-group'
# ------------------------------------------------------------#
# ECS Task Execution Role
# ------------------------------------------------------------#
ECSTaskExecutionRole:
Type: AWS::IAM::Role
Properties:
RoleName: !Sub '${NameTagPrefix}-${ProjectName}-ECSTaskExecutionRolePolicy'
Path: /
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service: ecs-tasks.amazonaws.com
Action: sts:AssumeRole
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy
# ------------------------------------------------------------#
# ECS TaskDefinition
# ------------------------------------------------------------#
ECSTaskDefinition:
Type: 'AWS::ECS::TaskDefinition'
Properties:
Cpu: !Ref ECSTaskCPUUnit
ExecutionRoleArn: !GetAtt ECSTaskExecutionRole.Arn
Family: !Sub '${NameTagPrefix}-${ProjectName}-${ECSTaskName}'
Memory: !Ref ECSTaskMemory
NetworkMode: awsvpc
RequiresCompatibilities:
- FARGATE
#ContainerDefinitions
ContainerDefinitions:
- Name: !Sub '${NameTagPrefix}-${ProjectName}-${ECSContainerName}'
Image: !Ref ECSImageName
LogConfiguration:
LogDriver: awslogs
Options:
awslogs-group: !Ref ECSLogGroup
awslogs-region: !Ref 'AWS::Region'
awslogs-stream-prefix: !Ref ProjectName
MemoryReservation: 128
PortMappings:
- HostPort: 80
Protocol: tcp
ContainerPort: 80
# ------------------------------------------------------------#
# ECS Service
# ------------------------------------------------------------#
ECSService:
Type: AWS::ECS::Service
DependsOn: ALBListener1
Properties:
Cluster: !Ref ECSCluster
DesiredCount: !Ref ECSTaskDesiredCount
DeploymentController:
Type: CODE_DEPLOY
LaunchType: FARGATE
LoadBalancers:
- TargetGroupArn: !Ref TargetGroup1
ContainerPort: 80
ContainerName: !Sub '${NameTagPrefix}-${ProjectName}-${ECSContainerName}'
NetworkConfiguration:
AwsvpcConfiguration:
AssignPublicIp: ENABLED
SecurityGroups:
- !Ref ECSSecurityGroupId
Subnets:
- !Ref ECSSubnetId1
- !Ref ECSSubnetId2
ServiceName: !Sub '${NameTagPrefix}-${ProjectName}-${ECSServiceName}'
TaskDefinition: !Ref ECSTaskDefinition
# ------------------------------------------------------------#
# Auto Scaling Service
# ------------------------------------------------------------#
ServiceAutoScalingRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service: application-autoscaling.amazonaws.com
Action: sts:AssumeRole
Path: /
Policies:
- PolicyName: !Sub '${NameTagPrefix}-${ProjectName}-${ECSContainerName}-autoscaling'
PolicyDocument:
Statement:
- Effect: Allow
Action:
- application-autoscaling:*
- cloudwatch:DescribeAlarms
- cloudwatch:PutMetricAlarm
- ecs:DescribeServices
- ecs:UpdateService
Resource: '*'
ServiceScalingTarget:
Type: AWS::ApplicationAutoScaling::ScalableTarget
Properties:
MinCapacity: !Ref TaskMinContainerCount
MaxCapacity: !Ref TaskMaxContainerCount
ResourceId: !Sub
- service/${EcsClusterName}/${EcsDefaultServiceName}
- EcsClusterName: !Ref ECSCluster
EcsDefaultServiceName: !Sub '${NameTagPrefix}-${ProjectName}-${ECSServiceName}'
RoleARN: !GetAtt ServiceAutoScalingRole.Arn
ScalableDimension: ecs:service:DesiredCount
ServiceNamespace: ecs
DependsOn:
- ECSService
ServiceScaleOutPolicy:
Type: AWS::ApplicationAutoScaling::ScalingPolicy
Properties:
PolicyName: !Sub '${NameTagPrefix}-${ProjectName}-${ECSServiceName}-ScaleOutPolicy'
PolicyType: StepScaling
ScalingTargetId: !Ref ServiceScalingTarget
StepScalingPolicyConfiguration:
AdjustmentType: ChangeInCapacity
Cooldown: 60
MetricAggregationType: Average
StepAdjustments:
- ScalingAdjustment: 1
MetricIntervalLowerBound: 0
ServiceScaleInPolicy:
Type: AWS::ApplicationAutoScaling::ScalingPolicy
Properties:
PolicyName: !Sub '${NameTagPrefix}-${ProjectName}-${ECSServiceName}-ScaleInPolicy'
PolicyType: StepScaling
ScalingTargetId: !Ref ServiceScalingTarget
StepScalingPolicyConfiguration:
AdjustmentType: ChangeInCapacity
Cooldown: 60
MetricAggregationType: Average
StepAdjustments:
- ScalingAdjustment: -1
MetricIntervalUpperBound: 0
ServiceScaleOutAlarm:
Type: AWS::CloudWatch::Alarm
Properties:
AlarmName: !Sub '${NameTagPrefix}-${ProjectName}-${ECSServiceName}-ScaleOutAlarm'
EvaluationPeriods: !Ref ServiceScaleEvaluationPeriods
Statistic: Average
TreatMissingData: notBreaching
Threshold: !Ref ServiceCpuScaleOutThreshold
AlarmDescription: Alarm to add capacity if CPU is high
Period: 60
AlarmActions:
- !Ref ServiceScaleOutPolicy
Namespace: AWS/ECS
Dimensions:
- Name: ClusterName
Value: !Ref ECSCluster
- Name: ServiceName
Value: !Sub '${NameTagPrefix}-${ProjectName}-${ECSServiceName}'
ComparisonOperator: GreaterThanThreshold
MetricName: CPUUtilization
DependsOn:
- ECSService
ServiceScaleInAlarm:
Type: AWS::CloudWatch::Alarm
Properties:
AlarmName: !Sub '${NameTagPrefix}-${ProjectName}-${ECSServiceName}-ScaleInAlarm'
EvaluationPeriods: !Ref ServiceScaleEvaluationPeriods
Statistic: Average
TreatMissingData: notBreaching
Threshold: !Ref ServiceCpuScaleInThreshold
AlarmDescription: Alarm to reduce capacity if container CPU is low
Period: 300
AlarmActions:
- !Ref ServiceScaleInPolicy
Namespace: AWS/ECS
Dimensions:
- Name: ClusterName
Value: !Ref ECSCluster
- Name: ServiceName
Value: !Sub '${NameTagPrefix}-${ProjectName}-${ECSServiceName}'
ComparisonOperator: LessThanThreshold
MetricName: CPUUtilization
DependsOn:
- ECSService
# ------------------------------------------------------------#
# Lambda
# ------------------------------------------------------------#
LambdaFunction:
Type: 'AWS::Lambda::Function'
DeletionPolicy: 'Delete'
Properties:
Code:
ZipFile: |
import boto3
import json
import logging
import cfnresponse
from botocore.exceptions import ClientError
logger = logging.getLogger()
logger.setLevel(logging.INFO)
client = boto3.client('codedeploy')
def lambda_handler(event, context):
appName = event['ResourceProperties']['appName']
deploymentGroup = event['ResourceProperties']['deploymentGroup']
clusterName = event['ResourceProperties']['ECSClusterName']
serviceName = event['ResourceProperties']['ECSServiceName']
print('REQUEST RECEIVED:\n' + json.dumps(event))
responseData = {}
try:
res = client.create_application(
applicationName=appName,
computePlatform='ECS'
)
logger.info(res)
logger.info("SUCCESS: CodeDeploy Application created.")
res = client.create_deployment_group(
applicationName=appName,
deploymentGroupName=deploymentGroup,
deploymentConfigName='CodeDeployDefault.ECSLinear10PercentEvery1Minutes',
serviceRoleArn=event['ResourceProperties']['CodeDeployServiceRoleArn'],
autoRollbackConfiguration={
'enabled': True,
'events': [
'DEPLOYMENT_FAILURE',
]
},
deploymentStyle={
'deploymentType': 'BLUE_GREEN',
'deploymentOption': 'WITH_TRAFFIC_CONTROL'
},
blueGreenDeploymentConfiguration={
'terminateBlueInstancesOnDeploymentSuccess': {
'action': 'TERMINATE',
'terminationWaitTimeInMinutes': 30
},
'deploymentReadyOption': {
'actionOnTimeout': 'STOP_DEPLOYMENT',
'waitTimeInMinutes': 5
}
},
loadBalancerInfo={
'targetGroupPairInfoList': [
{
'targetGroups': [
{
'name': event['ResourceProperties']['TargetGroup1']
},
{
'name': event['ResourceProperties']['TargetGroup2']
},
],
'prodTrafficRoute': {
'listenerArns': [
event['ResourceProperties']['ALBListener1'],
]
},
'testTrafficRoute': {
'listenerArns': [
event['ResourceProperties']['ALBListener2'],
]
}
},
]
},
ecsServices=[
{
'serviceName': serviceName,
'clusterName': clusterName
},
]
)
except ClientError as e:
logger.error("ERROR: Something error!")
logger.error(e)
responseData = {'error': str(e)}
cfnresponse.send(event, context, cfnresponse.FAILED, responseData)
else:
logger.info(res)
logger.info(
"SUCCESS: CodeDeploy Application and DeploymentGroup created.")
return cfnresponse.send(event, context, cfnresponse.SUCCESS, responseData)
Handler: index.lambda_handler
Role: !GetAtt LambdaRole.Arn
Runtime: python3.8
Timeout: 10
# ------------------------------------------------------------#
# Custom Resource
# ------------------------------------------------------------#
CreateCodeDeploy:
Type: Custom::CreateCodeDeploy
DependsOn:
- ECSService
Properties:
ServiceToken: !GetAtt LambdaFunction.Arn
Region: !Ref AWS::Region
ECSClusterName: !Sub '${NameTagPrefix}-${ProjectName}-${ECSClusterName}'
ECSServiceName: !Sub '${NameTagPrefix}-${ProjectName}-${ECSServiceName}'
CodeDeployServiceRoleArn: !Ref CodeDeployRoleArn
TargetGroup1: !Sub '${NameTagPrefix}-${ProjectName}-${TargetGroupName1}'
TargetGroup2: !Sub '${NameTagPrefix}-${ProjectName}-${TargetGroupName2}'
ALBListener1: !Ref ALBListener1
ALBListener2: !Ref ALBListener2
appName: !Sub '${NameTagPrefix}-${ProjectName}-${ECSClusterName}-${ECSServiceName}-${CodeDeployAppName}'
deploymentGroup: !Sub '${NameTagPrefix}-${ProjectName}-${ECSClusterName}-${ECSServiceName}-${CodeDeployDeploymentGroupName}'
# ------------------------------------------------------------#
# IAMRole For CustomResource Lambda
# ------------------------------------------------------------#
LambdaRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service:
- lambda.amazonaws.com
Action:
- sts:AssumeRole
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
- arn:aws:iam::aws:policy/AWSCodeDeployFullAccess
LambdaPolicy:
Type: AWS::IAM::Policy
Properties:
PolicyName: LambdaPolicy
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- ec2:*
- logs:*
Resource: '*'
- Effect: Allow
Resource: '*'
Action:
- iam:PassRole
Condition:
StringEqualsIfExists:
iam:PassedToService:
- codedeploy.amazonaws.com
Roles:
- !Ref LambdaRole
# ------------------------------------------------------------#
# Log Bucket
# ------------------------------------------------------------#
logsBacket:
Type: AWS::S3::Bucket
Properties:
BucketName: !Sub 'alb-log-${AWS::AccountId}'
AccessControl: LogDeliveryWrite
BucketEncryption:
ServerSideEncryptionConfiguration:
- ServerSideEncryptionByDefault:
SSEAlgorithm: 'AES256'
PublicAccessBlockConfiguration:
BlockPublicAcls: TRUE
BlockPublicPolicy: TRUE
IgnorePublicAcls: TRUE
RestrictPublicBuckets: TRUE
LifecycleConfiguration:
Rules:
- Id: 'Delete-After-400days'
Status: Enabled
ExpirationInDays: 400
logsBucketPolicy:
Type: AWS::S3::BucketPolicy
DependsOn: logsBacket
Properties:
Bucket: !Sub 'alb-log-${AWS::AccountId}'
PolicyDocument:
Statement:
- Action:
- 's3:PutObject'
Effect: 'Allow'
Resource:
- Fn::Join:
- ''
- - 'arn:aws:s3:::'
- !Sub 'alb-log-${AWS::AccountId}'
- '/*'
Principal:
AWS: '582318560864'
- Action:
- 's3:PutObject'
Effect: 'Allow'
Resource:
- Fn::Join:
- ''
- - 'arn:aws:s3:::'
- !Sub 'alb-log-${AWS::AccountId}'
- '/*'
Principal:
Service: 'delivery.logs.amazonaws.com'
Condition:
StringEquals:
's3:x-amz-acl':
- 'bucket-owner-full-control'
- Action:
- 's3:GetBucketAcl'
Effect: 'Allow'
Resource:
Fn::Join:
- ''
- - 'arn:aws:s3:::'
- !Sub 'alb-log-${AWS::AccountId}'
Principal:
Service: 'delivery.logs.amazonaws.com'
CodeCommit にプッシュする appspec.yaml
, taskdef.json
のコンテナ名やタスクロール定義なども構築したリソース名と合わせるように気を付けます。
4. CI/CD Pipeline and Environment と CloudWatchEvent の設定 (A)
CodePipelineは、[2パターン] CFn で Fargate の Blue/Green Deployment の CodePipeline を構築する | DevelopersIO のプレースホルダ版 Blue/Green デプロイを参照して下さい。
加えて、リポジトリ側からのイベント受信してパイプラインを実行する CloudWatchEvent を設定しています。
AWSTemplateFormatVersion: 2010-09-09
Description: Step 4, Environment and Pipeline (in Production Account)
# ------------------------------------------------------------#
# Parameters
# ------------------------------------------------------------#
Parameters:
NameTagPrefix:
Type: String
Default: system
Description: Prefix of Name tags.
RepositoryAccountId:
Type: String
Description: Prefix of RepositoryAccountId tag.
ServiceName:
Type: String
Default: myapp
Description: Prefix of Service tags.
CodeDeployAppName:
Type: String
CodeDeployDGName:
Type: String
ContainerName:
Type: String
ECRName:
Type: String
CodeCommitRepositoryName:
Description: Repository Account CodeCommit Repository Name (Created by 02-codecommit.yml)
Type: String
BranchName:
Type: String
Default: master
SnsTopicName:
Type: String
Default: deploytopic
ArtifactBucketName:
Description: Production Account S3 Bucket Name for Artifact (Created by 01-requirement.yml)
Type: String
CmkArn:
Description: Production Account CMK ARN (Created by 01-requirement.yml)
Type: String
CodePipelineRoleArn:
Description: Production Account CodePipeline Service Role ARN (Created by 01-requirement.yml)
Type: String
CodeCommitRoleArn:
Description: Repository Account CodeCommit Action Role ARN (Created by 02-codecommit.yml)
Type: String
CodeBuildRoleArn:
Description: Repository Account CodeBuild Action Role ARN (Created by 02-codecommit.yml)
Type: String
# ------------------------------------------------------------#
# Parameters
# ------------------------------------------------------------#
Resources:
# EventBus
SampleEventBusPolicy:
Type: AWS::Events::EventBusPolicy
Properties:
StatementId: MyStatement
Statement:
Effect: Allow
Principal:
AWS: !Sub arn:aws:iam::${RepositoryAccountId}:root
Action: events:PutEvents
Resource: !Sub arn:aws:events:${AWS::Region}:${AWS::AccountId}:event-bus/default
# CodeWatchEventを実行できるIAMRole
CloudwatchEventRole:
Type: AWS::IAM::Role
Properties:
RoleName: !Sub ${NameTagPrefix}-${ServiceName}-CloudWatchEventRole
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- events.amazonaws.com
Action: sts:AssumeRole
Path: /
Policies:
- PolicyName: CloudWatchEventsPipelineExecution
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action: codepipeline:StartPipelineExecution
Resource: !Sub arn:aws:codepipeline:${AWS::Region}:${AWS::AccountId}:${Pipeline}
# CloudWatchEventの実行ルール
AmazonCloudWatchEventRule:
Type: AWS::Events::Rule
Properties:
EventPattern:
source:
- aws.codecommit
detail-type:
- CodeCommit Repository State Change
resources:
- Fn::Join:
- ''
- - 'arn:aws:codecommit:'
- !Ref 'AWS::Region'
- ':'
- !Ref RepositoryAccountId
- ':'
- !Ref CodeCommitRepositoryName
detail:
event:
- referenceCreated
- referenceUpdated
referenceType:
- branch
referenceName:
- !Ref BranchName
Targets:
- Arn: !Sub arn:aws:codepipeline:${AWS::Region}:${AWS::AccountId}:${Pipeline}
RoleArn: !GetAtt CloudwatchEventRole.Arn
Id: codepipeline-AppPipeline
# CodeBuild
CodeBuildProject:
Type: AWS::CodeBuild::Project
Properties:
ServiceRole: !Ref CodeBuildRoleArn
Artifacts:
Type: CODEPIPELINE
Source:
Type: CODEPIPELINE
BuildSpec: |
version: 0.2
phases:
pre_build:
commands:
- echo Logging in to Amazon ECR...
- $(aws ecr get-login --no-include-email)
- IMAGE_TAG=$(echo $CODEBUILD_RESOLVED_SOURCE_VERSION | cut -c 1-7)
build:
commands:
- echo Build started on `date`
- echo Building the Docker image...
- docker build -t $REPOSITORY_URI:$IMAGE_TAG .
- docker tag $REPOSITORY_URI:$IMAGE_TAG $REPOSITORY_URI:$IMAGE_TAG
post_build:
commands:
- echo Build completed on `date`
- echo Pushing the Docker images...
- docker push $REPOSITORY_URI:$IMAGE_TAG
- echo Writing imageDetail json...
- echo "{\"name\":\"${ContainerName}\",\"ImageURI\":\"${REPOSITORY_URI}:${IMAGE_TAG}\"}" > imageDetail.json
artifacts:
files: imageDetail.json
Environment:
PrivilegedMode: true
ComputeType: BUILD_GENERAL1_SMALL
Image: aws/codebuild/standard:4.0
Type: LINUX_CONTAINER
EnvironmentVariables:
- Name: AWS_DEFAULT_REGION
Value: !Ref AWS::Region
- Name: REPOSITORY_URI
Value: !Sub ${AWS::AccountId}.dkr.ecr.${AWS::Region}.amazonaws.com/${ECRName}
- Name: ContainerName
Value: !Ref ContainerName
- Name: DOCKER_BUILDKIT
Value: '1'
Name: !Ref AWS::StackName
# ------------------------------------------------------------#
# CodePipeline
# ------------------------------------------------------------#
Pipeline:
Type: AWS::CodePipeline::Pipeline
Properties:
RoleArn: !Ref CodePipelineRoleArn
Name: !Sub ${NameTagPrefix}-${ServiceName}-pipeline
ArtifactStore:
Type: S3
Location: !Ref ArtifactBucketName
EncryptionKey:
Id: !Ref CmkArn
Type: KMS
Stages:
- Name: Source
Actions:
- Name: SourceAction
ActionTypeId:
Category: Source
Owner: AWS
Version: '1'
Provider: CodeCommit
Configuration:
RepositoryName: !Ref CodeCommitRepositoryName
PollForSourceChanges: false
BranchName: !Ref BranchName
RunOrder: 1
OutputArtifacts:
- Name: App
RoleArn: !Ref CodeCommitRoleArn
- Name: Build
Actions:
- Name: Build
ActionTypeId:
Category: Build
Owner: AWS
Version: '1'
Provider: CodeBuild
Configuration:
ProjectName: !Ref CodeBuildProject
RunOrder: 1
InputArtifacts:
- Name: App
OutputArtifacts:
- Name: BuildOutput
- Name: Approval
Actions:
- Name: Manual_Approval
ActionTypeId:
Category: Approval
Owner: AWS
Version: '1'
Provider: Manual
Configuration:
CustomData: !Sub '${ServiceName} will be updated. Do you want to deploy it?'
NotificationArn: arn:aws:sns:ap-northeast-1:{AWS::AccountId}:${SnsTopicName}
- Name: Deploy
Actions:
- Name: Deploy
ActionTypeId:
Category: Deploy
Owner: AWS
Version: '1'
Provider: CodeDeployToECS
Configuration:
AppSpecTemplateArtifact: App
AppSpecTemplatePath: appspec.yaml
TaskDefinitionTemplateArtifact: App
TaskDefinitionTemplatePath: taskdef.json
ApplicationName: !Ref CodeDeployAppName
DeploymentGroupName: !Ref CodeDeployDGName
Image1ArtifactName: BuildOutput
Image1ContainerName: IMAGE1_NAME
RunOrder: 1
InputArtifacts:
- Name: App
- Name: BuildOutput
Region: !Ref AWS::Region
# ------------------------------------------------------------#
# Outputs
# ------------------------------------------------------------#
Outputs:
PipelinelogicalID:
Description: logical ID.
Value: !Ref Pipeline
CodeCommit のブランチプッシュするとB/Gデプロイが実行されます。
さいごに
CodePipelineでアカウントをまたいだパイプラインを作成してみる | DevelopersIO でも記載されているように、設定作業はかなり煩雑になります。
GitHub のイベントフックで実行できるのならば、その方が効率的ではあります。
とはいえ、利用する機会も少なくはないと思いますので CloudFormationテンプレートをまとめておきました。
よしなに加工・修正を施して再利用して頂ければと思います。
以上です。
どこかのどなたかのお役に立てば幸いです。