この記事は公開されてから1年以上経過しています。情報が古い可能性がありますので、ご注意ください。
コンテナの実行基盤としてAWS Fargateを利用しています。FireLens(Fluent Bit)の動作確認のため手軽にクラスターごと作成・削除できる検証環境を欲していました。素のNginx(Webサーバ)にFireLens込みの実行環境をCloudFormationのテンプレートにまとめました。FireLensは最近検証していた内容を踏まえ補足説明します。
以下の環境を構築します。
本記事のCloudFormationのスタック作成にはrainコマンドを使用しています。
事前準備
VPC作成
Fargate作成のテンプレートではパブリックサブネットにELBを新規作成します。プライベートサブネットにNginxコンテナと、FireLensコンテナをデプロイします。既存VPCを流用しても問題ありません。 紹介するVPCテンプレートは以下のネットワーク構成を作成します。
検証環境のコスト考慮
検証環境ということもありVPCを維持するのにあたり、NAT Gatewayのランニングコストが気になります。NAT Gatewayの作成有無はテンプレートから切り替えできます。初期構築時に「Nat Gatewayを作るか、作らないか」という意図ではなく、NAT Gateway
が必要なときに具体的にはプライベートサブネットからインターネットアクセスが必要なときはEnabeleNatGateway
をtrueにして、スタックの更新をかけます。NAT Gatewayを新規作成し、ルートテーブルを設定し直します。検証が終わったあとはfalseにして更新をかけるとNAT Gatewayを削除します。NAT Gatewayを維持するコスト削減を目的としています。NAT GatewayにアタッチするEIPも保持による課金を抑えるため、NAT Gatewayの削除と同時にEIPを解放します。
VPCテンプレート
折りたたみ
---
AWSTemplateFormatVersion: "2010-09-09"
Description: Create network 3 layers with switching single Nat Gateway
Metadata:
AWS::CloudFormation::Interface:
ParameterGroups:
- Label:
default: Common Settings
Parameters:
- ProjectName
- Environment
- Label:
default: VPC Settings
Parameters:
- VPCCidr
- PublicSubnetCidr1
- PublicSubnetCidr2
- PrivateSubnetCidr1
- PrivateSubnetCidr2
- IsolatedSubnetCidr1
- IsolatedSubnetCidr2
- Label:
default: NAT Gateway
Parameters:
- EnableNatGateway
Parameters:
ProjectName:
Description: Project Name
Type: String
Default: unnamed
Environment:
Description: Environment
Type: String
Default: dev
AllowedValues:
- prod
- dev
- stg
VPCCidr:
Description: VPC IP Range
Type: String
Default: 10.0.0.0/16
AllowedPattern: '^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\/([0-9]|[1-2][0-9]|3[0-2]))$'
PublicSubnetCidr1:
Description: Public Subnet 1 IP Range
Type: String
Default: 10.0.1.0/24
AllowedPattern: '^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\/([0-9]|[1-2][0-9]|3[0-2]))$'
PublicSubnetCidr2:
Description: Public Subnet 2 IP Range
Type: String
Default: 10.0.2.0/24
AllowedPattern: '^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\/([0-9]|[1-2][0-9]|3[0-2]))$'
PrivateSubnetCidr1:
Description: Private Subnet 1 IP Range
Type: String
Default: 10.0.17.0/24
AllowedPattern: '^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\/([0-9]|[1-2][0-9]|3[0-2]))$'
PrivateSubnetCidr2:
Description: Private Subnet 2 IP Range
Type: String
Default: 10.0.18.0/24
AllowedPattern: '^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\/([0-9]|[1-2][0-9]|3[0-2]))$'
IsolatedSubnetCidr1:
Description: Isolated Subnet 1 IP Range
Type: String
Default: 10.0.33.0/24
AllowedPattern: '^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\/([0-9]|[1-2][0-9]|3[0-2]))$'
IsolatedSubnetCidr2:
Description: Isolated Subnet 2 IP Range
Type: String
Default: 10.0.34.0/24
AllowedPattern: '^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\/([0-9]|[1-2][0-9]|3[0-2]))$'
EnableNatGateway:
Description: Enable NAT Gateway.
Type: String
Default: true
AllowedValues: [true, false]
Conditions:
EnableNatGateway: !Equals [true, !Ref EnableNatGateway]
Resources:
# --------------------------------------------
# VPC
# --------------------------------------------
VPC:
Type: AWS::EC2::VPC
Properties:
CidrBlock: !Ref VPCCidr
EnableDnsSupport: true
EnableDnsHostnames: true
Tags:
- Key: Name
Value: !Sub ${ProjectName}-${Environment}-vpc
- Key: Environment
Value: !Sub ${Environment}
# --------------------------------------------
# Internet Gateway
# --------------------------------------------
# Create InternetGateway & VPC Attach
InternetGateway:
Type: AWS::EC2::InternetGateway
Properties:
Tags:
- Key: Name
Value: !Sub ${ProjectName}-${Environment}-igw
- Key: Environment
Value: !Sub ${Environment}
AttachGateway:
Type: AWS::EC2::VPCGatewayAttachment
Properties:
VpcId: !Ref VPC
InternetGatewayId: !Ref InternetGateway
# --------------------------------------------
# NAT Gateway
# --------------------------------------------
# Create NatGateway
NatGateway1:
Type: AWS::EC2::NatGateway
Condition: EnableNatGateway
Properties:
AllocationId: !GetAtt NatGatewayEIP1.AllocationId
SubnetId: !Ref PublicSubnet1
Tags:
- Key: Name
Value: !Sub ${ProjectName}-${Environment}-natgw1
- Key: Environment
Value: !Sub ${Environment}
NatGatewayEIP1:
Type: AWS::EC2::EIP
Condition: EnableNatGateway
Properties:
Domain: vpc
Tags:
- Key: Name
Value: !Sub ${ProjectName}-${Environment}-natgw-eip1
- Key: Environment
Value: !Sub ${Environment}
# --------------------------------------------
# Route Table
# --------------------------------------------
# Create Public RouteTable & Setting Routing
PublicRouteTable1:
Type: AWS::EC2::RouteTable
DependsOn: AttachGateway
Properties:
VpcId: !Ref VPC
Tags:
- Key: Name
Value: !Sub ${ProjectName}-${Environment}-public-rtb1
- Key: Environment
Value: !Sub ${Environment}
PublicRoute1:
Type: AWS::EC2::Route
DependsOn: AttachGateway
Properties:
RouteTableId: !Ref PublicRouteTable1
DestinationCidrBlock: 0.0.0.0/0
GatewayId: !Ref InternetGateway
# Create Private RouteTable & Setting Routing
PrivateRouteTable1:
Type: AWS::EC2::RouteTable
DependsOn: AttachGateway
Properties:
VpcId: !Ref VPC
Tags:
- Key: Name
Value: !Sub ${ProjectName}-${Environment}-private-rtb1
- Key: Environment
Value: !Sub ${Environment}
PrivateRouteNatGW1:
Type: AWS::EC2::Route
Condition: EnableNatGateway
DependsOn: AttachGateway
Properties:
RouteTableId: !Ref PrivateRouteTable1
DestinationCidrBlock: 0.0.0.0/0
NatGatewayId: !Ref NatGateway1
# Create Isolated RouteTable & Setting Routing
IsolatedRouteTable1:
Type: AWS::EC2::RouteTable
Properties:
VpcId: !Ref VPC
Tags:
- Key: Name
Value: !Sub ${ProjectName}-${Environment}-isolated-rtb1
- Key: Environment
Value: !Sub ${Environment}
# --------------------------------------------
# Public Subnet
# --------------------------------------------
# Public 1
PublicSubnet1:
Type: AWS::EC2::Subnet
DependsOn: AttachGateway
Properties:
VpcId: !Ref VPC
AvailabilityZone: !Select [0, !GetAZs ""]
CidrBlock: !Ref PublicSubnetCidr1
MapPublicIpOnLaunch: true
Tags:
- Key: Name
Value: !Sub ${ProjectName}-${Environment}-public-subnet1
- Key: Environment
Value: !Sub ${Environment}
PublicSubnetRouteTableAssociation1:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
SubnetId: !Ref PublicSubnet1
RouteTableId: !Ref PublicRouteTable1
# Public 2
PublicSubnet2:
Type: AWS::EC2::Subnet
DependsOn: AttachGateway
Properties:
VpcId: !Ref VPC
AvailabilityZone: !Select [1, !GetAZs ""]
CidrBlock: !Ref PublicSubnetCidr2
MapPublicIpOnLaunch: true
Tags:
- Key: Name
Value: !Sub ${ProjectName}-${Environment}-public-subnet2
- Key: Environment
Value: !Sub ${Environment}
PublicSubnetRouteTableAssociation2:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
SubnetId: !Ref PublicSubnet2
RouteTableId: !Ref PublicRouteTable1
# --------------------------------------------
# Private Subnet
# --------------------------------------------
# Private 1
PrivateSubnet1:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref VPC
AvailabilityZone: !Select [0, !GetAZs ""]
CidrBlock: !Ref PrivateSubnetCidr1
Tags:
- Key: Name
Value: !Sub ${ProjectName}-${Environment}-private-subnet1
PrivateSubnetRouteTableAssociation1:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
SubnetId: !Ref PrivateSubnet1
RouteTableId: !Ref PrivateRouteTable1
# Private 2
PrivateSubnet2:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref VPC
AvailabilityZone: !Select [1, !GetAZs ""]
CidrBlock: !Ref PrivateSubnetCidr2
Tags:
- Key: Name
Value: !Sub ${ProjectName}-${Environment}-private-subnet2
PrivateSubnetRouteTableAssociation2:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
SubnetId: !Ref PrivateSubnet2
RouteTableId: !Ref PrivateRouteTable1
# --------------------------------------------
# Isolated Subnet
# --------------------------------------------
# Isolated 1
IsolatedSubnet1:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref VPC
AvailabilityZone: !Select [0, !GetAZs ""]
CidrBlock: !Ref IsolatedSubnetCidr1
Tags:
- Key: Name
Value: !Sub ${ProjectName}-${Environment}-isolated-subnet1
- Key: Environment
Value: !Sub ${Environment}
IsolatedSubnetRouteTableAssociation1:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
SubnetId: !Ref IsolatedSubnet1
RouteTableId: !Ref IsolatedRouteTable1
# Isolated 2
IsolatedSubnet2:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref VPC
AvailabilityZone: !Select [1, !GetAZs ""]
CidrBlock: !Ref IsolatedSubnetCidr2
Tags:
- Key: Name
Value: !Sub ${ProjectName}-${Environment}-isolated-subnet2
- Key: Environment
Value: !Sub ${Environment}
IsolatedSubnetRouteTableAssociation2:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
SubnetId: !Ref IsolatedSubnet2
RouteTableId: !Ref IsolatedRouteTable1
# --------------------------------------------
# Network ACL
# --------------------------------------------
# Public NACL
PublicNetworkACL1:
Type: AWS::EC2::NetworkAcl
Properties:
VpcId: !Ref VPC
Tags:
- Key: Name
Value: !Sub ${ProjectName}-${Environment}-public-nacl1
NetworkACLEntryPublicIngress1:
Type: AWS::EC2::NetworkAclEntry
Properties:
CidrBlock: "0.0.0.0/0"
Egress: false
NetworkAclId: !Ref PublicNetworkACL1
Protocol: -1
RuleAction: "allow"
RuleNumber: 100
NetworkACLEntryPublicEgress1:
Type: "AWS::EC2::NetworkAclEntry"
Properties:
CidrBlock: "0.0.0.0/0"
Egress: true
NetworkAclId: !Ref PublicNetworkACL1
Protocol: -1
RuleAction: "allow"
RuleNumber: 100
# Private NACL
PrivateNetworkACL1:
Type: AWS::EC2::NetworkAcl
Properties:
VpcId: !Ref VPC
Tags:
- Key: Name
Value: !Sub ${ProjectName}-${Environment}-private-nacl1
NetworkACLEntryPrivateIngress1:
Type: AWS::EC2::NetworkAclEntry
Properties:
CidrBlock: "0.0.0.0/0"
Egress: false
NetworkAclId: !Ref PrivateNetworkACL1
Protocol: -1
RuleAction: "allow"
RuleNumber: 100
NetworkACLEntryPrivateEgress1:
Type: "AWS::EC2::NetworkAclEntry"
Properties:
CidrBlock: "0.0.0.0/0"
Egress: true
NetworkAclId: !Ref PrivateNetworkACL1
Protocol: -1
RuleAction: "allow"
RuleNumber: 100
# Isolated NACL
IsolatedNetworkACL1:
Type: AWS::EC2::NetworkAcl
Properties:
VpcId: !Ref VPC
Tags:
- Key: Name
Value: !Sub ${ProjectName}-${Environment}-isolated-nacl1
NetworkACLEntryIsolatedIngress1:
Type: AWS::EC2::NetworkAclEntry
Properties:
CidrBlock: "0.0.0.0/0"
Egress: false
NetworkAclId: !Ref IsolatedNetworkACL1
Protocol: -1
RuleAction: "allow"
RuleNumber: 100
NetworkACLEntryIsolatedEgress1:
Type: "AWS::EC2::NetworkAclEntry"
Properties:
CidrBlock: "0.0.0.0/0"
Egress: true
NetworkAclId: !Ref IsolatedNetworkACL1
Protocol: -1
RuleAction: "allow"
RuleNumber: 100
# NetworkACL Association
PublicNetworkACLAssocation1:
Type: AWS::EC2::SubnetNetworkAclAssociation
Properties:
SubnetId: !Ref PublicSubnet1
NetworkAclId: !Ref PublicNetworkACL1
PublicNetworkACLAssocation2:
Type: AWS::EC2::SubnetNetworkAclAssociation
Properties:
SubnetId: !Ref PublicSubnet2
NetworkAclId: !Ref PublicNetworkACL1
PrivateNetworkACLAssocation1:
Type: AWS::EC2::SubnetNetworkAclAssociation
Properties:
SubnetId: !Ref PrivateSubnet1
NetworkAclId: !Ref PrivateNetworkACL1
PrivateNetworkACLAssocation2:
Type: AWS::EC2::SubnetNetworkAclAssociation
Properties:
SubnetId: !Ref PrivateSubnet2
NetworkAclId: !Ref PrivateNetworkACL1
IsolatedNetworkACLAssocation1:
Type: AWS::EC2::SubnetNetworkAclAssociation
Properties:
SubnetId: !Ref IsolatedSubnet1
NetworkAclId: !Ref IsolatedNetworkACL1
IsolatedNetworkACLAssocation2:
Type: AWS::EC2::SubnetNetworkAclAssociation
Properties:
SubnetId: !Ref IsolatedSubnet2
NetworkAclId: !Ref IsolatedNetworkACL1
Outputs:
ExportVPC:
Value: !Ref VPC
Export:
Name: !Sub ${AWS::StackName}-VPC
ExportPublicSubnet1:
Value: !Ref PublicSubnet1
Export:
Name: !Sub ${AWS::StackName}-PublicSubnet1
ExportPublicSubnet2:
Value: !Ref PublicSubnet2
Export:
Name: !Sub ${AWS::StackName}-PublicSubnet2
ExportPrivateSubnet1:
Value: !Ref PrivateSubnet1
Export:
Name: !Sub ${AWS::StackName}-PrivateSubnet1
ExportPrivateSubnet2:
Value: !Ref PrivateSubnet2
Export:
Name: !Sub ${AWS::StackName}-PrivateSubnet2
ExportPrivateRoutetable1:
Value: !Ref PrivateRouteTable1
Export:
Name: !Sub ${AWS::StackName}-PrivateRoutetable1
ExportIsolatedSubnet1:
Value: !Ref IsolatedSubnet1
Export:
Name: !Sub ${AWS::StackName}-IsolatedSubnet1
ExportIsolatedSubnet2:
Value: !Ref IsolatedSubnet2
Export:
Name: !Sub ${AWS::StackName}-IsolatedSubnet2
ExportIsolatedetable1:
Value: !Ref IsolatedRouteTable1
Export:
Name: !Sub ${AWS::StackName}-Isolatedetable1
本検証環境は以下のVPCテンプレートをrainコマンドで以下のパラメータで作成した環境を用います。もちろんマネージドコンソール、AWS CLIから同様のCloudFormationのスタックを作成できます。
rain deploy ./vpc.yml sample-vpc-stack --params \
ProjectName=sample,\
Environment=dev,\
VPCCidr=10.0.0.0/16,\
PublicSubnetCidr1=10.0.1.0/24,\
PublicSubnetCidr2=10.0.2.0/24,\
PrivateSubnetCidr1=10.0.17.0/24,\
PrivateSubnetCidr2=10.0.18.0/24,\
IsolatedSubnetCidr1=10.0.33.0/24,\
IsolatedSubnetCidr2=10.0.34.0/24,\
EnableNatGateway=true
ECR作成
この後Fargateをデプロイするタスク定義の都合、FireLens(Fluent Bit)の独自設定ファイル込みイメージのを事前にアップロードする必要があります。本検証環境ではsample3-custom-logrouter-firelens
リポジトリを作成しました。
FireLensイメージ作成
先ほど作成したリポジトリにアップロードするイメージを作成します。
カレントディレクトリにあるFluent Bitの設定ファイル(extra.conf)をイメージ内の/fluent-bit/etc/extra.conf
パスへ保存します。このパスをFargateのタスク定義内で指定する箇所があります。設定ファイルをコピーするパスを変更する場合は、後ほど紹介するFargateのテンプレート内のタスク定義の修正も合わせて実施してください。
Dockerfile
FROM public.ecr.aws/aws-observability/aws-for-fluent-bit:2.19.0
COPY ./extra.conf /fluent-bit/etc/extra.conf
ディレクトリ構成は以下の状態です。
.
├── Dockerfile
└── extra.conf
Fluent Bitの設定ファイル(extra.conf)と、設定してある内容は以下です。
- ELBのヘルスチェックによるアクセスログは除外
- CloudWatch Logsにログを保存
- CloudWatch Logsのログストリーム名はタスクごとに作成
extra.conf
[SERVICE]
Flush 1
Grace 30
[FILTER]
Name grep
Match webapp-firelens*
Exclude log ^(?=.*ELB-HealthChecker\/2\.0).*$
[OUTPUT]
Name cloudwatch
Match webapp-firelens*
region ap-northeast-1
log_group_name /$(ecs_cluster)
log_stream_name App/$(ecs_task_id)
auto_create_group true
設定内容については以下のリンクで説明しています。
- FireLens(Fluent Bit)でELBヘルスチェックのログを除外してからCloudWatch Logsへログを保存してみる | DevelopersIO
- FireLens(Fluent Bit)からCloudWatch Logsへログを送信時、ThrottlingExceptionエラー回避のためログストリーム設定を見直す | DevelopersIO
イメージのアップロード
Fargateをデプロイするテンプレートのパラメータにイメージ名とタグ名を入力する項目があります。
本検証環境ではv1
タグを付けて、手元でビルドしたイメージをアップロードします。
aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin 123456789012.dkr.ecr.us-east-1.amazonaws.com
docker build -t sample3-custom-logrouter-firelens:v1 .
docker tag sample3-custom-logrouter-firelens:v1 123456789012.dkr.ecr.ap-northeast-1.amazonaws.com/sample3-custom-logrouter-firelens:v1
docker push 123456789012.dkr.ecr.ap-northeast-1.amazonaws.com/sample3-custom-logrouter-firelens:v1
VPCと、FireLensのイメージをアップロードしたら準備は完了です。
Fargate作成
パブリックサブネットにELBを、プライベートサブネットに素のNginxコンテナ+FireLensをデプロイします。そのため以下の情報が必要になります。VPCを今回のテンプレートから作成した場合はOutputに書き出してある値をそのまま使います。
VPC関連
各種IDを入力
- VPCID
- PublicSubnet1
- PublicSubnet2
- PrivateSubnet1
- PrivateSubnet2
FireLens
ECRにアップロードしたFireLensのイメージ名:タグを入力
- ImageNameFirelens
以下の環境が作成できます。
Fargateテンプレート
Fluent Bitの設定ファイル(extraf.conf)のイメージ内保存先のパスを変更している場合は以下の箇所を適切なパスに変更してください。
抜粋
FirelensConfiguration:
Type: "fluentbit"
Options:
config-file-type: "file"
config-file-value: "/fluent-bit/etc/extra.conf"
折りたたみ
AWSTemplateFormatVersion: "2010-09-09"
Description: Create Fargate*1 with B/G Deployment, AutoScaling and Firelens
Metadata:
AWS::CloudFormation::Interface:
ParameterGroups:
- Label:
default: Common Settings
Parameters:
- ProjectName
- Environment
- Label:
default: ECS VPC Settings
Parameters:
- VPCID
- PublicSubnet1
- PublicSubnet2
- PrivateSubnet1
- PrivateSubnet2
Parameters:
ProjectName:
Description: Project Name
Type: String
Default: unnamed
Environment:
Description: Environment
Type: String
Default: dev
AllowedValues:
- prod
- dev
- stg
VPCID:
Type: AWS::EC2::VPC::Id
PublicSubnet1:
Description: "ELB Subnet 1st"
Type: AWS::EC2::Subnet::Id
PublicSubnet2:
Description: "ELB Subnet 2nd"
Type: AWS::EC2::Subnet::Id
PrivateSubnet1:
Description: "ECS Subnet 1st"
Type: AWS::EC2::Subnet::Id
PrivateSubnet2:
Description: "ECS Subnet 2nd"
Type: AWS::EC2::Subnet::Id
DesiredCount:
Type: Number
Default: 1
ClusterName:
Type: String
Default: cluster
AppName:
Type: String
Default: webapp
ServiceName:
Type: String
Default: service
TaskDefinitionName:
Type: String
Default: taskdefinition
ImageNameWebApp:
Description: "Web Application Repository Name also Need to TagName"
Type: String
Default: "public.ecr.aws/nginx/nginx:latest"
ImageNameFirelens:
Description: "Firelens Repository Name also Need to TagName"
Type: String
Default: "public.ecr.aws/aws-observability/aws-for-fluent-bit:latest"
Resources:
# --------------------------------------------
# ELB
# --------------------------------------------
ELB1:
Type: AWS::ElasticLoadBalancingV2::LoadBalancer
Properties:
Type: "application"
Name: !Sub ${ProjectName}-${Environment}-elb
Scheme: "internet-facing"
SecurityGroups:
- !Ref SecurityGroup2
Subnets:
- !Ref PublicSubnet1
- !Ref PublicSubnet2
IpAddressType: "ipv4"
Tags:
- Key: Name
Value: !Sub ${ProjectName}-${Environment}-elb
ELBListener1:
Type: AWS::ElasticLoadBalancingV2::Listener
Properties:
DefaultActions:
- TargetGroupArn: !Ref TargetGroup1
Type: "forward"
LoadBalancerArn: !Ref ELB1
Port: 80
Protocol: "HTTP"
ELBListener2:
Type: AWS::ElasticLoadBalancingV2::Listener
Properties:
DefaultActions:
- TargetGroupArn: !Ref TargetGroup2
Type: "forward"
LoadBalancerArn: !Ref ELB1
Port: 8080
Protocol: "HTTP"
TargetGroup1:
Type: "AWS::ElasticLoadBalancingV2::TargetGroup"
Properties:
VpcId: !Ref VPCID
Name: !Sub ${ProjectName}-${Environment}-tg1
Protocol: "HTTP"
HealthCheckPath: "/"
Port: 80
TargetType: ip
HealthCheckIntervalSeconds: 10 # Default is 30.
HealthyThresholdCount: 2 # Default is 5.
HealthCheckTimeoutSeconds: 5
UnhealthyThresholdCount: 2
TargetGroupAttributes:
- Key: "stickiness.enabled"
Value: "false"
- Key: deregistration_delay.timeout_seconds
Value: "60" # default is 300.
- Key: "stickiness.type"
Value: "lb_cookie"
- Key: "stickiness.lb_cookie.duration_seconds"
Value: "86400"
- Key: "slow_start.duration_seconds"
Value: "0"
- Key: "load_balancing.algorithm.type"
Value: "round_robin"
TargetGroup2:
Type: "AWS::ElasticLoadBalancingV2::TargetGroup"
Properties:
VpcId: !Ref VPCID
Name: !Sub ${ProjectName}-${Environment}-tg2
Protocol: "HTTP"
HealthCheckPath: "/"
Port: 80
TargetType: ip
HealthCheckIntervalSeconds: 10 # Default is 30.
HealthyThresholdCount: 2 # Default is 5.
HealthCheckTimeoutSeconds: 5
UnhealthyThresholdCount: 2
TargetGroupAttributes:
- Key: "stickiness.enabled"
Value: "false"
- Key: deregistration_delay.timeout_seconds
Value: "60" # default is 300.
- Key: "stickiness.type"
Value: "lb_cookie"
- Key: "stickiness.lb_cookie.duration_seconds"
Value: "86400"
- Key: "slow_start.duration_seconds"
Value: "0"
- Key: "load_balancing.algorithm.type"
Value: "round_robin"
# --------------------------------------------
# CloudWatch Logs Group
# --------------------------------------------
# FireLens Stdout
FireLensLogGroup:
Type: "AWS::Logs::LogGroup"
Properties:
LogGroupName: !Sub "${ProjectName}-${Environment}-${ClusterName}-firelens-logs"
RetentionInDays: 400
# --------------------------------------------
# ECS Fargate
# --------------------------------------------
# Cluster
ECSCluster:
Type: "AWS::ECS::Cluster"
Properties:
ClusterName: !Sub "${ProjectName}-${Environment}-${ClusterName}"
ClusterSettings:
- Name: containerInsights
Value: enabled
CapacityProviders:
- "FARGATE_SPOT"
- "FARGATE"
# Service
ECSService:
Type: "AWS::ECS::Service"
Properties:
ServiceName: !Sub ${ProjectName}-${Environment}-${ServiceName}
Cluster: !Ref ECSCluster
LaunchType: "FARGATE"
PlatformVersion: "1.4.0"
DeploymentController:
Type: CODE_DEPLOY
DesiredCount: !Ref DesiredCount
LoadBalancers:
- TargetGroupArn: !Ref TargetGroup1
ContainerName: !Ref AppName
ContainerPort: 80
NetworkConfiguration:
AwsvpcConfiguration:
AssignPublicIp: "DISABLED"
SecurityGroups:
- !Ref SecurityGroup1
Subnets:
- !Ref PrivateSubnet1
- !Ref PrivateSubnet2
TaskDefinition: !Ref ECSTaskDefinition
DependsOn: ELBListener1
# ECS TaskDefinition
ECSTaskDefinition:
Type: "AWS::ECS::TaskDefinition"
Properties:
Family: !Sub "${ProjectName}-${Environment}-${AppName}-${TaskDefinitionName}"
TaskRoleArn: !GetAtt ECSTaskRole1.Arn
ExecutionRoleArn: !GetAtt ECSTaskExecutionRole1.Arn
NetworkMode: "awsvpc"
RequiresCompatibilities:
- "FARGATE"
Cpu: "256"
Memory: "512"
ContainerDefinitions:
- Essential: true
Name: !Ref AppName
Image: !Ref ImageNameWebApp
LogConfiguration:
LogDriver: "awsfirelens"
PortMappings:
- ContainerPort: 80
HostPort: 80
Protocol: "tcp"
- Essential: true
Name: "log_router"
Image: !Ref ImageNameFirelens
LogConfiguration:
LogDriver: "awslogs"
Options:
awslogs-group: !Ref FireLensLogGroup
awslogs-region: !Ref AWS::Region
awslogs-stream-prefix: "FireLens"
FirelensConfiguration:
Type: "fluentbit"
Options:
config-file-type: "file"
config-file-value: "/fluent-bit/etc/extra.conf"
User: "0"
# --------------------------------------------
# Security Group
# --------------------------------------------
# Security Group for WebApp
SecurityGroup1:
Type: AWS::EC2::SecurityGroup
Properties:
GroupName: !Sub ${ProjectName}-${Environment}-${AppName}-sg
GroupDescription: Web App Security Group
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: 80
ToPort: 80
SourceSecurityGroupId: !Ref SecurityGroup2
Description: "Access from ELB"
VpcId: !Ref VPCID
Tags:
- Key: Name
Value: !Sub ${ProjectName}-${Environment}-${AppName}-sg
# Security Group for ELB
SecurityGroup2:
Type: AWS::EC2::SecurityGroup
Properties:
GroupName: !Sub ${ProjectName}-${Environment}-elb-sg
GroupDescription: ELB Security Group
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: 80
ToPort: 80
CidrIp: "0.0.0.0/0"
Description: "Access from Public / Blue"
- IpProtocol: tcp
FromPort: 8080
ToPort: 8080
CidrIp: "0.0.0.0/0"
Description: "Access from Public / Green"
VpcId: !Ref VPCID
Tags:
- Key: Name
Value: !Sub ${ProjectName}-${Environment}-elb-sg
# --------------------------------------------
# IAM Role
# --------------------------------------------
# ECS Task Execution Role
ECSTaskExecutionRole1:
Type: "AWS::IAM::Role"
Properties:
RoleName: !Sub ${ProjectName}-${Environment}-${AppName}-ECSTaskExecutionRole
Path: "/"
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service: ecs-tasks.amazonaws.com
Action: sts:AssumeRole
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy
# ECS Task Role
ECSTaskRole1:
Type: "AWS::IAM::Role"
Properties:
Path: "/"
RoleName: !Sub ${ProjectName}-${Environment}-${AppName}-ECSTaskRole
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service: ecs-tasks.amazonaws.com
Action: sts:AssumeRole
ManagedPolicyArns:
- !Ref ECSExecPolicy
- !Ref SentCloudWatchLogsPolicy
# --------------------------------------------
# IAM Policy
# --------------------------------------------
# Allowed ECS Exec for Task Role
ECSExecPolicy:
Type: "AWS::IAM::ManagedPolicy"
Properties:
ManagedPolicyName: !Sub "${ProjectName}-${Environment}-ECSExecPolicy"
Path: "/"
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Action:
- ssmmessages:CreateControlChannel
- ssmmessages:CreateDataChannel
- ssmmessages:OpenControlChannel
- ssmmessages:OpenDataChannel
Resource: "*"
# Sent CloudWatch Logs for Task Role
SentCloudWatchLogsPolicy:
Type: "AWS::IAM::ManagedPolicy"
Properties:
ManagedPolicyName: !Sub "${ProjectName}-${Environment}-SentCloudWatchLogsPolicy"
Path: "/"
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Action:
- logs:CreateLogStream
- logs:CreateLogGroup
- logs:DescribeLogStreams
- logs:PutLogEvents
Resource: !Sub arn:aws:logs:${AWS::Region}:${AWS::AccountId}:log-group:*
# ------------------------------------------------------------#
# Auto Scaling Service
# ------------------------------------------------------------#
ServiceScalingTarget:
Type: AWS::ApplicationAutoScaling::ScalableTarget
Properties:
MinCapacity: 1
MaxCapacity: 4
# ResourceIdの必要書式: service/クラスター名/サービス名
ResourceId: !Sub service/${ECSCluster}/${ProjectName}-${Environment}-${ServiceName}
RoleARN: !Sub "arn:aws:iam::${AWS::AccountId}:role/aws-service-role/ecs.application-autoscaling.amazonaws.com/AWSServiceRoleForApplicationAutoScaling_ECSService"
ScalableDimension: ecs:service:DesiredCount
ServiceNamespace: ecs
DependsOn:
- ECSService # ResourIdでサービスを参照するため先に作成されている必要がある
ServiceScaleOutPolicy:
Type: AWS::ApplicationAutoScaling::ScalingPolicy
Properties:
PolicyName: !Sub "${ProjectName}-${Environment}-${ServiceName}-ScaleOutPolicy"
PolicyType: StepScaling
ScalingTargetId: !Ref ServiceScalingTarget
StepScalingPolicyConfiguration:
AdjustmentType: ChangeInCapacity
Cooldown: 60
MetricAggregationType: Average
StepAdjustments:
- ScalingAdjustment: 1
MetricIntervalLowerBound: 0
ServiceScaleInPolicy:
Type: AWS::ApplicationAutoScaling::ScalingPolicy
Properties:
PolicyName: !Sub "${ProjectName}-${Environment}-${ServiceName}-ScaleInPolicy"
PolicyType: StepScaling
ScalingTargetId: !Ref ServiceScalingTarget
StepScalingPolicyConfiguration:
AdjustmentType: ChangeInCapacity
Cooldown: 60
MetricAggregationType: Average
StepAdjustments:
- ScalingAdjustment: -1
MetricIntervalUpperBound: 0
# CloudWatch Alarms
ServiceScaleOutAlarm:
Type: AWS::CloudWatch::Alarm
Properties:
AlarmName: !Sub "${ProjectName}-${Environment}-${ServiceName}-ScaleOutAlarm"
EvaluationPeriods: 1
Statistic: Average
TreatMissingData: notBreaching
Threshold: 10
AlarmDescription: Alarm to add capacity if CPU is high
Period: 60
AlarmActions:
- !Ref ServiceScaleOutPolicy
Namespace: AWS/ECS
Dimensions:
- Name: ClusterName
Value: !Ref ECSCluster
- Name: ServiceName
Value: !Sub "${ProjectName}-${Environment}-${ServiceName}"
ComparisonOperator: GreaterThanThreshold
MetricName: CPUUtilization
ServiceScaleInAlarm:
Type: AWS::CloudWatch::Alarm
Properties:
AlarmName: !Sub "${ProjectName}-${Environment}-${ServiceName}-ScaleInAlarm"
EvaluationPeriods: 1
Statistic: Average
TreatMissingData: notBreaching
Threshold: 5
AlarmDescription: Alarm to reduce capacity if container CPU is low
Period: 300
AlarmActions:
- !Ref ServiceScaleInPolicy
Namespace: AWS/ECS
Dimensions:
- Name: ClusterName
Value: !Ref ECSCluster
- Name: ServiceName
Value: !Sub "${ProjectName}-${Environment}-${ServiceName}"
ComparisonOperator: LessThanThreshold
MetricName: CPUUtilization
本検証環境は以下のFargateテンプレートをrainコマンドで以下のパラメータで作成した環境を用います。
rain deploy ./fargate.yml sample3-fargate-stack --params \
ProjectName=sample3,\
Environment=dev,\
ClusterName=cluster,\
ServiceName=service,\
TaskDefinitionName=taskdefinition,\
AppName=webapp,\
DesiredCount=1,\
ImageNameWebApp=public.ecr.aws/nginx/nginx:latest,\
ImageNameFirelens=123456789012.dkr.ecr.ap-northeast-1.amazonaws.com/sample3-custom-logrouter-firelens:v1,\
VPCID=vpc-0f6d43eea7f6e7fdf,\
PublicSubnet1=subnet-059f8f126d382354b,\
PublicSubnet2=subnet-0ac4ece5ea47938a6,\
PrivateSubnet1=subnet-08a24e56d05376997,\
PrivateSubnet2=subnet-0f625692b77783cec
10分ほどでFargateの構築、コンテナの起動が完了します。
補足
起動してきた素のNginxコンテナにはAutoScalingの設定が加えてあります。当初Fluent Bitの設定を検証していたとき、AutoScalingは不要だろうと思い設定していませんでした。Fluent Bitのプラグインを検証してるとAutoScalingしてかつ、ログ流量が多くないと気が付かなかったログストリームのクォータの問題がありました。そのため、AutoScaling設定と、Fluent Bitの設定はその対策を施してあるものを載せています。
詳しくは以下のリンクをご参照ください。
- FireLens(Fluent Bit)からCloudWatch Logsへログを送信時、ThrottlingExceptionエラー回避のためログストリーム設定を見直す | DevelopersIO
- FireLens(Fluent Bit)CloudWatch Logsの新プラグインを使ったログストリーム作成の注意点 | DevelopersIO
コンテナの確認
タスク確認
Nginxコンテナ(webaap)と、自前の設定ファイル込みのFireLensコンテナ(log_router)が起動しています。
WebブラウザからELBにアクセスするとNginxのデフォルトページを確認できます。
CloudWatch Logsの確認
Nginxコンテナのログ
FireLensのFluent Bit経由でCloudWatch Logsへ保存されます。ロググループ、ログストリームはFluent Bitの自前の設定ファイルに書いた内容で作成されています。ログストリームの末尾にはタスクIDが付与され、タスクごとにログストリームも分けることができます。
Nginxコンテナのアクセスログは以下の内容が記録されていました。
{
"container_id": "c55844d79014427fad38dddb7ca529df-1393473712",
"container_name": "webapp",
"ecs_cluster": "sample3-dev-cluster",
"ecs_task_arn": "arn:aws:ecs:ap-northeast-1:123456789012:task/sample3-dev-cluster/c55844d79014427fad38dddb7ca529df",
"ecs_task_definition": "sample3-dev-webapp-taskdefinition:11",
"log": "10.0.1.142 - - [05/Sep/2021:05:42:46 +0000] \"GET / HTTP/1.1\" 200 612 \"-\" \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36\" \"11.22.33.44\"",
"source": "stdout"
}
FireLensのログ
Fluent Bitは経由せず標準のログドライバー(awslogs
)からCloudWatch Logsへ保存されます。ロググループと、ログストリームのタスクIDより前半部分はFargateのタスク定義で設定した内容です。こちらのログストリーム末尾のタスクIDはデフォルトで付与されます。
詳しくは以下をご参照ください。
- awslogs ログドライバーを使用する - Amazon Elastic Container Service
- FireLensコンテナのログをCloudWatch Logsへ保存したときのログストリームの名は。 | DevelopersIO
保存されていたログの内容はFluent Bit起動時のログしかありませんでした。こちらはFireLensコンテナのデバッグ目的のログ保存です。たとえばFluent BitからCloudWatch Logsへ送信時にスロットリングが発生している場合はこちらにログが記録されます。
再掲になりますが、以下のリンクはスロットリング発生時、FireLensコンテナに記録されたログを載せております。
- FireLens(Fluent Bit)からCloudWatch Logsへログを送信時、ThrottlingExceptionエラー回避のためログストリーム設定を見直す | DevelopersIO
CloudWatch Logsへログ送信する権限不足(IAMロースの設定ミス)のログもFireLensコンテナに記録されます。
おわりに
CloudFormationのテンプレートの紹介とともに、最近のFireLens検証から得た知見をあわせて紹介させていただきました。いろいろ検証して時間がかりました。今から勉強される方はショートカットできるように、どなたかのお役に立てれば幸いです。