AWS CloudFormationでVPCEndpoint + NLB + ALB + EC2の構成を作ってみる

AWS CloudFormationでVPCEndpoint + NLB + ALB + EC2の構成を作成しました！

#AWS

#AWS CloudFormation

#VPC Endpoint

#Network Load Balancer（NLB）

#Application Load Balancer（ALB）

おつまみ

2022.10.04

この記事は公開されてから1年以上経過しています。情報が古い可能性がありますので、ご注意ください。

こんにちは！AWS事業本部のおつまみです。

皆さん、AWS CloudFormationでVPCEndpoint + NLB + ALB + EC2の構成を作ってみたいなぁと思ったことはありますか？私はあります。

ちなみに2021年9月よりNLBのターゲットにALBを登録することができるようになってます。

今回同じようなシステム構成の案件に携わったので、せっかくなら自分でAWS CloudFormationを使って構築してみようと思い、やってみました。

ちなみにAWS CDKを使ったNLB + ALB + EC2の構成はこちらが参考になります。

実際にやってみた

構成図

ゴールはVPC AのEC2からVPC BのEC2に接続されることです。

VPC AのEC2にはSession Manager経由で接続します。
VPC BのEC2には事前にユーザデータで下記を設定しています。
- apacheをインストール
- ドキュメントルート配下(/var/www/html)にindex.htmlを作成し、「Hello World」を設定。

CloudFormationテンプレート

CloudFormationテンプレートはこのように3つに分けて作成しました。

VPCA.yml

AWSTemplateFormatVersion: "2010-09-09"
Description: "VPC-A Template."

Parameters:
  # ------------------------------------------------------------#
  # Common
  # ------------------------------------------------------------#
  Prefix:
    Type: String
    Default: "vpc-a"

  # ------------------------------------------------------------#
  # Network
  # ------------------------------------------------------------#
  VpcCidr:
    Type: String
    Default: "10.0.0.0/16"

  PrivateSubnetCidr:
    Type: String
    Default: "10.0.0.0/24"

  # ------------------------------------------------------------#
  # EC2
  # ------------------------------------------------------------#
  EC2InstanceName:
    Type: String
    Default: "ec2"
  EC2InstanceAMI:
    Type: AWS::EC2::Image::Id
    Default: "ami-078296f82eb463377" # Amazon Linux 2 AMI (HVM) - Kernel 5.10, SSD Volume Type
  EC2InstanceInstanceType:
    Type: String
    Default: "t3.nano"
  EC2InstanceVolumeType:
    Type: String
    Default: "gp2"
  EC2InstanceVolumeSize:
    Type: String
    Default: "8"

Resources:
  # ------------------------------------------------------------#
  #  Network
  # ------------------------------------------------------------#
  Vpc:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: !Ref VpcCidr
      EnableDnsSupport: true
      EnableDnsHostnames: true
      Tags:
        - Key: Name
          Value: !Sub ${Prefix}-vpc

  PrivateSubnet:
    Type: AWS::EC2::Subnet
    Properties:
      CidrBlock: !Ref PrivateSubnetCidr
      VpcId: !Ref Vpc
      AvailabilityZone:
        Fn::Select:
          - "0"
          - Fn::GetAZs: ""
      Tags:
        - Key: Name
          Value: !Sub ${Prefix}-private-subnet

  PrivateRouteTable: 
    Type: AWS::EC2::RouteTable
    Properties: 
      VpcId: !Ref Vpc 
      Tags: 
        - Key: Name
          Value: !Sub "${Prefix}-table"
  
  PrivateSubnetTableAssociation: 
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties: 
      SubnetId: !Ref PrivateSubnet
      RouteTableId: !Ref PrivateRouteTable
  
  EC2SecurityGroup:
    Type: "AWS::EC2::SecurityGroup"
    Properties:
      VpcId: !Ref Vpc
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 80
          ToPort: 80
          CidrIp: "0.0.0.0/0"
      GroupName: !Sub "${Prefix}-sg"
      GroupDescription: "-"
      Tags:
        - Key: "Name"
          Value: !Sub "${Prefix}-sg"

  # ------------------------------------------------------------#
  #  VPC Endpoint
  # ------------------------------------------------------------#
  SsmVpcEndpointSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: !Sub ${Prefix}-ssm-vpc-endpoint-sg
      GroupName: !Sub ${Prefix}-ssm-vpc-endpoint-sg
      VpcId: !Ref Vpc
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 443
          ToPort: 443
          SourceSecurityGroupId: !Ref EC2SecurityGroup
      Tags:
        - Key: Name
          Value: !Sub ${Prefix}-ssm-vpc-endpoint-sg

  SsmVpcEndpoint:
    Type: AWS::EC2::VPCEndpoint
    Properties:
      VpcEndpointType: Interface
      PrivateDnsEnabled: true
      ServiceName: !Sub com.amazonaws.${AWS::Region}.ssm
      VpcId: !Ref Vpc
      SubnetIds:
        - !Ref PrivateSubnet
      SecurityGroupIds:
        - !Ref SsmVpcEndpointSecurityGroup

  SsmMessagesVpcEndpointSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: !Sub ${Prefix}-ssmmessages-vpc-endpoint-sg
      GroupName: !Sub ${Prefix}-ssmmessages-vpc-endpoint-sg
      VpcId: !Ref Vpc
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 443
          ToPort: 443
          SourceSecurityGroupId: !Ref EC2SecurityGroup
      Tags:
        - Key: Name
          Value: !Sub ${Prefix}-ssmmessages-vpc-endpoint-sg

  SsmMessagesVpcEndpoint:
    Type: AWS::EC2::VPCEndpoint
    Properties:
      VpcEndpointType: Interface
      PrivateDnsEnabled: true
      ServiceName: !Sub com.amazonaws.${AWS::Region}.ssmmessages
      VpcId: !Ref Vpc
      SubnetIds:
        - !Ref PrivateSubnet
      SecurityGroupIds:
        - !Ref SsmMessagesVpcEndpointSecurityGroup

  # ------------------------------------------------------------#
  #  Ec2InstanceProfile
  # ------------------------------------------------------------#
  Ec2Role:
    Type: AWS::IAM::Role
    Properties:
      RoleName: !Sub ${Prefix}-ec2-role
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - ec2.amazonaws.com
            Action: sts:AssumeRole
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore

  Ec2InstanceProfile:
    Type: AWS::IAM::InstanceProfile
    Properties:
      InstanceProfileName: !Sub ${Prefix}-ec2-instance-profile
      Roles:
        - !Ref Ec2Role

  # ------------------------------------------------------------#
  #  EC2Instance
  # ------------------------------------------------------------#
  EC2Instance:
    Type: "AWS::EC2::Instance"
    Properties:
      Tags:
        - Key: Name
          Value: !Sub "${Prefix}-${EC2InstanceName}"
      ImageId: !Ref EC2InstanceAMI
      InstanceType: !Ref EC2InstanceInstanceType
      IamInstanceProfile: !Ref Ec2InstanceProfile
      DisableApiTermination: false
      EbsOptimized: false
      BlockDeviceMappings:
        - DeviceName: /dev/xvda
          Ebs:
            DeleteOnTermination: true
            VolumeType: !Ref EC2InstanceVolumeType
            VolumeSize: !Ref EC2InstanceVolumeSize
      SecurityGroupIds:
        - !Ref EC2SecurityGroup
      SubnetId: !Ref PrivateSubnet
      UserData: !Base64 |
        #! /bin/bash
        yum update -y

# ------------------------------------------------------------#
# Output Parameters
# ------------------------------------------------------------#   
Outputs:
  Vpc:
    Value: !Ref Vpc
    Export:
      Name: VpcA
  
  PrivateSubnet:
    Value: !Ref PrivateSubnet
    Export:
      Name: PrivateSubnet-VpcA

  EC2SecurityGroup:
    Value: !Ref EC2SecurityGroup
    Export:
      Name: EC2SecurityGroup-VpcA

VPCB.yml

AWSTemplateFormatVersion: "2010-09-09"
Description: "VPC-B Template."

Parameters:
  # ------------------------------------------------------------#
  # Common
  # ------------------------------------------------------------#
  Prefix:
    Type: String
    Default: "vpc-b"

  # ------------------------------------------------------------#
  # Network
  # ------------------------------------------------------------#
  VpcCidr:
    Type: String
    Default: "10.1.0.0/16"

  PrivateSubnetCidrA:
    Type: String
    Default: "10.1.0.0/24"
  
  PrivateSubnetCidrC:
    Type: String
    Default: "10.1.2.0/24"

  # ------------------------------------------------------------#
  # EC2
  # ------------------------------------------------------------#
  EC2InstanceName:
    Type: String
    Default: "ec2"
  EC2InstanceAMI:
    Type: AWS::EC2::Image::Id
    Default: "ami-078296f82eb463377" # Amazon Linux 2 AMI (HVM) - Kernel 5.10, SSD Volume Type
  EC2InstanceInstanceType:
    Type: String
    Default: "t3.nano"
  EC2InstanceVolumeType:
    Type: String
    Default: "gp2"
  EC2InstanceVolumeSize:
    Type: String
    Default: "8"

Resources:
  # ------------------------------------------------------------#
  #  Network
  # ------------------------------------------------------------#
  Vpc:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: !Ref VpcCidr
      EnableDnsSupport: true
      EnableDnsHostnames: true
      Tags:
        - Key: Name
          Value: !Sub ${Prefix}-vpc

  PrivateSubnetA:
    Type: AWS::EC2::Subnet
    Properties:
      CidrBlock: !Ref PrivateSubnetCidrA
      VpcId: !Ref Vpc
      AvailabilityZone:
        Fn::Select:
          - "0"
          - Fn::GetAZs: ""
      Tags:
        - Key: Name
          Value: !Sub ${Prefix}-private-subnet-a

  PrivateSubnetC:
    Type: AWS::EC2::Subnet
    Properties:
      CidrBlock: !Ref PrivateSubnetCidrC
      VpcId: !Ref Vpc
      AvailabilityZone:
        Fn::Select:
          - "1"
          - Fn::GetAZs: ""
      Tags:
        - Key: Name
          Value: !Sub ${Prefix}-private-subnet-c

  PrivateRouteTable: 
    Type: AWS::EC2::RouteTable
    Properties: 
      VpcId: !Ref Vpc 
      Tags: 
        - Key: Name
          Value: !Sub "${Prefix}-table"
  
  PrivateSubnetTableAssociationA: 
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties: 
      SubnetId: !Ref PrivateSubnetA
      RouteTableId: !Ref PrivateRouteTable

  PrivateSubnetTableAssociationC: 
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties: 
      SubnetId: !Ref PrivateSubnetC
      RouteTableId: !Ref PrivateRouteTable

  EC2SecurityGroup:
    Type: "AWS::EC2::SecurityGroup"
    Properties:
      VpcId: !Ref Vpc
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 80
          ToPort: 80
          CidrIp: "0.0.0.0/0"
      GroupName: !Sub "${Prefix}-ec2-sg"
      GroupDescription: "-"
      Tags:
        - Key: "Name"
          Value: !Sub "${Prefix}-ec2-sg"

  ALBSecurityGroup:
    Type: "AWS::EC2::SecurityGroup"
    Properties:
      VpcId: !Ref Vpc
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 80
          ToPort: 80
          CidrIp: "0.0.0.0/0"
      GroupName: !Sub "${Prefix}-alb-sg"
      GroupDescription: "-"
      Tags:
        - Key: "Name"
          Value: !Sub "${Prefix}-alb-sg"

  # ------------------------------------------------------------#
  #  VPC Endpoint
  # ------------------------------------------------------------#
  S3VPCEndpoint:
    Type: "AWS::EC2::VPCEndpoint"
    Properties:
      RouteTableIds: 
        - !Ref PrivateRouteTable
      ServiceName: !Sub "com.amazonaws.${AWS::Region}.s3"
      VpcEndpointType: Gateway
      VpcId: !Ref Vpc 

  # ------------------------------------------------------------#
  #  EC2Instance
  # ------------------------------------------------------------#
  EC2Instance1:
    Type: "AWS::EC2::Instance"
    Properties:
      Tags:
        - Key: Name
          Value: !Sub "${Prefix}-${EC2InstanceName}"
      ImageId: !Ref EC2InstanceAMI
      InstanceType: !Ref EC2InstanceInstanceType
      DisableApiTermination: false
      EbsOptimized: false
      BlockDeviceMappings:
        - DeviceName: /dev/xvda
          Ebs:
            DeleteOnTermination: true
            VolumeType: !Ref EC2InstanceVolumeType
            VolumeSize: !Ref EC2InstanceVolumeSize
      SecurityGroupIds:
        - !Ref EC2SecurityGroup
      SubnetId: !Ref PrivateSubnetA
      UserData: !Base64 |
        #! /bin/bash
        sudo yum update -y
        sudo yum -y install httpd
        sudo systemctl start httpd
        sudo systemctl enable httpd
        sudo echo "Hello World" >> /var/www/html/index.html
 
  EC2Instance2:
    Type: "AWS::EC2::Instance"
    Properties:
      Tags:
        - Key: Name
          Value: !Sub "${Prefix}-${EC2InstanceName}"
      ImageId: !Ref EC2InstanceAMI
      InstanceType: !Ref EC2InstanceInstanceType
      DisableApiTermination: false
      EbsOptimized: false
      BlockDeviceMappings:
        - DeviceName: /dev/xvda
          Ebs:
            DeleteOnTermination: true
            VolumeType: !Ref EC2InstanceVolumeType
            VolumeSize: !Ref EC2InstanceVolumeSize
      SecurityGroupIds:
        - !Ref EC2SecurityGroup
      SubnetId: !Ref PrivateSubnetC
      UserData: !Base64 |
        #! /bin/bash
        sudo yum update -y
        sudo yum -y install httpd
        sudo systemctl start httpd
        sudo systemctl enable httpd
        sudo echo "Hello World" >> /var/www/html/index.html

  # ------------------------------------------------------------#
  #  ALB
  # ------------------------------------------------------------#  
  ALBTargetGroup: 
    Type: AWS::ElasticLoadBalancingV2::TargetGroup
    Properties: 
      VpcId: !Ref Vpc
      Name: !Sub "${Prefix}-alb-tg"
      Protocol: HTTP
      Port: 80
      Tags: 
        - Key: Name
          Value: !Sub "${Prefix}-alb-tg"
      Targets: 
        - Id: !Ref EC2Instance1
          Port: 80
        - Id: !Ref EC2Instance2
          Port: 80
   
  ALB: 
    Type: AWS::ElasticLoadBalancingV2::LoadBalancer
    Properties: 
      IpAddressType: ipv4
      Name: !Sub "${Prefix}-alb"
      Tags: 
        - Key: Name
          Value: !Sub "${Prefix}-alb"
      Scheme: "internal"
      SecurityGroups:
        - !Ref ALBSecurityGroup
      Subnets: 
        - !Ref PrivateSubnetA
        - !Ref PrivateSubnetC
      Type: application

  ALBListener: 
    Type: AWS::ElasticLoadBalancingV2::Listener
    Properties: 
      DefaultActions: 
        - TargetGroupArn: !Ref ALBTargetGroup
          Type: forward
      LoadBalancerArn: !Ref ALB
      Port: 80
      Protocol: HTTP

  # ------------------------------------------------------------#
  #  NLB
  # ------------------------------------------------------------#  
  NLBTargetGroup: 
    DependsOn:
      - ALBListener
    Type: AWS::ElasticLoadBalancingV2::TargetGroup
    Properties: 
      VpcId: !Ref Vpc
      Name: !Sub "${Prefix}-nlb-tg"
      Protocol: TCP
      Port: 80
      Tags: 
        - Key: Name
          Value: !Sub "${Prefix}-nlb-tg"
      Targets: 
        - Id: !Ref ALB
          Port: 80
      TargetType: alb
   
  NLB: 
    Type: AWS::ElasticLoadBalancingV2::LoadBalancer
    Properties: 
      Name: !Sub "${Prefix}-nlb"
      Tags: 
        - Key: Name
          Value: !Sub "${Prefix}-nlb"
      Scheme: "internal"
      LoadBalancerAttributes: 
        - Key: "deletion_protection.enabled"
          Value: false
      Subnets: 
        - !Ref PrivateSubnetA
        - !Ref PrivateSubnetC
      Type: network

  NLBListener: 
    Type: AWS::ElasticLoadBalancingV2::Listener
    Properties: 
      DefaultActions: 
        - TargetGroupArn: !Ref NLBTargetGroup
          Type: forward
      LoadBalancerArn: !Ref NLB
      Port: 80
      Protocol: TCP

# ------------------------------------------------------------#
# Output Parameters
# ------------------------------------------------------------#   
Outputs:
  NLB:
    Value: !Ref NLB
    Export:
      Name: NLB

NLBVPCEndpoint.yml

AWSTemplateFormatVersion: "2010-09-09"
Description: "NLBVPCEndpoint Template."

Parameters:
  # ------------------------------------------------------------#
  # Common
  # ------------------------------------------------------------#
  Prefix:
    Type: String
    Default: "nlb-vpcendpoint"

Resources:
  # ------------------------------------------------------------#
  #  VPC Endpoint
  # ------------------------------------------------------------#
  NLBVPCEndpointService: 
    Type: "AWS::EC2::VPCEndpointService"
    Properties: 
      AcceptanceRequired: true
      NetworkLoadBalancerArns: 
        - !ImportValue NLB
  
  NLBVPCEndpointSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: !Sub ${Prefix}-nlb-vpc-endpoint-sg
      GroupName: !Sub ${Prefix}-nlb-vpc-endpoint-sg
      VpcId: !ImportValue VpcA
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 80
          ToPort: 80
          SourceSecurityGroupId: !ImportValue EC2SecurityGroup-VpcA
      Tags:
        - Key: Name
          Value: !Sub ${Prefix}-nlb-vpc-endpoint-sg

  NLBVPCEndpoint:
    DependsOn:
      - NLBVPCEndpointService
    Type: AWS::EC2::VPCEndpoint
    Properties:
      VpcEndpointType: Interface
      PrivateDnsEnabled: false
      ServiceName: !Sub com.amazonaws.vpce.${AWS::Region}.${NLBVPCEndpointService}
      VpcId: !ImportValue VpcA
      SubnetIds:
        - !ImportValue PrivateSubnet-VpcA
      SecurityGroupIds:
        - !Ref NLBVPCEndpointSecurityGroup

CloudFormation実行後には、エンドポイントサービスからエンドポイント接続を承認する必要があります。
そのため、下記の公式ドキュメントに記載の通りマネジメントコンソールもしくはAWS CLI経由で承認してあげましょう。

エンドポイントの接続リクエストを承諾または拒否する

接続確認

VPC AのEC2にSession Manager経由で接続します。
下記のコマンド通りNLBのエンドポイントにcurlをして、VPC Bへの接続を確認します。

 % curl <NLBのエンドポイント>

無事、Hello Worldと出力されました！

ハマったところ

ALBのヘルスチェックに403エラーで失敗する。

理由はVPC B側のEC2にindex.htmlを作成していないためでした。
そのため、ヘルスチェックをデフォルトの"/"にしている場合、apacheだと初期設定の/var/www/htmlにインデックスページを作成する必要があります。

NLBのターゲットグループ作成に失敗してしまう。

このようにCloudFormation実行が失敗しました。

理由はALB作成前にNLBのターゲットグループを作成しようとしているためでした。そのため、NLBのターゲットグループにDependsOn属性で依存関係を明示してあげましょう。

NLBTargetGroup: 
    DependsOn:
      - ALBListener
    Type: AWS::ElasticLoadBalancingV2::TargetGroup
    Properties: 
      VpcId: !Ref Vpc
      Name: !Sub "${Prefix}-nlb-tg"
      Protocol: TCP
      Port: 80
      Tags: 
        - Key: Name
          Value: !Sub "${Prefix}-nlb-tg"
      Targets: 
        - Id: !Ref ALB
          Port: 80
      TargetType: alb