![TerraformのCI/CDパイプラインを実装してみた](https://devio2023-media.developers.io/wp-content/uploads/2019/05/terraform-eyecatch.png)
TerraformのCI/CDパイプラインを実装してみた
こんにちは!AWS事業本部コンサルティング部のたかくに(@takakuni_)です。
今回は、TerraformのCI/CDパイプラインでどのような構成が取れるか考えてみました。
きっかけは、最近tfsecを使う機会がよくあり、CI/CDに組み込むと面白そうだなと思い、GWの宿題として考えてみました。
気がついたら、6月の終盤で超大作ブログになっていました。是非ともTerraformユーザーの方にご覧いただけるととても嬉しいです。
今回の構成が「必ずしも正解」というわけではなく、あくまで一例として参考程度にご覧いただけると幸いです。
「tfsec」って何?と言う方は以下のサイトも合わせてご覧いただけると幸いです。
全体の構成図
以下の構成図のようなTerraform実行パイプラインを作成しようと思います。
パイプラインが少し長いため、以下の区分で実装方式や解説をまとめていこうと思います。
- CodeCommit、CodeBuild(tfsec)の実装
- Backend(S3, DynamoDB)の実装
- CodeBuild(plan)、CodeBuild(apply)の実装
- CodePipelineで結合
実装はCloudFormationで行います。「Terraform使わずにCloudFormationでIaCやればいいじゃん」はその通りですがご容赦いただければと思います。
最終的なyamlファイルは以下になります。あまりにも長いため折り返しています。
ファイル全体(クリックで表示できます)
AWSTemplateFormatVersion: '2010-09-09' Description: Terraform pipeline template. Metadata: AWS::CloudFormation::Interface: ParameterGroups: - Label: default: "Project Name Prefix" Parameters: - PrjPrefix - Label: default: "Repository Configuration" Parameters: - BranchName - Label: default: "Docker Login Profile" Parameters: - DockerHubUserName - DockerHubUserPassword - Label: default: "Logs Configuration" Parameters: - BuildLogsRationDay - Label: default: "terraform build configuration" Parameters: - TerraformVersion Parameters: PrjPrefix: Type: String Description: "Enter resource prefix. (ex. terraform-dev)" Default: "terraform-dev" BranchName: Type: String Description: "Enter the name of the branch where you want to store exclude.yml" Default: "main" DockerHubUserName: Type: String Description: "Enter Docker Hub loging user name. (ex. terraform)" NoEcho: true DockerHubUserPassword: Type: String Description: "Enter Docker Hub loging user password. (ex. P@ssw0rd) If you enable MFA in Docker Hub, generate AccessToken and enter instead of user password." NoEcho: true BuildLogsRationDay: Type: Number Description: "Enter build log retention period." Default: 90 AllowedValues: - 1 - 3 - 5 - 7 - 14 - 30 - 60 - 90 - 120 - 150 - 180 - 365 - 400 - 545 - 731 - 1827 - 3653 TerraformVersion: Type: String Description: "Enter the version of Terraform that use with CodeBuild." Default: "1.2.0" Resources: ################################# # KMS (CloudWatch Logs) ################################# KeyCWL: Type: AWS::KMS::Key Properties: Description: "Terraform pipeline Build Logs Key" Enabled: true EnableKeyRotation: true KeyPolicy: Version: "2012-10-17" Statement: - Sid: "Enable IAM User Permissions" Effect: "Allow" Principal: AWS: !Sub "arn:${AWS::Partition}:iam::${AWS::AccountId}:root" Action: "kms:*" Resource: "*" - Sid: Allow use of the key from CWL Effect: "Allow" Principal: Service: !Sub "logs.${AWS::Region}.amazonaws.com" Action: - "kms:Encrypt" - "kms:Decrypt" - "kms:ReEncrypt" - "kms:GenerateDataKey" - "kms:Describe" Resource: "*" Condition: ArnLike: kms:EncryptionContext:aws:logs:arn: !Sub "arn:${AWS::Partition}:logs:${AWS::Region}:${AWS::AccountId}:*" KeySpec: "SYMMETRIC_DEFAULT" KeyUsage: "ENCRYPT_DECRYPT" Tags: - Key: "Name" Value: !Sub "${PrjPrefix}-tf-pipeline-logs-key" KeyAliasCWL: Type: AWS::KMS::Alias Properties: AliasName: !Sub "alias/${PrjPrefix}-tf-pipeline-logs-key" TargetKeyId: !Ref KeyCWL ################################# # KMS (S3 arthifact) ################################# KeyS3Arthifact: Type: AWS::KMS::Key Properties: Description: "Terraform pipeline arthifact Key" Enabled: true EnableKeyRotation: true KeyPolicy: Version: "2012-10-17" Statement: - Sid: "Enable IAM User Permissions" Effect: "Allow" Principal: AWS: !Sub "arn:${AWS::Partition}:iam::${AWS::AccountId}:root" Action: "kms:*" Resource: "*" KeySpec: "SYMMETRIC_DEFAULT" KeyUsage: "ENCRYPT_DECRYPT" Tags: - Key: "Name" Value: !Sub "${PrjPrefix}-tf-pipeline-artifact-key" KeyAliasS3Arthifact: Type: AWS::KMS::Alias Properties: AliasName: !Sub "alias/${PrjPrefix}-tf-pipeline-artifact-key" TargetKeyId: !Ref KeyS3Arthifact ################################# # KMS (S3 tfstate) ################################# KeyS3Tfstate: Type: AWS::KMS::Key Properties: Description: "Terraform pipeline tfstate Key" Enabled: true EnableKeyRotation: true KeyPolicy: Version: "2012-10-17" Statement: - Sid: "Enable IAM User Permissions" Effect: "Allow" Principal: AWS: !Sub "arn:${AWS::Partition}:iam::${AWS::AccountId}:root" Action: "kms:*" Resource: "*" KeySpec: "SYMMETRIC_DEFAULT" KeyUsage: "ENCRYPT_DECRYPT" Tags: - Key: "Name" Value: !Sub "${PrjPrefix}-tf-pipeline-tfstate-key" KeyAliasS3Tfstate: Type: AWS::KMS::Alias Properties: AliasName: !Sub "alias/${PrjPrefix}-tf-pipeline-tfstate-key" TargetKeyId: !Ref KeyS3Tfstate ################################# # KMS (Secrets Manager) ################################# KeySecretsManager: Type: AWS::KMS::Key Properties: Description: "Terraform pipeline Secrets Manager Key" Enabled: true EnableKeyRotation: true KeyPolicy: Version: "2012-10-17" Statement: - Sid: "Enable IAM User Permissions" Effect: "Allow" Principal: AWS: !Sub "arn:${AWS::Partition}:iam::${AWS::AccountId}:root" Action: "kms:*" Resource: "*" KeySpec: "SYMMETRIC_DEFAULT" KeyUsage: "ENCRYPT_DECRYPT" Tags: - Key: "Name" Value: !Sub "${PrjPrefix}-tf-pipeline-secretsmanager-key" KeyAliasSecretsManager: Type: AWS::KMS::Alias Properties: AliasName: !Sub "alias/${PrjPrefix}-tf-pipeline-secretsmanager-key" TargetKeyId: !Ref KeySecretsManager ################################# # KMS (DynamoDB) ################################# KeyDynamoDB: Type: AWS::KMS::Key Properties: Description: "Terraform pipeline DynamoDB Key" Enabled: true EnableKeyRotation: true KeyPolicy: Version: "2012-10-17" Statement: - Sid: "Enable IAM User Permissions" Effect: "Allow" Principal: AWS: !Sub "arn:${AWS::Partition}:iam::${AWS::AccountId}:root" Action: "kms:*" Resource: "*" KeySpec: "SYMMETRIC_DEFAULT" KeyUsage: "ENCRYPT_DECRYPT" Tags: - Key: "Name" Value: !Sub "${PrjPrefix}-tf-pipeline-dynamoDB-key" KeyAliasDynamoDB: Type: AWS::KMS::Alias Properties: AliasName: !Sub "alias/${PrjPrefix}-tf-pipeline-dynamoDB-key" TargetKeyId: !Ref KeyDynamoDB ################################# # CodeCommit ################################# CodeCommit: Type: AWS::CodeCommit::Repository Properties: RepositoryName: !Sub "${PrjPrefix}-tf-repo" RepositoryDescription: !Sub "${PrjPrefix}-tf-repo" Tags: - Key: "Name" Value: !Sub "${PrjPrefix}-tf-repo" ################################# # Custom Resource ################################# PutExcludeRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Version: "2012-10-17" Statement: - Effect: Allow Principal: Service: lambda.amazonaws.com Action: sts:AssumeRole Path: / ManagedPolicyArns: - "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole" - "arn:aws:iam::aws:policy/AWSCodeCommitPowerUser" RoleName: !Sub "${PrjPrefix}-tf-repo-${BranchName}-put-exclude-role" PutExcludeLog: Type: AWS::Logs::LogGroup Properties: LogGroupName: !Sub "/aws/lambda/${PrjPrefix}-tf-repo-${BranchName}-put-exclude" KmsKeyId: !GetAtt KeyCWL.Arn PutExcludeFunction: Type: AWS::Lambda::Function Properties: FunctionName: !Sub "${PrjPrefix}-tf-repo-${BranchName}-put-exclude" Code: ZipFile: | import json import boto3 import cfnresponse def handler(event, context): try: repository = event['ResourceProperties']['RepositoryName'] branch = event['ResourceProperties']['BranchName'] content = event['ResourceProperties']['FileContent'].encode() path = event['ResourceProperties']['FilePath'] if event['RequestType'] == 'Create': codecommit = boto3.client('codecommit') response = codecommit.put_file( repositoryName=repository, branchName=branch, fileContent=content, filePath=path, commitMessage='Initial Commit', name='Your Lambda Helper' ) cfnresponse.send(event, context, cfnresponse.SUCCESS, response) if event['RequestType'] == 'Delete': cfnresponse.send(event, context, cfnresponse.SUCCESS, {'Response': 'Success'}) if event['RequestType'] == 'Update': cfnresponse.send(event, context, cfnresponse.SUCCESS, {'Response': 'Success'}) except Exception as e: print(e) cfnresponse.send(event, context, cfnresponse.FAILED, {}) Handler: index.handler MemorySize: 128 Role: !GetAtt PutExcludeRole.Arn Runtime: "python3.9" Timeout: 60 Tags: - Key: "Name" Value: !Sub "${PrjPrefix}-tf-repo-${BranchName}-put-exclude" DependsOn: PutExcludeLog PutExclude: Type: Custom::CodeCommitPutExclude Properties: ServiceToken: !GetAtt PutExcludeFunction.Arn RepositoryName: !GetAtt CodeCommit.Name BranchName: !Ref BranchName FileContent: "---\nexclude:" FilePath: "exclude.yml" ################################# # Secrets Manager ################################# DockerLoginProfile: Type: AWS::SecretsManager::Secret Properties: Description: "Docker login profile for terraform pipelines" Name: !Sub "${PrjPrefix}/docker_hub" SecretString: !Sub '{"username":"${DockerHubUserName}","password":"${DockerHubUserPassword}"}' KmsKeyId: !GetAtt KeySecretsManager.Arn Tags: - Key: "Name" Value: !Sub "${PrjPrefix}/docker_hub" ################################# # CloudWatch Logs (CodeBuild tfsec) ################################# CWLTfsec: Type: AWS::Logs::LogGroup Properties: LogGroupName: !Sub "/aws/codebuild/${PrjPrefix}-tf-build-tfsec-project" RetentionInDays: !Ref BuildLogsRationDay KmsKeyId: !GetAtt KeyCWL.Arn Tags: - Key: "Name" Value: !Sub "/aws/codebuild/${PrjPrefix}-tf-build-tfsec-project" ################################# # S3 Bucket (Artifact) ################################# BucketArtifacts: Type: 'AWS::S3::Bucket' Properties: BucketName: !Sub "${PrjPrefix}-tf-pipeline-artifacts" BucketEncryption: ServerSideEncryptionConfiguration: - ServerSideEncryptionByDefault: SSEAlgorithm: "aws:kms" KMSMasterKeyID: !GetAtt KeyS3Arthifact.Arn BucketKeyEnabled: true PublicAccessBlockConfiguration: BlockPublicAcls: true BlockPublicPolicy: true IgnorePublicAcls: true RestrictPublicBuckets: true Tags: - Key: "Name" Value: !Sub "${PrjPrefix}-tf-pipeline-artifacts" BucketPolicyArtifacts: Type: AWS::S3::BucketPolicy Properties: Bucket: !Ref BucketArtifacts PolicyDocument: Version: '2012-10-17' Statement: - Sid: "AllowSSLRequestsOnly" Effect: "Deny" Principal: "*" Action: "s3:*" Resource: - !GetAtt BucketArtifacts.Arn - !Sub - "${BucketArn}/*" - { BucketArn: !GetAtt BucketArtifacts.Arn } Condition: Bool: aws:SecureTransport: "false" ################################# # CodeBuild (tfsec) ################################# PolicyTfsec: Type: AWS::IAM::ManagedPolicy Properties: ManagedPolicyName: !Sub "${PrjPrefix}-tf-build-tfsec-project-policy" PolicyDocument: Version: "2012-10-17" Statement: - Sid: "BuildLogs" Effect: "Allow" Action: - "logs:CreateLogGroup" - "logs:CreateLogStream" - "logs:PutLogEvents" Resource: - !Sub "arn:${AWS::Partition}:logs:${AWS::Region}:${AWS::AccountId}:log-group:${CWLTfsec}" - !GetAtt CWLTfsec.Arn - Sid: "BuildReports" Effect: "Allow" Action: - "codebuild:CreateReportGroup" - "codebuild:CreateReport" - "codebuild:UpdateReport" - "codebuild:BatchPutTestCases" - "codebuild:BatchPutCodeCoverages" Resource: - !Sub "arn:${AWS::Partition}:codebuild:${AWS::Region}:${AWS::AccountId}:report-group/${PrjPrefix}-tf-build-tfsec-project-reports" - Sid: "GitPull" Effect: "Allow" Action: "codecommit:GitPull" Resource: !GetAtt CodeCommit.Arn - Sid: "GetSecretValue" Effect: "Allow" Action: "secretsmanager:GetSecretValue" Resource: !Ref DockerLoginProfile - Sid: "KmsKey" Effect: "Allow" Action: - "kms:Decrypt" - "kms:DescribeKey" - "kms:Encrypt" - "kms:ReEncrypt" - "kms:GenerateDataKey" Resource: - !GetAtt KeyS3Arthifact.Arn - !GetAtt KeySecretsManager.Arn - Sid: "S3Artifact" Effect: "Allow" Action: - "s3:PutObject" - "s3:GetObject" - "s3:GetObjectVersion" - "s3:GetBucketAcl" - "s3:GetBucketLocation" Resource: - !GetAtt BucketArtifacts.Arn - !Sub - "${BucketArn}/*" - { BucketArn: !GetAtt BucketArtifacts.Arn } RoleTfsec: Type: AWS::IAM::Role Properties: RoleName: !Sub "${PrjPrefix}-tf-build-tfsec-project-role" AssumeRolePolicyDocument: Version: "2012-10-17" Statement: - Effect: "Allow" Principal: Service: "codebuild.amazonaws.com" Action: "sts:AssumeRole" ManagedPolicyArns: - !Ref PolicyTfsec Tags: - Key: "Name" Value: !Sub "${PrjPrefix}-tf-build-tfsec-project-role" ProjectTfsec: Type: AWS::CodeBuild::Project Properties: Name: !Sub "${PrjPrefix}-tf-build-tfsec-project" Description: "Analyze the code for vulnerabilities using tfsec." Source: Type: "CODEPIPELINE" BuildSpec: | version: 0.2 env: exported-variables: - BuildID - BuildTag phases: pre_build: commands: - "echo Executing tfsec" - "mkdir -p reports/tfsec/" build: commands: - "tfsec -s --no-color --config-file exclude.yml ." - "tfsec -s --no-color --config-file exclude.yml . --format junit > reports/tfsec/report.xml" post_build: commands: - "export BuildID=`echo $CODEBUILD_BUILD_ID | cut -d: -f1`" - "export BuildTag=`echo $CODEBUILD_BUILD_ID | cut -d: -f2`" reports: reports: files: - "reports/tfsec/report.xml" file-format: JUNITXML Artifacts: Type: "CODEPIPELINE" Cache: Type: "LOCAL" Modes: - "LOCAL_DOCKER_LAYER_CACHE" Environment: Type: "LINUX_CONTAINER" ComputeType: "BUILD_GENERAL1_SMALL" Image: "aquasec/tfsec:latest" ImagePullCredentialsType: "SERVICE_ROLE" RegistryCredential: Credential: !Ref DockerLoginProfile CredentialProvider: "SECRETS_MANAGER" PrivilegedMode: true LogsConfig: CloudWatchLogs: Status: "ENABLED" GroupName: !Ref CWLTfsec EncryptionKey: !GetAtt KeyS3Arthifact.Arn ResourceAccessRole: !GetAtt RoleTfsec.Arn ServiceRole: !GetAtt RoleTfsec.Arn TimeoutInMinutes: 60 Visibility: "PRIVATE" Tags: - Key: "Name" Value: !Sub "${PrjPrefix}-tf-build-tfsec-project" ReportGroupTfsec: Type: AWS::CodeBuild::ReportGroup Properties: Name: !Sub "${ProjectTfsec}-reports" Type: "TEST" DeleteReports: true ExportConfig: ExportConfigType: "NO_EXPORT" Tags: - Key: "Name" Value: !Sub "${ProjectTfsec}-reports" ################################# # S3 Bucket (tfstate) ################################# BucketTfstate: Type: 'AWS::S3::Bucket' Properties: BucketName: !Sub "${PrjPrefix}-tf-pipeline-tfstate" BucketEncryption: ServerSideEncryptionConfiguration: - ServerSideEncryptionByDefault: SSEAlgorithm: "aws:kms" KMSMasterKeyID: !GetAtt KeyS3Tfstate.Arn BucketKeyEnabled: true PublicAccessBlockConfiguration: BlockPublicAcls: true BlockPublicPolicy: true IgnorePublicAcls: true RestrictPublicBuckets: true VersioningConfiguration: Status: "Enabled" Tags: - Key: "Name" Value: !Sub "${PrjPrefix}-tf-pipeline-tfstate" BucketPolicyTfstate: Type: AWS::S3::BucketPolicy Properties: Bucket: !Ref BucketTfstate PolicyDocument: Version: '2012-10-17' Statement: - Sid: "AllowSSLRequestsOnly" Effect: "Deny" Principal: "*" Action: "s3:*" Resource: - !GetAtt BucketTfstate.Arn - !Sub - "${BucketArn}/*" - { BucketArn: !GetAtt BucketTfstate.Arn } Condition: Bool: aws:SecureTransport: "false" ################################# # DynamoDB table (state lock) ################################# DDBTable: Type: AWS::DynamoDB::Table Properties: TableName: !Sub "${PrjPrefix}-tf-pipeline-state-lock-table" KeySchema: - AttributeName: "LockID" KeyType: "HASH" AttributeDefinitions: - AttributeName: "LockID" AttributeType: "S" BillingMode: "PAY_PER_REQUEST" PointInTimeRecoverySpecification: PointInTimeRecoveryEnabled: true SSESpecification: KMSMasterKeyId: !GetAtt KeyDynamoDB.Arn SSEEnabled: true SSEType: "KMS" Tags: - Key: "Name" Value: !Sub "${PrjPrefix}-tf-pipeline-state-lock-table" ################################# # CloudWatch Logs (CodeBuild terraform plan) ################################# CWLTfplan: Type: AWS::Logs::LogGroup Properties: LogGroupName: !Sub "/aws/codebuild/${PrjPrefix}-tf-build-tfplan-project" RetentionInDays: !Ref BuildLogsRationDay KmsKeyId: !GetAtt KeyCWL.Arn Tags: - Key: "Name" Value: !Sub "/aws/codebuild/${PrjPrefix}-tf-build-tfplan-project" ################################# # CodeBuild (terraform plan) ################################# PolicyTfplan: Type: AWS::IAM::ManagedPolicy Properties: ManagedPolicyName: !Sub "${PrjPrefix}-tf-build-tfplan-project-policy" PolicyDocument: Version: "2012-10-17" Statement: - Sid: "BuildLogs" Effect: "Allow" Action: - "logs:CreateLogGroup" - "logs:CreateLogStream" - "logs:PutLogEvents" Resource: - !Sub "arn:${AWS::Partition}:logs:${AWS::Region}:${AWS::AccountId}:log-group:${CWLTfplan}" - !GetAtt CWLTfplan.Arn - Sid: "KmsKey" Effect: "Allow" Action: - "kms:Decrypt" - "kms:Encrypt" - "kms:ReEncrypt" - "kms:GenerateDataKey" Resource: - !GetAtt KeyS3Arthifact.Arn - !GetAtt KeyS3Tfstate.Arn - !GetAtt KeyDynamoDB.Arn - Sid: "S3Tfstate" Effect: "Allow" Action: - "s3:PutObject*" - "s3:DeleteObject*" Resource: - !Sub - "${BucketArn}/*" - { BucketArn: !GetAtt BucketTfstate.Arn } - Sid: "DynamoDB" Effect: "Allow" Action: - "dynamodb:PutItem" - "dynamodb:DeleteItem" Resource: !GetAtt DDBTable.Arn RoleTfplan: Type: AWS::IAM::Role Properties: RoleName: !Sub "${PrjPrefix}-tf-build-tfplan-project-role" AssumeRolePolicyDocument: Version: "2012-10-17" Statement: - Effect: "Allow" Principal: Service: "codebuild.amazonaws.com" Action: "sts:AssumeRole" ManagedPolicyArns: - !Ref PolicyTfplan - "arn:aws:iam::aws:policy/ReadOnlyAccess" Tags: - Key: "Name" Value: !Sub "${PrjPrefix}-tf-build-tfplan-project-role" ProjectTfplan: Type: AWS::CodeBuild::Project Properties: Name: !Sub "${PrjPrefix}-tf-build-tfplan-project" Description: "Execute terraform plan command." Source: Type: "CODEPIPELINE" BuildSpec: | version: 0.2 env: exported-variables: - BuildID - BuildTag phases: install: runtime-versions: golang: 1.14 commands: - "git clone https://github.com/tfutils/tfenv.git ~/.tfenv" - "ln -s ~/.tfenv/bin/* /usr/local/bin" - "tfenv install $TF_VERSION" - "tfenv use $TF_VERSION" pre_build: commands: - "terraform init -input=false -no-color" build: commands: - "terraform plan -input=false -no-color" post_build: commands: - "export BuildID=`echo $CODEBUILD_BUILD_ID | cut -d: -f1`" - "export BuildTag=`echo $CODEBUILD_BUILD_ID | cut -d: -f2`" Artifacts: Type: "CODEPIPELINE" Cache: Type: "LOCAL" Modes: - "LOCAL_DOCKER_LAYER_CACHE" Environment: Type: "LINUX_CONTAINER" ComputeType: "BUILD_GENERAL1_SMALL" Image: "aws/codebuild/amazonlinux2-x86_64-standard:3.0" ImagePullCredentialsType: "CODEBUILD" PrivilegedMode: true EnvironmentVariables: - Name: "TF_VERSION" Type: PLAINTEXT Value: !Ref TerraformVersion LogsConfig: CloudWatchLogs: Status: "ENABLED" GroupName: !Ref CWLTfplan EncryptionKey: !GetAtt KeyS3Arthifact.Arn ResourceAccessRole: !GetAtt RoleTfplan.Arn ServiceRole: !GetAtt RoleTfplan.Arn TimeoutInMinutes: 60 Visibility: "PRIVATE" Tags: - Key: "Name" Value: !Sub "${PrjPrefix}-tf-build-tfplan-project" ################################# # CloudWatch Logs (CodeBuild terraform apply) ################################# CWLTfapply: Type: AWS::Logs::LogGroup Properties: LogGroupName: !Sub "/aws/codebuild/${PrjPrefix}-tf-build-tfapply-project-role" RetentionInDays: !Ref BuildLogsRationDay KmsKeyId: !GetAtt KeyCWL.Arn Tags: - Key: "Name" Value: !Sub "/aws/codebuild/${PrjPrefix}-tf-build-tfapply-project-role" ################################# # CodeBuild (terraform apply) ################################# RoleTfapply: Type: AWS::IAM::Role Properties: RoleName: !Sub "${PrjPrefix}-tf-build-tfapply-project-role" AssumeRolePolicyDocument: Version: "2012-10-17" Statement: - Effect: "Allow" Principal: Service: "codebuild.amazonaws.com" Action: "sts:AssumeRole" ManagedPolicyArns: - "arn:aws:iam::aws:policy/AdministratorAccess" Tags: - Key: "Name" Value: !Sub "${PrjPrefix}-tf-build-tfapply-project-role" ProjectTfapply: Type: AWS::CodeBuild::Project Properties: Name: !Sub "${PrjPrefix}-tf-build-tfapply-project" Description: "Execute terraform apply command." Source: Type: "CODEPIPELINE" BuildSpec: | version: 0.2 phases: install: runtime-versions: golang: 1.14 commands: - "git clone https://github.com/tfutils/tfenv.git ~/.tfenv" - "ln -s ~/.tfenv/bin/* /usr/local/bin" - "tfenv install $TF_VERSION" - "tfenv use $TF_VERSION" pre_build: commands: - "terraform init -input=false -no-color" build: commands: - "terraform apply -input=false -no-color -auto-approve" Artifacts: Type: "CODEPIPELINE" Cache: Type: "LOCAL" Modes: - "LOCAL_DOCKER_LAYER_CACHE" Environment: Type: "LINUX_CONTAINER" ComputeType: "BUILD_GENERAL1_SMALL" Image: "aws/codebuild/amazonlinux2-x86_64-standard:3.0" ImagePullCredentialsType: "CODEBUILD" PrivilegedMode: true EnvironmentVariables: - Name: "TF_VERSION" Type: PLAINTEXT Value: !Ref TerraformVersion LogsConfig: CloudWatchLogs: Status: "ENABLED" GroupName: !Ref CWLTfapply EncryptionKey: !GetAtt KeyS3Arthifact.Arn ResourceAccessRole: !GetAtt RoleTfapply.Arn ServiceRole: !GetAtt RoleTfapply.Arn TimeoutInMinutes: 60 Visibility: "PRIVATE" Tags: - Key: "Name" Value: !Sub "${PrjPrefix}-tf-build-tfapply-project" ################################# # CodePipeline ################################# PolicyTfPipeline: Type: AWS::IAM::ManagedPolicy Properties: ManagedPolicyName: !Sub "${PrjPrefix}-tf-pipeline-policy" PolicyDocument: Version: "2012-10-17" Statement: - Sid: "S3Artifact" Effect: "Allow" Action: - "s3:GetObject*" - "s3:GetBucket*" - "s3:List*" - "s3:DeleteObject*" - "s3:PutObject" - "s3:Abort*" Resource: - !GetAtt BucketArtifacts.Arn - !Sub - "${BucketArn}/*" - { BucketArn: !GetAtt BucketArtifacts.Arn } - Sid: "KmsKey" Effect: "Allow" Action: - "kms:Decrypt" - "kms:DescribeKey" - "kms:Encrypt" - "kms:ReEncrypt*" - "kms:GenerateDataKey*" Resource: - !GetAtt KeyS3Arthifact.Arn - Sid: "CodeCommitRepo" Effect: "Allow" Action: - "codecommit:GetBranch" - "codecommit:GetCommit" - "codecommit:UploadArchive" - "codecommit:GetUploadArchiveStatus" - "codecommit:CancelUploadArchive" Resource: - !GetAtt CodeCommit.Arn - Sid: "CodeBuildProjects" Effect: "Allow" Action: - "codebuild:BatchGetBuilds" - "codebuild:StartBuild" - "codebuild:StopBuild" Resource: - !GetAtt ProjectTfsec.Arn - !GetAtt ProjectTfplan.Arn - !GetAtt ProjectTfapply.Arn RoleTfPipelne: Type: AWS::IAM::Role Properties: RoleName: !Sub "${PrjPrefix}-tf-pipeline-role" AssumeRolePolicyDocument: Version: "2012-10-17" Statement: - Effect: "Allow" Principal: Service: "codepipeline.amazonaws.com" Action: "sts:AssumeRole" ManagedPolicyArns: - !Ref PolicyTfPipeline Tags: - Key: "Name" Value: !Sub "${PrjPrefix}-tf-pipeline-role" CodePipeline: Type: AWS::CodePipeline::Pipeline Properties: Name: !Sub "${PrjPrefix}-tf-pipeline" ArtifactStore: EncryptionKey: Id: !GetAtt KeyS3Arthifact.Arn Type: "KMS" Location: !Ref BucketArtifacts Type: "S3" RoleArn: !GetAtt RoleTfPipelne.Arn Stages: - Name: "Source" Actions: - Name: "CodeCommit_Source" ActionTypeId: Category: "Source" Owner: "AWS" Provider: "CodeCommit" Version: "1" Configuration: RepositoryName: !GetAtt CodeCommit.Name BranchName: !Ref BranchName PollForSourceChanges: false OutputArtifacts: - Name: "Artifact_Source_CodeCommit_Source" RoleArn: !GetAtt RoleTfPipelne.Arn RunOrder: 1 - Name: "tfsec_Stage" Actions: - Name: "Terraform_Security_Analysis" Namespace: TFSEC ActionTypeId: Category: "Build" Owner: "AWS" Provider: "CodeBuild" Version: "1" Configuration: ProjectName: !Ref ProjectTfsec InputArtifacts: - Name: "Artifact_Source_CodeCommit_Source" RoleArn: !GetAtt RoleTfPipelne.Arn RunOrder: 1 - Name: "Terraform_Stages" Actions: - Name: "Terraform_Security_Analysis_Manual_Review" ActionTypeId: Category: "Approval" Owner: "AWS" Provider: "Manual" Version: "1" Configuration: CustomData: "tfsec review" ExternalEntityLink: !Sub "https://${AWS::Region}.console.aws.amazon.com/codesuite/codebuild/${AWS::AccountId}/projects/#{TFSEC.BuildID}/build/#{TFSEC.BuildID}%3A#{TFSEC.BuildTag}/?region=${AWS::Region}" RoleArn: !GetAtt RoleTfPipelne.Arn RunOrder: 1 - Name: "Terraform_Plan" Namespace: "TERRAFORM" ActionTypeId: Category: "Build" Owner: "AWS" Provider: "CodeBuild" Version: "1" Configuration: ProjectName: !Ref ProjectTfplan InputArtifacts: - Name: "Artifact_Source_CodeCommit_Source" RoleArn: !GetAtt RoleTfPipelne.Arn RunOrder: 2 - Name: "Terraform_Plan_Manual_Review" ActionTypeId: Category: "Approval" Owner: "AWS" Provider: "Manual" Version: "1" Configuration: CustomData: "Terraform plan review" ExternalEntityLink: !Sub "https://${AWS::Region}.console.aws.amazon.com/codesuite/codebuild/${AWS::AccountId}/projects/#{TERRAFORM.BuildID}/build/#{TERRAFORM.BuildID}%3A#{TERRAFORM.BuildTag}/?region=${AWS::Region}" RoleArn: !GetAtt RoleTfPipelne.Arn RunOrder: 3 - Name: "Terraform_Apply" ActionTypeId: Category: "Build" Owner: "AWS" Provider: "CodeBuild" Version: "1" Configuration: ProjectName: !Ref ProjectTfapply InputArtifacts: - Name: "Artifact_Source_CodeCommit_Source" RoleArn: !GetAtt RoleTfPipelne.Arn RunOrder: 4 ################################# # EventBridge (CodeCommit State Change) # ################################# PolicyEventBridge: Type: AWS::IAM::ManagedPolicy Properties: ManagedPolicyName: !Sub "${PrjPrefix}-tf-pipeline-event-policy" PolicyDocument: Version: "2012-10-17" Statement: - Sid: "CodePipelineExec" Effect: "Allow" Action: "codepipeline:StartPipelineExecution" Resource: !Sub "arn:${AWS::Partition}:codepipeline:${AWS::Region}:${AWS::AccountId}:${CodePipeline}" RoleEventBridge: Type: AWS::IAM::Role Properties: RoleName: !Sub "${PrjPrefix}-tf-pipeline-event-role" AssumeRolePolicyDocument: Version: "2012-10-17" Statement: - Effect: "Allow" Principal: Service: "events.amazonaws.com" Action: "sts:AssumeRole" ManagedPolicyArns: - !Ref PolicyEventBridge Tags: - Key: "Name" Value: !Sub "${PrjPrefix}-tf-pipeline-event-role" EventsTfPipeline: Type: AWS::Events::Rule Properties: Name: !Sub "${PrjPrefix}-tf-pipeline-event" EventPattern: source: - "aws.codecommit" resources: - !GetAtt CodeCommit.Arn detail-type: - "CodeCommit Repository State Change" detail: event: - referenceCreated - referenceUpdated referenceType: - "branch" referenceName: - !Ref BranchName State: "ENABLED" Targets: - Arn: !Sub "arn:${AWS::Partition}:codepipeline:${AWS::Region}:${AWS::AccountId}:${CodePipeline}" Id: "TerraformPipeline" RoleArn: !GetAtt RoleEventBridge.Arn Outputs: TfstateBucketName: Value: !Ref BucketTfstate Export: Name: !Sub "${AWS::StackName}-TfstateBucketName" DDBTableName: Value: !Ref DDBTable Export: Name: !Sub "${AWS::StackName}-DDBTableName"
CodeCommit、CodeBuild(tfsec)の実装
まずは、以下の赤線の部分を実装します。
KMS
各種リソースの暗号化を行うためにKMSキーを作成します。
今回、暗号化するリソースは以下の通りです。
- CloudWatch Logs(ビルドログ、Lambdaログ)
- S3(アーティファクト、tfstate用バケット)
- Secrets Manager(Dockerのログイン情報)
- DynamoDB(terraformコマンドの排他制御)
今回は、シンプルにIAMリソースに権限を寄せる方向でいこうと思います。
実装部分
Resources: ################################# # KMS (CloudWatch Logs) ################################# KeyCWL: Type: AWS::KMS::Key Properties: Description: "Terraform pipeline Build Logs Key" Enabled: true EnableKeyRotation: true KeyPolicy: Version: "2012-10-17" Statement: - Sid: "Enable IAM User Permissions" Effect: "Allow" Principal: AWS: !Sub "arn:${AWS::Partition}:iam::${AWS::AccountId}:root" Action: "kms:*" Resource: "*" - Sid: Allow use of the key from CWL Effect: "Allow" Principal: Service: !Sub "logs.${AWS::Region}.amazonaws.com" Action: - "kms:Encrypt" - "kms:Decrypt" - "kms:ReEncrypt" - "kms:GenerateDataKey" - "kms:Describe" Resource: "*" Condition: ArnLike: kms:EncryptionContext:aws:logs:arn: !Sub "arn:${AWS::Partition}:logs:${AWS::Region}:${AWS::AccountId}:*" KeySpec: "SYMMETRIC_DEFAULT" KeyUsage: "ENCRYPT_DECRYPT" Tags: - Key: "Name" Value: !Sub "${PrjPrefix}-tf-pipeline-logs-key" KeyAliasCWL: Type: AWS::KMS::Alias Properties: AliasName: !Sub "alias/${PrjPrefix}-tf-pipeline-logs-key" TargetKeyId: !Ref KeyCWL ################################# # KMS (S3 arthifact) ################################# KeyS3Arthifact: Type: AWS::KMS::Key Properties: Description: "Terraform pipeline arthifact Key" Enabled: true EnableKeyRotation: true KeyPolicy: Version: "2012-10-17" Statement: - Sid: "Enable IAM User Permissions" Effect: "Allow" Principal: AWS: !Sub "arn:${AWS::Partition}:iam::${AWS::AccountId}:root" Action: "kms:*" Resource: "*" KeySpec: "SYMMETRIC_DEFAULT" KeyUsage: "ENCRYPT_DECRYPT" Tags: - Key: "Name" Value: !Sub "${PrjPrefix}-tf-pipeline-artifact-key" KeyAliasS3Arthifact: Type: AWS::KMS::Alias Properties: AliasName: !Sub "alias/${PrjPrefix}-tf-pipeline-artifact-key" TargetKeyId: !Ref KeyS3Arthifact ################################# # KMS (S3 tfstate) ################################# KeyS3Tfstate: Type: AWS::KMS::Key Properties: Description: "Terraform pipeline tfstate Key" Enabled: true EnableKeyRotation: true KeyPolicy: Version: "2012-10-17" Statement: - Sid: "Enable IAM User Permissions" Effect: "Allow" Principal: AWS: !Sub "arn:${AWS::Partition}:iam::${AWS::AccountId}:root" Action: "kms:*" Resource: "*" KeySpec: "SYMMETRIC_DEFAULT" KeyUsage: "ENCRYPT_DECRYPT" Tags: - Key: "Name" Value: !Sub "${PrjPrefix}-tf-pipeline-tfstate-key" KeyAliasS3Tfstate: Type: AWS::KMS::Alias Properties: AliasName: !Sub "alias/${PrjPrefix}-tf-pipeline-tfstate-key" TargetKeyId: !Ref KeyS3Tfstate ################################# # KMS (Secrets Manager) ################################# KeySecretsManager: Type: AWS::KMS::Key Properties: Description: "Terraform pipeline Secrets Manager Key" Enabled: true EnableKeyRotation: true KeyPolicy: Version: "2012-10-17" Statement: - Sid: "Enable IAM User Permissions" Effect: "Allow" Principal: AWS: !Sub "arn:${AWS::Partition}:iam::${AWS::AccountId}:root" Action: "kms:*" Resource: "*" KeySpec: "SYMMETRIC_DEFAULT" KeyUsage: "ENCRYPT_DECRYPT" Tags: - Key: "Name" Value: !Sub "${PrjPrefix}-tf-pipeline-secretsmanager-key" KeyAliasSecretsManager: Type: AWS::KMS::Alias Properties: AliasName: !Sub "alias/${PrjPrefix}-tf-pipeline-secretsmanager-key" TargetKeyId: !Ref KeySecretsManager ################################# # KMS (DynamoDB) ################################# KeyDynamoDB: Type: AWS::KMS::Key Properties: Description: "Terraform pipeline DynamoDB Key" Enabled: true EnableKeyRotation: true KeyPolicy: Version: "2012-10-17" Statement: - Sid: "Enable IAM User Permissions" Effect: "Allow" Principal: AWS: !Sub "arn:${AWS::Partition}:iam::${AWS::AccountId}:root" Action: "kms:*" Resource: "*" KeySpec: "SYMMETRIC_DEFAULT" KeyUsage: "ENCRYPT_DECRYPT" Tags: - Key: "Name" Value: !Sub "${PrjPrefix}-tf-pipeline-dynamoDB-key" KeyAliasDynamoDB: Type: AWS::KMS::Alias Properties: AliasName: !Sub "alias/${PrjPrefix}-tf-pipeline-dynamoDB-key" TargetKeyId: !Ref KeyDynamoDB
カイゼンできる部分
上記のキーポリシーでは、「スタックを作成するアカウントのルートユーザー」と、「KMSの操作権限を持っているIAMリソース」にキーの操作を許可するように定義されています。
もし、大きな権限を持ったIAMリソースが意図せず作成された場合、今回の鍵で暗号化したリソースは復号されてしまう恐れがあります。そのため、本番運用する場合はキーポリシーも適切に権限管理する必要があります。
キーポリシーも厳重に管理したい方は、以下を重ねてご覧ください。
CloudWatch Logs
S3
Secrets Manager
DynamoDB
CodeCommit
続いて、Terraformのコードを格納するCodeCommitレポジトリを作成します。
実装部分
Resources: ################################# # CodeCommit ################################# CodeCommit: Type: AWS::CodeCommit::Repository Properties: RepositoryName: !Sub "${PrjPrefix}-tf-repo" RepositoryDescription: !Sub "${PrjPrefix}-tf-repo" Tags: - Key: "Name" Value: !Sub "${PrjPrefix}-tf-repo"
リポジトリ作成時にファイルを作成する
今回、「exclude.yml」というファイルを、「Lambda-backed カスタムリソース」で自動的に作成します。
「exclude.yml」は、tfsecで検知されたルールを無視(ignore)する際に使用します。
カスタムリソースの詳しい実装方法は、以下ブログをご覧ください。
実装部分
Resources: ################################# # Custom Resource ################################# PutExcludeRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Version: "2012-10-17" Statement: - Effect: Allow Principal: Service: lambda.amazonaws.com Action: sts:AssumeRole Path: / ManagedPolicyArns: - "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole" - "arn:aws:iam::aws:policy/AWSCodeCommitPowerUser" RoleName: !Sub "${PrjPrefix}-tf-repo-${BranchName}-put-exclude-role" PutExcludeLog: Type: AWS::Logs::LogGroup Properties: LogGroupName: !Sub "/aws/lambda/${PrjPrefix}-tf-repo-${BranchName}-put-exclude" KmsKeyId: !GetAtt KeyCWL.Arn PutExcludeFunction: Type: AWS::Lambda::Function Properties: FunctionName: !Sub "${PrjPrefix}-tf-repo-${BranchName}-put-exclude" Code: ZipFile: | import json import boto3 import cfnresponse def handler(event, context): try: repository = event['ResourceProperties']['RepositoryName'] branch = event['ResourceProperties']['BranchName'] content = event['ResourceProperties']['FileContent'].encode() path = event['ResourceProperties']['FilePath'] if event['RequestType'] == 'Create': codecommit = boto3.client('codecommit') response = codecommit.put_file( repositoryName=repository, branchName=branch, fileContent=content, filePath=path, commitMessage='Initial Commit', name='Your Lambda Helper' ) cfnresponse.send(event, context, cfnresponse.SUCCESS, response) if event['RequestType'] == 'Delete': cfnresponse.send(event, context, cfnresponse.SUCCESS, {'Response': 'Success'}) if event['RequestType'] == 'Update': cfnresponse.send(event, context, cfnresponse.SUCCESS, {'Response': 'Success'}) except Exception as e: print(e) cfnresponse.send(event, context, cfnresponse.FAILED, {}) Handler: index.handler MemorySize: 128 Role: !GetAtt PutExcludeRole.Arn Runtime: "python3.9" Timeout: 60 Tags: - Key: "Name" Value: !Sub "${PrjPrefix}-tf-repo-${BranchName}-put-exclude" DependsOn: PutExcludeLog PutExclude: Type: Custom::CodeCommitPutExclude Properties: ServiceToken: !GetAtt PutExcludeFunction.Arn RepositoryName: !GetAtt CodeCommit.Name BranchName: !Ref BranchName FileContent: "---\nexclude:" FilePath: "exclude.yml"
Secrets Manager
tfsecを実行するCodeBuildでは、Aqua Security(tfsecの開発元)のDockerイメージを使用します。
Docker Hubからイメージを取得するため、「レート制限」に引っかからないよう、Docker Hubへの認証情報をSecrets Managerに保存します。
Docker HubでTwo-Factor Authentication(MFA)を有効にしている場合は、passwordの代わりに「AccessToken」をDocker Hubで発行し設定します。
「CodeBuildを扱う上でのDocker Hubのレート制限」についてもっと知りたい方は、以下も重ねてご覧ください。
実装部分
Resources: ################################# # Secrets Manager ################################# DockerLoginProfile: Type: AWS::SecretsManager::Secret Properties: Description: "Docker login profile for terraform pipelines" Name: !Sub "${PrjPrefix}/docker_hub" SecretString: !Sub '{"username":"${DockerHubUserName}","password":"${DockerHubUserPassword}"}' KmsKeyId: !GetAtt KeySecretsManager.Arn Tags: - Key: "Name" Value: !Sub "${PrjPrefix}/docker_hub"
CloudWatch Logs(CodeBuild:tfsec)
tfsec用のCodeBuildのビルドログを保管するCloudWatch Logsを設定します。
保持期間はParameters
セクションから変更できるようにします。
実装部分
Parameters: BuildLogsRationDay: Type: Number Description: "Enter build log retention period." Default: 90 AllowedValues: - 1 - 3 - 5 - 7 - 14 - 30 - 60 - 90 - 120 - 150 - 180 - 365 - 400 - 545 - 731 - 1827 - 3653 Resources: ################################# # CloudWatch Logs (CodeBuild tfsec) ################################# CWLTfsec: Type: AWS::Logs::LogGroup Properties: LogGroupName: !Sub "/aws/codebuild/${PrjPrefix}-tf-build-tfsec-project" RetentionInDays: !Ref BuildLogsRationDay KmsKeyId: !GetAtt KeyCWL.Arn Tags: - Key: "Name" Value: !Sub "/aws/codebuild/${PrjPrefix}-tf-build-tfsec-project"
S3(Artifacts)
CodePipelineで使用するアーティファクト用のS3バケットを作成します。
CodeBuildのIAMロールでResource
セクションを厳密に制御するため、ビルドプロジェクトより先に作成します。
暗号化キーは先ほど作成した鍵を使用してSSE-KMS
とします。「バケットキー」を使用してリクエストコストの削減も行います。
また、オブジェクトACLは機能的に問題ないため、「無効」で設定します。
実装部分
################################# # S3 Bucket (Artifact) ################################# BucketArtifacts: Type: 'AWS::S3::Bucket' Properties: BucketName: !Sub "${PrjPrefix}-tf-pipeline-artifacts" BucketEncryption: ServerSideEncryptionConfiguration: - ServerSideEncryptionByDefault: SSEAlgorithm: "aws:kms" KMSMasterKeyID: !GetAtt KeyS3Arthifact.Arn BucketKeyEnabled: true PublicAccessBlockConfiguration: BlockPublicAcls: true BlockPublicPolicy: true IgnorePublicAcls: true RestrictPublicBuckets: true Tags: - Key: "Name" Value: !Sub "${PrjPrefix}-tf-pipeline-artifacts" BucketPolicyArtifacts: Type: AWS::S3::BucketPolicy Properties: Bucket: !Ref BucketArtifacts PolicyDocument: Version: '2012-10-17' Statement: - Sid: "AllowSSLRequestsOnly" Effect: "Deny" Principal: "*" Action: "s3:*" Resource: - !GetAtt BucketArtifacts.Arn - !Sub - "${BucketArn}/*" - { BucketArn: !GetAtt BucketArtifacts.Arn } Condition: Bool: aws:SecureTransport: "false"
カイゼンできる部分
Security Hubの「AWS Foundational Security Best Practices standard」に従い、セキュリティレベルを高めることができます。
具体的には、以下のコントロールが今回の実装では対応できていません。必要に応じて設定を行いましょう。
- [S3.9]S3バケットサーバーアクセスログ記録を有効にする必要があります
- [S3.10]バージョニングが有効なS3バケットでは、ライフサイクルポリシーを設定する必要があります
- [S3.11]S3バケットでは、イベント通知を有効にする必要があります
CodeBuild(tfsec)
tfsecが搭載されたDockerイメージで、ビルドプロジェクトを作成します。
Secrets Managerに保管したDocker Hubの認証情報も起動時に使用するよう作成します。
実装部分
################################# # CodeBuild (tfsec) ################################# PolicyTfsec: Type: AWS::IAM::ManagedPolicy Properties: ManagedPolicyName: !Sub "${PrjPrefix}-tf-build-tfsec-project-policy" PolicyDocument: Version: "2012-10-17" Statement: - Sid: "BuildLogs" Effect: "Allow" Action: - "logs:CreateLogGroup" - "logs:CreateLogStream" - "logs:PutLogEvents" Resource: - !Sub "arn:${AWS::Partition}:logs:${AWS::Region}:${AWS::AccountId}:log-group:${CWLTfsec}" - !GetAtt CWLTfsec.Arn - Sid: "BuildReports" Effect: "Allow" Action: - "codebuild:CreateReportGroup" - "codebuild:CreateReport" - "codebuild:UpdateReport" - "codebuild:BatchPutTestCases" - "codebuild:BatchPutCodeCoverages" Resource: - !Sub "arn:${AWS::Partition}:codebuild:${AWS::Region}:${AWS::AccountId}:report-group/${PrjPrefix}-tf-build-tfsec-project-reports" - Sid: "GitPull" Effect: "Allow" Action: "codecommit:GitPull" Resource: !GetAtt CodeCommit.Arn - Sid: "GetSecretValue" Effect: "Allow" Action: "secretsmanager:GetSecretValue" Resource: !Ref DockerLoginProfile - Sid: "KmsKey" Effect: "Allow" Action: - "kms:Decrypt" - "kms:DescribeKey" - "kms:Encrypt" - "kms:ReEncrypt" - "kms:GenerateDataKey" Resource: - !GetAtt KeyS3Arthifact.Arn - !GetAtt KeySecretsManager.Arn - Sid: "S3Artifact" Effect: "Allow" Action: - "s3:PutObject" - "s3:GetObject" - "s3:GetObjectVersion" - "s3:GetBucketAcl" - "s3:GetBucketLocation" Resource: - !GetAtt BucketArtifacts.Arn - !Sub - "${BucketArn}/*" - { BucketArn: !GetAtt BucketArtifacts.Arn } RoleTfsec: Type: AWS::IAM::Role Properties: RoleName: !Sub "${PrjPrefix}-tf-build-tfsec-project-role" AssumeRolePolicyDocument: Version: "2012-10-17" Statement: - Effect: "Allow" Principal: Service: "codebuild.amazonaws.com" Action: "sts:AssumeRole" ManagedPolicyArns: - !Ref PolicyTfsec Tags: - Key: "Name" Value: !Sub "${PrjPrefix}-tf-build-tfsec-project-role" ProjectTfsec: Type: AWS::CodeBuild::Project Properties: Name: !Sub "${PrjPrefix}-tf-build-tfsec-project" Description: "Analyze the code for vulnerabilities using tfsec." Source: Type: "CODEPIPELINE" BuildSpec: | version: 0.2 env: exported-variables: - BuildID - BuildTag phases: pre_build: commands: - "echo Executing tfsec" - "mkdir -p reports/tfsec/" build: commands: - "tfsec -s --no-color --config-file exclude.yml ." - "tfsec -s --no-color --config-file exclude.yml . --format junit > reports/tfsec/report.xml" post_build: commands: - "export BuildID=`echo $CODEBUILD_BUILD_ID | cut -d: -f1`" - "export BuildTag=`echo $CODEBUILD_BUILD_ID | cut -d: -f2`" reports: reports: files: - "reports/tfsec/report.xml" file-format: JUNITXML Artifacts: Type: "CODEPIPELINE" Cache: Type: "LOCAL" Modes: - "LOCAL_DOCKER_LAYER_CACHE" Environment: Type: "LINUX_CONTAINER" ComputeType: "BUILD_GENERAL1_SMALL" Image: "aquasec/tfsec:latest" ImagePullCredentialsType: "SERVICE_ROLE" RegistryCredential: Credential: !Ref DockerLoginProfile CredentialProvider: "SECRETS_MANAGER" PrivilegedMode: true LogsConfig: CloudWatchLogs: Status: "ENABLED" GroupName: !Ref CWLTfsec EncryptionKey: !GetAtt KeyS3Arthifact.Arn ResourceAccessRole: !GetAtt RoleTfsec.Arn ServiceRole: !GetAtt RoleTfsec.Arn TimeoutInMinutes: 60 Visibility: "PRIVATE" Tags: - Key: "Name" Value: !Sub "${PrjPrefix}-tf-build-tfsec-project" ReportGroupTfsec: Type: AWS::CodeBuild::ReportGroup Properties: Name: !Sub "${ProjectTfsec}-reports" Type: "TEST" DeleteReports: true ExportConfig: ExportConfigType: "NO_EXPORT" Tags: - Key: "Name" Value: !Sub "${ProjectTfsec}-reports"
buildspec.ymlの解説
ここでは、ビルドプロジェクトに設定したbuildspec.yml
について解説します。
version: 0.2 env: exported-variables: - BuildID - BuildTag phases: pre_build: commands: - "echo Executing tfsec" - "mkdir -p reports/tfsec/" build: commands: - "tfsec -s --no-color --config-file exclude.yml ." - "tfsec -s --no-color --config-file exclude.yml . --format junit > reports/tfsec/report.xml" post_build: commands: - "export BuildID=`echo $CODEBUILD_BUILD_ID | cut -d: -f1`" - "export BuildTag=`echo $CODEBUILD_BUILD_ID | cut -d: -f2`" reports: reports: files: - "reports/tfsec/report.xml" file-format: JUNITXML
env/exported-variables
では、ビルドプロジェクトで使用した値をエクスポートするために使用します。
「BuildID」と「BuildTag」をエクスポートするように設定しています。
各エクスポートした値は、CodePipelineの手動承認アクションで使用します。
build
フェーズで、tfsecを実行しています。
--no-color
オプションを使用することで、CodeBuildのログの文字化けを防ぎます。私も実装当初、以下のような事象に遭遇しエラー解決まで時間がかかりました。
--config-file
オプションでは、指定したファイルから無視する検知ルールを設定します。
CodeCommit作成時に、カスタムリソースで作成したexclude.yml
が、--config-file
の対象になっています。
--config-file
オプションで指定するファイルは空ファイルでも問題なく動作します。
-format junit
でレポートを「JUnit XML形式」で出力しています。
CodeBuildレポートグループでは、「JUnit XML形式」をサポートしています。
Backend(S3, DynamoDB)の実装
ここからは、以下の赤線の部分を実装します。
S3
tfstate用のS3バケットを作成します。
アーティファクト用と違い、tfstate用のS3バケットでは、バージョニング機能を設定します。
実装部分
################################# # S3 Bucket (tfstate) ################################# BucketTfstate: Type: 'AWS::S3::Bucket' Properties: BucketName: !Sub "${PrjPrefix}-tf-pipeline-tfstate" BucketEncryption: ServerSideEncryptionConfiguration: - ServerSideEncryptionByDefault: SSEAlgorithm: "aws:kms" KMSMasterKeyID: !GetAtt KeyS3Tfstate.Arn BucketKeyEnabled: true PublicAccessBlockConfiguration: BlockPublicAcls: true BlockPublicPolicy: true IgnorePublicAcls: true RestrictPublicBuckets: true VersioningConfiguration: Status: "Enabled" Tags: - Key: "Name" Value: !Sub "${PrjPrefix}-tf-pipeline-tfstate" BucketPolicyTfstate: Type: AWS::S3::BucketPolicy Properties: Bucket: !Ref BucketTfstate PolicyDocument: Version: '2012-10-17' Statement: - Sid: "AllowSSLRequestsOnly" Effect: "Deny" Principal: "*" Action: "s3:*" Resource: - !GetAtt BucketTfstate.Arn - !Sub - "${BucketArn}/*" - { BucketArn: !GetAtt BucketTfstate.Arn } Condition: Bool: aws:SecureTransport: "false"
DynamoDB
terraform
コマンドの排他制御を実装するため、DynamoDBテーブルを作成します。主キーには、LockID
を設定します。
実装部分
################################# # DynamoDB table (state lock) ################################# DDBTable: Type: AWS::DynamoDB::Table Properties: TableName: !Sub "${PrjPrefix}-tf-pipeline-state-lock-table" KeySchema: - AttributeName: "LockID" KeyType: "HASH" AttributeDefinitions: - AttributeName: "LockID" AttributeType: "S" BillingMode: "PAY_PER_REQUEST" PointInTimeRecoverySpecification: PointInTimeRecoveryEnabled: true SSESpecification: KMSMasterKeyId: !GetAtt KeyDynamoDB.Arn SSEEnabled: true SSEType: "KMS" Tags: - Key: "Name" Value: !Sub "${PrjPrefix}-tf-pipeline-state-lock-table"
CodeBuild(plan)、CodeBuild(apply)の実装
ここからは、以下の赤線の部分を実装します。
tfsecを行うCodeBuildと違い、ECRからAmazon LinuxのDockerイメージを使用します。
CodeBuild(plan)
terraform plan
用のCodeBuildプロジェクトでは、以下のブログによると「ReadOnlyAccess」と「DynamoDBへの操作権限」が必要です。
加えて、今回は以下のポリシーが必要です。
- アーティファクト用のS3バケットへの操作権限
- S3、DynamoDBの暗号化で使用するKMSキーへの操作権限
- CloudWatch Logsへのビルドログの配信権限
実装部分
################################# # CloudWatch Logs (CodeBuild terraform plan) ################################# CWLTfplan: Type: AWS::Logs::LogGroup Properties: LogGroupName: !Sub "/aws/codebuild/${PrjPrefix}-tf-build-tfplan-project" RetentionInDays: !Ref BuildLogsRationDay KmsKeyId: !GetAtt KeyCWL.Arn Tags: - Key: "Name" Value: !Sub "/aws/codebuild/${PrjPrefix}-tf-build-tfplan-project" ################################# # CodeBuild (terraform plan) ################################# PolicyTfplan: Type: AWS::IAM::ManagedPolicy Properties: ManagedPolicyName: !Sub "${PrjPrefix}-tf-build-tfplan-project-policy" PolicyDocument: Version: "2012-10-17" Statement: - Sid: "BuildLogs" Effect: "Allow" Action: - "logs:CreateLogGroup" - "logs:CreateLogStream" - "logs:PutLogEvents" Resource: - !Sub "arn:${AWS::Partition}:logs:${AWS::Region}:${AWS::AccountId}:log-group:${CWLTfplan}" - !GetAtt CWLTfplan.Arn - Sid: "KmsKey" Effect: "Allow" Action: - "kms:Decrypt" - "kms:Encrypt" - "kms:ReEncrypt" - "kms:GenerateDataKey" Resource: - !GetAtt KeyS3Arthifact.Arn - !GetAtt KeyS3Tfstate.Arn - !GetAtt KeyDynamoDB.Arn - Sid: "S3Tfstate" Effect: "Allow" Action: - "s3:PutObject*" - "s3:DeleteObject*" Resource: - !Sub - "${BucketArn}/*" - { BucketArn: !GetAtt BucketTfstate.Arn } - Sid: "DynamoDB" Effect: "Allow" Action: - "dynamodb:PutItem" - "dynamodb:DeleteItem" Resource: !GetAtt DDBTable.Arn RoleTfplan: Type: AWS::IAM::Role Properties: RoleName: !Sub "${PrjPrefix}-tf-build-tfplan-project-role" AssumeRolePolicyDocument: Version: "2012-10-17" Statement: - Effect: "Allow" Principal: Service: "codebuild.amazonaws.com" Action: "sts:AssumeRole" ManagedPolicyArns: - !Ref PolicyTfplan - "arn:aws:iam::aws:policy/ReadOnlyAccess" Tags: - Key: "Name" Value: !Sub "${PrjPrefix}-tf-build-tfplan-project-role" ProjectTfplan: Type: AWS::CodeBuild::Project Properties: Name: !Sub "${PrjPrefix}-tf-build-tfplan-project" Description: "Execute terraform plan command." Source: Type: "CODEPIPELINE" BuildSpec: | version: 0.2 env: exported-variables: - BuildID - BuildTag phases: install: runtime-versions: golang: 1.14 commands: - "git clone https://github.com/tfutils/tfenv.git ~/.tfenv" - "ln -s ~/.tfenv/bin/* /usr/local/bin" - "tfenv install $TF_VERSION" - "tfenv use $TF_VERSION" pre_build: commands: - "terraform init -input=false -no-color" build: commands: - "terraform plan -input=false -no-color" post_build: commands: - "export BuildID=`echo $CODEBUILD_BUILD_ID | cut -d: -f1`" - "export BuildTag=`echo $CODEBUILD_BUILD_ID | cut -d: -f2`" Artifacts: Type: "CODEPIPELINE" Cache: Type: "LOCAL" Modes: - "LOCAL_DOCKER_LAYER_CACHE" Environment: Type: "LINUX_CONTAINER" ComputeType: "BUILD_GENERAL1_SMALL" Image: "aws/codebuild/amazonlinux2-x86_64-standard:3.0" ImagePullCredentialsType: "CODEBUILD" PrivilegedMode: true EnvironmentVariables: - Name: "TF_VERSION" Type: PLAINTEXT Value: !Ref TerraformVersion LogsConfig: CloudWatchLogs: Status: "ENABLED" GroupName: !Ref CWLTfplan EncryptionKey: !GetAtt KeyS3Arthifact.Arn ResourceAccessRole: !GetAtt RoleTfplan.Arn ServiceRole: !GetAtt RoleTfplan.Arn TimeoutInMinutes: 60 Visibility: "PRIVATE" Tags: - Key: "Name" Value: !Sub "${PrjPrefix}-tf-build-tfplan-project"
buildspec.ymlの解説
terraform plan
用のビルドプロジェクトでも、簡単ではありますがbuildspec.ymlについて解説します。
version: 0.2 env: exported-variables: - BuildID - BuildTag phases: install: runtime-versions: golang: 1.14 commands: - "git clone https://github.com/tfutils/tfenv.git ~/.tfenv" - "ln -s ~/.tfenv/bin/* /usr/local/bin" - "tfenv install $TF_VERSION" - "tfenv use $TF_VERSION" pre_build: commands: - "terraform init -input=false -no-color" build: commands: - "terraform plan -input=false -no-color" post_build: commands: - "export BuildID=`echo $CODEBUILD_BUILD_ID | cut -d: -f1`" - "export BuildTag=`echo $CODEBUILD_BUILD_ID | cut -d: -f2`"
env/exported-variables
はtfsec用のビルドプロジェクトと同様に、CodePipelineの手動承認アクションで使用するためエクスポートしています。
install
フェーズでは、普段使っていることもあり、terraformのバージョン管理ツールとして「tfenv」をインストールしました。
CodeBuild(apply)
terraform apply
用のビルドプロジェクトでは、「AdministratorAccess」権限をIAMロールに付与します。
かなり強めな権限のため、必要に応じて権限周りの制限を行いましょう。
terraform apply
のビルドプロジェクトは、terraform plan
用とそこまで大差がないためbuildspec.ymlの解説は省略します。
実装部分
################################# # CloudWatch Logs (CodeBuild terraform apply) ################################# CWLTfapply: Type: AWS::Logs::LogGroup Properties: LogGroupName: !Sub "/aws/codebuild/${PrjPrefix}-tf-build-tfapply-project-role" RetentionInDays: !Ref BuildLogsRationDay KmsKeyId: !GetAtt KeyCWL.Arn Tags: - Key: "Name" Value: !Sub "/aws/codebuild/${PrjPrefix}-tf-build-tfapply-project-role" ################################# # CodeBuild (terraform apply) ################################# RoleTfapply: Type: AWS::IAM::Role Properties: RoleName: !Sub "${PrjPrefix}-tf-build-tfapply-project-role" AssumeRolePolicyDocument: Version: "2012-10-17" Statement: - Effect: "Allow" Principal: Service: "codebuild.amazonaws.com" Action: "sts:AssumeRole" ManagedPolicyArns: - "arn:aws:iam::aws:policy/AdministratorAccess" Tags: - Key: "Name" Value: !Sub "${PrjPrefix}-tf-build-tfapply-project-role" ProjectTfapply: Type: AWS::CodeBuild::Project Properties: Name: !Sub "${PrjPrefix}-tf-build-tfapply-project" Description: "Execute terraform apply command." Source: Type: "CODEPIPELINE" BuildSpec: | version: 0.2 phases: install: runtime-versions: golang: 1.14 commands: - "git clone https://github.com/tfutils/tfenv.git ~/.tfenv" - "ln -s ~/.tfenv/bin/* /usr/local/bin" - "tfenv install $TF_VERSION" - "tfenv use $TF_VERSION" pre_build: commands: - "terraform init -input=false -no-color" build: commands: - "terraform apply -input=false -no-color -auto-approve" Artifacts: Type: "CODEPIPELINE" Cache: Type: "LOCAL" Modes: - "LOCAL_DOCKER_LAYER_CACHE" Environment: Type: "LINUX_CONTAINER" ComputeType: "BUILD_GENERAL1_SMALL" Image: "aws/codebuild/amazonlinux2-x86_64-standard:3.0" ImagePullCredentialsType: "CODEBUILD" PrivilegedMode: true EnvironmentVariables: - Name: "TF_VERSION" Type: PLAINTEXT Value: !Ref TerraformVersion LogsConfig: CloudWatchLogs: Status: "ENABLED" GroupName: !Ref CWLTfapply EncryptionKey: !GetAtt KeyS3Arthifact.Arn ResourceAccessRole: !GetAtt RoleTfapply.Arn ServiceRole: !GetAtt RoleTfapply.Arn TimeoutInMinutes: 60 Visibility: "PRIVATE" Tags: - Key: "Name" Value: !Sub "${PrjPrefix}-tf-build-tfapply-project"
CodePipelineで結合
いよいよ最後のフェーズである「CodePipelineで結合」を行います。
今回は、パイプラインの要所で「手動承認」を組み込みます。
実装部分
################################# # CodePipeline ################################# PolicyTfPipeline: Type: AWS::IAM::ManagedPolicy Properties: ManagedPolicyName: !Sub "${PrjPrefix}-tf-pipeline-policy" PolicyDocument: Version: "2012-10-17" Statement: - Sid: "S3Artifact" Effect: "Allow" Action: - "s3:GetObject*" - "s3:GetBucket*" - "s3:List*" - "s3:DeleteObject*" - "s3:PutObject" - "s3:Abort*" Resource: - !GetAtt BucketArtifacts.Arn - !Sub - "${BucketArn}/*" - { BucketArn: !GetAtt BucketArtifacts.Arn } - Sid: "KmsKey" Effect: "Allow" Action: - "kms:Decrypt" - "kms:DescribeKey" - "kms:Encrypt" - "kms:ReEncrypt*" - "kms:GenerateDataKey*" Resource: - !GetAtt KeyS3Arthifact.Arn - Sid: "CodeCommitRepo" Effect: "Allow" Action: - "codecommit:GetBranch" - "codecommit:GetCommit" - "codecommit:UploadArchive" - "codecommit:GetUploadArchiveStatus" - "codecommit:CancelUploadArchive" Resource: - !GetAtt CodeCommit.Arn - Sid: "CodeBuildProjects" Effect: "Allow" Action: - "codebuild:BatchGetBuilds" - "codebuild:StartBuild" - "codebuild:StopBuild" Resource: - !GetAtt ProjectTfsec.Arn - !GetAtt ProjectTfplan.Arn - !GetAtt ProjectTfapply.Arn RoleTfPipelne: Type: AWS::IAM::Role Properties: RoleName: !Sub "${PrjPrefix}-tf-pipeline-role" AssumeRolePolicyDocument: Version: "2012-10-17" Statement: - Effect: "Allow" Principal: Service: "codepipeline.amazonaws.com" Action: "sts:AssumeRole" ManagedPolicyArns: - !Ref PolicyTfPipeline Tags: - Key: "Name" Value: !Sub "${PrjPrefix}-tf-pipeline-role" CodePipeline: Type: AWS::CodePipeline::Pipeline Properties: Name: !Sub "${PrjPrefix}-tf-pipeline" ArtifactStore: EncryptionKey: Id: !GetAtt KeyS3Arthifact.Arn Type: "KMS" Location: !Ref BucketArtifacts Type: "S3" RoleArn: !GetAtt RoleTfPipelne.Arn Stages: - Name: "Source" Actions: - Name: "CodeCommit_Source" ActionTypeId: Category: "Source" Owner: "AWS" Provider: "CodeCommit" Version: "1" Configuration: RepositoryName: !GetAtt CodeCommit.Name BranchName: !Ref BranchName PollForSourceChanges: false OutputArtifacts: - Name: "Artifact_Source_CodeCommit_Source" RoleArn: !GetAtt RoleTfPipelne.Arn RunOrder: 1 - Name: "tfsec_Stage" Actions: - Name: "Terraform_Security_Analysis" Namespace: TFSEC ActionTypeId: Category: "Build" Owner: "AWS" Provider: "CodeBuild" Version: "1" Configuration: ProjectName: !Ref ProjectTfsec InputArtifacts: - Name: "Artifact_Source_CodeCommit_Source" RoleArn: !GetAtt RoleTfPipelne.Arn RunOrder: 1 - Name: "Terraform_Stages" Actions: - Name: "Terraform_Security_Analysis_Manual_Review" ActionTypeId: Category: "Approval" Owner: "AWS" Provider: "Manual" Version: "1" Configuration: CustomData: "tfsec review" ExternalEntityLink: !Sub "https://${AWS::Region}.console.aws.amazon.com/codesuite/codebuild/${AWS::AccountId}/projects/#{TFSEC.BuildID}/build/#{TFSEC.BuildID}%3A#{TFSEC.BuildTag}/?region=${AWS::Region}" RoleArn: !GetAtt RoleTfPipelne.Arn RunOrder: 1 - Name: "Terraform_Plan" Namespace: "TERRAFORM" ActionTypeId: Category: "Build" Owner: "AWS" Provider: "CodeBuild" Version: "1" Configuration: ProjectName: !Ref ProjectTfplan InputArtifacts: - Name: "Artifact_Source_CodeCommit_Source" RoleArn: !GetAtt RoleTfPipelne.Arn RunOrder: 2 - Name: "Terraform_Plan_Manual_Review" ActionTypeId: Category: "Approval" Owner: "AWS" Provider: "Manual" Version: "1" Configuration: CustomData: "Terraform plan review" ExternalEntityLink: !Sub "https://${AWS::Region}.console.aws.amazon.com/codesuite/codebuild/${AWS::AccountId}/projects/#{TERRAFORM.BuildID}/build/#{TERRAFORM.BuildID}%3A#{TERRAFORM.BuildTag}/?region=${AWS::Region}" RoleArn: !GetAtt RoleTfPipelne.Arn RunOrder: 3 - Name: "Terraform_Apply" ActionTypeId: Category: "Build" Owner: "AWS" Provider: "CodeBuild" Version: "1" Configuration: ProjectName: !Ref ProjectTfapply InputArtifacts: - Name: "Artifact_Source_CodeCommit_Source" RoleArn: !GetAtt RoleTfPipelne.Arn RunOrder: 4 ################################# # EventBridge (CodeCommit State Change) # ################################# PolicyEventBridge: Type: AWS::IAM::ManagedPolicy Properties: ManagedPolicyName: !Sub "${PrjPrefix}-tf-pipeline-event-policy" PolicyDocument: Version: "2012-10-17" Statement: - Sid: "CodePipelineExec" Effect: "Allow" Action: "codepipeline:StartPipelineExecution" Resource: !Sub "arn:${AWS::Partition}:codepipeline:${AWS::Region}:${AWS::AccountId}:${CodePipeline}" RoleEventBridge: Type: AWS::IAM::Role Properties: RoleName: !Sub "${PrjPrefix}-tf-pipeline-event-role" AssumeRolePolicyDocument: Version: "2012-10-17" Statement: - Effect: "Allow" Principal: Service: "events.amazonaws.com" Action: "sts:AssumeRole" ManagedPolicyArns: - !Ref PolicyEventBridge Tags: - Key: "Name" Value: !Sub "${PrjPrefix}-tf-pipeline-event-role" EventsTfPipeline: Type: AWS::Events::Rule Properties: Name: !Sub "${PrjPrefix}-tf-pipeline-event" EventPattern: source: - "aws.codecommit" resources: - !GetAtt CodeCommit.Arn detail-type: - "CodeCommit Repository State Change" detail: event: - referenceCreated - referenceUpdated referenceType: - "branch" referenceName: - !Ref BranchName State: "ENABLED" Targets: - Arn: !Sub "arn:${AWS::Partition}:codepipeline:${AWS::Region}:${AWS::AccountId}:${CodePipeline}" Id: "TerraformPipeline" RoleArn: !GetAtt RoleEventBridge.Arn
以上、アーキテクチャの解説でした。
いざ検証
ここからは、実際にCodeCommitにTerraformのコードをプッシュして検証してみます。
CloudFormationスタックは、以下のパラメーターで作成しました。
設定値 | 値 | 備考 |
---|---|---|
BranchName | main | |
BuildLogsRationDay | 90 | |
DockerHubUserName | *** | Docker Hubのユーザー名 |
DockerHubUserPassword | *** | Docker HubのAccessToken |
PrjPrefix | tks-prd | 任意の値 |
TerraformVersion | 1.2.0 | 任意の値 |
検知されるルールについて
今回、tfsecによって評価されるルールは全部で5つあります。
- aws-vpc-add-description-to-security-group
- aws-vpc-add-description-to-security-group-rule
- aws-vpc-no-public-ingress-sgr
- aws-vpc-no-public-egress-sgr
- aws-iam-enforce-mfa
このうち「aws-iam-enforce-mfa」については、「exclude.yml
」の動作検証のためにルールを無視するよう設定します。
CodeCommitへの接続
gitコマンドを使用してCodeComitへ接続します。認証情報を聞かれるためIAMコンソールから認証情報を事前に作成しておきましょう。
git clone https://git-codecommit.ap-northeast-1.amazonaws.com/v1/repos/tks-prd-tf-repo
認証情報を入力します。
Username for 'https://git-codecommit.ap-northeast-1.amazonaws.com/v1/repos/tks-prd-tf-repo': user Password for 'https://user@git-codecommit.ap-northeast-1.amazonaws.com/v1/repos/tks-prd-tf-repo': ****** remote: Counting objects: 3, done. Unpacking objects: 100% (3/3), 231 bytes | 77.00 KiB/s, done.
terraformコードの作成
今回、検証用に以下のコードを用意しました。
terraform { required_providers { aws = { source = "hashicorp/aws" version = "4.19.0" } } backend "s3" { bucket = "tks-prd-tf-pipeline-tfstate" # S3(tfstate)で作成したバケット名 key = "prd/terraform.tfstate" # tfstateを保管するための任意の格納先 region = "ap-northeast-1" dynamodb_table = "tks-prd-tf-pipeline-state-lock-table" # 作成したDynamoDBテーブル } } provider "aws" { region = "ap-northeast-1" } module "vpc" { source = "terraform-aws-modules/vpc/aws" # source = "../modules/vpc" version = "3.12.0" # insert the 23 required variables here cidr = "10.0.0.0/16" name = "terraform-reintroduction" public_subnets = ["10.0.0.0/24", "10.0.1.0/24"] azs = ["ap-northeast-1a", "ap-northeast-1c"] } resource "aws_security_group" "allow_tls" { name = "allow_tls" description = "Allow TLS inbound traffic" vpc_id = module.vpc.vpc_id ingress { description = "TLS from VPC" from_port = 443 to_port = 443 protocol = "tcp" cidr_blocks = [module.vpc.vpc_cidr_block] } egress { from_port = 0 to_port = 0 protocol = "-1" cidr_blocks = ["0.0.0.0/0"] ipv6_cidr_blocks = ["::/0"] } tags = { Name = "allow_tls" } } data aws_caller_identity current {} resource aws_iam_group developers { name = "developersIO2022" }
検知されるルールについて
今回、tfsecで評価されるルールは全部で5つあります。
- aws-vpc-add-description-to-security-group
- aws-vpc-add-description-to-security-group-rule
- aws-vpc-no-public-ingress-sgr
- aws-vpc-no-public-egress-sgr
- aws-iam-enforce-mfa
このうち「aws-iam-enforce-mfa」については、「exclude.yml
」の動作検証のためにルールを無視するよう設定します。
exclude.ymlの修正
まずは、exclude.ymlの中身を確認します。
--- exclude:
exclude:
の下に、配列で無視したいルールを記述します。
--- exclude: + - aws-iam-enforce-mfa
レポジトリへプッシュ
今回は、そのままレポジトリへプッシュします。
git add . git commit -m "my first commit" git push
tfsecの確認
CodeCommitレポジトリの状態変更が起こると、CodePipelineがトリガーされる仕様になっています。
画面を確認すると「手動承認」で止まっているため、ビルドログとレポートを確認してみます。
「レビュー」ボタンをクリックすると「レビュー用URL」が表示されるようにしています。
レビュー用URLは、CloudFormationで「https://${AWS::Region}.console.aws.amazon.com/codesuite/codebuild/${AWS::AccountId}/projects/#{TFSEC.BuildID}/build/#{TFSEC.BuildID}%3A#{TFSEC.BuildTag}/?region=${AWS::Region}
」と動的に生成するよう定義しています。
buildspec.ymlでエクスポートした「BuildID」と「BuildTag」はこの実装のためにエクスポートしました。
生成したレポートURLからビルドログとレポートを確認してみます。
ビルドログを確認すると、tfsecでリポジトリのコードが評価されていることがわかります。
続いて「レポート」も確認してみます。
レポートでは、exclude.yml
で無視したルール以外の失敗したルールが一覧で表示されています。「レポートあたりのテストケースの最大数」がデフォルトでは500までのため、成功したテストケースについては表示していません。
成功したテストケースも表示したい場合は、tfsecコマンドに--include-passed
オプションを入れると表示されます。
失敗したルールをクリックすると、「どのファイルのどの部分が失敗しているのか」を確認できます。
terraform plan/apply
今回はtfsecで検知したルールは是正せず、そのままパイプラインの完了まで行おうと思います。
「手動承認」を承諾して、「Terraform_Plan」アクションへ進めます。
ビルドログの抜粋になりますが、問題なくterraform plan
が走っていることがわかります。
terraform plan
実行後も、「Terraform_Plan_Manual_Review」アクションで手動承認を組み込んでいます。tfsecの手動承認と同じ仕組みでレビュー用URLの作成をしています。
最後に「Terraform_Apply」アクションで、terraform apply
コマンドを実行します。
無事、リソースが作成されていることが確認できました。
参考
実装に当たってさまざまな人のTerraform Pipelineを見つけたためご紹介します。
おわりに
以上、「TerraformのCI/CDパイプラインを実装してみた」でした。今回、初めてCodePipelineを0から実装したため、かなり勉強になりました。
また、今回のような超大作ブログを長期休暇でかければいいなと思います。(比較的サーバーレスな構成のため課金に恐れずゆっくり書けました。)
このブログがどなたかのご参考になれば幸いです。
以上、AWS事業本部コンサルティング部のたかくに(@takakuni_)でした!