TerraformのAWS Provider Version 4へのアップグレードに伴うコード改修がおっくうな方へ 便利なツールありますよ

2022.02.28

先日TerraformのAWS Provider Version 4がリリースされました。

S3バケット周りで大規模なリファクタリングがあり、そのままのコードで単純にproviderだけアップグレードするとエラーになります。コードを書き換える必要があるのですが、中々面倒です。

そんな私のような方にピッタリなツールを見つけたのでご紹介します。tfrefactorです。既存のS3バケット(aws_s3_bucketリソース)のコードをコマンド一発でv4準拠のコードに書き換えてくれます。

インストール方法

バイナリをダウンロードしてパスを通す方法make installする方法があります。私は前者でやりましたが、「開発元を検証できないため開けません」のエラーでコマンド実行できなかったので、以下リンク先ページにあるようにシステム環境設定→セキュリティとプライバシーから設定変更して実行できるようにしました。

block-tfrefactor

使用方法

tfrefactor resource aws_s3_bucket (パス)というコマンドで、パスにあるaws_s3_bucketリソースをv4準拠のコードに書き直したファイルを出力してくれます。

使ってみた

前回作ったモリモリのコードを試してみます。

s3.tf

resource "aws_s3_bucket" "v3" {
  bucket = local.bucket_name

  acceleration_status = "Enabled"

  acl = "private"

  cors_rule {
    allowed_headers = ["*"]
    allowed_methods = ["PUT", "POST"]
    allowed_origins = ["https://s3-website-test.hashicorp.com"]
    expose_headers  = ["ETag"]
    max_age_seconds = 3000
  }

  # `acl`とコンフリクトするためコメントアウト  
  #   grant {
  #     type        = "Group"
  #     permissions = ["READ_ACP", "WRITE"]
  #     uri         = "http://acs.amazonaws.com/groups/s3/LogDelivery"
  #   }

  lifecycle_rule {
    id      = "log"
    enabled = true
    prefix  = "log/"
    tags = {
      rule      = "log"
      autoclean = "true"
    }
    transition {
      days          = 30
      storage_class = "STANDARD_IA"
    }
    transition {
      days          = 60
      storage_class = "GLACIER"
    }
    expiration {
      days = 90
    }
  }

  logging {
    target_bucket = aws_s3_bucket.log_bucket.id
    target_prefix = "log/"
  }

#   `replication_configuration`とコンフリクトするためコメントアウト 
#   object_lock_configuration {
#     object_lock_enabled = "Enabled"
# 
#     rule {
#       default_retention {
#         mode = "COMPLIANCE"
#         days = 3
#       }
#     }
#   }

  policy = <<-EOF
    {
    "Id": "Policy1446577137248",
    "Statement": [
        {
        "Action": "s3:PutObject",
        "Effect": "Allow",
        "Principal": {
            "AWS": "${data.aws_elb_service_account.current.arn}"
        },
        "Resource": "arn:${data.aws_partition.current.partition}:s3:::${local.bucket_name}/*",
        "Sid": "Stmt1446575236270"
        }
    ],
    "Version": "2012-10-17"
    }
  EOF

  replication_configuration {
    role = aws_iam_role.replication.arn
    rules {
      id     = "foobar"
      status = "Enabled"
      filter {
        tags = {}
      }
      destination {
        bucket        = aws_s3_bucket.destination.arn
        storage_class = "STANDARD"
        replication_time {
          status  = "Enabled"
          minutes = 15
        }
        metrics {
          status  = "Enabled"
          minutes = 15
        }
      }
    }
  }

  request_payer = "Requester"

  server_side_encryption_configuration {
    rule {
      apply_server_side_encryption_by_default {
        kms_master_key_id = aws_kms_key.mykey.arn
        sse_algorithm     = "aws:kms"
      }
    }
  }

  versioning {
    enabled = true
  }

  website {
    index_document = "index.html"
    error_document = "error.html"
  }
}

以下のコマンドを実行します。

% tfrefactor resource aws_s3_bucket s3.tf

すると、s3_migrated.tfというファイルが作成されます。

s3_migrated.tf

resource "aws_s3_bucket" "v3" {
  bucket = local.bucket_name
}

resource "aws_s3_bucket_acl" "v3_acl" {
  bucket = aws_s3_bucket.v3.id
  acl    = "private"
}

resource "aws_s3_bucket_accelerate_configuration" "v3_accelerate_configuration" {
  bucket = aws_s3_bucket.v3.id
  status = "Enabled"
}

resource "aws_s3_bucket_policy" "v3_policy" {
  bucket = aws_s3_bucket.v3.id
  policy = <<-EOF
    {
    "Id": "Policy1446577137248",
    "Statement": [
        {
        "Action": "s3:PutObject",
        "Effect": "Allow",
        "Principal": {
            "AWS": "${data.aws_elb_service_account.current.arn}"
        },
        "Resource": "arn:${data.aws_partition.current.partition}:s3:::${local.bucket_name}/*",
        "Sid": "Stmt1446575236270"
        }
    ],
    "Version": "2012-10-17"
    }
  EOF
}

resource "aws_s3_bucket_request_payment_configuration" "v3_request_payment_configuration" {
  bucket = aws_s3_bucket.v3.id
  payer  = "Requester"
}

resource "aws_s3_bucket_cors_configuration" "v3_cors_configuration" {
  bucket = aws_s3_bucket.v3.id
  cors_rule {
    allowed_headers = ["*"]
    allowed_methods = ["PUT", "POST"]
    allowed_origins = ["https://s3-website-test.hashicorp.com"]
    expose_headers  = ["ETag"]
    max_age_seconds = 3000
  }
}

resource "aws_s3_bucket_lifecycle_configuration" "v3_lifecycle_configuration" {
  bucket = aws_s3_bucket.v3.id
  rule {
    id     = "log"
    status = "Enabled"
    filter {
      and {
        tags = {
          rule      = "log"
          autoclean = "true"
        }
        prefix = "log/"
      }
    }
    transition {
      days          = 30
      storage_class = "STANDARD_IA"
    }
    transition {
      days          = 60
      storage_class = "GLACIER"
    }
    expiration {
      days = 90
    }
  }
}

resource "aws_s3_bucket_logging" "v3_logging" {
  bucket        = aws_s3_bucket.v3.id
  target_prefix = "log/"
  target_bucket = aws_s3_bucket.log_bucket.id
}

resource "aws_s3_bucket_versioning" "v3_versioning" {
  bucket = aws_s3_bucket.v3.id
  versioning_configuration {
    status = "Enabled"
  }
}

resource "aws_s3_bucket_replication_configuration" "v3_replication_configuration" {
  bucket = aws_s3_bucket.v3.id
  role   = aws_iam_role.replication.arn
  rule {
    id     = "foobar"
    status = "Enabled"
    filter {
      and {
        tags   = {}
        prefix = ""
      }
    }
    destination {
      bucket        = aws_s3_bucket.destination.arn
      storage_class = "STANDARD"
      replication_time {
        status = "Enabled"
        time {
          minutes = 15
        }
      }
      metrics {
        status = "Enabled"
        event_threshold {
          minutes = 15
        }
      }
    }
  }
}

resource "aws_s3_bucket_server_side_encryption_configuration" "v3_server_side_encryption_configuration" {
  bucket = aws_s3_bucket.v3.id
  rule {
    apply_server_side_encryption_by_default {
      kms_master_key_id = aws_kms_key.mykey.arn
      sse_algorithm     = "aws:kms"
    }
  }
}

resource "aws_s3_bucket_website_configuration" "v3_website_configuration" {
  bucket = aws_s3_bucket.v3.id
  error_document {
    key = "error.html"
  }
  index_document {
    suffix = "index.html"
  }
}

さらにさらに。-cオプションを付けると、対象になったリソースをCSVリストにして出力してくれます。

% tfrefactor resource aws_s3_bucket s3.tf -c

s3_new_resources.csv

aws_s3_bucket_policy.v3_policy,aws_s3_bucket.v3
aws_s3_bucket_request_payment_configuration.v3_request_payment_configuration,aws_s3_bucket.v3
aws_s3_bucket_accelerate_configuration.v3_accelerate_configuration,aws_s3_bucket.v3
aws_s3_bucket_acl.v3_acl,aws_s3_bucket.v3
aws_s3_bucket_cors_configuration.v3_cors_configuration,aws_s3_bucket.v3
aws_s3_bucket_lifecycle_configuration.v3_lifecycle_configuration,aws_s3_bucket.v3
aws_s3_bucket_logging.v3_logging,aws_s3_bucket.v3
aws_s3_bucket_versioning.v3_versioning,aws_s3_bucket.v3
aws_s3_bucket_replication_configuration.v3_replication_configuration,aws_s3_bucket.v3
aws_s3_bucket_server_side_encryption_configuration.v3_server_side_encryption_configuration,aws_s3_bucket.v3
aws_s3_bucket_website_configuration.v3_website_configuration,aws_s3_bucket.v3

upgrade手順案

このtfrefactorを使ってのv3からv4へのupgrade手順の案を考えました。

1.providerのversion constrait更新

  terraform {
    required_version = "= 1.1.5"

    required_providers {
      aws = {
        source  = "hashicorp/aws"
-      version = "3.74.2"
+      version = "4.2.0"
      }
    }  
}

2.lockfile更新

% terraform init --upgrade

Initializing the backend...

Initializing provider plugins...
- Finding hashicorp/aws versions matching "4.2.0"...
- Installing hashicorp/aws v4.2.0...
- Installed hashicorp/aws v4.2.0 (signed by HashiCorp)

Terraform has made some changes to the provider dependency selections recorded
in the .terraform.lock.hcl file. Review those changes and commit them to your
version control system if they represent changes you intended to make.

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

3.tfrefactor実行

% tfrefactor resource aws_s3_bucket s3.tf -c

4.元のファイルを除外

% mv s3.tf s3.tf.tmp

5.importスクリプトを作成

VSCodeでやる場合

  • 3で作られたCSVファイル内を正規表現オンで^(.+),.+\nで検索
  • terraform import $1 (バケット名)\nで置換

vscode-replace

6.スクリプト実行

5で作ったスクリプトを実行

% sh s3_new_resources.sh
aws_s3_bucket_policy.v3_policy: Importing from ID "test20220220"...
aws_s3_bucket_policy.v3_policy: Import prepared!
  Prepared aws_s3_bucket_policy for import
aws_s3_bucket_policy.v3_policy: Refreshing state... [id=test20220220]

Import successful!

The resources that were imported are shown above. These resources are now in
your Terraform state and will henceforth be managed by Terraform.

aws_s3_bucket_request_payment_configuration.v3_request_payment_configuration: Importing from ID "test20220220"...
aws_s3_bucket_request_payment_configuration.v3_request_payment_configuration: Import prepared!
  Prepared aws_s3_bucket_request_payment_configuration for import
aws_s3_bucket_request_payment_configuration.v3_request_payment_configuration: Refreshing state... [id=test20220220]

Import successful!

The resources that were imported are shown above. These resources are now in
your Terraform state and will henceforth be managed by Terraform.

aws_s3_bucket_accelerate_configuration.v3_accelerate_configuration: Importing from ID "test20220220"...
aws_s3_bucket_accelerate_configuration.v3_accelerate_configuration: Import prepared!
  Prepared aws_s3_bucket_accelerate_configuration for import
aws_s3_bucket_accelerate_configuration.v3_accelerate_configuration: Refreshing state... [id=test20220220]

Import successful!

The resources that were imported are shown above. These resources are now in
your Terraform state and will henceforth be managed by Terraform.

aws_s3_bucket_acl.v3_acl: Importing from ID "test20220220"...
aws_s3_bucket_acl.v3_acl: Import prepared!
  Prepared aws_s3_bucket_acl for import
aws_s3_bucket_acl.v3_acl: Refreshing state... [id=test20220220]

Import successful!

The resources that were imported are shown above. These resources are now in
your Terraform state and will henceforth be managed by Terraform.

aws_s3_bucket_cors_configuration.v3_cors_configuration: Importing from ID "test20220220"...
aws_s3_bucket_cors_configuration.v3_cors_configuration: Import prepared!
  Prepared aws_s3_bucket_cors_configuration for import
aws_s3_bucket_cors_configuration.v3_cors_configuration: Refreshing state... [id=test20220220]

Import successful!

The resources that were imported are shown above. These resources are now in
your Terraform state and will henceforth be managed by Terraform.

aws_s3_bucket_lifecycle_configuration.v3_lifecycle_configuration: Importing from ID "test20220220"...
aws_s3_bucket_lifecycle_configuration.v3_lifecycle_configuration: Import prepared!
  Prepared aws_s3_bucket_lifecycle_configuration for import
aws_s3_bucket_lifecycle_configuration.v3_lifecycle_configuration: Refreshing state... [id=test20220220]

Import successful!

The resources that were imported are shown above. These resources are now in
your Terraform state and will henceforth be managed by Terraform.

aws_s3_bucket_logging.v3_logging: Importing from ID "test20220220"...
aws_s3_bucket_logging.v3_logging: Import prepared!
  Prepared aws_s3_bucket_logging for import
aws_s3_bucket_logging.v3_logging: Refreshing state... [id=test20220220]

Import successful!

The resources that were imported are shown above. These resources are now in
your Terraform state and will henceforth be managed by Terraform.

aws_s3_bucket_versioning.v3_versioning: Importing from ID "test20220220"...
aws_s3_bucket_versioning.v3_versioning: Import prepared!
  Prepared aws_s3_bucket_versioning for import
aws_s3_bucket_versioning.v3_versioning: Refreshing state... [id=test20220220]

Import successful!

The resources that were imported are shown above. These resources are now in
your Terraform state and will henceforth be managed by Terraform.

aws_s3_bucket_replication_configuration.v3_replication_configuration: Importing from ID "test20220220"...
aws_s3_bucket_replication_configuration.v3_replication_configuration: Import prepared!
  Prepared aws_s3_bucket_replication_configuration for import
aws_s3_bucket_replication_configuration.v3_replication_configuration: Refreshing state... [id=test20220220]

Import successful!

The resources that were imported are shown above. These resources are now in
your Terraform state and will henceforth be managed by Terraform.

aws_s3_bucket_server_side_encryption_configuration.v3_server_side_encryption_configuration: Importing from ID "test20220220"...
aws_s3_bucket_server_side_encryption_configuration.v3_server_side_encryption_configuration: Import prepared!
  Prepared aws_s3_bucket_server_side_encryption_configuration for import
aws_s3_bucket_server_side_encryption_configuration.v3_server_side_encryption_configuration: Refreshing state... [id=test20220220]

Import successful!

The resources that were imported are shown above. These resources are now in
your Terraform state and will henceforth be managed by Terraform.

aws_s3_bucket_website_configuration.v3_website_configuration: Importing from ID "test20220220"...
aws_s3_bucket_website_configuration.v3_website_configuration: Import prepared!
  Prepared aws_s3_bucket_website_configuration for import
aws_s3_bucket_website_configuration.v3_website_configuration: Refreshing state... [id=test20220220]

Import successful!

The resources that were imported are shown above. These resources are now in
your Terraform state and will henceforth be managed by Terraform.

7.planで差分が出ないか確認

% terraform plan
(略)
  # aws_s3_bucket_acl.v3_acl will be updated in-place
  ~ resource "aws_s3_bucket_acl" "v3_acl" {
      + acl    = "private"
        id     = "test20220220"
        # (1 unchanged attribute hidden)

        # (1 unchanged block hidden)
    }

…差分が出ました。これが正しそうなのでこのあとapplyしました。

(2022/3/1追記)

aws_s3_bucket_aclに対するimportコマンドをterraform import aws_s3_bucket_acl.v3_acl test20220220というようにしていたのですが、 terraform import aws_s3_bucket_acl.v3_acl test20220220,privateとすると上記差分は出ないようです。

8.ファイル置き換え

% mv s3_migrated.tf s3.tf
% rm s3.tf.tmp

参考情報