AWS-managed buckets that must be allowed when restricting an S3 gateway endpoint policy to your own account only

Hello, I'm Hayashi.

By adding an aws:ResourceAccount condition to the policy of an S3 gateway endpoint, you can restrict the buckets accessible from your VPC to only those in your own account.

The official documentation has the following note about this condition.

To specify the AWS account ID of the resource being accessed, you can use either the aws:ResourceAccount or the s3:ResourceAccount key in your IAM policy. Note, however, that some AWS services rely on access to AWS managed buckets. Therefore, using the aws:ResourceAccount or s3:ResourceAccount key in your IAM policy might also affect access to these resources.

https://docs.aws.amazon.com/ja_jp/AmazonS3/latest/userguide/privatelink-interface-endpoints.html

The AWS managed buckets mentioned in the note are, for example, the buckets that hold ECR image layers and Amazon Linux 2023 packages. If you restrict access to your own account only, access to these is also denied.

On an EC2 instance in a private subnet with no internet access, I checked what fails and which buckets need to be allowed for things to succeed.

Conclusion first

I tried a policy that allows only my own account in the Tokyo region. Retrieving from a bucket in my own account succeeded, and four operations failed with S3 403 errors.

When I allowed s3:GetObject to the AWS managed buckets including these, all four succeeded.
Here is the policy with the added permissions.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Access-to-own-account-buckets-only",
      "Effect": "Allow",
      "Principal": "*",
      "Action": "s3:*",
      "Resource": "*",
      "Condition": {
        "StringEquals": {
          "aws:ResourceAccount": "123456789012"
        }
      }
    },
    {
      "Sid": "Access-to-AWS-managed-buckets",
      "Effect": "Allow",
      "Principal": "*",
      "Action": "s3:GetObject",
      "Resource": [
        "arn:aws:s3:::prod-ap-northeast-1-starport-layer-bucket/*",
        "arn:aws:s3:::al2023-repos-ap-northeast-1-de612dc2/*",
        "arn:aws:s3:::amazon-ssm-ap-northeast-1/*",
        "arn:aws:s3:::aws-ssm-ap-northeast-1/*",
        "arn:aws:s3:::patch-baseline-snapshot-ap-northeast-1/*",
        "arn:aws:s3:::aws-patch-manager-ap-northeast-1-4849fa78f/*",
        "arn:aws:s3:::ap-northeast-1-birdwatcher-prod/*"
      ]
    }
  ]
}

The only action allowed is s3:GetObject. The examples in the ECR and Systems Manager guides also use only s3:GetObject, and in my testing the five operations succeeded without s3:ListBucket.

aws-patch-manager-ap-northeast-1-4849fa78f and ap-northeast-1-birdwatcher-prod were not verified in this test, but I included them following the list in the Systems Manager guide.

Bucket names are determined per region, and are documented in the AWS user guides and an AWS re:Post Knowledge Center article.

https://docs.aws.amazon.com/ja_jp/AmazonECR/latest/userguide/vpc-endpoints.html

https://repost.aws/knowledge-center/ec2-al1-al2-update-yum-without-internet

https://docs.aws.amazon.com/ja_jp/systems-manager/latest/userguide/ssm-agent-technical-details.html#ssm-agent-minimum-s3-permissions

{suffix} differs by region. The actual names for each region are listed in "Reference: Amazon S3 buckets for patching operations".

https://docs.aws.amazon.com/ja_jp/systems-manager/latest/userguide/patch-operations-s3-buckets.html

The Systems Manager guide also has the following note about not using a wildcard for the region.

We recommend that you do not use a wildcard character (*) in place of a specific Region in this policy. For example, use arn:aws:s3:::aws-ssm-us-east-2/* and do not use arn:aws:s3:::aws-ssm-*/*. Using a wildcard could provide access to S3 buckets you do not intend to grant access to. If you want to use the instance profile for more than one Region, we recommend repeating the first Statement block for each Region.

If you use Windows Server, you also need aws-windows-downloads-{region}, which is listed in the same guide. Since this test used Amazon Linux 2023, I did not include it in the policy.

Test method

The EC2 IAM role allowed s3:Get* and s3:List* on all resources, so that the only changing condition was the endpoint policy. The docker pull target was amazonlinux:2023 placed in a private ECR repository in my own account.

I switched the S3 gateway endpoint policy among three types and ran the same five operations with Run Command for each.

Here is the "Own account only" policy.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Access-to-own-account-buckets-only",
      "Effect": "Allow",
      "Principal": "*",
      "Action": "s3:*",
      "Resource": "*",
      "Condition": {
        "StringEquals": {
          "aws:ResourceAccount": "123456789012"
        }
      }
    }
  ]
}

The five operations I ran are as follows.

1. Retrieve an object from a bucket in my own account with aws s3 cp
2. Fetch packages with dnf install
3. Pull an ECR image with docker pull
4. Run AWS-RunPatchBaseline with Scan
5. Run AWS-UpdateSSMAgent

Results

○ means success and × means failure.

Running Run Command and connecting with Session Manager do not use S3, so they succeeded with every policy.

Errors on failure

These are the errors from the four operations that failed with the "Own account only" policy. In every case, the name of the inaccessible bucket appears in the 403 error URL.

dnf

It failed when fetching the repository mirror list.

Error: Failed to download metadata for repo 'kernel-livepatch': Cannot prepare internal mirrorlist: Status code: 403 for https://al2023-repos-ap-northeast-1-de612dc2.s3.dualstack.ap-northeast-1.amazonaws.com/kernel-livepatch/mirrors/al2023/aarch64/mirror.list?instance_id=i-0554d801acea99583 (IP: 52.219.16.122)

docker pull

ECR authentication and manifest retrieval succeeded, and it failed when fetching the layers. The error message also shows that it was the VPC endpoint policy that denied the request.

denied: User: arn:aws:sts::xxxxxxxxxxxx:assumed-role/storage-access-role-prod-ap-northeast-1/xxxxxxxx is not authorized to perform: s3:GetObject on resource: "arn:aws:s3:::prod-ap-northeast-1-starport-layer-bucket/..." because no VPC endpoint policy allows the s3:GetObject action

AWS-RunPatchBaseline

It failed when fetching the patching materials.

root [ERROR]: Unable to download payload: https://s3.ap-northeast-1.amazonaws.com/aws-ssm-ap-northeast-1/patchbaselineoperations/linux/payloads/patch-baseline-operations-1.252.tar.gz.

When the materials were cached from a previous run, it failed at the next step, fetching the patch baseline snapshot. This error does not show a bucket name, so I confirmed patch-baseline-snapshot-ap-northeast-1, where the snapshots are stored, in the Systems Manager guide.

root [ERROR]: Could not download a valid snapshot

AWS-UpdateSSMAgent

It failed when fetching the manifest used to check whether an update is available.

failed to download file reliably, https://s3.ap-northeast-1.amazonaws.com/amazon-ssm-ap-northeast-1/ssm-agent-manifest.json, http request failed. status:403 Forbidden statuscode:403

Summary

I checked which AWS services fail when the S3 gateway endpoint policy is restricted to your own account only, and which buckets need to be allowed.

• ECR image pulls, dnf on Amazon Linux 2023, Patch Manager, and SSM Agent updates cannot fetch from AWS managed buckets and fail with 403 errors
• The errors on failure show the name of the bucket that could not be accessed
• The required bucket names are determined per region and are documented in the AWS user guides and AWS re:Post
• Allowing s3:GetObject to the AWS managed buckets makes every operation succeed

When restricting access with aws:ResourceAccount, you need to check which AWS managed buckets the services you use rely on, and allow them.

I hope this article is helpful to someone. Thank you for reading to the end!

References

https://docs.aws.amazon.com/AmazonS3/latest/userguide/privatelink-interface-endpoints.html

https://docs.aws.amazon.com/AmazonECR/latest/userguide/vpc-endpoints.html

https://docs.aws.amazon.com/systems-manager/latest/userguide/ssm-agent-technical-details.html

https://docs.aws.amazon.com/systems-manager/latest/userguide/patch-operations-s3-buckets.html

https://repost.aws/knowledge-center/ec2-al1-al2-update-yum-without-internet

https://dev.classmethod.jp/articles/private-ec2-ssm-vpcendpoint-policy/

https://dev.classmethod.jp/articles/tsnote-ecs-ecr-private-subnet-CannotPullContainerError-403-forbidden-solution/