I tried processing S3 data encrypted with KMS using Lambda

Introduction

Hello, I'm Akito.

In this post, I'll introduce how to process encrypted data using Lambda, one of AWS's services.

It's well known that data can be encrypted when storing it in S3, but I was curious about how this encrypted data can actually be processed within AWS, so I researched and tested whether Lambda can encrypt data and save it in S3, and whether it can decrypt the stored data.

Prerequisites

• Understanding of basic AWS services (S3, Lambda, IAM, KMS)
• Basic knowledge of Python

Let's Try It

Step 1: Creating and Configuring a KMS Key

1-1. Creating a KMS Key

Please create a customer managed key with default settings.
We will edit the key policy later.

1-2. Creating an IAM Policy for KMS Operations

Create an IAM policy as follows to grant KMS permissions for Lambda:

			
			{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "S3Read",
      "Effect": "Allow",
      "Action": "s3:GetObject",
      "Resource": "arn:aws:s3:::<BUCKET_NAME>/*"
    },
    {
      "Sid": "S3WriteWithKmsOnly",
      "Effect": "Allow",
      "Action": "s3:PutObject",
      "Resource": "arn:aws:s3:::<BUCKET_NAME>/*"
    },
    {
      "Sid": "KmsForSseKmsFromS3Only",
      "Effect": "Allow",
      "Action": [
        "kms:Decrypt",
        "kms:GenerateDataKey*"
      ],
      "Resource": "arn:aws:kms:ap-northeast-1:<ACCOUNT_ID>:key/<KMS_KEY_ID>",
      "Condition": {
        "StringEquals": {
          "kms:ViaService": "s3.ap-northeast-1.amazonaws.com"
        },
        "ForAnyValue:StringLike": {
          "kms:EncryptionContext:aws:s3:arn": "arn:aws:s3:::<BUCKET_NAME>/*"
        }
      }
    }
  ]
}

		

Explanation of each permission:

• s3:GetObject: Allows reading objects from S3
• s3:PutObject: Allows writing objects to S3
• kms:Decrypt: Allows decrypting encrypted objects stored in S3
• kms:GenerateDataKey*: Allows generating data keys needed when writing new objects to S3
• Condition (kms:ViaService): Allows operations only on objects under a specific bucket via S3

1-3. Creating an IAM Role for Lambda

To create an IAM role for Lambda, set the use case to Lambda. Add AWSLambdaBasicExecutionRole for writing to CloudWatch Logs and the policy you just created as permission policies, then create it.#### 1-4. KMS Key Configuration
Change the Default key policy of the created KMS key as follows.

			
			{
	"Version": "2012-10-17",
	"Id": "key-consolepolicy-3",
	"Statement": [
		{
			"Sid": "Enable IAM User Permissions",
			"Effect": "Allow",
			"Principal": {
				"AWS": "arn:aws:iam::<ACCOUNT_ID>:root"
			},
			"Action": "kms:*",
			"Resource": "*"
		},
		{
			"Sid": "AllowLambdaRoleUseOfKey",
			"Effect": "Allow",
			"Principal": {
				"AWS": "arn:aws:iam::<ACCOUNT_ID>:role/<LAMBDA_ROLE_NAME>"
			},
			"Action": [
				"kms:Decrypt",
				"kms:GenerateDataKey*"
			],
			"Resource": "*",
			"Condition": {
				"StringEquals": {
					"kms:ViaService": "s3.ap-northeast-1.amazonaws.com"
				}
			}
		}
	]
}


		

Key Policy Explanation:

• Principal: Specifies the IAM role (in this case, the Lambda execution role) that can use this policy
• Condition (kms:ViaService): A restriction that allows operations only through S3 (prevents usage from other services)

Step 2: Lambda Function Implementation and Testing#### 2-1. Basic Lambda Implementation

Creating a Lambda function in Python 3.13 to process S3 encrypted with KMS:

			
			import json
import boto3
import os

# --- Create S3 client ---
s3 = boto3.client("s3")

# Get S3 bucket name and KMS key ARN from environment variables
BUCKET = os.environ["BUCKET"]
KMS_KEY_ARN = os.environ["KMS_KEY_ARN"]

def lambda_handler(event, context):
    # --- Test data to store (fixed value) ---
    payload = {"message": "Hello from Lambda with SSE-KMS"}
    # Convert to JSON format and encode to byte array
    body = json.dumps(payload).encode("utf-8")

    # --- PutObject: Save encrypted to S3 ---
    s3.put_object(
        Bucket=BUCKET,                      # Target bucket
        Key="sample.json",                  # Key of object to save (equivalent to file name)
        Body=body,                          # Data body to save
        ContentType="application/json",     # Content type (specify JSON)
        ServerSideEncryption="aws:kms",     # Specify server-side encryption method (KMS)
        SSEKMSKeyId=KMS_KEY_ARN,            # Specify the KMS key ARN to use
        BucketKeyEnabled=True               # Optional: Enable S3 Bucket Keys (reduces KMS requests)
    )

    # --- GetObject: Retrieve the saved encrypted object ---
    response = s3.get_object(
        Bucket=BUCKET, 
        Key="sample.json"
    )

    # Decode the retrieved data and load as JSON
    content = response['Body'].read().decode('utf-8')
    data = json.loads(content)

    # --- Lambda return value ---
    return {
        'statusCode': 200,
        'body': json.dumps({
            'message': 'Successfully processed KMS encrypted S3 object',
            'data': data
        })
    }


		

2-2. Setting Environment Variables

Set the following in Lambda environment variables:

• BUCKET: Target S3 bucket name
• KMS_KEY_ARN: ARN of the KMS key to use

2-3. Testing the Lambda Function

Set the handler to lambda_function.lambda_handler and
run the Lambda test.

Step 3: Verification in S3 Console#### 3-1. Checking the Encryption Status

Let's check the encryption settings of the created object.
An object called sample.json encrypted with the target KMS will be created in S3.

From the object's properties, we can confirm that it is properly encrypted with SSE-KMS.

3-2. Operation Verification Results

When execution is successful, a response like the following will be returned:

			
			{
  "statusCode": 200,
  "body": "{\"message\": \"Successfully processed KMS encrypted S3 object\", \"data\": {\"message\": \"Hello from Lambda with SSE-KMS\"}}"
}

		

Summary

In this article, we tested how to process S3 data encrypted with KMS using Lambda.

By correctly configuring IAM policies and KMS key policies, we confirmed that data is automatically encrypted when stored in S3 and decrypted when retrieved (however, SSE-KMS must be specified when saving). It's important to note that with this method, encryption and decryption are automatically handled by KMS and S3, not by the application code.

I hope this article is helpful to someone.

References

• Creating a symmetric KMS key - AWS Key Management Service
• S3 - Boto3 1.40.21 documentation
• Specifying server-side encryption with AWS KMS (SSE-KMS) - Amazon Simple Storage Service

About Annotation Inc.

Annotation Inc., as a group company of Classmethod, continues to challenge itself to be a company that can deliver "Operation Excellence." Under the slogan "Work like yourself, live like yourself," diverse members with various backgrounds have been providing services to customers through highly flexible work styles. We are currently recruiting members who can help us build our company. If you are interested, please visit the Annotation Inc. website.