I tried the new Kinesis Data Streams feature "Streaming tables" for direct delivery to S3 Tables

Introduction

On August 28, 2026, Streaming tables for Amazon Kinesis Data Streams was announced. This feature allows data flowing through a stream to be delivered to Apache Iceberg tables on S3 Tables without writing a consumer. On the 29th, delivery to general-purpose S3 buckets was also announced, making S3 Tables and general-purpose S3 buckets the two delivery destination options available with KDS alone.

https://aws.amazon.com/about-aws/whats-new/2026/08/kinesis/data-delivery-s3-tables

https://dev.classmethod.jp/articles/kinesis-data-streams-s3-general-purpose-delivery/

In this article, I created one delivery to an S3 Tables destination and one delivery to a general-purpose S3 bucket destination from a single stream. I ran the same SQL queries in Athena, only swapping the referenced source, to compare how the same records were delivered to both destinations.

Validation Details

Prerequisites for Streaming tables

Both Streaming tables and general-purpose S3 delivery apply to streams in On-Demand Standard or On-Demand Advantage mode. They cannot be used with Provisioned mode streams (Developer Guide). The maximum number of deliveries that can be set up per stream is two total — one for S3 Tables and one for general-purpose S3 — and this cannot be increased (same quota table).

Delivery to S3 Tables has additional requirements that delivery to general-purpose S3 buckets does not.

For delivery to S3 Tables, specifying a schema registered in AWS Glue Schema Registry is mandatory. A dead-letter queue S3 bucket, which serves as the output destination for error information about records that could not be delivered, is also required (Developer Guide). Columns used for partitioning must be of type timestamptz in the Iceberg table. In JSON Schema, they are defined as string type with format: date-time (Developer Guide). Cross-account and cross-region delivery are not supported, so the stream, table bucket, and Glue Schema Registry must be placed in the same account and same region (Developer Guide).

Validation Environment

Preparing Related Resources

I created the resources that would serve as delivery destinations and input sources.

# ODS stream as input source
aws kinesis create-stream --stream-name kdsst-0902-stream \
  --stream-mode-details StreamMode=ON_DEMAND

# Table bucket as destination for S3 Tables delivery
aws s3tables create-table-bucket --name kdsst-0902-tb

# General-purpose S3 bucket serving as both the destination for general-purpose S3 delivery and the dead-letter queue
aws s3api create-bucket --bucket kdsst-0902-123456789012 \
  --create-bucket-configuration LocationConstraint=ap-northeast-1

Next, I prepared the JSON Schema required for S3 Tables delivery as event-schema.json.

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "title": "event",
  "type": "object",
  "properties": {
    "event_time": { "type": "string", "format": "date-time" },
    "user_id": { "type": "string" },
    "action": { "type": "string" },
    "region": { "type": "string" },
    "latency_ms": { "type": "integer" },
    "payload": { "type": "string" }
  },
  "required": ["event_time", "user_id", "action"]
}

Items listed in required become NOT NULL columns in the Iceberg table. The event_time used for partitioning corresponds to the timestamptz column mentioned above.

I registered this file in the Glue Schema Registry. I created the registry first, then registered the schema.

aws glue create-registry --registry-name kdsst-0902-registry

aws glue create-schema \
  --registry-id RegistryName=kdsst-0902-registry \
  --schema-name events \
  --data-format JSON \
  --compatibility NONE \
  --schema-definition file://event-schema.json

I specified JSON for the data format and NONE for compatibility. The schema ARN included in the creation result was specified in the subsequent delivery creation.

I created a service execution role for Kinesis Data Streams to assume during delivery processing. The trust policy was prepared as trust-policy.json.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": { "Service": "kinesis.amazonaws.com" },
      "Action": "sts:AssumeRole",
      "Condition": {
        "StringEquals": { "aws:SourceAccount": "123456789012" },
        "ArnLike": { "aws:SourceArn": "arn:aws:kinesis:ap-northeast-1:123456789012:channel/*" }
      }
    }
  ]
}

The permission policy was consolidated in permission-policy.json. The permissions configured in this validation are of three types: S3 Tables actions for the destination table bucket, Glue Schema Registry actions for reading the schema, and write access to the general-purpose S3 bucket. The latter bucket serves as both the destination for general-purpose S3 delivery and the dead-letter queue. For S3 Tables actions, all actions listed in the policy examples in the Developer Guide were granted, even for a configuration that does not use CMK.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "S3TablesAccess",
      "Effect": "Allow",
      "Action": [
        "s3tables:GetTable",
        "s3tables:GetTableBucket",
        "s3tables:GetTableMetadataLocation",
        "s3tables:UpdateTableMetadataLocation",
        "s3tables:CreateTable",
        "s3tables:CreateNamespace",
        "s3tables:PutTableData",
        "s3tables:GetTableData",
        "s3tables:TagResource",
        "s3tables:PutTableRecordExpirationConfiguration",
        "s3tables:PutTableEncryption"
      ],
      "Resource": [
        "arn:aws:s3tables:ap-northeast-1:123456789012:bucket/kdsst-0902-tb",
        "arn:aws:s3tables:ap-northeast-1:123456789012:bucket/kdsst-0902-tb/table/*"
      ]
    },
    {
      "Sid": "GlueSchemaRegistryAccess",
      "Effect": "Allow",
      "Action": [
        "glue:GetSchemaVersion"
      ],
      "Resource": [
        "arn:aws:glue:ap-northeast-1:123456789012:registry/kdsst-0902-registry",
        "arn:aws:glue:ap-northeast-1:123456789012:schema/kdsst-0902-registry/events"
      ]
    },
    {
      "Sid": "DeliveryBucketList",
      "Effect": "Allow",
      "Action": [
        "s3:ListBucket",
        "s3:ListBucketMultipartUploads"
      ],
      "Resource": [
        "arn:aws:s3:::kdsst-0902-123456789012",
        "arn:aws:s3:::kdsst-0902-123456789012/*"
      ]
    },
    {
      "Sid": "DeliveryBucketWrite",
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",
        "s3:CreateMultipartUpload",
        "s3:UploadPart",
        "s3:CompleteMultipartUpload",
        "s3:ListMultipartUploads",
        "s3:ListMultipartUploadParts"
      ],
      "Resource": [
        "arn:aws:s3:::kdsst-0902-123456789012/*"
      ]
    }
  ]
}

I created the role and attached this policy.

aws iam create-role --role-name kdsst-0902-delivery-role \
  --assume-role-policy-document file://trust-policy.json

aws iam put-role-policy --role-name kdsst-0902-delivery-role \
  --policy-name kdsst-0902-delivery-policy \
  --policy-document file://permission-policy.json

Creating Deliveries

Here is the JSON specified in the create-channel for the S3 Tables destination. I saved it as create-channel-tables.json.

{
  "ChannelName": "kdsst-0902-tables",
  "ServiceExecutionRoleARN": "arn:aws:iam::123456789012:role/kdsst-0902-delivery-role",
  "StreamConfigurationList": [
    {
      "StreamARN": "arn:aws:kinesis:ap-northeast-1:123456789012:stream/kdsst-0902-stream",
      "RecordConfiguration": {
        "RecordFormatType": "JSON",
        "GSRSchemaARN": "arn:aws:glue:ap-northeast-1:123456789012:schema/kdsst-0902-registry/events"
      }
    }
  ],
  "S3TablesDestinationConfiguration": {
    "DataFreshnessInSeconds": 300,
    "DeadLetterQueueS3Configuration": {
      "BucketARN": "arn:aws:s3:::kdsst-0902-123456789012",
      "ExpectedBucketOwner": "123456789012",
      "ErrorOutputPrefix": "dlq-tables/"
    },
    "S3TablesConfigurationList": [
      {
        "TableBucketARN": "arn:aws:s3tables:ap-northeast-1:123456789012:bucket/kdsst-0902-tb",
        "Namespace": "kdsdemo",
        "TableName": "events",
        "CompressionType": "ZSTD",
        "PartitionSpec": {
          "PartitionFields": [
            { "Transform": "TIME_HOUR", "SourceName": "event_time" }
          ]
        }
      }
    ]
  }
}

By specifying JSON for RecordFormatType and passing GSRSchemaARN, you can put in JSON records that have not been encoded with the Glue Schema Registry serializer. You can also choose GSR_JSON, which embeds the schema ID in records using the Glue Schema Registry serializer (Developer Guide). I used JSON this time.

Next, here is the JSON specified in the create-channel for the general-purpose S3 destination. I saved it as create-channel-s3.json.

{
  "ChannelName": "kdsst-0902-s3",
  "ServiceExecutionRoleARN": "arn:aws:iam::123456789012:role/kdsst-0902-delivery-role",
  "StreamConfigurationList": [
    {
      "StreamARN": "arn:aws:kinesis:ap-northeast-1:123456789012:stream/kdsst-0902-stream",
      "RecordConfiguration": { "RecordFormatType": "JSON" }
    }
  ],
  "S3DestinationConfiguration": {
    "DataFreshnessInSeconds": 300,
    "DeadLetterQueueS3Configuration": {
      "BucketARN": "arn:aws:s3:::kdsst-0902-123456789012",
      "ExpectedBucketOwner": "123456789012",
      "ErrorOutputPrefix": "dlq-s3/"
    },
    "StorageConfiguration": {
      "BucketARN": "arn:aws:s3:::kdsst-0902-123456789012",
      "ExpectedBucketOwner": "123456789012",
      "OutputKeyTemplate": "data/!{yyyy}/!{MM}/!{dd}/!{HH}/!{channel-name}!{extension}",
      "StorageClass": "STANDARD",
      "CompressionType": "GZIP"
    }
  }
}

I specified each input file in create-channel to create the deliveries.

aws kinesis create-channel --cli-input-json file://create-channel-tables.json

aws kinesis create-channel --cli-input-json file://create-channel-s3.json

The destination specifications are separated into S3TablesDestinationConfiguration and S3DestinationConfiguration. I executed the two create-channel commands at 04:12:25Z and 04:12:46Z, and both were ACTIVE by 04:12:57Z.

The destination table was automatically created by Streaming tables at the time of this delivery creation. Delivery to existing tables is not supported (Developer Guide).

{
    "name": "events",
    "type": "customer",
    "tableARN": "arn:aws:s3tables:ap-northeast-1:123456789012:bucket/kdsst-0902-tb/table/<table-id>",
    "namespace": [
        "kdsdemo"
    ],
    "metadataLocation": "s3://<table-id-prefix>-<suffix>--table-s3/metadata/00001-<metadata-id>.metadata.json",
    "warehouseLocation": "s3://<table-id-prefix>-<suffix>--table-s3",
    "createdBy": "123456789012",
    "managedByService": "kinesis.amazonaws.com",
    "ownerAccountId": "123456789012",
    "format": "ICEBERG"
}

The auto-created table is a service-managed table with managedByService set to kinesis.amazonaws.com. It is not intended to have its schema or table properties modified externally (Developer Guide).

In Athena, the JSON Schema columns were accessible with the following types.

Only event_time became timestamp with time zone type, while other columns were accessible as Athena types corresponding to their JSON Schema types.

Ingesting Data

With both deliveries running, I put records into the stream.

For the payload column, I inserted a value consisting of the same character repeated 900 times. Since gzip is highly effective on this type of content, the size and scan amount comparison results below are measurements taken under conditions favorable to general-purpose S3 delivery.

The two records inserted as non-schema-conforming were: one record missing action, and one record with a string value in latency_ms. I saved them separately as bad-record-1.json and bad-record-2.json.

{"event_time":"2026-09-02T04:15:00.000Z","user_id":"user-bad1","region":"ap-northeast-1","latency_ms":10,"payload":"missing-action"}
{"event_time":"2026-09-02T04:15:01.000Z","user_id":"user-bad2","action":"view","region":"ap-northeast-1","latency_ms":"not-a-number","payload":"bad-type"}

These 2 records were put individually using PutRecord. I specified raw-in-base64-out for --cli-binary-format to pass raw JSON.

aws kinesis put-record --stream-name kdsst-0902-stream \
  --partition-key bad1 \
  --cli-binary-format raw-in-base64-out \
  --data file://bad-record-1.json

DataFreshnessInSeconds was set to 300 for both, but the first object and Iceberg table creation were confirmed approximately 13 minutes after ingestion. The first time includes destination preparation, so delivery does not arrive within this exact value.

Verifying the Ingested Data

I verified the delivery throughput using CloudWatch metrics. These are metrics with the DeliveryToIceberg and DeliveryToS3 prefixes in the AWS/Kinesis namespace, aggregated with the Sum statistic for the period 2026-09-02 04:10Z–05:15Z.

The input byte counts matched perfectly between the two deliveries. This result confirmed that the two deliveries each independently read the same records. The output volume was 264,801 bytes for the Streaming tables side, approximately one-third of the 799,726 bytes for general-purpose S3 delivery. The former is Parquet with ZSTD, and the latter is GZIP JSON. The actual objects for general-purpose S3 delivery numbered 3, and the total size of 799,726 bytes matched BytesOut.

During this retrieval period, record count metrics for the Streaming tables side returned no values, and only Bytes-related metrics and DLQDeliverySuccess values could be confirmed. This single observation should not be generalized to determine the availability of metrics overall.

The catalog is referenced as s3tablescatalog/<table bucket name>, the database as the namespace, and the table as the table name specified in the delivery.

SELECT action, count(*) AS n
FROM "s3tablescatalog/kdsst-0902-tb"."kdsdemo"."events"
GROUP BY action
ORDER BY action

In this account, analytics service integration was already enabled in advance, so it could be referenced without adding additional permissions in Lake Formation. A Glue federated catalog called s3tablescatalog already existed targeting all table buckets in the account. The default permissions were also ALL for IAM_ALLOWED_PRINCIPALS. In accounts where this integration is not set up, it is necessary to first enable analytics service integration on the S3 Tables side (User Guide).

Next, I compared the Athena scan amounts. For the general-purpose S3 delivery side, I created an external table to read the delivered GZIP JSON.

CREATE EXTERNAL TABLE IF NOT EXISTS kdsst_0902.events_json (
  event_time string,
  user_id string,
  action string,
  region string,
  latency_ms bigint,
  payload string
)
ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe'
LOCATION 's3://kdsst-0902-123456789012/data/'

The executed SQL differed only in the FROM reference, with the same columns and aggregations for both.

For the aggregation, the scan amount was approximately 1/96th. Since Parquet stores data by column, only the action column needs to be read, whereas the JSON side requires reading the entire object. The total record count retrieval was 0 bytes scanned on the Streaming tables side, because it could be answered using only Iceberg metadata.

Here are the count aggregation results by action side by side.

The handling of the 2 non-schema-conforming records differed between the destinations. On the Streaming tables side, the 2 records that failed schema validation were sent to the dead-letter queue and did not enter the table. With general-purpose S3 delivery, the records were stored without validation, and in the Athena results, the record missing action appeared as a row with a NULL action. The record with a string in latency_ms was counted as a row with action view.

The dead-letter queue object was placed under dlq-tables/DESERIALIZATION_ERROR/2026/09/02/04/. After the specified ErrorOutputPrefix, an error type directory is inserted, followed by a time-based directory.

{"approximateArrivalTimestamp":1788322490093,"streamArn":"arn:aws:kinesis:ap-northeast-1:123456789012:stream/kdsst-0902-stream","shardId":"shardId-000000000001","sequenceNumber":"49677952019003285752333393925921343430349053876775157778","errorCode":"Iceberg.MissingColumnWithinRecord","errorMessage":"Data Channel is unable to deliver to the table since a required column within the schema is missing within the record."}
{"approximateArrivalTimestamp":1788322490551,"streamArn":"arn:aws:kinesis:ap-northeast-1:123456789012:stream/kdsst-0902-stream","shardId":"shardId-000000000003","sequenceNumber":"49677952019047887242730455172492139211962632343367188530","errorCode":"Iceberg.MalformedFieldWithinRecord","errorMessage":"Data Channel is unable to convert column data in your record to the column type specified within the schema."}

The record body was not included. The objects in this case contained only the stream ARN, shard ID, sequence number, and error type, written in one-record-per-line JSON format. To resubmit, you would need to re-read from the stream using the sequence number.

Teardown

Since billing continues until deliveries are deleted, I deleted both deliveries first after completing the validation. Since the table and namespace are automatically created by the delivery, I deleted them individually before deleting the table bucket. The teardown was executed in this order.

aws kinesis delete-channel --channel-arn <channel-arn>
aws kinesis delete-stream --stream-name kdsst-0902-stream --enforce-consumer-deletion

aws s3tables delete-table --table-bucket-arn <table-bucket-arn> --namespace kdsdemo --name events
aws s3tables delete-namespace --table-bucket-arn <table-bucket-arn> --namespace kdsdemo
aws s3tables delete-table-bucket --table-bucket-arn <table-bucket-arn>

aws s3 rm s3://kdsst-0902-<account-id> --recursive
aws s3api delete-bucket --bucket kdsst-0902-<account-id>

aws glue delete-schema --schema-id SchemaArn=<schema-arn>
aws glue delete-registry --registry-id RegistryName=kdsst-0902-registry
aws glue delete-table --database-name kdsst_0902 --name events_json

aws iam delete-role-policy --role-name kdsst-0902-delivery-role --policy-name kdsst-0902-delivery-policy
aws iam delete-role --role-name kdsst-0902-delivery-role

Pricing

Delivery pricing differs by destination. I calculated using On-Demand Standard, US East (Ohio) unit prices, normalized to 15,000 GB per month (500 GB/day × 30 days), because the calculation examples on the pricing page use different data volumes for each destination.

The $0.035/GB for Streaming tables is set lower than delivering to Iceberg tables via Amazon Data Firehose. The Firehose unit price for an Iceberg destination with Kinesis Data Streams as the source was $0.045/GB (calculation example in Amazon Data Firehose pricing).

Billing applies to the amount of data successfully delivered. For Streaming tables, this is counted based on the pre-compression size, so delivery costs do not decrease even when data is converted to Parquet and becomes smaller. The reduction in size benefits Athena's scan-based billing ($5/TB), which is proportional to the amount read.

In addition to this, there are charges for the stream itself. The billable items for On-Demand Standard are stream hours, data ingestion, data retrieval, and data retention period.

Summary

With Streaming tables, Kinesis Data Streams data could be delivered to Iceberg tables on S3 Tables without an intermediary custom consumer. Because the data arrives as Parquet and Iceberg metadata, Athena reads only the necessary columns and returns total record counts from metadata alone.

Configurations that write to Iceberg tables via a custom consumer or another service can be replaced with Streaming tables, provided the destination table can be recreated. Compared to configurations that go through Amazon Data Firehose, the unit price is lower and only one channel is needed. If you are analyzing with Athena, which can take advantage of Parquet and Iceberg metadata, give it a try.