I Tried Direct Delivery to S3 Tables with the New Kinesis Data Streams Feature "Streaming tables"

Introduction

On August 28, 2026, Streaming tables for Amazon Kinesis Data Streams was announced. It is a feature that delivers data flowing through a stream to Apache Iceberg tables on S3 Tables without writing a consumer. On the 29th, delivery to general purpose S3 buckets was also announced, so KDS alone now offers two delivery destinations: S3 Tables and general purpose S3 buckets.

https://aws.amazon.com/about-aws/whats-new/2026/08/kinesis/data-delivery-s3-tables

https://dev.classmethod.jp/articles/kinesis-data-streams-s3-general-purpose-delivery/

In this article, I created one delivery to an S3 Tables destination and one delivery to a general purpose S3 bucket destination on a single stream. To see how the same records arrived at both destinations, I ran the same SQL in Athena, changing only the table it referenced, and compared the results.

Verification

Prerequisites for Streaming tables

Both Streaming tables and general purpose S3 delivery are available only for On-Demand Standard or On-Demand Advantage streams. They cannot be used with Provisioned mode streams (Developer Guide). A single stream can have at most two deliveries, one for S3 Tables and one for general purpose S3, and this limit cannot be raised (same quota table).

Delivery to S3 Tables has additional requirements that delivery to general purpose S3 buckets does not.

Delivery to S3 Tables requires you to specify a schema registered in AWS Glue Schema Registry. An S3 bucket for a dead-letter queue, which is the output destination for error information about records that could not be delivered, is also required (Developer Guide). The column used for partitioning must be of type timestamptz in the Iceberg table. In JSON Schema, you define it as a string type with format: date-time (Developer Guide). Cross-account delivery and cross-Region delivery are not supported, so the stream, table bucket, and Glue Schema Registry must be placed in the same account and the same Region (Developer Guide).

Verification environment

Preparing related resources

I created the resources that serve as the delivery destinations and the input source.

# ODS stream as the input source
aws kinesis create-stream --stream-name kdsst-0902-stream \
  --stream-mode-details StreamMode=ON_DEMAND

# Table bucket that serves as the destination for S3 Tables delivery
aws s3tables create-table-bucket --name kdsst-0902-tb

# General purpose S3 bucket serving as both the general purpose S3 delivery destination and the dead-letter queue
aws s3api create-bucket --bucket kdsst-0902-123456789012 \
  --create-bucket-configuration LocationConstraint=ap-northeast-1

Next, I prepared the JSON Schema required for S3 Tables delivery as event-schema.json.

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "title": "event",
  "type": "object",
  "properties": {
    "event_time": { "type": "string", "format": "date-time" },
    "user_id": { "type": "string" },
    "action": { "type": "string" },
    "region": { "type": "string" },
    "latency_ms": { "type": "integer" },
    "payload": { "type": "string" }
  },
  "required": ["event_time", "user_id", "action"]
}

The items listed in required become NOT NULL columns in the Iceberg table. The event_time used for partitioning is the timestamptz column mentioned above.

I registered this file in Glue Schema Registry. I created the registry first and then registered the schema.

aws glue create-registry --registry-name kdsst-0902-registry

aws glue create-schema \
  --registry-id RegistryName=kdsst-0902-registry \
  --schema-name events \
  --data-format JSON \
  --compatibility NONE \
  --schema-definition file://event-schema.json

I specified JSON as the data format and NONE as the compatibility. The schema ARN included in the creation result was used when creating the deliveries that follow.

I created a service execution role that Kinesis Data Streams assumes for delivery processing. The trust policy was prepared as trust-policy.json.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": { "Service": "kinesis.amazonaws.com" },
      "Action": "sts:AssumeRole",
      "Condition": {
        "StringEquals": { "aws:SourceAccount": "123456789012" },
        "ArnLike": { "aws:SourceArn": "arn:aws:kinesis:ap-northeast-1:123456789012:channel/*" }
      }
    }
  ]
}

I put the permissions policy together in permission-policy.json. The permissions configured in this verification are of three kinds: S3 Tables actions on the destination table bucket, Glue Schema Registry actions for reading the schema, and writes to the general purpose S3 bucket. This bucket serves as both the general purpose S3 delivery destination and the dead-letter queue. For the S3 Tables actions, I granted all of the ones listed in the policy example in the Developer Guide, even though this configuration does not use a CMK.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "S3TablesAccess",
      "Effect": "Allow",
      "Action": [
        "s3tables:GetTable",
        "s3tables:GetTableBucket",
        "s3tables:GetTableMetadataLocation",
        "s3tables:UpdateTableMetadataLocation",
        "s3tables:CreateTable",
        "s3tables:CreateNamespace",
        "s3tables:PutTableData",
        "s3tables:GetTableData",
        "s3tables:TagResource",
        "s3tables:PutTableRecordExpirationConfiguration",
        "s3tables:PutTableEncryption"
      ],
      "Resource": [
        "arn:aws:s3tables:ap-northeast-1:123456789012:bucket/kdsst-0902-tb",
        "arn:aws:s3tables:ap-northeast-1:123456789012:bucket/kdsst-0902-tb/table/*"
      ]
    },
    {
      "Sid": "GlueSchemaRegistryAccess",
      "Effect": "Allow",
      "Action": [
        "glue:GetSchemaVersion"
      ],
      "Resource": [
        "arn:aws:glue:ap-northeast-1:123456789012:registry/kdsst-0902-registry",
        "arn:aws:glue:ap-northeast-1:123456789012:schema/kdsst-0902-registry/events"
      ]
    },
    {
      "Sid": "DeliveryBucketList",
      "Effect": "Allow",
      "Action": [
        "s3:ListBucket",
        "s3:ListBucketMultipartUploads"
      ],
      "Resource": [
        "arn:aws:s3:::kdsst-0902-123456789012",
        "arn:aws:s3:::kdsst-0902-123456789012/*"
      ]
    },
    {
      "Sid": "DeliveryBucketWrite",
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",
        "s3:CreateMultipartUpload",
        "s3:UploadPart",
        "s3:CompleteMultipartUpload",
        "s3:ListMultipartUploads",
        "s3:ListMultipartUploadParts"
      ],
      "Resource": [
        "arn:aws:s3:::kdsst-0902-123456789012/*"
      ]
    }
  ]
}

I created the role and attached this policy.

aws iam create-role --role-name kdsst-0902-delivery-role \
  --assume-role-policy-document file://trust-policy.json

aws iam put-role-policy --role-name kdsst-0902-delivery-role \
  --policy-name kdsst-0902-delivery-policy \
  --policy-document file://permission-policy.json

Creating the deliveries

This is the JSON specified for create-channel for the S3 Tables destination. I saved it as create-channel-tables.json.

{
  "ChannelName": "kdsst-0902-tables",
  "ServiceExecutionRoleARN": "arn:aws:iam::123456789012:role/kdsst-0902-delivery-role",
  "StreamConfigurationList": [
    {
      "StreamARN": "arn:aws:kinesis:ap-northeast-1:123456789012:stream/kdsst-0902-stream",
      "RecordConfiguration": {
        "RecordFormatType": "JSON",
        "GSRSchemaARN": "arn:aws:glue:ap-northeast-1:123456789012:schema/kdsst-0902-registry/events"
      }
    }
  ],
  "S3TablesDestinationConfiguration": {
    "DataFreshnessInSeconds": 300,
    "DeadLetterQueueS3Configuration": {
      "BucketARN": "arn:aws:s3:::kdsst-0902-123456789012",
      "ExpectedBucketOwner": "123456789012",
      "ErrorOutputPrefix": "dlq-tables/"
    },
    "S3TablesConfigurationList": [
      {
        "TableBucketARN": "arn:aws:s3tables:ap-northeast-1:123456789012:bucket/kdsst-0902-tb",
        "Namespace": "kdsdemo",
        "TableName": "events",
        "CompressionType": "ZSTD",
        "PartitionSpec": {
          "PartitionFields": [
            { "Transform": "TIME_HOUR", "SourceName": "event_time" }
          ]
        }
      }
    ]
  }
}

If you specify JSON for RecordFormatType and pass GSRSchemaARN, you can ingest JSON records that have not been encoded with the Glue Schema Registry serializer. You can also choose GSR_JSON, where the serializer embeds the schema ID in each record (Developer Guide). I used JSON this time.

Next is the JSON specified for create-channel for the general purpose S3 destination. I saved it as create-channel-s3.json.

{
  "ChannelName": "kdsst-0902-s3",
  "ServiceExecutionRoleARN": "arn:aws:iam::123456789012:role/kdsst-0902-delivery-role",
  "StreamConfigurationList": [
    {
      "StreamARN": "arn:aws:kinesis:ap-northeast-1:123456789012:stream/kdsst-0902-stream",
      "RecordConfiguration": { "RecordFormatType": "JSON" }
    }
  ],
  "S3DestinationConfiguration": {
    "DataFreshnessInSeconds": 300,
    "DeadLetterQueueS3Configuration": {
      "BucketARN": "arn:aws:s3:::kdsst-0902-123456789012",
      "ExpectedBucketOwner": "123456789012",
      "ErrorOutputPrefix": "dlq-s3/"
    },
    "StorageConfiguration": {
      "BucketARN": "arn:aws:s3:::kdsst-0902-123456789012",
      "ExpectedBucketOwner": "123456789012",
      "OutputKeyTemplate": "data/!{yyyy}/!{MM}/!{dd}/!{HH}/!{channel-name}!{extension}",
      "StorageClass": "STANDARD",
      "CompressionType": "GZIP"
    }
  }
}

I specified each input file in create-channel to create the deliveries.

aws kinesis create-channel --cli-input-json file://create-channel-tables.json

aws kinesis create-channel --cli-input-json file://create-channel-s3.json

The destination settings are split into S3TablesDestinationConfiguration and S3DestinationConfiguration. I ran the two create-channel commands at 04:12:25Z and 04:12:46Z, and by 04:12:57Z both were ACTIVE.

The destination table was created automatically by Streaming tables when the delivery was created this time. You cannot deliver to an existing table (Developer Guide).

{
    "name": "events",
    "type": "customer",
    "tableARN": "arn:aws:s3tables:ap-northeast-1:123456789012:bucket/kdsst-0902-tb/table/<table-id>",
    "namespace": [
        "kdsdemo"
    ],
    "metadataLocation": "s3://<table-id-prefix>-<suffix>--table-s3/metadata/00001-<metadata-id>.metadata.json",
    "warehouseLocation": "s3://<table-id-prefix>-<suffix>--table-s3",
    "createdBy": "123456789012",
    "managedByService": "kinesis.amazonaws.com",
    "ownerAccountId": "123456789012",
    "format": "ICEBERG"
}

The automatically created table is a service-managed table whose managedByService is kinesis.amazonaws.com. Its schema and table properties are not meant to be modified from outside (Developer Guide).

In Athena, the columns from the JSON Schema could be referenced with the following types.

Only event_time became the timestamp with time zone type; the other columns could be referenced as the Athena types corresponding to their JSON Schema types.

Ingesting data

With both deliveries running, I ingested records into the stream.

For the payload column, I used a value consisting of the same character repeated 900 times. Because this content compresses easily with gzip, the size and scan volume comparisons that follow are measurements under conditions favorable to general purpose S3 delivery.

The records I ingested as not conforming to the schema were one missing action and one with a string in latency_ms. I saved them separately as bad-record-1.json and bad-record-2.json.

{"event_time":"2026-09-02T04:15:00.000Z","user_id":"user-bad1","region":"ap-northeast-1","latency_ms":10,"payload":"missing-action"}
{"event_time":"2026-09-02T04:15:01.000Z","user_id":"user-bad2","action":"view","region":"ap-northeast-1","latency_ms":"not-a-number","payload":"bad-type"}

I ingested these two individually with PutRecord. Because I passed raw JSON, I specified raw-in-base64-out for --cli-binary-format.

aws kinesis put-record --stream-name kdsst-0902-stream \
  --partition-key bad1 \
  --cli-binary-format raw-in-base64-out \
  --data file://bad-record-1.json

I set DataFreshnessInSeconds to 300 for both, but I could confirm the first object and the Iceberg table creation only about 13 minutes after ingestion. The first delivery includes preparing the destination, so data does not arrive within this value.

Checking the ingested data

I checked the delivery throughput with CloudWatch metrics. These are the metrics with the DeliveryToIceberg and DeliveryToS3 prefixes in the AWS/Kinesis namespace, aggregated with the Sum statistic for 04:10Z to 05:15Z on 2026-09-02.

The input byte counts matched exactly across the two deliveries. From this result, I could confirm that the two deliveries each read the same records independently. The output volume on the Streaming tables side was 264,801 bytes, about one third of the 799,726 bytes for general purpose S3 delivery. The former is Parquet with ZSTD, and the latter is GZIP-compressed JSON. General purpose S3 delivery produced 3 actual objects, and their total size of 799,726 bytes matched BytesOut.

In this retrieval period, no values were returned for the record-count metrics on the Streaming tables side, and I could confirm only the Bytes metrics and DLQDeliverySuccess. This single observation does not generalize whether those metrics are available.

The catalog is referenced as s3tablescatalog/<table bucket name>, the database as the namespace, and the table as the table name specified in the delivery.

SELECT action, count(*) AS n
FROM "s3tablescatalog/kdsst-0902-tb"."kdsdemo"."events"
GROUP BY action
ORDER BY action

In this account, the integration with analytics services had already been enabled, so I could query without adding extra permissions in Lake Formation. A Glue federated catalog called s3tablescatalog, covering all table buckets in the account, already existed. The default permission was also ALL for IAM_ALLOWED_PRINCIPALS. In an account without this integration, you first need to enable the integration with analytics services on the S3 Tables side (User Guide).

Next, I compared the Athena scan volumes. For general purpose S3 delivery, I created an external table that reads the delivered GZIP JSON.

CREATE EXTERNAL TABLE IF NOT EXISTS kdsst_0902.events_json (
  event_time string,
  user_id string,
  action string,
  region string,
  latency_ms bigint,
  payload string
)
ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe'
LOCATION 's3://kdsst-0902-123456789012/data/'

The SQL I ran differed only in the FROM target; the selected columns and aggregation were the same in both.

The aggregation needed about 1/96 of the scan volume. Parquet stores data by column, so only the action column needs to be read, whereas the JSON side has to read entire objects. For getting the total count, the scan on the Streaming tables side was 0 bytes, because it could be answered from the Iceberg metadata alone.

Here are the results of the count per action side by side.

The handling of the two records that do not conform to the schema differed by destination. On the Streaming tables side, the two records that failed schema validation were sent to the dead-letter queue and did not enter the table. With general purpose S3 delivery, records were stored without validation, and in the Athena results the record missing action appeared as a row with a NULL action. The record with a string in latency_ms was counted as a row with action view.

The dead-letter queue objects were placed under dlq-tables/DESERIALIZATION_ERROR/2026/09/02/04/. An error type directory is inserted after the specified ErrorOutputPrefix, followed by time directories.

{"approximateArrivalTimestamp":1788322490093,"streamArn":"arn:aws:kinesis:ap-northeast-1:123456789012:stream/kdsst-0902-stream","shardId":"shardId-000000000001","sequenceNumber":"49677952019003285752333393925921343430349053876775157778","errorCode":"Iceberg.MissingColumnWithinRecord","errorMessage":"Data Channel is unable to deliver to the table since a required column within the schema is missing within the record."}
{"approximateArrivalTimestamp":1788322490551,"streamArn":"arn:aws:kinesis:ap-northeast-1:123456789012:stream/kdsst-0902-stream","shardId":"shardId-000000000003","sequenceNumber":"49677952019047887242730455172492139211962632343367188530","errorCode":"Iceberg.MalformedFieldWithinRecord","errorMessage":"Data Channel is unable to convert column data in your record to the column type specified within the schema."}

The record bodies were not included. In this case, the objects contained only the stream ARN, shard ID, sequence number, and error type, written as JSON with one record per line. To re-ingest, you need to re-read from the stream using the sequence number as a clue.

Cleanup

Because deliveries continue to be billed until deleted, I deleted the two deliveries first after finishing the verification. The table and namespace automatically created by the delivery were deleted individually before deleting the table bucket. I carried out the cleanup in this order this time.

aws kinesis delete-channel --channel-arn <channel-arn>
aws kinesis delete-stream --stream-name kdsst-0902-stream --enforce-consumer-deletion

aws s3tables delete-table --table-bucket-arn <table-bucket-arn> --namespace kdsdemo --name events
aws s3tables delete-namespace --table-bucket-arn <table-bucket-arn> --namespace kdsdemo
aws s3tables delete-table-bucket --table-bucket-arn <table-bucket-arn>

aws s3 rm s3://kdsst-0902-<account-id> --recursive
aws s3api delete-bucket --bucket kdsst-0902-<account-id>

aws glue delete-schema --schema-id SchemaArn=<schema-arn>
aws glue delete-registry --registry-id RegistryName=kdsst-0902-registry
aws glue delete-table --database-name kdsst_0902 --name events_json

aws iam delete-role-policy --role-name kdsst-0902-delivery-role --policy-name kdsst-0902-delivery-policy
aws iam delete-role --role-name kdsst-0902-delivery-role

Pricing

Delivery pricing differs per destination. I calculated using the On-Demand Standard, US East (Ohio) unit prices, normalized to 15,000 GB per month (500 GB/day × 30 days). This is because the example calculations on the pricing page assume different data volumes for each destination.

The $0.035/GB for Streaming tables is set lower than delivering to Iceberg tables via Amazon Data Firehose. The Firehose unit price was $0.045/GB for an Iceberg destination with Kinesis Data Streams as the source (from the example calculation in Amazon Data Firehose pricing).

Billing is based on the volume of data successfully delivered. For Streaming tables it is calculated on the pre-compression size, so the delivery charge does not go down even if the data becomes smaller after conversion to Parquet. The reduction in size pays off in Athena's scan charges ($5/TB), which are proportional to the amount read.

Separately from this, charges for the stream itself apply. The billable items for On-Demand Standard are stream hours, data ingestion, data retrieval, and data retention period.

Summary

With Streaming tables, I was able to deliver Kinesis Data Streams data to Iceberg tables on S3 Tables without placing a custom consumer in between. Because the data arrives as Parquet and Iceberg metadata, Athena reads only the columns it needs and returns the total record count from metadata alone.

If you currently write to Iceberg tables through a custom consumer or another service, you can replace that setup with Streaming tables as long as you can recreate the destination table. Compared with a setup that puts Amazon Data Firehose in between, the unit price is also lower, and all you need is a single channel. If you analyze with Athena and can take advantage of Parquet and Iceberg metadata, give it a try.