I tried out the new "Streaming tables" feature of Kinesis Data Streams for direct delivery to S3 Tables

Introduction

On August 28, 2026, Streaming tables for Amazon Kinesis Data Streams was announced. It is a feature that allows data flowing through a stream to be delivered to Apache Iceberg tables on S3 Tables without writing a consumer. On the 29th, delivery to general-purpose S3 buckets was also announced, making the available delivery destinations for KDS alone two options: S3 Tables and general-purpose S3 buckets.

https://aws.amazon.com/about-aws/whats-new/2026/08/kinesis/data-delivery-s3-tables

https://dev.classmethod.jp/articles/kinesis-data-streams-s3-general-purpose-delivery/

In this article, I created one delivery each to an S3 Tables destination and a general-purpose S3 bucket destination from a single stream. To compare how the same records arrived at both destinations, I ran the same SQL queries in Athena, only swapping out the referenced destination.

Verification Details

Prerequisites for Streaming tables

Both Streaming tables and general-purpose S3 delivery target On-Demand Standard or On-Demand Advantage streams. They cannot be used with Provisioned mode streams (Developer Guide). The number of deliveries that can be created per stream is a maximum of two total: one for S3 Tables and one for general-purpose S3, and this cannot be increased (same quota table).

Delivery to S3 Tables has additional requirements not present in delivery to general-purpose S3 buckets.

For delivery to S3 Tables, specifying a schema registered in AWS Glue Schema Registry is mandatory. An S3 bucket for a dead-letter queue, which serves as the output destination for error information about records that could not be delivered, is also required (Developer Guide). Columns used for partitioning must be of type timestamptz in the Iceberg table. In JSON Schema, they are defined as string type with format: date-time (Developer Guide). Since cross-account and cross-region delivery are not supported, the stream, table bucket, and Glue Schema Registry must be placed in the same account and same region (Developer Guide).

Verification Environment

Preparing Related Resources

I created the resources that would serve as delivery destinations and input sources.

# ODS stream as input source
aws kinesis create-stream --stream-name kdsst-0902-stream \
  --stream-mode-details StreamMode=ON_DEMAND

# Table bucket as destination for S3 Tables delivery
aws s3tables create-table-bucket --name kdsst-0902-tb

# General-purpose S3 bucket serving as both destination for general-purpose S3 delivery and dead-letter queue
aws s3api create-bucket --bucket kdsst-0902-123456789012 \
  --create-bucket-configuration LocationConstraint=ap-northeast-1

Next, I prepared the JSON Schema required for S3 Tables delivery as event-schema.json.

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "title": "event",
  "type": "object",
  "properties": {
    "event_time": { "type": "string", "format": "date-time" },
    "user_id": { "type": "string" },
    "action": { "type": "string" },
    "region": { "type": "string" },
    "latency_ms": { "type": "integer" },
    "payload": { "type": "string" }
  },
  "required": ["event_time", "user_id", "action"]
}

Items listed in required become NOT NULL columns in the Iceberg table. The event_time used for partitioning corresponds to the timestamptz column mentioned above.

I registered this file in the Glue Schema Registry. I first created the registry, then registered the schema.

aws glue create-registry --registry-name kdsst-0902-registry

aws glue create-schema \
  --registry-id RegistryName=kdsst-0902-registry \
  --schema-name events \
  --data-format JSON \
  --compatibility NONE \
  --schema-definition file://event-schema.json

I specified JSON for the data format and NONE for compatibility. The schema ARN included in the creation result was specified in the subsequent delivery creation.

I created the service execution role that Kinesis Data Streams assumes for delivery processing. The trust policy was prepared as trust-policy.json.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": { "Service": "kinesis.amazonaws.com" },
      "Action": "sts:AssumeRole",
      "Condition": {
        "StringEquals": { "aws:SourceAccount": "123456789012" },
        "ArnLike": { "aws:SourceArn": "arn:aws:kinesis:ap-northeast-1:123456789012:channel/*" }
      }
    }
  ]
}

The permission policy was compiled in permission-policy.json. The permissions configured in this verification are three types: S3 Tables actions for the destination table bucket, Glue Schema Registry actions for reading the schema, and write access to the general-purpose S3 bucket. This bucket serves as both the destination for general-purpose S3 delivery and the dead-letter queue. For S3 Tables actions, all actions listed in the Developer Guide policy examples were granted, even for a configuration not using CMK.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "S3TablesAccess",
      "Effect": "Allow",
      "Action": [
        "s3tables:GetTable",
        "s3tables:GetTableBucket",
        "s3tables:GetTableMetadataLocation",
        "s3tables:UpdateTableMetadataLocation",
        "s3tables:CreateTable",
        "s3tables:CreateNamespace",
        "s3tables:PutTableData",
        "s3tables:GetTableData",
        "s3tables:TagResource",
        "s3tables:PutTableRecordExpirationConfiguration",
        "s3tables:PutTableEncryption"
      ],
      "Resource": [
        "arn:aws:s3tables:ap-northeast-1:123456789012:bucket/kdsst-0902-tb",
        "arn:aws:s3tables:ap-northeast-1:123456789012:bucket/kdsst-0902-tb/table/*"
      ]
    },
    {
      "Sid": "GlueSchemaRegistryAccess",
      "Effect": "Allow",
      "Action": [
        "glue:GetSchemaVersion"
      ],
      "Resource": [
        "arn:aws:glue:ap-northeast-1:123456789012:registry/kdsst-0902-registry",
        "arn:aws:glue:ap-northeast-1:123456789012:schema/kdsst-0902-registry/events"
      ]
    },
    {
      "Sid": "DeliveryBucketList",
      "Effect": "Allow",
      "Action": [
        "s3:ListBucket",
        "s3:ListBucketMultipartUploads"
      ],
      "Resource": [
        "arn:aws:s3:::kdsst-0902-123456789012",
        "arn:aws:s3:::kdsst-0902-123456789012/*"
      ]
    },
    {
      "Sid": "DeliveryBucketWrite",
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",
        "s3:CreateMultipartUpload",
        "s3:UploadPart",
        "s3:CompleteMultipartUpload",
        "s3:ListMultipartUploads",
        "s3:ListMultipartUploadParts"
      ],
      "Resource": [
        "arn:aws:s3:::kdsst-0902-123456789012/*"
      ]
    }
  ]
}

I created the role and attached this policy.

aws iam create-role --role-name kdsst-0902-delivery-role \
  --assume-role-policy-document file://trust-policy.json

aws iam put-role-policy --role-name kdsst-0902-delivery-role \
  --policy-name kdsst-0902-delivery-policy \
  --policy-document file://permission-policy.json

Creating the Deliveries

The following is the JSON specified for the S3 Tables destination create-channel. It was saved as create-channel-tables.json.

{
  "ChannelName": "kdsst-0902-tables",
  "ServiceExecutionRoleARN": "arn:aws:iam::123456789012:role/kdsst-0902-delivery-role",
  "StreamConfigurationList": [
    {
      "StreamARN": "arn:aws:kinesis:ap-northeast-1:123456789012:stream/kdsst-0902-stream",
      "RecordConfiguration": {
        "RecordFormatType": "JSON",
        "GSRSchemaARN": "arn:aws:glue:ap-northeast-1:123456789012:schema/kdsst-0902-registry/events"
      }
    }
  ],
  "S3TablesDestinationConfiguration": {
    "DataFreshnessInSeconds": 300,
    "DeadLetterQueueS3Configuration": {
      "BucketARN": "arn:aws:s3:::kdsst-0902-123456789012",
      "ExpectedBucketOwner": "123456789012",
      "ErrorOutputPrefix": "dlq-tables/"
    },
    "S3TablesConfigurationList": [
      {
        "TableBucketARN": "arn:aws:s3tables:ap-northeast-1:123456789012:bucket/kdsst-0902-tb",
        "Namespace": "kdsdemo",
        "TableName": "events",
        "CompressionType": "ZSTD",
        "PartitionSpec": {
          "PartitionFields": [
            { "Transform": "TIME_HOUR", "SourceName": "event_time" }
          ]
        }
      }
    ]
  }
}

By specifying JSON for RecordFormatType and passing GSRSchemaARN, you can insert JSON records that have not been encoded with the Glue Schema Registry serializer. GSR_JSON, which embeds the schema ID in records using the serializer, is also available (Developer Guide). I used JSON this time.

Next is the JSON specified for the general-purpose S3 destination create-channel. It was saved as create-channel-s3.json.

{
  "ChannelName": "kdsst-0902-s3",
  "ServiceExecutionRoleARN": "arn:aws:iam::123456789012:role/kdsst-0902-delivery-role",
  "StreamConfigurationList": [
    {
      "StreamARN": "arn:aws:kinesis:ap-northeast-1:123456789012:stream/kdsst-0902-stream",
      "RecordConfiguration": { "RecordFormatType": "JSON" }
    }
  ],
  "S3DestinationConfiguration": {
    "DataFreshnessInSeconds": 300,
    "DeadLetterQueueS3Configuration": {
      "BucketARN": "arn:aws:s3:::kdsst-0902-123456789012",
      "ExpectedBucketOwner": "123456789012",
      "ErrorOutputPrefix": "dlq-s3/"
    },
    "StorageConfiguration": {
      "BucketARN": "arn:aws:s3:::kdsst-0902-123456789012",
      "ExpectedBucketOwner": "123456789012",
      "OutputKeyTemplate": "data/!{yyyy}/!{MM}/!{dd}/!{HH}/!{channel-name}!{extension}",
      "StorageClass": "STANDARD",
      "CompressionType": "GZIP"
    }
  }
}

I specified each input file to create-channel and created the deliveries.

aws kinesis create-channel --cli-input-json file://create-channel-tables.json

aws kinesis create-channel --cli-input-json file://create-channel-s3.json

The destination specifications are separated into S3TablesDestinationConfiguration and S3DestinationConfiguration. I ran the two create-channel commands at 04:12:25Z and 04:12:46Z, and by 04:12:57Z both had become ACTIVE.

The destination table was automatically created by Streaming tables when the delivery was created this time. Delivery to an existing table is not possible (Developer Guide).

{
    "name": "events",
    "type": "customer",
    "tableARN": "arn:aws:s3tables:ap-northeast-1:123456789012:bucket/kdsst-0902-tb/table/<table-id>",
    "namespace": [
        "kdsdemo"
    ],
    "metadataLocation": "s3://<table-id-prefix>-<suffix>--table-s3/metadata/00001-<metadata-id>.metadata.json",
    "warehouseLocation": "s3://<table-id-prefix>-<suffix>--table-s3",
    "createdBy": "123456789012",
    "managedByService": "kinesis.amazonaws.com",
    "ownerAccountId": "123456789012",
    "format": "ICEBERG"
}

The automatically created table is a service-managed table with managedByService set to kinesis.amazonaws.com. It is not meant to have its schema or table properties modified externally (Developer Guide).

In Athena, the JSON Schema columns were accessible with the following types.

Only event_time became a timestamp with time zone type; the other columns were accessible as the Athena types corresponding to their JSON Schema types.

Inserting Data

With two deliveries running, I inserted records into the stream.

The payload column was filled with a value of 900 of the same character. Since this content is gzip-friendly, the subsequent size and scan volume comparison results are measurements taken under conditions favorable to general-purpose S3 delivery.

The records inserted as non-conforming to the schema were one record missing action and one record with a string value in latency_ms. They were saved separately as bad-record-1.json and bad-record-2.json.

{"event_time":"2026-09-02T04:15:00.000Z","user_id":"user-bad1","region":"ap-northeast-1","latency_ms":10,"payload":"missing-action"}
{"event_time":"2026-09-02T04:15:01.000Z","user_id":"user-bad2","action":"view","region":"ap-northeast-1","latency_ms":"not-a-number","payload":"bad-type"}

These 2 records were inserted individually using PutRecord. To pass raw JSON, --cli-binary-format was set to raw-in-base64-out.

aws kinesis put-record --stream-name kdsst-0902-stream \
  --partition-key bad1 \
  --cli-binary-format raw-in-base64-out \
  --data file://bad-record-1.json

DataFreshnessInSeconds was set to 300 for both, but the first object and Iceberg table creation were confirmed approximately 13 minutes after insertion. On the first run, destination preparation is included, so delivery does not occur within this value.

Verifying Inserted Data

The delivery throughput was confirmed with CloudWatch metrics. These are metrics with the DeliveryToIceberg and DeliveryToS3 prefixes, aggregated with the Sum statistic for the period 2026-09-02 04:10Z–05:15Z in the AWS/Kinesis namespace.

The input byte counts matched exactly between the two deliveries. This result confirmed that the two deliveries each independently read the same records. The output volume was 264,801 bytes for Streaming tables, approximately one-third of the 799,726 bytes for general-purpose S3 delivery. The former is Parquet with ZSTD, and the latter is GZIP JSON. The actual objects in general-purpose S3 delivery were 3 files, with a total size of 799,726 bytes matching BytesOut.

During this retrieval period, the record count metrics for the Streaming tables side returned no values, and only the Bytes metrics and DLQDeliverySuccess values could be confirmed. This single observation should not be generalized to conclude the availability of metrics.

The catalog is referenced as s3tablescatalog/<table bucket name>, the database as the namespace, and the table as the table name specified in the delivery.

SELECT action, count(*) AS n
FROM "s3tablescatalog/kdsst-0902-tb"."kdsdemo"."events"
GROUP BY action
ORDER BY action

In this account, integration with analytics services had been enabled in advance, so the table could be referenced without adding additional permissions in Lake Formation. A Glue federated catalog called s3tablescatalog, targeting all table buckets in the account, already existed. The default permissions were also ALL for IAM_ALLOWED_PRINCIPALS. For accounts where this integration is not set up, it is necessary to first enable integration with analytics services on the S3 Tables side (User Guide).

Next, I compared the scan volumes in Athena. For the general-purpose S3 delivery side, I created an external table to read the delivered GZIP JSON.

CREATE EXTERNAL TABLE IF NOT EXISTS kdsst_0902.events_json (
  event_time string,
  user_id string,
  action string,
  region string,
  latency_ms bigint,
  payload string
)
ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe'
LOCATION 's3://kdsst-0902-123456789012/data/'

The SQL executed differed only in the FROM reference, with the same columns and aggregations for both.

For aggregation, the scan volume was approximately 1/96th. Since Parquet stores data by column, only the action column needs to be read, whereas the JSON side requires reading the entire object. For retrieving the total record count, the Streaming tables side scanned 0 bytes, because the response could be served from Iceberg metadata alone.

The results of the count aggregation by action are listed below.

The handling of the 2 non-conforming records differed between destinations. On the Streaming tables side, the 2 records that failed schema validation were sent to the dead-letter queue and did not enter the table. With general-purpose S3 delivery, the records were stored without validation, and in the Athena results, the 1 record missing action appeared as a row with a NULL action. The 1 record with a string in latency_ms was counted as a row with action view.

The dead-letter queue object was placed under dlq-tables/DESERIALIZATION_ERROR/2026/09/02/04/. An error type directory is inserted after the specified ErrorOutputPrefix, followed by a time directory beneath it.

{"approximateArrivalTimestamp":1788322490093,"streamArn":"arn:aws:kinesis:ap-northeast-1:123456789012:stream/kdsst-0902-stream","shardId":"shardId-000000000001","sequenceNumber":"49677952019003285752333393925921343430349053876775157778","errorCode":"Iceberg.MissingColumnWithinRecord","errorMessage":"Data Channel is unable to deliver to the table since a required column within the schema is missing within the record."}
{"approximateArrivalTimestamp":1788322490551,"streamArn":"arn:aws:kinesis:ap-northeast-1:123456789012:stream/kdsst-0902-stream","shardId":"shardId-000000000003","sequenceNumber":"49677952019047887242730455172492139211962632343367188530","errorCode":"Iceberg.MalformedFieldWithinRecord","errorMessage":"Data Channel is unable to convert column data in your record to the column type specified within the schema."}

The record body was not included. The objects in this case contained only the stream ARN, shard ID, sequence number, and error type, written in JSON format with one record per line. To resubmit, it is necessary to re-read from the stream using the sequence number.

Teardown

Since billing continues until deliveries are deleted, I deleted the two deliveries first after completing the verification. The table and namespace automatically created by the delivery were individually deleted before deleting the table bucket. The teardown was performed in this order.

aws kinesis delete-channel --channel-arn <channel-arn>
aws kinesis delete-stream --stream-name kdsst-0902-stream --enforce-consumer-deletion

aws s3tables delete-table --table-bucket-arn <table-bucket-arn> --namespace kdsdemo --name events
aws s3tables delete-namespace --table-bucket-arn <table-bucket-arn> --namespace kdsdemo
aws s3tables delete-table-bucket --table-bucket-arn <table-bucket-arn>

aws s3 rm s3://kdsst-0902-<account-id> --recursive
aws s3api delete-bucket --bucket kdsst-0902-<account-id>

aws glue delete-schema --schema-id SchemaArn=<schema-arn>
aws glue delete-registry --registry-id RegistryName=kdsst-0902-registry
aws glue delete-table --database-name kdsst_0902 --name events_json

aws iam delete-role-policy --role-name kdsst-0902-delivery-role --policy-name kdsst-0902-delivery-policy
aws iam delete-role --role-name kdsst-0902-delivery-role

Pricing

Delivery pricing differs per destination. Using On-Demand Standard, US East (Ohio) unit prices, I calculated for a uniform 15,000 GB per month (500 GB/day × 30 days). This is because the calculation examples on the pricing page use different data volumes per destination.

The $0.035/GB for Streaming tables is priced lower than delivering to Iceberg tables via Amazon Data Firehose. The Firehose unit price for an Iceberg destination using Kinesis Data Streams as the source was $0.045/GB (calculation example from Amazon Data Firehose pricing).

The billing target is the amount of data successfully delivered. For Streaming tables, billing is calculated based on the pre-compression size, so even if conversion to Parquet results in a smaller size, the delivery cost does not decrease. The savings from the smaller size apply to Athena scan charges ($5/TB), which are proportional to the amount read.

Separately, charges for the stream itself apply. For On-Demand Standard, billable items are stream hours, data ingestion, data retrieval, and data retention period.

Summary

With Streaming tables, data in Kinesis Data Streams could be delivered to Iceberg tables in S3 Tables without going through a custom-built consumer. Because the data arrives as Parquet with Iceberg metadata, Athena reads only the necessary columns and returns the total record count from metadata alone.

Configurations that write to Iceberg tables via a custom consumer or another service can be replaced with Streaming tables if the destination table can be recreated. The unit price is lower compared to configurations that route through Amazon Data Firehose, and only a single channel is needed. If you are analyzing with Athena, which can take advantage of Parquet and Iceberg metadata, give it a try.