Amazon Redshift added support for Apache Iceberg v3 tables, so I tried it out on rg.large

Amazon Redshift added support for Apache Iceberg v3 tables, so I tried it out on rg.large

Amazon Redshift has added support for reading and writing Apache Iceberg v3 tables. From a cluster with the rg.large instance type, I tried creating v3 tables via the Glue Data Catalog and experimenting with default column values and row lineage pseudo-columns using only SQL.
2026.09.03

This page has been translated by machine translation. View original

Introduction

On August 31, 2026, Amazon Redshift added support for reading and writing Apache Iceberg v3 tables on the data lake.

https://aws.amazon.com/about-aws/whats-new/2026/08/amazon-redshift-supports-apache-iceberg-v3

Among the additions in v3, Redshift now supports three features: default column values, row lineage, and deletion vectors. The execution infrastructure that supports reading and writing v3 tables is either the RG instance type or Redshift Serverless.

Some features are not supported for v3 tables.

  • You can't read or write complex types (struct, list, map, variant) in Iceberg v3 tables.
  • You can't use Iceberg v3 tables that contain equality deletes.
  • You can't create materialized views on Iceberg v3 tables.

https://docs.aws.amazon.com/redshift/latest/dg/iceberg-v3-features.html

With v3 tables, you cannot read or write columns of complex types, and materialized views cannot be created either.

You can query Apache Iceberg tables cataloged in the AWS Glue Data Catalog with Amazon Redshift. RG instance types and Redshift Serverless use their own compute to process data lake queries, while RA3 instance types use Redshift Spectrum.

When you access Iceberg tables from an RG cluster or a Redshift Serverless workgroup, data lake queries run on the cluster's or workgroup's own compute resources, so there is no separate charge for data lake queries. When you access Iceberg tables from a DC2 or RA3 cluster, you are charged Redshift Spectrum pricing.

https://docs.aws.amazon.com/redshift/latest/dg/querying-iceberg.html

RG and Serverless execute data lake queries using their own compute.

In this article, we create an Iceberg v3 table via the Glue Data Catalog from an rg.large cluster. After completing the initial setup for the cluster, S3, Glue, and IAM, all table operations were performed using SQL.

What Was Tested

We examined how the v3 additions appear from SQL, by comparing them with v2.

Test Environment

Since the instance type affects the execution infrastructure for Iceberg queries, the configuration of the cluster used is shown below.

Item Value
NodeType rg.large
NumberOfNodes 2
ClusterVersion 1.0
AvailabilityZone ap-northeast-1c
ClusterStatus available

The database name is dev. The ApplyStatus of the IAM role applied to the cluster was in-sync.

Before operating on Iceberg tables, we created an external schema pointing to the Glue Data Catalog database.

CREATE EXTERNAL SCHEMA iceberg FROM DATA CATALOG DATABASE 'devio_iceberg_v3' IAM_ROLE 'arn:aws:iam::123456789012:role/devio-iceberg-v3-redshift';

The role ARN has been replaced with a masked value.

Default Column Values

We created a v3 table with format-version set to 3. We inserted 3 rows while omitting the status column, which has a DEFAULT defined.

CREATE TABLE iceberg.orders_v3 (order_id int, status varchar DEFAULT 'pending', amount int)
USING ICEBERG
LOCATION 's3://amzn-s3-demo-bucket/orders_v3/'
TABLE PROPERTIES ('format-version'='3');

INSERT INTO iceberg.orders_v3 (order_id, amount) VALUES (1,100),(2,200),(3,300);

The bucket name has been masked with an actual value.

The result of the SELECT is as follows.

order_id status amount
1 pending 100
2 pending 200
3 pending 300

The status column, which was not specified in the INSERT, was populated with the defined default value pending.

Next, we added a region column with DEFAULT to a table that already had existing rows.

ALTER TABLE iceberg.orders_v3 ADD COLUMN region varchar DEFAULT 'ap-northeast-1';
SELECT order_id,status,amount,region FROM iceberg.orders_v3 ORDER BY order_id;
order_id status amount region
1 pending 100 ap-northeast-1
2 pending 200 ap-northeast-1
3 pending 300 ap-northeast-1

The 3 rows that existed before the column was added returned a value for region without any UPDATE.

Differences from v2

We created a v2 table without using DEFAULT, and performed only the column addition using the same operation as with v3. Since no format-version is specified in the table properties, it is created as v2.

CREATE TABLE iceberg.orders_v2 (order_id int, status varchar, amount int)
USING ICEBERG
LOCATION 's3://amzn-s3-demo-bucket/orders_v2/';

INSERT INTO iceberg.orders_v2 VALUES (1,'pending',100),(2,'pending',200),(3,'pending',300);

When we added the same region column with DEFAULT to this table as in v3, the statement failed.

ALTER TABLE iceberg.orders_v2 ADD COLUMN region varchar DEFAULT 'ap-northeast-1';
ERROR: Columns constraints and attributes are not supported for an "iceberg" table.
  Hint: Default values are only supported with Iceberg version "3".

Removing the DEFAULT allowed the column addition to succeed.

ALTER TABLE iceberg.orders_v2 ADD COLUMN region varchar;
SELECT order_id,status,amount,region FROM iceberg.orders_v2 ORDER BY order_id;
order_id status amount region
1 pending 100 NULL
2 pending 200 NULL
3 pending 300 NULL

With v2, an UPDATE to fill in existing rows is required after adding a column.

Row Lineage

In v3 tables, you can reference per-row identifiers _row_id and the last-updated sequence number _last_updated_sequence_number as pseudo-columns.

We confirmed the values before the update, then ran an UPDATE on only the row where order_id is 2.

SELECT _row_id,_last_updated_sequence_number,order_id,status FROM iceberg.orders_v3 ORDER BY order_id;
UPDATE iceberg.orders_v3 SET status='shipped' WHERE order_id=2;
order_id Before update _row_id Before update _last_updated_sequence_number After update _row_id After update _last_updated_sequence_number After update status
1 2 1 2 1 pending
2 1 1 1 3 shipped
3 0 1 0 1 pending

Only the row where order_id is 2 had its sequence number change from 1 to 3, while _row_id remained 1. This makes it possible to track the same row across updates while distinguishing only the updated rows.

We ran an incremental extraction using the sequence number 3 measured after the UPDATE as the condition.

SELECT _row_id,_last_updated_sequence_number,order_id,status FROM iceberg.orders_v3 WHERE _last_updated_sequence_number >= 3 ORDER BY order_id;
_row_id _last_updated_sequence_number order_id status
1 3 2 shipped

Only the one updated row was returned. You can extract updated records without needing to add an updated timestamp column or flag column to the table.

Teardown

The rg.large cluster continues to incur charges until it is deleted. We dropped the tables and external schema using SQL.

DROP TABLE IF EXISTS iceberg.orders_v3;
DROP TABLE IF EXISTS iceberg.orders_v2;
DROP SCHEMA IF EXISTS iceberg;

At this point, 39 objects remained in S3. Dropping an Iceberg table is a metadata operation on the catalog side, so the data files remain in S3. The remaining resources were deleted using the AWS CLI.

aws s3 rm s3://<bucket name> --recursive
aws s3api delete-bucket --bucket <bucket name>
aws glue delete-database --name devio_iceberg_v3
aws redshift delete-cluster --cluster-identifier <cluster identifier> --skip-final-cluster-snapshot
aws redshift wait cluster-deleted --cluster-identifier <cluster identifier>
aws iam delete-role-policy --role-name <role name> --policy-name <policy name>
aws iam delete-role --role-name <role name>

Since cluster deletion is asynchronous, we waited for it to complete before deleting the IAM role.

Summary

From an rg.large cluster, we created an Iceberg v3 table and were able to execute everything in SQL alone — from returning default values to existing rows when adding a column, to extracting updated rows.

ETL pipelines that maintain their own updated timestamp or flag columns for incremental extraction can be replaced with row lineage pseudo-columns, and workflows that run bulk UPDATEs on existing data every time a column is added can be replaced with DEFAULT-based column additions. There is no need for custom columns to detect updates, and no need to rewrite existing rows when adding a column.

Since RG clusters execute data lake queries using their own compute, there are no separate Spectrum charges for queries against Iceberg tables. When using Iceberg v3, please review the constraints such as complex types and materialized views, then try it out by specifying format-version in the table properties.

Share this article

AWSのお困り事はクラスメソッドへ