Amazon Redshift Now Supports Apache Iceberg v3 Tables, So I Tried It Out on rg.large
This page has been translated by machine translation. View original
Introduction
On August 31, 2026, Amazon Redshift added support for reading and writing Apache Iceberg v3 tables on data lakes.
Of the elements added in v3, Redshift now supports three: default column values, row lineage, and deletion vectors. The execution platforms that support reading and writing v3 tables are the RG instance type and Redshift Serverless.
There are also features that are not supported for v3 tables.
- You can't read or write complex types (struct, list, map, variant) in Iceberg v3 tables.
- You can't use Iceberg v3 tables that contain equality deletes.
- You can't create materialized views on Iceberg v3 tables.
With v3 tables, you cannot read or write columns of complex types, and you cannot create materialized views either.
You can query Apache Iceberg tables cataloged in the AWS Glue Data Catalog with Amazon Redshift. RG instance types and Redshift Serverless use their own compute to process data lake queries, while RA3 instance types use Redshift Spectrum.
When you access Iceberg tables from an RG cluster or a Redshift Serverless workgroup, data lake queries run on the cluster's or workgroup's own compute resources, so there is no separate charge for data lake queries. When you access Iceberg tables from a DC2 or RA3 cluster, you are charged Redshift Spectrum pricing.
RG and Serverless run data lake queries on their own compute.
In this article, I create an Iceberg v3 table via the Glue Data Catalog from a cluster of the RG instance type rg.large. After completing the initial setup of the cluster, S3, Glue, and IAM, I performed all table operations in SQL.
What I Tested
I checked how the elements added in v3 appear from SQL, by comparing against v2.
Test Environment
Because the instance type affects the execution platform for Iceberg queries, here is the configuration of the cluster I used.
| Item | Value |
|---|---|
| NodeType | rg.large |
| NumberOfNodes | 2 |
| ClusterVersion | 1.0 |
| AvailabilityZone | ap-northeast-1c |
| ClusterStatus | available |
The database name is dev. The ApplyStatus of the IAM role applied to the cluster was in-sync.
Before working with Iceberg tables, I created an external schema that points to a database in the Glue Data Catalog.
CREATE EXTERNAL SCHEMA iceberg FROM DATA CATALOG DATABASE 'devio_iceberg_v3' IAM_ROLE 'arn:aws:iam::123456789012:role/devio-iceberg-v3-redshift';
The role ARN has been replaced with a masked value.
Default Column Values
I created a v3 table with format-version set to 3. I INSERTed three rows, omitting the status column for which DEFAULT is defined.
CREATE TABLE iceberg.orders_v3 (order_id int, status varchar DEFAULT 'pending', amount int)
USING ICEBERG
LOCATION 's3://amzn-s3-demo-bucket/orders_v3/'
TABLE PROPERTIES ('format-version'='3');
INSERT INTO iceberg.orders_v3 (order_id, amount) VALUES (1,100),(2,200),(3,300);
The bucket name has been masked from the actual value.
The SELECT results are as follows.
| order_id | status | amount |
|---|---|---|
| 1 | pending | 100 |
| 2 | pending | 200 |
| 3 | pending | 300 |
The status column, which was not specified in the INSERT, received the defined default value pending.
Next, I added a region column with DEFAULT to the table that already has rows.
ALTER TABLE iceberg.orders_v3 ADD COLUMN region varchar DEFAULT 'ap-northeast-1';
SELECT order_id,status,amount,region FROM iceberg.orders_v3 ORDER BY order_id;
| order_id | status | amount | region |
|---|---|---|---|
| 1 | pending | 100 | ap-northeast-1 |
| 2 | pending | 200 | ap-northeast-1 |
| 3 | pending | 300 | ap-northeast-1 |
The three rows that existed before the column was added returned the region value without any UPDATE.
Differences from v2
I created a v2 table without using DEFAULT, and ran only the same column addition operation as for v3. Since format-version is not specified in the table properties, it is created as v2.
CREATE TABLE iceberg.orders_v2 (order_id int, status varchar, amount int)
USING ICEBERG
LOCATION 's3://amzn-s3-demo-bucket/orders_v2/';
INSERT INTO iceberg.orders_v2 VALUES (1,'pending',100),(2,'pending',200),(3,'pending',300);
When I added the same region column with DEFAULT to this table as in v3, the statement failed.
ALTER TABLE iceberg.orders_v2 ADD COLUMN region varchar DEFAULT 'ap-northeast-1';
ERROR: Columns constraints and attributes are not supported for an "iceberg" table.
Hint: Default values are only supported with Iceberg version "3".
When I removed DEFAULT, the column addition succeeded.
ALTER TABLE iceberg.orders_v2 ADD COLUMN region varchar;
SELECT order_id,status,amount,region FROM iceberg.orders_v2 ORDER BY order_id;
| order_id | status | amount | region |
|---|---|---|---|
| 1 | pending | 100 | NULL |
| 2 | pending | 200 | NULL |
| 3 | pending | 300 | NULL |
With v2, after adding a column you need an UPDATE to fill in the existing rows.
Row Lineage
In v3 tables, you can reference the per-row identifier _row_id and the last updated sequence number _last_updated_sequence_number as pseudo-columns.
After checking the values before the update, I UPDATEd only the row where order_id is 2.
SELECT _row_id,_last_updated_sequence_number,order_id,status FROM iceberg.orders_v3 ORDER BY order_id;
UPDATE iceberg.orders_v3 SET status='shipped' WHERE order_id=2;
| order_id | _row_id before update | _last_updated_sequence_number before update | _row_id after update | _last_updated_sequence_number after update | status after update |
|---|---|---|---|---|---|
| 1 | 2 | 1 | 2 | 1 | pending |
| 2 | 1 | 1 | 1 | 3 | shipped |
| 3 | 0 | 1 | 0 | 1 | pending |
Only for the row where order_id is 2, the sequence number changed from 1 to 3, while _row_id stayed at 1. This lets you track the same row across UPDATEs while identifying only the rows that were updated.
Using the sequence number 3 measured after the UPDATE as the condition, I ran an incremental extraction.
SELECT _row_id,_last_updated_sequence_number,order_id,status FROM iceberg.orders_v3 WHERE _last_updated_sequence_number >= 3 ORDER BY order_id;
| _row_id | _last_updated_sequence_number | order_id | status |
|---|---|---|---|
| 1 | 3 | 2 | shipped |
Only the single updated row was returned. You can extract the updated rows without giving the table an update timestamp column or a flag column.
Cleanup
The rg.large cluster keeps incurring charges until it is deleted. I deleted the tables and the external schema with SQL.
DROP TABLE IF EXISTS iceberg.orders_v3;
DROP TABLE IF EXISTS iceberg.orders_v2;
DROP SCHEMA IF EXISTS iceberg;
At this point, 39 objects remained in S3. Dropping an Iceberg table is a metadata operation on the catalog side, so the data files remain in S3. I deleted the rest with the AWS CLI.
aws s3 rm s3://<バケット名> --recursive
aws s3api delete-bucket --bucket <バケット名>
aws glue delete-database --name devio_iceberg_v3
aws redshift delete-cluster --cluster-identifier <クラスター識別子> --skip-final-cluster-snapshot
aws redshift wait cluster-deleted --cluster-identifier <クラスター識別子>
aws iam delete-role-policy --role-name <ロール名> --policy-name <ポリシー名>
aws iam delete-role --role-name <ロール名>
Cluster deletion is asynchronous, so I waited for it to complete before deleting the IAM role.
Summary
From an rg.large cluster, I created an Iceberg v3 table and, using only SQL, confirmed the default values returned for existing rows when a column is added, as well as the extraction of updated rows.
ETL that keeps its own update timestamp or flag columns to compute differences can move to the row lineage pseudo-columns, and operations that bulk UPDATE existing data every time a column is added can move to adding columns with DEFAULT. You no longer need a self-made column to detect updates, and you no longer need to rewrite existing rows just to add a column.
Because an RG cluster runs data lake queries on its own compute, no separate Spectrum charge is incurred for queries against Iceberg tables. When using Iceberg v3, please check the restrictions such as complex types and materialized views, and then try it out by specifying format-version in the table properties.
