[Update] System-Managed Iceberg Materialized Views Are Now Available in AWS Glue, So I Compared Writes with Regular Materialized Views

With AWS Glue's system-managed Apache Iceberg materialized views now available, I tested their write behavior and compared it side by side with regular materialized views.

石川覚

2026.10.13

This page has been translated by machine translation. View original

I'm Ishikawa from the Cloud Business Division. AWS Glue Apache Iceberg materialized views now offer a "system-managed" mode that limits writes to the data and definition to AWS Glue only. I created a regular materialized view and a system-managed one with the same definition side by side, tried writing to them from Spark, Amazon Athena, the AWS Glue API, and the Amazon S3 Tables API, and compared the results.

An AWS Glue materialized view (MV) precomputes the result of a query defined in SQL and stores it as an Apache Iceberg table. The result is an Iceberg table registered in the AWS Glue Data Catalog, so Iceberg-compatible engines such as Amazon Athena and Amazon EMR can read it.

Because the result is an open Iceberg table, any user with write permission can change the contents of an MV, either through the catalog or by writing directly to the files on Amazon S3. A system-managed MV limits the principals that can write the MV's data and definition to AWS Glue only. No matter who holds write permission on the table or its underlying S3 files, the computed result stays exactly as AWS Glue computed it.

System-managed MVs are available in all Regions where Apache Iceberg MVs are supported. The prerequisite AWS Glue 6.0 became generally available in August 2026.

What is a system-managed materialized view?

The AWS Glue Developer Guide describes a system-managed MV as "an MV that only AWS Glue can write to." AWS Glue writes both the data and the definition, and other engines and users cannot change them. It is stored in an Amazon S3 Tables bucket in your own account, in standard Apache Iceberg table format.

The requirements are as follows.

AWS Glue 6.0 or later
Set the Spark session configuration spark.sql.mv.managed.enabled to true
Create it with the MANAGED keyword, as in CREATE MANAGED MATERIALIZED VIEW
Storage is Amazon S3 Tables only (S3 general purpose buckets are not supported)

On creation, AWS Glue creates an empty table in S3 Tables and then runs the definition query to populate it. The CREATE MANAGED MATERIALIZED VIEW statement does not return until AWS Glue finishes creating it, so it can take longer than for a regular MV. You can check creation progress by specifying IncludeStatusDetails=true in the GetTable API.

Write protection covers the MV's data and its metadata such as schema and storage location. The documentation says changes from the following paths are rejected.

Other query engines
Direct calls to the AWS Glue UpdateTable API
Direct writes to the underlying Apache Iceberg table

The following operations can still be performed.

Refreshing the MV (REFRESH). AWS Glue runs the refresh and writes the result
Changing the refresh schedule and table properties
Deleting the MV (if you have delete permission)

Schema evolution is not supported. To change the SQL definition, drop the MV and recreate it.

The differences from a regular MV are as follows.

Item	Regular MV	System-managed MV
Required AWS Glue version	5.1 or later	6.0 or later
Storage	S3 Tables or S3 general purpose bucket	S3 Tables only
Create syntax	`CREATE MATERIALIZED VIEW`	`CREATE MANAGED MATERIALIZED VIEW`
Principals that can write data and definition	Users and engines with write permission	AWS Glue only
Schema evolution	-	Not supported (drop and recreate)

Trying it out

This was a complicated test, so here is the conclusion first: the results were largely as expected.

I created a regular MV and a system-managed MV with the same definition in the same S3 Tables namespace, ran the same write operations against both, and compared the results.

Conclusion first: summary of results

Operation	Role used	Regular MV	System-managed MV
Spark `INSERT INTO`	Job role	Succeeded	Failed (S3 403)
Spark DataFrame `append`	Job role	Succeeded	Failed (S3 403)
Spark `DELETE`	Job role	Succeeded	Failed (S3 403)
Spark `ALTER TABLE ADD COLUMNS`	Job role	Not run	Rejected
Athena `SELECT`	Verifier	Succeeded	Succeeded
Athena `INSERT INTO`	Verifier	Succeeded	Failed (S3 403)
`UpdateTable` (change `IsManaged` to `false`)	Verifier	-	Rejected
`UpdateTable` (without `ViewDefinition`)	Verifier	Returned success, but the definition remained	Rejected
`UpdateTable` (rewrite definition SQL)	Verifier	Rejected	Rejected
S3 Tables `UpdateTableMetadataLocation`	Verifier	Succeeded (then unreadable due to integrity check)	Rejected (explicit Deny)
`REFRESH ... FULL`	Job role	Failed (integrity check)	Succeeded (456.5 s)
`SET TBLPROPERTIES`	Job role	Not run	Succeeded
`ADD SCHEDULE` / `DROP SCHEDULE`	Job role	Not run	Succeeded (schedule remained after `DROP`)
`DROP MATERIALIZED VIEW`	Job role	Succeeded	Succeeded

Prerequisites

Region: Asia Pacific (Tokyo) ap-northeast-1
AWS Glue 6.0 (Spark 4.1.1-amzn-0), Spark job with G.1X × 2 workers
Amazon Athena (engine version 3, primary workgroup)
Amazon S3 Tables and AWS Glue Data Catalog integration (s3tablescatalog) already set up

The principals I used to attempt writes were these two.

The Glue job role (blog-mv-e603a380). It is the role that created the MVs (the definer role), and it is allowed s3tables:* on this S3 Tables bucket and its tables
The verifier's role (cm-user). I used it for the AWS CLI and Athena, and also to create and delete the test resources. It is set as a Lake Formation data lake administrator

Create the test resources

Create an S3 general purpose bucket and Glue database for the source table, plus an S3 Tables bucket and namespace to store the MVs.

% export AWS_REGION=ap-northeast-1
% export ACCOUNT_ID=123456789012
% export BUCKET=blog-mv-e603a380
% export TABLE_BUCKET=blog-mv-e603a380
% export SRC_DB=blog_mv_src_e603a380
% export S3T_CATALOG=${ACCOUNT_ID}:s3tablescatalog/${TABLE_BUCKET}
% export TB_ARN=arn:aws:s3tables:${AWS_REGION}:${ACCOUNT_ID}:bucket/${TABLE_BUCKET}

% aws s3api create-bucket --bucket $BUCKET \
  --create-bucket-configuration LocationConstraint=$AWS_REGION

aws s3tables create-table-bucket --name $TABLE_BUCKET

aws s3tables create-namespace --table-bucket-arn $TB_ARN --namespace analytics

aws glue create-database --database-input \
  "{\"Name\":\"${SRC_DB}\",\"LocationUri\":\"s3://${BUCKET}/warehouse/${SRC_DB}.db\"}"

{
    "arn": "arn:aws:s3tables:ap-northeast-1:123456789012:bucket/blog-mv-e603a380"
}
{
    "tableBucketARN": "arn:aws:s3tables:ap-northeast-1:123456789012:bucket/blog-mv-e603a380",
    "namespace": [
        "analytics"
    ]
}

Because the S3 Tables integration is set up, the table bucket I created appears in the Glue Data Catalog as a catalog named s3tablescatalog/<table bucket name>.

% aws glue get-databases --catalog-id $S3T_CATALOG \
  --query 'DatabaseList[].{Name:Name,CatalogId:CatalogId}' --output table
-----------------------------------------------------------------------
|                            GetDatabases                             |
+--------------------------------------------------------+------------+
|                        CatalogId                       |   Name     |
+--------------------------------------------------------+------------+
|  123456789012:s3tablescatalog/blog-mv-e603a380  |  analytics |
+--------------------------------------------------------+------------+

Create the IAM role for the Glue job

Create the Glue job role. So that AWS Glue can assume this role for automatic MV refresh, the trust policy allows glue.amazonaws.com.

% aws iam create-role --role-name blog-mv-e603a380 \
  --assume-role-policy-document '{
    "Version": "2012-10-17",
    "Statement": [{"Effect": "Allow", "Principal": {"Service": "glue.amazonaws.com"}, "Action": "sts:AssumeRole"}]
  }'

For permissions, I took the contents of "IAM permissions for AWS Glue jobs" in the documentation and added iam:PassRole on the role itself and s3tables:* on this S3 Tables bucket.

{
  "Version": "2012-10-17",
  "Statement": [
    {"Sid": "GlueCatalog", "Effect": "Allow", "Action": [
      "glue:GetCatalog", "glue:GetCatalogs", "glue:GetTable", "glue:GetTables",
      "glue:CreateTable", "glue:UpdateTable", "glue:DeleteTable",
      "glue:GetDatabase", "glue:GetDatabases", "cloudwatch:PutMetricData"], "Resource": "*"},
    {"Sid": "S3Objects", "Effect": "Allow", "Action": ["s3:GetObject", "s3:PutObject", "s3:DeleteObject"],
      "Resource": ["arn:aws:s3:::blog-mv-e603a380/*"]},
    {"Sid": "S3List", "Effect": "Allow", "Action": ["s3:ListBucket"], "Resource": ["arn:aws:s3:::blog-mv-e603a380"]},
    {"Sid": "Logs", "Effect": "Allow", "Action": ["logs:CreateLogGroup", "logs:CreateLogStream", "logs:PutLogEvents"],
      "Resource": ["arn:aws:logs:*:*:*:/aws-glue/*"]},
    {"Sid": "PassSelf", "Effect": "Allow", "Action": ["iam:PassRole"],
      "Resource": ["arn:aws:iam::123456789012:role/blog-mv-e603a380"]},
    {"Sid": "S3TablesBucket", "Effect": "Allow", "Action": ["s3tables:*"],
      "Resource": ["arn:aws:s3tables:ap-northeast-1:123456789012:bucket/blog-mv-e603a380",
                   "arn:aws:s3tables:ap-northeast-1:123456789012:bucket/blog-mv-e603a380/table/*"]},
    {"Sid": "S3TablesList", "Effect": "Allow", "Action": ["s3tables:ListTableBuckets"], "Resource": "*"},
    {"Sid": "LF", "Effect": "Allow", "Action": ["lakeformation:GetDataAccess"], "Resource": "*"}
  ]
}

% aws iam put-role-policy --role-name blog-mv-e603a380 \
  --policy-name mv-test --policy-document file://policy.json

As described later, REFRESH failed with this policy, so I added one more permission.

Create the Glue 6.0 job

Create a Glue 6.0 Spark job. For --conf, I used the "For S3 Tables buckets" settings from the documentation plus spark.sql.mv.managed.enabled=true. The source table is handled through glue_catalog (the default catalog of the Glue Data Catalog), and the MVs through s3t_catalog (s3tablescatalog/<table bucket name>).

{
  "Name": "blog-mv-e603a380",
  "Role": "arn:aws:iam::123456789012:role/blog-mv-e603a380", 
  "Command": {"Name": "glueetl", "ScriptLocation": "s3://blog-mv-e603a380/scripts/mv_test.py", "PythonVersion": "3"},
  "DefaultArguments": {
    "--enable-glue-datacatalog": "true",
    "--job-language": "python",
    "--src_db": "blog_mv_src_e603a380",
    "--ns": "analytics",
    "--phase": "phase1",
    "--conf": "spark.sql.extensions=org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions --conf spark.sql.catalog.glue_catalog=org.apache.iceberg.spark.SparkCatalog --conf spark.sql.catalog.glue_catalog.type=glue --conf spark.sql.catalog.glue_catalog.warehouse=s3://blog-mv-e603a380/warehouse --conf spark.sql.catalog.glue_catalog.glue.region=ap-northeast-1 --conf spark.sql.catalog.glue_catalog.glue.id=123456789012 --conf spark.sql.catalog.glue_catalog.glue.account-id=123456789012 --conf spark.sql.catalog.s3t_catalog=org.apache.iceberg.spark.SparkCatalog --conf spark.sql.catalog.s3t_catalog.type=glue --conf spark.sql.catalog.s3t_catalog.glue.id=123456789012:s3tablescatalog/blog-mv-e603a380 --conf spark.sql.catalog.s3t_catalog.glue.account-id=123456789012 --conf spark.sql.catalog.s3t_catalog.warehouse=s3://blog-mv-e603a380/mv-warehouse --conf spark.sql.catalog.s3t_catalog.glue.region=ap-northeast-1 --conf spark.sql.defaultCatalog=s3t_catalog --conf spark.sql.optimizer.answerQueriesWithMVs.enabled=true --conf spark.sql.materializedViews.metadataCache.enabled=true --conf spark.sql.mv.managed.enabled=true"
  },
  "GlueVersion": "6.0",
  "WorkerType": "G.1X",
  "NumberOfWorkers": 2,
  "Timeout": 30
}

% aws glue create-job --cli-input-json file://job.json

The job script runs SQL in order according to the --phase argument, and writes the success or failure, elapsed time, and result rows of each statement to standard output as one line of JSON. Even if one statement fails, the following statements continue to run. I retrieved the results from the CloudWatch Logs log group /aws-glue/jobs/output.

def run(step, sql, show=True):
    start = time.time()
    try:
        df = spark.sql(sql)
        rows = [r.asDict() for r in df.collect()] if show else None
        log({"step": step, "sql": sql, "status": "SUCCESS",
             "elapsed_sec": round(time.time() - start, 1), "rows": rows})
        return True
    except Exception as e:
        log({"step": step, "sql": sql, "status": "FAILED",
             "elapsed_sec": round(time.time() - start, 1), "error": err_info(e)})
        return False

Grant Lake Formation permissions

In the account used for this test, Lake Formation's CreateDatabaseDefaultPermissions and CreateTableDefaultPermissions are empty (new databases and tables do not automatically get IAMAllowedPrincipals permissions). IAM policies alone therefore did not allow catalog operations. I granted Lake Formation permissions limited to the test resources. I also gave the verifier's role S3 Tables-side permissions so that it can write to the MVs from Athena.

JOB_ROLE=arn:aws:iam::${ACCOUNT_ID}:role/blog-mv-e603a380
ME=arn:aws:iam::${ACCOUNT_ID}:role/cm-user

# Job role: source Glue database and tables
aws lakeformation grant-permissions --principal DataLakePrincipalIdentifier=$JOB_ROLE \
  --resource "{\"Database\":{\"Name\":\"${SRC_DB}\"}}" --permissions ALL
aws lakeformation grant-permissions --principal DataLakePrincipalIdentifier=$JOB_ROLE \
  --resource "{\"Table\":{\"DatabaseName\":\"${SRC_DB}\",\"TableWildcard\":{}}}" --permissions ALL

# Job role: S3 Tables catalog
aws lakeformation grant-permissions --principal DataLakePrincipalIdentifier=$JOB_ROLE \
  --resource "{\"Catalog\":{\"Id\":\"${S3T_CATALOG}\"}}" --permissions DESCRIBE

# Job role and verifier's role: S3 Tables namespace and tables
for P in $JOB_ROLE $ME; do
  aws lakeformation grant-permissions --principal DataLakePrincipalIdentifier=$P \
    --resource "{\"Database\":{\"CatalogId\":\"${S3T_CATALOG}\",\"Name\":\"analytics\"}}" --permissions ALL
  aws lakeformation grant-permissions --principal DataLakePrincipalIdentifier=$P \
    --resource "{\"Table\":{\"CatalogId\":\"${S3T_CATALOG}\",\"DatabaseName\":\"analytics\",\"TableWildcard\":{}}}" --permissions ALL
done

Create the regular MV and the system-managed MV

In the first job run (--phase phase1), I create the source table, insert 5 rows, and create a regular MV and a system-managed MV with the same definition. The only difference is the MANAGED keyword.

CREATE TABLE IF NOT EXISTS glue_catalog.blog_mv_src_e603a380.orders (
    id INT,
    customer_name STRING,
    amount DECIMAL(10,2),
    order_date DATE
) USING iceberg
TBLPROPERTIES ('format-version'='2');

INSERT INTO glue_catalog.blog_mv_src_e603a380.orders VALUES
    (1, 'Alice', 150.00, DATE '2026-10-01'),
    (2, 'Bob',   200.50, DATE '2026-10-01'),
    (3, 'Alice', 75.25,  DATE '2026-10-02'),
    (4, 'Carol', 320.00, DATE '2026-10-02'),
    (5, 'Bob',   99.99,  DATE '2026-10-03');

-- Regular MV
CREATE MATERIALIZED VIEW s3t_catalog.analytics.customer_summary_regular
AS
SELECT
    customer_name,
    COUNT(*) AS order_count,
    SUM(amount) AS total_amount
FROM glue_catalog.blog_mv_src_e603a380.orders
GROUP BY customer_name;

-- System-managed MV
CREATE MANAGED MATERIALIZED VIEW s3t_catalog.analytics.customer_summary_managed
AS
SELECT
    customer_name,
    COUNT(*) AS order_count,
    SUM(amount) AS total_amount
FROM glue_catalog.blog_mv_src_e603a380.orders
GROUP BY customer_name;

aws glue start-job-run --job-name blog-mv-e603a380 --arguments '{"--phase":"phase1"}'

The results for each statement are as follows.

Statement	Result	Elapsed time
CREATE TABLE (source)	SUCCESS	10.3 s
INSERT INTO (source, 5 rows)	SUCCESS	7.0 s
CREATE MATERIALIZED VIEW (regular)	SUCCESS	9.6 s
CREATE MANAGED MATERIALIZED VIEW	SUCCESS	211.9 s

Even with a 5-row source, creating the system-managed MV took 211.9 seconds. As the documentation says, the CREATE MANAGED MATERIALIZED VIEW statement did not return until AWS Glue finished creating it.

SHOW VIEWS showed both MVs in the FRESH state.

SHOW VIEWS IN s3t_catalog.analytics;

namespace	viewName	isTemporary	stalenessStatus
analytics	customer_summary_managed	false	FRESH
analytics	customer_summary_regular	false	FRESH

This session runs with spark.sql.mv.managed.enabled=true, but an MV created without MANAGED was created as a regular MV, as shown below.

Check the differences between the two MVs

Compare the two MVs by running aws glue get-table with --include-status-details.

% aws glue get-table --catalog-id $S3T_CATALOG --database-name analytics \
  --name customer_summary_managed --include-status-details \
  --query 'Table.{Name:Name,IsMaterializedView:IsMaterializedView,IsManaged:ViewDefinition.IsManaged,IsProtected:ViewDefinition.IsProtected,Definer:ViewDefinition.Definer,StatusAction:Status.Action,StatusState:Status.State,RequestTime:Status.RequestTime,UpdateTime:Status.UpdateTime}' \
  --output json
{
    "Name": "customer_summary_managed",
    "IsMaterializedView": true,
    "IsManaged": true,
    "IsProtected": true,
    "Definer": "arn:aws:iam::123456789012:role/blog-mv-e603a380",
    "StatusAction": "CREATE",
    "StatusState": "SUCCESS",
    "RequestTime": "2026-10-13T00:50:59+09:00",
    "UpdateTime": "2026-10-13T00:54:24+09:00"
}

Here is the result of the same command against the regular MV.

% aws glue get-table --catalog-id $S3T_CATALOG --database-name analytics \
  --name customer_summary_regular --include-status-details \
  --query 'Table.{Name:Name,IsMaterializedView:IsMaterializedView,IsManaged:ViewDefinition.IsManaged,IsProtected:ViewDefinition.IsProtected,Definer:ViewDefinition.Definer,StatusAction:Status.Action,StatusState:Status.State,RequestTime:Status.RequestTime,UpdateTime:Status.UpdateTime}' \
  --output json
{
    "Name": "customer_summary_regular",
    "IsMaterializedView": true,
    "IsManaged": false,
    "IsProtected": true,
    "Definer": "arn:aws:iam::123456789012:role/blog-mv-e603a380",
    "StatusAction": null,
    "StatusState": null,
    "RequestTime": null,
    "UpdateTime": null
}

You can tell whether an MV is system-managed from ViewDefinition.IsManaged. Only the system-managed MV returned a Status, which recorded the times from the create request (00:50:59) to completion (00:54:24).

In Spark's DESCRIBE EXTENDED, the columns differed. The system-managed MV has two extra columns not in the definition: __ivm_s_total_amount and __ivm_num_rec.

DESCRIBE EXTENDED s3t_catalog.analytics.customer_summary_managed;

col_name                      data_type                                       comment
customer_name                 string                                          NULL
order_count                   bigint                                          NULL
total_amount                  decimal(20,2)                                   NULL
__ivm_s_total_amount          bigint                                          NULL
__ivm_num_rec                 bigint                                          NULL

# Metadata Columns
_spec_id                      int
_partition                    struct<>
_file                         string
_pos                          bigint
_deleted                      boolean

# Detailed Table Information
Name                          s3t_catalog.analytics.customer_summary_managed
Type                          MANAGED
Location                      s3://7a560e21-ea48-467e-eufmujof614fp454ydr7rox8idknkapn1b--table-s3
Provider                      iceberg
Table_type                    ICEBERG
Table Properties              [createdBy=123456789012,current-snapshot-id=8320873279919889256,format=iceberg/parquet,format-version=2,ownerAccountId=123456789012,s3TableArn=arn:aws:s3tables:ap-northeast-1:123456789012:bucket/blog-mv-e603a380/table/7a560e21-ea48-467e-88ee-5d9475f8e45f,warehouse_location=s3://7a560e21-ea48-467e-eufmujof614fp454ydr7rox8idknkapn1b--table-s3,write.parquet.compression-codec=zstd]
Materialized View Properties  [IMV_mvNamespace=analytics,subObjectBaseTableStats=[{"SourceType":"ICEBERG"}],IMV_mvCatalogID=123456789012:s3tablescatalog/blog-mv-e603a380,isMaterializedView=true,IMV_sparkVersion=4.1.1-amzn-0,viewVersionToken=3efb8bcc81583ddfd543,subObjects=[{"Version":"405373681366950278","DatabaseName":"blog_mv_src_e603a380","Region":"ap-northeast-1","CatalogId":"123456789012","Name":"orders"}],viewVersionId=8320873279919889256,tableVersionToken=3efb8bcc81583ddfd543,IMV_ansiEnabled=true,IMV_catalogInfo=[{"catalogId":"123456789012","catalogName":"glue_catalog"}],stalenessStatus=FRESH,IMV_region=ap-northeast-1,lastRefreshType=FULL,isManaged=true,viewOriginalText=SELECT
    customer_name,
    COUNT(*) AS order_count,
    SUM(amount) AS total_amount
FROM glue_catalog.blog_mv_src_e603a380.orders
GROUP BY customer_name]
Statistics                    1741 bytes, 3 rows                              NULL

The regular MV has only three columns, customer_name, order_count, and total_amount, and its Materialized View Properties showed isManaged=false. Note that Type shows MANAGED for regular MVs too, so use isManaged to tell whether an MV is system-managed. The documentation says columns starting with __ivm are reserved for system use.

Write to the MVs from Spark

The job role is allowed s3tables:* on the S3 Tables bucket. With this role, I tried the same writes against both MVs.

-- Run against both MVs
INSERT INTO s3t_catalog.analytics.customer_summary_regular VALUES ('Mallory_sql', 999, 99999.99);
INSERT INTO s3t_catalog.analytics.customer_summary_managed VALUES ('Mallory_sql', 999, 99999.99, 1, 1);
-- Also append 1 row via the DataFrame API (writeTo(...).append())

-- Delete
DELETE FROM s3t_catalog.analytics.customer_summary_regular WHERE customer_name = 'Mallory_df';
DELETE FROM s3t_catalog.analytics.customer_summary_managed WHERE customer_name = 'Alice';

-- System-managed MV only: add a column
ALTER TABLE s3t_catalog.analytics.customer_summary_managed ADD COLUMNS (note STRING);

At first I ran the INSERT against the system-managed MV with 3 columns. It failed with INSERT_COLUMN_ARITY_MISMATCH.NOT_ENOUGH_DATA_COLUMNS because it was short by the two __ivm columns. This was not a rejection by write protection, so I reran it with 5 columns.

The results are as follows.

Operation	Regular MV	System-managed MV
`INSERT INTO`	SUCCESS	FAILED
DataFrame `writeTo(...).append()`	SUCCESS	FAILED
`DELETE FROM`	SUCCESS	FAILED
`ALTER TABLE ADD COLUMNS`	Not run	FAILED

INSERT, append, and DELETE against the system-managed MV all failed at the stage where Spark tasks write data files. The cause (Caused by) in the error log was S3 AccessDenied (403).

This is the full log the driver wrote to the Glue error log (/aws-glue/jobs/error) during the INSERT. The task failed four times, and the stack trace of the last attempt (TID 3) was recorded.

ALTER TABLE ADD COLUMNS was rejected at the Spark stage, because a materialized view cannot be modified.

UnsupportedOperationException: Cannot apply unsupported change to managed materialized view: org.apache.spark.sql.connector.catalog.TableChange$AddColumn@4181d81

After the writes, the regular MV contained rows that are not in the result of the definition query (the Mallory_df row was later removed by DELETE).

SELECT * FROM s3t_catalog.analytics.customer_summary_regular ORDER BY customer_name;

customer_name	order_count	total_amount
Alice	2	225.25
Bob	2	300.49
Carol	1	320.00
Mallory_df	1	88888.88
Mallory_sql	999	99999.99

The Iceberg snapshots of the regular MV also increased with every write. The system-managed MV's snapshots stayed at the single one from creation (committed_at is UTC).

SELECT committed_at, snapshot_id, operation
FROM s3t_catalog.analytics.customer_summary_regular.snapshots ORDER BY committed_at;

committed_at	snapshot_id	operation
2026-10-12 15:50:57.455	8870572473500367875	overwrite
2026-10-12 15:54:37.040	7556935525565095869	append
2026-10-12 15:54:41.483	7176343283781109168	append
2026-10-12 15:57:45.607	3453113367357939935	delete

SELECT committed_at, snapshot_id, operation
FROM s3t_catalog.analytics.customer_summary_managed.snapshots ORDER BY committed_at;

committed_at	snapshot_id	operation
2026-10-12 15:54:07.033	8320873279919889256	overwrite

Read and write the MVs from Athena

Next, I operate from Athena using the verifier's role. First I read both MVs.

% aws athena start-query-execution \
  --work-group primary \
  --query-execution-context Catalog=s3tablescatalog/${TABLE_BUCKET},Database=analytics \
  --result-configuration OutputLocation=s3://${BUCKET}/athena-results/ \
  --query-string "SELECT * FROM customer_summary_managed ORDER BY customer_name"

customer_name	order_count	total_amount	__ivm_s_total_amount	__ivm_num_rec
Alice	2	225.25	2	2
Bob	2	300.49	2	2
Carol	1	320.00	1	1

The system-managed MV could also be read from Athena. SELECT * also returns the two __ivm columns. The regular MV could be read too, and it included the Mallory_sql row written from Spark.

Next, I run INSERT against both MVs.

% aws athena start-query-execution \
  --work-group primary \
  --query-execution-context Catalog=s3tablescatalog/${TABLE_BUCKET},Database=analytics \
  --result-configuration OutputLocation=s3://${BUCKET}/athena-results/ \
  --query-string "INSERT INTO customer_summary_managed VALUES ('Mallory_athena', 1, 77777.77, 1, 1)"

% aws athena get-query-execution --query-execution-id b518c362-b82b-4574-834b-654fc84c4ed8 \
  --query 'QueryExecution.Status.{State:State,Reason:StateChangeReason,ErrorCategory:AthenaError.ErrorCategory,ErrorType:AthenaError.ErrorType}'

INSERT INTO customer_summary_regular VALUES ('Mallory_athena', 1, 77777.77) against the regular MV ended in SUCCEEDED. The INSERT against the system-managed MV failed with an S3 403.

{
    "State": "FAILED",
    "Reason": "PERMISSION_DENIED: Access Denied (Service: S3, Status Code: 403, Request ID: TMS75TKSBSYR1HS9, Extended Request ID: Knkn42T+WVzFwW1tRifvN7yxUzFE19U+7fA/5cgWQPARx/8aL75/ldxSzuKmQFStWfFewe7yios=). If a data manifest file was generated at 's3://blog-mv-e603a380/athena-results/b518c362-b82b-4574-834b-654fc84c4ed8-manifest.csv', you may need to manually clean the data from locations specified in the manifest. Athena will not delete data in your account.",
    "ErrorCategory": 2,
    "ErrorType": 1500
}

Change the definition with the AWS Glue UpdateTable API

As a change through the catalog, I called aws glue update-table directly with the verifier's role. I build a TableInput from the get-table result and set ViewDefinition.IsManaged to false to see whether I can remove the protection.

Running update-table against an MV in the S3 Tables catalog required the following three things. All of them were found through errors returned during my attempts.

Specify --version-id (otherwise S3 Tables returns an error that versionToken is null)
Specify --view-update-action when including ViewDefinition
Set TableType to EXTERNAL_TABLE (get-table returns customer, but passing it as is gives Materialized views must have table type EXTERNAL_TABLE.)

% aws glue get-table --catalog-id $S3T_CATALOG --database-name analytics \
  --name customer_summary_managed --output json > current_managed.json

% jq '.Table | {Name, Description: "tampered by UpdateTable", TableType: "EXTERNAL_TABLE", Parameters, StorageDescriptor,
    ViewDefinition: (.ViewDefinition | {IsProtected, Definer, ViewVersionId, ViewVersionToken,
      SubObjects, SubObjectVersionIds, LastRefreshType,
      Representations: [.Representations[] | {Dialect, DialectVersion, ViewOriginalText}],
      IsManaged: false})}' current_managed.json > table_input.json

% aws glue update-table --catalog-id $S3T_CATALOG --database-name analytics \
  --version-id $(jq -r '.Table.VersionId' current_managed.json) \
  --view-update-action REPLACE \
  --table-input file://table_input.json

An error occurred (InvalidInputException) when calling the UpdateTable operation: Cannot modify ViewDefinition.IsManaged on a system-managed materialized view. This field is immutable after creation.

IsManaged was rejected as a field that cannot be changed after creation. Passing a TableInput without ViewDefinition was also rejected for the system-managed MV.

An error occurred (InvalidInputException) when calling the UpdateTable operation: Cannot modify ViewDefinition on a system-managed materialized view. This field is immutable after creation.

An update-table without ViewDefinition returned success for the regular MV. However, the following get-table still showed ViewDefinition, so the MV definition had not been removed.

An update that rewrote the definition SQL (ViewOriginalText) to SUM(amount) * 2 was rejected with the same error for both the regular and system-managed MVs. Rewriting the SQL definition is not accepted for any MV, so there was no difference between the two.

An error occurred (InvalidInputException) when calling the UpdateTable operation: Update of ViewOriginalText for Materialized View is not available.

An update that changed only Description without changing ViewDefinition returned success for both MVs. However, Description in get-table remained null for both.

Swap the Iceberg metadata with the S3 Tables API

As a direct write to the underlying Iceberg table, I use the S3 Tables UpdateTableMetadataLocation API to swap the Iceberg metadata file that the table references. For the regular MV, I revert to the metadata file from before the Athena INSERT (00004-5734b581-e4f9-4ae7-832c-b8939b15f275.metadata.json).

% aws s3tables get-table-metadata-location --table-bucket-arn $TB_ARN \
  --namespace analytics --name customer_summary_regular
{
    "versionToken": "1a99953d59198d0ce3e4",
    "metadataLocation": "s3://f1b961f4-c1f9-4ffd-mec3po58st48mi4a5t3a4bjea9gfhapn1b--table-s3/metadata/00005-98a09a9f-9073-4150-8b85-d4501244a6bb.metadata.json",
    "warehouseLocation": "s3://f1b961f4-c1f9-4ffd-mec3po58st48mi4a5t3a4bjea9gfhapn1b--table-s3"
}

% aws s3tables update-table-metadata-location --table-bucket-arn $TB_ARN \
  --namespace analytics --name customer_summary_regular \
  --version-token 1a99953d59198d0ce3e4 \
  --metadata-location s3://f1b961f4-c1f9-4ffd-mec3po58st48mi4a5t3a4bjea9gfhapn1b--table-s3/metadata/00004-5734b581-e4f9-4ae7-832c-b8939b15f275.metadata.json
{
    "name": "customer_summary_regular",
    "tableARN": "arn:aws:s3tables:ap-northeast-1:123456789012:bucket/blog-mv-e603a380/table/f1b961f4-c1f9-4ffd-bd45-5c5ff2a62d62",
    "namespace": [
        "analytics"
    ],
    "versionToken": "cf43e6e34e76a2516aa1",
    "metadataLocation": "s3://f1b961f4-c1f9-4ffd-mec3po58st48mi4a5t3a4bjea9gfhapn1b--table-s3/metadata/00004-5734b581-e4f9-4ae7-832c-b8939b15f275.metadata.json"
}

The swap succeeded for the regular MV. The system-managed MV's metadata file is still the first one (00000-db037a92-7d84-405d-a549-b333c14fe896.metadata.json) and there is nothing to revert to, so I check whether even a call that specifies unchanged contents is rejected at the API level. Calling the same API on the system-managed MV with its current metadata file and versionToken as is was rejected.

% aws s3tables update-table-metadata-location --table-bucket-arn $TB_ARN \
  --namespace analytics --name customer_summary_managed \
  --version-token 3efb8bcc81583ddfd543 \
  --metadata-location s3://7a560e21-ea48-467e-eufmujof614fp454ydr7rox8idknkapn1b--table-s3/metadata/00000-db037a92-7d84-405d-a549-b333c14fe896.metadata.json

An error occurred (AccessDeniedException) when calling the UpdateTableMetadataLocation operation: User: arn:aws:sts::123456789012:assumed-role/cm-user/ishikawa is not authorized to perform: s3tables:UpdateTableMetadataLocation on resource: arn:aws:s3tables:ap-northeast-1:123456789012:bucket/blog-mv-e603a380/table/7a560e21-ea48-467e-88ee-5d9475f8e45f with an explicit deny in a resource-based policy

The error message indicates denial by an explicit Deny in a resource-based policy. However, get-table-policy and get-table-bucket-policy showed no policy on either of the two tables or on the table bucket.

% aws s3tables get-table-policy --table-bucket-arn $TB_ARN \
  --namespace analytics --name customer_summary_managed
An error occurred (NotFoundException) when calling the GetTablePolicy operation: The specified table policy does not exist.

After swapping the metadata, I read the regular MV from Athena.

% aws athena start-query-execution \
  --work-group primary \
  --query-execution-context Catalog=s3tablescatalog/${TABLE_BUCKET},Database=analytics \
  --result-configuration OutputLocation=s3://${BUCKET}/athena-results/ \
  --query-string "SELECT * FROM customer_summary_regular ORDER BY customer_name"

The query failed; the regular MV could no longer be read from Athena.

NOT_SUPPORTED: Materialized view integrity check failed for (catalog:123456789012:s3tablescatalog/blog-mv-e603a380$schema:analytics, customer_summary_regular): versionId (cf43e6e34e76a2516aa1) != viewVersionToken (1a99953d59198d0ce3e4)

The error says the S3 Tables-side version (cf43e6e34e76a2516aa1) does not match the version recorded in the Glue Data Catalog's MV definition (1a99953d59198d0ce3e4).

Next, I read the regular MV from Spark in a Glue job and ran the REFRESH suggested by the error message.

SELECT * FROM s3t_catalog.analytics.customer_summary_regular ORDER BY customer_name;

REFRESH MATERIALIZED VIEW s3t_catalog.analytics.customer_summary_regular FULL;

Both failed with a ValidationException from the same integrity check.

org.apache.iceberg.exceptions.ValidationException: Materialized view customer_summary_regular integrity check failed: versionId (cf43e6e34e76a2516aa1) != viewVersionToken (1a99953d59198d0ce3e4). Please recreate the materialized view or run a full refresh

However, a SELECT * run in the same job after the failed REFRESH succeeded and returned the following 4 rows.

customer_name	order_count	total_amount
Alice	2	225.25
Bob	2	300.49
Carol	1	320.00
Mallory_sql	999	99999.99

This matches the contents of the metadata file I swapped to (what I read before the Athena INSERT): there is no Mallory_athena row and the Mallory_sql row remains. It is not a result recomputed by the definition query. A read of the snapshots metadata table (customer_summary_regular.snapshots) immediately afterward failed again with the same ValidationException. I could not confirm why only this SELECT * succeeded.

Try the permitted operations

I tried the operations that the documentation allows for system-managed MVs.

REFRESH

After adding 2 rows to the source table, the system-managed MV's stalenessStatus became STALE.

INSERT INTO glue_catalog.blog_mv_src_e603a380.orders VALUES
    (6, 'Carol', 180.00, DATE '2026-10-04'),
    (7, 'Dave',  50.00,  DATE '2026-10-04');

REFRESH MATERIALIZED VIEW s3t_catalog.analytics.customer_summary_managed FULL;

The first REFRESH failed due to insufficient IAM permissions. Here too, the job script recorded only the first 1,500 characters of the error message, so this is the part before the stack trace.

An error occurred while calling o147.sql.
: org.apache.iceberg.shaded.software.amazon.awssdk.services.glue.model.AccessDeniedException: User: arn:aws:sts::123456789012:assumed-role/blog-mv-e603a380/GlueJobRunnerSession is not authorized to perform: glue:StartMaterializedViewRefreshTaskRun because no identity-based policy allows the glue:StartMaterializedViewRefreshTaskRun action (Service: Glue, Status Code: 400, Request ID: e072e336-6e4c-48c0-96dc-061df4f29da5) (SDK Attempt Count: 1)

glue:StartMaterializedViewRefreshTaskRun is a permission not included in the Developer Guide's IAM policy example. The Glue Materialized view API has four refresh-task operations (start, stop, get, list), so I added all four to the job role.

{
  "Sid": "MVRefreshTaskRun",
  "Effect": "Allow",
  "Action": [
    "glue:StartMaterializedViewRefreshTaskRun",
    "glue:StopMaterializedViewRefreshTaskRun",
    "glue:GetMaterializedViewRefreshTaskRun",
    "glue:ListMaterializedViewRefreshTaskRuns"
  ],
  "Resource": "*"
}

After adding the permissions and rerunning, REFRESH succeeded. It took 456.5 seconds.

SELECT * FROM s3t_catalog.analytics.customer_summary_managed ORDER BY customer_name;

customer_name	order_count	total_amount	__ivm_s_total_amount	__ivm_num_rec
Alice	2	225.25	2	2
Bob	2	300.49	2	2
Carol	2	500.00	2	2
Dave	1	50.00	1	1

The added Carol and Dave orders were reflected, and one snapshot (overwrite) was added. You can check the refresh task records with list-materialized-view-refresh-task-runs.

% aws glue list-materialized-view-refresh-task-runs --catalog-id $S3T_CATALOG \
  --database-name analytics --table-name customer_summary_managed
{
    "MaterializedViewRefreshTaskRuns": [
        {
            "CustomerId": "123456789012",
            "MaterializedViewRefreshTaskRunId": "b604f9ff-8373-421e-92cb-9f951246e0b6",
            "DatabaseName": "analytics",
            "TableName": "customer_summary_managed",
            "CatalogId": "123456789012:s3tablescatalog/blog-mv-e603a380",
            "Role": "arn:aws:iam::123456789012:role/blog-mv-e603a380",
            "Status": "SUCCEEDED",
            "CreationTime": "2026-10-13T01:06:35+09:00",
            "LastUpdated": "2026-10-13T01:10:19+09:00",
            "EndTime": "2026-10-13T01:10:18+09:00",
            "DPUSeconds": 240.3
        }
    ]
}

The refresh task was created at 01:06:35 and finished at 01:10:18, with 240.3 DPU-seconds recorded. The Spark REFRESH statement started at 01:06:29 and returned at 01:14:06. The updated 4 rows could also be read from Athena.

Change table properties

ALTER TABLE ... SET TBLPROPERTIES succeeded. The snapshots did not change.

ALTER TABLE s3t_catalog.analytics.customer_summary_managed SET TBLPROPERTIES ('blog.note'='updated-by-owner');
SHOW TBLPROPERTIES s3t_catalog.analytics.customer_summary_managed;

key	value
blog.note	updated-by-owner
createdBy	123456789012
current-snapshot-id	8320873279919889256
format	iceberg/parquet
format-version	2
ownerAccountId	123456789012
s3TableArn	arn:aws:s3tables:ap-northeast-1:123456789012:bucket/blog-mv-e603a380/table/7a560e21-ea48-467e-88ee-5d9475f8e45f
warehouse_location	s3://7a560e21-ea48-467e-eufmujof614fp454ydr7rox8idknkapn1b--table-s3
write.parquet.compression-codec	zstd

Change the refresh schedule

The documentation says that on AWS Glue 6.0 or later the minimum automatic refresh interval is 10 minutes. I specified EVERY 10 MINUTES, but got a syntax error. The singular EVERY 10 MINUTE gave the same syntax error.

[PARSE_SYNTAX_ERROR] Syntax error at or near 'MINUTES'. SQLSTATE: 42601 (line 1, pos 101)

== SQL ==
ALTER MATERIALIZED VIEW s3t_catalog.analytics.customer_summary_managed ADD SCHEDULE REFRESH EVERY 10 MINUTES
-----------------------------------------------------------------------------------------------------^^^

Specifying the same time unit as the documentation's example succeeded, and schedule=3600 was added to Materialized View Properties in DESCRIBE EXTENDED.

ALTER MATERIALIZED VIEW s3t_catalog.analytics.customer_summary_managed ADD SCHEDULE REFRESH EVERY 1 HOUR;
ALTER MATERIALIZED VIEW s3t_catalog.analytics.customer_summary_managed DROP SCHEDULE;

The DROP SCHEDULE I ran next also returned success. However, the following get-table still showed RefreshSeconds as 3600.

% aws glue get-table --catalog-id $S3T_CATALOG --database-name analytics \
  --name customer_summary_managed \
  --query 'Table.{IsManaged:ViewDefinition.IsManaged,RefreshSeconds:ViewDefinition.RefreshSeconds,LastRefreshType:ViewDefinition.LastRefreshType}'
{
    "IsManaged": true,
    "RefreshSeconds": 3600,
    "LastRefreshType": "FULL"
}

Furthermore, about one minute after I ran DROP SCHEDULE at 01:14:21, a refresh task started at 01:15:22. I had not started it manually. I stopped it with stop-materialized-view-refresh-task-run and confirmed it became STOPPED (0.0 DPU-seconds).

% aws glue stop-materialized-view-refresh-task-run --catalog-id $S3T_CATALOG \
  --database-name analytics --table-name customer_summary_managed
--------------------------------------------------------------------------------------------------------------------------
|                                           ListMaterializedViewRefreshTaskRuns                                          |
+---------------------------+--------+----------------------------+----------------------------------------+-------------+
|          Created          |  DPU   |            End             |                  Id                    |   Status    |
+---------------------------+--------+----------------------------+----------------------------------------+-------------+
|  2026-10-13T01:15:22+09:00|  0.0   |  2026-10-13T01:16:04+09:00 |  76566af4-49a0-4767-9590-072d6610a86d  |  STOPPED    |
|  2026-10-13T01:06:35+09:00|  240.3 |  2026-10-13T01:10:18+09:00 |  b604f9ff-8373-421e-92cb-9f951246e0b6  |  SUCCEEDED  |
+---------------------------+--------+----------------------------+----------------------------------------+-------------+

Discussion

Where writes were rejected

Writes to the system-managed MV were rejected at three stages.

Spark: ALTER TABLE ADD COLUMNS returned Cannot apply unsupported change to managed materialized view from Spark
AWS Glue API: trying to change ViewDefinition or IsManaged with UpdateTable returned This field is immutable after creation.
Storage: data writes from Spark and Athena failed with S3 403, and S3 Tables UpdateTableMetadataLocation was rejected by "an explicit Deny in a resource-based policy." This Deny was not in the table policy that can be checked with GetTablePolicy

The principals that ran the rejected operations were the job role, which is allowed s3tables:* on the table bucket, and the verifier's role. Against the regular MV, both roles succeeded at writing data (such as Spark and Athena INSERT) and swapping the metadata file. This matches the What's New statement that the computed results are maintained without modification no matter who has write access to the table or its underlying S3 files.

What happened with the regular MV

With the regular MV, INSERT and DELETE from Spark and Athena completed without errors, and rows that are not in the definition query result (such as Mallory_sql with order_count of 999) became readable as they were.

When I swapped the metadata file with the S3 Tables API, it no longer matched the version recorded in the Glue Data Catalog, and reads failed in both Athena and Spark. REFRESH ... FULL failed on the same check, so in this test I could not restore this regular MV to a readable state with REFRESH. After the failed REFRESH, a Spark SELECT * returned the contents of the swapped-in metadata file once, but the next read failed again.

Time and DPU consumption

The source had only 7 rows, but processing the system-managed MV took the following time.

CREATE MANAGED MATERIALIZED VIEW: 211.9 seconds (the regular MV CREATE took 9.6 seconds)
REFRESH ... FULL: 456.5 seconds (the refresh task itself ran from 01:06:35 to 01:10:18, 240.3 DPU-seconds)

Spark statements do not return until AWS Glue finishes. During that time, the Glue job's workers stay allocated. The six job runs together totaled 2,089 DPU-seconds, of which the one run including REFRESH was 1,021 DPU-seconds. When running creation or REFRESH from a Spark job, keep in mind that the waiting time is also consumed as job DPU. The AWS blog describes the automatic refresh price as 0.44 USD per DPU-hour (billed per second, 1-minute minimum).

Points to note

System-managed MVs require AWS Glue 6.0 and S3 Tables. MVs on S3 general purpose buckets are not supported
__ivm columns such as __ivm_s_total_amount are added and are also returned by Athena SELECT *. It is better to tell consumers to specify the columns they need when reading
REFRESH requires glue:StartMaterializedViewRefreshTaskRun, which is not in the documentation's IAM policy example
In accounts where default permissions for new resources are disabled in Lake Formation, Lake Formation grants are needed in addition to the IAM policy
In this test, minute-based schedule specifications (MINUTES / MINUTE) gave syntax errors, and only the hour-based specification worked
Even when DROP SCHEDULE returned success, RefreshSeconds in get-table sometimes remained. After removing a schedule, check RefreshSeconds and list-materialized-view-refresh-task-runs
Schema evolution is not supported, so to change the definition, drop and recreate

What I hope for

Documentation of how to specify refresh intervals in minutes
Adding the MV refresh task APIs to the Developer Guide's IAM policy example
An explanation or fix for the schedule remaining after DROP SCHEDULE

Conclusion

I tried AWS Glue's system-managed MV side by side with a regular MV. With the system-managed MV, data writes from Spark and Athena, definition changes via UpdateTable, and metadata swaps via the S3 Tables API were all rejected, even from roles with write permission. On the other hand, reads from Athena, REFRESH, table property changes, and deletion all worked.

If you provide aggregated results to other teams as an MV and want to keep the results from being changed regardless of how consumers' write permissions are set, a system-managed MV is an option. If you adopt it, assuming AWS Glue 6.0 and S3 Tables, check in advance how to handle the __ivm columns, the IAM permissions needed for REFRESH, and the time required for creation and refresh.