[Update] Tried out how AWS Glue zero-ETL integration now stops duplicate writes to the same table with a ConflictException

[Update] Tried out how AWS Glue zero-ETL integration now stops duplicate writes to the same table with a ConflictException

AWS Glue zero-ETL integration now records ownership in target table properties, and double writes to the same table are detected and rejected at creation time. Let's actually cause a conflict to verify this.
2026.09.21

This page has been translated by machine translation. View original

This is Ishikawa from the Cloud Business Division. AWS Glue zero-ETL integration now records ownership in target table properties, making it impossible to create a configuration where two integrations write to the same table. I actually tried causing a conflict to verify this.

https://aws.amazon.com/jp/about-aws/whats-new/2026/09/glue-zero-etl-ownership-conflicts/

AWS Glue zero-ETL integration is a feature that replicates data from DynamoDB or SaaS applications to Amazon S3 Tables or SageMaker Lakehouse catalogs without writing ETL jobs.

The target table properties written by zero-ETL integration now record "which integration owns it." When you try to create a new integration for a combination already owned by a different integration, AWS Glue detects the conflict and rejects the creation.

When operating multiple zero-ETL integrations, you might accidentally create a duplicate integration pointing the same source table to the same target, causing one to overwrite the other's data. This update stops that accident at creation time.

https://docs.aws.amazon.com/glue/latest/dg/zero-etl-target.html

What is ownership in target table properties

According to the official documentation, target table properties are identified by two things: the "target resource ARN" and the "source table name." The target resource refers to the AWS Glue database when using a general-purpose S3 bucket as the target, or the S3 Tables catalog when using Amazon S3 Tables as the target.

Only one integration can be associated with each such combination at a time. As a result, two integrations cannot write the same source table to the same target resource. This restriction applies regardless of whether you use the management console, AWS CLI, or API.

Trying it out

Prerequisites

  • Verification region: ap-northeast-1
  • AWS CLI: 2.36.45 (updated to the latest version at the time of writing)
  • Source: Amazon DynamoDB table (single table)
  • Target: AWS Glue database associated with a general-purpose S3 bucket

I used DynamoDB as the source this time. For the target, instead of Amazon S3 Tables specifically mentioned in this update, I chose a general-purpose S3 bucket + Glue database configuration. Since the ownership identification unit is "target resource × source table name" in both cases, I verified using the latter, which requires fewer prerequisite resources.

Verification environment

Source side

On the source side, I created a DynamoDB table zetl_orders and inserted 3 sample records. Enabling Point-in-time recovery (PITR) is required to use it as a zero-ETL source. Additionally, to allow AWS Glue to export the table, I granted glue.amazonaws.com permissions for dynamodb:ExportTableToPointInTime, dynamodb:DescribeTable, and dynamodb:DescribeExport via a resource-based policy.

Target side

On the target side, I created an S3 bucket and a Glue database zetl_blog with that path specified as LocationUri. Specifying LocationUri is a mandatory requirement for zero-ETL. Next, I prepared an IAM role for AWS Glue to write to this database and associated it with the target resource using create-integration-resource-property. Finally, I added glue:CreateInboundIntegration and glue:AuthorizeInboundIntegration to the Glue data catalog resource policy. The former is granted to the integration creator, and the latter to glue.amazonaws.com — both are required for integration creation to succeed.

Creating target table properties

I create the target table properties, specifying orders_i1 as the output table name.

% aws glue create-integration-table-properties \
  --resource-arn arn:aws:glue:ap-northeast-1:123456789012:database/zetl_blog \
  --table-name zetl_orders \
  --target-table-config '{"UnnestSpec":"FULL","TargetTableName":"orders_i1"}'

Here, let me check the state using the newly added list-integration-table-properties. You can narrow down the target using --filters.

% aws glue list-integration-table-properties \
  --filters 'Name=TargetArn,Values=arn:aws:glue:ap-northeast-1:123456789012:database/zetl_blog'
{
    "IntegrationTablePropertiesList": [
        {
            "ResourceArn": "arn:aws:glue:ap-northeast-1:123456789012:database/zetl_blog",
            "TableName": "zetl_orders",
            "TargetTableConfig": {
                "UnnestSpec": "FULL",
                "PartitionSpec": [],
                "TargetTableName": "orders_i1"
            }
        }
    ]
}

At this point, IntegrationArn is not returned. The table properties have been created, but there is no owner yet.

Ownership is recorded when an integration is created

I create a zero-ETL integration.

% aws glue create-integration \
  --integration-name zetl-blog-i1 \
  --source-arn arn:aws:dynamodb:ap-northeast-1:123456789012:table/zetl_orders \
  --target-arn arn:aws:glue:ap-northeast-1:123456789012:database/zetl_blog
{
    "SourceArn": "arn:aws:dynamodb:ap-northeast-1:123456789012:table/zetl_orders",
    "TargetArn": "arn:aws:glue:ap-northeast-1:123456789012:database/zetl_blog",
    "IntegrationName": "zetl-blog-i1",
    "IntegrationArn": "arn:aws:glue:ap-northeast-1:123456789012:integration:be1697bf-04db-4fea-9de2-6c65f873f4aa",
    "Status": "CREATING",
    "CreateTime": "2026-09-15T19:54:37.291000+09:00",
    "DataFilter": "include: *"
}

While the Status is still CREATING, let me check the table properties again.

% aws glue list-integration-table-properties \
  --filters 'Name=TargetArn,Values=arn:aws:glue:ap-northeast-1:123456789012:database/zetl_blog'
{
    "IntegrationTablePropertiesList": [
        {
            "ResourceArn": "arn:aws:glue:ap-northeast-1:123456789012:database/zetl_blog",
            "TableName": "zetl_orders",
            "TargetTableConfig": {
                "UnnestSpec": "FULL",
                "PartitionSpec": [],
                "TargetTableName": "orders_i1",
                "IntegrationArn": "arn:aws:glue:ap-northeast-1:123456789012:integration:be1697bf-04db-4fea-9de2-6c65f873f4aa"
            }
        }
    ]
}

IntegrationArn has been added, containing the ARN of the created integration. Ownership is recorded at the CREATING stage, without waiting for the integration to become ACTIVE.

Creating a second integration with the same combination causes a conflict

Let me try creating a second integration pointing the same DynamoDB table to the same Glue database.

% aws glue create-integration \
  --integration-name zetl-blog-i2 \
  --source-arn arn:aws:dynamodb:ap-northeast-1:123456789012:table/zetl_orders \
  --target-arn arn:aws:glue:ap-northeast-1:123456789012:database/zetl_blog

An error occurred (ConflictException) when calling the CreateIntegration operation:
Source table 'zetl_orders' is already being written to the same target by integration(s)
arn:aws:glue:ap-northeast-1:123456789012:integration:be1697bf-04db-4fea-9de2-6c65f873f4aa.
Please choose a different target ARN or check the existing integration(s).

It was rejected with a ConflictException. One notable point is that the error message includes the ARN of the owning integration directly. Rather than just saying "someone is already writing to this," it returns exactly which integration holds ownership. When a team member encounters this conflict, they can pass this ARN to describe-integrations to identify the name and creation time of the conflicting integration.

% aws glue describe-integrations \
  --integration-identifier arn:aws:glue:ap-northeast-1:123456789012:integration:be1697bf-04db-4fea-9de2-6c65f873f4aa \
  --query 'Integrations[].{Name:IntegrationName,Status:Status,Created:CreateTime}' \
  --output table
-------------------------------------------------------------------
|                      DescribeIntegrations                       |
+---------------+----------+--------------------------------------+
|     Name      | Status   |               Created                |
+---------------+----------+--------------------------------------+
|  zetl-blog-i1 |  ACTIVE  |  2026-09-15T19:54:37.268000+09:00    |
+---------------+----------+--------------------------------------+

Note that you must pass the ARN rather than the integration name to --integration-identifier. Passing a name returned a ValidationException.

Discussion

Here is a summary of what was confirmed in this verification.

  • Immediately after creating target table properties, IntegrationArn is not attached; it is attached when an integration is created. The creation of table properties and the recording of ownership happen at different times.

  • Ownership is recorded at the point when the integration enters CREATING state. It does not wait for the integration to become ACTIVE. Immediately after create-integration returns a response, those table properties were already in an owned state.

  • A second integration pointing the same source table to the same target resource is rejected with a ConflictException. The fact that the error message includes the ARN of the owning integration has practical operational value. When multiple teams operate zero-ETL integrations in the same account, they can identify the conflicting party from the error message alone.

According to the official documentation, the unit of restriction is "target resource × source table name." Even with the same source table, different target resources are treated as different combinations. Conversely, the path for feeding the same source table into a single Glue database is always limited to one. Additionally, concurrent modification detection is handled differently from other conditions and does not block creation. If another user updates the same output settings, you can choose to either retrieve the latest settings or proceed as-is.

https://docs.aws.amazon.com/glue/latest/dg/zero-etl-target.html

Closing

AWS Glue zero-ETL integration target table properties now record ownership, making it impossible to create duplicate integrations writing the same source table to the same target resource. Conflicts are returned as ConflictException at creation time, and the message includes the ARN of the owning integration.

The simultaneously added ListIntegrationTableProperties now allows you to list target table properties across your account. In this verification as well, I used this API to confirm the difference before and after ownership was recorded. In environments running multiple zero-ETL integrations, you can use this listing to track which integration owns which table properties.

Usage requires AWS CLI 2.36.45 or later, or botocore 1.43.94 or later. Older versions cannot handle the IntegrationArn field or the listing API.

Share this article

AWSのお困り事はクラスメソッドへ