I tried out the new CloudWatch Logs feature "Automatic Indexing"

Introduction

On September 29, 2026, automatic indexing for Amazon CloudWatch Logs was announced.

https://aws.amazon.com/jp/about-aws/whats-new/2026/09/cloudwatch-logs-auto-indexes-fields/

In this article, I output CloudTrail logs to CloudWatch Logs and verified how long it takes for automatic indexes to be created from queries, and how the number of scanned records changes.

Targets for Automatic Indexing and How to Check Them

Automatic indexes are created for fields used with = or IN in Logs Insights filters. They can be used at no additional cost. This is separate from the index policy limit of 20 per log group.

Automatic indexes are rotated based on recent queries and retained for 30 days. Fields that you always want indexed should be added to the field index policy.

Created automatic indexes can be checked using the DescribeFieldIndexes API. With the CLI, specify AUTO for --index-categories.

aws logs describe-field-indexes \
  --log-group-identifiers "/cloudtrail/members-20261002" \
  --index-categories AUTO

In the response, entries where indexCategory is AUTO are automatic indexes.

{
    "fieldIndexes": [
        {
            "fieldIndexName": "sourceIPAddress",
            "firstEventTime": 1791139845913,
            "type": "FIELD_INDEX",
            "indexCategory": "AUTO"
        }
    ]
}

Verification Environment

I output CloudTrail trails to a CloudWatch Logs log group and ran queries with approximately 160,000 log entries accumulated over the past 3 days. No index policy was configured.

Time from Query to Automatic Index Creation

For CloudTrail, I tried both = and IN syntax. As examples, here are two queries filtered by eventName.

fields @timestamp, eventName, eventSource | filter eventName = "DescribeLogGroups" | limit 20
fields @timestamp, eventName, eventSource | filter eventName IN ["GetCallerIdentity","DescribeLogStreams","DescribeFlowLogs"] | limit 20

I retrieved the list of automatic index fields immediately after running the queries and approximately 25 minutes later. I also ran similar queries on VPC flow logs (default VPC).

Automatic Index Fields Are Rotated

It took approximately 15 minutes for automatic indexes to be created after using awsRegion, eventType, sourceIPAddress, and recipientAccountId with =. The following shows 4 snapshots recorded at 60-second intervals, showing the state just before and after the fields were rotated.

2026-10-05 03:50:19 +0900 AUTO=[eventSource readOnly errorCode eventName]
2026-10-05 03:51:19 +0900 AUTO=[awsRegion eventName readOnly eventSource eventType]
2026-10-05 03:52:19 +0900 AUTO=[eventSource eventName eventType readOnly awsRegion]
2026-10-05 03:53:20 +0900 AUTO=[readOnly eventSource eventName recipientAccountId sourceIPAddress]

In this observation, the maximum number of automatic index fields was 5.

How the Number of Scanned Records Changes

For sourceIPAddress, only logs ingested after the automatic index was created are indexed. I compared recordsScanned for indexed logs versus non-indexed logs.

Indexed Logs (873 entries ingested after index creation)

For fields with an automatic index, only 2 records were scanned, including the matching one. For fields without an index, all 873 records were scanned.

Non-Indexed Logs (160,762 entries ingested before index creation)

filter scanned all non-indexed logs and found the matching entry, but filterIndex searches only indexed logs, so no matches were found. For this reason, it is recommended to use filter rather than filterIndex for automatic index fields.

https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CloudWatchLogs-Field-Indexing-Automatic.html

Summary

I confirmed that automatic indexes (AUTO) are created for fields filtered using = or IN in CloudWatch Logs Insights without configuring a policy, making queries more efficient.

When filtering and aggregating logs that meet specific conditions in CloudWatch Logs Insights, this can be expected to reduce scanning costs and processing time compared to before.

However, automatic index (AUTO) creation is asynchronous, and logs ingested before creation are not covered. Additionally, indexes may be removed after creation due to rotation. If it is necessary to avoid gaps caused by rotation, it is recommended to use an index policy.