I tried out the new CloudWatch Logs feature "Automatic Indexing"

Introduction

On September 29, 2026, automatic indexing for Amazon CloudWatch Logs was announced.

https://aws.amazon.com/jp/about-aws/whats-new/2026/09/cloudwatch-logs-auto-indexes-fields/

This article outputs CloudTrail logs to CloudWatch Logs and examines how long it takes for automatic indexes to be created from queries, and how the number of scanned records changes.

Targets and Verification Method for Automatic Indexes

Automatic indexes are created for fields that use = or IN in Logs Insights filters. They can be used at no additional cost. They are separate from the index policy limit of 20 per log group.

Automatic indexes rotate based on recent queries and are retained for 30 days. Fields you want to keep indexed at all times should be added to the field index policy.

Created automatic indexes can be confirmed using the DescribeFieldIndexes API. With the CLI, specify AUTO for --index-categories.

aws logs describe-field-indexes \
  --log-group-identifiers "/cloudtrail/members-20261002" \
  --index-categories AUTO

In the response, items where indexCategory is AUTO are automatic indexes.

{
    "fieldIndexes": [
        {
            "fieldIndexName": "sourceIPAddress",
            "firstEventTime": 1791139845913,
            "type": "FIELD_INDEX",
            "indexCategory": "AUTO"
        }
    ]
}

Verification Environment

CloudTrail trails were output to a CloudWatch Logs log group, and queries were executed with approximately 160,000 logs from the past 3 days accumulated. No index policy was configured.

Time from Query to Automatic Index Creation

For CloudTrail, both = and IN syntax were tested. As examples, two queries filtered by eventName are shown below.

fields @timestamp, eventName, eventSource | filter eventName = "DescribeLogGroups" | limit 20
fields @timestamp, eventName, eventSource | filter eventName IN ["GetCallerIdentity","DescribeLogStreams","DescribeFlowLogs"] | limit 20

The list of automatic index fields was retrieved immediately after query execution and approximately 25 minutes later. Similar queries were also executed for VPC flow logs (default VPC).

Automatic Index Fields Rotate

It took approximately 15 minutes for automatic indexes to be created after using awsRegion, eventType, sourceIPAddress, and recipientAccountId with =. The following shows 4 consecutive entries, recorded at 60-second intervals, around the time the fields rotated.

2026-10-05 03:50:19 +0900 AUTO=[eventSource readOnly errorCode eventName]
2026-10-05 03:51:19 +0900 AUTO=[awsRegion eventName readOnly eventSource eventType]
2026-10-05 03:52:19 +0900 AUTO=[eventSource eventName eventType readOnly awsRegion]
2026-10-05 03:53:20 +0900 AUTO=[readOnly eventSource eventName recipientAccountId sourceIPAddress]

In this observation, the maximum number of automatic index fields was 5.

How Does the Number of Scanned Records Change?

For sourceIPAddress, only logs ingested after the automatic index was created are indexed. The recordsScanned values were compared between indexed and non-indexed logs.

Indexed Logs (873 records ingested after index creation)

For fields with an automatic index, only 2 records including the matching one were scanned. For fields without one, all 873 records were scanned.

Non-Indexed Logs (160,762 records ingested before index creation)

filter scanned all non-indexed logs and found the matching record, while filterIndex searches only indexed logs, resulting in 0 matches. For this reason, it is recommended to use filter rather than filterIndex for automatic index fields.

https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CloudWatchLogs-Field-Indexing-Automatic.html

Summary

We confirmed that automatic indexes (AUTO) are created for fields filtered using = or IN in CloudWatch Logs Insights without needing to configure a policy, making queries more efficient.

When filtering and aggregating logs that meet specific conditions in CloudWatch Logs Insights, this feature is expected to reduce scanning costs and processing time compared to before.

However, automatic index (AUTO) creation is asynchronous, and logs ingested before creation are not covered. Additionally, indexes may be removed after creation due to rotation. If it is necessary to avoid missing data due to rotation, using an index policy is recommended.