I tried out the new commands and functions in CloudWatch Logs Insights (parse logfmt, case, expand, and more)

Introduction

On May 21, 2026, 13 new commands and functions were added to CloudWatch Logs Insights. They are available in all commercial regions.

https://aws.amazon.com/jp/about-aws/whats-new/2026/06/amazon-cloudwatch-logs-insights/

The list of added features is as follows.

This article covers 11 of the features in practice, excluding urlencode / base64encode. These two were not tested hands-on this time and are only introduced in the list.

Verification Environment

A log group /test/insights-new-features was created in ap-northeast-1, and 115 test data entries were loaded for verification. The steps to reproduce are described in the collapsible section at the end of the article.

parse logfmt

It is now possible to directly parse structured logs in logfmt format.

Original log:

level=info service=payment-service method=POST path=/api/payment duration=234ms status=200 user_id=usr_12345

Query:

parse @message logfmt as lf
| filter lf.service = 'payment-service'
| display lf.service, lf.method, lf.path, lf.duration, lf.status, lf.user_id

Result:

The syntax is parse @message logfmt as lf. Use as to assign an alias, and access each field using dot notation like lf.key. Note that writing it as parse logfmt @message will result in a syntax error.

startsWith / endsWith

These are functions that filter by prefix match or suffix match on strings. The same result can be achieved with the traditional like operator (regular expressions), but startsWith has the advantage of not requiring escaping and being more readable.

Written with like:

filter path like /^\/api\//

Written with startsWith:

filter startsWith(path, '/api/')

filter startsWith(path, '/api/')
| display path, urldecode(path)

The return value of startsWith / endsWith is not a boolean but a numeric value (1 or 0). It works fine within filter since it is implicitly evaluated as a boolean, but when used with fields, it returns 1/0.

fields path, startsWith(path, '/api/') as starts_api, endsWith(path, 'users') as ends_users

regex_replace

This is string replacement using regular expressions. It uses RE2 syntax.

parse @message logfmt as lf
| filter ispresent(lf.path) and endsWith(lf.path, 'payment')
| display lf.path, regex_replace(lf.duration, 'ms$', '') as duration_num

The unit string was removed to extract only the numeric portion.

base64decode / urldecode

Encoded values can be decoded inline directly in the query.

base64decode

filter ispresent(payload_b64)
| display service, payload_b64, base64decode(payload_b64) as decoded

urldecode

As shown in the startsWith section, URL-encoded parameters can be decoded inline with urldecode(path).

case

This is a conditional branching function. The syntax is case(condition1, value1, condition2, value2, ..., default_value), which evaluates conditions from top to bottom and returns the value of the first condition that is true. If no condition matches, the default value at the end is returned.

filter ispresent(level)
| fields service, level,
  case(
    level = 'ERROR', 'critical',
    level = 'WARN', 'warning',
    'normal'
  ) as severity

Result (excerpt):

The 'normal' at the end is the default value with no condition. The default value is correctly returned even for levels like INFO and DEBUG that are not explicitly specified as conditions.

round + haversine

round is a function for rounding numbers, and haversine is a function that calculates the distance (in km) between two coordinates. The argument order is haversine(latitude1, longitude1, latitude2, longitude2).

filter service = 'geo-service' or service = 'cdn-edge'
| display service, haversine(origin_lat, origin_lon, dest_lat, dest_lon) as distance_km

The values were reasonable: geo-service (Tokyo → Osaka) at approximately 403 km, and cdn-edge (New York → London) at approximately 5,570 km.

round can be used to round to a specified number of decimal places.

filter ispresent(response_time_ms)
| fields response_time_ms, round(response_time_ms, 0) as rounded

relevantfields

This is a command that automatically identifies fields highly correlated with a specific condition. It can be used for initial investigation to find fields that are characteristic when errors occur.

It was executed against 100 test data entries (designed so that ERROR logs are concentrated on specific paths / services).

relevantfields service, path where level = 'ERROR'

The score for path is high, indicating that errors are concentrated on /api/auth. Specifically, the occurrence rate of /api/auth under ERROR conditions is 75% (compared to 14% under normal conditions), and this difference (frequencyChange) is reflected in the score. It is also possible to omit the field list and analyze all fields with the where condition, but when you have a hypothesis, specifying fields is more efficient.

Note that in this verification, using relevantfields alone (without a where clause) resulted in a syntax error, so it was executed in the form relevantfields <fields> where <condition>.

expand

This is a command that expands array fields into multiple records.

Original log:

event_type=order items=["apple","banana","cherry"] user=alice

Query:

filter @message like /order/
| parse @message 'items=* user=*' as raw_items, user
| expand raw_items
| display raw_items, user

Result:

It was expanded into 3 records. The key point is that expand is applied after extracting the array portion as a string using the glob pattern in parse.

In this verification, expand could be applied to fields that have a string representation of a JSON array. The documentation states that it operates on array-type fields, but it did not expand as expected with list types generated by jsonParse, and operation was confirmed only when extracting as a string using glob/regex in parse. Even with fields that were flattened by automatic JSON parsing (expanded with indexes like items.0, items.1), expansion did not work as expected in this verification.

Notes

• After ingesting logs, there may be an ingestion delay of 2–3 minutes before the results are reflected in Insights queries
• regex_replace uses RE2 syntax. Regular expression features not supported by RE2, such as backreferences and lookarounds, cannot be used

Summary

This update increases the amount of preprocessing that can be done within Logs Insights, reducing the need to export logs externally for processing. In particular, parse logfmt will be frequently used for application log analysis, and case-based conditional branching enhances the expressiveness of aggregation queries. Field processing with regex_replace and anomalous field identification with relevantfields should also help streamline day-to-day log investigations.

Reference Links

https://aws.amazon.com/jp/about-aws/whats-new/2026/06/amazon-cloudwatch-logs-insights/

https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_AnalyzeLogData_LogsInsights.html

aws logs create-log-group --log-group-name /test/insights-new-features --region ap-northeast-1
aws logs create-log-stream --log-group-name /test/insights-new-features --log-stream-name test-stream-1 --region ap-northeast-1

NOW_MS=$(date +%s%3N)

cat << EOF > /tmp/log_events.json
[
  {"timestamp": $((NOW_MS - 5000)), "message": "{\\"level\\":\\"INFO\\",\\"service\\":\\"api-gateway\\",\\"path\\":\\"/api/users?name=%E7%94%B0%E4%B8%AD&page=1\\",\\"status\\":200,\\"lat\\":35.6812,\\"lon\\":139.7671}"},
  {"timestamp": $((NOW_MS - 4000)), "message": "level=info service=payment-service method=POST path=/api/payment duration=234ms status=200 user_id=usr_12345"},
  {"timestamp": $((NOW_MS - 3000)), "message": "{\\"level\\":\\"DEBUG\\",\\"service\\":\\"data-pipeline\\",\\"payload_b64\\":\\"SGVsbG8gV29ybGQgZnJvbSBDbG91ZFdhdGNoIExvZ3M=\\",\\"tags\\":[\\"prod\\",\\"critical\\",\\"us-east-1\\"]}"},
  {"timestamp": $((NOW_MS - 2000)), "message": "{\\"level\\":\\"INFO\\",\\"service\\":\\"geo-service\\",\\"origin_lat\\":35.6812,\\"origin_lon\\":139.7671,\\"dest_lat\\":34.6937,\\"dest_lon\\":135.5023}"},
  {"timestamp": $((NOW_MS - 1000)), "message": "event_type=order items=[\\"apple\\",\\"banana\\",\\"cherry\\"] user=alice"}
]
EOF

aws logs put-log-events \
  --log-group-name /test/insights-new-features \
  --log-stream-name test-stream-1 \
  --log-events file:///tmp/log_events.json \
  --region ap-northeast-1

START_TIME=$(( $(date +%s) - 300 ))
END_TIME=$(( $(date +%s) + 60 ))

QUERY_ID=$(aws logs start-query \
  --log-group-name /test/insights-new-features \
  --start-time $START_TIME --end-time $END_TIME \
  --query-string 'parse @message logfmt as lf | filter lf.service = "payment-service" | display lf.service, lf.method, lf.path, lf.duration' \
  --region ap-northeast-1 \
  --query queryId --output text)

sleep 5
aws logs get-query-results --query-id $QUERY_ID --region ap-northeast-1

QUERY_ID=$(aws logs start-query \
  --log-group-name /test/insights-new-features \
  --start-time $START_TIME --end-time $END_TIME \
  --query-string 'filter @message like /order/ | parse @message "items=* user=*" as raw_items, user | expand raw_items | display raw_items, user' \
  --region ap-northeast-1 \
  --query queryId --output text)

sleep 5
aws logs get-query-results --query-id $QUERY_ID --region ap-northeast-1

aws logs delete-log-group --log-group-name /test/insights-new-features --region ap-northeast-1