I Tried Outputting CMCD and Additional Fields from CloudFront Standard Logs (v2) to CloudWatch Logs

I Tried Outputting CMCD and Additional Fields from CloudFront Standard Logs (v2) to CloudWatch Logs

I output CMCD and additional fields to CloudWatch Logs using Amazon CloudFront standard logging (v2). Both appear when specified with --record-fields, but CMCD fields were populated only for curl requests sent with them in the query or headers.
2026.11.19

This page has been translated by machine translation. View original

Introduction

In an announcement on November 18, 2026, CloudFront standard logging (v2) became able to include real-time log fields such as CMCD (Common Media Client Data). Below, fields other than CMCD that are not included in the default fields for CloudWatch Logs are called additional fields.

https://aws.amazon.com/jp/about-aws/whats-new/2026/11/cloudfront-standard-logging-cmcd-fields/

In this article, I sent requests with curl to a distribution whose origin is an S3 static website, and checked the output of the additional fields and CMCD in CloudWatch Logs (JSON).

Output of Additional Fields

The fields that can be output to CloudWatch Logs fall into the following three categories.

Default fields (33)
Field
date
time
x-edge-location
sc-bytes
c-ip
cs-method
cs(Host)
cs-uri-stem
sc-status
cs(Referer)
cs(User-Agent)
cs-uri-query
cs(Cookie)
x-edge-result-type
x-edge-request-id
x-host-header
cs-protocol
cs-bytes
time-taken
x-forwarded-for
ssl-protocol
ssl-cipher
x-edge-response-result-type
cs-protocol-version
fle-status
fle-encrypted-fields
c-port
time-to-first-byte
x-edge-detailed-result-type
sc-content-type
sc-content-len
sc-range-start
sc-range-end

Of the additional fields, those other than the ones added in this announcement are the following 13.

Field
timestamp
DistributionId
timestamp(ms)
origin-fbl
origin-lbl
asn
c-country
cache-behavior-path-pattern
distributionid
distribution-tenant-id
connection-id
viewer-request-log-data
viewer-response-log-data

The following 6 were added in this announcement.

Field
s-ip
primary-distribution-id
primary-distribution-dns-name
r-host
sr-reason
x-edge-mqcs

The CMCD fields you can select are the following 18.

Field
cmcd-encoded-bitrate
cmcd-buffer-length
cmcd-buffer-starvation
cmcd-content-id
cmcd-object-duration
cmcd-deadline
cmcd-measured-throughput
cmcd-next-object-request
cmcd-next-range-request
cmcd-object-type
cmcd-playback-rate
cmcd-requested-maximum-throughput
cmcd-streaming-format
cmcd-session-id
cmcd-stream-type
cmcd-startup
cmcd-top-bitrate
cmcd-version

Creating the Log Delivery

The standard logging (v2) API must be called in us-east-1, so I ran the following commands with the AWS CLI region set to us-east-1. First, create the log group and the resource policy.

Creating the log group and resource policy
aws logs create-log-group --log-group-name cf-stdlog-cmcd-test

Save the following content as artifacts/log-policy.json.

{
  "Version": "2012-10-17",
  "Statement": [{
    "Sid": "AllowCloudFrontLogDelivery",
    "Effect": "Allow",
    "Principal": {"Service": "delivery.logs.amazonaws.com"},
    "Action": ["logs:CreateLogStream", "logs:PutLogEvents"],
    "Resource": "arn:aws:logs:us-east-1:123456789012:log-group:cf-stdlog-cmcd-test:*",
    "Condition": {
      "StringEquals": {"aws:SourceAccount": "123456789012"},
      "ArnLike": {"aws:SourceArn": "arn:aws:logs:us-east-1:123456789012:delivery-source:*"}
    }
  }]
}
aws logs put-resource-policy --policy-name cf-stdlog-cmcd-test-policy --policy-document file://artifacts/log-policy.json

Next, create the delivery source, delivery destination, and delivery in that order.

aws logs put-delivery-source --name cf-stdlog-cmcd-source --resource-arn arn:aws:cloudfront::123456789012:distribution/EDFDVBD6EXAMPLE --log-type ACCESS_LOGS --output json
aws logs put-delivery-destination --name cf-stdlog-cmcd-destination --output-format json --delivery-destination-configuration destinationResourceArn=arn:aws:logs:us-east-1:123456789012:log-group:cf-stdlog-cmcd-test --output json

The additional fields and CMCD are included in the selectable fields (allowedFields). The list above is the result of the following command.

aws logs describe-configuration-templates --service cloudfront --log-type ACCESS_LOGS --delivery-destination-types CWL --output json

For the delivery's --record-fields, I specified 10 fields chosen from the default fields, plus 8 of the additional fields (including timestamp(ms)) and all 18 CMCD fields.

aws logs create-delivery \
  --delivery-source-name cf-stdlog-cmcd-source \
  --delivery-destination-arn arn:aws:logs:us-east-1:123456789012:delivery-destination:cf-stdlog-cmcd-destination \
  --record-fields timestamp\(ms\) date time x-edge-location x-edge-request-id cs-method cs-uri-stem cs-uri-query sc-status x-edge-result-type cs\(User-Agent\) \
    s-ip asn c-country origin-fbl origin-lbl cache-behavior-path-pattern r-host \
    cmcd-encoded-bitrate cmcd-buffer-length cmcd-buffer-starvation cmcd-content-id cmcd-object-duration cmcd-deadline cmcd-measured-throughput \
    cmcd-next-object-request cmcd-next-range-request cmcd-object-type cmcd-playback-rate cmcd-requested-maximum-throughput cmcd-streaming-format \
    cmcd-session-id cmcd-stream-type cmcd-startup cmcd-top-bitrate cmcd-version \
  --output json

Output Records

This time, it took about 79 seconds from sending the first request until the 6 log entries appeared in CloudWatch Logs.

Here is the record for a normal request without CMCD (Miss). x-edge-request-id and cmcd-* are omitted. All 18 cmcd-* fields were -.

{
  "s-ip": "192.0.2.54",
  "r-host": "amzn-s3-demo-bucket.s3-website-ap-northeast-1.amazonaws.com",
  "timestamp(ms)": "1795051531698",
  "date": "2026-11-19",
  "time": "01:25:31",
  "x-edge-location": "LAX54-P8",
  "cs-method": "GET",
  "cs-uri-stem": "/",
  "cs-uri-query": "r=2",
  "sc-status": "200",
  "x-edge-result-type": "Miss",
  "cs(User-Agent)": "curl/8.18.0",
  "asn": "64496",
  "c-country": "JP",
  "origin-fbl": "0.272",
  "origin-lbl": "0.663",
  "cache-behavior-path-pattern": "*"
}

In the Hit record, origin-fbl, origin-lbl, and r-host were -. The definition of each field is in the documentation.

https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/standard-logging.html

To compare s-ip with curl's connection destination, I sent three requests, changing cs-uri-query from ip=1 to ip=3. curl's %{remote_ip} was 192.0.2.193, 192.0.2.54, and 192.0.2.140 in that order, each matching the s-ip of the corresponding record.

Applying Configuration Changes

To add fields to an existing delivery, pass --id and --record-fields to update-delivery-configuration. Here, I specified the same 36 fields as in create-delivery plus Referer.

aws logs update-delivery-configuration --id <delivery-id> --record-fields <create-delivery と同じ36個> cs\(Referer\)

This time, I ran the update command at 10:30:07 (JST) and, one second later at 10:30:08, sent a curl request with a Referer header. That record contained "cs(Referer)": "https://example.com/probe-1".

Output of CMCD

How to Send CMCD

CMCD is playback information that the player attaches to each request, such as bitrate, buffer length, and session ID. With curl, I sent it either as the CMCD query parameter or with four types of headers (CMCD-Request, CMCD-Object, CMCD-Status, CMCD-Session). For the query, the whole value was URL-encoded, and for the headers, I passed values in the form bl=21300,....

curl 'https://d111111abcdef8.cloudfront.net/sample.png?CMCD=bl%3D21300%2Cbr%3D3200%2Cbs%2Ccid%3D%22c-query-default%22%2Cd%3D4004%2Cdl%3D18500%2Cmtp%3D48100%2Cnor%3D%22..%252Fnext.m4v%22%2Cnrr%3D%2212323-48763%22%2Cot%3Dv%2Cpr%3D1.08%2Crtp%3D15000%2Csf%3Dh%2Csid%3D%22s-query-default%22%2Cst%3Dv%2Csu%2Ctb%3D6000%2Cv%3D1'
curl \
  -H 'CMCD-Request: bl=21300,dl=18500,mtp=48100,nor="..%2Fnext.m4v",nrr="12323-48763",su' \
  -H 'CMCD-Object: br=3200,d=4004,ot=v,tb=6000' \
  -H 'CMCD-Status: bs,rtp=15000' \
  -H 'CMCD-Session: cid="c-header-default",pr=1.08,sf=h,sid="s-header-default",st=v,v=1' \
  https://d111111abcdef8.cloudfront.net/sample.png

Values Shown in the Logs

These are the CMCD fields of the record when all keys were sent via headers. The record sent via query had the same values, except for cmcd-content-id and cmcd-session-id (c-query-default, s-query-default).

"cmcd-encoded-bitrate": "3200",
"cmcd-buffer-length": "21300",
"cmcd-buffer-starvation": "1",
"cmcd-content-id": "c-header-default",
"cmcd-object-duration": "4004",
"cmcd-deadline": "18500",
"cmcd-measured-throughput": "48100",
"cmcd-next-object-request": "..%252Fnext.m4v",
"cmcd-next-range-request": "12323-48763",
"cmcd-object-type": "v",
"cmcd-playback-rate": "1.08",
"cmcd-requested-maximum-throughput": "15000",
"cmcd-streaming-format": "h",
"cmcd-session-id": "s-header-default",
"cmcd-stream-type": "v",
"cmcd-startup": "1",
"cmcd-top-bitrate": "6000",
"cmcd-version": "1"

bs and su, which were sent without values, were output as 1 in cmcd-buffer-starvation and cmcd-startup. The ..%2Fnext.m4v sent in nor had its % re-encoded and was output as ..%252Fnext.m4v.

When I sent only sid, or only br and ot, the fields corresponding to bs, su, and v, which I had not sent, also got values.

What was sent Fields that got values
Nothing None (all 18 cmcd-* fields are -)
Only sid in the CMCD-Session header cmcd-session-id, cmcd-buffer-starvation (0), cmcd-startup (0), cmcd-version (1)
Only br and ot in the CMCD-Object header cmcd-encoded-bitrate (3200), cmcd-object-type (v), cmcd-buffer-starvation (0), cmcd-startup (0), cmcd-version (1)

I sent the CMCD-Session header (cid, sid, st) and the query (br, cid, ot, sid, st, tb) at the same time. The values of cid and sid differed between the header and the query. cmcd-content-id and cmcd-session-id contained the cid and sid from the header. The three fields corresponding to br, ot, and tb, which were only in the query, were -.

Relationship with the Cache Policy

Using the following two behaviors, I checked Hit / Miss for requests with different CMCD values. I sent the requests after the distribution became Deployed.

  • Default behavior (*): the managed cache policy Managed-CachingOptimized. Query strings are not included in the cache key.
  • /sample.png behavior: a custom policy. All query strings are included in the cache key, while CMCD headers and cookies are not.

For CMCD sent via query, I changed only cid and sid.

Path How sent Request x-cache
/ Query cid/sid with the first values Hit
/ Query cid/sid with the second values Hit
/sample.png Query cid/sid with the first values Miss
/sample.png Query cid/sid with the second values Miss
/sample.png Query Resend of the first values Hit

The CMCD values were output to the logs in both behaviors. In the behavior that includes all query strings in the cache key, a change in CMCD values resulted in a Miss, and resending the same values resulted in a Hit.

Cache policy used

Save the following content as artifacts/cache-policy-qs-all.json.

{
  "Name": "cf-stdlog-cmcd-qs-all",
  "Comment": "query strings (all) in cache key, no headers, no cookies",
  "MinTTL": 0,
  "DefaultTTL": 86400,
  "MaxTTL": 31536000,
  "ParametersInCacheKeyAndForwardedToOrigin": {
    "EnableAcceptEncodingGzip": true,
    "EnableAcceptEncodingBrotli": true,
    "HeadersConfig": {"HeaderBehavior": "none"},
    "CookiesConfig": {"CookieBehavior": "none"},
    "QueryStringsConfig": {"QueryStringBehavior": "all"}
  }
}
aws cloudfront create-cache-policy --cache-policy-config file://artifacts/cache-policy-qs-all.json --query CachePolicy.Id --output text

Summary

CloudFront standard logging (v2) can now include 6 additional fields such as s-ip and r-host, as well as CMCD fields.

If the standard log fields were insufficient and you were compensating with something like Kinesis Data Streams real-time logs, try the newly available fields.

Note that adding log fields may also require adjustments in tools such as Athena that you use to analyze standard logs (v2) output to S3.

Share this article

AWSのお困り事はクラスメソッドへ