I Tried Sending Traces from an OpenTelemetry Collector on EC2 to Amazon CloudWatch Using an API Key (Bearer Token)

Introduction

On November 3, 2026, the X-Ray OTLP trace endpoint began supporting API key (bearer token) authentication in addition to SigV4.

https://aws.amazon.com/jp/about-aws/whats-new/2026/11/amazon-cloudwatch-api-keys-opentelemetry

In this article, I sent traces with an API key from an OpenTelemetry Collector on EC2 in the Tokyo Region, and confirmed that they can be viewed with CloudWatch Logs Insights and the X-Ray retrieval APIs.

Test Environment

Configuration

IAM user and API key

I created an IAM user, attached the AWSXRayAPIKeyAccess policy, and issued an API key with xray.amazonaws.com as the service name. The key's validity period was set to 1 day.

aws iam create-user --user-name xray-traces-api-key-user
aws iam attach-user-policy --user-name xray-traces-api-key-user --policy-arn arn:aws:iam::aws:policy/AWSXRayAPIKeyAccess
aws iam create-service-specific-credential --user-name xray-traces-api-key-user --service-name xray.amazonaws.com --credential-age-days 1

The issuance procedure and required permissions are described in the developer guide.

https://docs.aws.amazon.com/xray/latest/devguide/xray-opentelemetry-bearer-token.html

EC2 role

When I evaluated xray:PutTraceSegments and xray:CallWithBearerToken for the sender EC2's role with simulate-principal-policy, both were implicitDeny.

aws iam simulate-principal-policy --policy-source-arn arn:aws:iam::<ACCOUNT>:role/xray-apikey-verify-ec2 --action-names xray:PutTraceSegments xray:CallWithBearerToken --query 'EvaluationResults[].[EvalActionName,EvalDecision]'

[
    [
        "xray:PutTraceSegments",
        "implicitDeny"
    ],
    [
        "xray:CallWithBearerToken",
        "implicitDeny"
    ]
]

With this role's permissions, traces cannot be sent to X-Ray.

Collector and sender

I saved the ServiceCredentialSecret value from the issuance response to /etc/otelcol-contrib/xray-api-key without a trailing newline. The permissions are 600, and the owner is the otelcol-contrib user. The Collector's bearertokenauth extension reads the API key from this file and sets it in the Authorization header.

I saved the Collector configuration to /etc/otelcol-contrib/config.yaml.

extensions:
  bearertokenauth:
    filename: "/etc/otelcol-contrib/xray-api-key"

receivers:
  otlp:
    protocols:
      http:
        endpoint: "127.0.0.1:4318"

processors:
  batch:

exporters:
  otlp_http/xray:
    compression: gzip
    traces_endpoint: "https://xray.ap-northeast-1.amazonaws.com/v1/traces"
    auth:
      authenticator: bearertokenauth

service:
  extensions: [bearertokenauth]
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlp_http/xray]

The sender is a Python script that creates one parent span and three child spans, attaches a test.run_id attribute to each, and sends them to the Collector at 127.0.0.1:4318.

import sys
import time
import uuid

from opentelemetry import trace
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor

run_id = sys.argv[1] if len(sys.argv) > 1 else uuid.uuid4().hex[:8]

provider = TracerProvider(resource=Resource.create({"service.name": "apikey-demo"}))
provider.add_span_processor(
    BatchSpanProcessor(OTLPSpanExporter(endpoint="http://127.0.0.1:4318/v1/traces"))
)
trace.set_tracer_provider(provider)
tracer = trace.get_tracer("apikey-demo")

with tracer.start_as_current_span("parent-job") as parent:
    parent.set_attribute("test.run_id", run_id)
    for i in range(3):
        with tracer.start_as_current_span(f"child-step-{i}") as child:
            child.set_attribute("test.run_id", run_id)
            time.sleep(0.05)

provider.shutdown()
print(f"sent run_id={run_id}")

Viewing Traces

Logs Insights

For the first run, I specified run092751 as test.run_id and sent one trace (4 spans). The spans went into the aws/spans log group in CloudWatch Logs and could be searched with the following query.

fields @timestamp, name, spanId, parentSpanId, attributes.test.run_id | filter attributes.test.run_id = 'run092751' | sort @timestamp asc

The parentSpanId of the three child spans matches the spanId of parent-job.

Next, I set test.run_id to idx-092911 and sent one trace at a time, 300 times, adding 300 traces (1200 spans). The Collector metrics are as follows. 1204 is the total of the 4 spans from the first run and 300 × 4 spans.

In Logs Insights, I counted the number of spans and traces with the following query.

filter attributes.test.run_id = 'idx-092911' | stats count(*) as spans, count_distinct(traceId) as traces

About 2 minutes after sending finished, spans was 1144, which had not reached 1200. When I re-ran it about 4 minutes later, spans was 1200 and traces was 300.

X-Ray retrieval API

I specified --start-time 1794269290 --end-time 1794270790 for get-trace-summaries. This range includes the 301 traces I sent (1 trace from the first run and the additional 300 traces). 4 were returned. When I retrieved again with the same range about 2 days and 17 hours later, it was still 4.

The 4 returned is a number consistent with the 1% indexing rate setting.

Summary

I confirmed that traces can be sent to the X-Ray OTLP trace endpoint with API key (bearer token) authentication.

This seems useful when sending to the X-Ray OTLP trace endpoint from an environment outside AWS, when it is hard to attach an IAM role for X-Ray even in an AWS environment, or when you want to send to X-Ray in another account.

Please attach only the minimum necessary permissions to the IAM user that creates the API key. If possible, I recommend applying IP restrictions and preparing for rotation before using it.