Amazon ECS Managed Instances Now Supports the json-file Log Driver, So I Tried Forwarding Logs to S3 and CloudWatch Logs

Introduction

On November 4, 2026, Amazon ECS Managed Instances added support for the json-file log driver. A container that specifies json-file writes its logs to a local JSON file. If you mount the log driver volume newly added to Managed Daemons into a daemon, the daemon can read that file.

https://aws.amazon.com/jp/about-aws/whats-new/2026/11/amazon-ecs-managed-instances-json-file-log-driver/

In this article, I read nginx logs with Fluent Bit in a daemon and forward them to S3 and CloudWatch Logs.

Test Environment

Configuration

The log path is determined by two task definitions: one for the app side and one for the daemon side.

App Task Definition

On the app side, you only need to specify json-file in logConfiguration. The task definition is as follows.

{
  "family": "jsonlog-demo-nginx",
  "requiresCompatibilities": ["MANAGED_INSTANCES"],
  "networkMode": "awsvpc",
  "cpu": "256",
  "memory": "512",
  "runtimePlatform": {"cpuArchitecture": "X86_64", "operatingSystemFamily": "LINUX"},
  "containerDefinitions": [
    {
      "name": "nginx",
      "image": "public.ecr.aws/nginx/nginx:latest",
      "essential": true,
      "portMappings": [{"containerPort": 80, "protocol": "tcp"}],
      "logConfiguration": {"logDriver": "json-file"}
    }
  ]
}

The official nginx image links access.log to stdout and error.log to stderr, so both the access log and the error log go into json-file.

Daemon Task Definition

On the daemon side, you define a log driver volume in volumes and mount it into the container with mountPoints. Only the logs of containers that specify json-file go into the volume.

https://docs.aws.amazon.com/AmazonECS/latest/developerguide/managed-daemons-task-definitions.html

  "volumes": [
    {
      "name": "app-logs",
      "logDriverVolumeConfiguration": {
        "driver": "json-file"
      }
    }
  ],
      "mountPoints": [
        {
          "sourceVolume": "app-logs",
          "containerPath": "/var/log/tasks"
        }
      ],

I saved the definition containing this excerpt as daemon-taskdef.json and registered it with the following command.

aws ecs register-daemon-task-definition --cli-input-json file://daemon-taskdef.json

Forwarding

Fluent Bit Configuration

Fluent Bit in the daemon reads the volume with the tail input and sends it to two outputs: s3 and cloudwatch_logs. Under the mount destination, a directory is created for each container, and <runtimeId>-json.log is placed inside it. The documentation calls the directory name the "container ID," but in my measurements it matched the runtimeId returned by describe-tasks.

For the daemon container, I used the AWS-provided public.ecr.aws/aws-observability/aws-for-fluent-bit:stable as is. I did not specify the container user and ran it with the image default. Because the image reads its default configuration file from a fixed path, I overrode the startup command to pass my own configuration. The configuration embedded in the environment variable FB_CONF is written out to /tmp/fluent-bit.conf and passed to fluent-bit -c. The relevant part of the container definition is as follows.

"environment": [{"name": "FB_CONF", "value": "<下の設定ファイルの内容>"}],
"command": ["sh", "-c", "printf '%s' \"$FB_CONF\" > /tmp/fluent-bit.conf && exec /fluent-bit/bin/fluent-bit -c /tmp/fluent-bit.conf"]

[SERVICE]
    Flush         5
    Log_Level     info
    Parsers_File  /fluent-bit/parsers/parsers.conf

[INPUT]
    Name              tail
    Path              /var/log/tasks/*/*-json.log
    Tag_Regex         /var/log/tasks/[^/]+/(?<runtime_id>[^/]+)-json\.log$
    Tag               <runtime_id>
    Parser            docker
    Read_from_Head    On
    Refresh_Interval  5

[OUTPUT]
    Name              s3
    Match             *
    bucket            amzn-s3-demo-bucket
    region            ap-northeast-1
    s3_key_format     /logs/$TAG/%Y%m%d%H%M%S-$UUID.json
    use_put_object    On
    total_file_size   1M
    upload_timeout    20s

[OUTPUT]
    Name               cloudwatch_logs
    Match              *
    region             ap-northeast-1
    log_group_name     /jsonlog-demo/app
    log_stream_prefix  app-
    auto_create_group  On

Tag_Regex extracts the runtimeId from the file name and uses it as the Tag. This Tag goes into the $TAG part of the S3 key, and after log_stream_prefix in the CloudWatch Logs stream name. For Parser, I specified the docker parser from the parsers.conf bundled with the image, which splits each json-file line into log, stream, and time. Because I set auto_create_group to On, the plugin created the log group.

In this configuration, I did not specify the tail DB. Specifying a DB lets it carry over the read position after a restart, so if you operate by restarting the daemon, you may need to specify a DB.

IAM Permissions

I attached the following policy to the daemon's task role. Since this is for testing, I did not narrow it down to least privilege.

{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Action":["s3:PutObject"],"Resource":"arn:aws:s3:::amzn-s3-demo-bucket/*"},{"Effect":"Allow","Action":["logs:CreateLogGroup","logs:CreateLogStream","logs:PutLogEvents"],"Resource":"*"}]}

With these permissions, the logs reached both S3 and CloudWatch Logs.

The Delivered Logs

The S3 key is logs/<runtimeId>/<YYYYmmddHHMMSS>-<UUID>.json. For the nginx task, the startup logs and the shutdown logs became separate objects.

The timestamps in the listing are JST, and the timestamps in the keys are UTC.

2026-11-13 02:58:09       3058 logs/2624f628xxxxxxxxxxxxxxxxxxxxxxxx-2531612879/20261112175744-Sq2rToP8.json
2026-11-13 03:01:03       1973 logs/2624f628xxxxxxxxxxxxxxxxxxxxxxxx-2531612879/20261112180039-Ah3pdjzR.json

Below are the first two lines of the first object. The content is JSON with one record per line, and log contains each nginx line as is. date is a field added by the S3 output plugin and is not present in CloudWatch Logs events.

{"date":"2026-11-12T17:57:35.880309Z","log":"/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration\n","stream":"stdout","time":"2026-11-12T17:57:35.880309457Z"}
{"date":"2026-11-12T17:57:35.880475Z","log":"/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/\n","stream":"stdout","time":"2026-11-12T17:57:35.880475542Z"}

The CloudWatch Logs log stream is app-<runtimeId>. The body of each event is JSON with log, stream, and time. The event timestamp matched the body's time down to the millisecond digits. Below are the timestamps (UNIX time in milliseconds) and bodies of the first two events extracted from the stream of a different nginx task.

1794506627468	{"log":"/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration\n","stream":"stdout","time":"2026-11-12T18:03:47.468205463Z"}
1794506627468	{"log":"/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/\n","stream":"stdout","time":"2026-11-12T18:03:47.468320715Z"}

Both the S3 keys and the CloudWatch Logs stream names are separated by runtimeId, so logs can be categorized per container.

Summary

With the json-file log driver now available on ECS Managed Instances, application containers can forward logs externally simply by writing them as local files in the standard Docker json-file format.

The written logs can be read directly by the daemon through the volume mount, so you can handle them with Fluent Bit or similar tools without adjusting bind mount paths or doing work as root.

When forwarding logs with FireLens, you need to place a log router container in each task and write an awsfirelens configuration in the application containers. With the combination of json-file and a daemon, you can collect the logs of all containers that specify json-file with a single daemon per instance, without placing sidecars. If configuring log forwarding middleware or managing sidecars has been a pain point, give this update a try.