Memory usage control was added to Mountpoint for Amazon S3, so I checked the number of files that can be opened simultaneously for writing

Introduction

On August 26, 2026, Mountpoint for Amazon S3 added --memory-target, which specifies the target value for the total process memory usage in MiB.

https://aws.amazon.com/about-aws/whats-new/2026/08/mountpoint-for-S3-adds-memory-usage-controls

https://dev.classmethod.jp/articles/mountpoint-for-S3-adds-memory-usage-controls/

https://github.com/awslabs/mountpoint-s3/blob/main/doc/CONFIGURATION.md

In the What's New post, the previous behavior is described as follows:

Previously, Mountpoint's memory usage could expand over time based on usage patterns, potentially causing performance or stability issues when competing with other memory-intensive applications.

What changed is not just the ability to specify a value. Even when no value is specified, a target value determined from installed memory or cgroup limits is applied. The method of determining the target value from cgroup limits is also described in the official documentation, but what was verified this time is the case where the target value is determined from installed memory.

The behavior under memory pressure is described in the CHANGELOG as follows:

This target is not a guaranteed limit but Mountpoint manages the memory available for data buffers to stay within the target: under memory pressure it slows down I/O, reclaims buffers it no longer needs and reduces prefetching.

--memory-target was added in v1.24.0 (released August 24, 2026), and this verification was also performed with the same version. The maximum number of files that can be open simultaneously for writing is classified in the CHANGELOG as a Breaking change rather than a new feature. Since the limit applies regardless of whether --memory-target is specified, the behavior changes as soon as the version is upgraded.

Mountpoint now limits how many files can be open for writing at the same time, derived from --memory-target and --write-part-size. Once the limit is reached, opening a file for writing fails with ENOMEM until an existing write file handle is closed.

In this article, we verified the maximum number of files that can be open simultaneously for writing (hereinafter referred to as the limit count) determined by each target value, and the behavior when the limit count is exceeded.

Verification Details

The two things verified were remounting with different target values and continuously opening files for writing in Python without closing them.

Verification Environment

In v1.24.0, the limit count is calculated from the target value and the read part size and write part size. Since no cgroup memory limit is applied in the verification environment, the default target value when unspecified is determined from installed memory.

• EC2 t3.medium (free -m total is 3835 MiB), Amazon Linux 2023
• Mountpoint is RPM package version 1.24.0
• Region is ap-northeast-1, target is one general-purpose bucket
• Default values for both --read-part-size and --write-part-size are 8388608 bytes (8 MiB)

The relevant portion of mount-s3 --help is as follows:

      --memory-target <MiB>
          Target for Mountpoint's total memory usage, in MiB.

          Mountpoint manages the memory used to buffer reads and writes to stay within this target but it is
          not a guaranteed limit. Lowering it may reduce throughput and increase latency.

      --read-part-size <SIZE>
          Part size for GET in bytes [default: 8388608]

      --write-part-size <SIZE>
          Part size for multi-part PUT in bytes [default: 8388608]

Limit Count per Target Value

The operation of mounting with a different target value, checking the startup log, and then unmounting was repeated 4 times. Mounting was done with the following command, and logs were retrieved.

mount-s3 amzn-s3-demo-bucket /mnt/s3 --memory-target 512
journalctl -e SYSLOG_IDENTIFIER=mount-s3 --no-pager -o cat | grep write_handle_limiter

From the output of 4 patterns, only the successful mount line and the relevant log line are extracted.

===== memory-target=512 =====
bucket amzn-s3-demo-bucket is mounted at /mnt/s3
[INFO] ThreadId(01) mountpoint_s3_fs::memory::write_handle_limiter: max files concurrently open for write: 47 (memory target 512 MiB, write part size 8 MiB)
===== memory-target=1024 =====
bucket amzn-s3-demo-bucket is mounted at /mnt/s3
[INFO] ThreadId(01) mountpoint_s3_fs::memory::write_handle_limiter: max files concurrently open for write: 111 (memory target 1024 MiB, write part size 8 MiB)
===== memory-target=2048 =====
bucket amzn-s3-demo-bucket is mounted at /mnt/s3
[INFO] ThreadId(01) mountpoint_s3_fs::memory::write_handle_limiter: max files concurrently open for write: 223 (memory target 2048 MiB, write part size 8 MiB)
===== memory-target=none =====
bucket amzn-s3-demo-bucket is mounted at /mnt/s3
[INFO] ThreadId(01) mountpoint_s3_fs::memory::write_handle_limiter: max files concurrently open for write: 397 (memory target 3644 MiB, write part size 8 MiB)

The default target value of 3644 MiB is approximately 95% of the installed memory of 3835 MiB. This matches the description in the official documentation of "95% of total memory or cgroup limit."

The limit count for all 4 patterns matched the calculation formula in the official documentation. The formula is described in "Maximum number of files open for writing" in CONFIGURATION.md.

https://github.com/awslabs/mountpoint-s3/blob/main/doc/CONFIGURATION.md

cap = (memory_target − max(128 MiB, memory_target / 8) − read_part_size) / write_part_size

When the target value is 512 MiB, Mountpoint reserves 128 MiB for overhead and 8 MiB for reading. The amount available for writing is 376 MiB, and the limit is (512 − 128 − 8) / 8 = 47 files. Fractions resulting from the calculation are truncated.

Doubling the target value does not double the limit count. The overhead portion and the read portion are deducted first. Furthermore, when the target value exceeds 1024 MiB, the overhead calculation switches to memory_target / 8, increasing the amount deducted (256 MiB for 2048 MiB).

Since the denominator of the formula is the write part size, reducing --write-part-size will increase the limit count. However, since multipart upload has a maximum of 10,000 parts, the maximum size of a single object also decreases. Since the write part size was not changed in this verification, please verify separately if you plan to change it.

Open Failure When Limit Is Exceeded

Mounted with a target value of 512 MiB (limit of 47 files), and to reliably exceed the limit, files were opened for writing in Python without closing them, increasing up to 60 files.

[INFO] ThreadId(01) mountpoint_s3_fs::memory::write_handle_limiter: max files concurrently open for write: 47 (memory target 512 MiB, write part size 8 MiB)
opened successfully: 47
failed at file #48: errno=12 (ENOMEM) Cannot allocate memory

As per the limit count shown in the startup log, opening up to 47 files succeeded, and opening the 48th file failed with ENOMEM (Cannot allocate memory).

Subsequently, one open handle was closed and a re-open was attempted.

opened=47, next open failed: errno=12 (ENOMEM)
after closing 1 handle, waited 0s cumulative: open failed errno=12 (ENOMEM)
after closing 1 handle, waited 1s cumulative: open SUCCESS

In this environment, re-opening immediately after closing a handle failed, but the retry after 1 second succeeded. The CHANGELOG states that opening for writing fails until an existing write handle is closed. However, a slot does not necessarily become available immediately after closing.

When the limit count is reached, open() fails rather than slowing down. Applications where the number of simultaneously open files may reach the limit count need to either control the number of files opened or handle ENOMEM. When implementing retries, consider incorporating a wait period rather than repeating immediate retries.

Summary

Previously, Mountpoint's memory usage could increase over time, but this update introduced controls to keep memory usage within the target value. Even without specifying a target value, the target value is determined from installed memory or cgroup limits. The target value is applied simply by upgrading the version. If you have been experiencing memory shortages or OOM issues with Mountpoint, please try upgrading the version and adjusting --memory-target.

On the other hand, there is a trade-off: the limit count is determined from the target value, and once the limit count is reached, subsequent opens fail with ENOMEM. First, check the limit count for your environment in the startup log. The limit count can be increased by raising the target value, but since a buffer equal to the part size is reserved for each file being written, memory corresponding to the number of simultaneously open files is required. If you continue to address this solely by increasing infrastructure memory, the memory allocated to EC2 or Fargate will increase, adding to operational burden and AWS usage costs. As application-side countermeasures, please consider limiting the number of files open simultaneously or performing writes using the AWS SDK or AWS CLI.