I Benchmarked Amazon S3's New Checksum Algorithm on the Latest-Generation Intel/AMD/Graviton4 EC2 Instances

I Benchmarked Amazon S3's New Checksum Algorithm on the Latest-Generation Intel/AMD/Graviton4 EC2 Instances

S3 now supports 10 checksum algorithms. We outline the new XXHash3, XXHash64, XXHash128, SHA-512 and MD5, and measure checksum speed and S3 transfer speed on c8i (Intel), c8a (AMD) and c8g (Graviton4) 4xlarge instances.
2026.04.24

This page has been translated by machine translation. View original

On April 23, 2026, Amazon S3 added support for five new checksum algorithms, bringing the total to 10.

https://aws.amazon.com/jp/about-aws/whats-new/2026/04/s3-five-additional-checksum-algorithms/

In this article, we examine how algorithm choice and instance type affect performance, using measured benchmarks on the latest-generation EC2 instances across three architectures (Intel Emerald Rapids / AMD EPYC / AWS Graviton4).

What Changed

Category Previously Newly Added
CRC family CRC32, CRC32C, CRC64NVME ―
XXHash family ― XXHash3, XXHash64, XXHash128
SHA family SHA-1, SHA-256 SHA-512
Other ― MD5

Available in 37 Regions. The default remains CRC64NVME as before.

Algorithm Characteristics

Non-cryptographic hashes (fast, suited to integrity checks)

Algorithm Bit length Characteristics
CRC32 / CRC32C 32bit Lightweight. Supports hardware acceleration
CRC64NVME 64bit S3 default. Highly optimized in the AWS CRT
XXHash64 64bit Fast general-purpose hash. Widely used in data pipelines
XXHash3 64bit Latest version of XXHash. SIMD-optimized
XXHash128 128bit 128-bit version of XXHash3. Even lower collision probability

Cryptographic hashes (suited to security and compliance)

Algorithm Bit length Characteristics
MD5 128bit Collision attacks have been demonstrated. For legacy compatibility
SHA-1 160bit Collisions have been demonstrated. Deprecated
SHA-256 256bit Currently secure. Widely used
SHA-512 512bit Longer hash value than SHA-256. Can be more efficient on 64bit CPUs

If the goal is detecting data corruption in transit, non-cryptographic hashes are sufficient. Protection against tampering with data on S3 is fundamentally ensured not by checksums but by IAM policies and S3 Object Lock.

Measured Benchmarks

Test Environment

We tested three instance types in the Tokyo Region (ap-northeast-1).

Item c8i.4xlarge c8a.4xlarge c8g.4xlarge
CPU Intel Xeon 6975P-C (Emerald Rapids) AMD EPYC 9R45 AWS Graviton4
Architecture x86_64 x86_64 aarch64
vCPU / Physical cores 16 / 8 (with SMT) 16 / 16 (no SMT) 16 / 16 (no SMT)
Memory 32GB DDR5 32GB DDR5 32GB DDR5
Network bandwidth Up to 12.5 Gbps Up to 15 Gbps Up to 12.5 Gbps
On-demand price $0.4718/h $0.5426/h $0.4003/h

The test data (5GB) was placed on tmpfs (a RAM disk) to eliminate storage I/O as a bottleneck. Checksum computation speed was measured single-threaded on in-memory data using Python's xxhash (C extension) and hashlib, and S3 transfer speed was measured with aws s3 cp from AWS CLI 2.34.35. All figures are averages of three runs.

Checksum computation speed (5GB, single-threaded, average of 3 runs)

Algorithm c8i (Intel) c8a (AMD) c8g (Graviton4)
XXHash3_64 14.8 GB/s 36.7 GB/s 23.4 GB/s
XXHash3_128 14.8 GB/s 36.8 GB/s 23.3 GB/s
XXHash64 7.0 GB/s 22.0 GB/s 18.3 GB/s
SHA-256 1.7 GB/s 2.1 GB/s 1.7 GB/s
CRC32 1.3 GB/s 1.7 GB/s 1.0 GB/s
SHA-512 0.8 GB/s 1.2 GB/s 1.0 GB/s
MD5 0.8 GB/s 0.9 GB/s 0.6 GB/s

Variation across the three runs was extremely small (coefficient of variation under 1%), so the results are highly reproducible.

AMD was the fastest for every algorithm. With XXHash3 it reached 36.7 GB/s, 2.5 times Intel and 1.6 times Graviton4. The AVX-512 optimizations in the xxhash library are thought to be taking effect. Graviton4 also significantly outperformed Intel, with 23.4 GB/s for XXHash3 and 18.3 GB/s for XXHash64, showing that the ARM NEON optimizations are working effectively.

S3 transfer speed (5GB, tmpfs → S3, average of 3 runs)

This is the upload throughput of aws s3 cp. A 5GB file is processed internally as a multipart upload, with multiple parts transferred in parallel.

Algorithm c8i (Intel) c8a (AMD) c8g (Graviton4)
CRC32C 507 MB/s 494 MB/s 425 MB/s
CRC32 470 MB/s 405 MB/s 413 MB/s
XXHASH3 461 MB/s 411 MB/s 428 MB/s
SHA-256 458 MB/s 382 MB/s 450 MB/s
CRC64NVME 455 MB/s 421 MB/s 421 MB/s
XXHASH128 432 MB/s 480 MB/s 443 MB/s
XXHASH64 423 MB/s 434 MB/s 480 MB/s
SHA-1 403 MB/s 469 MB/s 491 MB/s
SHA-512 373 MB/s 355 MB/s 395 MB/s

In checksum computation speed, there was up to a 47x difference between algorithms (XXHash3 vs MD5), but in S3 transfer speed all three environments fell within the 350–510 MB/s range, and the differences between algorithms shrank considerably.

Discussion

The impact of algorithm choice on S3 transfers is limited

With network bandwidth secured by 4xlarge instances and storage I/O eliminated by tmpfs, all three environments recorded throughput of around 400–500 MB/s regardless of algorithm. Even though checksum computation speed differs by tens of times, the effect on total S3 transfer throughput is small. This is because network I/O and SDK overhead dominate, and checksum computation is only a small part of the whole.

The differences among the three environments are small

In S3 transfer speed, there was no clear winner among the three architectures. However, differences were seen in run-to-run variation (standard deviation): c8i was the largest (average SD 79.5 MB/s) and c8a the smallest (average SD 41.4 MB/s).

In addition, when we ran the same tests beforehand on .large (2 vCPU) instances, c8i.large (1 physical core / SMT) showed a drop to 141 MB/s for some algorithms. c8a.large / c8g.large (2 physical cores) showed no such drop, and it was also resolved on c8i when changed to 4xlarge, so a ceiling on parallel processing due to a shortage of physical cores is thought to be one factor. On .large class instances, EBS throughput and network bandwidth constraints also overlap, so it cannot be attributed to a single cause, but in environments that make heavy use of multipart uploads, it is important to choose an instance size with ample physical cores.

Architecture differences are pronounced in checksum computation speed

On the other hand, in pure checksum computation speed, AMD (c8a) was overwhelmingly faster. It is expected to be effective for workloads with heavy checksum computation, and for multipart uploads in environments with ample network bandwidth (25 Gbps or more).

Guidelines for Choosing an Architecture

Priority Recommendation
Checksum computation speed c8a (AMD) is fastest
S3 transfer speed All three are equivalent (small differences)
Cost performance c8g (Graviton4) is cheapest

c8g is 15% cheaper than c8i and 26% cheaper than c8a. Given that there is no large difference in S3 transfer speed, c8g (Graviton4) is suitable if cost is the priority, and c8a (AMD) if performance is the priority.

Use case Recommendation Reason
General use CRC64NVME (default) No change needed. Sufficient speed and accuracy
Integration with existing pipelines XXHash64 / XXHash3 Compatibility with Spark, ClickHouse, etc.
Large-scale data lakes XXHash128 128bit gives an extremely low collision risk
Compliance requirements SHA-256 / SHA-512 FIPS compliance; finance, healthcare, government
Legacy system compatibility MD5 Only for specifying precomputed values in headers

Notes

CLI / SDK versions

Using the new algorithms requires updating the CLI / SDK. We confirmed support with CLI 2.34.35.

aws --version
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip && sudo ./aws/install --update

The existing algorithms (CRC32, CRC32C, SHA-1, SHA-256) have been supported for some time. CRC64NVME was added in early 2025, so it cannot be used with versions earlier than that.

MD5

It is supported at the S3 API level, but automatic computation via --checksum-algorithm MD5 in the SDK / CLI is not supported. You need to specify a precomputed value with the x-amz-checksum-md5 header. The MD5 of a multipart upload can be computed afterward with Compute checksum in S3 Batch Operations.

Directory buckets

Specifying a checksum algorithm is not supported for directory buckets (S3 Express One Zone).

Summary

With this announcement, S3's checksum algorithms have been expanded from the previous five to ten. The addition of the XXHash family makes integration with existing data pipelines easier, and the addition of SHA-512 and MD5 broadens the options for compliance and legacy compatibility.

For many workloads, simply using the default CRC64NVME provides sufficient speed and integrity. There is no need to deliberately change the algorithm.

On the other hand, this benchmark also confirmed that the combination of CPU architecture and algorithm produces clear differences in checksum computation speed. For workloads where integration with existing data pipelines or faster computation is desired, please try the newly added algorithms.

Share this article

AWSのお困り事はクラスメソッドへ