I compared auto-vectorization with std::simd in C++26 using GCC 16 for audio DSP

Introduction

When accelerating signal processing in C++, you may wonder whether to leave ordinary loops to the compiler or write SIMD processing using CPU-specific intrinsics. Ordinary loops are easy to read, but there is no guarantee they will be vectorized in the intended direction. With intrinsics you can control processing in detail, but you need separate implementations and maintenance for each instruction set.

GCC 16 experimentally added std::simd, the C++26 data-parallel type, to libstdc++. It is now possible to write SIMD processing with standard C++ types and operations. This is a middle ground that is neither of the previous options. I tried it right away.

As a result, for a simple gain, the performance of auto-vectorization and std::simd were close. On the other hand, for a 64-tap FIR, explicitly specifying the direction of parallelization with std::simd yielded a 3.50–14.14x speedup over scalar. (However, these results are from this particular verification environment.) In this article, I would like to think about criteria for choosing between ordinary C++ and std::simd.

What is std::simd

std::simd is a C++26 API for applying the same operation to multiple elements of the same type. You specify the element type as in std::simd::vec<float>, and can describe processing on multiple loaded values using ordinary arithmetic operators.

In this article, I used std::simd::vec<float> with the element count omitted. Under the compilation conditions used here, the number of float elements was 4 for SSE2, 8 for AVX2, and 16 for AVX-512.

Verification Environment

• Amazon EC2 c6i.large
• Intel Xeon Platinum 8375C
• GCC 16.2.0
• C++26, -O3
• 48 kHz, 64, 128, 256, 512 frames

Target Audience

• Those considering SIMD optimization for audio DSP in C++
• Those who want to know how to choose between auto-vectorization and std::simd
• Those who want to know the extent of what is possible with standard C++ before writing CPU-specific intrinsics

References

• GCC 16 Changes
• GCC 16.2 Release
• Patch implementing std::simd in GCC 16's libstdc++
• P1928R15 adding std::simd to C++26

Verification Method

As operations with different characteristics, I compared a gain that multiplies all samples by the same coefficient, and a 64-tap FIR that performs multiply-accumulate operations. For each, I prepared a scalar version, an auto-vectorized version, and a std::simd version. For the FIR, I also added an ordinary C++ version that processes multiple frames in the same order as the std::simd version.

Each implementation was compiled for SSE2, AVX2, and AVX-512, and I observed the correctness of the output, the instructions generated by the compiler, and the processing time. Rather than judging the success of SIMD optimization by processing time alone, I also examined the compiler's optimization report and disassembly.

Implementation

First, I implemented a gain that multiplies consecutive samples by the same coefficient. The auto-vectorized version compiles an ordinary loop with -O3.

void gain_auto(const float* input, float* output, std::size_t frames,
               float gain) {
  for (std::size_t frame = 0; frame < frames; ++frame) {
    output[frame] = input[frame] * gain;
  }
}

The std::simd version loads multiple samples, multiplies them as a vector, and then stores them. The tail that cannot be divided evenly by the vector width is handled with an ordinary loop.

using FloatVec = std::simd::vec<float>;

void gain_simd(const float* input, float* output, std::size_t frames,
               float gain) {
  constexpr std::size_t width = FloatVec::size();
  std::size_t frame = 0;
  const FloatVec gains(gain);

  for (; frame + width <= frames; frame += width) {
    const auto samples = std::simd::unchecked_load<FloatVec>(
        std::span(input + frame, width));
    std::simd::unchecked_store(samples * gains,
                               std::span(output + frame, width));
  }

  for (; frame < frames; ++frame) {
    output[frame] = input[frame] * gain;
  }
}

Next, I implemented a 64-tap FIR as a computationally intensive operation. The ordinary loop accumulates multiply-add operations in the tap direction, one frame at a time.

for (std::size_t frame = 0; frame < frames; ++frame) {
  float accumulator = 0.0F;
  for (std::size_t tap = 0; tap < taps; ++tap) {
    accumulator += input[frame + taps - 1 - tap] * coefficients[tap];
  }
  output[frame] = accumulator;
}

The std::simd version holds independent multiply-accumulate results for multiple frames in a single vector. It accumulates the vector until the tap-direction loop finishes, then stores to the output.

for (; frame + width <= frames; frame += width) {
  FloatVec accumulator(0.0F);
  for (std::size_t tap = 0; tap < taps; ++tap) {
    const auto samples = std::simd::unchecked_load<FloatVec>(
        std::span(input + frame + taps - 1 - tap, width));
    accumulator += samples * FloatVec(coefficients[tap]);
  }
  std::simd::unchecked_store(accumulator,
                             std::span(output + frame, width));
}

With just these two implementations, in addition to the API difference, a difference in loop structure—tap direction versus frame direction—is also included. Therefore, I also added an ordinary C++ version that holds accumulated values for multiple frames in a std::array<float, N> and processes them in the same order as the std::simd version. N was compiled as 4 for SSE2, 8 for AVX2, and 16 for AVX-512.

The scalar version was compiled with auto-vectorization disabled.

Each condition was pinned to logical CPU 0, and after adjusting the number of internal iterations so that one trial took 100 ms or more, measurements were taken 30 times with the order shuffled. The reported figures are the median throughput values obtained by dividing the batch execution time by the number of internal iterations.

Results

Since the trend was consistent across all block sizes, results for 128 frames are shown as a representative. The multiplier is the speedup ratio relative to the scalar version with the same instruction set.

The gain results are as follows.

The FIR results are as follows. "Ordinary C++ expressing frame parallelism" is a controlled experiment using arrays of fixed lane counts to represent accumulated values for multiple frames.

The speedup of the std::simd FIR version across 64–512 frames was 3.50–3.52x for SSE2, 7.62–7.82x for AVX2, and 14.04–14.14x for AVX-512.

In the compiler's optimization report, the auto-vectorized gain version was converted into 16-, 32-, and 64-byte vectors corresponding to each instruction set. For the ordinary C++ version expressing frame parallelism, it was the tap loop, not the lane loop written in the source code, that was vectorized to the same width. (Confirmed in disassembly as well: mulps for SSE2, YMM registers for AVX2, and vmulps using ZMM registers for AVX-512.)

On the other hand, the ordinary C++ version expressing frame parallelism contained many input reorderings and temporary stack spills for AVX2 and AVX-512. Even after aligning the traversal order in the source code, the vectorization direction and instruction sequence chosen by the compiler did not match those of the std::simd version.

Regarding errors, I compared all outputs against a reference FIR that converts inputs and coefficients to double before multiplication. The maximum absolute error was 6.2434349412665568e-8.

Discussion

For simple processing like gain, I think it is sufficient to first check whether an ordinary loop has been auto-vectorized, rather than immediately replacing it with explicit SIMD code. On the other hand, for multiply-accumulate operations like FIR, it seems important to communicate to the compiler which direction contains independent computations.

Note that the speed difference observed here cannot be attributed solely to the std::simd API. This is because the ordinary C++ version used fixed lane counts per ISA, and the generated instructions did not match those of the std::simd version. Also, no performance comparison with CPU-specific intrinsics was conducted. Therefore, it cannot be concluded that intrinsics can be fully replaced.

Within the scope of this investigation, it was possible to utilize the target SIMD width without writing intrinsics and to achieve a clear performance improvement for FIR. It seems best to first verify ordinary C++ and auto-vectorization, and then consider std::simd for operations where the intended parallelism cannot be expressed.

Summary

Using C++26 std::simd in GCC 16.2, I compared audio DSP gain and 64-tap FIR. For simple gain, the performance of auto-vectorization and std::simd were close, and for FIR, the std::simd version that explicitly expressed parallelism in the frame direction achieved a 3.50–14.14x speedup over scalar.

There does not appear to be a need to rewrite operations where an ordinary loop is sufficient. On the other hand, for cases where auto-vectorization does not produce the intended result, I think it is worth trying std::simd while checking the generated instructions and the support status for the target CPU. I hope this article serves as useful information when choosing how to write SIMD in audio DSP with C++.