Benchmarking How C++ Ring Buffer Implementations Work with SIMD in Real-Time Audio Processing

Introduction

When writing real-time audio processing, have you ever struggled with an audio callback that becomes heavy, or with crackling noise from implementations based on std::queue and std::mutex? This article focuses on the ring buffer pattern working behind the scenes, and summarizes its C++ implementation and how well SIMD works on it, together with benchmark results.

What is SIMD

SIMD (Single Instruction Multiple Data) is a mechanism that increases throughput by applying the same operation to multiple pieces of data simultaneously with a single instruction. In the previous article What is SIMD - Speeding up audio buffer gain processing with C++, we measured the effect of SIMD using gain processing on a linear buffer as the subject.

Target audience

• Those who write audio callbacks or real-time audio processing in C++ and are troubled by CPU load or crackling noise
• Those who implement audio buffer sharing between threads with std::queue or std::mutex and are unsure whether this is really fine
• Those who know the term ring buffer but want to sort out what role it plays in audio processing

References

• SIMD 入門（C 言語 SIMD 入門：目次）
• Intro to practical SIMD for graphics
• Circular Buffer - Baeldung on Computer Science

Real-time audio processing and ring buffers

Problems that tend to occur in audio callbacks

When writing real-time audio processing code, you may encounter situations like the following.

• When passing audio buffers between threads using std::queue and std::mutex, crackling noise occasionally appears
• Decoding or memory allocation is performed inside the audio callback, so the load graph becomes jagged and unstable, and in the worst case dropouts occur
• Audio playback falls apart all at once just because the network or file reading stalls for a moment

They look like separate troubles on the surface, but in many cases they boil down to the following two points.

• The processing time of the audio thread frequently exceeds the allowed time window
• The buffering and design are not sufficient to pass audio data smoothly

The ring buffer is a standard design pattern for mitigating both of these problems at the same time.

Overview of the ring buffer pattern

A ring buffer (circular buffer) is a data structure that treats a fixed-length array as a ring by connecting its ends. When you use it from the start to the end, you return to the start and continue, overwriting old data.

A ring buffer has the following characteristics.

• Memory is allocated at a fixed length and reads and writes cycle within that range, so dynamic reallocation is unnecessary
• Even when used as a queue, there is no need to repack or shift elements
• A first-in, first-out buffer can be built simply by managing the read position and the write position

In real-time processing, there are many situations where you want to avoid unpredictable delays caused by the heap or locks. For this reason, a ring buffer that completes within a predetermined region is a good fit.

The role of ring buffers in audio processing

In audio playback, there are the following two processes.

On the non-real-time side, the network or decoding may become heavy for a moment. On the audio callback side, if it does not return the specified number of samples every few milliseconds without fail, noise or dropouts occur immediately. Therefore, the samples decoded on the non-real-time side are stored in a ring buffer for the time being, and the callback side takes out a fixed size at a time from it, absorbing the timing variation between the two.

However, depending on how the ring buffer is written, even if the pattern itself is correct, CPU load can become high and it can become hard to benefit from SIMD.

Naive ring buffer implementation and compatibility with SIMD

In the previous article, we performed gain processing on a simple linear buffer and confirmed the speedup from SIMD. The linear buffer version of the processing looked like this.

void process_linear_block(float* data, int numSamples, float gain)
{
    for (int i = 0; i < numSamples; ++i) {
        data[i] *= gain;
    }
}

In contrast, when trying to do the same processing with a ring buffer, I think many people would first write code like the following.

void process_ring_naive(float* data, int capacity, int& readPos,
                        int numSamples, float gain)
{
    for (int i = 0; i < numSamples; ++i) {
        int idx = (readPos + i) % capacity;
        data[idx] *= gain;
    }
    readPos = (readPos + numSamples) % capacity;
}

It is easy to understand at a glance and it works, but from the SIMD perspective it has the following problems.

• % capacity is executed on every loop iteration, making index calculation heavy
• Array access is split at the end of the ring, making it hard to load as a large contiguous chunk
• From the compiler's point of view, each loop iteration no longer has a simple contiguous subscript, so it is hard to make it a target for auto-vectorization

As a result, even the same gain processing can end up being much slower than the linear buffer version.

Ring buffer design points from the SIMD perspective

So how can we benefit from SIMD while using a ring buffer? For example, one approach is to write the core numerical processing against a simple linear array as much as possible, and another is to confine the wrap-around handling specific to ring buffers in a thin outer wrapper.

The contiguous numSamples on the ring buffer can be divided into two chunks: the section from readPos to the end of the array, and the remaining section from the start of the array. By treating each of these as a linear block, the inner loop can remain in a simple form like process_linear_block.

void process_ring_chunked(float* data, int capacity, int& readPos,
                          int numSamples, float gain)
{
    int firstChunk  = std::min(numSamples, capacity - readPos);
    int secondChunk = numSamples - firstChunk;

    // The first chunk is [readPos, readPos + firstChunk)
    process_linear_block(data + readPos, firstChunk, gain);

    // The second chunk is the remainder from the start
    if (secondChunk > 0) {
        process_linear_block(data, secondChunk, gain);
    }

    readPos = (readPos + numSamples) % capacity;
}

Designing it this way achieves both the functionality of a ring buffer and ease of applying SIMD.

Comparing ring buffer implementations with benchmarks

From here, we will write several ring buffer implementation patterns in C++ and look at the benchmark results.

Test environment

The test environment is as follows.

• CPU: Intel(R) Core(TM) i7-11700F @ 2.50GHz
• Compiler: MSVC v143 - VS 2022 C++ x64/x86 build tools
• Language mode: C++ 17
• Build configuration: Release x64
• Examples of compile options are as follows
  • Optimization options such as /O2 /arch:AVX2 /GL /Oi
  • /Qvec-report:2 added to output the vectorization report

Since the target is in the form of sample code, absolute values will vary depending on the CPU generation and detailed options. Here, the purpose is only to confirm the relative trends between patterns.

Benchmark code overview

First, we define the ring buffer structure used in common for the tests.

struct RingBuffer {
    std::vector<float> data;
    int capacity{};
    int readPos{};
};

RingBuffer makeRingBuffer(int capacity)
{
    RingBuffer rb;
    rb.data.resize(static_cast<std::size_t>(capacity));
    rb.capacity = capacity;
    rb.readPos = 0;

    // Fill with pseudo-random initial values
    for (int i = 0; i < capacity; ++i) {
        rb.data[static_cast<std::size_t>(i)] =
            static_cast<float>((i * 13) % 100) / 100.0f;
    }

    return rb;
}

The core processing for the linear buffer is the following simple gain processing.

void process_linear_block(float* data, int numSamples, float gain)
{
    for (int i = 0; i < numSamples; ++i) {
        data[i] *= gain;
    }
}

void process_linear_entry(RingBuffer& rb, int numSamples, float gain)
{
    process_linear_block(rb.data.data(), numSamples, gain);
}

The benchmark function runs the same processing many times and obtains the average time per sample.

using Func = void(*)(RingBuffer&, int, float);

double benchmark(Func f, RingBuffer rb, int numSamples, float gain)
{
    constexpr int NumIterations = 100000;

    auto start = std::chrono::steady_clock::now();

    for (int i = 0; i < NumIterations; ++i) {
        f(rb, numSamples, gain);
    }

    auto end = std::chrono::steady_clock::now();

    std::chrono::duration<double> diff = end - start;
    double totalSec = diff.count();

    double totalSamples = static_cast<double>(NumIterations) * numSamples;
    double nsPerSample = (totalSec * 1e9) / totalSamples;

    // Take a checksum so the processing is not optimized away
    volatile float checksum = 0.0f;
    for (float v : rb.data) {
        checksum += v;
    }
    (void)checksum;

    return nsPerSample;
}

The capacity was fixed at 4096 samples (a power of 2), and block sizes of 64 / 128 / 256 / 512 samples were tried. The gain is 0.5f, but it has almost no effect on performance.

The four patterns to compare

The implementation patterns compared are the following four.

Linear buffer version

This is the simplest form, which processes only numSamples from the start of the array.

void process_linear_entry(RingBuffer& rb, int numSamples, float gain)
{
    process_linear_block(rb.data.data(), numSamples, gain);
}

This loop is the form in which SIMD auto-vectorization works most easily.

Naive ring buffer version

This is a naive implementation that processes from the read position readPos while wrapping around with % capacity.

void process_ring_naive(RingBuffer& rb, int numSamples, float gain)
{
    for (int i = 0; i < numSamples; ++i) {
        int idx = (rb.readPos + i) % rb.capacity;
        rb.data[static_cast<std::size_t>(idx)] *= gain;
    }
    rb.readPos = (rb.readPos + numSamples) % rb.capacity;
}

This is a form seen in many sample codes, but because % and index calculation are inside the loop, it is disadvantageous from the viewpoint of SIMD.

Ring buffer version using a bit mask

This is an implementation that uses a bit mask instead of %, on the premise that capacity can be a power of 2.

void process_ring_mask(RingBuffer& rb, int numSamples, float gain)
{
    int mask = rb.capacity - 1;

    for (int i = 0; i < numSamples; ++i) {
        int idx = (rb.readPos + i) & mask;
        rb.data[static_cast<std::size_t>(idx)] *= gain;
    }
    rb.readPos = (rb.readPos + numSamples) & mask;
}

It is lighter than %, but it still performs index calculation every time.

Chunk-splitting ring buffer version

This method splits the contiguous region on the ring buffer into at most two chunks and processes each as a linear buffer.

void process_ring_chunked(RingBuffer& rb, int numSamples, float gain)
{
    int firstChunk  = std::min(numSamples, rb.capacity - rb.readPos);
    int secondChunk = numSamples - firstChunk;

    // First chunk
    process_linear_block(rb.data.data() + rb.readPos, firstChunk, gain);

    // Second chunk
    if (secondChunk > 0) {
        process_linear_block(rb.data.data(), secondChunk, gain);
    }

    rb.readPos = (rb.readPos + numSamples) % rb.capacity;
}

What is called internally is the linear buffer version's process_linear_block, so that part can be expected to be SIMD-ized under the same conditions as the linear buffer.

Measurement results

As a result of running the benchmark in a Release x64 build, the processing time per sample was as follows.

capacity,4096
blockSize,linear_ns_per_sample,naive_ns_per_sample,mask_ns_per_sample,chunked_ns_per_sample
64,0.0542188,1.96087,0.973609,0.162484
128,0.0346016,1.7579,0.731492,0.124078
256,0.0304609,1.46162,0.573102,0.0586641
512,0.0266738,1.39959,0.526123,0.041375

As a table, it looks like this.

Looking at the ratios with the linear buffer version as 1.0, the trend is as follows.

The following can be seen from these results.

• The naive %-based ring buffer implementation is always dozens of times slower than the linear buffer version
• Even when capacity is made a power of 2 and replaced with a bit mask, a gap of roughly ten to twenty times remains
• On the other hand, the chunk-splitting method approaches the performance of the linear buffer especially as the block size grows, and at blockSize 512 it stays within about 1.6x

In other words, even though all patterns logically perform the same gain processing, the performance difference varies by not just one but two orders of magnitude depending on the implementation method.

Supporting evidence from the vectorization report

Next, we built with /Qvec-report:2 enabled and checked which loops the compiler vectorized. The logs of the relevant parts are as follows.

• The loop in process_linear_block

  --- Analyzing function: void __cdecl process_linear_block(float * __ptr64,int,float)
  ... info C5001: loop vectorized
  

• The loop in process_ring_naive

  --- Analyzing function: void __cdecl process_ring_naive(struct RingBuffer & __ptr64,int,float)
  ... info C5002: loop not vectorized due to reason '1200'
  

• The loop in process_ring_mask

  --- Analyzing function: void __cdecl process_ring_mask(struct RingBuffer & __ptr64,int,float)
  ... info C5002: loop not vectorized due to reason '1200'
  

It can be seen that the linear buffer version process_linear_block is auto-vectorized. On the other hand, the loops of the ring buffer versions containing % or & mask are not vectorized due to reason code 1200.

Reason code 1200
Loop-carried data dependencies prevent vectorization. Different iterations of the loop interfere with each other such that vectorizing the loop would produce wrong answers, and the auto-vectorizer cannot prove to itself that there are no such data dependencies.
\(From Vectorizer and parallelizer messages)

Meanwhile, the loop of the chunk-splitting method pushes the ring-buffer-specific processing out to a thin outer wrapper, and the part that actually scans the samples is confined inside process_linear_block. Therefore, the chunk-splitting version is also considered to be able to use the auto-vectorized loop, just like the linear buffer version.

From this, the performance difference this time can be interpreted as being mainly due to the following two causes.

• Because the index calculation containing % or & mask is inside the loop, the overhead of scalar operations becomes large
• By eliminating index calculation and moving processing toward a linear buffer, the compiler's auto-vectorization works more easily

An audio callback repeatedly processes small blocks of a few hundred samples, so this difference directly appears as a difference in CPU load and headroom.

Summary

In this article, we sorted out what role the ring buffer pattern in C++ plays in real-time audio processing, and confirmed through benchmark results and vectorization reports that how well SIMD works and the resulting performance change greatly depending on the implementation method. The naive %-based implementation and the bit mask implementation are easy to understand, but in some cases they show a performance gap of ten to several tens of times compared with the linear buffer version, and we found that a design that moves the core processing onto a linear array, like the chunk-splitting method, is effective. If you feel your audio callback is struggling, review the ring buffer implementation and loop structure, and use these results as a reference to consider whether there is room to draw out the benefits of SIMD.