Aurora DSQL Python Connector Has Been Released, So I Measured and Compared Performance on Lambda (psycopg2 vs asyncpg)

Aurora DSQL Python Connector Has Been Released, So I Measured and Compared Performance on Lambda (psycopg2 vs asyncpg)

Quick test of Aurora DSQL's official Python connector on Lambda: psycopg2 vs asyncpg performance, x86 vs Graviton measurements, connection reuse to cut overhead, and ways to avoid connection rate limits in production.
2026.02.20

This page has been translated by machine translation. View original

On February 19, 2026, official connectors for Aurora DSQL were released for three languages: Go, Python, and Node.js.

https://aws.amazon.com/jp/about-aws/whats-new/2026/02/aurora-dsql-launches-go-python-nodejs-connectors/

https://dev.classmethod.jp/articles/aurora-dsql-launches-go-python-nodejs-connectors/

The Python package aurora-dsql-python-connector supports three drivers (psycopg / psycopg2 / asyncpg) and is a transparent authentication layer that automates IAM authentication token generation when connecting to DSQL.

Until now, connecting to DSQL from Python required creating a DSQL client with boto3, generating a token with generate_db_connect_admin_auth_token, and passing it as the password of psycopg2.connect.

import psycopg2
import boto3

def get_dsql_connection():
    client = boto3.client('dsql', region_name='us-west-2')
    token = client.generate_db_connect_admin_auth_token(DSQL_ENDPOINT, 'us-west-2')
    return psycopg2.connect(
        host=DSQL_ENDPOINT, port=5432, database='postgres',
        user='admin', password=token, sslmode='require'
    )

With the new connector, this becomes a single line.

import aurora_dsql_psycopg2 as dsql

conn = dsql.connect(host=DSQL_ENDPOINT, region='us-west-2', user='admin')

The boto3 call, token generation, and sslmode specification are all no longer needed. With asyncpg, you can likewise use it simply by importing aurora_dsql_asyncpg.

Motivation: Why I Tested on Lambda

Following the release of the new connector, I wanted to understand the improvement it would bring to Lambda workloads that use psycopg2. My current environment had three issues.

First, it depends on an x86 Lambda layer for psycopg2-binary, so it cannot migrate to arm64 (Graviton). Second, some Lambdas run for up to 230 minutes via Step Functions, so the token can expire because the run exceeds DSQL's maximum connection lifetime (60 minutes). Third, the same get_dsql_connection() is copy-pasted across multiple files, which hurts maintainability.

Since the new connector also supports asyncpg, I decided to measure the improvement, including a driver change, in an actual Lambda environment.

Note that the new connector also supports psycopg (v3), but since this test assumes migrating an existing environment, I limited it to psycopg2 and asyncpg.

Test Method

Test Setup

I created two Lambda functions with SAM: one for x86_64 and one for arm64 (Graviton), each attached to a layer containing psycopg2 and asyncpg. The runtime is Python 3.12, memory is 256 MB, and the DSQL cluster is in us-west-2, the same region as Lambda. After testing, you can delete the whole stack with sam delete.

Test Patterns

I measured the following six patterns.

# Pattern Architecture Processing per iteration
1 psycopg2 (connect every time) x86_64 Automatic token generation → TCP → TLS → PG authentication → SQL → disconnect
2 asyncpg (connect every time) x86_64 Same as above (driver is asyncpg)
3 asyncpg (connection reuse) x86_64 Connection established only the first time; the same connection is reused afterward
4 psycopg2 (connect every time) arm64 Pattern 1 run on Graviton
5 asyncpg (connect every time) arm64 Pattern 2 run on Graviton
6 asyncpg (connection reuse) arm64 Pattern 3 run on Graviton

The "connect every time" patterns are not intended for production use; they are deliberate anti-patterns for accurately measuring the overhead of a single connection establishment.

Measurement Method

I used SELECT 1 as the query to minimize processing on the DB side. Within a single Lambda invoke, I ran 30 iterations and measured "connection establishment time" and "query execution time" separately in each iteration using time.perf_counter(). I invoked each architecture multiple times (x86: 6 times, arm64: 3 times) and used the median.

Measurements were taken in the in-handler loop during warm starts and do not include Init Duration. "Connection time" is the sum of IAM token generation + TCP connection + TLS handshake + PostgreSQL authentication.

Reference Values: Cold Start

x86_64 arm64
Init Duration 149ms 123ms
First connection approx. 1.0–1.5 seconds approx. 1.0 seconds
Total cold start approx. 1.2–1.7 seconds approx. 1.1–1.2 seconds

There was ample margin relative to API Gateway's default timeout (29 seconds).

Below is a conceptual diagram of the Lambda lifecycle and connection establishment timing.

What I measured in this test is the in-handler loop during this warm start.

Results

x86_64 Lambda (median, 6 runs)

Pattern Connection (avg) Query (avg)
psycopg2 (connect every time) 873.0ms 12.95ms
asyncpg (connect every time) 807.7ms 7.53ms
asyncpg (connection reuse) 0.02ms 2.50ms

arm64 (Graviton) Lambda (median, 3 runs)

Pattern Connection (avg) Query (avg)
psycopg2 (connect every time) 673.6ms 12.56ms
asyncpg (connect every time) 633.1ms 6.67ms
asyncpg (connection reuse) 0.02ms 2.44ms

Discussion

Driver Comparison: Small Difference in Connection, Clear Difference in Queries

The difference in connection establishment time between drivers was small: 7% on x86 and 6% on arm64. Most of the connection time is taken up by IAM token generation (STS calls via boto3), and the driver-specific processing is only a small part of it.

On the other hand, there was a clear difference in query execution time. asyncpg was 42% faster on x86 and 47% faster on arm64. This is thanks to PostgreSQL's binary protocol and result parsing with Cython.

However, in the Lambda environment, connection establishment (600–870ms) accounts for most of the processing time, so a difference of a few milliseconds in queries is within the margin of error relative to the total.

Connection Reuse: Connection Cost Reduced to Zero

In the connection reuse pattern, obtaining a connection dropped from 600–870ms to 0.02ms. Because an already established connection is used as is, neither TCP connection nor token generation occurs.

I implemented it with asyncpg.create_pool(min_size=1), but since Lambda processes one request at a time serially, the effect is essentially the same as keeping a single connection in a global variable. In Lambda, it is simpler to hold the connection in global scope as shown below, without using a pool library.

import aurora_dsql_asyncpg as dsql

conn = None

async def handler(event, context):
    global conn
    if conn is None or conn.is_closed():
        conn = await dsql.connect(host=DSQL_ENDPOINT, region='us-west-2', user='admin')
    row = await conn.fetchrow("SELECT ...")
    return row

Because global variables are preserved during warm starts, connection establishment can be skipped on the second and subsequent invokes.

Architecture Comparison: arm64 Was 22–23% Faster at Connection Establishment

The reduction was 23% for psycopg2 (873→674ms) and 22% for asyncpg (808→633ms). It is thought that Graviton processed CPU-bound work such as signature calculation for IAM token generation faster. Query execution time was nearly the same because network I/O dominates it.

Operational Notes for Lambda × DSQL

Connection Rate Limit: The Biggest Pitfall

DSQL has a hard limit of 100 connections per second on the connection rate. When Lambda scales out under spiky access, each container tries to establish connections all at once, and the 100/sec limit is reached instantly. Since DSQL has no intermediary like RDS Proxy, control on the Lambda side is essential.

Possible countermeasures include limiting concurrency with Reserved Concurrency, retrying with exponential backoff on connection errors, and absorbing spikes by placing SQS or EventBridge in between.

Here is a summary of the main DSQL constraints and their impact on Lambda.

DSQL constraint Value Impact on Lambda
Connection rate 100 connections/sec Most dangerous during simultaneous scale-out
Maximum connection lifetime 60 minutes Reconnection required for long-running batches
Maximum transaction duration 5 minutes Large INSERTs must be split
Maximum rows per transaction 3,000 rows Affects batch size design
Maximum connections 10,000 Needs attention in large-scale Lambda operations

Should You Adopt asyncpg?

asyncpg's query execution was 42–47% faster, but the decision to adopt it in Lambda is not determined by speed alone.

asyncpg is suitable when you want to break the dependency on the x86 psycopg2 layer by migrating to arm64, when you run multiple queries in parallel with asyncio.gather, or when connection reuse is important in batch processing.

On the other hand, for simple processing with one or two queries per request, the psycopg2 connector is sufficient. No SQL rewrites are needed and the adoption cost is low, and the benefits of eliminating manual token generation and reducing boilerplate are gained on their own.

Notes When Migrating to asyncpg

Adopting asyncpg requires some code changes.

SQL placeholders change from %s to $1, $2, ..., and parameters are passed as positional arguments rather than a tuple.

# psycopg2
cur.execute("SELECT * FROM t WHERE id = %s AND status = %s", (article_id, 'pending'))
row = cur.fetchone()

# asyncpg
row = await conn.fetchrow("SELECT * FROM t WHERE id = $1 AND status = $2", article_id, 'pending')

Method names change from fetchone → fetchrow and fetchall → fetch.

Transaction handling also differs. psycopg2 defaults to autocommit=False and starts transactions implicitly, whereas asyncpg defaults to autocommit. When using transactions explicitly, wrap them in async with conn.transaction():.

# psycopg2
conn.commit()

# asyncpg
async with conn.transaction():
    await conn.execute("INSERT ...")
    await conn.execute("UPDATE ...")

Since DSQL's maximum transaction duration is 5 minutes, asyncpg's autocommit default is less likely to run into this limit and errs on the safe side.

Summary

Through testing on Lambda, I was able to confirm the effect of the new connector.

The cost of connection establishment is large in the Lambda environment at 600–870ms, and it can be reduced to zero with connection reuse. On arm64, connection establishment is 22–23% faster than on x86, and a 20% cost reduction can also be expected. Because the connector automatically generates IAM tokens, the risk of authentication expiring in long-running batches is also eliminated.

If you are looking for a way to connect to DSQL, I think it is best to start with the psycopg2 connector, which can be introduced without changing existing code. If arm64 migration or batch processing optimization is on the horizon, consider stepping up to the asyncpg connector.

Note that these measurements are warm-start values in the same region (us-west-2), and are medians of 30 iterations × multiple invokes. Results will vary depending on the network environment and the load on the DSQL cluster.

Reference: Test Code

Here are the SAM template and Lambda handler used for the test. You can deploy with sam build && sam deploy, run with sam remote invoke, and tear down with sam delete.

template.yaml
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: Aurora DSQL Connector Benchmark on Lambda

Parameters:
  DsqlEndpoint:
    Type: String
  DsqlRegion:
    Type: String
    Default: us-west-2
  BenchIterations:
    Type: String
    Default: '30'

Globals:
  Function:
    Runtime: python3.12
    Timeout: 300
    MemorySize: 256
    Environment:
      Variables:
        DSQL_ENDPOINT: !Ref DsqlEndpoint
        DSQL_REGION: !Ref DsqlRegion
        BENCH_ITERATIONS: !Ref BenchIterations

Resources:
  Psycopg2Layer:
    Type: AWS::Serverless::LayerVersion
    Properties:
      LayerName: dsql-bench-psycopg2-x86
      ContentUri: layers/psycopg2-x86/
      CompatibleRuntimes: [python3.12]
      CompatibleArchitectures: [x86_64]
    Metadata:
      BuildMethod: python3.12
      BuildArchitecture: x86_64

  AsyncpgLayer:
    Type: AWS::Serverless::LayerVersion
    Properties:
      LayerName: dsql-bench-asyncpg-arm64
      ContentUri: layers/asyncpg-arm64/
      CompatibleRuntimes: [python3.12]
      CompatibleArchitectures: [arm64]
    Metadata:
      BuildMethod: python3.12
      BuildArchitecture: arm64

  BenchX86:
    Type: AWS::Serverless::Function
    Properties:
      FunctionName: dsql-bench-x86
      Handler: handler.lambda_handler
      CodeUri: src/
      Architectures: [x86_64]
      Layers: [!Ref Psycopg2Layer]
      Policies:
        - Statement:
            - Effect: Allow
              Action: dsql:DbConnectAdmin
              Resource: !Sub arn:aws:dsql:${DsqlRegion}:${AWS::AccountId}:cluster/*

  BenchArm64:
    Type: AWS::Serverless::Function
    Properties:
      FunctionName: dsql-bench-arm64
      Handler: handler.lambda_handler
      CodeUri: src/
      Architectures: [arm64]
      Layers: [!Ref AsyncpgLayer]
      Policies:
        - Statement:
            - Effect: Allow
              Action: dsql:DbConnectAdmin
              Resource: !Sub arn:aws:dsql:${DsqlRegion}:${AWS::AccountId}:cluster/*
handler.py
import asyncio
import json
import os
import platform
import statistics
import time

DSQL_ENDPOINT = os.environ['DSQL_ENDPOINT']
DSQL_REGION = os.environ.get('DSQL_REGION', 'us-west-2')
ITERATIONS = int(os.environ.get('BENCH_ITERATIONS', '30'))
QUERY = 'SELECT 1'
ARCH = platform.machine()

def fmt(s):
    return round(s * 1000, 2)

def calc_stats(connect_times, query_times):
    n = len(connect_times)
    total = sum(connect_times) + sum(query_times)
    return {
        'connect_avg_ms': fmt(statistics.mean(connect_times)),
        'connect_p50_ms': fmt(statistics.median(connect_times)),
        'connect_p95_ms': fmt(sorted(connect_times)[int(n * 0.95)]),
        'query_avg_ms': fmt(statistics.mean(query_times)),
        'query_p50_ms': fmt(statistics.median(query_times)),
        'total_ms': fmt(total),
        'qps': round(n / total, 1),
    }

def bench_psycopg2():
    import aurora_dsql_psycopg2 as dsql
    ct, qt = [], []
    for _ in range(ITERATIONS):
        t0 = time.perf_counter()
        conn = dsql.connect(host=DSQL_ENDPOINT, region=DSQL_REGION, user='admin')
        t1 = time.perf_counter()
        ct.append(t1 - t0)
        cur = conn.cursor()
        cur.execute(QUERY)
        cur.fetchall()
        qt.append(time.perf_counter() - t1)
        cur.close()
        conn.close()
    return {'pattern': 'psycopg2 (every connect)', **calc_stats(ct, qt)}

async def bench_asyncpg_no_pool():
    import aurora_dsql_asyncpg as dsql
    ct, qt = [], []
    for _ in range(ITERATIONS):
        t0 = time.perf_counter()
        conn = await dsql.connect(host=DSQL_ENDPOINT, region=DSQL_REGION, user='admin')
        t1 = time.perf_counter()
        ct.append(t1 - t0)
        await conn.fetch(QUERY)
        qt.append(time.perf_counter() - t1)
        await conn.close()
    return {'pattern': 'asyncpg (every connect)', **calc_stats(ct, qt)}

async def bench_asyncpg_reuse():
    import aurora_dsql_asyncpg as dsql
    t0 = time.perf_counter()
    pool = await dsql.create_pool(
        host=DSQL_ENDPOINT, region=DSQL_REGION, user='admin',
        min_size=1, max_size=3)
    pool_create = time.perf_counter() - t0
    ct, qt = [], []
    for _ in range(ITERATIONS):
        t1 = time.perf_counter()
        async with pool.acquire() as conn:
            t2 = time.perf_counter()
            ct.append(t2 - t1)
            await conn.fetch(QUERY)
            qt.append(time.perf_counter() - t2)
    await pool.close()
    stats = calc_stats(ct, qt)
    stats['pool_create_ms'] = fmt(pool_create)
    return {'pattern': 'asyncpg (connection reuse)', **stats}

def lambda_handler(event, context):
    results = []
    results.append(bench_psycopg2())
    loop = asyncio.get_event_loop()
    results.append(loop.run_until_complete(bench_asyncpg_no_pool()))
    results.append(loop.run_until_complete(bench_asyncpg_reuse()))
    body = {
        'arch': ARCH,
        'runtime': f"python{platform.python_version()}",
        'iterations': ITERATIONS,
        'query': QUERY,
        'endpoint': DSQL_ENDPOINT,
        'results': results,
    }
    print(json.dumps(body, ensure_ascii=False, indent=2))
    return body

References

Share this article

AWSのお困り事はクラスメソッドへ