Aurora DSQL Python Connector Has Been Released, So I Measured and Compared Performance on Lambda (psycopg2 vs asyncpg)
This page has been translated by machine translation. View original
On February 19, 2026, official connectors for Aurora DSQL were released for three languages: Go, Python, and Node.js.
The Python package aurora-dsql-python-connector supports three drivers (psycopg / psycopg2 / asyncpg) and is a transparent authentication layer that automates IAM authentication token generation when connecting to DSQL.
Until now, connecting to DSQL from Python required creating a DSQL client with boto3, generating a token with generate_db_connect_admin_auth_token, and passing it as the password of psycopg2.connect.
import psycopg2
import boto3
def get_dsql_connection():
client = boto3.client('dsql', region_name='us-west-2')
token = client.generate_db_connect_admin_auth_token(DSQL_ENDPOINT, 'us-west-2')
return psycopg2.connect(
host=DSQL_ENDPOINT, port=5432, database='postgres',
user='admin', password=token, sslmode='require'
)
With the new connector, this becomes a single line.
import aurora_dsql_psycopg2 as dsql
conn = dsql.connect(host=DSQL_ENDPOINT, region='us-west-2', user='admin')
The boto3 call, token generation, and sslmode specification are all no longer needed. With asyncpg, you can likewise use it simply by importing aurora_dsql_asyncpg.
Motivation: Why I Tested on Lambda
Following the release of the new connector, I wanted to understand the improvement it would bring to Lambda workloads that use psycopg2. My current environment had three issues.
First, it depends on an x86 Lambda layer for psycopg2-binary, so it cannot migrate to arm64 (Graviton). Second, some Lambdas run for up to 230 minutes via Step Functions, so the token can expire because the run exceeds DSQL's maximum connection lifetime (60 minutes). Third, the same get_dsql_connection() is copy-pasted across multiple files, which hurts maintainability.
Since the new connector also supports asyncpg, I decided to measure the improvement, including a driver change, in an actual Lambda environment.
Note that the new connector also supports psycopg (v3), but since this test assumes migrating an existing environment, I limited it to psycopg2 and asyncpg.
Test Method
Test Setup
I created two Lambda functions with SAM: one for x86_64 and one for arm64 (Graviton), each attached to a layer containing psycopg2 and asyncpg. The runtime is Python 3.12, memory is 256 MB, and the DSQL cluster is in us-west-2, the same region as Lambda. After testing, you can delete the whole stack with sam delete.
Test Patterns
I measured the following six patterns.
| # | Pattern | Architecture | Processing per iteration |
|---|---|---|---|
| 1 | psycopg2 (connect every time) | x86_64 | Automatic token generation → TCP → TLS → PG authentication → SQL → disconnect |
| 2 | asyncpg (connect every time) | x86_64 | Same as above (driver is asyncpg) |
| 3 | asyncpg (connection reuse) | x86_64 | Connection established only the first time; the same connection is reused afterward |
| 4 | psycopg2 (connect every time) | arm64 | Pattern 1 run on Graviton |
| 5 | asyncpg (connect every time) | arm64 | Pattern 2 run on Graviton |
| 6 | asyncpg (connection reuse) | arm64 | Pattern 3 run on Graviton |
The "connect every time" patterns are not intended for production use; they are deliberate anti-patterns for accurately measuring the overhead of a single connection establishment.
Measurement Method
I used SELECT 1 as the query to minimize processing on the DB side. Within a single Lambda invoke, I ran 30 iterations and measured "connection establishment time" and "query execution time" separately in each iteration using time.perf_counter(). I invoked each architecture multiple times (x86: 6 times, arm64: 3 times) and used the median.
Measurements were taken in the in-handler loop during warm starts and do not include Init Duration. "Connection time" is the sum of IAM token generation + TCP connection + TLS handshake + PostgreSQL authentication.
Reference Values: Cold Start
| x86_64 | arm64 | |
|---|---|---|
| Init Duration | 149ms | 123ms |
| First connection | approx. 1.0–1.5 seconds | approx. 1.0 seconds |
| Total cold start | approx. 1.2–1.7 seconds | approx. 1.1–1.2 seconds |
There was ample margin relative to API Gateway's default timeout (29 seconds).
Below is a conceptual diagram of the Lambda lifecycle and connection establishment timing.
What I measured in this test is the in-handler loop during this warm start.
Results
x86_64 Lambda (median, 6 runs)
| Pattern | Connection (avg) | Query (avg) |
|---|---|---|
| psycopg2 (connect every time) | 873.0ms | 12.95ms |
| asyncpg (connect every time) | 807.7ms | 7.53ms |
| asyncpg (connection reuse) | 0.02ms | 2.50ms |
arm64 (Graviton) Lambda (median, 3 runs)
| Pattern | Connection (avg) | Query (avg) |
|---|---|---|
| psycopg2 (connect every time) | 673.6ms | 12.56ms |
| asyncpg (connect every time) | 633.1ms | 6.67ms |
| asyncpg (connection reuse) | 0.02ms | 2.44ms |
Discussion
Driver Comparison: Small Difference in Connection, Clear Difference in Queries
The difference in connection establishment time between drivers was small: 7% on x86 and 6% on arm64. Most of the connection time is taken up by IAM token generation (STS calls via boto3), and the driver-specific processing is only a small part of it.
On the other hand, there was a clear difference in query execution time. asyncpg was 42% faster on x86 and 47% faster on arm64. This is thanks to PostgreSQL's binary protocol and result parsing with Cython.
However, in the Lambda environment, connection establishment (600–870ms) accounts for most of the processing time, so a difference of a few milliseconds in queries is within the margin of error relative to the total.
Connection Reuse: Connection Cost Reduced to Zero
In the connection reuse pattern, obtaining a connection dropped from 600–870ms to 0.02ms. Because an already established connection is used as is, neither TCP connection nor token generation occurs.
I implemented it with asyncpg.create_pool(min_size=1), but since Lambda processes one request at a time serially, the effect is essentially the same as keeping a single connection in a global variable. In Lambda, it is simpler to hold the connection in global scope as shown below, without using a pool library.
import aurora_dsql_asyncpg as dsql
conn = None
async def handler(event, context):
global conn
if conn is None or conn.is_closed():
conn = await dsql.connect(host=DSQL_ENDPOINT, region='us-west-2', user='admin')
row = await conn.fetchrow("SELECT ...")
return row
Because global variables are preserved during warm starts, connection establishment can be skipped on the second and subsequent invokes.
Architecture Comparison: arm64 Was 22–23% Faster at Connection Establishment
The reduction was 23% for psycopg2 (873→674ms) and 22% for asyncpg (808→633ms). It is thought that Graviton processed CPU-bound work such as signature calculation for IAM token generation faster. Query execution time was nearly the same because network I/O dominates it.
Operational Notes for Lambda × DSQL
Connection Rate Limit: The Biggest Pitfall
DSQL has a hard limit of 100 connections per second on the connection rate. When Lambda scales out under spiky access, each container tries to establish connections all at once, and the 100/sec limit is reached instantly. Since DSQL has no intermediary like RDS Proxy, control on the Lambda side is essential.
Possible countermeasures include limiting concurrency with Reserved Concurrency, retrying with exponential backoff on connection errors, and absorbing spikes by placing SQS or EventBridge in between.
Here is a summary of the main DSQL constraints and their impact on Lambda.
| DSQL constraint | Value | Impact on Lambda |
|---|---|---|
| Connection rate | 100 connections/sec | Most dangerous during simultaneous scale-out |
| Maximum connection lifetime | 60 minutes | Reconnection required for long-running batches |
| Maximum transaction duration | 5 minutes | Large INSERTs must be split |
| Maximum rows per transaction | 3,000 rows | Affects batch size design |
| Maximum connections | 10,000 | Needs attention in large-scale Lambda operations |
Should You Adopt asyncpg?
asyncpg's query execution was 42–47% faster, but the decision to adopt it in Lambda is not determined by speed alone.
asyncpg is suitable when you want to break the dependency on the x86 psycopg2 layer by migrating to arm64, when you run multiple queries in parallel with asyncio.gather, or when connection reuse is important in batch processing.
On the other hand, for simple processing with one or two queries per request, the psycopg2 connector is sufficient. No SQL rewrites are needed and the adoption cost is low, and the benefits of eliminating manual token generation and reducing boilerplate are gained on their own.
Notes When Migrating to asyncpg
Adopting asyncpg requires some code changes.
SQL placeholders change from %s to $1, $2, ..., and parameters are passed as positional arguments rather than a tuple.
# psycopg2
cur.execute("SELECT * FROM t WHERE id = %s AND status = %s", (article_id, 'pending'))
row = cur.fetchone()
# asyncpg
row = await conn.fetchrow("SELECT * FROM t WHERE id = $1 AND status = $2", article_id, 'pending')
Method names change from fetchone → fetchrow and fetchall → fetch.
Transaction handling also differs. psycopg2 defaults to autocommit=False and starts transactions implicitly, whereas asyncpg defaults to autocommit. When using transactions explicitly, wrap them in async with conn.transaction():.
# psycopg2
conn.commit()
# asyncpg
async with conn.transaction():
await conn.execute("INSERT ...")
await conn.execute("UPDATE ...")
Since DSQL's maximum transaction duration is 5 minutes, asyncpg's autocommit default is less likely to run into this limit and errs on the safe side.
Summary
Through testing on Lambda, I was able to confirm the effect of the new connector.
The cost of connection establishment is large in the Lambda environment at 600–870ms, and it can be reduced to zero with connection reuse. On arm64, connection establishment is 22–23% faster than on x86, and a 20% cost reduction can also be expected. Because the connector automatically generates IAM tokens, the risk of authentication expiring in long-running batches is also eliminated.
If you are looking for a way to connect to DSQL, I think it is best to start with the psycopg2 connector, which can be introduced without changing existing code. If arm64 migration or batch processing optimization is on the horizon, consider stepping up to the asyncpg connector.
Note that these measurements are warm-start values in the same region (us-west-2), and are medians of 30 iterations × multiple invokes. Results will vary depending on the network environment and the load on the DSQL cluster.
Reference: Test Code
Here are the SAM template and Lambda handler used for the test. You can deploy with sam build && sam deploy, run with sam remote invoke, and tear down with sam delete.
template.yaml
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: Aurora DSQL Connector Benchmark on Lambda
Parameters:
DsqlEndpoint:
Type: String
DsqlRegion:
Type: String
Default: us-west-2
BenchIterations:
Type: String
Default: '30'
Globals:
Function:
Runtime: python3.12
Timeout: 300
MemorySize: 256
Environment:
Variables:
DSQL_ENDPOINT: !Ref DsqlEndpoint
DSQL_REGION: !Ref DsqlRegion
BENCH_ITERATIONS: !Ref BenchIterations
Resources:
Psycopg2Layer:
Type: AWS::Serverless::LayerVersion
Properties:
LayerName: dsql-bench-psycopg2-x86
ContentUri: layers/psycopg2-x86/
CompatibleRuntimes: [python3.12]
CompatibleArchitectures: [x86_64]
Metadata:
BuildMethod: python3.12
BuildArchitecture: x86_64
AsyncpgLayer:
Type: AWS::Serverless::LayerVersion
Properties:
LayerName: dsql-bench-asyncpg-arm64
ContentUri: layers/asyncpg-arm64/
CompatibleRuntimes: [python3.12]
CompatibleArchitectures: [arm64]
Metadata:
BuildMethod: python3.12
BuildArchitecture: arm64
BenchX86:
Type: AWS::Serverless::Function
Properties:
FunctionName: dsql-bench-x86
Handler: handler.lambda_handler
CodeUri: src/
Architectures: [x86_64]
Layers: [!Ref Psycopg2Layer]
Policies:
- Statement:
- Effect: Allow
Action: dsql:DbConnectAdmin
Resource: !Sub arn:aws:dsql:${DsqlRegion}:${AWS::AccountId}:cluster/*
BenchArm64:
Type: AWS::Serverless::Function
Properties:
FunctionName: dsql-bench-arm64
Handler: handler.lambda_handler
CodeUri: src/
Architectures: [arm64]
Layers: [!Ref AsyncpgLayer]
Policies:
- Statement:
- Effect: Allow
Action: dsql:DbConnectAdmin
Resource: !Sub arn:aws:dsql:${DsqlRegion}:${AWS::AccountId}:cluster/*
handler.py
import asyncio
import json
import os
import platform
import statistics
import time
DSQL_ENDPOINT = os.environ['DSQL_ENDPOINT']
DSQL_REGION = os.environ.get('DSQL_REGION', 'us-west-2')
ITERATIONS = int(os.environ.get('BENCH_ITERATIONS', '30'))
QUERY = 'SELECT 1'
ARCH = platform.machine()
def fmt(s):
return round(s * 1000, 2)
def calc_stats(connect_times, query_times):
n = len(connect_times)
total = sum(connect_times) + sum(query_times)
return {
'connect_avg_ms': fmt(statistics.mean(connect_times)),
'connect_p50_ms': fmt(statistics.median(connect_times)),
'connect_p95_ms': fmt(sorted(connect_times)[int(n * 0.95)]),
'query_avg_ms': fmt(statistics.mean(query_times)),
'query_p50_ms': fmt(statistics.median(query_times)),
'total_ms': fmt(total),
'qps': round(n / total, 1),
}
def bench_psycopg2():
import aurora_dsql_psycopg2 as dsql
ct, qt = [], []
for _ in range(ITERATIONS):
t0 = time.perf_counter()
conn = dsql.connect(host=DSQL_ENDPOINT, region=DSQL_REGION, user='admin')
t1 = time.perf_counter()
ct.append(t1 - t0)
cur = conn.cursor()
cur.execute(QUERY)
cur.fetchall()
qt.append(time.perf_counter() - t1)
cur.close()
conn.close()
return {'pattern': 'psycopg2 (every connect)', **calc_stats(ct, qt)}
async def bench_asyncpg_no_pool():
import aurora_dsql_asyncpg as dsql
ct, qt = [], []
for _ in range(ITERATIONS):
t0 = time.perf_counter()
conn = await dsql.connect(host=DSQL_ENDPOINT, region=DSQL_REGION, user='admin')
t1 = time.perf_counter()
ct.append(t1 - t0)
await conn.fetch(QUERY)
qt.append(time.perf_counter() - t1)
await conn.close()
return {'pattern': 'asyncpg (every connect)', **calc_stats(ct, qt)}
async def bench_asyncpg_reuse():
import aurora_dsql_asyncpg as dsql
t0 = time.perf_counter()
pool = await dsql.create_pool(
host=DSQL_ENDPOINT, region=DSQL_REGION, user='admin',
min_size=1, max_size=3)
pool_create = time.perf_counter() - t0
ct, qt = [], []
for _ in range(ITERATIONS):
t1 = time.perf_counter()
async with pool.acquire() as conn:
t2 = time.perf_counter()
ct.append(t2 - t1)
await conn.fetch(QUERY)
qt.append(time.perf_counter() - t2)
await pool.close()
stats = calc_stats(ct, qt)
stats['pool_create_ms'] = fmt(pool_create)
return {'pattern': 'asyncpg (connection reuse)', **stats}
def lambda_handler(event, context):
results = []
results.append(bench_psycopg2())
loop = asyncio.get_event_loop()
results.append(loop.run_until_complete(bench_asyncpg_no_pool()))
results.append(loop.run_until_complete(bench_asyncpg_reuse()))
body = {
'arch': ARCH,
'runtime': f"python{platform.python_version()}",
'iterations': ITERATIONS,
'query': QUERY,
'endpoint': DSQL_ENDPOINT,
'results': results,
}
print(json.dumps(body, ensure_ascii=False, indent=2))
return body
