I Tried Placing the Cache for Lambda Web Functions on ElastiCache Serverless for Valkey
This page has been translated by machine translation. View original
Introduction
Recently, we migrated the DevelopersIO backend from ECS (EC2) to Lambda Web Functions.
When running on ECS (EC2), we used Next.js's memory cache and file cache to reduce calls to the Content Delivery API (CDA) of the CMS (Contentful). In the migration to Lambda Web Functions, execution environments (instances) disappear as they scale in and out or get replaced, so we abolished the Next.js-side cache.
Instead, we set up a separate Lambda Web Functions function from Next.js that caches CDA responses (valkey-cda-mr; hereafter, the cache Lambda), and Amazon ElastiCache Serverless for Valkey (hereafter, Valkey Serverless).
Architecture
Overall flow
The red frame is the part we added this time. Next.js calls a CloudFront distribution for the CDA instead of the Contentful CDA. The cache Lambda behind it reads from the Valkey Serverless in the same region, and fetches from the Contentful CDA only when there is no value.
Lifetime of execution environments
Each Lambda Web Functions execution environment has fixed memory of 2 GB, and environments disappear as they scale in and out or get replaced.
Using the cache Lambda's logs, we tallied the uptime of about 480 environments that terminated in the past week (5 regions). Uptime is the time from startup until the last request was processed; the median was about 30 minutes and the longest was about 6.5 hours.
| Uptime (minutes) | ap-northeast-1 | us-east-1 | us-east-2 | us-west-2 | eu-central-1 | Total | Share |
|---|---|---|---|---|---|---|---|
| Under 1 | 7 | 3 | 6 | 6 | 9 | 31 | 6% |
| 1–5 | 11 | 13 | 10 | 7 | 5 | 46 | 9% |
| 5–10 | 51 | 27 | 19 | 2 | 0 | 99 | 20% |
| 10–20 | 14 | 13 | 7 | 2 | 1 | 37 | 8% |
| 20–30 | 9 | 12 | 5 | 0 | 0 | 26 | 5% |
| 30–60 | 14 | 8 | 3 | 0 | 1 | 26 | 5% |
| 60–120 | 16 | 26 | 8 | 4 | 3 | 57 | 12% |
| 120–240 | 12 | 17 | 17 | 6 | 6 | 58 | 12% |
| 240–330 | 6 | 4 | 7 | 2 | 5 | 24 | 5% |
| 330–360 | 0 | 4 | 3 | 3 | 3 | 13 | 3% |
| 360–375 | 8 | 5 | 3 | 10 | 9 | 35 | 7% |
| 375–390 | 5 | 5 | 6 | 9 | 8 | 33 | 7% |
| Total | 153 | 137 | 94 | 51 | 50 | 485 | 100% |
About one-third of the environments terminated in under 10 minutes, and a little over 10% ran for 6 hours or more, so uptime split into short-lived environments and environments that last about 6 hours.
Caches placed in an environment's memory or files disappear along with the environment. So we made the Valkey Serverless in each region a data store shared by all environments in that region, so the cache is preserved even when environments are replaced. Contentful CDA responses stored in Valkey Serverless are handled in the following states depending on their staleness.
| State | Staleness of response | Behavior |
|---|---|---|
| HIT | Fresh | Return as is |
| STALE | Old | Return immediately as is, and refresh in the background (stale-while-revalidate) |
| EXPIRED | Even older | Fetch again from the Contentful CDA, then return |
| MISS | Not in Valkey Serverless | Fetch from the Contentful CDA and return |
Connecting to Valkey
In this architecture, we connect to the public endpoint of Valkey Serverless using IAM authentication and TLS 1.3. With the public endpoint, reconnecting for every operation makes p50 about 200 times slower than reusing a connection.
So the cache Lambda starts the connection when the environment starts up, and reuses the single established connection for all requests.
Here is an excerpt from cache.mjs.
let client = null;
let connecting = null; // 接続中の Promise（接続中だけ非 null）
let retryAt = 0;
/** モジュール読み込み時に接続を開始する（最初のリクエストより前に完了させる） */
export function start() {
connect();
}
How things are handled while the connection is not yet established is in the get function.
/**
* 接続済みのクライアントを返す。接続の確立中なら waitMs まで待つ（コールドスタート直後の 1 回目用）。
* 接続に失敗している間は待たずに null を返す。
*/
async function get(waitMs) {
if (client) return client;
if (!connecting && Date.now() >= retryAt) connect();
if (connecting && waitMs > 0) {
await Promise.race([connecting, new Promise((r) => setTimeout(r, waitMs))]);
}
return client;
}
Measurements
All Valkey Serverless values are for Tokyo.
Hit rate
The period is the past week. The CacheHitRate of Tokyo's Valkey Serverless (the proportion of Valkey GETs where a value was found; includes entries that were too old and had to be refetched) stayed at roughly 70–90% on an hourly average.
Response time
The period is the past day. The p50 response time measured inside the cache Lambda is shown by state. The values are estimates derived from the cda.request.duration histogram, aggregated across the 5 regions.
| State | In-function response time (p50) |
|---|---|
| HIT | About 1 ms |
| STALE | About 1 ms |
| MISS | About 130 ms |
| EXPIRED | About 120 ms |
Cost
The period is the past week. The cost of Tokyo's Valkey Serverless, on an on-demand basis in Cost Explorer, was about 1.2 USD per day, or about 40 USD per month.
| Item | Per day (on-demand basis) | Breakdown |
|---|---|---|
| Storage | About 1.2 USD | About 12 GB-hours (average about 0.5 GB) |
| ECPU | Under 0.01 USD | About 2 million ECPU/day |
| Total | About 1.2 USD |
Valkey Serverless billing is determined by the amount of stored data (GB-hours, minimum 100 MB) and ECPU, and there are no node charges.
Since there are no node charges and nearly all of the cost is storage, the fixed cost stays small even when placing a Valkey Serverless in each region.
During traffic spikes
The period is the night of 10/30. The number of cache Lambda requests peaked at about 4,600 per minute (about 80 rps). The normal level is about 600 per minute, so the peak was about 8 times higher. There were zero 5XX errors in all 5 regions, and zero Valkey Serverless errors (connection and command).
The peak hour had about 140,000 requests, about 4 times the roughly 35,000 you get by converting the normal level of about 600 per minute to an hour.
Summary
The cache that was held in Next.js memory and files on ECS (EC2) has been replaced with Valkey Serverless on Lambda Web Functions. How the cache is held has also become more flexible, for example changing the TTL based on an article's update date.
Lambda Web Functions has limits on execution environment memory and storage capacity, and on continuous running time. Valkey Serverless has no node charges and is easy to place in each region, so please try it as one way to compensate for these limits in a multi-region setup.
