Cutting CDK Deployment Wait Time to 1/6! Testing the Real Power of the High-Speed Deployment Tool "cdkd"
This page has been translated by machine translation. View original
When you create an IAM Role + InstanceProfile + EC2 with CloudFormation, you can sometimes be kept waiting for several minutes. In particular, there are cases where it takes about 2 minutes for the InstanceProfile creation to complete, and even with CDK it is the same because CloudFormation is running behind the scenes.
I was wondering, "Can't this be made faster?" and then I learned about a tool called "cdkd" that deploys directly with the AWS SDK without going through CloudFormation, so I tried it out.
What is cdkd
cdkd (CDK Direct) is an experimental tool that deploys CDK apps directly with the AWS SDK / Cloud Control API without going through CloudFormation. For supported resources, you can use it simply by replacing cdk deploy with cdkd deploy, without changing your existing CDK code.
Installation and initial setup
# Install
npm install -g @go-to-k/cdkd
# Initial setup (once per account. Separate from CDK bootstrap)
cdkd bootstrap
# Deploy (just run it instead of cdk deploy)
cdkd deploy
In this test, I isolated the environment and ran it with Docker. The Dockerfile is included in the appendix.
Test: Comparing 4 methods with the same configuration
Configuration
It consists of 5 resources: IAM Role, IAM Policy, InstanceProfile, SecurityGroup, and EC2 Instance.
IAM Role → IAM Policy
→ InstanceProfile → EC2 Instance ← SecurityGroup
Methods compared
| Method | Description |
|---|---|
| CloudFormation | aws cloudformation deploy directly |
| CDK | cdk deploy (via CloudFormation) |
| cdkd | cdkd deploy (direct SDK) |
| Optimized CLI | bash + AWS CLI parallelization (reference value for hand-written optimization) |
Execution environment
- Region: ap-northeast-1 (Tokyo)
- Run on Docker (node:20-slim)
- aws-cli 2.34.45 / node v20.20.2 / cdk 2.1121.0 / cdkd 0.58.0
- t3.micro / Amazon Linux 2023 (x86_64)
- Each method run 3 times, median adopted
Results
| Method | Run 1 | Run 2 | Run 3 | Median | Speedup |
|---|---|---|---|---|---|
| CloudFormation | 188s | 187s | 188s | 188s | 1.0x |
| CDK | 188s | 188s | 188s | 188s | 1.0x |
| cdkd | 29s | 28s | 28s | 28s | 6.7x |
| Optimized CLI | 30s | 30s | 31s | 30s | 6.3x |
With this EC2 + IAM InstanceProfile configuration, cdkd was about 6.7 times faster than CloudFormation/CDK.
Why this difference arose
The main cause is the InstanceProfile stabilization check
Looking at CloudFormation's Stack Events, you can see that creating the InstanceProfile took 2 minutes 11 seconds.
BenchEc2IamStack | 2/6 | 8:57:55 AM | CREATE_IN_PROGRESS | AWS::IAM::InstanceProfile | Resource creation Initiated
BenchEc2IamStack | 4/6 | 9:00:06 AM | CREATE_COMPLETE | AWS::IAM::InstanceProfile
The SDK's CreateInstanceProfile itself returns immediately, but CloudFormation performs an internal stabilization check after creation ("Eventual consistency check initiated" is recorded in Stack Events). This wait time is presumed to be a mechanism to guarantee that IAM propagation has completed. The official documentation does not publish details about the target resource types or the time required.
cdkd's approach: optimistic execution + retry
With cdkd, the long wait for the stabilization check does not occur, and all resources were created in a short time.
Deploying 5 resource(s) (DAG: 3 levels, max parallel: 10)
[1/5] ✅ BenchSecurityGroup (AWS::EC2::SecurityGroup) created
[2/5] ✅ BenchRole (AWS::IAM::Role) created
[3/5] ✅ BenchPolicy (AWS::IAM::Policy) created
[4/5] ✅ BenchInstanceProfile (AWS::IAM::InstanceProfile) created
[5/5] ✅ BenchInstance (AWS::EC2::Instance) created
Deployment Summary:
Duration: 19.83s
- The Duration of 19.83s is the time for the cdkd deployment itself. The 28s in the results table is the total time for the whole command, including synth + deployment + script overhead.
If IAM propagation is not in time, it detects the error pattern and retries with backoff (the retry patterns are defined in src/deployment/retryable-errors.ts). In the optimized CLI's log, I was able to confirm that retries actually occurred.
[00:06] Retry 1: IAM not propagated yet (waiting 2s)...
[00:08] Retry 2: IAM not propagated yet (waiting 4s)...
[00:14] EC2 Instance launched
At least in this environment, the wait for the IAM propagation required to launch EC2 was covered by 2 retries, 6 seconds in total.
Other cases where this is likely effective beyond InstanceProfile
CloudFormation's Eventual consistency check occurs not only for InstanceProfile but for many resources. In the Stack Events of this test, it was also confirmed for SecurityGroup and EC2 Instance.
BenchSecurityGroup | CREATE_IN_PROGRESS | Eventual consistency check initiated
BenchInstance | CREATE_IN_PROGRESS | Eventual consistency check initiated
The cdkd approach (immediate SDK completion + retry on error) is expected to save time for resources in general where the Eventual consistency check is long. However, since stabilization conditions and SDK behavior differ by resource type, the same time savings as in this test will not always be obtained.
Speed equivalent to a hand-written optimized CLI, with declarative code
The optimized CLI, with parallelization and retries hand-implemented in bash, took 30 seconds, while deploying the declarative TypeScript CDK code as-is with cdkd took 28 seconds. I found that a major advantage of cdkd is that it achieves an equivalent duration while keeping the declarative CDK code, without having to hand-implement parallelization or retries.
Notes and limitations
- It does not support all resource types (90+ SDK Providers + Cloud Control API fallback)
- Rollback (can be disabled with
--no-rollback) and drift detection (cdkd drift, supports--accept/--revert) have been implemented since v0.35.0. However, some bugs, such as drift detection for properties not specified in CFn, are still being fixed (see the CHANGELOG) - State management is S3-based (no DynamoDB needed, but it does not have as much operational track record as CloudFormation)
- Since it is under active development, I recommend checking the official repository for the latest support status
Summary
Traditionally, there was a workaround of separating IAM into a different stack to avoid waiting for InstanceProfile creation, but splitting stacks had the side effect of making it difficult to minimize permissions on a per-resource basis. With cdkd, you can deploy quickly while keeping IAM and EC2 in the same stack, so you can achieve both least privilege and speed.
When minimizing IAM permissions, where you repeat deploy → test → fix while gradually narrowing down the policy, or in workflows that create and destroy a test environment for each PR, or in prototyping where you experiment with configurations, going from 3 minutes to 30 seconds per deployment greatly changes the development experience.
If you feel that the wait time for CDK or CloudFormation deployments in development and testing environments is a problem, please give it a try.
Reference links
Code for reproduction (click to expand)
CDK app (bin/app.ts)
#!/usr/bin/env node
import 'source-map-support/register';
import * as cdk from 'aws-cdk-lib';
import { BenchStack } from '../lib/stack';
const app = new cdk.App();
const vpcId = app.node.tryGetContext('vpcId');
const subnetId = app.node.tryGetContext('subnetId');
const amiId = app.node.tryGetContext('amiId');
if (!vpcId || !subnetId || !amiId) {
throw new Error('Required: -c vpcId=xxx -c subnetId=xxx -c amiId=xxx');
}
new BenchStack(app, 'BenchEc2IamStack', { vpcId, subnetId, amiId });
CDK app (cdk.json)
{
"app": "npx ts-node --prefer-ts-exts bin/app.ts",
"versionReporting": false
}
CDK app (lib/stack.ts)
import * as cdk from 'aws-cdk-lib';
import * as iam from 'aws-cdk-lib/aws-iam';
import * as ec2 from 'aws-cdk-lib/aws-ec2';
import { Construct } from 'constructs';
interface BenchStackProps extends cdk.StackProps {
vpcId: string;
subnetId: string;
amiId: string;
}
export class BenchStack extends cdk.Stack {
constructor(scope: Construct, id: string, props: BenchStackProps) {
super(scope, id, props);
const role = new iam.CfnRole(this, 'BenchRole', {
assumeRolePolicyDocument: {
Version: '2012-10-17',
Statement: [{ Effect: 'Allow', Principal: { Service: 'ec2.amazonaws.com' }, Action: 'sts:AssumeRole' }],
},
});
new iam.CfnPolicy(this, 'BenchPolicy', {
policyName: 'bench-policy',
roles: [role.ref],
policyDocument: {
Version: '2012-10-17',
Statement: [{ Effect: 'Allow', Action: 's3:GetObject', Resource: '*' }],
},
});
const instanceProfile = new iam.CfnInstanceProfile(this, 'BenchInstanceProfile', {
roles: [role.ref],
});
const sg = new ec2.CfnSecurityGroup(this, 'BenchSecurityGroup', {
groupDescription: 'Benchmark security group',
vpcId: props.vpcId,
});
new ec2.CfnInstance(this, 'BenchInstance', {
instanceType: 't3.micro',
imageId: props.amiId,
subnetId: props.subnetId,
iamInstanceProfile: instanceProfile.ref,
securityGroupIds: [sg.attrGroupId],
tags: [{ key: 'Name', value: 'bench-test' }],
});
}
}
Dockerfile
Optimized CLI script (04-cli-optimized-deploy.sh)
#!/bin/bash
set -euo pipefail
VPC_ID="${1:?Usage: $0 <vpc-id> <subnet-id> <ami-id>}"
SUBNET_ID="${2:?}"
AMI_ID="${3:?}"
RUN_ID="bench-$(date +%s)"
ROLE_NAME="bench-role-${RUN_ID}"
PROFILE_NAME="bench-profile-${RUN_ID}"
START=$(date +%s)
ts() { local e=$(( $(date +%s) - START )); printf '[%02d:%02d]' $((e/60)) $((e%60)); }
echo "$(ts) === Optimized CLI deploy (parallel + retry) ==="
# Phase 1: Role + SecurityGroup in parallel
echo "$(ts) Creating IAM Role & SecurityGroup (parallel)..."
aws iam create-role --role-name "$ROLE_NAME" \
--assume-role-policy-document '{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"Service":"ec2.amazonaws.com"},"Action":"sts:AssumeRole"}]}' \
--no-cli-pager > /dev/null &
ROLE_PID=$!
SG_ID=$(aws ec2 create-security-group --group-name "bench-sg-${RUN_ID}" \
--description "Benchmark SG" --vpc-id "$VPC_ID" --query 'GroupId' --output text)
echo "$(ts) SecurityGroup created: ${SG_ID}"
wait $ROLE_PID
echo "$(ts) IAM Role created: ${ROLE_NAME}"
# Phase 2: Policy + InstanceProfile in parallel (depend on Role)
echo "$(ts) Creating Policy & InstanceProfile (parallel)..."
aws iam put-role-policy --role-name "$ROLE_NAME" --policy-name "bench-policy" \
--policy-document '{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Action":"s3:GetObject","Resource":"*"}]}' &
POLICY_PID=$!
aws iam create-instance-profile --instance-profile-name "$PROFILE_NAME" --no-cli-pager > /dev/null
aws iam add-role-to-instance-profile --instance-profile-name "$PROFILE_NAME" --role-name "$ROLE_NAME"
echo "$(ts) InstanceProfile created: ${PROFILE_NAME}"
wait $POLICY_PID
echo "$(ts) Policy attached"
# Phase 3: EC2 Instance (retry for IAM propagation)
echo "$(ts) Launching EC2 Instance (with IAM propagation retry)..."
PROFILE_ARN=$(aws iam get-instance-profile --instance-profile-name "$PROFILE_NAME" \
--query 'InstanceProfile.Arn' --output text)
INSTANCE_ID=""
for i in 1 2 3 4 5 6 7 8 9 10; do
INSTANCE_ID=$(aws ec2 run-instances --instance-type t3.micro --image-id "$AMI_ID" \
--subnet-id "$SUBNET_ID" --iam-instance-profile "Arn=${PROFILE_ARN}" \
--security-group-ids "$SG_ID" \
--tag-specifications "ResourceType=instance,Tags=[{Key=Name,Value=bench-test}]" \
--query 'Instances[0].InstanceId' --output text 2>/dev/null) && break
echo "$(ts) Retry ${i}: IAM not propagated yet (waiting $((i*2))s)..."
sleep $((i * 2))
done
[ -z "$INSTANCE_ID" ] && echo "ERROR: Failed to launch instance" && exit 1
echo "$(ts) EC2 Instance launched: ${INSTANCE_ID}"
# Phase 4: Wait for running
echo "$(ts) Waiting for instance running..."
aws ec2 wait instance-running --instance-ids "$INSTANCE_ID"
END=$(date +%s)
echo "$(ts) === Done: $((END - START))s ==="
FROM node:20-slim
RUN apt-get update && apt-get install -y --no-install-recommends \
curl unzip jq bash ca-certificates python3 \
&& rm -rf /var/lib/apt/lists/*
# AWS CLI v2 (for aarch64. Change the URL for x86_64)
RUN curl -sL "https://awscli.amazonaws.com/awscli-exe-linux-aarch64.zip" -o /tmp/awscli.zip \
&& unzip -q /tmp/awscli.zip -d /tmp \
&& /tmp/aws/install \
&& rm -rf /tmp/aws /tmp/awscli.zip
# CDK + cdkd
RUN npm install -g aws-cdk @go-to-k/cdkd
WORKDIR /benchmark
COPY . .
RUN cd cdk-app && npm install
ENTRYPOINT ["/bin/bash"]
How to run
# Build
docker build -t bench-ec2-iam .
# cdkd bootstrap (first time only)
docker run --rm -v ~/.aws:/root/.aws -e AWS_REGION=ap-northeast-1 \
bench-ec2-iam -c "cdkd bootstrap"
# cdkd deploy
docker run --rm -v ~/.aws:/root/.aws -e AWS_REGION=ap-northeast-1 \
-w /benchmark/cdk-app \
bench-ec2-iam -c "cdkd deploy -c vpcId=<vpc-id> -c subnetId=<subnet-id> -c amiId=<ami-id>"
