I tried using ARM64 / FARGATE_SPOT / ECS Exec which are not supported in ECS Express Mode

# ECS Express Mode Limitations Workaround: Two-Stage Deployment Process ## Background ECS Express Mode provides simplified ECS cluster deployment, but it has the following limitations: - Cannot specify ARM64 (Graviton) architecture - Cannot enable Fargate Spot - Cannot enable ECS Exec This article explains a two-stage deployment process that combines CloudFormation and AWS CLI to work around these limitations. --- ## Overview of the Two-Stage Deployment Process ``` Stage 1: Initial deployment with CloudFormation (ECS Express Mode) ↓ Stage 2: Post-deployment updates via AWS CLI - Switch to ARM64 (Graviton) - Enable Fargate Spot - Enable ECS Exec ``` --- ## Stage 1: Initial Deployment with CloudFormation ### Template Structure ```yaml # ecs-express-base.yaml AWSTemplateFormatVersion: '2010-09-09' Description: 'ECS Express Mode Base Deployment' Parameters: AppName: Type: String Default: myapp ContainerImage: Type: String Description: 'ECR Image URI' ContainerPort: Type: Number Default: 8080 TaskCpu: Type: String Default: '512' TaskMemory: Type: String Default: '1024' VpcId: Type: AWS::EC2::VPC::Id SubnetIds: Type: List<AWS::EC2::Subnet::Id> Resources: # ECS Cluster ECSCluster: Type: AWS::ECS::Cluster Properties: ClusterName: !Sub '${AppName}-cluster' CapacityProviders: - FARGATE DefaultCapacityProviderStrategy: - CapacityProvider: FARGATE Weight: 1 ClusterSettings: - Name: containerInsights Value: enabled # CloudWatch Log Group LogGroup: Type: AWS::Logs::LogGroup Properties: LogGroupName: !Sub '/ecs/${AppName}' RetentionInDays: 30 # IAM Role for Task Execution TaskExecutionRole: Type: AWS::IAM::Role Properties: RoleName: !Sub '${AppName}-task-execution-role' AssumeRolePolicyDocument: Version: '2012-10-17' Statement: - Effect: Allow Principal: Service: ecs-tasks.amazonaws.com Action: sts:AssumeRole ManagedPolicyArns: - arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy # IAM Role for Task (with ECS Exec permissions) TaskRole: Type: AWS::IAM::Role Properties: RoleName: !Sub '${AppName}-task-role' AssumeRolePolicyDocument: Version: '2012-10-17' Statement: - Effect: Allow Principal: Service: ecs-tasks.amazonaws.com Action: sts:AssumeRole Policies: - PolicyName: ECSExecPolicy PolicyDocument: Version: '2012-10-17' Statement: # Permissions required for ECS Exec - Effect: Allow Action: - ssmmessages:CreateControlChannel - ssmmessages:CreateDataChannel - ssmmessages:OpenControlChannel - ssmmessages:OpenDataChannel Resource: '*' # CloudWatch Logs permissions - Effect: Allow Action: - logs:DescribeLogGroups - logs:CreateLogStream - logs:DescribeLogStreams - logs:PutLogEvents Resource: '*' # Security Group ServiceSecurityGroup: Type: AWS::EC2::SecurityGroup Properties: GroupDescription: !Sub 'Security group for ${AppName} ECS service' VpcId: !Ref VpcId SecurityGroupIngress: - IpProtocol: tcp FromPort: !Ref ContainerPort ToPort: !Ref ContainerPort CidrIp: '0.0.0.0/0' SecurityGroupEgress: - IpProtocol: '-1' CidrIp: '0.0.0.0/0' # Task Definition (initially x86_64) TaskDefinition: Type: AWS::ECS::TaskDefinition Properties: Family: !Sub '${AppName}-task' Cpu: !Ref TaskCpu Memory: !Ref TaskMemory NetworkMode: awsvpc RequiresCompatibilities: - FARGATE ExecutionRoleArn: !GetAtt TaskExecutionRole.Arn TaskRoleArn: !GetAtt TaskRole.Arn # x86_64 for initial deployment RuntimePlatform: CpuArchitecture: X86_64 OperatingSystemFamily: LINUX ContainerDefinitions: - Name: !Ref AppName Image: !Ref ContainerImage PortMappings: - ContainerPort: !Ref ContainerPort Protocol: tcp LogConfiguration: LogDriver: awslogs Options: awslogs-group: !Ref LogGroup awslogs-region: !Ref AWS::Region awslogs-stream-prefix: ecs Environment: - Name: APP_ENV Value: production Essential: true # ECS Service (initially FARGATE only) ECSService: Type: AWS::ECS::Service Properties: ServiceName: !Sub '${AppName}-service' Cluster: !Ref ECSCluster TaskDefinition: !Ref TaskDefinition DesiredCount: 2 LaunchType: FARGATE # ECS Exec disabled initially EnableExecuteCommand: false NetworkConfiguration: AwsvpcConfiguration: SecurityGroups: - !Ref ServiceSecurityGroup Subnets: !Ref SubnetIds AssignPublicIp: ENABLED DeploymentConfiguration: MaximumPercent: 200 MinimumHealthyPercent: 100 Outputs: ClusterName: Value: !Ref ECSCluster Export: Name: !Sub '${AppName}-cluster-name' ServiceName: Value: !GetAtt ECSService.Name Export: Name: !Sub '${AppName}-service-name' TaskDefinitionFamily: Value: !Sub '${AppName}-task' Export: Name: !Sub '${AppName}-task-family' TaskRoleArn: Value: !GetAtt TaskRole.Arn Export: Name: !Sub '${AppName}-task-role-arn' TaskExecutionRoleArn: Value: !GetAtt TaskExecutionRole.Arn Export: Name: !Sub '${AppName}-task-execution-role-arn' SecurityGroupId: Value: !Ref ServiceSecurityGroup Export: Name: !Sub '${AppName}-sg-id' ``` ### Stage 1 Deployment Script ```bash #!/bin/bash # deploy-stage1.sh set -euo pipefail # Configuration APP_NAME="myapp" STACK_NAME="${APP_NAME}-ecs-stack" REGION="ap-northeast-1" CONTAINER_IMAGE="123456789012.dkr.ecr.ap-northeast-1.amazonaws.com/myapp:latest" VPC_ID="vpc-xxxxxxxxxxxxxxxxx" SUBNET_IDS="subnet-xxxxxxxxxxxxxxxxx,subnet-yyyyyyyyyyyyyyyyy" echo "=== Stage 1: CloudFormation Deployment ===" # Stack deployment aws cloudformation deploy \ --template-file ecs-express-base.yaml \ --stack-name "${STACK_NAME}" \ --region "${REGION}" \ --parameter-overrides \ AppName="${APP_NAME}" \ ContainerImage="${CONTAINER_IMAGE}" \ VpcId="${VPC_ID}" \ SubnetIds="${SUBNET_IDS}" \ --capabilities CAPABILITY_NAMED_IAM \ --no-fail-on-empty-changeset echo "Stage 1 deployment complete" # Retrieve output values CLUSTER_NAME=$(aws cloudformation describe-stacks \ --stack-name "${STACK_NAME}" \ --region "${REGION}" \ --query 'Stacks[0].Outputs[?OutputKey==`ClusterName`].OutputValue' \ --output text) SERVICE_NAME=$(aws cloudformation describe-stacks \ --stack-name "${STACK_NAME}" \ --region "${REGION}" \ --query 'Stacks[0].Outputs[?OutputKey==`ServiceName`].OutputValue' \ --output text) echo "Cluster: ${CLUSTER_NAME}" echo "Service: ${SERVICE_NAME}" # Save to environment variables file cat > .env.deployment << EOF APP_NAME=${APP_NAME} STACK_NAME=${STACK_NAME} REGION=${REGION} CLUSTER_NAME=${CLUSTER_NAME} SERVICE_NAME=${SERVICE_NAME} CONTAINER_IMAGE=${CONTAINER_IMAGE} EOF echo "Environment variables saved to .env.deployment" ``` --- ## Stage 2: Post-Deployment Updates via AWS CLI ### Stage 2 Script: ARM64 + Fargate Spot + ECS Exec Enable ```bash #!/bin/bash # deploy-stage2.sh set -euo pipefail # Load environment variables source .env.deployment echo "=== Stage 2: ARM64 / Fargate Spot / ECS Exec Configuration ===" # ───────────────────────────────────────── # Step 2-1: Add Fargate Spot to Cluster # ───────────────────────────────────────── echo "[Step 2-1] Adding FARGATE_SPOT to cluster capacity providers..." aws ecs put-cluster-capacity-providers \ --cluster "${CLUSTER_NAME}" \ --region "${REGION}" \ --capacity-providers FARGATE FARGATE_SPOT \ --default-capacity-provider-strategy \ capacityProvider=FARGATE_SPOT,weight=3,base=0 \ capacityProvider=FARGATE,weight=1,base=1 echo "Fargate Spot capacity provider configured" # ───────────────────────────────────────── # Step 2-2: Create New Task Definition (ARM64) # ───────────────────────────────────────── echo "[Step 2-2] Creating ARM64 task definition..." # Retrieve existing task definition TASK_FAMILY="${APP_NAME}-task" CURRENT_TASK_DEF=$(aws ecs describe-task-definition \ --task-definition "${TASK_FAMILY}" \ --region "${REGION}" \ --query 'taskDefinition') EXECUTION_ROLE_ARN=$(echo "${CURRENT_TASK_DEF}" | jq -r '.executionRoleArn') TASK_ROLE_ARN=$(echo "${CURRENT_TASK_DEF}" | jq -r '.taskRoleArn') CPU=$(echo "${CURRENT_TASK_DEF}" | jq -r '.cpu') MEMORY=$(echo "${CURRENT_TASK_DEF}" | jq -r '.memory') CONTAINER_DEFS=$(echo "${CURRENT_TASK_DEF}" | jq '.containerDefinitions') # Create new task definition with ARM64 NEW_TASK_DEF=$(cat << EOF { "family": "${TASK_FAMILY}", "executionRoleArn": "${EXECUTION_ROLE_ARN}", "taskRoleArn": "${TASK_ROLE_ARN}", "networkMode": "awsvpc", "requiresCompatibilities": ["FARGATE"], "cpu": "${CPU}", "memory": "${MEMORY}", "runtimePlatform": { "cpuArchitecture": "ARM64", "operatingSystemFamily": "LINUX" }, "containerDefinitions": ${CONTAINER_DEFS} } EOF ) # Register task definition NEW_TASK_DEF_ARN=$(aws ecs register-task-definition \ --region "${REGION}" \ --cli-input-json "${NEW_TASK_DEF}" \ --query 'taskDefinition.taskDefinitionArn' \ --output text) echo "ARM64 task definition created: ${NEW_TASK_DEF_ARN}" # ───────────────────────────────────────── # Step 2-3: Update Service # - Switch to ARM64 task definition # - Enable ECS Exec # - Enable Fargate Spot # ───────────────────────────────────────── echo "[Step 2-3] Updating ECS service..." aws ecs update-service \ --cluster "${CLUSTER_NAME}" \ --service "${SERVICE_NAME}" \ --region "${REGION}" \ --task-definition "${NEW_TASK_DEF_ARN}" \ --enable-execute-command \ --capacity-provider-strategy \ capacityProvider=FARGATE_SPOT,weight=3,base=0 \ capacityProvider=FARGATE,weight=1,base=1 \ --deployment-configuration \ maximumPercent=200,minimumHealthyPercent=100 \ --force-new-deployment echo "Service update initiated" # ───────────────────────────────────────── # Step 2-4: Wait for Deployment Stability # ───────────────────────────────────────── echo "[Step 2-4] Waiting for service stability..." aws ecs wait services-stable \ --cluster "${CLUSTER_NAME}" \ --services "${SERVICE_NAME}" \ --region "${REGION}" echo "Service is stable" # ───────────────────────────────────────── # Step 2-5: Verify Configuration # ───────────────────────────────────────── echo "[Step 2-5] Verifying configuration..." # Verify service settings SERVICE_INFO=$(aws ecs describe-services \ --cluster "${CLUSTER_NAME}" \ --services "${SERVICE_NAME}" \ --region "${REGION}" \ --query 'services[0]') EXEC_ENABLED=$(echo "${SERVICE_INFO}" | jq -r '.enableExecuteCommand') CAPACITY_STRATEGY=$(echo "${SERVICE_INFO}" | jq -r '.capacityProviderStrategy') RUNNING_TASK_DEF=$(echo "${SERVICE_INFO}" | jq -r '.taskDefinition') echo "ECS Exec enabled: ${EXEC_ENABLED}" echo "Capacity provider strategy: ${CAPACITY_STRATEGY}" echo "Running task definition: ${RUNNING_TASK_DEF}" # Verify running tasks TASKS=$(aws ecs list-tasks \ --cluster "${CLUSTER_NAME}" \ --service-name "${SERVICE_NAME}" \ --region "${REGION}" \ --query 'taskArns' \ --output json) echo "Running tasks: ${TASKS}" # Verify architecture of each task for TASK_ARN in $(echo "${TASKS}" | jq -r '.[]'); do TASK_INFO=$(aws ecs describe-tasks \ --cluster "${CLUSTER_NAME}" \ --tasks "${TASK_ARN}" \ --region "${REGION}" \ --query 'tasks[0]') LAUNCH_TYPE=$(echo "${TASK_INFO}" | jq -r '.launchType // "CAPACITY_PROVIDER"') CAPACITY_PROVIDER=$(echo "${TASK_INFO}" | jq -r '.capacityProviderName // "N/A"') echo "Task: ${TASK_ARN##*/}" echo " Launch type: ${LAUNCH_TYPE}" echo " Capacity provider: ${CAPACITY_PROVIDER}" done echo "" echo "=== Stage 2 configuration complete ===" ``` --- ## Integrated Deployment Script ```bash #!/bin/bash # deploy-all.sh set -euo pipefail # ───────────────────────────────────────── # Configuration # ───────────────────────────────────────── APP_NAME="${1:-myapp}" REGION="${2:-ap-northeast-1}" CONTAINER_IMAGE="${3}" VPC_ID="${4}" SUBNET_IDS="${5}" # Validate required parameters if [[ -z "${CONTAINER_IMAGE}" || -z "${VPC_ID}" || -z "${SUBNET_IDS}" ]]; then echo "Usage: $0 <app-name> <region> <container-image> <vpc-id> <subnet-ids>" echo "Example: $0 myapp ap-northeast-1 123456789012.dkr.ecr.ap-northeast-1.amazonaws.com/myapp:latest vpc-xxx subnet-xxx,subnet-yyy" exit 1 fi STACK_NAME="${APP_NAME}-ecs-stack" echo "========================================" echo " ECS Two-Stage Deployment" echo "========================================" echo " App Name : ${APP_NAME}" echo " Region : ${REGION}" echo " Image : ${CONTAINER_IMAGE}" echo " VPC : ${VPC_ID}" echo " Subnets : ${SUBNET_IDS}" echo "========================================" # ───────────────────────────────────────── # Stage 1: CloudFormation Deployment # ───────────────────────────────────────── echo "" echo ">>> Stage 1: CloudFormation Base Deployment" aws cloudformation deploy \ --template-file ecs-express-base.yaml \ --stack-name "${STACK_NAME}" \ --region "${REGION}" \ --parameter-overrides \ AppName="${APP_NAME}" \ ContainerImage="${CONTAINER_IMAGE}" \ VpcId="${VPC_ID}" \ SubnetIds="${SUBNET_IDS}" \ --capabilities CAPABILITY_NAMED_IAM \ --no-fail-on-empty-changeset # Retrieve output values CLUSTER_NAME=$(aws cloudformation describe-stacks \ --stack-name "${STACK_NAME}" \ --region "${REGION}" \ --query 'Stacks[0].Outputs[?OutputKey==`ClusterName`].OutputValue' \ --output text) SERVICE_NAME=$(aws cloudformation describe-stacks \ --stack-name "${STACK_NAME}" \ --region "${REGION}" \ --query 'Stacks[0].Outputs[?OutputKey==`ServiceName`].OutputValue' \ --output text) echo "Stage 1 complete: Cluster=${CLUSTER_NAME}, Service=${SERVICE_NAME}" # ───────────────────────────────────────── # Stage 2: Advanced Configuration # ───────────────────────────────────────── echo "" echo ">>> Stage 2: ARM64 / Fargate Spot / ECS Exec Configuration" # 2-1: Add Fargate Spot capacity provider echo "Adding FARGATE_SPOT capacity provider..." aws ecs put-cluster-capacity-providers \ --cluster "${CLUSTER_NAME}" \ --region "${REGION}" \ --capacity-providers FARGATE FARGATE_SPOT \ --default-capacity-provider-strategy \ capacityProvider=FARGATE_SPOT,weight=3,base=0 \ capacityProvider=FARGATE,weight=1,base=1 # 2-2: Create ARM64 task definition echo "Creating ARM64 task definition..." TASK_FAMILY="${APP_NAME}-task" CURRENT_TASK_DEF=$(aws ecs describe-task-definition \ --task-definition "${TASK_FAMILY}" \ --region "${REGION}" \ --query 'taskDefinition') # Update RuntimePlatform to ARM64 ARM64_TASK_DEF=$(echo "${CURRENT_TASK_DEF}" | jq ' del(.taskDefinitionArn, .revision, .status, .requiresAttributes, .placementConstraints, .compatibilities, .registeredAt, .registeredBy) | .runtimePlatform.cpuArchitecture = "ARM64" ') NEW_TASK_DEF_ARN=$(aws ecs register-task-definition \ --region "${REGION}" \ --cli-input-json "${ARM64_TASK_DEF}" \ --query 'taskDefinition.taskDefinitionArn' \ --output text) echo "ARM64 task definition: ${NEW_TASK_DEF_ARN}" # 2-3: Update service echo "Updating service..." aws ecs update-service \ --cluster "${CLUSTER_NAME}" \ --service "${SERVICE_NAME}" \ --region "${REGION}" \ --task-definition "${NEW_TASK_DEF_ARN}" \ --enable-execute-command \ --capacity-provider-strategy \ capacityProvider=FARGATE_SPOT,weight=3,base=0 \ capacityProvider=FARGATE,weight=1,base=1 \ --force-new-deployment \ > /dev/null # 2-4: Wait for stability echo "Waiting for service to stabilize..." aws ecs wait services-stable \ --cluster "${CLUSTER_NAME}" \ --services "${SERVICE_NAME}" \ --region "${REGION}" # ───────────────────────────────────────── # Final Verification # ───────────────────────────────────────── echo "" echo ">>> Verifying deployment results" SERVICE_INFO=$(aws ecs describe-services \ --cluster "${CLUSTER_NAME}" \ --services "${SERVICE_NAME}" \ --region "${REGION}" \ --query 'services[0]') echo "Service status : $(echo ${SERVICE_INFO} | jq -r '.status')" echo "Running count : $(echo ${SERVICE_INFO} | jq -r '.runningCount')" echo "Desired count : $(echo ${SERVICE_INFO} | jq -r '.desiredCount')" echo "ECS Exec enabled : $(echo ${SERVICE_INFO} | jq -r '.enableExecuteCommand')" echo "Capacity strategy : $(echo ${SERVICE_INFO} | jq -c '.capacityProviderStrategy')" echo "Task definition : $(echo ${SERVICE_INFO} | jq -r '.taskDefinition')" echo "" echo "========================================" echo " Deployment Complete!" echo "========================================" echo "" echo "ECS Exec usage example:" echo " TASK_ARN=\$(aws ecs list-tasks --cluster ${CLUSTER_NAME} --service-name ${SERVICE_NAME} --query 'taskArns[0]' --output text)" echo " aws ecs execute-command --cluster ${CLUSTER_NAME} --task \${TASK_ARN} --container ${APP_NAME} --interactive --command '/bin/sh'" ``` --- ## Troubleshooting ### ECS Exec Not Working ```bash #!/bin/bash # troubleshoot-ecs-exec.sh CLUSTER_NAME="${1}" SERVICE_NAME="${2}" REGION="${3:-ap-northeast-1}" echo "=== ECS Exec Diagnostics ===" # Get task ARN TASK_ARN=$(aws ecs list-tasks \ --cluster "${CLUSTER_NAME}" \ --service-name "${SERVICE_NAME}" \ --region "${REGION}" \ --query 'taskArns[0]' \ --output text) echo "Target task: ${TASK_ARN}" # Check prerequisites with amazon-ecs-exec-checker echo "" echo "Checking prerequisites..." TASK_ID="${TASK_ARN##*/}" # Verify task role TASK_DEF_ARN=$(aws ecs describe-tasks \ --cluster "${CLUSTER_NAME}" \ --tasks "${TASK_ARN}" \ --region "${REGION}" \ --query 'tasks[0].taskDefinitionArn' \ --output text) TASK_ROLE_ARN=$(aws ecs describe-task-definition \ --task-definition "${TASK_DEF_ARN}" \ --region "${REGION}" \ --query 'taskDefinition.taskRoleArn' \ --output text) echo "Task role: ${TASK_ROLE_ARN}" # Check ssmmessages permissions echo "" echo "Checking ssmmessages permissions..." aws iam simulate-principal-policy \ --policy-source-arn "${TASK_ROLE_ARN}" \ --action-names \ ssmmessages:CreateControlChannel \ ssmmessages:CreateDataChannel \ ssmmessages:OpenControlChannel \ ssmmessages:OpenDataChannel \ --query 'EvaluationResults[].{Action:EvalActionName,Decision:EvalDecision}' \ --output table # Verify ECS Exec is enabled EXEC_ENABLED=$(aws ecs describe-services \ --cluster "${CLUSTER_NAME}" \ --services "${SERVICE_NAME}" \ --region "${REGION}" \ --query 'services[0].enableExecuteCommand' \ --output text) echo "" echo "ECS Exec enabled: ${EXEC_ENABLED}" if [[ "${EXEC_ENABLED}" == "false" ]]; then echo "WARNING: ECS Exec is disabled. Enabling..." aws ecs update-service \ --cluster "${CLUSTER_NAME}" \ --service "${SERVICE_NAME}" \ --region "${REGION}" \ --enable-execute-command \ --force-new-deployment fi ``` ### Fargate Spot Interruption Handling ```bash # Monitor Spot interruption events aws events put-rule \ --name "fargate-spot-interruption" \ --event-pattern '{ "source": ["aws.ecs"], "detail-type": ["ECS Task State Change"], "detail": { "stopCode": ["SpotInterruption"] } }' \ --state ENABLED \ --region "${REGION}" ``` --- ## Key Points Summary | Feature | Stage 1 (CloudFormation) | Stage 2 (AWS CLI) | |---|---|---| | ECS Cluster | Created | Capacity provider update | | Task Definition | x86_64 | ARM64 re-registration | | ECS Service | Created (Exec disabled) | Exec enabled | | Fargate Spot | Not configured | Capacity strategy set | | IAM Roles | Created | No change required | By using this two-stage deployment process, you can leverage ECS Express Mode's simplicity while applying advanced configurations such as ARM64, Fargate Spot, and ECS Exec.