I tried running After Effects with AWS Deadline Cloud's CMF + my own S3 conda channel

Introduction

In render farm operations for DCC (Digital Content Creation) applications used in 3DCG / VFX production, it is common to increase the number of DCC license contracts in proportion to the number of Workers. AWS Deadline Cloud has a mechanism called UBL (Usage-Based Licensing) that can supply licenses with pay-as-you-go billing based on job execution time, and it is available by default in SMF (Service-Managed Fleet). The official documentation states that UBL can also be used in CMF (Customer-Managed Fleet) via a License Endpoint.

However, Adobe After Effects (AE) is not subject to UBL. The official Adobe help explains that since After Effects CS6, aerender or Watch Folder can be run in non-royalty bearing mode (render-only mode), and serialization on render-only machines is not required. Therefore, the UBL framework itself does not appear for AE.

On the other hand, running AE on CMF presents a different challenge. deadline-cloud-for-after-effects does not provide a Worker-side CLI in the same form as the adaptor packages for 3ds Max or Maya, and in the configuration of this article, the AE binary is supplied to Workers as a conda package. AWS provides AE conda packages for SMF from the deadline-cloud channel, but CMF Workers cannot access it. Therefore, when supplying AE to CMF Workers following the flow of the official AWS AE Submitter and conda Queue Environment, a configuration where AE conda packages are placed in a self-managed S3 conda channel is practical.

This article shares the results of confirming that AE rendering works on actual hardware in this "CMF + self-managed S3 conda channel" configuration, along with 3 insights gained during verification.

What is AWS Deadline Cloud

AWS Deadline Cloud is a managed service for building render farms in the cloud for 3DCG / VFX production. It provides the features necessary for render farm operations, including Worker auto-scaling, job management, and license supply.

Target Audience

• Those who are building or considering building a CMF with AWS Deadline Cloud
• Those who want to build an Adobe After Effects render farm on AWS
• Those considering a render farm configuration using AE's render-only mode on AWS

References

• AWS Deadline Cloud Developer Guide: Using software licenses
• AWS Deadline Cloud Developer Guide: Create a conda channel using S3
• aws-deadline/deadline-cloud-for-after-effects
• aws-deadline/deadline-cloud-samples: aftereffects-25.1 recipe
• aws-deadline/deadline-cloud-samples: queue_environments
• Adobe After Effects CS6 Official Help PDF (non-royalty bearing mode, p.605)

Verification Goals and Configuration

The verification had the following 4 goals, all of which needed to be satisfied.

1. Jobs submitted from the AE Submitter plugin run to SUCCEEDED on Worker EC2
2. aerender starts on the Worker during job execution without requiring Adobe sign-in
3. The rendered PNG sequence can be retrieved via S3
4. The Worker can resolve and install the AE conda package from the self-managed S3 conda channel

The verification configuration is as follows. The Submitter uses macOS with AE Trial and the Submitter plugin, and on the AWS side, one Render Worker EC2 (Windows Server 2022) and an S3 bucket with a self-managed conda channel were prepared. AE 2026 was used in the verification.

In addition to Python and the Worker Agent, the Render Worker needs py-rattler for conda package resolution, Git for Windows for calling bash inside the Queue Environment, and ae_render_only_node.txt to enable Adobe non-royalty bearing mode, all pre-installed. Since jobs will fail if these prerequisites are not in place, Worker setup plays an important role in the build flow.

Verification Results

As a verification scene, a minimal AEP project with a single solid layer in a 640 x 480 / 24fps / 1 second composition was prepared, and a job was submitted from the Submitter plugin. The output module was set to PNG Sequence.

The job status transitioned from READY through STARTING and RUNNING to SUCCEEDED, with 3 tasks completing in 36 seconds. The job output was retrieved locally with deadline job download-output, yielding 24 PNG frames in sequence.

In this verification, a single-color gray PNG like the following was output.

Checking CloudWatch Logs confirmed that aerender on the Worker output Finished composition and Total Time Elapsed: 1 Seconds before exiting with No errors detected, exiting with process return code: 0, and that no Adobe sign-in prompt appeared.

It was confirmed on actual hardware that AE on CMF Fleet can resolve a conda package from a self-managed S3 conda channel and render in non-royalty bearing mode.

Overall Flow

The verification proceeded in the following 6 Phases. The purpose and what was done in each Phase is briefly described below. For detailed procedures, please refer to the reference links and official documentation.

Phase 1. AWS Resource Construction

Farm / Queue / Fleet / Queue Environment / Storage Profile / IAM roles / S3 bucket / Security Groups / Key Pair are all created together using the AWS CLI. When creating a new CMF Fleet, the Fleet Worker Role specified needs to have an inline policy for CloudWatch Logs permissions manually attached in addition to the managed policy AWSDeadlineCloud-FleetWorker. This differs from SMF, and details are covered in our previous article Trying to run 3ds Max with AWS Deadline Cloud CMF + UBL.

Phase 2. Obtaining the AE Binary and Uploading to S3

A Windows Server 2022 EC2 is temporarily launched, AE Trial is installed from the Creative Cloud Desktop App, the AE Support Files folder is zipped, and uploaded to the archive_files/ prefix in S3. This follows the same procedure as the aftereffects-25.1 recipe README provided by AWS.

In this verification, an Adobe account was only required at this stage. It was confirmed that no Adobe sign-in prompt appears when rendering on the Render Worker.

Phase 3. Building the conda Package and Placing It in the S3 Channel

Miniforge and conda-build are installed on local macOS, and an aftereffects-26.0 recipe is created based on the official AWS aftereffects-25.1 recipe. The version and sha256 in meta.yaml are updated, the conda package is built targeting win-64, and uploaded to the S3 channel.

Normally, building via the AWS Deadline Cloud Package Build Queue is the officially recommended route, but AE conda packages do not involve native code compilation (only copying Support Files and generating activate.d scripts), making cross-building from macOS possible. This method was adopted in this verification.

The S3 conda channel is created as a private bucket, with read permissions for s3://<your-bucket>/Conda/Default granted to the Queue service role and the Worker role. The official AWS S3 conda channel procedure also shows an example of giving production queues read-only permissions to the /Conda prefix.

Phase 4. Render Worker EC2 Setup

A Render Worker EC2 (Windows Server 2022) is launched, and Python 3.11, Worker Agent, py-rattler, Git for Windows, and ae_render_only_node.txt under Administrator Documents are installed in sequence. py-rattler is version-pinned to >=0.18,<0.19. The 0.19 series did not match the API calls on the Queue Environment side, so the 0.18 series was adopted in this verification. When registering the Worker Agent service, run_jobs_as_agent_user=true in worker.toml is enabled, and the Worker Agent service is restarted to join the Fleet in IDLE state.

Phase 5. macOS Submitter Setup

AE 2026 Trial is installed from the Creative Cloud Desktop App on the local macOS, and AE script execution permissions (Allow Scripts To Write Files And Access Network) are enabled. The AWS Deadline Cloud Submitter installer is run to install the AE Submitter plugin, and the deadline CLI was installed in isolation in a dedicated venv (~/venvs/ae-submitter).

Phase 6. Job Submission and Verification

The verification AEP was opened in AE, added to the Render Queue, then the Submitter UI was launched and the job was submitted.

Job status transitions and aerender behavior can be monitored with CloudWatch Logs and aws deadline list-jobs.

Insights Gained from Verification

deadline-cloud-for-after-effects does not provide a Worker-side CLI. Workers obtain aerender via conda packages

At the start of verification, it was assumed that, similar to adaptor packages for 3ds Max and Maya, pip installing deadline-cloud-for-after-effects would add a Worker-side CLI to PATH. In the 3ds Max verification, pip installing deadline-cloud-for-3ds-max added 3dsmax-openjd.exe to PATH, and this was the form in which jobs were executed on the Worker.

However, checking pyproject.toml reveals that the [project.scripts] section is empty and no Worker-side CLI is provided at all. deadline-cloud-for-after-effects operates only as an AE Submitter on the Submitter side, and the created job bundle is designed to use aerender bundled with AE on the Worker side.

The route that worked in this verification is a design where the AE Submitter embeds a CondaPackages parameter (e.g., aftereffects=26.0) in the OpenJD job template, and the Worker resolves the AE conda package from the conda channel via the Queue Environment to launch aerender. The contents of the AE conda package are a zip of the AE Trial installation. AWS provides AE 24.6 / 25.1 / 25.2 / 25.6 / 26.0 for SMF from the deadline-cloud channel, but since CMF Workers cannot access it, you need to create your own AE conda package.

The pyrattler version Queue Environment from AWS Samples does not work on Windows Workers

The conda_queue_env_pyrattler.yaml in AWS Deadline Cloud Samples is a Queue Environment that can resolve conda packages via py-rattler without installing conda itself on the Worker. Since it reduces Worker setup, it was initially assumed that attaching this as-is would work.

However, when running a job on a Windows Worker, it fails at the conda env activation stage with FileNotFoundError: The system cannot find the file specified. The cause is that the Python script inside the Queue Environment calls bash via subprocess.check_call(["bash", capture_script]) during activation, and within capture_script, further Python is called using sys.executable. Since the Worker Agent starts Python with the relative path .\python.EXE, sys.executable is also propagated as a relative path, and when called from bash from a different CWD, the file cannot be found and it fails.

The route that worked in this verification was adding a fallback to absolutize sys.executable in the Python script inside the Queue Environment. The full patch is included in the appendix at the end of this article. Since no publicly available cases with this insight were found at the time of writing this article, I hope it will be helpful for those who get stuck with the same configuration.

macOS can cross-build conda packages for Windows, but adding build.sh is required

For building AE conda packages, the README in AWS Deadline Cloud Samples introduces a method using the Package Build Queue (a configuration with a dedicated Build Fleet and Build Worker EC2). Initially, the plan was to follow this method.

However, the contents of the AE conda package are simply copying AE Support Files and generating activate.d scripts, with no native code compilation occurring. Therefore, cross-building is possible even from macOS by specifying CONDA_SUBDIR=win-64.

However, the official AWS AE recipe only contains bld.bat (Windows wrapper) and build_win.sh (bash body), and macOS conda-build cannot find a build script and generates an empty package (displaying WARNING: No files or script found for output).

The workaround adopted this time was simply copying build_win.sh as build.sh. macOS conda-build detects this and executes it with bash, correctly generating the win-64 conda package. Build time is about 30 minutes, significantly shorter compared to via the Package Build Queue (90 - 120 minutes including Build Worker EC2 startup and configuration).

cp aftereffects-26.0/recipe/build_win.sh aftereffects-26.0/recipe/build.sh
chmod +x aftereffects-26.0/recipe/build.sh
CONDA_SUBDIR=win-64 conda-build aftereffects-26.0/recipe --no-test --output-folder ./conda-build-output

The output is generated as conda-build-output/win-64/aftereffects-26.0-0.conda (approximately 3.9 GB) and can be placed in the S3 conda channel using aws s3 sync.

Summary

It was confirmed on actual hardware that AE rendering works with the combination of AWS Deadline Cloud CMF and a self-managed S3 conda channel. In this verification, it was confirmed that aerender completes without an Adobe sign-in prompt on a Worker with ae_render_only_node.txt placed. Since this verification covers only a single minimal composition, confirming configurations that include plugins such as Pencil+ and V-Ray, multi-frame rendering, and Fleet scaling behavior will be the next steps. I hope this article will be helpful when considering CMF and AE configurations with AWS Deadline Cloud.

Appendix: Scripts and Patches Created During Verification

The custom scripts and patches created during verification that cannot be used as-is from existing public information are listed below in collapsed sections.

import shutil as _shutil
_py_exe = sys.executable
if not os.path.isfile(_py_exe):
    for _cand in (
        _shutil.which("python.exe"),
        _shutil.which("python"),
        r"C:\Program Files\Python311\python.exe",
    ):
        if _cand and os.path.isfile(_cand):
            _py_exe = _cand
            break
with open(capture_script, "w", encoding="utf8") as f:
    f.write(activation.script)
    f.write("\n")
    f.write(shlex.join([_py_exe, vars_capture_script, vars_file]))
    f.write("\n")

# Run in the recipe directory
cp aftereffects-26.0/recipe/build_win.sh aftereffects-26.0/recipe/build.sh
chmod +x aftereffects-26.0/recipe/build.sh

mkdir -p archive_files
aws s3 cp s3://<your-bucket>/archive_files/Adobe_AfterEffects_26_0_Windows_installation.zip archive_files/

mkdir -p conda-build-output
CONDA_SUBDIR=win-64 conda-build aftereffects-26.0/recipe --no-test --output-folder ./conda-build-output

aws s3 sync ./conda-build-output/ s3://<your-bucket>/Conda/Default/

# 1. Python 3.11 silent install
$pyUrl = "https://www.python.org/ftp/python/3.11.9/python-3.11.9-amd64.exe"
$pyInstaller = "$env:TEMP\python-3.11.9-amd64.exe"
Invoke-WebRequest -Uri $pyUrl -OutFile $pyInstaller
Start-Process -FilePath $pyInstaller -ArgumentList "/quiet","InstallAllUsers=1","PrependPath=1","Include_pip=1","Include_launcher=1" -Wait
$env:Path = [System.Environment]::GetEnvironmentVariable("Path", "Machine") + ";" + [System.Environment]::GetEnvironmentVariable("Path", "User")

# 2. boto3 region configuration
New-Item -ItemType Directory -Force -Path "$env:USERPROFILE\.aws" | Out-Null
@"
[default]
region = ap-northeast-1
output = json
"@ | Out-File -Encoding ASCII "$env:USERPROFILE\.aws\config"

# 3. Worker Agent + py-rattler install
python -m pip install --upgrade pip
python -m pip install deadline-cloud-worker-agent "py-rattler>=0.18,<0.19" "pyyaml>=6,<7"

# 4. Git for Windows silent install + Machine PATH addition
$gitUrl = "https://github.com/git-for-windows/git/releases/download/v2.45.2.windows.1/Git-2.45.2-64-bit.exe"
$gitInstaller = "$env:TEMP\GitInstaller.exe"
Invoke-WebRequest -Uri $gitUrl -OutFile $gitInstaller
Start-Process -FilePath $gitInstaller -ArgumentList "/VERYSILENT","/NORESTART","/SUPPRESSMSGBOXES" -Wait

$existingPath = [System.Environment]::GetEnvironmentVariable("Path", "Machine")
[System.Environment]::SetEnvironmentVariable("Path", "C:\Program Files\Git\bin;C:\Program Files\Git\usr\bin;$existingPath", "Machine")

# 5. Place Adobe non-royalty bearing mode activation marker
New-Item -Path "C:\Users\Administrator\Documents\ae_render_only_node.txt" -ItemType File -Force