Prevent Forgetting to Stop SageMaker Real-time Endpoints Launched from Vercel by Automatic Deletion

I was able to automatically delete a SageMaker Real-time Endpoint launched from a Next.js app on Vercel even after closing the browser. In additional verification, I set the stop deadline to 1,800 seconds, which allowed approximately 30 minutes from when the model became available.

越井琢巳 (Koshii Takumi)

2026.07.31

This page has been translated by machine translation. View original

Introduction

With a configuration that starts a SageMaker Real-time Endpoint only when needed from a Next.js app deployed on Vercel, you can avoid the cost of running a GPU 24/7. However, if a user forgets to stop it, the Endpoint keeps running even after they close their browser.

Screen showing a self-hosted model being operated in the Next.js verification app on Vercel

Therefore, I added a mechanism on the AWS side to automatically delete the Endpoint even after the browser is closed and Vercel Functions have finished executing. This article introduces that procedure.

Target Audience

Those who want to start a SageMaker Real-time Endpoint only when needed from a Next.js app deployed on Vercel
Those who want to avoid the cost of running a GPU 24/7
Those who do not want to rely on the browser or serverless functions for shutdown processing

Verification Environment

Model: lmstudio-community/Qwen3.6-35B-A3B-GGUF
File: Qwen3.6-35B-A3B-Q4_K_M.gguf
Instance: ml.g5.2xlarge
Region: us-west-2
Terraform: 1.15.8
AWS Provider: 6.49.0
Monitoring interval: 1 minute

References

Verification Configuration

On the Next.js side, starting the Endpoint, checking status, streaming generation, and stopping are separated into different requests. It is not necessary to maintain an HTTP connection for generation while the model is starting up.

Screen showing SageMaker Endpoint status being checked in the Next.js verification app

For automatic shutdown, instead of a timer on the application side, I used a method that calls Lambda every 1 minute from EventBridge Scheduler.

A browser timer cannot guarantee shutdown after a tab is closed. A method of keeping Vercel Functions waiting is also affected by function termination and redeployments. By periodically reconciling state on the AWS side, processing can continue even if the user's device and application server have shut down.

The shutdown deadline is calculated by adding the maximum session time to the CreationTime returned by SageMaker. Even if an Endpoint is recreated with the same name, the CreationTime is updated, so it does not inherit the previous creation time.

Implementing Automatic Shutdown

Lambda calls DescribeEndpoint and calculates the elapsed time from the difference between the current time and CreationTime. If the maximum session time has not been reached, the Endpoint is maintained.

lambda_function.py

description = client.describe_endpoint(EndpointName=endpoint_name)
status = description["EndpointStatus"]
creation_time = _as_utc(description["CreationTime"])
age_seconds = max(0, int((now - creation_time).total_seconds()))

if status == "Deleting":
    return {"action": "deletion_pending"}
if age_seconds < max_session_seconds:
    return {"action": "retained"}

When the deadline is reached, DeleteEndpoint is called. If the Endpoint no longer exists because a manual shutdown overlapped, it is treated as already deleted.

lambda_function.py

try:
    client.delete_endpoint(EndpointName=endpoint_name)
except Exception as error:
    if is_missing_error(error):
        return {"action": "absent_after_race"}
    raise
return {"action": "delete_requested"}

Normal termination also occurs when the Endpoint does not exist at the time periodic reconciliation starts. When the status is Deleting, deletion is not attempted again. If DeleteEndpoint fails for any other reason, an exception is returned, allowing reconciliation again via Lambda's asynchronous execution retry or the next scheduled execution.

The EventBridge Scheduler was set to a 1-minute interval. A delay of up to about 1 minute from deadline reached to delete request is acceptable.

main.tf

resource "aws_scheduler_schedule" "reconciler" {
  schedule_expression = "rate(1 minute)"
  state               = "ENABLED"

  flexible_time_window {
    mode = "OFF"
  }

  target {
    arn      = aws_lambda_function.reconciler.arn
    role_arn = aws_iam_role.scheduler.arn

    retry_policy {
      maximum_event_age_in_seconds = 60
      maximum_retry_attempts       = 2
    }
  }
}

Lambda's permissions were limited to DescribeEndpoint and DeleteEndpoint for the target Endpoint, and dedicated CloudWatch Logs only.

Results

First, I set the shutdown deadline to 900 seconds after creating the Qwen3.6 Endpoint. After confirming normal generation and client cancel, I closed the browser. After that, DeleteEndpoint was not called from the local verification process.

Item	Result
Endpoint creation to ready	312.363 seconds
Warm TTFT	1.838 seconds
Normal generation completion	2.625 seconds
Client cancel	0.612 seconds
Automatic shutdown deadline	900 seconds after Endpoint creation
Lambda delete request	914 seconds after Endpoint creation
Endpoint absent	20.488 seconds after shutdown deadline
`DeleteEndpoint` from verification process	0 times
Conservative GPU cost	approximately 0.387 USD

Even after closing the browser, Scheduler and Lambda continued to operate, and the Endpoint was deleted after the deadline.

What I Learned

After testing several more times, I found that when using the Endpoint creation time as the starting point for the deadline, the time the model is available is affected by how long startup takes.

In another test where I set the shutdown deadline to 1,800 seconds, the Endpoint remained in Creating state beyond 1,800 seconds, and DeleteEndpoint was rejected with the following error.

Cannot update in-progress endpoint

After the Endpoint transitioned to InService, Lambda requested deletion at 2,114 seconds after creation. It took 2,175 seconds from creation to confirming the Endpoint was absent.

Additional Verification Based on Completion of Startup

Therefore, to make the time the model is available consistent, I conducted additional verification using the time Lambda first observes InService as the starting point for the deadline.

The starting time, shutdown deadline, and Endpoint generation were saved to DynamoDB. The SageMaker CreationTime is recorded as the generation to prevent an Endpoint recreated with the same name from inheriting the previous shutdown deadline.

In the additional verification, the shutdown deadline was set to 1,800 seconds after the first InService observation. Lambda requested deletion 1,859 seconds after the observation, and the Endpoint became absent about 6 seconds later. Approximately 30 minutes of availability was secured after becoming available.

Summary

Using EventBridge Scheduler and Lambda, I was able to automatically delete a SageMaker Real-time Endpoint even after closing the browser. With a configuration that calculates the deadline from SageMaker's CreationTime, no data store for saving the shutdown deadline is needed.

To make the time the model is available consistent, I also verified a method that uses the first InService observation as the starting point for the deadline. In the additional verification with the shutdown deadline set to 1,800 seconds, automatic deletion was achieved after securing approximately 30 minutes of availability.

Appendix

The appendix includes minimal implementations for both the creation-time-based and InService-based approaches.

Reproduction Conditions

Item	Value
Repository	`lmstudio-community/Qwen3.6-35B-A3B-GGUF`
Revision	`c7ed48a94a6a082167b768bab350745695824e0a`
File	`Qwen3.6-35B-A3B-Q4_K_M.gguf`
Size	21,166,757,728 bytes
SHA-256	`4ac6a06bce551257267f49ad2226f8671a22519ccc1a4dde9d5b433d1f2a410d`
llama.cpp	`86a9c79f866799eb0e7e89c03578ccfbcc5d808e`
Region	`us-west-2`
Instance	`ml.g5.2xlarge`
Context	16,384
Parallel slot	1

Creation-Time-Based Lambda

lambda_function.py

import json
import os
from datetime import datetime, timezone

def _as_utc(value):
    if value.tzinfo is None:
        return value.replace(tzinfo=timezone.utc)
    return value.astimezone(timezone.utc)

def _is_missing_endpoint(error):
    details = getattr(error, "response", {}).get("Error", {})
    return (
        details.get("Code") == "ResourceNotFound"
        or (
            details.get("Code") == "ValidationException"
            and "Could not find endpoint" in details.get("Message", "")
        )
    )

def reconcile_endpoint(
    *,
    client,
    endpoint_name,
    max_session_seconds,
    now,
    is_missing_error=_is_missing_endpoint,
):
    try:
        description = client.describe_endpoint(EndpointName=endpoint_name)
    except Exception as error:
        if is_missing_error(error):
            return {"action": "absent"}
        raise

    status = description["EndpointStatus"]
    creation_time = _as_utc(description["CreationTime"])
    age_seconds = max(0, int((now - creation_time).total_seconds()))
    result = {
        "endpoint_status": status,
        "age_seconds": age_seconds,
        "max_session_seconds": max_session_seconds,
    }

    if status == "Deleting":
        return {**result, "action": "deletion_pending"}
    if age_seconds < max_session_seconds:
        return {**result, "action": "retained"}

    try:
        client.delete_endpoint(EndpointName=endpoint_name)
    except Exception as error:
        if is_missing_error(error):
            return {**result, "action": "absent_after_race"}
        raise
    return {**result, "action": "delete_requested"}

def handler(event, context):
    del event, context
    import boto3

    result = reconcile_endpoint(
        client=boto3.client("sagemaker"),
        endpoint_name=os.environ["ENDPOINT_NAME"],
        max_session_seconds=int(os.environ["MAX_SESSION_SECONDS"]),
        now=datetime.now(timezone.utc),
    )
    print(json.dumps(result, sort_keys=True, separators=(",", ":")))
    return result

Creation-Time-Based Terraform

versions.tf

terraform {
  required_version = ">= 1.15.0, < 2.0.0"

  required_providers {
    archive = {
      source  = "hashicorp/archive"
      version = "~> 2.7"
    }
    aws = {
      source  = "hashicorp/aws"
      version = "~> 6.49"
    }
  }
}

variables.tf

variable "endpoint_name" {
  type = string
}

variable "name_prefix" {
  type    = string
  default = "sagemaker-autostop-poc"
}

variable "max_session_seconds" {
  type    = number
  default = 900
}

main.tf

data "aws_caller_identity" "current" {}
data "aws_partition" "current" {}
data "aws_region" "current" {}

locals {
  function_name    = "${var.name_prefix}-session-autostop"
  endpoint_arn     = "arn:${data.aws_partition.current.partition}:sagemaker:${data.aws_region.current.region}:${data.aws_caller_identity.current.account_id}:endpoint/${var.endpoint_name}"
  lambda_log_group = "/aws/lambda/${local.function_name}"
}

data "archive_file" "lambda" {
  type        = "zip"
  source_file = "${path.module}/lambda_function.py"
  output_path = "${path.module}/.terraform/${local.function_name}.zip"
}

resource "aws_cloudwatch_log_group" "lambda" {
  name              = local.lambda_log_group
  retention_in_days = 14
}

data "aws_iam_policy_document" "lambda_assume" {
  statement {
    actions = ["sts:AssumeRole"]

    principals {
      type        = "Service"
      identifiers = ["lambda.amazonaws.com"]
    }
  }
}

resource "aws_iam_role" "lambda" {
  name               = "${local.function_name}-execution"
  assume_role_policy = data.aws_iam_policy_document.lambda_assume.json
}

data "aws_iam_policy_document" "lambda" {
  statement {
    actions = [
      "sagemaker:DeleteEndpoint",
      "sagemaker:DescribeEndpoint",
    ]
    resources = [local.endpoint_arn]
  }

  statement {
    actions = [
      "logs:CreateLogStream",
      "logs:PutLogEvents",
    ]
    resources = ["${aws_cloudwatch_log_group.lambda.arn}:*"]
  }
}

resource "aws_iam_role_policy" "lambda" {
  name   = "${local.function_name}-policy"
  role   = aws_iam_role.lambda.id
  policy = data.aws_iam_policy_document.lambda.json
}

resource "aws_lambda_function" "reconciler" {
  function_name                  = local.function_name
  role                           = aws_iam_role.lambda.arn
  handler                        = "lambda_function.handler"
  runtime                        = "python3.13"
  architectures                  = ["arm64"]
  filename                       = data.archive_file.lambda.output_path
  source_code_hash               = data.archive_file.lambda.output_base64sha256
  memory_size                    = 128
  timeout                        = 30
  reserved_concurrent_executions = 1

  environment {
    variables = {
      ENDPOINT_NAME       = var.endpoint_name
      MAX_SESSION_SECONDS = tostring(var.max_session_seconds)
    }
  }

  depends_on = [
    aws_cloudwatch_log_group.lambda,
    aws_iam_role_policy.lambda,
  ]
}

resource "aws_lambda_function_event_invoke_config" "reconciler" {
  function_name                = aws_lambda_function.reconciler.function_name
  maximum_event_age_in_seconds = 60
  maximum_retry_attempts       = 2
}

data "aws_iam_policy_document" "scheduler_assume" {
  statement {
    actions = ["sts:AssumeRole"]

    principals {
      type        = "Service"
      identifiers = ["scheduler.amazonaws.com"]
    }
  }
}

resource "aws_iam_role" "scheduler" {
  name               = "${var.name_prefix}-session-scheduler"
  assume_role_policy = data.aws_iam_policy_document.scheduler_assume.json
}

data "aws_iam_policy_document" "scheduler" {
  statement {
    actions   = ["lambda:InvokeFunction"]
    resources = [aws_lambda_function.reconciler.arn]
  }
}

resource "aws_iam_role_policy" "scheduler" {
  name   = "${var.name_prefix}-session-scheduler-policy"
  role   = aws_iam_role.scheduler.id
  policy = data.aws_iam_policy_document.scheduler.json
}

resource "aws_scheduler_schedule_group" "reconciler" {
  name = "${var.name_prefix}-session"
}

resource "aws_scheduler_schedule" "reconciler" {
  name       = "${var.name_prefix}-session-reconcile"
  group_name = aws_scheduler_schedule_group.reconciler.name
  state      = "ENABLED"

  flexible_time_window {
    mode = "OFF"
  }

  schedule_expression = "rate(1 minute)"

  target {
    arn      = aws_lambda_function.reconciler.arn
    role_arn = aws_iam_role.scheduler.arn

    retry_policy {
      maximum_event_age_in_seconds = 60
      maximum_retry_attempts       = 2
    }
  }
}

resource "aws_lambda_permission" "scheduler" {
  statement_id  = "AllowEventBridgeScheduler"
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.reconciler.function_name
  principal     = "scheduler.amazonaws.com"
  source_arn    = aws_scheduler_schedule.reconciler.arn
}

`InService`-Based Lambda

InService-based lambda_function.py

lambda_function.py

import json
import os
from datetime import datetime, timedelta, timezone

def _as_utc(value):
    if value.tzinfo is None:
        return value.replace(tzinfo=timezone.utc)
    return value.astimezone(timezone.utc)

def _generation(creation_time):
    return _as_utc(creation_time).isoformat()

def _is_missing_endpoint(error):
    details = getattr(error, "response", {}).get("Error", {})
    return (
        details.get("Code") == "ResourceNotFound"
        or (
            details.get("Code") == "ValidationException"
            and "Could not find endpoint" in details.get("Message", "")
        )
    )

class DynamoDbStateStore:
    def __init__(self, client, table_name):
        self.client = client
        self.table_name = table_name

    def initialize_ready(
        self,
        *,
        endpoint_name,
        generation,
        ready_at,
        deadline_at,
    ):
        try:
            self.client.put_item(
                TableName=self.table_name,
                Item={
                    "EndpointName": {"S": endpoint_name},
                    "Generation": {"S": generation},
                    "ReadyAt": {"S": ready_at.isoformat()},
                    "DeadlineAt": {"S": deadline_at.isoformat()},
                },
                ConditionExpression=(
                    "attribute_not_exists(#name) "
                    "OR #generation <> :generation"
                ),
                ExpressionAttributeNames={
                    "#name": "EndpointName",
                    "#generation": "Generation",
                },
                ExpressionAttributeValues={
                    ":generation": {"S": generation},
                },
            )
        except self.client.exceptions.ConditionalCheckFailedException:
            pass

    def get(self, endpoint_name):
        response = self.client.get_item(
            TableName=self.table_name,
            Key={"EndpointName": {"S": endpoint_name}},
            ConsistentRead=True,
        )
        item = response.get("Item")
        if not item:
            return None
        return {
            "generation": item["Generation"]["S"],
            "ready_at": datetime.fromisoformat(item["ReadyAt"]["S"]),
            "deadline_at": datetime.fromisoformat(item["DeadlineAt"]["S"]),
        }

def reconcile_endpoint(
    *,
    client,
    state_store,
    endpoint_name,
    max_session_seconds,
    now,
    is_missing_error=_is_missing_endpoint,
):
    try:
        description = client.describe_endpoint(EndpointName=endpoint_name)
    except Exception as error:
        if is_missing_error(error):
            return {"action": "absent"}
        raise

    status = description["EndpointStatus"]
    generation = _generation(description["CreationTime"])

    if status == "Deleting":
        return {"action": "deletion_pending"}
    if status != "InService":
        return {"action": "provisioning", "endpoint_status": status}

    state_store.initialize_ready(
        endpoint_name=endpoint_name,
        generation=generation,
        ready_at=now,
        deadline_at=now + timedelta(seconds=max_session_seconds),
    )
    state = state_store.get(endpoint_name)
    if state is None or state["generation"] != generation:
        return {"action": "stale_generation"}
    if now < state["deadline_at"]:
        return {"action": "retained"}

    try:
        current = client.describe_endpoint(EndpointName=endpoint_name)
    except Exception as error:
        if is_missing_error(error):
            return {"action": "absent_after_race"}
        raise

    if _generation(current["CreationTime"]) != generation:
        return {"action": "stale_generation"}
    if current["EndpointStatus"] != "InService":
        return {"action": "status_changed_before_delete"}

    try:
        client.delete_endpoint(EndpointName=endpoint_name)
    except Exception as error:
        if is_missing_error(error):
            return {"action": "absent_after_race"}
        raise
    return {"action": "delete_requested"}

def handler(event, context):
    del event, context
    import boto3

    result = reconcile_endpoint(
        client=boto3.client("sagemaker"),
        state_store=DynamoDbStateStore(
            boto3.client("dynamodb"),
            os.environ["STATE_TABLE_NAME"],
        ),
        endpoint_name=os.environ["ENDPOINT_NAME"],
        max_session_seconds=int(os.environ["MAX_SESSION_SECONDS"]),
        now=datetime.now(timezone.utc),
    )
    print(json.dumps(result, sort_keys=True, separators=(",", ":")))
    return result

Terraform for `InService` Criteria

variables.tf for InService criteria

variables.tf

variable "endpoint_name" {
  type = string
}

variable "name_prefix" {
  type    = string
  default = "sagemaker-ready-autostop-poc"
}

variable "max_session_seconds" {
  type    = number
  default = 1800
}

main.tf for InService criteria

main.tf

data "aws_caller_identity" "current" {}
data "aws_partition" "current" {}
data "aws_region" "current" {}

locals {
  function_name    = "${var.name_prefix}-session-autostop"
  endpoint_arn     = "arn:${data.aws_partition.current.partition}:sagemaker:${data.aws_region.current.region}:${data.aws_caller_identity.current.account_id}:endpoint/${var.endpoint_name}"
  lambda_log_group = "/aws/lambda/${local.function_name}"
}

data "archive_file" "lambda" {
  type        = "zip"
  source_file = "${path.module}/lambda_function.py"
  output_path = "${path.module}/.terraform/${local.function_name}.zip"
}

resource "aws_dynamodb_table" "session" {
  name         = "${var.name_prefix}-session"
  billing_mode = "PAY_PER_REQUEST"
  hash_key     = "EndpointName"

  attribute {
    name = "EndpointName"
    type = "S"
  }

  server_side_encryption {
    enabled = true
  }
}

resource "aws_cloudwatch_log_group" "lambda" {
  name              = local.lambda_log_group
  retention_in_days = 14
}

data "aws_iam_policy_document" "lambda_assume" {
  statement {
    actions = ["sts:AssumeRole"]

    principals {
      type        = "Service"
      identifiers = ["lambda.amazonaws.com"]
    }
  }
}

resource "aws_iam_role" "lambda" {
  name               = "${local.function_name}-execution"
  assume_role_policy = data.aws_iam_policy_document.lambda_assume.json
}

data "aws_iam_policy_document" "lambda" {
  statement {
    actions = [
      "sagemaker:DeleteEndpoint",
      "sagemaker:DescribeEndpoint",
    ]
    resources = [local.endpoint_arn]
  }

  statement {
    actions = [
      "dynamodb:GetItem",
      "dynamodb:PutItem",
    ]
    resources = [aws_dynamodb_table.session.arn]
  }

  statement {
    actions = [
      "logs:CreateLogStream",
      "logs:PutLogEvents",
    ]
    resources = ["${aws_cloudwatch_log_group.lambda.arn}:*"]
  }
}

resource "aws_iam_role_policy" "lambda" {
  name   = "${local.function_name}-policy"
  role   = aws_iam_role.lambda.id
  policy = data.aws_iam_policy_document.lambda.json
}

resource "aws_lambda_function" "reconciler" {
  function_name                  = local.function_name
  role                           = aws_iam_role.lambda.arn
  handler                        = "lambda_function.handler"
  runtime                        = "python3.13"
  architectures                  = ["arm64"]
  filename                       = data.archive_file.lambda.output_path
  source_code_hash               = data.archive_file.lambda.output_base64sha256
  memory_size                    = 128
  timeout                        = 30
  reserved_concurrent_executions = 1

  environment {
    variables = {
      ENDPOINT_NAME       = var.endpoint_name
      MAX_SESSION_SECONDS = tostring(var.max_session_seconds)
      STATE_TABLE_NAME    = aws_dynamodb_table.session.name
    }
  }

  depends_on = [
    aws_cloudwatch_log_group.lambda,
    aws_iam_role_policy.lambda,
  ]
}

resource "aws_lambda_function_event_invoke_config" "reconciler" {
  function_name                = aws_lambda_function.reconciler.function_name
  maximum_event_age_in_seconds = 60
  maximum_retry_attempts       = 2
}

data "aws_iam_policy_document" "scheduler_assume" {
  statement {
    actions = ["sts:AssumeRole"]

    principals {
      type        = "Service"
      identifiers = ["scheduler.amazonaws.com"]
    }
  }
}

resource "aws_iam_role" "scheduler" {
  name               = "${var.name_prefix}-session-scheduler"
  assume_role_policy = data.aws_iam_policy_document.scheduler_assume.json
}

data "aws_iam_policy_document" "scheduler" {
  statement {
    actions   = ["lambda:InvokeFunction"]
    resources = [aws_lambda_function.reconciler.arn]
  }
}

resource "aws_iam_role_policy" "scheduler" {
  name   = "${var.name_prefix}-session-scheduler-policy"
  role   = aws_iam_role.scheduler.id
  policy = data.aws_iam_policy_document.scheduler.json
}

resource "aws_scheduler_schedule_group" "reconciler" {
  name = "${var.name_prefix}-session"
}

resource "aws_scheduler_schedule" "reconciler" {
  name       = "${var.name_prefix}-session-reconcile"
  group_name = aws_scheduler_schedule_group.reconciler.name
  state      = "ENABLED"

  flexible_time_window {
    mode = "OFF"
  }

  schedule_expression = "rate(1 minute)"

  target {
    arn      = aws_lambda_function.reconciler.arn
    role_arn = aws_iam_role.scheduler.arn

    retry_policy {
      maximum_event_age_in_seconds = 60
      maximum_retry_attempts       = 2
    }
  }
}

resource "aws_lambda_permission" "scheduler" {
  statement_id  = "AllowEventBridgeScheduler"
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.reconciler.function_name
  principal     = "scheduler.amazonaws.com"
  source_arn    = aws_scheduler_schedule.reconciler.arn
}

Prevent Forgetting to Stop SageMaker Real-time Endpoints Launched from Vercel by Automatic Deletion

Introduction

Target Audience

Verification Environment

References

Verification Configuration

Implementing Automatic Shutdown

Results

What I Learned

Additional Verification Based on Completion of Startup

Summary

Appendix

Reproduction Conditions

Creation-Time-Based Lambda

Creation-Time-Based Terraform

`InService`-Based Lambda

Terraform for `InService` Criteria

AWS Topics

Trending Topics

Products & Services

Features and Series

Introduction

Target Audience

Verification Environment

References

Verification Configuration

Implementing Automatic Shutdown

Results

What I Learned

Additional Verification Based on Completion of Startup

Summary

Appendix

Reproduction Conditions

Creation-Time-Based Lambda

Creation-Time-Based Terraform

InService-Based Lambda

Terraform for InService Criteria

`InService`-Based Lambda

Terraform for `InService` Criteria