Try out OpenAI API Flex processing available at the same price as the Batch API

This is Suenaga from the Retail App Co-Creation Division.

The Batch API is available at half the regular price, but it runs asynchronously and can take up to 24 hours to complete (though it may finish sooner in practice). When you want results faster than the Batch API and within the same request, Flex processing is the option to use.

What is Flex processing

Flex processing is a processing mode that allows you to use the Responses API or Chat Completions API at a lower price, in exchange for tolerating slower response times and temporary resource unavailability.

The pricing is the same as the Batch API — half the regular price. Prompt caching discounts can also be combined on top of that.

Since you only need to add service_tier: "flex" to your request, there is no need for file creation, job registration, status checking, or result file retrieval like with the Batch API. You receive the generated result directly as the API response.

On the other hand, there are also the following drawbacks.

• Responses may take longer than standard processing
• If resources are insufficient, a 429 Resource Unavailable error may occur
• It is in beta and supported models are limited (all GPT-5.6 series models are supported)

The main use cases are for processing where some delay is acceptable — such as evaluation, data enrichment, and overnight batch jobs — rather than tasks where an end user is waiting in front of a screen.

Differences between Standard, Flex, and Batch API

Here is a rough summary.

The Batch API also has a separate, larger rate limit quota from the synchronous API. If you can process a large number of requests in bulk and tolerate up to 24 hours for completion, the Batch API may be a better fit depending on your requirements.

Flex is a middle-ground option for cases where you want to maintain a per-item processing flow but prioritize cost over response speed.

Pricing

At the time of writing, the pricing for gpt-5.6-sol, gpt-5.6-terra, and gpt-5.6-luna, which support Flex processing, is all half of Standard. Here we use gpt-5.6-terra as an example. Units are per 1 million tokens, for short context.

Input, output, and Prompt caching are all half price. Please check OpenAI API Pricing for supported models and the latest pricing.

Trying it out

With the official OpenAI SDK, you specify flex for the service_tier in the Responses API.

import OpenAI from "openai";

const client = new OpenAI({
  timeout: 15 * 60 * 1000,
});

const response = await client.responses.create({
  model: "gpt-5.6-terra",
  input: "この文章を3行で要約してください。\n\n<要約する文章>",
  service_tier: "flex",
});

console.log(response.output_text);
console.log(response.service_tier);

The calling method is almost identical to the regular Responses API. By checking service_tier in the response, you can also confirm which processing mode was actually applied.

Since Flex may take longer than usual, the official OpenAI Flex sample sets the SDK timeout to 15 minutes. When using a third-party SDK, it is a good idea to set a longer timeout on that SDK's side as well.

For example, when using the Vercel AI SDK's OpenAI provider, you can specify Flex via providerOptions.

import { openai } from "@ai-sdk/openai";
import { generateText } from "ai";

const result = await generateText({
  model: openai.responses("gpt-5.6-terra"),
  prompt: "この文章を3行で要約してください。\n\n<要約する文章>",
  timeout: 15 * 60 * 1000,
  providerOptions: {
    openai: {
      serviceTier: "flex",
    },
  },
});

console.log(result.text);
console.log(result.providerMetadata?.openai?.serviceTier);

Running 100 times to check latency and rejection rate

I verified how much slower Flex actually is in practice and how often it gets rejected due to resource unavailability.

The test conditions were as follows.

• Model: gpt-5.6-terra
• 100 runs each for Standard and Flex
• Maximum 20 concurrent requests per processing mode
• Short Japanese judgment/summarization tasks with output limited to within 2 sentences
• Average of approximately 190 input tokens and approximately 65 output tokens per request
• Reasoning effort set to none
• Prompt caching not used
• No retry logic

Retry logic was not used this time in order to observe the raw rejection rate of Flex itself.

Here are the results. Time is measured from the start of the request until the full response is received.

In this test, all 100 Flex requests succeeded and no 429 Resource Unavailable errors occurred. Latency was also nearly the same as Standard, with Flex coming in slightly shorter.

Flex happened to be faster this time around. On the other hand, in actual use I have experienced cases where processing stalled due to errors during peak times or retries occurred, ultimately resulting in longer processing times.

I also calculated the cost based on the actual tokens consumed.

The output token count varied slightly between generations so it is not exactly 50%, but the measured result came out to roughly half. With the same token count, it would be exactly 50% as stated in the pricing table.

Thinking about errors and retries

If Flex cannot secure resources, a 429 Resource Unavailable is returned and you are not charged for that request.

Following the official documentation, I recommend handling errors by retrying with Flex using exponential backoff (progressively increasing the wait time between retries — 1 second, 2 seconds, 4 seconds, and so on), or if completion is a priority, switching to Standard and retrying.

One approach would be to retry with Flex for processes such as overnight jobs where you can wait until the next execution opportunity, and fall back to Standard for processes where you want to ensure completion within the current run.

Also, unlike the Batch API's guarantee of "within 24 hours," Flex offers no such completion time guarantee. Since timeouts can occur with long-running requests, it is safest to use Flex for processes that can be safely re-executed.

Closing thoughts

I think Flex processing is worth considering for workloads where you want to receive results synchronously but API costs are a concern.
As a side note, Amazon Bedrock also has Service Tiers including a similarly named Flex Tier, which lets you choose a processing mode based on a similar concept (though supported models and pricing differ).

See you 👋