I Tried Playwright E2E Testing of LLM Streaming in Next.js Without an External API

Introduction

In a chat screen that uses an LLM, it is important that the content being generated is displayed little by little, rather than being shown all at once after the answer is complete.

On the other hand, if you connect streaming E2E tests to a real LLM provider, API usage fees are incurred. The response content and speed are also not constant. It is also difficult to reproduce HTTP 429 or mid-stream disconnections at the intended timing.

This time, using a Next.js application that uses OpenRouter as the subject, I built a browser E2E test that does not communicate with any external LLM API. It verifies not only the incremental display in the browser, but also errors before the stream starts, mid-stream disconnections, and missing completion notifications.

I do not add a test provider to the production code. The setup keeps the path through the Next.js Route Handler and the existing OpenRouter adapter.

Target audience

• Those who stream LLM responses in Next.js
• Those considering swapping out LLM providers
• Those who want to run browser E2E tests without paid APIs or real API keys
• Those who want to reproduce not only the normal case but also mid-stream disconnections and missing completion notifications

Test environment

References

• Playwright: Web server
• Playwright: Mock APIs
• Playwright: page.route()
• Node.js: --require CLI option
• Next.js: Route Handlers

Architecture

The communication path under test

Next.js Route Handlers can use the standard Web Request and Response APIs. The POST /api/generate in this article is also structured to relay a Response holding an upstream ReadableStream to the browser.

The /api/generate that the browser connects to is not mocked. The application runs the same Route Handler as usual, and the OpenRouter adapter builds the request. Only the OpenRouter URL that the adapter finally calls is replaced inside the Next.js server process.

Fixing the stream contract visible from the browser

Before swapping out the backend, I fixed the stream contract visible from the browser.

request:
  POST /api/generate
  { "messages": [{ "role": "user|assistant", "content": "..." }] }

success:
  Content-Type: text/event-stream
  data: {"choices":[{"delta":{"content":"..."}}]}

completion:
  data: [DONE]

pre-stream failure:
  HTTP status + JSON { "error": "...", "retryAfter"?: "..." }

On the server side, provider-specific processing is separated into the following interface.

export type StreamChatRequest = {
  messages: ChatMessage[];
  signal?: AbortSignal;
};

export interface StreamingLlmBackend {
  readonly id: string;
  streamChat(request: StreamChatRequest): Promise<Response>;
}

The OpenRouter adapter sends the request with stream: true. If another provider returns a different event format, the adapter converts it into the application's common SSE format.

The browser side does not know the provider name. It handles only the following three kinds of events.

export type AppStreamEvent =
  | { type: "delta"; text: string }
  | { type: "error"; message: string }
  | { type: "done" };

With this boundary in place, even if you change the backend's authentication, URL, request format, or provider-specific events, you do not need to change the incremental rendering on the React side.

Implementation

Replacing fetch in the Next.js server process

Using Playwright's webServer setting, the Next.js development server is started when the tests begin.

Playwright can mock requests issued by a browser page using page.route() or browserContext.route().

This time, the requests issued by the browser go only as far as /api/generate. The request to OpenRouter is issued from the Next.js server process. If you mock /api/generate on the browser side, the Route Handler, prompt generation, OpenRouter adapter, and SSE relay are no longer exercised.

So I set a CommonJS preload on the Next.js process that Playwright launches. Node.js's --require has a feature that loads the specified module at startup. For details, see the Node.js CLI documentation.

import { defineConfig } from "@playwright/test";
import path from "node:path";

const preloadPath = path.resolve("tests/e2e/mock-openrouter.cjs");

export default defineConfig({
  testDir: "./tests/e2e",
  workers: 1,
  use: {
    baseURL: "http://127.0.0.1:3100",
    browserName: "chromium",
    channel: "chrome",
    headless: true,
  },
  webServer: {
    command: "npm run dev -- --hostname 127.0.0.1 --port 3100",
    url: "http://127.0.0.1:3100",
    reuseExistingServer: false,
    env: {
      ...process.env,
      NODE_ENV: "development",
      NODE_OPTIONS: `--require=${preloadPath}`,
      LLM_E2E_FETCH_MOCK: "1",
      OPENROUTER_API_KEY: "e2e-placeholder-not-a-secret",
    },
  },
});

Preventing the test preload from starting in production

Because it rewrites the server process's fetch, I make sure the test preload is never loaded in a production environment.

At the top of the preload, the following two conditions are checked.

"use strict";

if (process.env.NODE_ENV === "production") {
  throw new Error("E2E fetch mock must never run with NODE_ENV=production");
}

if (process.env.LLM_E2E_FETCH_MOCK !== "1") {
  throw new Error("E2E fetch mock requires an explicit local-only opt-in");
}

The URL to be replaced must exactly match OpenRouter's Chat Completions endpoint. All other fetch calls are passed to the original implementation.

const OPENROUTER_URL = "https://openrouter.ai/api/v1/chat/completions";
const originalFetch = globalThis.fetch;

globalThis.fetch = async function controlledFetch(input, init) {
  if (String(input) !== OPENROUTER_URL) {
    return originalFetch(input, init);
  }

  // Return a response according to the synthetic scenario
};

An unknown scenario arriving at the OpenRouter URL is not allowed to fall back to the real network.

return new Response(
  JSON.stringify({ error: { message: "unknown E2E scenario" } }),
  {
    status: 400,
    headers: { "Content-Type": "application/json" },
  },
);

This closes off the path by which a mistake in the test input could lead to a connection to the paid API. The preload's startup conditions are also verified by a separate unit test.

Creating a time-staggered stream

Simply returning the SSE string all at once cannot confirm that React rendered the intermediate states. I created a ReadableStream that calls enqueue() for each chunk after a specified time.

const encoder = new TextEncoder();

function delayedStream(steps) {
  return new ReadableStream({
    start(controller) {
      let elapsed = 0;
      for (const step of steps) {
        elapsed += step.afterMs;
        setTimeout(() => {
          if (step.error) {
            controller.error(new Error(step.error));
          } else if (step.close) {
            controller.close();
          } else {
            controller.enqueue(encoder.encode(step.data));
          }
        }, elapsed);
      }
    },
  });
}

In the normal case, the first delta, a malformed JSON event, the next delta, and [DONE] are returned at staggered times. CRLF and LF are also mixed.

return streamResponse([
  {
    afterMs: 100,
    data: 'data: {"choices":[{"delta":{"content":"第一段階"}}]}\r\n\r\n',
  },
  { afterMs: 250, data: "data: this-is-not-json\n\n" },
  {
    afterMs: 650,
    data: 'data: {"choices":[{"delta":{"content":"第二段階"}}]}\n\n',
  },
  { afterMs: 100, data: "data: [DONE]\n\n" },
  { afterMs: 10, close: true },
]);

In a separate unit test, I split UTF-8 Japanese strings and SSE delimiters in the middle of network chunks. This confirms that the stream can be reconstructed without depending on the provider's chunk boundaries.

Checking intermediate states in the browser

In the E2E test, I check not only the final result but also the period between the first delta and the next delta.

test("renders token-equivalent deltas incrementally", async ({ page }) => {
  await page.goto("/");
  await page.getByTestId("prompt-input")
    .fill("[[e2e:success]] 公開可能な合成入力");
  await page.getByTestId("send-button").click();

  const assistant = page.getByTestId("assistant-message").last();

  await expect(page.getByTestId("stream-status")).toBeVisible();
  await expect(assistant).toHaveText("第一段階");
  await expect(assistant).not.toContainText("第二段階");

  await expect(assistant).toHaveText("第一段階第二段階");
  await expect(page.getByTestId("stream-status")).toBeHidden();
});

At the point when the first toHaveText("第一段階") succeeds, the in-progress indicator is still displayed. 第二段階 is not yet in the DOM. After that, the second delta and [DONE] are received, and the state becomes complete.

A test that checks only the final string would also pass an implementation that buffers everything at once. By checking the intermediate state, you can directly test the user experience we want to maintain.

Preparing scenarios other than the normal case

When swapping out the backend, the moment at which a failure occurs is also important. This time I prepared the following four scenarios.

HTTP errors before the start are handled before beginning to read the SSE. In a mid-stream disconnection, the partial response already received is not erased.

Even without a transport error, a stream may close without [DONE] ever arriving. Under this contract, a normal EOF is not treated as equivalent to [DONE]. The partial response is kept, and the screen shows that the stream ended before the completion notification.

Results

Running the tests

I ran the following commands.

npm test
npm run lint
npx tsc --noEmit
npm run build
npm run test:e2e

Running 4 tests using 1 worker
4 passed (16.1s)

In the mid-stream disconnection test, failed to pipe response was output to the Next.js development server. This is the result of the synthetic disconnection reaching the server's SSE relay. It is an expected diagnostic for this scenario.

When I first wrote the E2E tests, only 3 of the 4 passed. The application displays an existing Japanese message for HTTP 429, but the test expected the string HTTP 429.

In this case, I did not change the application to fit the test; I checked the current screen specification and fixed only the test's expected value. When an E2E test fails, the production code is not always the one that is wrong.

What I learned

With this E2E test, I was able to confirm the whole flow: from input in the browser, through the Next.js Route Handler, the OpenRouter adapter, and the SSE relay, to the answer being displayed little by little on the screen. Beyond normal completion, it can also distinguish errors before the start, mid-stream disconnections, and cases that end without [DONE].

On the other hand, I cannot tell whether Vercel or a CDN buffers the response, or how the deployment environment's timeouts behave. The response speed and error formats of the actual backend are also unverified. These will need to be checked in a preview environment after the backend candidate is decided.

When adding another streaming backend, it seems easy to work with a design that confines differences in authentication and request format inside the adapter and returns the common delta, error, and done to the browser. However, this is a design policy derived from this result, not a fact confirmed on another backend. If an asynchronous job approach is chosen, rather than forcing it into the same shape, I would treat it as a separate API and screen design.

Summary

By replacing only the fetch addressed to OpenRouter in the Next.js server process, I built a browser E2E test that does not communicate with any external LLM API. With this mechanism, even when changing the LLM backend in the future, the current streaming experience can be verified without depending on external API costs or response variability.

I hope this is helpful for those considering E2E tests for LLM streaming.

Appendix

The full code used on the E2E test side is included here. The application side needs data-testid attributes for prompt-input, send-button, assistant-message, stream-status, error-panel, and retry-button.

import { defineConfig } from "@playwright/test";
import path from "node:path";

const preloadPath = path.resolve("tests/e2e/mock-openrouter.cjs");

export default defineConfig({
    testDir: "./tests/e2e",
    fullyParallel: false,
    workers: 1,
    reporter: "line",
    outputDir: "test-results",
    use: {
        baseURL: "http://127.0.0.1:3100",
        browserName: "chromium",
        channel: "chrome",
        headless: true,
        trace: "retain-on-failure",
    },
    webServer: {
        command: "npm run dev -- --hostname 127.0.0.1 --port 3100",
        url: "http://127.0.0.1:3100",
        reuseExistingServer: false,
        timeout: 120_000,
        env: {
            ...process.env,
            NODE_ENV: "development",
            NODE_OPTIONS: `--require=${preloadPath}`,
            LLM_E2E_FETCH_MOCK: "1",
            OPENROUTER_API_KEY: "e2e-placeholder-not-a-secret",
        },
    },
});

"use strict";

if (process.env.NODE_ENV === "production") {
    throw new Error("E2E fetch mock must never run with NODE_ENV=production");
}
if (process.env.LLM_E2E_FETCH_MOCK !== "1") {
    throw new Error("E2E fetch mock requires an explicit local-only opt-in");
}

const OPENROUTER_URL = "https://openrouter.ai/api/v1/chat/completions";
const originalFetch = globalThis.fetch;
const encoder = new TextEncoder();

function delayedStream(steps) {
    return new ReadableStream({
        start(controller) {
            let elapsed = 0;
            for (const step of steps) {
                elapsed += step.afterMs;
                setTimeout(() => {
                    if (step.error) {
                        controller.error(new Error(step.error));
                    } else if (step.close) {
                        controller.close();
                    } else {
                        controller.enqueue(encoder.encode(step.data));
                    }
                }, elapsed);
            }
        },
    });
}

function streamResponse(steps) {
    return new Response(delayedStream(steps), {
        status: 200,
        headers: { "Content-Type": "text/event-stream; charset=utf-8" },
    });
}

globalThis.fetch = async function controlledFetch(input, init) {
    if (String(input) !== OPENROUTER_URL) {
        return originalFetch(input, init);
    }

    const request = JSON.parse(String(init?.body ?? "{}"));
    const messages = Array.isArray(request.messages) ? request.messages : [];
    const lastUser = [...messages].reverse().find((message) => message?.role === "user");
    const scenario = String(lastUser?.content ?? "");

    if (scenario.includes("[[e2e:success]]")) {
        return streamResponse([
            { afterMs: 100, data: "data: {\"choices\":[{\"delta\":{\"content\":\"第一段階\"}}]}\r\n\r\n" },
            { afterMs: 250, data: "data: this-is-not-json\n\n" },
            { afterMs: 650, data: "data: {\"choices\":[{\"delta\":{\"content\":\"第二段階\"}}]}\n\n" },
            { afterMs: 100, data: "data: [DONE]\n\n" },
            { afterMs: 10, close: true },
        ]);
    }

    if (scenario.includes("[[e2e:pre-error]]")) {
        return new Response(JSON.stringify({ error: { message: "synthetic rate limit" } }), {
            status: 429,
            headers: { "Content-Type": "application/json", "Retry-After": "3" },
        });
    }

    if (scenario.includes("[[e2e:disconnect]]")) {
        return streamResponse([
            { afterMs: 100, data: "data: {\"choices\":[{\"delta\":{\"content\":\"部分応答\"}}]}\n\n" },
            { afterMs: 250, error: "synthetic mid-stream disconnect" },
        ]);
    }

    if (scenario.includes("[[e2e:missing-done]]")) {
        return streamResponse([
            { afterMs: 100, data: "data: {\"choices\":[{\"delta\":{\"content\":\"完了前\"}}]}\n\n" },
            { afterMs: 100, close: true },
        ]);
    }

    return new Response(JSON.stringify({ error: { message: "unknown E2E scenario" } }), {
        status: 400,
        headers: { "Content-Type": "application/json" },
    });
};

import { expect, test } from "@playwright/test";

async function send(page: import("@playwright/test").Page, prompt: string) {
    await page.goto("/");
    await page.getByTestId("prompt-input").fill(prompt);
    await page.getByTestId("send-button").click();
}

test("renders token-equivalent deltas incrementally and completes despite a malformed event", async ({ page }) => {
    await send(page, "[[e2e:success]] 公開可能な合成入力");

    const assistant = page.getByTestId("assistant-message").last();
    await expect(page.getByTestId("stream-status")).toBeVisible();
    await expect(assistant).toHaveText("第一段階");
    await expect(assistant).not.toContainText("第二段階");
    await expect(assistant).toHaveText("第一段階第二段階");
    await expect(page.getByTestId("stream-status")).toBeHidden();
    await expect(page.getByTestId("error-panel")).toHaveCount(0);
});

test("shows a pre-stream provider error separately from a stream interruption", async ({ page }) => {
    await send(page, "[[e2e:pre-error]] 公開可能な合成入力");

    await expect(page.getByTestId("error-panel")).toContainText("レート制限中");
    await expect(page.getByText("サーバからの推奨待機秒: 3s")).toBeVisible();
    await expect(page.getByTestId("assistant-message").last()).toContainText("レート制限中");
    await expect(page.getByTestId("stream-status")).toBeHidden();
});

test("retains partial output and identifies a mid-stream disconnect", async ({ page }) => {
    await send(page, "[[e2e:disconnect]] 公開可能な合成入力");

    const assistant = page.getByTestId("assistant-message").last();
    await expect(assistant).toContainText("部分応答");
    await expect(page.getByTestId("error-panel")).toContainText("途中で切断");
    await expect(assistant).toContainText("途中で切断");
    await expect(page.getByTestId("retry-button")).toBeVisible();
    await expect(page.getByTestId("stream-status")).toBeHidden();
});

test("treats EOF without the completion event as an incomplete stream", async ({ page }) => {
    await send(page, "[[e2e:missing-done]] 公開可能な合成入力");

    const assistant = page.getByTestId("assistant-message").last();
    await expect(assistant).toContainText("完了前");
    await expect(page.getByTestId("error-panel")).toContainText("完了通知前");
    await expect(page.getByTestId("stream-status")).toBeHidden();
});

import assert from "node:assert/strict";
import { spawnSync } from "node:child_process";
import path from "node:path";
import test from "node:test";

const preload = path.resolve("tests/e2e/mock-openrouter.cjs");

function requireMock(environment) {
    return spawnSync(process.execPath, ["--require", preload, "--eval", "void 0"], {
        cwd: process.cwd(),
        encoding: "utf8",
        env: { ...process.env, ...environment },
    });
}

test("the E2E fetch mock refuses production", () => {
    const result = requireMock({ NODE_ENV: "production", LLM_E2E_FETCH_MOCK: "1" });
    assert.notEqual(result.status, 0);
    assert.match(result.stderr, /must never run with NODE_ENV=production/);
});
test("the E2E fetch mock requires explicit local opt-in", () => {
    const result = requireMock({ NODE_ENV: "development", LLM_E2E_FETCH_MOCK: "0" });
    assert.notEqual(result.status, 0);
    assert.match(result.stderr, /requires an explicit local-only opt-in/);
});

test("the E2E fetch mock can preload only with both development guards", () => {
    const result = requireMock({ NODE_ENV: "development", LLM_E2E_FETCH_MOCK: "1" });
    assert.equal(result.status, 0, result.stderr);
});