I tried a setup to make LLMs automatically play Unity games using Claude Desktop × MCP

Target Audience

• People who want to try operating applications using LLM
• People who want to link Unity games with external tools
• People interested in Claude Desktop or MCP (Model Context Protocol)

Introduction

Recently, configurations that allow LLMs to use external tools, such as Claude Desktop and GitHub Copilot's Agent mode, have been attracting attention. At the core of this is the MCP (Model Context Protocol). In this article, I implemented an MCP Server in TypeScript that operates a Unity game from Claude Desktop in accordance with the MCP philosophy.

Keywords: MCP / Unity / Automatic Debugging / Automatic Testing / Gameplay AI Agent

What is MCP?

MCP (Model Context Protocol) is a protocol that enables the following for LLMs:

• Calling external tools (Tool)
• Retrieving resources (files and data)
• Providing prompts (instructions)

The characteristics of MCP are as follows:

• Tool definitions are described in JSON schema
• Communication is done via STDIO or HTTP+SSE
• The LLM side (MCP Client) gets a list of Tools and calls appropriate Tools according to the context

An MCP Server is a "lightweight program for LLM use" and can be understood as an AI agent-like entity.

Value Created by "MCP × Games"

The combination of MCP and games creates the following value:

• Debugging Efficiency
  You can check the behavior of the game while asking the LLM "How should it behave in this state?"

• Balance Adjustment through Automated Play
  Have the LLM play thousands of times to analyze win rates and behavior patterns for game balance adjustments

• Collaborative Development with Non-Engineers
  Checking and operating the game state in natural language through LLM enables smoother coordination with planners and designers

Made Claude Desktop Operate a Unity Game

In this experiment, I had Claude Desktop operate a simple 3x3 grid game created in Unity.

• The player can move up, down, left, and right
• Picking up a key and opening the door completes the game
• The game state is expressed as text and returned in JSON format

Player: (1,1)
HasKey: true
Key: (1,0)
Door: (1,1)
LastInput: move north
Status: cleared

Configuration Diagram

In this configuration, we adopted the following flow to exchange commands between Claude Desktop and the Unity game:

1. Prepare an HttpListener in the Unity application
2. Build an MCP Server with TypeScript and send it to the Unity application
3. Access the MCP Server from Claude Desktop

Let me introduce each briefly.

Preparation on Unity Side

On the Unity side, we set up a simple HTTP server using HttpListener. It receives requests from the MCP Server (e.g., move north), advances the game, and returns the state.

{
  "role": "system",
  "name": "game_state",
  "content": "Player: (1,0)\nHasKey: true\nKey: (0,-1)\nDoor: (2,2)\nLastInput: move east\nStatus: in_progress"
}

Implementing an MCP Server in TypeScript

Role of MCP Server

An MCP Server is a lightweight program that provides "Tools" to be called from Claude Desktop (MCP Client). In this case, we define "move" and "game_state" Tools to manipulate the Unity game, allowing the player to move in four directions and retrieve the in-game situation.

Tool Definition

A Tool is an external process that the LLM (Claude) calls when it determines "this prompt needs this processing." On the MCP Server side, we define the tool's name, description, argument schema, and execution function. Claude gets the list of Tools and selects and calls the appropriate Tool based on the prompt.

This code initializes the MCP Server and defines "move" and "get_state" Tools that send requests to the Unity-side HTTP server. We use zod for argument validation in the Tool definition.

// src/index.ts
import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";
import { z } from "zod";
import fetch from "node-fetch";

const server = new McpServer({
  name: "UnityController",
  version: "0.1.0",
});

// Tool: Move the player
server.tool(
    "move",  // Tool name
    "Move the player in a specified direction.",  // Description
    {
        direction: z.enum(["north", "south", "east", "west"]),  // Argument schema
    },
    async ({ direction }) => {
        const res = await fetch("http://127.0.0.1:8080/mcp", {
            method: "POST",
            headers: { "Content-Type": "application/json" },
            body: JSON.stringify({
                role: "user",
                name: "player_move",
                content: `move ${direction}`,
            }),
        });

        const json = await res.json() as {
            role: string;
            name: string;
            content: string;
        };
        return {
            content: [
                {
                    type: "text",
                    text: json.content,
                },
            ],
        };
    }
);

// Tool: Get game state
server.tool(
    "get_state",  // Tool name
    "Get the current game state.",  // Description
    {},  // Argument schema
    async () => {
        const res = await fetch("http://127.0.0.1:8080/mcp", {
            method: "POST",
            headers: { "Content-Type": "application/json" },
            body: JSON.stringify({
                role: "user",
                name: "player_move",
                content: "get_state",
            }),
        });

        const json = await res.json() as {
            role: string;
            name: string;
            content: string;
        };
        return {
            content: [
                {
                    type: "text",
                    text: json.content,
                },
            ],
        };
    }
);

// Start MCP Server
async function main() {
  const transport = new StdioServerTransport();
  await server.connect(transport);
  console.error("MCP Server running on stdio");
}

main().catch((err) => {
  console.error("Fatal error:", err);
  process.exit(1);
});

Registering MCP Server with Claude Desktop

To register the MCP Server with Claude Desktop, add the following to the claude_desktop_config.json configuration file:

{
  "mcpServers": {
    "unity-controller": {
      "command": "node",
      "args": [
        "/absolute/path/to/build/index.js"
      ]
    }
  }
}

After saving the configuration, restart Claude Desktop. If recognized correctly, the registered MCP Server will appear in the settings screen.

Trying It Out

Let's input the following to Claude Desktop:

Clear the game

The following GIF shows Claude automatically playing the Unity game using the "move" and "gate_state" Tools. Claude understands the prompt's intent and calls the "gate_state" Tool. The MCP Server sends "get_state" to the Unity-side HTTP server. The Unity side returns the current state in JSON. Claude reads that state and determines the next action. In this case, it found a key to the east, so it next calls the "move" Tool with the argument {"direction": "east"}.

{}
Player: (0,0)
HasKey: false
Key: (1,0)
Door: (1,1)
LastInput: move unknown
Status: in_progress

{
  `direction`: `east`
}
Player: (1,0)
HasKey: true
Key: (1,0)
Door: (1,1)
LastInput: move east
Status: in_progress

{
  `direction`: `north`
}
Player: (1,1)
HasKey: true
Key: (1,0)
Door: (1,1)
LastInput: move north
Status: cleared

Impressions and Future Outlook

With the Claude Desktop × MCP Server × Unity configuration, I was able to have the LLM operate an application in a natural way. What was particularly impressive was that by just defining Tools, Claude appropriately called them according to the context. I found that by refining the Tool descriptions and schemas, I could more naturally control the LLM's behavior.

Future Outlook

• Add Tools like reset_game, take_key, open_door to enable more complex game control
• Return detailed Unity-side state in JSON to improve LLM judgment accuracy
• Consider integration with MCP Clients other than Claude Desktop (e.g., VS Code, Copilot Agent)