Prompt Injection Countermeasures for Custom Chat Agents in Amazon Quick
This page has been translated by machine translation. View original
I'm Ishikawa from the Cloud Business Division. When rolling out custom chat agents in Amazon Quick within a company, one concern is how to defend against prompt injection. Based on the official AWS documentation and my own experience, this article explains the guardrails that Amazon Quick provides out of the box, as well as the measures that administrators and agent creators can each take, from a defense-in-depth perspective.
What Are Amazon Quick Chat Agents?
Amazon Quick has two types of chat agents: the system chat agent (default), which is automatically provided when you create a subscription, and custom chat agents, which you create for a specific team or use case by configuring your own instructions (persona) and knowledge sources.
Custom chat agents can be linked to spaces, dashboards, topics, files, and so on as knowledge sources, integrated with action connectors (Slack, Outlook, Jira, etc.), and then shared within the organization. While convenient, because they are chat applications that embed an LLM, they need to be designed with the risk of prompt injection in mind.
What Is Prompt Injection?
The AWS Security Blog post Safeguard your generative AI workloads from prompt injections classifies prompt injection into two broad types.
- Direct prompt injection: An attack in which a user explicitly enters a command such as "ignore all previous instructions" in an attempt to overwrite the agent's original instructions
- Indirect prompt injection: An attack in which malicious instructions are embedded in an external source the agent references (such as an uploaded document or a web page), causing unintended behavior when the agent processes it. An example is given of planting instructions hidden in a document as white text on a white background
Because a custom chat agent performs both "conversation with users" and "referencing knowledge sources," countermeasures are needed for each of these two paths.
Overview of Countermeasures
Prompt injection countermeasures in Amazon Quick can be organized into the following four layers.
Layer 1: Default Guardrails (Automatically Applied)
The first thing to understand is that all chat agents in Amazon Quick (system and custom) as well as flows have default guardrails and safety controls built in from the start.
The official documentation Chat agent customization in Amazon Quick lists the following three.
- Prompt leak protection: Automatically enabled, it prevents prompt injection and other attacks that attempt to break the LLM
- Prompt safety: Protects against common security threats, such as malicious instructions and instructions like "ignore the guardrails"
- Blocked words and phrases: Protects both chat requests and responses from inappropriate content such as insults, hate speech, sexual content, violence, and misconduct
These are standard protections applied by default, with no need for users to enable them. The fact that you don't need to build and configure guardrails yourself, as you would with Amazon Bedrock Guardrails, is a benefit unique to a managed service.
Layer 2: Instance-Wide Settings by Administrators
Adding Blocked Words and Phrases
Administrators can define up to 50 blocked words and phrases that apply to all chat agents and flows in the instance. The configured words and phrases will no longer be included in responses. Nothing is set by default, so if there are words you don't want output under your company's policy, add them explicitly. However, as of this writing (2026/07/17), entering Japanese results in an error and cannot be registered.
Configuration is done from the management console.
- Log in to the Amazon Quick console and select "Manage account"
- From the left navigation menu, select "Customize" → "Agent customization"
- Under "Guardrails and safety controls," in "Blocked words and phrases," click the [Add] button to add words and phrases
Since the limit of 50 is small, it is practical to assume that the default guardrails cover general inappropriate content and to concentrate the 50 slots on "words that can't be blocked without your company's own domain knowledge."
Controlling Whether External Links Are Clickable
Administrators can control whether URLs in chat responses are displayed as clickable hyperlinks or as plain text. This setting applies to all chat agents and flows in the instance.
In indirect prompt injection, a possible tactic is to include a malicious link in a response to lure the user, so if hyperlinks are not required by your requirements, displaying them as plain text (toggle off) can reduce the risk involved in clicking.
Layer 3: Countermeasures in Agent Design
Place Instructions That Must Not Be Altered in Reference Documents
As means of defining the behavior of a custom chat agent, the official documentation Chat agent context sources and best practices lists three: Persona instructions, Reference documents, and Spaces, each with different characteristics.
| Method | Characteristics |
|---|---|
| Persona instructions | High-level guidance that generative AI interprets, expands, and optimizes |
| Reference documents | What you write is retained as is and becomes the agent's persistent context |
| Spaces | A dynamic knowledge repository that is searched and referenced during conversations |
What is important from a security perspective is that persona instructions may have their original wording or structure changed by the AI's enhancement processing. The official documentation explicitly states that "critical guidelines that must not be altered should be placed in reference documents instead."
Furthermore, regarding reference documents, it states that "behavioral frameworks and reference data provided via files remain in effect even when the user attaches other content." Fixing the business rules and response policies you want the agent to follow as reference documents helps prepare for overwriting of instructions by user input (direct prompt injection). However, be aware that the format of response templates may be temporarily adjusted if the user gives conflicting formatting instructions.
Explicitly Scope Knowledge Sources and Actions
According to the official documentation "Create, customize, and deploy AI-powered chat agents in Amazon Quick," when resources (spaces and action connectors) are explicitly configured at creation time, a custom chat agent by default orchestrates answers and actions using only those configured resources. Conversely, if no resources are configured, all spaces and actions that the user can access at chat time may become targets of use.
AWS Prescriptive Guidance's Mapping to OWASP top 10 for LLM applications also lists "limiting the attack surface by scoping the agent" as a countermeasure for LLM01 (prompt injection) of the OWASP Top 10 for LLM Applications. When creating a custom chat agent, linking only the trusted spaces and actions needed for the use case narrows both the entry paths for indirect prompt injection and the blast radius if an attack succeeds.
Layer 4: Permission Management
Filtering Responses by User Permissions
A custom chat agent accesses content with the permissions of the invoking user themselves, not the permissions of the agent itself. Spaces maintain a security boundary through Amazon Quick's permission system, so even if the agent's behavior is manipulated through prompt injection, the design ensures that data the user could not originally access is not included in responses.
Restricting Features with Custom Permissions
In the Enterprise Edition, you can use custom permission profiles to restrict features per account, role, or user. For chat agents, the following controls are available.
- Restricting Create Chat Agents: Allows interaction with agents while restricting only new creation. The official documentation also recommends using only this restriction when you want to stop creation while keeping interaction with the system agent
- Restricting Use internet to enhance results: Restricts internet access used to enhance results. Because web content is a typical path for indirect prompt injection, you can consider restricting it depending on your requirements
- Restricting Chat Agents: Disables the entire chat agent feature, including interaction with the system agent.
Note: If the goal is to prevent any use of agents, not assigning the PRO role is more rational in terms of cost as well, so this setting is actually useful when there are users or roles that need other PRO features such as creating dashboards but should be blocked from chat agents only, or when you want to temporarily shut everything down until a security review is complete
By limiting the users who can create agents to a trusted scope and managing sharing with two permission types, Owner / Viewer, you can prevent agents with unintended configurations from spreading across the organization.
Usage Notes
Even though default guardrails are built in, there is no single measure that can prevent prompt injection 100%. The AWS Security Blog also recommends taking a multi-layered approach rather than relying on a single defense. In addition to combining the four layers introduced in this article, I recommend establishing a management process for documents imported into spaces (an operation that keeps untrusted files from getting mixed in).
Conclusion
Amazon Quick chat agents have three guardrails built in as standard—prompt leak protection, prompt safety, and blocking of inappropriate content—so basic protection against prompt injection works without additional configuration. On top of that, by combining administrator control of blocked words and link display, agent design that makes use of reference documents, and feature restrictions through custom permissions, you can build a multi-layered defense.
When proceeding with the internal rollout of custom chat agents, why not first understand what the default guardrails cover, and then consider the Layer 2–4 settings that fit your company's policy?
