Amazon Q in Connect self-service: I tried automatically answering only specific questions while escalating others
Introduction
In Amazon Q in Connect self-service, when automatically answering customer questions with the QUESTION tool, if the necessary information is not available in the knowledge source, a fixed message "I don't have an answer" is displayed.
Here is an actual example of the behavior:
BOT : Please tell us your inquiry.
Customer : Please tell me where Classmethod's headquarters is located.
QUESTION tool is selected
BOT : I don't have an answer
Customer : You don't know?
QUESTION tool is selected
BOT : I don't have an answer
Customer : Please connect me to an operator.
ESCALATION tool is selected
BOT : Understood. I will connect you to a specialist, please wait a moment.
(Transitions to the next flow and transfers to queue)
The English message "I don't have an answer" diminishes the customer experience when providing services in Japanese.
Additionally, the repeated display of this message may cause customer dissatisfaction and frustration.
One solution to this problem is to create a custom AI prompt in Amazon Q in Connect self-service that automatically answers only specific questions and escalates everything else.
In this article, we will try this approach.
This approach directly sets question and answer pairs in the prompt, rather than referring to knowledge sources like the QUESTION tool.
Benefits:
- Can be implemented within the scope of Amazon Q in Connect self-service
- Reliably answers specific questions
- Complete control over response content
Drawbacks:
- Concerns about decreased response accuracy and speed as question-answer sets increase
- Regular maintenance of prompt settings required
- Limited types of questions that can be handled
- For unanticipated questions, escalation occurs, limiting the effect of automation## AI Prompt Creation
For the AI prompt, we'll use the Claude 3.5 Sonnet model. Since it can't be configured in the console, we'll create it by executing commands from AWS CloudShell.
First, let's create the following script file.
The prompt name will be smart-inquiry-auto-triage-escalation-handler-claude-3-5-sonnet-Processing
.
The assistant_id
can be confirmed from the default AI prompt confirmation screen in the console.
cat << 'EOF' > prompt.py
import boto3
import yaml
import json
assistant_id = 'a1793008-f4de-481b-a9ed-3697ef373ff2'
prompt_name = 'smart-inquiry-auto-triage-escalation-handler-claude-3-5-sonnet-Processing'
yaml_template = """system: |
You are an experienced customer service assistant. Please respond to customer phone inquiries using the following procedure.
Important instructions:
Use the ANSWER tool ONLY for the specific questions below. For all other questions, use the ESCALATION tool or COMPLETE.
Questions and answers that can be handled with the ANSWER tool:
1. About changing email address
Example question: I'd like to change my registered email address, how can I do that?
Answer: Let me guide you through the email address change process. First, please log in to our service and go to the account settings page. There, select the option to change email address and enter your new email address. After completing the verification process, your change will be complete.
2. About password reset
Example question: I forgot my password. How can I reset it?
Answer: Let me guide you through the password reset process. Please access the login page and click on the "Forgot password?" link. There, enter your registered email address, and we will send you an email with password reset instructions. Please follow the instructions in the email to set a new password.
3. About product returns
Example question: I want to return a product I purchased, can you tell me how to process it?
Answer: Let me guide you through the return process. First, log in to your account and select the item you wish to return from your purchase history page. Next, click the return process button and select the reason for return. Then, follow the instructions on screen to send the product to the specified address. Once we confirm the return, we will process your refund.
4. About checking shipping status
Example question: How can I check the shipping status of my order?
Answer: Let me guide you through checking your shipping status. Please log in to your account and go to your order history page. By selecting the relevant order, you can check the current shipping status and tracking number.
5. About canceling subscriptions
Example question: I want to cancel my subscription, how do I do that?
Answer: Let me guide you through the subscription cancellation process. First, log in to your account and access the subscription management page. Select the subscription you wish to cancel and click the cancel button. On the confirmation screen, select "confirm cancellation" to complete the cancellation process.
6. About updating credit card information
Example question: I want to update my registered credit card information, how do I do that?
Answer: Let me guide you through updating your credit card information. Log in to your account settings, select the payment methods section. There, enter your new card information and press the save button. This completes the update process.
7. About product warranty periods
Example question: I'd like to know the warranty period for a product I purchased, where can I check this?
Answer: Let me guide you on how to check the warranty period. The warranty period is listed on the product page or in the email sent to you at the time of purchase. Additionally, you can check warranty information by selecting the relevant order from your purchase history in your account.
8. About checking point balance
Example question: Where can I check my current point balance?
Answer: Let me guide you on how to check your point balance. Log in to your account and navigate to the points history page. There, you can check your current point balance.
9. About unsubscribing from email newsletters
Example question: How do I unsubscribe from email newsletters?
Answer: Let me guide you on how to unsubscribe from our newsletters. At the bottom of any email we send, there is an unsubscribe link that you can click. Alternatively, you can also unsubscribe through the notification settings in your account settings.
10. About account deletion
Example question: I want to delete my account, how do I do that?
Answer: Let me guide you through the account deletion process. Log in to your account settings and select the account management option. There, choose the option to delete your account and follow the on-screen instructions to complete the process.
For all other questions (missing products, defect reports, issues with specific orders, technical troubles, etc.), please use the ESCALATION tool.
tools:
- name: ANSWER
description: Use only for the 10 specific questions above and provide the corresponding answer.
input_schema:
type: object
properties:
message:
type: string
description: The corresponding answer from the question and answer list above
required:
- message
- name: ESCALATION
description: Escalate to an agent. Use for all inquiries that are not among the answerable questions.
input_schema:
type: object
properties:
message:
type: string
description: Message to the customer during escalation
summary:
type: string
description: A concise statement describing the reason for the customer's contact and important details.
inquiry:
type: string
description: The customer's inquiry content exactly as stated.
required:
- message
- summary
- inquiry
- name: COMPLETE
description: End the conversation with the customer.
input_schema:
type: object
properties:
message:
type: string
description: The final message you want to send to the customer to end the interaction. This message should be contextual to the conversation and polite.
required:
- message
messages:
- role: user
content: |
<examples>
<example>
<conversation>
[CUSTOMER] I'd like to reset my password.
</conversation>
<thinking>The customer wants to reset their password. This corresponds to question #2 in the list of answerable questions, so I'll use the ANSWER tool.</thinking>
<tool> [ANSWER(message="Let me guide you through the password reset process. Please access the login page and click on the "Forgot password?" link. There, enter your registered email address, and we will send you an email with password reset instructions. Please follow the instructions in the email to set a new password.")] </tool>
</example>
<example>
<conversation>
[CUSTOMER] I ordered a product online 3 days ago and it still hasn't arrived. The estimated delivery date was yesterday. Is there a problem? My order number is 12345.
</conversation>
<thinking>The customer is inquiring about a missing product. This is not included in the list of answerable questions, so I need to use the ESCALATION tool.</thinking>
<tool> [ESCALATION(message="I'll connect you with a representative, please wait a moment.", summary="Customer inquiry about a product ordered 3 days ago (order #12345) that hasn't arrived despite passing the estimated delivery date", inquiry="I ordered a product online 3 days ago and it still hasn't arrived. The estimated delivery date was yesterday. Is there a problem? My order number is 12345.")] </tool>
</example>
<example>
<conversation>
[CUSTOMER] No, that's fine.
</conversation>
<thinking>I asked the customer if they had any inquiries, and they responded that they don't need anything. This conversation has reached a logical conclusion.</thinking>
<tool> [COMPLETE(message="I'm glad I could help. Thank you.")] </tool>
</example>
</examples>
Only use the ANSWER tool when the customer's question corresponds to one of the 10 items above and provide the corresponding response. For everything else, use the ESCALATION tool.
<conversation>
{{$.transcript}}
</conversation>
"""
client = boto3.client('qconnect')
response = client.create_ai_prompt(
apiFormat='MESSAGES',
assistantId=assistant_id,
modelId='apac.anthropic.claude-3-5-sonnet-20241022-v2:0',
name=prompt_name,
templateConfiguration={'textFullAIPromptEditTemplateConfiguration': {'text': yaml_template}},
templateType='TEXT',
type='SELF_SERVICE_PRE_PROCESSING',
visibilityStatus='PUBLISHED'
)
print(json.dumps(response, ensure_ascii=False, indent=2, default=str))
EOF
This prompt is configured to use a custom tool called ANSWER only for specific 10 types of questions (email address change, password reset, product returns, etc.), and to use the ESCALATION tool or COMPLETE tool for all other questions.Run the script and confirm that there are no errors.
python3 prompt.py
After that, create an AI agent with the added AI prompt on the console and set it as the default self-service AI agent.
Connect Flow
This Connect flow performs the following processes:
- Configure initial settings for Japanese voice and [Amazon Q Connect] block
- Call a Lex bot to accept customer inquiries
- Execute branching processes based on the tool selected by the custom AI prompt
- ANSWER tool: Call the Lex bot again after automatic response to accept additional questions
- ESCALATION tool: Execute escalation process
- COMPLETE tool: End the conversation
The custom tool (ANSWER) transitions to the next block after providing an answer, which prevents continuous dialogue with the customer.
Therefore, by calling the Lex bot again after the automatic response, it enables handling additional questions.
This two-stage bot call design creates a flexible flow that allows for additional support after an automated response.
Verification
Conversation Example 1
This is a case that matches a question that can be handled as defined in the prompt - changing an email address.
The ANSWER tool is selected and provides an appropriate response in Japanese.
After that, it checks if there are any additional questions, and since the customer was satisfied, the conversation ends with the COMPLETE tool.
BOT : Please tell me what you're inquiring about.
Customer : Please tell me how to change my email address
ANSWER tool selected
BOT : Let me explain how to change your email address. First, please log in to our service and proceed to the account settings page. There, select the option to change your email address and enter your new email address. Once you complete the confirmation process, the change will be complete.
(Transitions to the next block and calls the Lex bot again)
BOT : Do you have any other questions or concerns? I can also connect you to an operator.
Customer : Thank you. That solved my issue.
COMPLETE tool selected
BOT : I'm very pleased that we were able to resolve your question. If you have any other questions in the future, please feel free to contact us. Thank you.
(Transitions to the next block)
Conversation Example 2
This is a question about product returns, which is also a question that can be handled, so the ANSWER tool is selected.
After the automated response, the customer requested to be transferred to an operator, so the ESCALATION tool was selected and the escalation process was executed.
BOT : Please tell me what you're inquiring about.
Customer : I don't understand how to return an item
ANSWER tool selected
BOT : Let me guide you through the product return process. First, please log in to your account and select the product you wish to return from your purchase history page. Next, click the return procedure button and select your reason for return. Then, follow the instructions displayed on the screen and send the product to the specified address. Once we confirm the return, we will process your refund.
(Transitions to the next flow and calls the Lex bot again)
BOT : Do you have any other questions or concerns? I can also connect you to an operator.
Customer : Could I please speak with an operator?
ESCALATION tool selected
BOT : I'll connect you with an operator now. Please wait a moment.
(Transitions to the next block)
```### Conversation Example 3
This is a highly specific inquiry about undelivered products, which does not fall under the questions defined as answerable in the prompt.
From the beginning, the ESCALATION tool is selected, and the case is appropriately escalated to an operator.
Instead of inappropriate messages like "I don't have an answer," it explains the reason for escalation in polite Japanese.
BOT : Please tell me how I can help you.
Customer : The product that was supposed to arrive yesterday hasn't come yet, what's happening?
ESCALATION tool is selected
BOT : We apologize for the inconvenience. We will check about the delivery delay, so I will connect you to a representative. Please wait a moment.
(Transition to next block)
## In Conclusion
Using Amazon Q in Connect self-service, we built a system that automatically answers specific questions while escalating others.
With this method, common questions can be reliably answered while complex inquiries are appropriately escalated, which can lead to improved customer satisfaction and reduced operator workload.
However, it's necessary to consider operational methods, taking into account that the questions that can be handled are limited and regular maintenance will be required.