I tried implementing a follow-up question widget in Google Chat Bot — Using inline forms in cardsV2

Introduction

In the previous article, we implemented a progressive update UX with cardsV2 for a Google Chat Bot, enabling real-time display of RAG pipeline progress.

This time, we're adding an interactive clarification widget to that bot. When a user sends an ambiguous query (e.g., just the single word "macro"), instead of immediately generating a response, the bot asks "What are you looking for regarding this topic?" and displays option buttons along with a free-text input field.

We also implemented a mechanism to reliably append escalation guidance to fallback responses (when information is not in the knowledge base) and confidential information refusals (passwords, etc.).

Configuration

Problem: Responding Immediately to Ambiguous Queries

During ongoing operation of the RAG bot, we frequently encountered cases where users sent ambiguous queries.

User: "macro"
Bot:  Let me explain Excel macros... (the user actually wanted to know about Outlook macros)

When the knowledge base contains multiple topics such as "Excel macros," "Outlook macros," and "Google Sheets macros," the LLM picks one and responds. Returning a response that misses the user's intent is a worse experience than not responding at all.

The ideal flow looks like this:

User: "macro"
Bot:  Regarding "macro," what are you looking for?
      [Excel macros] [Outlook macros] [Spreadsheet macros]
      [Free text input] [Submit]
User: Clicks [Excel macros]
Bot:  Here are the steps for setting up Excel macros...

Implementation 1: Detecting Ambiguity with Pydantic Structured Output

Extending the RagAnswer Schema

First, we add clarification fields to the LLM output schema. The google-genai SDK can pass a Pydantic BaseModel directly to response_schema, allowing type-safe definition of structured output.

from pydantic import BaseModel, Field

class RagAnswer(BaseModel):
    """Structured output from Gemini for RAG responses."""

    answer: str = Field(
        description="Answer based on knowledge base information (Markdown format)"
    )
    is_answerable: bool = Field(
        description="True if relevant information was found in the knowledge base and a response was provided"
    )
    needs_clarification: bool = Field(
        default=False,
        description="True if the query is ambiguous and intent cannot be determined. "
        "If true, put the clarification question in answer "
        "and put the options in clarification_options",
    )
    clarification_options: list[str] = Field(
        default_factory=list,
        description="Options for clarification (up to 3). "
        "Only used when needs_clarification is true",
    )

Adding Instructions to the System Prompt

The schema alone won't make the LLM ask clarifying questions. We include clear judgment criteria in the system prompt.

## Handling Ambiguous Queries
- If the user's question is ambiguous and the knowledge base search results match multiple different topics:
  - Set needs_clarification to true
  - Put the clarification question in answer
  - Put up to 3 options based on search results in clarification_options
- Even if the question is a single word, if the search results concentrate on one topic, respond immediately without asking for clarification
- If there are 0 search results, do not ask for clarification; set is_answerable to false

The key point is that "single word ≠ ambiguous". For a word like "VPN" where all search results point to the same topic (VPN connection troubleshooting), it's the right experience to respond immediately even with a single word. We have the model determine ambiguity based on the diversity of search results.

Generation Call

The call using the google-genai SDK is the same as regular structured output.

from google import genai
from google.genai import types

client = genai.Client(vertexai=True, project=PROJECT_ID, location=LOCATION)

response = client.models.generate_content(
    model="gemini-2.5-flash",
    contents=prompt,
    config=types.GenerateContentConfig(
        system_instruction=SYSTEM_PROMPT,
        response_schema=RagAnswer,
        response_mime_type="application/json",
    ),
)

result = response.parsed  # → RagAnswer instance

Since response.parsed directly returns a Pydantic model instance, there's no need to write parsing logic. If result.needs_clarification is True, we branch to the clarification flow.

Implementation 2: Building the Clarification Card with cardsV2

Card Structure

The clarification card consists of the following elements:

1. textParagraph — The clarification question generated by the LLM (e.g., "Regarding macros, what are you looking for?")
2. buttonList — Up to 3 option buttons
3. textInput — Free text input field (for cases not covered by the button options)
4. buttonList — Submit button

def _build_clarification_section(clarification_text, options,
                                 message_name, endpoint_url,
                                 space_name, thread_name, user_name,
                                 original_query=""):
    function_url = endpoint_url or ""
    base_params = [
        {"key": "action", "value": "clarify"},
        {"key": "message_name", "value": message_name or ""},
        {"key": "space_name", "value": space_name or ""},
        {"key": "thread_name", "value": thread_name or ""},
        {"key": "user_name", "value": user_name or ""},
        {"key": "original_query", "value": original_query or ""},
    ]

    widgets = [
        {"textParagraph": {"text": clarification_text}},
    ]

    # Option buttons (up to 3)
    if options:
        buttons = []
        for option in options[:3]:
            buttons.append({
                "text": option,
                "onClick": {
                    "action": {
                        "function": function_url,
                        "parameters": base_params + [
                            {"key": "query", "value": option},
                        ],
                    }
                },
            })
        widgets.append({"buttonList": {"buttons": buttons}})

    # Free text input field
    widgets.append({
        "textInput": {
            "label": "Enter here if none of the above apply",
            "type": "SINGLE_LINE",
            "name": "clarify_text",
        }
    })

    # Submit button (for free text input)
    widgets.append({
        "buttonList": {
            "buttons": [{
                "text": "Submit",
                "onClick": {
                    "action": {
                        "function": function_url,
                        "parameters": base_params + [
                            {"key": "query", "value": ""},
                        ],
                    }
                },
            }]
        }
    })

    return {"widgets": widgets}

Can textInput Be Used in Inline Cards?

Before implementing, one concern was Google Chat's textInput widget. While the documentation primarily covers it in the context of dialogs (modals displayed via renderActions), it works fine in inline message cards as well.

When a user enters text and clicks the submit button, the CARD_CLICKED event's commonEventObject.formInputs contains the form data.

{
  "commonEventObject": {
    "formInputs": {
      "clarify_text": {
        "stringInputs": {
          "value": ["Text entered by the user"]
        }
      }
    },
    "parameters": {
      "action": "clarify",
      "query": ""
    }
  }
}

Notes on Parameter Design

Google Chat's action.parameters are all string types. Numeric values and booleans cannot be passed. Therefore, values like message_name (the resource name of an existing message) and thread_name are also passed as strings, with empty strings treated as "not set."

Also, for option buttons, the button text is placed directly in the query parameter, while the submit button uses an empty string for query. This makes it easy for the click handler to distinguish between "button click" and "free text input."

Implementation 3: Handling CARD_CLICKED Events

Routing

We process the CARD_CLICKED event that Google Chat sends when a button is clicked. Since it uses the same event format as the feedback buttons implemented in the previous article, we branch using parameters.action.

if "buttonClickedPayload" in chat:
    action = params.get("action")

    if action == "clarify":
        return handle_clarify(body, endpoint_url=endpoint_url)

    if action == "feedback":
        # Existing feedback processing
        ...

Click Handler

import threading
from bot.worker import process_message

def handle_clarify(event_body, endpoint_url=""):
    common_event = event_body.get("commonEventObject", {})
    params = common_event.get("parameters", {})
    form_inputs = common_event.get("formInputs", {})

    # Button click → text is in params["query"]
    # Free text + submit button → params["query"] is empty, text is in formInputs
    query = params.get("query", "")
    if not query:
        text_input = form_inputs.get("clarify_text", {})
        values = text_input.get("stringInputs", {}).get("value", [])
        query = values[0].strip() if values else ""

    if not query:
        return {}  # Ignore empty query

    # Combine original query and clarification response to preserve context
    original_query = params.get("original_query", "")
    if original_query and query != original_query:
        query = f"{original_query} (about {query})"

    message_name = params.get("message_name", "")
    space_name = params.get("space_name", "")
    thread_name = params.get("thread_name", "")
    user_name = params.get("user_name", "")

    user = event_body.get("chat", {}).get("user", {})
    sender = user.get("displayName", "")

    thread = threading.Thread(
        target=process_message,
        args=(space_name, query, sender),
        kwargs={
            "endpoint_url": endpoint_url,
            "thread_name": thread_name or None,
            "user_name": user_name,
            "message_name": message_name or None,
            "is_clarification": True,
        },
    )
    thread.start()

    return {}

There are four key points.

Prioritize the button's query

When a user clicks an option button, even if something is entered in the textInput, a non-empty string is present in params["query"]. Since the button click intent is clear, this takes priority. For the submit button where query is empty, we retrieve the value from formInputs.

Preserve context of the original question with original_query

The original query (original_query) is embedded in the base_params of the clarification card. When the user responds to the clarification, this original query is combined with the user's response before the pipeline is re-executed.

For example, if the original query was "Tell me how to apply for a tool" and the user entered "kintone" in the free text field, the final query becomes Tell me how to apply for a tool (about kintone). This allows KB searches while preserving the context of "how to apply."

Why this is necessary is explained in detail in the "Pitfalls" section below.

Pass message_name to patch the existing card

In normal MESSAGE events, process_message creates a new message card, but when re-executing from a clarification, we want to overwrite the original clarification card. By passing message_name (the resource name of the existing message), the worker uses patch_message instead of create_message.

# Inside process_message
if message_name is None:
    message_name = chat_client.create_message(space_name, body, thread_name=thread_name)
else:
    chat_client.patch_message(message_name, body, "cardsV2")

Return {} immediately

Returning {} to a CARD_CLICKED event tells Google Chat to make no changes (the original card remains). Since the background thread patches the card via the Chat API, the user experiences a seamless transition from the clarification card to the answer card.

Implementation 4: Preventing Infinite Loops

What happens if the result of a clarification is again judged as "ambiguous"?

User: "macro"
Bot:  Clarification card → [Excel macros]
Bot:  Another clarification card → Infinite loop!

To prevent this, when executed with is_clarification=True, we forcibly override needs_clarification to False even if the LLM returns needs_clarification=True.

# Infinite loop prevention
if is_clarification and result.needs_clarification:
    result = RagAnswer(
        answer=result.answer,
        is_answerable=result.is_answerable,
        needs_clarification=False,
        clarification_options=[],
    )

If insufficient information is obtained from one clarification, it's a better user experience to provide the best possible answer with the available information rather than asking again.

Implementation 5: Reliably Appending Escalation Text

Separately from the clarification feature, there was also an issue where escalation guidance to the responsible department was missing from fallback responses (when information is not in the knowledge base) and confidential information refusals.

Instructions in the System Prompt

## Constraints
- Do not directly answer passwords or account information
  (only provide procedural guidance, and always append the following sentence at the end of the response):
  "Please contact YOUR_ESCALATION_TEXT."
- If there is no relevant information in the knowledge base, always append the following sentence at the end of the response:
  "Please contact YOUR_ESCALATION_TEXT."

Safety Net: Forced Appending via Post-Processing

LLMs sometimes don't follow instructions. In particular, for boilerplate text like escalation messages, there were cases where the model would omit them by judging them as "unnecessary."

Therefore, we check the LLM output in post-processing and forcibly add the escalation text if is_answerable=False but the escalation text is not included.

ESCALATION_TEXT = "Please contact YOUR_ESCALATION_TEXT."

def _ensure_escalation(result: RagAnswer) -> RagAnswer:
    """Append escalation text to non-answerable responses if missing."""
    if result.needs_clarification:
        return result  # Skip during clarification
    if not result.is_answerable and ESCALATION_TEXT not in result.answer:
        return RagAnswer(
            answer=result.answer.rstrip() + "\n" + ESCALATION_TEXT,
            is_answerable=False,
        )
    return result

The key point is skipping when needs_clarification=True. Since clarification represents a state of "not yet answered," adding escalation text at this point would be inappropriate.

We apply this function to all return paths of generate_answer().

def generate_answer(question, contexts, history=None):
    if not contexts:
        return RagAnswer(answer=NO_RESULT_MESSAGE, is_answerable=False)

    # ... LLM call ...

    try:
        if isinstance(response.parsed, RagAnswer):
            return _ensure_escalation(response.parsed)
        if response.text is not None:
            return _ensure_escalation(RagAnswer.model_validate_json(response.text))
    except (ValueError, TypeError):
        logger.warning("Failed to parse structured response")
    return RagAnswer(answer=NO_RESULT_MESSAGE, is_answerable=False)

Overall Flow

Here is the complete picture of the final clarification flow.

Phase 1 (Initial Query): When the user sends "macro," the RAG pipeline searches the KB and returns needs_clarification=True if multiple topics are detected. A clarification card (option buttons + text input + submit) is patched via the Chat API.

Phase 2 (Clarification Response): When the user clicks a button or enters text, handle_clarify() extracts the query from the CARD_CLICKED event. It combines the original_query (original query) embedded in the card parameters with the user's response to re-execute the pipeline with is_clarification=True while preserving context. A second clarification is forcibly disabled, and the existing card is overwritten with the answer via patch_message.

Pitfalls

formInputs Are Sent Even in Inline Cards

The Google Chat documentation explains formInputs in the context of dialogs, but even when textInput is placed in an inline message card, formInputs is correctly sent when the button is clicked. There is no need to use dialogs.

All Parameters Are Strings

The value in action.parameters is string type only. All information needed for re-execution, such as message_name and thread_name, is passed as strings. Empty strings "" are treated as "not set" and converted to or None on the handler side.

The Context of the Original Query Disappears in Clarification Responses

This was the trickiest problem. It occurs in the following scenario.

User: "Tell me how to apply for a tool"
Bot:  Clarification card → [kintone] [Salesforce] [Microsoft 365] + free text input
User: Enters "kintone" in free text and submits
Bot:  Responds with a general description of kintone ← The "how to apply" context is gone!

The cause is that the original query is not saved to conversation history when clarification occurs. With normal follow-up questions, the query is reconstructed using the Firestore conversation history, but during clarification, save_turn() is not called because no answer has been returned. As a result, the KB search is performed using only the clarification response "kintone," and the "how to apply" context is completely lost.

As a solution, we adopted an approach of embedding the original query (original_query) in the clarification card's action.parameters.

# cards.py — add original_query to base_params
base_params = [
    {"key": "action", "value": "clarify"},
    {"key": "original_query", "value": original_query or ""},
    # ...
]

# clarify.py — combine and re-execute the pipeline
original_query = params.get("original_query", "")
if original_query and query != original_query:
    query = f"{original_query} (about {query})"

We also considered a Firestore-based approach, but the card parameter approach is superior in the following ways:

• Deterministic — No LLM-based reconstruction is needed, incurring no additional latency or cost
• Firestore-independent — Since the context is embedded in the card widget itself, it works even during Firestore outages
• No history pollution — No need to save incomplete clarification exchanges to conversation history

Don't Save Clarification Exchanges to Conversation History

The clarification exchange ("Which one are you asking about?" → button selection) is meta-conversation, not actual Q&A. Saving it to conversation history would make the context incorrect in subsequent follow-up questions. We skip save_turn() when needs_clarification=True.

Priority Between Button query and formInputs

When an option button is clicked, if the user has entered something in the textInput, both pieces of data are included in the CARD_CLICKED event. We designed it so that if a non-empty query is present in the button's parameters, that takes priority, and formInputs is only referenced for the submit button (with query="").

Summary

• By delegating ambiguity judgment to the LLM via Pydantic structured output, we naturally achieved the determination of "is this query ambiguous?" which would be difficult with rule-based approaches
• We built an interactive clarification UI with cardsV2's ButtonList + TextInput. It works entirely with inline cards without needing dialogs
• The CARD_CLICKED → {} + background thread pattern enables seamless overwriting of existing cards
• Embedding original_query in card parameters preserves the context of the original question (e.g., "how to apply") when responding to clarifications — a deterministic approach that doesn't rely on conversation history
• The is_clarification flag for infinite loop prevention is essential. A policy of returning the best possible answer after one clarification is simple and effective
• The _ensure_escalation() post-processing pattern is generally applicable for cases where you want to reliably include boilerplate text in LLM structured output

Google Chat's cardsV2 can receive form data with textInput even without dialogs, making it sufficiently practical as a means of achieving interactive chat bot experiences.

References

• Google Chat cards v2 widgets | Google for Developers
• Receive and respond to interactions | Google Chat
• TextInput widget | Google Chat cards v2
• google-genai Python SDK | GitHub
• Previous article: The Many Walls We Hit Implementing cardsV2 Progressive UX in Google Chat Bot