Twilio ConversationRelay: Comparing How ElevenLabs v4 and Amazon Polly Read Out Names and Numbers

Introduction

With Twilio ConversationRelay, you can easily build an AI that talks over the phone. Twilio handles speech recognition and text-to-speech, so developers only need to receive text over a WebSocket and send back text as the reply.

The phone reception AI I am building reads back the names and phone numbers it has heard, in the form "Is your answer ~, correct?", so the other party can confirm them. I use Amazon Polly for text-to-speech, and to prevent misreadings I had been converting names and numbers into hiragana readings before sending them.

In October 2026, ElevenLabs v4 became generally available (GA) in ConversationRelay. So I decided to have Polly and ElevenLabs v4 read 20 sentences used in phone reception and compare how they sound.

As a result, ElevenLabs v4 showed improved ability to read out names and numbers compared with Polly. However, v4 does not support SSML or speed settings. I have introduced how to fix Polly's pronunciation with SSML in a previous article. Please refer to it as well.

https://dev.classmethod.jp/articles/twilio-conversationrelay-polly-ssml-japanese-pronunciation/

What is ElevenLabs v4?

ElevenLabs v4 is a text-to-speech model that ElevenLabs released in September 2026. It supports Japanese. In ConversationRelay, it became GA on October 28, 2026 (Twilio changelog).

To use it in ConversationRelay, add -v4 after the voice ID in the voice attribute. Writing -v4_turbo also reads aloud with the same v4 engine (Picking a voice).

<ConversationRelay url="wss://example.com/ws" language="ja-JP"
  ttsProvider="ElevenLabs" voice="3JDquces8E8bkmvbh6Bc-v4" />

Test environment

• Twilio ConversationRelay (language: ja-JP)
• Amazon Polly Takumi-Neural (ttsProvider is Amazon)
• ElevenLabs v4 (the default ja-JP voice 3JDquces8E8bkmvbh6Bc)
• One Node.js test server process
• Test date: November 2, 2026

Target audience

• Those building a Japanese phone AI with Twilio ConversationRelay
• Those who carefully craft the text to be read aloud in order to prevent misreadings of names and numbers

References

• Picking a voice (Twilio Docs)
• ConversationRelay TwiML (Twilio Docs)

Background

Among the sentences to be read aloud, the guidance and question sentences are fixed. If there is a part that gets misread, I can check it in advance and fix it with SSML.

On the other hand, the content of the sentences that confirm names and numbers changes from call to call depending on the other party's answer. It cannot be checked and fixed in advance. Also, the reading of the kanji in a name is not uniquely determined.

In addition, in my AI I also wanted numbers to be read one digit at a time.

So I had the LLM that analyzes the answer produce the reading in hiragana, and sent that hiragana to Polly instead of the written form, as follows.

表記:   ご回答は、XYZ5678、でよろしいですか。
送る文: ご回答は、えっくす わい ぜっと、ごー ろく なな はち、でよろしいですか。

This requires the effort of instructing how to produce the reading for each type of value, such as names, numbers, and dates, and of checking the results.

Test method

Assuming a general reservation reception, I prepared 20 sentences to be read aloud and divided them into the following 2 experiments.

Experiment 1: Fixed sentences

The following 13 sentences were read by Polly and v4 using the same strings.

1. お電話ありがとうございます。ご用件をお聞かせください。
2. お名前をフルネームでお聞かせください。
3. ご連絡のつくお電話番号をお聞かせください。
4. ご予約番号をお聞かせください。英字 3 文字と数字 4 桁です。
5. ご希望の日時をお聞かせください。
6. 何名様でご利用になりますか。
7. ご予約の変更についてのお問い合わせですね。こちらのご用件でよろしいですか。
15. 恐れ入ります。もう一度お聞かせいただけますでしょうか。
16. キャンセルでしょうか、日時の変更でしょうか。
17. 確認のため担当者におつなぎします。少々お待ちください。
18. 本日の受付は午後六時までです。
19. ご予約の変更を承りました。確認のメールをお送りします。
20. お電話ありがとうございました。失礼いたします。

Experiment 2: Sentences confirming names and numbers

The following 7 sentences were read under 4 conditions. Every sentence has the form "Is your answer ○○, correct?", and only the ○○ part differs.

Evaluation criteria

I evaluated on the following 3 levels.

• Good: No sense of unnaturalness in pronunciation
• Acceptable: Pronunciation sounds somewhat off, but it is barely usable for business
• Unacceptable: Not usable for business (in this test, for example, splitting a phone number at positions without a break, or reading the digits of a reservation number as a number, like "five thousand six hundred seventy-eight")

Time until reading starts

I measured, on the server side, the time from sending a sentence until receiving on of agentSpeaking. This is not the time until the sound starts being heard on the handset, but it serves as a guide for comparing the models.

Results

Experiment 1: Fixed sentences

For fixed sentences, v4 was Good for all of them, and about 30% of Polly's were Acceptable. Neither had any Unacceptable results, so for fixed sentences, both models provided speech usable for business.

Experiment 2: Sentences confirming names and numbers

For sentences confirming names and numbers, v4 was Good for 80% or more regardless of how the text was passed, and all were Good when passed in the written form as is. For Polly, converting to hiragana did not raise the proportion of Good, and the phone number (sentence 8) and the reservation number (sentence 11) were Unacceptable with either way of passing.

Time until reading starts

In both experiments, the difference in medians was within 30 ms.

Discussion

At least within the range I tested, v4 played back without any problem even when I passed the sentences without converting them to hiragana, so I think switching to it may reduce the work of crafting the text to be read aloud.

However, after switching, SSML and speed settings can no longer be used. When you want numbers read slowly, you would have to choose a voice whose way of speaking fits. Also, the only fix I tried with Polly was one kind of hiragana notation, and I did not try things like SSML say-as. The judgments were made by me alone, and the speakers also differ between Polly and v4. I think the most reliable approach is to listen to the voices and compare them yourself.

Summary

If you are carefully crafting the text to be read aloud in order to prevent misreadings of names and numbers, please try comparing by ear with ElevenLabs v4 once. When doing so, prepare both the crafted sentences and the sentences in the written form as is, and judge them using recordings with the condition names hidden; this will also let you check whether continuing the crafting is worthwhile. I hope this serves as a reference when choosing a text-to-speech model for a phone AI.