The theory that voice is the next big interface has gained considerable momentum, with investors pouring billions of dollars into voice AI startups working in areas ranging from model makers to enterprise customer service providers to meeting note-taking and AI-powered dictation.
Every week there’s a new model or tool version that claims to look and converse like a human. However, in reality, this may not be the case. Shawn Wen, CTO of enterprise voice AI platform PolyAI, believes that despite the release of full-duplex models – which can speak while listening to you – voice AI has not yet had its “ChatGPT moment”.
“We’ve reached the stage of developing full-duplex models. The next challenge is to make the reasoning very fast, so that the models can get answers quickly and the conversation feels natural,” he told me on stage at the HumanX conference last month.
He also said that AI customer service agents should not appear robotic and should give callers enough confidence in their ability to resolve problems.
“I think the next step will be slightly different because once the voice is good enough and the customer is willing to interact with it for the first two or three rounds, they start to gain confidence and, over time, they will feel like I probably don’t need to talk to a human if the agent can solve my problem,” he said.
Alex Gay, CMO of meeting note taker Otter, said identifying the speaker, capturing their intent and capturing it with organizational knowledge is a key step in enabling automation. The company is also working on digital twins that could represent people in meetings. For this technology, he said it was essential that the output voice gave the same emotional expressions as when speaking to a human in a meeting.
“If you think about the meetings you’re in right now, the best conversations you have are the ones where you can have debates and strategic discussions, and where you feel like there’s a relationship underlying it. If you can’t have that with an avatar, then it’s just AQ and a chatbot,” Gay noted.
Understanding and Transparency of Voice AI
Although voice AI models have improved, AI assistants often don’t understand users, or your meeting’s note-taker displays a bad transcript or summary.
Wen believes that Automatic Speech Recognition (ASR) models often miss important keywords, which creates a problem in capturing the entire context.
Otter’s Gay agrees, adding that the company continues to work on improving transcription. He also said that language is an area where speech patterns need improvement.
“For Otter, you know, transcription was never the end point. It was just the layer where we could start to drive some productivity gains. But if your original transcription didn’t have the accuracy that you needed, all the follow-up actions that you needed became imperfect. And as soon as you start taking action, it’s wrong. You lose confidence in the platform. It’s critical for us to continue to improve this ASR model because all the downstream impacts are important,” he said.
With the new voice tools, there is also a question of transparency. Tools must declare to customers that they are being recorded or talking to the AI. Otter said he wants to instill confidence in those attending a meeting. So even for a meeting where the bot is not present, it wants to try methods such as notifying everyone in the chat that the meeting is being recorded. PolyAI’s Wen also said it was important to establish that people are talking to an AI on business calls.
When you purchase through links in our articles, we may earn a small commission. This does not affect our editorial independence.
Gn tech
