A study by Applause [1], a world leader in testing and digital quality, reveals that nearly one-third (30%) of users of chatbots, interactive voice response (IVR) services and conversational assistants are dissatisfied with their experiences. In addition, the study has revealed concerns about inherent bias that can negatively influence engagements and outcomes with AI-driven voice applications.
The results are drawn from Applause's second annual AI and Voice Applications Survey to gain insights into AI user experiences. Conducted in February and March of 2023, the survey collected responses from nearly 5,200 digital quality testing professionals from around the globe.
ChatGPT is proving to be popular among users, with up to three-quarters (74%) responding positively about their experience with the recently released generative AI application. When asked if ChatGPT had understood their questions and provided them with helpful answers, more than two-thirds (67%) responded with "yes". When asked whether ChatGPT or similar technology would help them to complete their work-related tasks, the vast majority responded that it would be "very" (43%) or "somewhat" (37%) helpful.
Conversely, when asked about interactions with chatbots in general, up to 30% said they were dissatisfied with the experience - referring to wrong answers (29%) and a lack of understanding (24%) as reasons for dissatisfaction. Respondents were also frustrated by the failure of chatbots to appreciate language variations or the phrasing of questions. However, one-third indicated that they would use chatbots more if they responded more accurately to questions and voice commands.
Despite these complaints, 92% of respondents expect companies to have chatbots on their apps or websites, eliminating the need for a call. A similar number (86%) said they expect to be greeted by an IVR system when they call a business. Nearly half of respondents (44%) expect mobile apps to have voice assistants.
The study also looked at sentiment around the inherent biases that can affect interactions with AI powered chatbots and smart voice assistants. Bias occurs when the underlying algorithm has been trained with poor and limited data. This impacts its decision making and leads to negative or unanticipated experiences. When questioned about bias in generative AI technology like ChatGPT, an overwhelming 86% of survey respondents expressed their concern.
In the UK, over 7 in 10 (72%) UK respondents had used ChatGPT. More than three-quarters (77%) of those were satisfied with their ChatGPT experience. UK attitudes towards chatbots, IVR and voice assistants were very similar to the global results, with respondents sharing many of the same frustrations. However, nearly a third (27%) commented that chatbots do not always have the answers they're after.
"As AI technology becomes ubiquitous and we rely on it more heavily for mission-critical business functions, it is essential that we focus on quality - both from a data collection standpoint, and from a training and testing perspective. For a second year in-a-row, a vast number of survey respondents shared their frustration with current chatbot interactions. Organizations must focus on bringing real world, unbiased insights into their development processes to drive improvements to their AI experiences and meet their customers' rising expectations," said Luke Damian, Chief Growth Officer for Applause.