Google Assistant beats rivals to top accuracy tests

Posted by Edith MacLeod on 27 Aug, 2019
View comments Search News
Google Assistant beats Siri and Alexa, but all show continuing improvements in understanding and accuracy as voice assistant use reported to have reached "critical mass".

Voice assistants.

In a study by Loop Venture, Google Assistant came out top over rivals Siri and Alexa, answering 92.9% of 800 questions asked correctly. Apple’s Siri came second with 83.1%, and Amazon's Alexa third with 79.8%.

Each assistant was asked the same questions and graded on 2 metrics:

1. Did it understand what was being asked?
2. Did it deliver a correct response?

In the metric on comprehension, Google Assistant scored 100%, correctly understanding each question. Alexa scored 99,9%, misunderstanding one, while Siri scored 99.8%, misunderstanding two questions.

Test results.

Image source: Loop Venture

All 3 platforms have continued to show improvements in accuracy. The 2018 test scores for correct answers were:  Google Assistant 86%, Siri 79% and Alexa 61%. Cortana was eliminated from the test due to Microsoft’s shift in Cortana’s strategic positioning.

The questions were broken into 5 categories:

  • Local – Where is the nearest coffee shop?
  • Commerce – Order me more paper towels.
  • Navigation – How do I get to Uptown on the bus?
  • Information – Who do the Twins play tonight?
  • Command – Remind me to call Jerome at 2 pm today.

Category queries.

Image source: Loop Venture

Loop Venture said it believed the number of consumers making purchases through voice commands was not significant, with queries more related to product information and reseearch.

“We believe, based on surveying consumers and our experience using digital assistants, that the number of consumers making purchases through voice commands is insignificant. We think commerce-related queries are more geared toward product and service research and local business discovery than actually purchasing something, and our question set reflects that.”

It noted that with scores nearing 100%, it may seem that virtual assistants may soon be able to answer any question you pose, but that this was not the case.

“Today, they are able to understand, within reason, everything you say to them, and the primary use cases are well built out, but they are not generally intelligent.

Further improvements will come from extending the feature sets of these assistants. New skills are hard to discover and, therefore, don’t get used. To become a habit, new use cases need to be well understood, simple to use, and solve a problem that voice is uniquely suited to solve.“

Voice assistant use moves into the mainstream

In a separate report earlier this year, eMarketer said that voice assistant use had reached “critical mass”, moving out of the early adopter phase and into the mainstream.

eMarketer estimated that 111.8m people in the US will use a voice assistant at least monthly in 2019, up 9.5% from 102m last year. This represents 39.4% of US internet users and 33.8% of the overall population.  

Voice assistant use.

Most people use assistants on smartphones which come with the technology pre-installed or downloadable. eMarketer said it expected the number of voice assistant users to rise as the software becomes available on more smart devices and gadgets. It forecasts US voice assistant users to reach 122.7 million by 2021, representing 42.2% of US internet users and 36.6% of the overall population.

Further reading

Your essential guide to researching keywords for Voice Search Optimization

The 4 best ways to optimize your site for voice search

Voice search 101 for marketers - infogfraphic

Recent articles

How to use word-of-mouth marketing [infographic]
Posted by Wordtracker on 3 May 2022
New Microsoft and Google digital marketing certifications
Posted by Edith MacLeod on 3 May 2022
New profile link feature announced by LinkedIn
Posted by Edith MacLeod on 3 May 2022
5 website strategies to engender user trust
Posted by Julie Weishaar on 30 April 2022
How to recognize and avoid phishing attacks [infographic]
Posted by Wordtracker on 26 April 2022