Google Assistant beats rivals to top accuracy tests

Posted by Edith MacLeod on 27 Aug, 2019
View comments Search News
Google Assistant beats Siri and Alexa, but all show continuing improvements in understanding and accuracy as voice assistant use reported to have reached "critical mass".

Voice assistants.

In a study by Loop Venture, Google Assistant came out top over rivals Siri and Alexa, answering 92.9% of 800 questions asked correctly. Apple’s Siri came second with 83.1%, and Amazon's Alexa third with 79.8%.

Each assistant was asked the same questions and graded on 2 metrics:

1. Did it understand what was being asked?
2. Did it deliver a correct response?

In the metric on comprehension, Google Assistant scored 100%, correctly understanding each question. Alexa scored 99,9%, misunderstanding one, while Siri scored 99.8%, misunderstanding two questions.

Test results.

Image source: Loop Venture

All 3 platforms have continued to show improvements in accuracy. The 2018 test scores for correct answers were:  Google Assistant 86%, Siri 79% and Alexa 61%. Cortana was eliminated from the test due to Microsoft’s shift in Cortana’s strategic positioning.

The questions were broken into 5 categories:

  • Local – Where is the nearest coffee shop?
  • Commerce – Order me more paper towels.
  • Navigation – How do I get to Uptown on the bus?
  • Information – Who do the Twins play tonight?
  • Command – Remind me to call Jerome at 2 pm today.

Category queries.

Image source: Loop Venture

Loop Venture said it believed the number of consumers making purchases through voice commands was not significant, with queries more related to product information and reseearch.

“We believe, based on surveying consumers and our experience using digital assistants, that the number of consumers making purchases through voice commands is insignificant. We think commerce-related queries are more geared toward product and service research and local business discovery than actually purchasing something, and our question set reflects that.”

It noted that with scores nearing 100%, it may seem that virtual assistants may soon be able to answer any question you pose, but that this was not the case.

“Today, they are able to understand, within reason, everything you say to them, and the primary use cases are well built out, but they are not generally intelligent.

Further improvements will come from extending the feature sets of these assistants. New skills are hard to discover and, therefore, don’t get used. To become a habit, new use cases need to be well understood, simple to use, and solve a problem that voice is uniquely suited to solve.“

Voice assistant use moves into the mainstream

In a separate report earlier this year, eMarketer said that voice assistant use had reached “critical mass”, moving out of the early adopter phase and into the mainstream.

eMarketer estimated that 111.8m people in the US will use a voice assistant at least monthly in 2019, up 9.5% from 102m last year. This represents 39.4% of US internet users and 33.8% of the overall population.  

Voice assistant use.

Most people use assistants on smartphones which come with the technology pre-installed or downloadable. eMarketer said it expected the number of voice assistant users to rise as the software becomes available on more smart devices and gadgets. It forecasts US voice assistant users to reach 122.7 million by 2021, representing 42.2% of US internet users and 36.6% of the overall population.

Further reading

Your essential guide to researching keywords for Voice Search Optimization

The 4 best ways to optimize your site for voice search

Voice search 101 for marketers - infogfraphic

Recent articles

Google launches new personalisation options in Search
Posted by Edith MacLeod on 27 November 2023
Google adds small business filter to Search and Maps
Posted by Edith MacLeod on 21 November 2023
Google releases Nov 2023 reviews update
Posted by Edith MacLeod on 9 November 2023
Interactive content: engaging your audience in the digital age
Posted by Brian Shelton on 8 November 2023
Google releases November 2023 core update
Posted by Edith MacLeod on 3 November 2023