Googlebot will soon start crawling over HTTP/2

Posted by Edith MacLeod on 22 Sep, 2020
View comments SEO
Google is moving forward with selected sites from November. Here’s what you need to know.

Googlebot HTTP/2

HTTP/2 (h2) is the upgraded, more modern version of HTTP, the protocol used to transfer data over the internet. It’s a major revision which will be more efficient and so faster in transferring data.

With Google announcing it's now ready to move ahead, the question arises - should you switch over to HTTP/2?

For the moment, there’s no rush. Google has confirmed there is no ranking benefit to HTTP/2.

They will start with a small number of sites from mid-November which are both ready for HTTP/2 and will benefit from the switch, but continue using HTTP/1.1 for the rest.  There’s no disadvantage to your site if it doesn’t support HTTP/2.

The detail

Ranking and indexing

Google says categorically, there is no ranking benefit for a site being crawled over HTTP/2.

Preliminary tests showed no issues or negative impact on indexing.

“Whether we crawl using h1 or h2 does not affect how your site is indexed, and hence it does not affect how much we plan to crawl from your site.”

Switching sites

Google will only switch over sites that support HTTP/2 and which will clearly benefit from it.

It said tests showed "little to no benefit" for certain sites when crawling over HTTP/2. If there's no clear benefit for a site then Googlebot will continue to crawl over HTTP/1.1, as now.

Performance gains will be monitored and the criteria for switching may change over time.

Starting from mid-November, the first phase will involve a small number of sites, ramping up gradually to more sites “that may benefit from the initially supported features, like request multiplexing”.

Opting out

If your site is eligible for the connection upgrade you don’t need to do anything; you can’t opt in.

You can, however, opt out if you wish, by responding with a 421 HTTP status code when Googlebot attempts to crawl your site over HTTP/2, or, as a temporary solution, contacting Googlebot.

Google says there’s no drawback if your site doesn’t support HTTP/2, “crawling will remain the same, quality and quantity wise”.

Monitoring your site

There’s a handy online tool you can use to check if your site supports HTTP/2.

HTTP/2 test.

When your site becomes eligible for crawling over HTTP/2, you’ll see a message to this effect in Search Console. You can also check your server logs.

Benefits of HTTP/2

In their blog announcement, Google says the change makes crawling more efficient, as Googlebot will be able to transfer multiple files over a single TCP connection, and save resources through efficient compression of HTTP header fields

“With h2, Googlebot is able to open a single TCP connection to the server and efficiently transfer multiple files over it in parallel, instead of requiring multiple connections. The fewer connections open, the fewer resources the server and Googlebot have to spend on crawling.”

Google outlined the benefits of HTTP/2 as:

  • Multiplexing and concurrency: Fewer TCP connections open means fewer resources spent.
  • Header compression: Drastically reduced HTTP header sizes will save resources.
  • Server push: This feature is not yet enabled; it's still in the evaluation phase. It may be beneficial for rendering, but we don't have anything specific to say about it at this point.

A post on Google’s Web Fundamentals further details the benefits and potential of HTTP/2:

“HTTP/2 will make our applications faster, simpler, and more robust — a rare combination — by allowing us to undo many of the HTTP/1.1 workarounds previously done within our applications and address these concerns within the transport layer itself. Even better, it also opens up a number of entirely new opportunities to optimize our applications and improve performance!”

https://developers.google.com/web/fundamentals/performance/http2

It also chronicles the development history and technical details.

Recent articles

Google launches Ads in AI Overviews
Posted by Edith MacLeod on 7 October 2024
Google rolls out AI-organized search results
Posted by Edith MacLeod on 7 October 2024
New AI video capabilities for Ads and YouTube
Posted by Edith MacLeod on 24 September 2024
Why web hosting matters - optimising UX and Core Web Vitals
Posted by Daniel Watkinson on 18 September 2024
Google introduces confidential matching
Posted by Edith MacLeod on 15 September 2024