HTTP is at the foundation of the World Wide Web. Analysing the HTTP status code of your URLs and ensuring that they respond as expected is at the foundation of your website’s digital presence and performance.
HTTP refers to the Hypertext Transfer Protocol. In the most simple terms, HTTP is a protocol used to exchange data between web clients (like browsers) and servers (where data is stored).
Clients, or web browsers, use HTTP requests to ask the server for the information they need to load a website, once a user has clicked on a link in search results, for example. The server then receives the request, processes it, and returns an HTTP response to the browser based on what was contained in the request.
The HTTP status code is a part of the HTTP request, alongside HTTP response headers and the HTTP body. These contain the files and data needed to correctly load the content of a page (HTML, CSS, JS, images, XML and JSON).
The HTTP status code is a 3-digit code contained in the server response. It indicates whether a request has been successfully completed, redirected, or whether an error arose.
HTTP status codes can be grouped into 5 categories:
These are informational status codes, issues provisionally to communicate to the client that the request has been received and is being processed.
2xx are success codes. They indicate that the request was successfully processed and the response contains the information needed to load the page.
When it comes to SEO and organic performance, we expect all relevant live URLs to return 200 OK responses, to be eligible for indexing.
3xx codes indicate redirection. The client’s request was received but further action is needed to complete the request. There are several types of 3xx redirects, with different implications for SEO.
Correctly using and implementing redirects is essential to the correct management of the crawl budget, website migrations, and passing on relevance signals when pages or resources are moved, among others.
Your SEO strategy should include the regular review of URLs with 3xx responses, to ensure that these are used appropriately, that redirect chains are avoided and that permanently moved URLs and resources are not linked internally.
HTTP 4xx status codes indicate that there was an error on the client’s side when processing the request. This may be a request for a non-existent page, too many requests, or a lack of authentication, among others.
When returning this kind of response, the server will generally include some content to give the client an explanation of the error.
All of the 4xx errors described above are treated in the same way by Googlebot. The crawler understands that the content has been removed, the crawling frequency decreases and they are ultimately removed from the index if the URL was indexed to begin with.
Both 4xx errors should be addressed as a priority in any SEO campaign, as these codes mean that your users may be encountering pages that do not load, cutting their journey on your website short. The same is true for crawlers that may be encountering these pages through internal linking.
Ranking URLs that have been removed for justified reasons, should present a 404 code or be redirected where an equivalent or relevant page is available, as indexed pages that consistently return an error will be dropped from rankings.
HTTP 5xx status codes indicate that the client’s request was valid but there was an error on the server’s side, preventing it from completing the request.
Just like with 4xx errors, it is essential to keep tabs on and address 5xx errors as part of your SEO strategy. When a website consistently returns 5xx errors, search crawlers are likely to drop these results from rankings and slow down the crawling of the website if they conclude that the server can’t handle the current rate of requests.
Being able to find, identify and analyse successful requests, redirects and errors is essential to managing and optimising the crawling and indexing of your website. Making sure that URLs return the HTTP status codes we expect them to, help us ensure that sites are optimised for both users and search engines. As such, understanding what different status codes mean can help identify which codes will deliver the desired result.
There are over 70 HTTP status codes, as listed in the official HTTP Status Code Registry. While these are essential to the workings of the World Wide Web, it is unlikely for most users to come across more than a handful of these.
You can monitor Your website’s response stats through a variety of tools.
Google Search Console provides a breakdown of crawled URLs by response. This report allows you to investigate the issues that the Google crawlers encounter on your website and monitor these over time. To understand how Google treats the errors flagged up in the Google Search Console report, see the official documentation.
Website crawling tools like Sitebulb and Screaming Frog also allow you to analyse the response codes for all of the pages and resources on your website, find potential issues, as well as flag up URLs where the crawler was unable to gather a response from the server.
Understanding status codes is essential in making sure that your website is crawlable and responding as expected to crawler’s and user’s requests. At Wildcat Digital, making sure that your website can be crawled, indexed and follows best practice is at the core of our SEO strategies.
Rely on the experts to improve your website’s organic performance and punch above the waist online. Get in touch today for a free consultation.
