HTTP is at the foundation of the World Wide Web. Analysing the HTTP status code of your URLs and ensuring that they respond as expected is at the foundation of your website’s digital presence and performance.
What is HTTP?
HTTP refers to the Hypertext Transfer Protocol. In the most simple terms, HTTP is a protocol used to exchange data between web clients (like browsers) and servers (where data is stored).
Requests and Responses
Clients, or web browsers, use HTTP requests to ask the server for the information they need to load a website, once a user has clicked on a link in search results, for example. The server then receives the request, processes it, and returns an HTTP response to the browser based on what was contained in the request.
The HTTP status code is a part of the HTTP request, alongside HTTP response headers and the HTTP body. These contain the files and data needed to correctly load the content of a page (HTML, CSS, JS, images, XML and JSON).
What are HTTP Status Codes?
The HTTP status code is a 3-digit code contained in the server response. It indicates whether a request has been successfully completed, redirected, or whether an error arose.
HTTP status codes can be grouped into 5 categories:
1xx status codes – informational
These are informational status codes, issues provisionally to communicate to the client that the request has been received and is being processed.
2xx status codes – success
2xx are success codes. They indicate that the request was successfully processed and the response contains the information needed to load the page.
- 200 OK: The page requested exists and the server response was successful.
When it comes to SEO and organic performance, we expect all relevant live URLs to return 200 OK responses, to be eligible for indexing.
3xx status codes – redirection
3xx codes indicate redirection. The client’s request was received but further action is needed to complete the request. There are several types of 3xx redirects, with different implications for SEO.
- 301 Moved Permanently: Indicates that the resource has been permanently moved to a different location. It sends a strong signal that all signals should be passed onto the destination URL. Googlebot may understand the destination URL as canonical and cache the destination URL.
- 302 Found: Indicates that the resource has temporarily moved to another location. It sends weak signals, and relevance signals won’t be passed on internally. However, if left in place for extended periods of time, crawlers will treat them as permanent redirects.
- 303 See Other: This response is used in particular situations when the server wants the client to use a different URL to access the resource requested. This may be used to avoid form resubmission or
- 307 Temporary Redirect: Equivalent to 302. While there are some
- technical differences between the two, Googlebot and other crawlers will treat these similarly.
Correctly using and implementing redirects is essential to the correct management of the crawl budget, website migrations, and passing on relevance signals when pages or resources are moved, among others.
Your SEO strategy should include the regular review of URLs with 3xx responses, to ensure that these are used appropriately, that redirect chains are avoided and that permanently moved URLs and resources are not linked internally.
4xx status codes – client error
HTTP 4xx status codes indicate that there was an error on the client’s side when processing the request. This may be a request for a non-existent page, too many requests, or a lack of authentication, among others.
When returning this kind of response, the server will generally include some content to give the client an explanation of the error.
- 400 Bad Request: The server will not or cannot process the request due to an error on the client’s side
- 401 Unauthorised: The client does not have the credentials to access the requested page or resource. This may be used to stop search engines from crawling and indexing staging websites, for example.
- 403 Forbidden: The server understands the request but refuses it. This is normally a permanent measure implemented by the server and the client should not repeat the request.
- 404 Not Found: The resource requested cannot be found by the server. This is the most common 4xx error, used to indicate that a page has been removed, a URL has been mistyped, or a link is broken, among others.
- 410 Gone: A previously existing resource is no longer available on the server.
All of the 4xx errors described above are treated in the same way by Googlebot. The crawler understands that the content has been removed, the crawling frequency decreases and they are ultimately removed from the index if the URL was indexed to begin with.
- 429 Too Many Requests: This error indicates that the client has made too many requests within a limited period of time and the server has blocked their access temporarily because of this limit. It can be understood as a temporary 403 error. Gogglebot understands this code as a server error, with the assumption that the server was overloaded. Over time, this signal can lead to a reduction in the crawl budget.
Both 4xx errors should be addressed as a priority in any SEO campaign, as these codes mean that your users may be encountering pages that do not load, cutting their journey on your website short. The same is true for crawlers that may be encountering these pages through internal linking.
Ranking URLs that have been removed for justified reasons, should present a 404 code or be redirected where an equivalent or relevant page is available, as indexed pages that consistently return an error will be dropped from rankings.
5xx status codes – server error
HTTP 5xx status codes indicate that the client’s request was valid but there was an error on the server’s side, preventing it from completing the request.
- 500 Internal Server Error: An unexpected issue has prevented the server from fulfilling the request. This is a catch-all response, which indicates that the server was not able to pinpoint exactly what went wrong.
- 503 Service Unavailable: Indicates that the server is temporarily unavailable and the client should retry later.
Just like with 4xx errors, it is essential to keep tabs on and address 5xx errors as part of your SEO strategy. When a website consistently returns 5xx errors, search crawlers are likely to drop these results from rankings and slow down the crawling of the website if they conclude that the server can’t handle the current rate of requests.
Why are HTTP Status Codes Important for SEO?
Being able to find, identify and analyse successful requests, redirects and errors is essential to managing and optimising the crawling and indexing of your website. Making sure that URLs return the HTTP status codes we expect them to, help us ensure that sites are optimised for both users and search engines. As such, understanding what different status codes mean can help identify which codes will deliver the desired result.
Most Common HTTP Status Codes
There are over 70 HTTP status codes, as listed in the official HTTP Status Code Registry. While these are essential to the workings of the World Wide Web, it is unlikely for most users to come across more than a handful of these.
- 200 – OK
- 301 – Moved Permanently
- 302 – Found
- 404 – Not Found
- 503 – Service Unavailable
How to Monitor HTTP Status Codes
You can monitor Your website’s response stats through a variety of tools.
Google Search Console provides a breakdown of crawled URLs by response. This report allows you to investigate the issues that the Google crawlers encounter on your website and monitor these over time. To understand how Google treats the errors flagged up in the Google Search Console report, see the official documentation.
Website crawling tools like Sitebulb and Screaming Frog also allow you to analyse the response codes for all of the pages and resources on your website, find potential issues, as well as flag up URLs where the crawler was unable to gather a response from the server.
Optimise Your Website’s Performance with Wildcat Digital
Understanding status codes is essential in making sure that your website is crawlable and responding as expected to crawler’s and user’s requests. At Wildcat Digital, making sure that your website can be crawled, indexed and follows best practice is at the core of our SEO strategies.
Rely on the experts to improve your website’s organic performance and punch above the waist online. Get in touch today for a free consultation.
