5xx Server Errors: A Guide to Finding & Fixing 5xx Issues on Your Website for SEO
Have you ever clicked on a search result only to get a notification stating that “this page isn’t working”? Usually, these errors come in the form of 5xx or 4xx error codes. Having an HTTP availability error or server error on your website can be detrimental for both SEO and conversions. But what are server errors anyway?
There are several types of server errors that can be issued as a response to a client’s request to access your site. If server errors occur quite frequently on your site, it could have a negative impact on your SEO efforts. This is because some HTTP status codes (like 5xx errors) could signal to search engines that they should crawl those pages less frequently, or could lead to a page being dropped from the search engine index. But not all HTTP status codes are created equal, so it’s worth understanding the different types of errors — and how they might affect your search ranking.
In this article, we will dig into the different types of 5xx server errors, what each one means, and how to find and fix them.
What is a 5xx server error?
A 5xx error message occurs when a website’s server can not successfully meet the client’s request to access a specific page.
Often, you will see 5xx status codes appear in your browser like this:
5xx simply refers to any HyperText Transfer Protocol (HTTP) status code in the fifth class of errors. HTTP status codes range from 1xx to 5xx. The first number—which signifies the class—allows us to understand the type of response the website is returning:
- 1xx – Informational
- 2xx – Success
- 3xx – Redirection
- 4xx – Client-side error
- 5xx – Server-side error
As you can see from the list above, only 4xx and 5xx codes are considered errors.
What causes 5xx errors?
A server error can happen at multiple points during the response and can happen across multiple layers of the server environment. The most common causes on websites are:
- Bugs on a web server like Nginx or Apache
- Backend failure of a content management system (CMS)
- Incompatible plugins running on the CMS
- The website development framework (like PHP) not being compatible with the CMS
- A content distribution network (CDN) like Cloudflare misconfiguration
Are 5xx server errors bad for SEO?
There are several reasons why 5xx errors may impact your SEO efforts.
First and foremost, if users are unable to access pages on your site, it results in a bad user experience for your website visitors. For example, if you run a major eCommerce website and your users keep encountering errors when they are looking for different products, they are unlikely to stay on the site. Not only will this impact how users interact with your site (and whether they bother to return), it will impact metrics like bounce rate and, more importantly, conversions. With the launch of Core Web Vitals last year, it’s clear that Google is increasingly looking at page experience in its search algorithms.
Additionally, if a 5xx code is returned when Googlebot attempts to access a page on your website, it could result in Google dropping that page from its index.
Google’s John Mueller has said that Google will initially retry a page that has a 5xx error, but “if we continue to see …the 500 errors then we will …slow down crawling. And if we continue to see that there are 500 errors, then we will drop those URLs from the index.”
Different types of 5xx server errors
As mentioned above, there is a range of 5xx errors that can be returned by the server. Although the 5 indicates it is a server-side issue, the numbers that follow allow us to understand the exact issue in order to rectify it.
500 – Internal server error
A 500 error code is generated when the server encounters an unexpected issue. This is probably the most commonly seen 5xx error and can happen for numerous reasons. Often, this is the server’s way of saying “we are currently experiencing technical difficulties. Please try again later.”
501 – Not implemented
A 501 error happens when the server can not support the functions needed to make the request happen.
502 – Bad gateway
A 502 response occurs when the server is acting as a gateway or proxy. If the upstream server you are expecting a response from does not return a valid response while the server in question is trying to process the request, this results in a 502 error.
503 – Service unavailable
This error is returned when the server is unable to process your request at a particular time. Usually, this is only temporary (often due to server maintenance) and is nothing to worry about.
504 – Gateway timeout
Similar to the 502 response above, this happens when your server is acting as a proxy or gateway. On this occasion, the upstream service did not return a response in a timely manner.
505 – HTTP version not supported
If a web server does not support (or refuses to support) the HTTP protocol used in the request then it will return a 505 response code. When this happens, the response should explain why it is not supported.
506 – Variant also negotiates
Content negotiation is the HTTP mechanism that allows websites to serve content in different formats. For example, if you have an image that can be served in PNG or GIF format (variants), this mechanism allows a browser to show whichever one it can support.
If there is an internal error within the configuration of the chosen variant, that means there is no proper endpoint in the content negotiation process. In which case, you will receive a 506 error response.
507 – Insufficient storage
Similar to a 503 error response, this is often temporary. A 507 response is served when there is not enough storage available to store the representation it needs in order to complete the request.
508 – Loop detected
A 508 code indicates the operation has failed due to the server encountering an infinite loop when attempting to process the request.
509 – Bandwidth limit exceeded
This code appears most often on shared hosting environments and means your website has now exceeded its allotted bandwidth on the server and is no longer accessible to the public.
510 – Not extended
A 510 response indicates the server needs further extensions in order to fulfill the request.
511 – Network authentication required
This happens when the client needs to authenticate themselves before being granted full access to the website. Often, this can happen due to the location you are sending the request from or if you are using a virtual private network (VPN).
When this response is returned it should include an accessible link to allow the user to enter their credentials and gain authorization.
524 – A timeout occurred
This error was created by the content delivery network (CDN) Cloudflare and shows when your website doesn’t respond in time. That means Cloudflare can’t provide a response to visitors or crawlers and results in this familiar screen:
How to find 5XX errors on your site
Often, it can be difficult to find 5xx errors unless a website user points one out or you happen to stumble on one yourself. This is especially true for large websites where an error may only occur on one particular page.
However, permanent 5xx errors are a serious SEO issue. That’s why regularly auditing your website to find these errors (and any other issues affecting your website’s health) is so important.
The best way to do this is to use a technical SEO tool like Deepcrawl’s Analytics Hub to automatically audit your site at regular intervals. Let’s look at how to find 5xx issues with Deepcrawl.
Find 5xx errors with Deepcrawl
First, add your website as a project within the tool and fill in all of the required settings. Then let the crawler do its magic.
Once your crawl is complete you will get a detailed overview of your site and any errors that have been identified as part of the audit.
To find 5xx errors, go to the left-hand menu and select Indexation > Overview:
On the Indexation dashboard, if you scroll down on the right-hand side, you will see the HTTP status breakdown. This will show you all of the pages by their HTTP response code, and you can see if there are any pages showing errors:
As you can see above, you can see how many pages are associated with each HTTP status code type. If you have any 5xx errors showing, select it from the menu and you will see a detailed breakdown of each URL.
Find 5xx errors with Google Search Console
You can also find 5xx errors in the coverage report within Google Search Console. To do this, head over to your Google Search Console account and select the correct property.
Then, in the left-hand menu under the “Index” tab, select “Coverage”.
Scroll down and you will see the 5xx errors listed below:
If you click “Server error 5xx” you will be taken to an overview of the affected pages:
You can also use GSC to find 5xx errors by checking the crawl responses in the Crawl Stats Report. In the left-hand menu, select “Settings”. Once on the settings screen, select “Open report” under “Crawling”
Once the report is open, scroll down to Crawl Requests Breakdown > By Response
If you have any 5xx pages on your site, they will show here and you can select “Server error (5xx)” to view a detailed report.
Find 5xx errors in your log files
Considering Google Search Console will only show you 5xx server errors that Google receives, this may leave you in the dark about certain pages Google hasn’t stumbled upon. That’s why one of the best ways to look for 5xx errors is directly in your log files.
Log files are usually found directly from your web host, although in some cases, these can have strict security measures and be difficult to access.
How you access your log files through your host differs depending on what service you are using. However, as an example, with Siteground, I can access my error logs simply by opening Site tools > Statistics > Error Logs
One great way to analyze your log files is with Deepcrawl’s file integration with Logz.io which allows you to perform log file analysis directly in Deepcrawl’s Analyze Tool.
How to fix 5xx errors
It’s important to fix 5xx errors as soon as you are aware of them. Here are some of the best steps to take in order to fix 5xx issues:
Reload the page: Reloading the page usually works if the server issue is only temporary. You can do this by hitting F5 or Ctrl-R on your keyboard or submitting the URL again from the address bar in your browser.
Clear cookies: Occasionally, 5xx issues are caused by cookies associated with the website; so, clearing these cookies and reloading the browser may fix the issue. Enter the History section of your web browser and choose Delete to do this. You might need to check the box next to Cookies before pressing the delete key on some devices in order to clear the cookies. Check to see whether the error code still appears after refreshing your page.
Rollback your site: If you have just made any updates or changes to the site, roll them back and see if that fixes the problem (this can often be the case if a newly updated plugin is incompatible with your CMS).
Contact your host: If all of the above fails, it may be time to call in the troops. Contact your developer or create a support ticket with your hosting provider so that they can help you resolve the server issue.
Final thoughts on 5xx errors
Every website is bound to run into 5xx errors at some point, whether they are temporary or more serious. For anyone who wants to ensure a good user experience and avoid traffic losses from organic search, it’s worth understanding how to monitor your site for server errors so that you can take the action needed to fix them when they happen — before they have a chance to negatively impact your website’s health and performance.
To learn more about these server errors, check out Deepcrawl’s SEO Office Hours notes on 5xx errors.