Jul 27, 2023

15 Steps to Fix Website Indexing Issues

1. URL blocked by robots.txt

Edit your robots.txt file and remove the line that blocks the URL.

2. Not found (404)

Make sure that the page exists and is accessible. If the page no longer exists, you need to remove it from your site.

3. Page with redirect

Firstly, determine whether the redirect is temporary or permanent then;

For permanent redirect: update your robots.txt file to allow the redirect.

For temporary redirect: remove the redirect from your site.

4. Soft 404

To fix this, either remove the page from your site or add new content to the page and resubmit it to Google.

5. URL marked 'noindex'

Remove the ‘noindex’ tag from the important pages you want to index.

6. Duplicate without user-selected canonical

Select a canonical URL for the pages. You can do this by adding a rel="canonical" tag to the page that you want to be the canonical URL.

7. Crawled currently not indexed

Firstly, identify the reason why the page is not being indexed. Take steps to fix the issue such as providing high-quality content, limiting duplicate content, and correcting technical errors.

8. Server error (5xx)

There are many different types of server errors, so the specific steps to fix this issue will vary depending on the error. However, here are some general steps that you can take:

  1. Check the server logs to see more information about the error.
  2. Contact your hosting provider for help in resolving the error.
  3. Update your server software to the latest version.

9. Blocked due to unauthorized request (401)

Make sure that the user has the correct permissions to access the page, optimize the structured data, and optimize your crawl budgets.

10. Blocked due to access forbidden (403)

Ensure user permissions are correct.

Check the robots.txt file for Googlebot access.

Check the htaccess file for access rules.

Contact a web hosting provider if none of the above steps work.

11. URL blocked due to another 4xx issue

Identify the reason why the server is returning a 4xx error and then take steps to fix the issue.

12. Blocked by page removal tool

To fix this issue, you need to remove the page or cancel the request from the Page Removal Tool.

13. Discovered - currently not indexed

Identify the reason why the page is not being indexed and then take the following steps to fix the issue;

  1. Request indexing via GSC
  2. Improve your crawl budget
  3. Ensure high-quality and unique content
  4. Do internal linking
  5. Add backlinks

15. Redirect error

Check the redirect and make sure that it is working properly.

Digital Services
About
Contact
THE SQUID WAY

We do things differently at Squid, creating brands and experiences which evolve the way businesses are seen. We are based in Auckland, offering digital marketing, web design, web development, branding and content marketing across NZ and Australia

Terms of Engagement

Copyright 2022 Squid Group. All rights reserved.

Group