8 tips to help index the site in Google

Organic search traffic is critical to the growth of your site. It cannot be obtained if the resource does not appear in the search results at all. SEO and keywords will not help if the site is not in the index. You can just wait for Google to index it, but sometimes it doesn’t.

Before Google starts showing your site in search results, it indexes it. Indexing is the storage of web pages in a global search engine database. When a user enters a search query, Google’s algorithm looks for relevant results in the index.

With a reliable VPS from TutHost you do not have to worry about limited hosting resources, and if attendance continues to grow, we can rent a physical server. You can also choose a reliable SSL certificate from us.

What are crawling and indexing?

When you do a Google search, it displays a list of websites that can answer your question in a split second. But how does Google know which result is best? This is preceded by a complex process of crawling and indexing.

• Crowling. The search engine scans the Internet with the help of search robots.
• Indexing. This is archiving and organizing all found content in a large database called an index. Algorithms then find pages here to show in search results. Google can only rank web pages that are in the index.

For search engine crawlers to crawl the site, it must always be accessible. Order server administration from us and we will see to the stable operation of the site.

How do you know if your site is indexed by Google?

If you are planning to promote in Google, be sure to connect to Google Search Console and Google Analytics. To see which pages of your site are indexed, see Indexing > Coverage. You will see the number of valid pages. You can also enter a specific URL to see if it is in the index.

How to help Google index your site?

The easiest way to help a search engine index pages on your site is to send a direct request to Search Console.

1. Find the “URL Verification” section.
2. In the search box, enter the URL of the page you want to index.
3. Wait for the results of the check, it may take a few minutes.
4. If the URL is not found in the index, click the “Request Indexing” button.

The process can be repeated when new pages are added to the site. However, if you have a large portal or online store, it is not advisable to send requests manually. Perhaps the site settings are preventing indexing.

1. Remove the indexing ban from the robots.txt file

The robots.txt file can be used to disallow indexing of the site. This is done, for example, at the stage of development, when the resource is not yet ready for full operation. Check to see if you have the entire site blocked:

User-agent: Googlebot

Disallow: /

User-agent: *

Disallow: /

This text prohibits the search robot Google to index the site. Remove it to bring the bot back. Also, the ban on indexing may apply only to individual web pages:

User-agent: *

Disallow: /page.html

Remove this text if the page needs to be indexed.

2. Remove irrelevant noindex tags

A ban on indexing can also be set using the robots tag in the section of the web page:

<meta name=”robots” content=”noindex”>

or

<meta name=” googlebot” content=”noindex”>

You can use site audit services to find such tags. Remove tags from all pages that need to be indexed.

3. Set up the sitemap

Site map, or sitemap, tells Google which pages of the site are important and which are not. Note that search engine crawlers crawl the site without this file, but sitemap promotes fast indexing.

You can find the file at:

site.com/sitemap.xml

Data about pages in sitemap is also in Search Console. To check if a web page is listed in the sitemap file, run URL Checker. If you see the error “URL not in Google index” and “Sitemap: n/a”, then it is not in the index and sitemap file.

4. Remove unnecessary canonical attributes

This attribute is written in the head block for all duplicate pages, with a link to the main, canonical version.

<link rel=”canonical” href=”/page.html”/>

So you show Google where the main, canonical page, which should be indexed. This tag helps avoid duplication in the search engine index.

But if a web page has an unnecessary canonical tag, it can direct Google to a canonical version of the page that doesn’t exist. Because of this the existing page will not be indexed.

Run a URL check in Search Console. If the canonical tag points to a different page, you will see a “Alternate canonical page” warning.

5. Check the orphan pages

Google’s robots follow the links on your site, and that’s how they find all the pages. If no link leads to a page, it makes it difficult to scan the site. It will also make it harder for your visitors to find new content.

Various tools are available to find such orphan pages: Netpeak Spider, Site Audit from Ahrefs, Screaming Frog SEO Spider and others. Analytical reports will help you find all orphan pages.

6. Set up internal nofollow links

The nofollow attribute prevents PageRank (link weight) from being passed to the page specified in the tag:

<a href=”http://site.com” rel=”nofollow”>Interesting article</a>

Remove the nofollow attribute from internal links if you want Google to index the pages they lead to. Tip: Close the pages you do not want to show to search engines, it is better to use the noindex tag.

7. Add internal links to new pages

The rate of crawling and indexing directly depends on the link structure of the site. Good linking helps search engine crawlers and is also important for SEO optimization. With proper linking, Googlebot will go around the site faster, looking in even the most inaccessible places. If it is difficult to navigate and there are 404 errors, there may be problems.

Google is more likely to crawl pages with high PR, so it is important to place links to new pages that need to be indexed.

8. Get quality backlinks

Backlinks from quality resources signal Google that the page content is of interest. Such pages are indexed by Google more often and much faster. In Google Analytics you will find a list of sites linking to your pages.

The last but the most important tip – create quality and useful content, use keywords. Google is not indexing your site for two reasons:

• technical problems, such as a ban in robots.txt or the noindex tag;
• low-quality content that Google does not consider useful enough to show to users.

Keep an eye on the quality of your content and do analytical work so that pages get into the index quickly. If you are just working on a website, we can register a domain, order hosting and SSL-certificate. And if you’re thinking about getting your own server, we have a bargain colocation with free connection.