How To Check If Your URLs Are Indexed In Google

Getting your URLs indexed in Google is a vital step in the process of getting free, organic traffic to your website.

If your URLs aren’t in Google in the first place, then they won’t be found, so ensuring all of the content you want to be visible, is visible, should be one of the first things you consider if you want more organic traffic from your website.

N.B. You most likely won’t want ALL of your web pages indexed. The majority of sites will have some level of irrelevant, duplicated or sensitive content that shouldn’t be indexed – so a URL not being indexed isn’t necessarily a bad thing

 Here are Three Different Ways to Check Which of Your URLs Are Indexed

 Site: Search in Google

There are many different search commands that can be used in Google to ascertain information about yours, and other people’s websites, but one of the most useful and commonly used commands is the ‘site:’ search command.

This will tell you how many pages from a particular domain or subdomain are indexed in Google.

In this instance, we can see that our website has 2,850 pages indexed according to Google.

2. Google Search Console Coverage Report

Of course, just checking the actual live Google index alone doesn’t tell you too much.

You’ll want to know if this is indeed the number of pages you could and should have indexed.

Google Search Console will tell you the amount of valid URLs (i.e. valid for indexing) on your domain

Ideally, this should broadly match the number of URLs returned by a site: command search.

In this instance we can see that there are 3,160 valid pages .

This is MORE than we can see in the live index; and roughly a 10% difference.

There is likely to always be a difference between these two figures and we can’t rely on either as 100% accurate, but when we see a differential like this there may be an opportunity to investigate further.

In this instance we may want to compare the number of indexed pages in the live Google index, to what Google are reporting.

 Google Search Console URL Inspection

Occasionally, we may want to check whether an individual URL is indexed, rather than the index status of every single URL on the domain.

In these situations, we can use the URL inspection tool on Search Console

Most importantly, this will tell us whether the URL is indexed, and if not, why not.

It’ll also give us some other potentially useful information such as;

• Whether the URL was found in the XML Sitemap
• When the URL was last crawled
• The user declared and Google-selected canonicals

4. Site Crawl

Another way to gauge your domains indexability is with a crawl.

Crawling your website with a tool like Screaming Frog can give you several different data points relating to your site; on of the most useful is whether or not a page is indexable.

N.B. Indexable is not the same as indexed – but all indexable pages should be indexed (unless they’re unintentionally indexable).

Pages may not be indexable because of robots.txt rules, status codes (e.g. 404s or 301s), no index tags, or canonical tags. Crawl data can be used to diagnose the reason why a page may not be indexable.

Tips For Getting Your Pages Indexed

Once you’ve resolved any glaring issues surrounding indexability of your domain, you’ll want to ensure that going forwards, your pages get indexed as quickly as possible, and remain indexed for as long as they’re live.

To do this, you want to;

1. Ensure all indexable URLs are Submitted in the XML Sitemap

When/If all else fails – the XML sitemap is a place google will be able to access a list of all of your indexable URLs, so ensure this is maintained properly, and that any new URLs are added.

Dynamic sitemaps can help you with this, but you’ll want to ensure that the settings exclude any canonicalized, noindexed, blocked or non-200 URLs.

2. Submit New URLs via GSC

This may not speed up indexing, but for something that takes a few seconds, it’s worth a try.

When you inspect a URL via Search Console, you can Request Indexing, this will add the URL to Googles crawl queue, quicker than it may have been otherwise

3. Add Internal links

If you add a new page to your that you want to be discovered, make sure you implement internal links pointing this page. These internal links should be both templated (main nav, footer, breadcrumbs) and in-content (e.g. cross-linking from related pages or blog post, and of course, the homepage).

Summary

Ensuring your URLs are indexable is the minimum basic requirement for ensuring they receive traffic, yet it’ something that’s overlooked by many sites.