Introducing DeepCrawl’s Advanced Google Search Console Integration
Google Search Console (GSC) is a powerful, yet underutilised tool, in every SEO’s toolkit. At DeepCrawl, we identified an opportunity to harness the power of GSC’s Search Analytics data to provide you with a more complete picture of the URLs on the websites you manage.
After much discussion and deliberation, we are excited to announce the launch of our Advanced Google Search Console Integration.
Our latest update takes advantage of the insights from GSC’s Search Analytics’ SERP information and combines it with the wealth of DeepCrawl’s existing crawl data.
Using DeepCrawl’s GSC integration, I found a set of orphaned pages on a client site that were receiving search traffic but were not linked to from anywhere on the site. That discovery led me down a very deep rabbit hole of SEO issues. DeepCrawl’s ability to compare site crawl data to Search Console data is immensely powerful.
How Can You Add GSC Data to Your Crawls?
Adding Search Console as a Crawl Source
Taking advantage of our new integration with Search Console is a simple process requiring minimal effort to add it as a crawl source.
When setting up a new crawl you will need to select Google Search Console as a source in the second stage of the crawl set up. When you first add Search Console as a crawl source you will need to enable permissions to connect with your Google account, and select the verified domain(s) relevant to your crawl.
Filter Down Search Console Data to Only Include What’s Relevant
We’ve built a flexible integration with GSC, so that you can decide what SERP data you want to include in your crawl.
After the connecting your Search Console account, you’ll have a range of filters at your disposal, so that you can drill the data down as you would in Search Console.
Specifically, you can filter Search Console data to:
- Include and/or exclude search queries – This could be used to focus on non-branded queries by filtering out brand terms
- Look at a specific date period – Look at search performance data from the last 7 days to the last 90 days
- Only include queries with a minimum number of clicks – Weed out low performing queries by only showing queries with a minimum number of clicks
- Specify a location – Select specific countries to be added or excluded from your crawl
- Limit to a search type – Restrict data to only show information for web, image or video queries
What Does the GSC Integration Add to Your Crawl
17 New SERP Metrics to Take Your Crawls to the Next Level
Unlike competitor GSC integrations, DeepCrawl provides device specific metrics for all search metrics. That means you can see the number of impressions, clicks, CTR and average position separately for mobile, desktop and tablet for every single URL on your site!
We have also pooled together these device specific metrics to provide you with a total. The final 17th addition is the “Found In” metric that will tell you whether or not a URL was found in the Search Console source.
Together our new metrics provide the most comprehensive Search Console integration of any website crawler and have formed the basis of a whole host of new reports.
10 New Reports to Evaluate the Effectiveness of Your Indexable Pages
The additional information within DeepCrawl from the GSC integration has allowed us to develop the following reports to help you uncover insights and issues on the websites you manage.
Traffic Reports
In our Traffic category you’ll find 7 new reports:
- Indexable Pages Without Search Impressions – Identifies pages that should be indexable but aren’t showing up in search. Use this report as a starting point to highlight valuable pages on your site that are in need of optimisation.
- Indexable Pages With Search Impressions – Finds pages on your site that are being indexed and appearing in search. This report can be used to identify your best performing pages in search and to track the number of indexable pages in search from crawl to crawl to identify and understand fluctuations.
- Non-Indexable Pages With Search Impressions – Flags any pages that shouldn’t be indexed but are receiving impressions in search. This can happen when Google is ranking pages that are duplicates, canonicalised or noindexed.
- Mobile/AMP Pages With Search Impressions – Reports all of your AMP pages and/or separate mobile pages with impressions in search. This report is useful in confirming that mobile and AMP pages are being indexed and receiving search impressions.
- Mobile/AMP Pages With Desktop Search Clicks – Find AMP and Mobile pages that are getting clicks from desktop users. If a crawl returns any pages in this category you will need to check how this page has been (mis)configured for mobile.
- Desktop Pages With Mobile Search Clicks – Conversely, any desktop pages receiving clicks from mobile users will be shown in this report and again some investigatory work will need to be carried to identify the issue.
- Broken Pages With Traffic – Any URLs returning 4xx or 5xx errors but receiving Search Console clicks or Analytics traffic will be displayed in this report. Pages flagged here should be dealt with as a matter of priority because search engines will remove these pages from search and it results in poor user experience if error pages are landed on.
Gap Analysis Reports
The Search Console integration has also resulted in 3 new reports in the Source Gap category allowing you to find all URLs in search and compare them against other sources.
Two of the reports (Pages in & not in Search Console) show pages included in the crawl that were and weren’t found in Search Console. These can be used in gap analysis to gain an overview of pages in and not in Search Console as a proportion of the total pages found in the crawl.
The final Source Gap report is the Orphaned Search Console Pages, which highlights pages receiving clicks which aren’t linked internally on the site. You can link to these pages internally to make these pages easier to find which may result in an increase in rankings.
Two New Visualisations
The icing on the cake of our GSC integration update is the introduction of two new graphs to the DeepCrawl Dashboard.
The first new visualisation is the Search Clicks by Device Chart. This chart breaks down clicks to your site for indexable pages, non-indexable and mobile alternate pages for mobile, desktop and tablet.
The Indexable Pages in Search Graph provides you with a quick overview of the proportion of your indexable pages that are and aren’t receiving search impressions.
This second graph is a convenient starting point in identifying pages which should be showing in search but aren’t, before drilling down into the Indexable Pages Without Search Impressions Report and deciding which of these pages to focus your optimisation efforts on.
The data insights now available with the GSC integration is potentially game changing for technical SEO.
What Are You Waiting For?
Now you’ve got the skinny on our latest update, you can go forth and crawl with the combined power of GSC and DeepCrawl.
If you’re new to DeepCrawl then take us for a spin, by signing up for an account. If you’re a DC veteran then simply login to the app, add your GSC property as a crawl source and get crawling!
And finally, if you fancy some extra reading then check out our guides on the advanced GSC integration and how to optimise for mobile using GSC.
Find out how to get the most out of our integration by watching our webinar
Sam Marsden
Sam Marsden is Deepcrawl's Former SEO & Content Manager. Sam speaks regularly at marketing conferences, like SMX and BrightonSEO, and is a contributor to industry publications such as Search Engine Journal and State of Digital.
