Technical SEO Library

How do Search Engines Work?

In this guide we will cover the processes of crawling and indexing as well as concepts such as crawl budget & PageRank.

Search Engine Crawling

Let's delve deeper into the processes that search engines use to understand the web, otherwise known as the crawling.

Search Engine Indexing

Let’s take a look at the indexing process that search engines use to store information about web pages.

Search Engine Differences

Here we break down some of the key differences between some of the world's major search engines.

Crawl Budget

Learn about the main rules and conditions around crawling that search engines incorporate as part of their algorithms.

Metadata

Discover how metadata provides information about a page’s structure & content to search engines and social networks to better understand webpages.

Page Titles

Learn about the most important type of metadata, the page title, can be used to influence search engines and rankings.

Meta Descriptions

This guide covers best practices, details as well as do's and don't's around providing effective meta descriptions.

Schema.org Microdata

Discover how you can add metadata that provides machine-readable context into webpage content with microdata.

Website-level Schema.org Microdata

This guide looks at some specific examples of markup you might want to apply at a site level.

Product Microdata

Learn about the types of markup you can implement on product pages to give details of items that are sold on a site and obtain rich snippets.

Review Microdata

Find out how review markup can be applied to a variety of entities such as local businesses, products, films and books to enhance webpages.

Other Microdata

The possibilities to markup webpages with schema are vast. Here we cover some of the other types that are available.

International Site Architecture

Learn how to optimize multilingual & multiregional websites, to ensure they are indexed and served to the correct audiences.

Hreflang

Find out how hreflang tells search engines about the relationship between international versions of a page.

Robots.txt

Read about the robots.txt text file and how it can be used to instruct the search engine web crawlers.

URL-level Robots Directives

Let's take a look at ways that you can apply robots directives at a page level to instruct search engine bots.