EDGE of the Web - The Best SEO Podcast for Today's Digital Marketer cover image

524 | The Evolution of Google Updates w/ Dawn Anderson

EDGE of the Web - The Best SEO Podcast for Today's Digital Marketer

00:00

How to Get to the Frontiers of Web Crawlers

Most web crawlers don't crawl the entire publicly available internet and they're not intended to, but they're actually intended to decide which pages to crawl based on the number of other pages that they link to correct. But in actual fact, they do want to crawl everything, but they can't. They have a degree of low priority on certain things like pagination and these kind of comments. The caffeine update in 2009, it was focused on fresher search results. It was about the freshness of the web.

Transcript
Play full episode

The AI-powered Podcast Player

Save insights by tapping your headphones, chat with episodes, discover the best highlights - and more!
App store bannerPlay store banner
Get the app