How SEO Changed in 2024: The web almanac with Jamie Indigo
Jan 6, 2025
auto_awesome
Jamie Indigo, a Technical SEO expert and author of the Rich Snippets newsletter, shares insights from the 2024 Web Almanac. The conversation dives into the significant changes in SEO, including the rising impact of AI and the pivotal role of robots.txt. Jamie discusses the evolution of Core Web Vitals and how structured data is becoming essential for user experience. He highlights the need for clearer communication around SEO directives and reflects on the future of SEO in an ever-changing digital landscape.
The Web Almanac highlights a paradigm shift in technical SEO methodology by examining both home pages and inner pages distinctly for better data insights.
Increased awareness of robots.txt among developers indicates a move towards more responsible content management and control over search engine interactions.
Core Web Vitals have evolved, with new metrics improving user experience, as evidenced by 58% of desktop sites now meeting performance standards.
Deep dives
Introduction to the Web Almanac
The Web Almanac serves as a significant resource for understanding the state of the web, offering insights derived from the analysis of millions of web pages. This initiative, led by the HTTP Archive, involves crawling approximately 17 million pages and allows researchers to examine vast amounts of data to gauge changes and trends in web practices. The approach of studying both home pages and inner pages this year marks an evolution in methodology, revealing different behaviors and patterns between these page types. By closely analyzing this data, stakeholders can better comprehend technical SEO dynamics and their implications for website performance.
The Importance of Robots.txt and Directives
An examination of robots.txt and its associated directives has unveiled an increased awareness among web developers about controlling how search engines interact with their sites. This year marked the first time the authoring team explored robots directives in depth, revealing that a significant percentage of sites utilize rules that are technically misleading, such as 'no follow' and 'no index'. By studying how major publishers are blocking AI crawlers, the data highlights a shift towards more responsible content management practices. Overall, this understanding allows SEOs to better strategize the accessibility and visibility of their content online.
Changes in Core Web Vitals Measurement
Core Web Vitals continue to evolve, with notable improvements in how interactivity is assessed through the introduction of the Interaction to Next Paint (INP) metric. This shift reflects the growing need for websites to accommodate Single Page Applications (SPAs), leading to enhancements in user experience across various platforms. Statistics revealing that 58% of desktop sites now pass the Core Web Vitals standards indicate a positive shift towards better web performance. As a result, webmasters are encouraged to prioritize optimizing their sites to enhance user satisfaction and engagement.
The Role of Structured Data
Structured data plays a crucial role in helping search engines understand the context of web content, allowing for improved indexing and rich results representation. This year's insights showcase that while JSON-LD is increasingly adopted as the preferred format for structured data, many websites still resort to less effective methods, such as RDFa. By delivering clearly defined structured data on inner pages—especially for product listings or article author attributes—webmasters can provide search engines with all necessary information to enhance search visibility. Moreover, using structured data effectively can lead to richer user experiences and a more direct path to conversions.
The Continued Impact of Generative AI on Content
As generative AI continues to proliferate, questions around the authenticity and ownership of content are becoming paramount. The conversation highlights that many AI-generated outputs lack grounding in verified data, leading to inaccuracies and misinformation. Discussions surrounding AI's impact on original content draw attention to the growing need for clarity on content ownership and the monetization of creative works. Moving forward, maintaining the integrity and trustworthiness of web content in an AI-dominated environment will be a significant challenge for the community.
In the first episode of 2025, host Jack Chambers-Ward is joined by Jamie Indigo, an expert in technical SEO and an overall delightful human being.
Jamie guides Jack through the findings from the 2024 Web Almanac's SEO chapter including technical data changes over recent years, offering insights and predictions for the future.
Jamie also highlights AI's rising influence, the importance of robots.txt, Core Web Vitals updates, and structured data advancements.
Maggie Appleton Dark Forest: https://maggieappleton.com/ai-dark-forest
https://youtu.be/1OvasjokuaA?si=UBNiK07uyWHjd2HX by Jess Sankhavaram-Peck
Chapters
00:00 Welcome to 2025: Kicking Off the New Year
01:11 Sponsored by SE Ranking https://seranking.com/?sou1=influencers&gr1=podcast&tg1=search_with_candour&kw1=com_first_ad&utm_source=influencers&utm_medium=referral&utm_campaign=com_first_ad_search_with_candour
02:45 Introducing Jamie Indigo: Technical SEO Expert
03:56 What is the Web Almanac?
04:43 Exploring the Methodology: Data and Analysis
13:47 Dynamic Rendering and Robots.txt: Trends and Changes
20:44 Meta Robots Directives: New Data Insights
29:03 Canonical Tags Explained
29:37 Consistency in SEO
33:53 JavaScript and SEO Misconceptions
34:51 Structured Data Insights
39:34 The Role of Knowledge Graphs
46:10 Core Web Vitals Update
50:05 Wrapping Up and Future Plans
Remember Everything You Learn from Podcasts
Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
