Summary

Artificial Intelligence (AI) has significantly transformed search engine algorithms, making them more efficient, accurate, and user-centric. This article delves into how AI has changed search engine algorithms over the years, and shares insights on how it will continue to change in the future. We will explore the evolution of Google's search algorithm, its current state, and future predictions. We will also touch upon other platforms like Bing, and emerging search engines like ChatGPT, Perplexity, You.com, and Claude, tying these changes back to Search Engine Optimization (SEO).

Evolution of Google's Search Algorithm

Past : Pre AI Era

A few years ago, Google's search algorithm relied heavily on keyword matching and basic ranking factors such as backlinks and on-page optimization. The algorithm was less sophisticated in understanding user intent and context, often leading to irrelevant search results. Techniques like keyword stuffing and link farms were commonly used to manipulate rankings, resulting in a poor user experience.

PageRank Algorithm (2000): Google introduced the PageRank algorithm in 2000, which revolutionized how search results were ranked by considering the number and quality of links to a page. This was a significant departure from previous methods that primarily focused on keyword density.

The Google Dance (2003): In 2003, Google introduced several updates, including the "Cassandra," "Dominic," and "Esmeralda" updates, which refined how backlinks were counted and eliminated hidden links and texts. This period, known as the "Google Dance", marked the beginning of more sophisticated algorithmic changes.

Panda and Penguin Updates (2011-2012): The Panda update in 2011 targeted low-quality sites with thin content, while the Penguin update in 2012 focused on penalizing sites using manipulative link-building tactics. These updates aimed to improve the quality of search results by prioritizing high-quality, relevant content.

Present : Leveraging Artificial Intelligence

Today, Google's search algorithm is powered by advanced AI technologies such as RankBrain, BERT (Bidirectional Encoder Representations from Transformers), and MUM (Multitask Unified Model). These AI systems enable the algorithm to understand natural language, context, and user intent more accurately.

RankBrain (2015): RankBrain was one of the first AI systems to help Google process search queries by understanding the context and semantics behind them. It uses machine learning to improve search results based on user interactions.

BERT (2019): BERT further improved the algorithm's ability to understand the nuances of language, making search results more relevant and accurate. It uses deep learning to process the context of words in search queries, allowing for better interpretation of user intent.

MUM (2021): MUM is 1,000 times more powerful than BERT and capable of multitasking to understand complex queries and provide comprehensive answers. It can analyze text, images, and videos in multiple languages, making it a significant advancement in search technology.

Recent Updates (2023-2024): Google's recent updates, such as the March 2024 Core Update, have focused on reducing low-quality content and promoting high-quality, helpful content. These updates have led to a 45% reduction in low-quality content in search results, emphasizing Google's commitment to improving search quality.

Future: AI-Driven Search

The future of search algorithms will likely see even deeper integration of AI, with generative AI playing a crucial role. AI will enable search engines to provide more detailed and accurate answers, reducing the need for users to browse through multiple search result listings. The algorithms will become more conversational, understanding and responding to natural language queries more effectively.

AI Overviews: Google's introduction of AI-generated answers, known as "AI Overviews", represents a significant shift in how search results are presented. These AI-generated summaries provide users with concise, accurate information directly in the search results, potentially reducing the need for users to click through to individual websites.

Personalization and Contextual Understanding: Future algorithms will likely focus on even greater personalization, using AI to tailor search results based on individual user preferences, search history, and contextual factors. This will enhance the relevance and accuracy of search results, providing a more personalized search experience.

Bing's Search Algorithm

Bing, Microsoft's search engine, also employs advanced AI and machine learning technologies to deliver relevant search results. While Bing's market share is smaller than Google's, it remains a significant player in the search engine landscape.

AI Integration: Bing's algorithm relies heavily on machine learning, with majority search results influenced by AI. This allows Bing to rank documents in the same order as humans would, focusing on understanding query intent and delivering relevant results. With Microsoft's key partnership with OpenAI, we expect Bing's search engine to become more AI-native over time, and working similar to "sister" products like Microsoft's CoPilot, or OpenAI's ChatGPT. It would not be a surprise if Microsoft affiliated browsers and search platforms ended up using the same search engine to find relevant documents and respond to search queries. This would allow them to create network effects in their products, and build a better search experience long term.

Ranking Factors: Bing's ranking factors include content relevance, page authority, user engagement metrics, and social signals. Unlike Google, Bing places more emphasis on exact keyword matches and social media signals, making it essential for SEO professionals to tailor their strategies accordingly.

User Experience: Previously, Bing valued user experience metrics such as page load speed, mobile responsiveness, and intuitive navigation. Websites that offer a seamless user experience are more likely to rank higher in Bing's search results. Google's recent updates on prioritizing mobile-first websites, and user experience may bridge this in the future.

Emerging Search Engines: ChatGPT, Perplexity, You.com, Claude

In addition to Google and Bing, several emerging search engines are leveraging AI to provide innovative search experiences. These platforms offer unique features and capabilities that differentiate them from traditional search engines.

ChatGPT : We were all blown away when OpenAI released ChatGPT in November 2022. Since then, it has acquired 180M+ users, and become a part of people's workflow. OpenAI has been rumored to venture into the search engine market with its ChatGPT Search product, rivaling the likes of Google and other search engines. It's also not a stretch from what ChatGPT offers today : the key difference would be to plug into real time data as opposed to a 3-4 month lag. With OpenAI's partnership with Microsoft, and access to browsers and search engines like Bing and CoPilot, ChatGPT is in a powerful position to take over a good chunk of Google's market share.

Perplexity : Perplexity.ai is an AI-driven search engine that focuses on providing concise, accurate answers to user queries. It uses advanced natural language processing and machine learning algorithms to understand and respond to user questions, offering a more conversational search experience. Perplexity aims to reduce the need for users to sift through multiple search results by delivering direct, relevant answers.

You.com: You.com is a personalization-focused search engine that integrates a consumer-facing Large Language Model (LLM) with real-time internet access. You.com offers AI-powered chat capabilities, providing users with personalized search results and various response formats, including text, images, and videos. The platform's AI Modes allow users to tailor their search experience based on their needs, from quick information retrieval to complex problem-solving.

Claude : Anthropic's Claude is another AI-driven platform that has made significant strides in the search engine landscape. Claude's models, including Claude 3 Sonnet, Haiku, and Opus, offer advanced capabilities for search and information retrieval. As of today, the platform seems to be more developer focused (with the integration of developer tools, function calling like features, and a focus on APIs, etc). However, Claude's output quality has improved exponentially, and it would not be a surprise to see Anthropic launch its own natural language search interface built on top of its existing technology soon.

The Implications of AI on Search Engine Optimization

Google was the gold standard for search and established itself as an action verb over the last decade (Just "Google" it). Yes, Google still has the lion's share of the market, but it is naive to assume that it will continue to be like this over the next 5 years, or at the very least that Google will not evolve its own search engine to keep up with the new kids on the block.

These changes : (1) Search Engines changing how they "understand" the internet and interact with end users, and (2) the emergence of other search engines challenging Google's monopoly, makes this one of the most transformational times for the Search Engine Optimization.

If Search changes, so does Search Engine Optimization.

The recent changes (Featured Snippets, AI Overviews, etc.) has made it very clear that the importance of SEO will only increase over time. I'll show you why. The two most importance steps for any search engine (may that be Google, ChatGPT, Perplexity, or anyone out there) is to find relevant websites for a user's search query, and understand the content on that website. It will then use this understanding to answer the user question - in the form of a simple response using an LLM, or helpful links. Guess what? SEO is what helps these search engines find your website, and helps LLMs understand your website to answer user questions (even if it's in the form of Featured Snippets or AI Overviews).

Don't get me wrong : SEO is changing, and I do believe traditional, generic SEO checklists will no longer be effective. What I'm saying is that SEO will look and feel VERY different than what has been the industry standard over the last decade.

Think of Search as Finding Resources in a Forest

One of our mentors drew a great analogy that illustrates the rapidly evolving search landscape - I'll do my best to replicate that.

Think of every search engine out there as a "Forest". People navigate the forest to find resources - fruits, nuts, leaves, timber, or whatever "product" they're looking for. The goal of marketing is to "scatter breadcrumbs" around people wandering about these forests to bring them onto a particular trail at the end of which they can find the "product" that they're looking for. Think of SEO as these "breadcrumbs" that are being scattered around - the goal being increasing discoverability, and more importantly, revenue.

So, what has changed here?

First, the number of forests. Google was the only forest we needed to think about : that's no longer true. Your users don't just search for products in one forest : in fact, you don't even know which forest their search journey started on. Perhaps they asked ChatGPT a question like "How to do SEO", ChatGPT answered with some general guidelines and links to some SEO agencies and products. Good start, but not actionable. The user then "transported" to Google, to learn more about the solutions referred by ChatGPT to make a purchase decision.

Guess what? If you do not show up where the user's search journey starts, you will likely not be a part of their purchase decision.

Second, the "breadcrumbs". The analogy is pushed to the limits of our creativity here, but this is the gist : different search engines are leveraging AI to "understand" the internet differently. The "breadcrumbs" in the "forests" are the various aspects of SEO (Technical, On Page, Off-Page, Editorial, etc.) that allow these search engines and users to "understand" your offering (website) better. Now, we can't just scatter one-size-fits-all, generic "breadcrumbs" out there hoping that they will attract the people wandering the forests. The SEO strategy needs to be data-driven, complex, and thought of as running marketing experiments : we need to know which type of bread the user and search engine prefers, are there certain breads that are more attractive than others, where in the forest to scatter the crumbs, which type of people prefer which type of crumbs, when in their hiking journey to people look for these breadcrumbs, etc. One size fits all, generic, traditional SEO playbooks are no longer effective.

How You Should Elevate Your SEO Strategy

If all of this sounds complicated and overwhelming, don't worry : it is. The SEO industry is going through a paradigm shift, and I expect a lot of folks out there (agencies, freelancers, etc.) to not be able to keep up with the evolving landscape.

The only way to keep up with the changes is to be tech-first, data-driven SEO mavericks : let go of the traditional SEO checklists, and create hyper personalized strategies, coupled with fast execution, and continuous strategy iteration accounting for not just Google, but all the "forests" that the end user is searching on.

The good news is that you don't have to do this by yourself. Passionfruit can help out : we're a VC-backed team of AI experts, SEO experts, and Content Writers. We've built partnerships with key search platforms and SEO analytic platforms, and figured out how different search engines "understand" websites differently. Most importantly, we've created human-in-the-loop workflows that help our clients execute and iterate SEO strategies 20x faster than anyone out there