What is Google BERT?

Google BERT is a language model from Google that works out the meaning of a word from the context of the words before and after it. The name stands for Bidirectional Encoder Representations from Transformers. Google has been using BERT in Search since October 2019 to better understand search queries and text passages, especially long queries in natural language.

Term profile

How does Google BERT work?

Google BERT reads every word together with its left and its right context at the same time and derives what the word means in exactly this sentence. Earlier language models mostly processed text in one direction, from left to right. The “B” in BERT stands for this bidirectional reading, which builds on the transformer architecture from 2017.

BERT learned this in two phases:

1. Pre-training: The model read the English Wikipedia with around 2.5 billion words and a collection of books with around 800 million words. In the process, 15 percent of the words were hidden, and BERT had to guess them from the surrounding text. A second exercise was to recognize whether two sentences follow each other in the original.
2. Fine-tuning: The pre-trained model is then tailored to a specific task with a small amount of additional training, such as answering questions or classifying texts.

The result is a model that does not know words as fixed entries but represents them differently depending on the sentence. Technically, BERT creates an embedding for every word, a vector of numbers that captures its meaning in context. The word “bank” gets a different vector next to “withdraw money” than next to “by the river”.

What did BERT change in Google Search?

With BERT, Google understands small words such as “for”, “to”, or “without” that change the meaning of a search query. Google announced the rollout on October 25, 2019 and called it the biggest leap forward of the past five years. At launch, BERT affected about one in ten search queries in English in the US. A year later, Google said it was using BERT for almost every English-language query.

Google explained the effect with its own examples:

• “2019 brazil traveler to usa need a visa”: Before BERT, Search overlooked the word “to” and showed results for US citizens traveling to Brazil. With BERT, it recognized that a traveler from Brazil is looking for a visa for the US.
• “can you get medicine for someone pharmacy”: With BERT, Search grasped that the query is about picking up medicine for another person and not about prescriptions in general.

In addition, Google started using BERT for featured snippets in 2019, in all of the roughly two dozen countries where they existed at the time. Long queries that sound spoken benefited most, precisely the longtail keywords in which every single word counts. The search intent of such queries often only becomes clear from their prepositions.

Does Google still use BERT today?

Yes, Google still lists BERT as an active system in its official guide to Google Search ranking systems. There, Google describes BERT as an AI system that understands how combinations of words express different meanings and intents. During the US government’s antitrust case against Google, it became known in 2023 that Google internally calls the use of BERT in ranking DeepRank.

BERT does not work alone. RankBrain was added back in 2015, the Multitask Unified Model (MUM) followed in 2021, and today models from the Gemini family write the answers in AI Overviews. Together, these systems form the foundation of semantic search.

Outside Google, BERT is one of the most influential language models of all. Numerous offshoots are based on its architecture, such as RoBERTa from Meta, DistilBERT from Hugging Face, and Sentence-BERT, which encodes sentences so they can be compared by meaning. For the German language, the Berlin company deepset released its own BERT model in 2019.

What is the difference between BERT and GPT?

BERT is designed to understand language, GPT to generate it. Both model families are based on the transformer architecture but use different halves of it.

Modern large language models such as GPT or Gemini are many times larger than BERT. For quickly understanding and matching millions of search queries, compact encoder models like BERT nevertheless remain in use.

Can you optimize for Google BERT?

No, there is no separate optimization for BERT. Danny Sullivan, Search Liaison at Google at the time, made it clear in October 2019 that there is nothing to optimize for with BERT and that the fundamentals remain unchanged: Google wants to reward helpful content. BERT does shift which texts benefit, however. This leads to 4 consequences:

1. Write naturally: A text that answers a question in normal language fits a model that understands normal language. Strings of search terms such as “dentist munich cheap fast” do not help and are closer to a case of keyword stuffing.
2. Answer the question precisely: When small words determine the meaning, the page that answers exactly the question asked wins, not a related one.
3. Take long queries seriously: Since BERT, Google can match specific questions with several conditions much better. A section that fully resolves such a question has a real chance there.
4. Use terms unambiguously: If you name a topic consistently and provide the context, you make it easy for a language model to choose the right meaning.

The same rules apply to AI systems that put answers together from web pages. Our SEO services describe how we prepare content for both sides, for classic search and for AI answers. The page on answer engine optimization (AEO, also known as GEO) and AI visibility goes deeper into the part aimed specifically at AI systems.

Frequently asked questions about Google BERT

Is Google BERT a Google update?
Google announced BERT in October 2019 as a change to Search, which is why people often speak of the BERT update. However, BERT is not a core update but a permanently running system for understanding language.

Does BERT also apply to German search queries?
Yes. In December 2019, Google expanded BERT to more than 70 languages, including German.

What is the difference between BERT and RankBrain?
RankBrain relates the words in a search query to concepts and, from 2015 on, helped above all with unfamiliar queries. Since 2019, BERT has understood how the words within a sentence relate to each other. Both systems run in parallel.