AI Overviews Gone Wild (+ What It Means for Businesses)

Generative AI, since its public release in the form of OpenAI’s ChatGPT in 2022, has been transforming almost every digital technology. It has decreased the technological barrier further, allowing knowledge workers to access information and complete tasks faster and with more accuracy.

And the search is no exception.

Solutions like Perplexity.ai, Microsoft Copilot, and ChatGPT are allowing internet users to find the exact information they are looking for, instead of returning ten blue links.

To keep up with the Joneses, Google, the go-to search engine for most of us, launched something similar. AI Overviews was made available to the public on May 14, 2024, and announced at Google’s annual I/O conference.

However, to put it simply, it was a disaster. The tool, while innovative, spit out incorrect results to the degree that it became a laughing stock.

Following the backlash, Google dialled it down significantly — only 15% of queries returned AI Overviews — by June 5, 2024, merely a few weeks after it was released.

In this article, let’s understand what AI Overviews are and look at the inaccuracies that led to its restricted implementation.

What are AI Overviews

Usually, when a query on the search engine will return ten blue links (until Google implemented infinite scrolling) related to the topic. Then, the user has to go through the links individually, until they find what they have been looking for.

AI Overviews do it for you. 

It analyses the relevant pages and provides the user with a comprehensive blob of text with the relevant information.

source

This shift has significant implications for both users and businesses, as it changes how information is accessed and utilised online.

While this technology promises efficiency and convenience, it is not without its flaws. Instances, where these AI overviews have provided incorrect or useless answers, can lead to misinformation and frustration for users. 

And such instances were so common and frequent that, as we mentioned, Google limited its usage to only certain queries — within weeks after its launch.

Let’s look at a few such examples that were funny, absurd, and sometimes plain dangerous.

1. Funny mistakes

One of the many things that AI struggles to get right is humour. The answers that were meant to be a joke were displayed by Google in case of serious queries.

source

1. Blinker fluid is real

A long-standing joke in the automotive industry — an electronic device on your car (blinker lights) requires a hypothetical fluid (blinker fluid) — never gets old. 

A joke that, unfortunately, Google’s AI Overviews do not get:

source

2. Dangerous advice

Even the most advanced large-language models (LLMs) struggle to understand the nuanced context of a conversation, leading to useless outputs. In the case of AI Overviews, some of the advice in the results could hurt the user if put into practice.

1. Chlorine gas cleans washing machines

A user looking to clean their washing machine got some advice that could get them fatally hurt by creating chlorine gas:

source

2. Don’t let the dogs out

If you are going somewhere with your dog, you can just leave them in the car. Even if it’s hot and sunny outside. Well, that’s what Google recommends:

source

3. Mental health advice

A user queried that they were feeling depressed and asked for ways to curb it. Suffice to say, the results were disturbing:

source

4. Making spicy spaghetti

We all know that gasoline and food should stay apart. However, AI Overviews confidently explained how you can use the former to make your spaghetti finger-licking good.

Source

5. Smoking during pregnancy

Yes, you read it right. And apparently, doctors recommend it.

source

6. Using glue to stick cheese onto pizza

A user asked what they could do to ensure cheese sticks to the pizza, and AI Overview recommended glue. The answer originated from a satirical answer on Reddit 11 years ago:

Source

7. Eating rocks for nutrition

Google ‘quoted’ UC Berkeley geologists when suggesting that eating at least one rock a day will be beneficial for health:

8. Recipes for chicken

All we can say is that — don’t use AI Overviews if you need advice in the kitchen.

source

3. Plagiarised content

As if the incorrect and absurd answers weren’t enough, AI Overviews went berserk further by pulling content from other sources without sending any visitors their way.

1. No traffic for you

It is quite obvious that AI Overviews pull content from different sources, but it is interesting to note that those sources don’t even get a mention. It can be difficult for content writers to accept that their work, which allows Google to attract users, won’t be rewarded with organic traffic.

Source

2. Stealing content

AI Overviews ‘generates’ answers for the users. The only problem is that the answers are actually content taken (read: stolen) from other blog articles.

Despite copying word for word, there is still no link to the source:

source

3. Other AI-generated content

The whole point of searching on Google is to find what another human who has more knowledge and expertise shares their opinion and advice. AI Overviews, based on what they generate, seem to have missed that point.

On May 30, 2024, Perplexity.ai launched a feature called “Perplexity Pages” that allowed users to find, save, restructure, and share information they find on the internet. Basically, the chat history with a few bells and whistles.

Some results in AI Overviews were content straight from these pages:

source

We could go on.

But as we enjoy these lighthearted moments by looking at the blunders of a new AI tool, we need to think about how it might affect us.

Google’s reply: “Most AI Overviews are useful”

Liz Reid, VP, Head of Google Search addressed all the concerns about the errors and oddities regarding AI Overviews.

source

They maintain their stance that AI Overviews have enhanced the user experience on search and send quality visitors to websites. Liz clarified that it can be backed by feedback data. The accuracy of AI Overviews, as per these numbers (which are private), is as good as featured snippets.

While they acknowledged the odd results, they stated that a lot of the inaccurate results were fake screenshots circulating around the web. For instance, the ones about leaving dogs in cars, smoking while pregnant, and jumping off of the Golden Gate Bridge during depression simply didn’t happen.

She explains, “Some of these faked results have been obvious and silly. Others have implied that we returned dangerous results for topics like leaving dogs in cars, smoking while pregnant, and depression. Those AI Overviews never appeared. So we’d encourage anyone encountering these screenshots to do a search themselves to check.”

However, it will be difficult to prove or refute these claims as AI Overviews have been pulled back massively following the online backlash.

Either way, there were still odd results and Google accepted them. The ones about geologists recommending eating rocks and suggesting users to use glue to stick cheese onto pizza were very real.

Liz did explain why AI Overviews gave those answers — AI can sometimes fail to detect satire in content. Moreover, queries like “how many rocks should I eat in a day” were asked by no one, as you may expect, leading to an “information gap”.

As the satirical piece from the Onion was the only contextually relevant information it could find, it quickly fetched the same for the user.

To mitigate these issues in the future and improve the quality of answers fetched by AI Overviews, Google has already taken a few steps.

As per the statement from Liz Reid that we quoted earlier, these include limiting the use of satire and humour content, restricting user-generated content, and providing AI Overviews for selected queries.

The impact on users

How AI Overviews will affect users, businesses, and content creators is still a complex question. 

It is not just a new technology, its application also has been limited to selected queries. 

From the comments of the VP of search, it seems that some types of content (like user-generated content with humour) will have a harder time making their way into the AI Overviews.

Only time will tell how it will change the search experience for the general audience on the internet. 

Amid all these unanswered questions, a few things are certain. People still value helpful content created by real people over anything else. Google is always enhancing its search algorithms to make that a reality, as reflected by the steps taken by their teams so far after the backlash.

Liz Reid, in the statement linked earlier, concludes, “We’ll keep improving when and how we show AI Overviews and strengthen our protections, including for edge cases, and we’re very grateful for the ongoing feedback.”

The steps they have taken so far — restricting AI Overviews to certain queries, not showing humour or satire in them, etc., — point toward a future where the end users can find the things they are looking for quickly.

If these improvements are successful, AI Overviews could enhance the efficiency of information retrieval for users. People can find contextual answers to their specific questions immediately without going through ten (or more) blue links.

But how does this affect the companies and creators that produce content?

What it means for businesses

The uncertainties around AI-powered search and its broader implications may leave companies and writers with questions corresponding to the nature and type of content they should create.

These questions are more important for small to medium businesses that usually have limited budgets and resources for content production. It is now more important than ever for teams to produce and publish content that can reach their audience with or without AI-powered search technologies.

So, what kind of content should businesses create?

To answer that more comprehensively, let’s summarise the things we have discussed so far:

• AI Overviews gave some odd answers due to the knowledge gap but some of them are likely fake screenshots overshared for their entertainment value
• The general internet users find these errors frustrating as it keeps them from relevant and helpful information
• Google has proactively taken steps to ensure these errors stay minimal and people can get contextually relevant information
• Although its present applications are limited, there is a good chance that AI Overviews could become the future of search if improved

Based on the above points, the most logical thing businesses could do is to create content for their audience. Articles, for instance, that answer questions, provide practical advice, share personal experience, and explain concepts for people will fare better, with or without AI Overviews.

Simply put, following the E-E-A-T framework to produce content centred around your audience’s needs will bring you relevant visitors, which can ultimately drive conversions.

Wrapping up

Google’s AI Overviews go through relevant pages to return a specific answer for a query to the user. While this makes the search experience better for the users, it can also increase the chances of odd responses that are unhelpful.

This has resulted in an online backlash which has prompted Google to address the issues and take steps to mitigate the challenges. The steps taken to minimise these oddities included installing guard rails and limiting AI Overviews to certain use cases.

It is crucial to recognize that the backlash against Google and the search giant’s future action plans about AI Overviews stems from a common concern — keeping the web helpful.

Businesses and creators alike should focus on creating content that their audience finds applicable and relevant for their use cases. These kinds of content have the potential to reach the right audience, even if AI transforms web search entirely.