The AI Overview feature, designed to improve Google’s search engine, continues to be the subject of criticism. AI Overview’s recommendations, such as adding glue to pizza to keep the cheese from slipping off or eating one small stone a day, have raised doubts about AI’s ability to separate jokes and implausible stories from the truth. Google has stated that it has taken the necessary steps to prevent meaningless AI advice, but errors continue to plague generative search.
AI Overview’s Controversial Recommendations
Instead of quoting humorous comments from Reddit users or satirical articles from The Onion, Google’s AI-generated search summaries began citing their own errors. Game developer and former Google employee Colin McMillen highlighted this issue on the social network Bluesky, sharing a screenshot of a response to the search query “How much glue to put on pizza?”
According to Business Insider’s Katie Notopoulos, Google’s AI-generated summary suggested adding ⅛ cup or 2 tablespoons of white non-toxic glue to pizza sauce to prevent the cheese from sliding off. The summary claimed that Notopoulos said adding glue did not significantly change the sauce’s consistency and that the resulting pizza sauce was orange, similar to vodka sauce. However, this was a misrepresentation. Notopoulos had written an article last month highlighting the absurdity of such advice, repeatedly warning readers not to add glue to pizza sauce.
Questioning AI’s Effectiveness
One could argue that McMillen asked Google a leading question that elicited an absurd response. Nonetheless, this example calls into question the effectiveness of Google’s measures to eliminate the risk of ridiculous AI Overview responses.
Last month, Liz Reid, Google’s vice president of search, stated that the company had created “better mechanisms for detecting meaningless queries related to satirical and humorous content.” Reid explained that Google has limited the use of user data in responses that lead to misleading advice, especially regarding health and trending news. However, it appears that AI Overview has not yet learned to filter out the meaningless information received by the AI model during the training process, NIX Solutions.
In February, Google agreed to pay Reddit $60 million a year to provide access to its database of user posts and comments for use in training its AI models. It looks like Google might be starting to regret this deal. We’ll keep you updated on any further developments regarding AI Overview and its implications for Google’s search engine.