AI Overview jokes

Language Log 2025-02-26

There's recently been a minor social- and mass-media fad for weird "AI Overview" answers from Google. The results are a moving target, either because of back-end fixes or because of the inherently stochastic nature of LLM results, but some of them are funny while they last. One query that still works this morning is a request for "elements that end in um but not ium", which sometimes answers

and sometimes answers

Of course the answer should be something like "Molybdenum, Tantalum, Platinum, and Lanthanum" (also "Aluminum" in places that spell it that way — see below), and no doubt Future Gemini will figure that out or be instructed to "know" it.

The funniest one that I've seen so far is a response to the query "is tripe kosher", which once answered that it depends on the religion of the cow:

I don’t know if Google can top this, their finest work to date

[image or embed]

— Delbert Grady Seasons (@delbertgseasons.bsky.social) February 25, 2025 at 1:31 AM

…but now gives a correct answer, at least when I ask…

Update: Now when I ask about "elements that end in um but not ium", AI Overview tells me that aluminum is the only one:

So LLMs apparently continue to have difficulties with spelling and counting.

Update #2: Or maybe the count is zero?