A curious question from my kids sent Gemini into a hallucination.
Google's AI Overview can be 100% wrong, even when SERP is right.
Google’s AI Overview can be 100% wrong, even when SERP is right.
It could be a trap when you least expect it.
Today, my kids asked me, “What is Buah Long Long?” I quickly Googled it to show them images. The initial search was good: both the AI Overview and SERP correctly identified “Buah Long Long” as Ambarella Fruit.
However, when I tapped on “Benefits of Buah Long Long,” the AI Overview hallucinated, confidently stating that Buah Long Long is also known as longan. This led to a cascade of completely wrong information.
Ironically, the SERP results for “Benefits of Buah Long Long” were accurate. AI Overview is supposed to be grounded in Google Search, so why was it so far off?
This isn’t just about fruit; it’s a critical reminder about the reliability of information, especially from AI summaries.
I tested the same prompt, “Benefits of Buah Long Long,” on Perplexity, and it had no issues. This incident, combined with my past experiences, indicates that Gemini still exhibits the worst hallucinations among frontier LLMs.
My takeaway: Always, always, always double-check facts presented by AI Overview.
What are the worst hallucinations you’ve encountered recently?
Technical observation:
It hallucinates when I least expect it. But here’s an interesting observation: Buah Long Long and Longan both contain the word “Long”. Could this be related to tokenization?
#gemini #googleai #ai #google #artificialintelligence #chatgpt #perplexity #openai #genai #llm #hallucinations
Tap to expand
Enjoyed this? Subscribe for more.
Practical insights on AI, growth, and independent learning. No spam.
More in AI Agents
From ChatGPT to Claude Code: A Non-Techie’s Introduction to the Raw Power of AI by a Techie
Not because I don't like sharing. But because the only tool I use for 99% of my AI needs is Claude Code. And while I think it is not hard to learn, I'm never...
💡 Little-known hack to get the most out of Cursor for FREE
If you're using Cursor on the free plan, you will eventually hit the dreaded "servers overload" screen.
The Worst Job Displacement of Software Engineers Is Yet to Come.
This is not another fear mongering post.
How I Collaborate with AI to Refine What I Know
In this article, I share examples of how I practice it in actual work.
Most OpenClaw workshops skip the fundamentals and teach you 1-click setup on AI or cloud providers...
I decided to go for a different path. In my workshop, I guide learners to set up with the original experience: install it on your laptop, but with security i...
Yesterday, we were discussing when we could actually let AI agents auto-merge and publish code.
My take - probably not so soon.
From ChatGPT to Claude Code: A Non-Techie’s Introduction to the Raw Power of AI by a Techie
Not because I don't like sharing. But because the only tool I use for 99% of my AI needs is Claude Code. And while I think it is not hard to learn, I'm never...
The Worst Job Displacement of Software Engineers Is Yet to Come.
This is not another fear mongering post.
Most OpenClaw workshops skip the fundamentals and teach you 1-click setup on AI or cloud providers...
I decided to go for a different path. In my workshop, I guide learners to set up with the original experience: install it on your laptop, but with security i...
💡 Little-known hack to get the most out of Cursor for FREE
If you're using Cursor on the free plan, you will eventually hit the dreaded "servers overload" screen.
How I Collaborate with AI to Refine What I Know
In this article, I share examples of how I practice it in actual work.
Yesterday, we were discussing when we could actually let AI agents auto-merge and publish code.
My take - probably not so soon.