A curious question from my kids sent Gemini into a hallucination.
Google's AI Overview can be 100% wrong, even when SERP is right.
Google’s AI Overview can be 100% wrong, even when SERP is right.
It could be a trap when you least expect it.
Today, my kids asked me, “What is Buah Long Long?” I quickly Googled it to show them images. The initial search was good: both the AI Overview and SERP correctly identified “Buah Long Long” as Ambarella Fruit.
However, when I tapped on “Benefits of Buah Long Long,” the AI Overview hallucinated, confidently stating that Buah Long Long is also known as longan. This led to a cascade of completely wrong information.
Ironically, the SERP results for “Benefits of Buah Long Long” were accurate. AI Overview is supposed to be grounded in Google Search, so why was it so far off?
This isn’t just about fruit; it’s a critical reminder about the reliability of information, especially from AI summaries.
I tested the same prompt, “Benefits of Buah Long Long,” on Perplexity, and it had no issues. This incident, combined with my past experiences, indicates that Gemini still exhibits the worst hallucinations among frontier LLMs.
My takeaway: Always, always, always double-check facts presented by AI Overview.
What are the worst hallucinations you’ve encountered recently?
Technical observation:
It hallucinates when I least expect it. But here’s an interesting observation: Buah Long Long and Longan both contain the word “Long”. Could this be related to tokenization?
#gemini #googleai #ai #google #artificialintelligence #chatgpt #perplexity #openai #genai #llm #hallucinations
Tap to expand
Enjoyed this? Subscribe for more.
Practical insights on AI, growth, and independent learning. No spam.
More in AI Agents
Not every automation needs an AI agent. After burning $25+ with a browser agent just to download analytics of my top LinkedIn posts, I decided to build a simple automation tool that costs nothing to run.
--
Stop using AI as a calculator.
One of the most common mistakes people make with AI is asking it to do the calculation directly.
I spent years looking for one book that lists the exhaustive set of science concepts applied in...
AI agent + skill is commoditizing (or democratizing for a better word) knowledge at a faster pace than many realized.
We often hear that AI is taking over Google as the go-to source of information.
But in my experience, sometimes Google is still faster and better than AI.
Business+AI Lab with AWS: Event Recap
Held at AWS Singapore, this hands-on GenAI Lab gave participants the rare opportunity to build and deploy their own enterprise-grade chatbot, in under 3 hour...
Most OpenClaw workshops skip the fundamentals and teach you 1-click setup on AI or cloud providers...
I decided to go for a different path. In my workshop, I guide learners to set up with the original experience: install it on your laptop, but with security i...
Not every automation needs an AI agent. After burning $25+ with a browser agent just to download analytics of my top LinkedIn posts, I decided to build a simple automation tool that costs nothing to run.
--
I spent years looking for one book that lists the exhaustive set of science concepts applied in...
AI agent + skill is commoditizing (or democratizing for a better word) knowledge at a faster pace than many realized.
Most OpenClaw workshops skip the fundamentals and teach you 1-click setup on AI or cloud providers...
I decided to go for a different path. In my workshop, I guide learners to set up with the original experience: install it on your laptop, but with security i...
Stop using AI as a calculator.
One of the most common mistakes people make with AI is asking it to do the calculation directly.
We often hear that AI is taking over Google as the go-to source of information.
But in my experience, sometimes Google is still faster and better than AI.
Business+AI Lab with AWS: Event Recap
Held at AWS Singapore, this hands-on GenAI Lab gave participants the rare opportunity to build and deploy their own enterprise-grade chatbot, in under 3 hour...