Google Says Its Error-Ridden "AI Overviews" Will Now Give Health Advice

Google's crappy "AI Overviews" regularly spits out dangerous and incorrect answers — and now it's being entrusted with medical advice.

In a self-congratulatory blog post, Google's chief health officer Karen DeSalvo claimed that "recent health-focused advancements on Gemini models" and "our best-in-class quality and ranking systems" will allow the janky feature to "cover thousands more health topics."

Troublingly, the update will include a new feature called "What People Suggest," which will provide health advice from amateurs around the internet. Though DeSalvo said the feature is available on mobile devices in the US, we weren't able to access it on either the Google app or the web, which uses Google — and we've reached out to the company to ask what's up with that.

As Futurism and other outlets have extensively documented, Google's AI Overviews are cartoonishly bad at providing accurate information. From claiming that baby elephants can fit in the palm of a human hand to suggesting you put glue on your pizza, Google's in-search AI has consistently struggled to differentiate between fact and fiction in the nearly two years since its launch.

Indeed, a recent study from Columbia's Tow Center for Digital Journalism found that the search giant's Gemini chatbot, which undergirds AI Overviews, got basic questions wrong an astounding 60 percent of the time — and Google hasn't explained how it plans to fix that problem.

Ironically, Google used to have hundreds of human employees whose jobs centered around health — but in 2021, the company closed its health division and either laid off or reshuffled the people who staffed it. Citing past failures at follow-through on big-vision health tech projects, dozens of experts who spoke to Bloomberg last summer said they were concerned that Google's AI will give accurate and high-quality healthcare information — and there's been little in the intervening period to suggest that those anxieties have been assuaged.

"If Google’s overall ambition is to completely disrupt the health-care industry, well, nobody in the big tech world has succeeded," explained Emarketer senior analyst Rajiv Leventhal told Bloomberg at the time. "Health care is a unique beast."

We've reached out to Google to ask about how it plans to verify that the health information in its AI provides accurate and safe health responses. The last time we asked the company about AI Overviews, it told us that there "may not be a lot of high-quality web content available" for some questions, and as such, the feature will sometimes simply get it wrong.

"We have guardrails and policies in place to protect against low-quality responses," a company spokesperson told Futurism a few weeks ago, "and when issues arise we use those examples to improve and take appropriate action under our policies."

More on Google AI: Did Google Test an Experimental AI on Kids, With Tragic Results?

The post Google Says Its Error-Ridden "AI Overviews" Will Now Give Health Advice appeared first on Futurism.