Search Engine Optimization (SEO) was the name of the web search game until perhaps the last few years; now, however, AI chatbot summaries have taken over. More and more people rely on these quick and handy summaries, which provide—or at least are expected to provide—a refined overview of the topic in question. But is that actually the case, and if not, what could the consequences be? That is the question this article addresses, though it wasn’t where I originally started.
The topic of this article was supposed to be science and its headlines, with an emphasis on microplastics. However, research training has taught me that once you start exploring, results often steer you in new directions. That is the beauty of the scientific method: the hypothesis is only the first step. You then begin asking questions and working your way through the labyrinth of results to reach a conclusion.
My hypothesis: Shock-value science journalism misleads the public by oversimplifying complex observational studies.
My question—or rather, my Google search—was simply “microplastics and brain.” Google’s AI mode came back with the following statement: “Recent studies published in Nature Medicine show that human brain tissue accumulates measurable and increasing amounts of microplastics and nanoplastics.” The search cited five different sources it had used to develop this summary. Interestingly, the actual Nature Medicine article by the University of New Mexico (UNM) research group, although open access, was missing from the source list. Instead, its primary citations—as shown in a screenshot below—were materials published in news articles: The Guardian, Health Policy Watch, Smithsonian Magazine, the University of New Mexico press release, and a research article published in Frontiers in Toxicology (available on PubMed Central).
I had set out to investigate these headlines, knowing that the primary UNM study had received what is fairly common in new avenues of research: a scientific challenge. The Nature Medicine article used Py-GC/MS detection of polymer solids, and other experts, in a correspondence piece, put forth arguments concerning the limitations of Py-GC/MS for microplastics detection in tissues. This was precisely my follow-up question to the AI bot. Conveniently enough, the bot came back with a nuanced summary: “The use of Pyrolysis-Gas Chromatography-Mass Spectrometry (Py-GC/MS) to detect microplastics in human organs has triggered intense debate within the analytical chemistry and toxicology communities. While Py-GC/MS is uniquely valuable because it measures total polymer mass rather than individual particle counts—allowing it to capture ‘invisible’ sub-micron nanoplastics—its application to complex biological tissues carries significant technical arguments.”
Funnily enough, while I was researching how news media articles latch onto “shock-value” research results, The Guardian was once again the top source for the AI bot. Credit goes to The Guardian for publishing a detailed story on the challenges faced by the original research article, and credit to the scientific publishing enterprise for strengthening scientific dialogue and the scientific method.
My initial hypothesis was partially correct: news media headlines on scientific discoveries can sometimes be misleading, likely because that is the nature of their relationship. Science is always exploring and evolving. A new discovery makes for a good headline, but good journalism follows up when that discovery is challenged and makes sure the response gets attention too.
However, my hypothesis and my question led me to an unexpected result that wasn’t even on my radar: AI chatbots. Increasingly integrated into daily life, AI is marketed as an amazingly intelligent companion to help with all our needs. Yet, much like humans, it has blind spots and limitations. I am a strong proponent of the value LLMs can bring to our lives, but we must recognize that they are simply tools providing curated access to vast reserves of human knowledge. Just like any tool, we need to learn its proper usage, applications, and limitations.
Traditional search engines used to hand you a scattered list of links, forcing you to do the heavy lifting of clicking through, cross-referencing, and evaluating what was credible. But an LLM doesn’t just search; it weaves. It takes a handful of disparate sources, blends them together, and presents them back to you in a single, polished, authoritative-sounding narrative. Because the output reads so cohesive and complete, it lulls you into a false sense of security. You are far less likely to question or cross-check an answer that feels like it has already done all the thinking for you—even when, as in my case, it completely bypassed the primary academic paper in favor of secondary news reports.
I am glad my training in critical thinking helped me look past this blind spot, and I hope this reflection can help others do the same.
Leave a Reply