The AI Gap Your Doctor Doesn’t Know Exists
When a child named Maria spent five years being tested, biopsied, and hospitalized for what doctors called an autoimmune liver disease, her parents went broke trying to save her. Then a geneticist using an AI expert system diagnosed her in days. The solution? Change her diet. Eighteen months later, she was thriving.
Pete Martinez, who built that AI platform, put it this way:
“The basic cost and why we have such an expensive medical system right now is the amount of unnecessary testing that we do. In the trial and error format.”
Five to seven years. That’s the average time to diagnose a rare disease. Five to seven years of wrong treatments, wrong medications, and—as Pete put it—” the progression of the disease is monstrous.”
But here’s what most people don’t understand about AI in medicine: not all AI is created equal. And the difference could mean everything for your diagnosis.
The Hallucination Problem
You’ve probably uploaded a scan or a blood test result into ChatGPT and asked what it means. Most of us have. But there’s a problem with that approach that goes beyond the obvious privacy concerns.
When researchers at Deakin University analyzed medical literature reviews generated by ChatGPT, they found that roughly one in five academic citations were completely fabricated. More than half of all citations were either fake or contained errors. The AI didn’t just make mistakes—it invented references that sounded authoritative but didn’t exist.
A separate study in the Journal of Medical Internet Research found that when ChatGPT generated medical references, 47% were fabricated entirely, 46% were real but inaccurate, and only 7% were both real and accurate.
Pete Martinez described this problem in plain terms: “You gotta be careful with the information that you put into a ChatGPT or anything else. Consider that a ChatGPT, a Copilot, any one of these systems is just scraping the whole internet for information. It could be relevant information, it could be non-relevant, it could be outdated, it could be misinformation.”
The phenomenon has a name: hallucination. Though as one researcher pointed out in Nature, calling it that is too generous. These aren’t perceptual errors—they’re fabrications.
What Trusted AI Actually Looks Like
The AI platform Pete built works differently. Instead of scraping the entire internet, it draws from curated, validated sources: Johns Hopkins’ Online Mendelian Inheritance in Man database (OMIM), the National Institutes of Health, the National Center for Biotechnology Information.
“We concentrate on very, very trusted data,” Pete explained. “Data that we either produce from our own analysis, our own AI, our own genomics work that we do, and or we license from trusted sources.”
This distinction matters more than most people realize. A 2024 meta-analysis of 83 studies published in npj Digital Medicine found that generative AI models showed no significant performance difference from physicians overall. But here’s the crucial finding: AI performed significantly worse than expert physicians.
The difference? Expert physicians bring specialized knowledge and pattern recognition that general-purpose AI can’t match. But when that expert knowledge is codified into a purpose-built system—when you put “the mind of an expert, a highly skilled individual” into algorithms, as Pete described—you get something different.
“What we’re doing is amplifying the brain to be a super brain,” Pete told me, “taking a look at the whole human body as an entity, not in very fragmented pieces.”
The Fragmentation Problem
This is where AI could genuinely transform cancer diagnosis. Our medical system has created specialists who only concentrate on very small parts of the body. You see a cardiologist who looks at your heart. An oncologist who looks at your tumor. A neurologist who looks at your nervous system. But as Pete pointed out, “The body doesn’t work that way. The body is a set of integrated systems all having, trying to reach some kind of equilibrium.”
AI expert systems can see patterns across those boundaries—connections between symptoms that no single specialist would notice because they’re looking at different fragments.
Pete shared a story about a 16-year-old girl who’d been experiencing dizziness, then started having fainting spells—sixteen per day. She went from specialist to specialist. Nobody could figure it out. Then she started having seizures. A teenager in a wheelchair, unable to attend school.
When she finally reached Mayo Clinic, they diagnosed her with POTS—a condition involving both the heart and the nervous system. Pete entered her symptoms into his AI platform and got the same diagnosis.
“How did you do that?” her mother asked.
“It’s the collective minds of very, very brilliant people that have produced significant amount of data,” Pete said, “and all we’re doing is compressing it, synthesizing, aggregating it, and turning it into actionable intelligence.”
What This Means for You
So where does this leave you when you’re facing a cancer diagnosis and trying to figure out which AI tools might actually help?
First, understand the difference between general-purpose AI and specialized clinical decision support. ChatGPT can help you understand what a term means. It cannot reliably diagnose your condition or verify that the information it gives you is accurate.
Second, know that tools like Pete’s exist—but they’re designed for clinicians, not consumers. The question to ask your doctor isn’t “Can I use ChatGPT?” It’s “Are you using any AI-based clinical decision support tools? What data sources do they draw from?”
Third, remember Pete’s advice about physicians who dismiss your requests for more comprehensive testing: “Walk away.”
AI won’t replace your doctor. But the right AI tools, in the hands of a doctor who knows how to use them, could compress your diagnostic journey from years to days.
That’s not hype. That’s 162,000 patients whose lives were changed because someone asked better questions.
As Dr. Eric Topol, one of the most cited researchers in medicine, puts it: “AI will always need human backup... Doctors shouldn’t be dealing with things that machines will do better than them. But serious conditions, like getting a cancer diagnosis, are what doctors should be working on.”
The technology exists. The question is whether you’ll push for it—and whether your doctor will listen.
🎧 Hear the full conversation: Listen to this week’s episode of Kicking Cancer’s Ass wherever you get your podcasts.
Key Links
Sivotec - Pete’s company
THEME 1: The Scale of Diagnostic Harm
Dr. David Newman-Toker, Director, Armstrong Institute Center for Diagnostic Excellence, Johns Hopkins:
“Reducing diagnostic errors by 50% for stroke, sepsis, pneumonia, pulmonary embolism and lung cancer could cut permanent disabilities and deaths by 150,000 per year.”
Source: Johns Hopkins Medicine press release, July 2023 URL: https://www.hopkinsmedicine.org/news/newsroom/news-releases/2023/07/report-highlights-public-health-impact-of-serious-harms-from-diagnostic-error-in-us Context: His research estimates 795,000 Americans die or are permanently disabled annually due to diagnostic errors—371,000 deaths and 424,000 permanent disabilities.
Dr. Jeffrey Schnipper, Brigham and Women’s Hospital:
“We know diagnostic errors are dangerous, and hospitals are obviously interested in reducing their frequency, but it’s much harder to do this when we don’t know what’s causing these errors or what their direct impact is on individual patients.”
Source: Harvard Gazette, January 2024 URL: https://news.harvard.edu/gazette/story/2024/01/research-assesses-rates-causes-of-diagnostic-errors/ Context: His study found 23% of seriously ill hospitalized patients experienced a diagnostic error, with errors contributing to 1 in 15 deaths.
Dr. Andrew Auerbach, UCSF Division of Hospital Medicine:
“Our study is similar to studies from the ‘90s describing the prevalence and impact of common patient safety events, such as medication errors, studies which catalyzed the patient safety movement. We hope our work provides a similar call to action to academic medical centers, researchers and policymakers.”
Source: UCSF News, January 2024 URL: https://www.ucsf.edu/news/2024/01/426941/diagnostic-errors-are-common-seriously-ill-hospitalized-adults Context: Co-author on the same study showing 23% diagnostic error rate in seriously ill patients.
THEME 2: AI’s Potential to Transform Diagnosis
Dr. Eric Topol, Executive Vice President, Scripps Research (author of Deep Medicine):
“Machine eyes will see things that humans will never see. It’s actually quite extraordinary.”
Source: NIH Record, November 2024 URL: https://nihrecord.nih.gov/2024/11/22/topol-discusses-potential-ai-transform-medicine Context: Topol cited a 2023 Johns Hopkins study estimating 800,000 Americans are seriously disabled or die from incorrect diagnoses each year.
Dr. Eric Topol:
“AI will always need human backup... Doctors shouldn’t be dealing with things that machines will do better than them. But serious conditions, like getting a cancer diagnosis, are what doctors should be working on.”
Source: TIME interview, March 2019 URL: https://time.com/collection/life-reinvented/5551296/cardiologist-eric-topol-artificial-intelligence-interview/ Context: This aligns perfectly with Pete’s “clinical decision support” framing—AI assists, the physician decides.
Dr. Eric Topol:
“The doctor’s role is to interpret and communicate information, not hoard it.”
Source: Bookey compilation of Topol quotes URL: https://www.bookey.app/quote-author/eric-topol Context: Reinforces patient empowerment and the partnership model Pete advocates.
Dr. Eric Topol:
“This is not the right type of medicine that we want to practice. We want to have presence, trust and a bond that’s ameliorated and built upon during a visit.”
Source: NIH Record, November 2024 URL: https://nihrecord.nih.gov/2024/11/22/topol-discusses-potential-ai-transform-medicine Context: Speaking about how doctors spend so much time on administrative tasks they can’t adequately examine patients.
Dr. Eric Topol (from his book description):
“By freeing physicians from the tasks that interfere with human connection, AI will create space for the real healing that takes place between a doctor who can listen and a patient who needs to be heard.”
Source: Scripps Research press release, March 2019 URL: https://www.scripps.edu/news-and-events/press-room/2019/20190312-topol-deep-medicine.html Context: Core thesis of his book Deep Medicine.
Stanford HAI (Human-Centered Artificial Intelligence):
“AI is not replacing doctors. Only your doctor will prescribe medications, perform operations, or administer any other interventions.” — Dr. Ethan Goh, Stanford School of Medicine
Source: Stanford HAI, November 2024 URL: https://hai.stanford.edu/news/can-ai-improve-medical-diagnostic-accuracy Context: From a study showing ChatGPT alone scored 92% on diagnostic accuracy while physicians scored 74-76%.
THEME 3: Patient Empowerment Changes Outcomes
AHRQ (Agency for Healthcare Research and Quality):
“Failure to use advocacy and assertion has been frequently identified as a primary contributor to the clinical errors found in malpractice cases and sentinel events. You should advocate for the patient even when your viewpoint is unpopular, is in opposition to another person’s view, or questions authority.”
Source: AHRQ TeamSTEPPS toolkit URL: https://www.ahrq.gov/teamstepps-program/curriculum/mutual/tools/advocacy.html Context: From the federal government’s patient safety resources—validates Pete’s “walk away” advice.
World Health Organization:
“Investment in patient engagement, if done well, can reduce the burden of harm by up to 15%.”
Source: WHO Patient Safety Fact Sheet, September 2023 URL: https://www.who.int/news-room/fact-sheets/detail/patient-safety Context: Validates that patient advocacy isn’t just feel-good advice—it measurably reduces harm.
THEME 4: The Hallucination Problem (supporting research)
Journal of Medical Internet Research:
“Given their current performance, it is not recommended for LLMs to be deployed as the primary or exclusive tool for conducting systematic reviews. Any references generated by such models warrant thorough validation by researchers.”
Source: JMIR, May 2024 URL: https://www.jmir.org/2024/1/e53164/ Context: Study on ChatGPT hallucination rates in medical literature searches.
Nature (Schizophrenia journal):
“One study investigating the authenticity and accuracy of references in medical articles generated by ChatGPT found that of 115 references that were generated, 47% were fabricated, 46% were authentic but inaccurate, and only 7% were authentic and accurate.”
Source: Nature Schizophrenia, August 2023 URL: https://www.nature.com/articles/s41537-023-00379-4 Context: The researcher argues these aren’t “hallucinations”—they’re fabrications and falsifications.