UW: Artificial Intelligence is headed towards doctors’ offices and it is not always a good thing

Imagine you’re at the doctor’s office worried about an illness, and the physician turns to his computer and pulls up Artificial Intelligence (AI) for a diagnosis. We’re not far from that happening and that could be a good thing, but it comes with pitfalls.

In the same way patients scour WebMD, tapping into their symptoms and doomscrolling a long list of possible problems, doctors now use Google as a source of information.

UW News reports AI bots may be next. Dr. Gary Franklin, a University of Washington research professor, described an experience with Google’s Gemini chatbot. When Franklin asked Gemini for information on the outcomes of a specific procedure the bot gave a detailed answer that cited two medical studies, neither of which existed.

MyNorthwest News: Black Friday, Cyber Monday sales break records with buyers using AI, phones

Franklin wrote that it’s “buyer beware when it comes to using AI chatbots to extract accurate scientific information or evidence-based guidance.” He recommended that AI experts develop specialized chatbots that pull information only from verified sources.

One expert working toward a solution is Lucy Lu Wang, a UW assistant professor in the Information School. Wang has developed tools to extract important information from medical research papers, verify scientific claims, and make scientific images accessible to blind and low-vision readers.

“Doctors use Google a lot, but they also rely on services like UpToDate, which provide really great summaries of medical information and research,” Franklin said. “Most doctors have zero time and just want to be able to read something very quickly that is well documented. So from a physician’s perspective, trying to find truthful answers, trying to make my practice more efficient, trying to coordinate things better — if this technology could meaningfully contribute to any of those things, then it would be unbelievably great.”

Franklin added, “I’m not sure how much doctors will use AI, but for many years, patients have been coming in with questions about what they found on the internet, like on WebMD. AI is just the next step of patients doing this, getting some guidance about what to do with the advice they’re getting. As an example, if a patient sees a surgeon who’s overly aggressive and says they need a big procedure, the patient could ask an AI tool what the broader literature might recommend. And I have concerns about that.”

National News: OpenAI’s Sam Altman ‘not that worried’ about rival Elon Musk’s influence in the Trump administration

Wang echoed Franklin’s sentiments, noting that clinicians want to look up information very quickly because they’re so taxed and there’s limited time to treat patients. “You can imagine if the tools that we have, these chatbots, were actually very good at searching for information and very good at citing accurately, that they could become a better replacement for a type of tool like UpToDate,” she said. “Because UpToDate is good, it’s human-curated, but it doesn’t always contain the most fine-grained information you might be looking for.”

Wang also highlighted the potential benefits for patient communication. “These tools could also potentially help clinicians with patient communication, because there’s not always enough time to follow up or explain things in a way that patients can understand. It’s an add-on part of the job for clinicians, and that’s where I think language models and these tools, in an ideal world, could be really beneficial.”

On the patient’s side, Wang said it would be amazing to develop tools that help with patient education and increase the overall health literacy of the population, beyond what WebMD or Google does. “These tools could engage patients with their own health and health care more than before,” she said.

Franklin and Wang both see potential for AI to improve coordination across the health care system and between physicians. “There was a book called ‘Crossing the Quality Chasm’ that argued the main problem in American medicine is poor coordination,” Franklin said.

Wang added that socio-technical problems need to be addressed, such as developing models with the specific goal of supporting scientific search. “People are, in fact, working toward these things and have demonstrated good preliminary results,” she said.

“I think the citation problem has already been overcome in research demonstration cases,” Wang explained. “If we, for example, hook up an LLM to PubMed search and allow it only to cite conclusions based on articles that are indexed in PubMed, then actually the models are very faithful to citations that are retrieved from that search engine. But if you use Gemini and ChatGPT, those are not always hooked up to those research databases.”

“The problem is that a person trying to search using those tools doesn’t know that,” Franklin noted.

Reflecting on the impact of resources like WebMD, Wang said, “Before its existence, patients really did have a hard time finding any information at all. And of course, there’s limited face time with clinicians where people actually get to ask those questions. So for every patient who wrongly self-diagnoses on WebMD, there are probably also hundreds of patients who found a quick answer to a question. I think that with these models, it’s going to be similar. They’re going to help address some of the gaps in clinical care where we don’t currently have enough resources.”

Bill Kaczaraba is a content editor at MyNorthwest. You can read his stories here. Follow Bill on X, formerly known as Twitter, here and email him here.