Eric Topol with Medscape’s Medicine and the Machine podcast.
We have a new episode today with a really interesting fellow physician, Dr Adam Rodman. He’s an internist at Beth Deaconess Medical Center. He’s an educator and instructor at Harvard Medical School, author of the book Short Cuts: Medicine, and has a podcast called Bedside Rounds. He also had articles in both the August 3 and August 10 issues of The New England Journal of Medicine.
Adam Rodman, MD, MPH: Thank you. Eric, I read your book years ago.
Read More
- Omani researchers discover genetic link to early-onset Parkinsonism
- Trufud brings cleaner, greener, homegrown produce to consumers
- Dr. Ebrahimi brings 15 years of surgical expertise to AdLife Hospital
- Free yoga session in Muscat for expectant mothers
- AdLife Hospital welcomes new Orthodontic specialist to elevate dental care
I’m a little starstruck talking to you.
Topol: You’ve been really lighting up lately. Before we talk about medical education, which is where you are leading the charge, I thought we would start with where we stand with the large language models (LLMs): ChatGPT, GPT-3, GPT-4, Gemini and many others that are out there already. How do you see these becoming rooted in the daily practice of medicine? We saw early on the hype about the US Medical Licensing Examination (USMLE), but then we got to patient questions, front door for the doctor, clinical reasoning, diagnosis, etc. What is your vision for where this is going over time?
Rodman: In addition to education, I do clinical reasoning research. There are a couple of domains. I’m sure you saw the preprint from Stanford looking at summarization of documents. We generate a lot of text. Much of what we do is summarize text. My job as an internist is obviously collecting information from patients and building relationships, but it’s also absorbing a lot of information that’s stored in the chart. Much of the near-term uses that we’re seeing, and what people are building right now, is text summarization.
You mentioned clinical reasoning. Most older studies on clinical reasoning were done on the MedQA database for benchmarking, the USMLE questions or USMLE-style questions. My research has been a bit more in-depth about what it means for an LLM to show clinical reasoning or to act as clinical decision support (CDS).
We’ve actually been thinking about this since the 1950s, but we hadn’t taken it too seriously until a year ago. Between 1995 and 2019, at least 15 papers were published evaluating CDS for diagnosis. In the past 6 months, there have been four times that number of papers. So, clearly interest has exploded. This is exciting. If you look from a cognitive psychology standpoint on how doctors make clinical reasoning decisions, a lot of it has to do with words. It has to do with script theory, with semantics, and the understanding that our internal map of diseases and disease processes is mapped out in association with these different semantic qualities.
LLMs don’t work the same way as the human mind, but the evidence is starting to show that general-purpose models appear to make medical decisions, or at least diagnoses, equivalent to or even better than humans in very specific experimental situations. It’s something we haven’t dealt with before in the long history of CDS.
Topol: Well, hopefully everybody is up on how CDS has been evolving. Whereas before it was heuristic primitive algorithms, now it’s becoming multimodal with the foundation models. It’s much more sophisticated than it used to be, when doctors didn’t want CDS in the EHR because it was often stupid or unhelpful. Now it has a chance to really make a difference.