Electronic Health Record (EHR) Based Conversational System for Clinical Decision Support
PI: Vasant Honavar (IST)
The PI will be responsible for covering tuition for the graduate student.
Proposal Description:
Recent advances in Large Language Models (LLMs) have led to remarkable progress in medical consultation. However, existing work on medical LLMs have focused primarily on clinical diagnosis and do not take into account the clinical workflow and the central role of the physician in clinical decision-making. Consequently, such work has focused on LLMs that can pass the US Medical Licensing Exam (USMLE) or automate aspects of clinical diagnosis. However, our preliminary research with clinical collaborators has revealed that their need is better served by an AI-powered decision support system that augments the human physicians, improving their performance, especially in high-stress, time-constrained settings. Against this background, this project aims to assess the feasibility of an LLM-powered, EHR-based conversational system for clinical decision support.
Specifically, the project aims to adapt a general purpose LLM by finetuning it with EHR data, clinical guidelines, clinical tests and the guidance for interpretation of test outcomes (normal versus abnormal in a given context), to realize a conversational system that can be used as a learning tool by physicians in training, as a decision support tool by practicing physicians or physician assistants (in low-resource, e.g., rural, settings), and as an information source by patients and caregivers.
The specific aims of this research include:
• Assembling curated set of data for finetuning an LLM to achieve the desired functionality (in close collaboration with expert clinicians who are working with the PI)
• Finetuning off-the-shelf state-of-the-art LLM using the assembled data. This will require addressing several challenges (including aligning the LLM responses with user preferences (for different classes of users, including physicians-in-training, practicing physicians, patients and caregivers), detecting and mitigating LLM hallucinations, etc.
• Designing benchmarks to evaluate the LLM along the key dimensions (accuracy and relevance of responses, logical coherence of the conversations, quality of explanations offered, user satisfaction, etc.)
• Implementing and evaluating a prototype version of the EHR-based conversational system for clinical decision support through carefully designed empirical studies
Long-Term Goal:
The long-term goals of the project are to develop validated, robust, conversational clinical decision support systems for different classes of users. The current project will help initiate the proposed work, help gather preliminary data, and help lay the foundations for an ambitious, ideally externally funded, high-impact interdisciplinary research project.
Connection to ICDS Mission:
This project directly supports the ICDS mission by advancing core methods related to the development, validation and evaluation of advanced LLM-powered decision support systems for high-impact applications in general, and clinical decision support in particular.
Ideal student background:
Deep knowledge of LLM (including methods for aligning LLM with human preferences), and familiarity with electronic health records data.