Paper
A prospective clinical feasibility study of a conversational diagnostic AI in an ambulatory primary care clinic
Authors
Peter Brodeur, Jacob M. Koshy, Anil Palepu, Khaled Saab, Ava Homiar, Roma Ruparel, Charles Wu, Ryutaro Tanno, Joseph Xu, Amy Wang, David Stutz, Hannah M. Ferrera, David Barrett, Lindsey Crowley, Jihyeon Lee, Spencer E. Rittner, Ellery Wulczyn, Selena K. Zhang, Elahe Vedadi, Christine G. Kohn, Kavita Kulkarni, Vinay Kadiyala, Sara Mahdavi, Wendy Du, Jessica Williams, David Feinbloom, Renee Wong, Tao Tu, Petar Sirkovic, Alessio Orlandi, Christopher Semturs, Yun Liu, Juraj Gottweis, Dale R. Webster, Joëlle Barral, Katherine Chou, Pushmeet Kohli, Avinatan Hassidim, Yossi Matias, James Manyika, Rob Fields, Jonathan X. Li, Marc L. Cohen, Vivek Natarajan, Mike Schaekermann, Alan Karthikesalingam, Adam Rodman
Abstract
Large language model (LLM)-based AI systems have shown promise for patient-facing diagnostic and management conversations in simulated settings. Translating these systems into clinical practice requires assessment in real-world workflows with rigorous safety oversight. We report a prospective, single-arm feasibility study of an LLM-based conversational AI, the Articulate Medical Intelligence Explorer (AMIE), conducting clinical history taking and presentation of potential diagnoses for patients to discuss with their provider at urgent care appointments at a leading academic medical center. 100 adult patients completed an AMIE text-chat interaction up to 5 days before their appointment. We sought to assess the conversational safety and quality, patient and clinician experience, and clinical reasoning capabilities compared to primary care providers (PCPs). Human safety supervisors monitored all patient-AMIE interactions in real time and did not need to intervene to stop any consultations based on pre-defined criteria. Patients reported high satisfaction and their attitudes towards AI improved after interacting with AMIE (p < 0.001). PCPs found AMIE's output useful with a positive impact on preparedness. AMIE's differential diagnosis (DDx) included the final diagnosis, per chart review 8 weeks post-encounter, in 90% of cases, with 75% top-3 accuracy. Blinded assessment of AMIE and PCP DDx and management (Mx) plans suggested similar overall DDx and Mx plan quality, without significant differences for DDx (p = 0.6) and appropriateness and safety of Mx (p = 0.1 and 1.0, respectively). PCPs outperformed AMIE in the practicality (p = 0.003) and cost effectiveness (p = 0.004) of Mx. While further research is needed, this study demonstrates the initial feasibility, safety, and user acceptance of conversational AI in a real-world setting, representing crucial steps towards clinical translation.
Metadata
Related papers
Gen-Searcher: Reinforcing Agentic Search for Image Generation
Kaituo Feng, Manyuan Zhang, Shuang Chen, Yunlong Lin, Kaixuan Fan, Yilei Jian... • 2026-03-30
On-the-fly Repulsion in the Contextual Space for Rich Diversity in Diffusion Transformers
Omer Dahary, Benaya Koren, Daniel Garibi, Daniel Cohen-Or • 2026-03-30
Graphilosophy: Graph-Based Digital Humanities Computing with The Four Books
Minh-Thu Do, Quynh-Chau Le-Tran, Duc-Duy Nguyen-Mai, Thien-Trang Nguyen, Khan... • 2026-03-30
ParaSpeechCLAP: A Dual-Encoder Speech-Text Model for Rich Stylistic Language-Audio Pretraining
Anuj Diwan, Eunsol Choi, David Harwath • 2026-03-30
RAD-AI: Rethinking Architecture Documentation for AI-Augmented Ecosystems
Oliver Aleksander Larsen, Mahyar T. Moghaddam • 2026-03-30
Raw Data (Debug)
{
"raw_xml": "<entry>\n <id>http://arxiv.org/abs/2603.08448v1</id>\n <title>A prospective clinical feasibility study of a conversational diagnostic AI in an ambulatory primary care clinic</title>\n <updated>2026-03-09T14:43:40Z</updated>\n <link href='https://arxiv.org/abs/2603.08448v1' rel='alternate' type='text/html'/>\n <link href='https://arxiv.org/pdf/2603.08448v1' rel='related' title='pdf' type='application/pdf'/>\n <summary>Large language model (LLM)-based AI systems have shown promise for patient-facing diagnostic and management conversations in simulated settings. Translating these systems into clinical practice requires assessment in real-world workflows with rigorous safety oversight. We report a prospective, single-arm feasibility study of an LLM-based conversational AI, the Articulate Medical Intelligence Explorer (AMIE), conducting clinical history taking and presentation of potential diagnoses for patients to discuss with their provider at urgent care appointments at a leading academic medical center. 100 adult patients completed an AMIE text-chat interaction up to 5 days before their appointment. We sought to assess the conversational safety and quality, patient and clinician experience, and clinical reasoning capabilities compared to primary care providers (PCPs). Human safety supervisors monitored all patient-AMIE interactions in real time and did not need to intervene to stop any consultations based on pre-defined criteria. Patients reported high satisfaction and their attitudes towards AI improved after interacting with AMIE (p < 0.001). PCPs found AMIE's output useful with a positive impact on preparedness. AMIE's differential diagnosis (DDx) included the final diagnosis, per chart review 8 weeks post-encounter, in 90% of cases, with 75% top-3 accuracy. Blinded assessment of AMIE and PCP DDx and management (Mx) plans suggested similar overall DDx and Mx plan quality, without significant differences for DDx (p = 0.6) and appropriateness and safety of Mx (p = 0.1 and 1.0, respectively). PCPs outperformed AMIE in the practicality (p = 0.003) and cost effectiveness (p = 0.004) of Mx. While further research is needed, this study demonstrates the initial feasibility, safety, and user acceptance of conversational AI in a real-world setting, representing crucial steps towards clinical translation.</summary>\n <category scheme='http://arxiv.org/schemas/atom' term='cs.HC'/>\n <category scheme='http://arxiv.org/schemas/atom' term='cs.AI'/>\n <category scheme='http://arxiv.org/schemas/atom' term='cs.CL'/>\n <category scheme='http://arxiv.org/schemas/atom' term='cs.LG'/>\n <published>2026-03-09T14:43:40Z</published>\n <arxiv:primary_category term='cs.HC'/>\n <author>\n <name>Peter Brodeur</name>\n </author>\n <author>\n <name>Jacob M. Koshy</name>\n </author>\n <author>\n <name>Anil Palepu</name>\n </author>\n <author>\n <name>Khaled Saab</name>\n </author>\n <author>\n <name>Ava Homiar</name>\n </author>\n <author>\n <name>Roma Ruparel</name>\n </author>\n <author>\n <name>Charles Wu</name>\n </author>\n <author>\n <name>Ryutaro Tanno</name>\n </author>\n <author>\n <name>Joseph Xu</name>\n </author>\n <author>\n <name>Amy Wang</name>\n </author>\n <author>\n <name>David Stutz</name>\n </author>\n <author>\n <name>Hannah M. Ferrera</name>\n </author>\n <author>\n <name>David Barrett</name>\n </author>\n <author>\n <name>Lindsey Crowley</name>\n </author>\n <author>\n <name>Jihyeon Lee</name>\n </author>\n <author>\n <name>Spencer E. Rittner</name>\n </author>\n <author>\n <name>Ellery Wulczyn</name>\n </author>\n <author>\n <name>Selena K. Zhang</name>\n </author>\n <author>\n <name>Elahe Vedadi</name>\n </author>\n <author>\n <name>Christine G. Kohn</name>\n </author>\n <author>\n <name>Kavita Kulkarni</name>\n </author>\n <author>\n <name>Vinay Kadiyala</name>\n </author>\n <author>\n <name>Sara Mahdavi</name>\n </author>\n <author>\n <name>Wendy Du</name>\n </author>\n <author>\n <name>Jessica Williams</name>\n </author>\n <author>\n <name>David Feinbloom</name>\n </author>\n <author>\n <name>Renee Wong</name>\n </author>\n <author>\n <name>Tao Tu</name>\n </author>\n <author>\n <name>Petar Sirkovic</name>\n </author>\n <author>\n <name>Alessio Orlandi</name>\n </author>\n <author>\n <name>Christopher Semturs</name>\n </author>\n <author>\n <name>Yun Liu</name>\n </author>\n <author>\n <name>Juraj Gottweis</name>\n </author>\n <author>\n <name>Dale R. Webster</name>\n </author>\n <author>\n <name>Joëlle Barral</name>\n </author>\n <author>\n <name>Katherine Chou</name>\n </author>\n <author>\n <name>Pushmeet Kohli</name>\n </author>\n <author>\n <name>Avinatan Hassidim</name>\n </author>\n <author>\n <name>Yossi Matias</name>\n </author>\n <author>\n <name>James Manyika</name>\n </author>\n <author>\n <name>Rob Fields</name>\n </author>\n <author>\n <name>Jonathan X. Li</name>\n </author>\n <author>\n <name>Marc L. Cohen</name>\n </author>\n <author>\n <name>Vivek Natarajan</name>\n </author>\n <author>\n <name>Mike Schaekermann</name>\n </author>\n <author>\n <name>Alan Karthikesalingam</name>\n </author>\n <author>\n <name>Adam Rodman</name>\n </author>\n </entry>"
}