When Experts and Students Disagree: Divergent Perceptions of Bias Mitigation Strategies In a Stroke Prognosis Support Chatbot
Authors
Hansen, Daniel Dugenio Boiskov ; Lind, Arlonsompoon Phoonjaroen
Term
4. term
Education
Publication year
2025
Pages
10
Abstract
Kunstig intelligens bruges i stigende grad i kliniske beslutningsstøttesystemer (CDSS) til at hjælpe klinikere med at udarbejde prognoser (vurderinger af patienters fremtidige forløb). Menneskelig tænkning er dog sårbar over for kognitive skævheder (bias) – mentale genveje, der kan føre til fejl. Dette studie afprøvede to enkle tiltag, der skal mindske for tidlig konklusion (premature closure), altså tendensen til at stoppe for tidligt med at overveje alternativer. Vi sammenlignede to versioner af en AI-chatbot til prognosestøtte: Én brugte en enkelt strategi, "hør historien først", som beder brugeren gennemgå patientdata før AI'ens anbefaling; den anden kombinerede dette med "overvej det modsatte", som opfordrer brugeren til at tænke i alternative prognoser. Vi gennemførte et blandet metode-studie med 10 medialogi-studerende og 3 praktiserende klinikere og indsamlede vurderinger af brugbarhed og arbejdsbyrde (CUQ og TLX) samt kvalitativ feedback. Blandt de studerende blev chatbotten med én strategi vurderet som mere brugbar (CUQ 69,1 vs. 59,8), uden væsentlig forskel i oplevet arbejdsbyrde. De studerende overså dog ofte opfordringerne. Klinikere foretrak derimod "overvej det modsatte", fordi det fremmede kritisk refleksion og transparens. De fandt "hør historien først" overflødig, hvilket peger på forskellige behov hos eksperter og ikke-eksperter. Resultaterne viser, at bias-tiltag bør tilpasses målgruppen: Passive opfordringer kan gå ikke-eksperter forbi, mens klinikere værdsætter prompts, der aktivt udfordrer deres ræsonnement. Studiet advarer også mod at lægge for stor vægt på feedback fra ikke-eksperter, da klinikernes input ændrede vores forståelse af effektiv AI-støtte. Fremtidigt arbejde bør afprøve en selvstændig udgave af "overvej det modsatte".
Artificial intelligence is increasingly used in clinical decision support systems (CDSS) to help clinicians form prognoses (predictions about patient outcomes). Human thinking, however, is vulnerable to cognitive biases—mental shortcuts that can lead to error. This study tested two simple prompts designed to reduce premature closure, the tendency to stop considering alternatives too early. We compared two versions of an AI chatbot for prognosis support: one used a single strategy, "hear the story first," which asks users to review patient data before seeing the AI's recommendation; the other combined that with "consider the opposite," which urges users to think through alternative prognoses. We ran a mixed-methods study with 10 medialogy students and 3 practising clinicians, gathering usability and workload ratings (CUQ and TLX) and qualitative feedback. Among students, the single-strategy chatbot was rated as more usable (CUQ 69.1 vs. 59.8), with no significant difference in perceived workload. However, students often overlooked the prompts. Clinicians, in contrast, preferred "consider the opposite" because it encouraged critical reflection and transparency. They viewed "hear the story first" as redundant, highlighting different needs for expert and non-expert users. These findings suggest bias mitigation should be tailored to the audience: passive reminders may be ignored by non-experts, while clinicians value prompts that actively challenge their reasoning. The study also cautions against relying too heavily on non-expert feedback during design, as clinician input reshaped our understanding of effective AI support. Future work should evaluate standalone implementations of "consider the opposite."
[This summary has been rewritten with the help of AI based on the project's original abstract]
Keywords
Documents