When Experts and Students Disagree: Divergent Perceptions of Bias Mitigation Strategies In a Stroke Prognosis Support Chatbot
Authors
Hansen, Daniel Dugenio Boiskov ; Lind, Arlonsompoon Phoonjaroen
Term
4. term
Education
Publication year
2025
Pages
10
Abstract
This study investigates the effectiveness of cognitive bias mitigation strategies in an AI-powered CDSS, focusing on reducing premature closure bias among clinicians during prognosis. We designed and evaluated two experimental conditions: one employing a single strategy "hear the story first," which prompts users to review patient data before receiving AI recommendations, and another condition which adds with "consider the opposite," which encourages reflection on alternative prognoses. Our mixed methods evaluation involved 10 medialogy students and 3 practising clinicians. Quantitative results from the TLX and CUQ revealed that students perceived the single-mitigation chatbot as more usable (CUQ score: 69.1 vs. 59.8), with no significant differences in workload. Qualitative feedback showed that students often overlooked mitigation prompts, while clinicians strongly favoured "consider the opposite" for its role in fostering critical reflection and transparency. Notably, clinicians dismissed "hear the story first" as redundant, highlighting a divergence between expert and non expert user needs. The findings underscore the importance of tailoring bias mitigation strategies to the target audience: passive prompts may go unnoticed by non experts, whereas clinicians value active challenges to their reasoning. The study also demonstrates the risks of over reliance on non expert feedback during design, as clinician insights fundamentally reshaped our understanding of effective AI support. Future work should explore standalone implementations of "consider the opposite".
Keywords
Documents