Evaluating Ambient AI for Clinical Documentation: A Mixed-Methods Study
Translated title
Evaluering af Ambient AI til klinisk dokumentation: Et mixed-methods studie
Authors
Moss, Mille Borregaard ; Schultz, Robert Bjeldskov ; Nielsen, Stine Bagger ; Sørensen, Louise Riise
Term
3. term
Education
Publication year
2025
Submitted on
2025-12-17
Pages
22
Abstract
Baggrund og formål: Dokumentation i elektroniske patientjournaler øger klinikeres arbejdsbyrde og risiko for udbrændthed. “Ambient” AI-skrivere—AI, der lytter under konsultationer og laver udkast til noter—kan måske lette denne byrde. Dette studie undersøger, hvordan sådan AI påvirker dokumentationsarbejdet i klinisk praksis. Metode: Et mixed-methods, forklarende sekventielt design med først kvantitative målinger og dernæst kvalitative forklaringer. To klinikere deltog (en psykolog og en sygeplejerske). Vi målte dokumentationstid og fejlrater og analyserede dem med standardstatistiske test (bl.a. Mann-Whitney U samt Pearson- og Spearman-korrelationer). Derudover gennemførtes et semistruktureret interview, som blev tematisk analyseret efter Braun og Clarke. Resultater: For sygeplejersken steg dokumentationstiden fra 7,5 til 21,5 minutter (p = 0,024). For psykologen sås ingen statistisk signifikant forskel (62,0 vs. 86,4 minutter; p = 0,190). En læringseffekt (hurtigere over tid) blev kun fundet for psykologen (ρ = –0,689; p = 0,040; 95 % KI [–0,932; –0,022]). Den gennemsnitlige fejlrate var højere for psykologen end for sygeplejersken (34,11 [SD 12,33] vs. 9,33 [SD 3,50]). Fejlraten var ikke forbundet med dokumentationstid eller antal sessioner. Kvalitativt blev AI’en oplevet som intuitiv og nyttig til at strukturere udkast, men hyppige fejl, manglende oplysninger og “hallucinationer” (AI, der finder på indhold) svækkede tilliden og forhindrede tidsbesparelser. Konklusion: I dette lille studie forbedrede ambient AI ikke dokumentationseffektiviteten, gav blandet brugertilfredshed og viste ikke tilstrækkelig effektivitet. Yderligere forskning er nødvendig på grund af studiets begrænsninger.
Background and objective: Documentation in electronic health records increases clinicians’ workload and burnout risk. Ambient AI scribes—AI that listens during visits and drafts notes—might reduce this burden. This study examines how such AI affects documentation work in clinical practice. Methods: We used a mixed-methods, explanatory sequential design: quantitative measurements first, followed by qualitative explanations. Two clinicians participated (a psychologist and a nurse). We measured documentation time and error rates and analyzed them with standard statistical tests (including Mann–Whitney U and Pearson’s and Spearman’s correlations). We also conducted a semistructured interview and performed thematic analysis following Braun and Clarke. Results: For the nurse, documentation time increased from 7.5 to 21.5 minutes (p = 0.024). For the psychologist, there was no statistically significant difference (62.0 vs 86.4 minutes; p = 0.190). A learning effect (faster over time) appeared only for the psychologist (ρ = –0.689; p = 0.040; 95% CI [–0.932, –0.022]). Mean error rates were higher for the psychologist than the nurse (34.11 [SD 12.33] vs 9.33 [SD 3.50]). Error rate was not associated with documentation time or number of sessions. Qualitatively, the AI was seen as intuitive and helpful for structuring drafts, but frequent errors, missing information, and ‘hallucinations’ (made‑up content) undermined trust and prevented time savings. Conclusion: In this small study, ambient AI did not improve documentation efficiency, was associated with ambivalent user satisfaction, and did not show sufficient effectiveness. Further research is needed given the study’s limitations.
[This summary has been rewritten with the help of AI based on the project's original abstract]
Documents