Development of tool for remote asynchronous usability and user experience testing of desktop applications
Author
Eriksen, Asger Møller
Term
4. term
Education
Publication year
2023
Submitted on
2023-06-02
Pages
77
Abstract
People increasingly expect software to be easy to use. This project explored a way to evaluate usability without bringing users and researchers together at the same place or time. We developed and tested an asynchronous remote usability testing (ARUT) approach, where participants complete tasks on their own computers, in their own time, while data is collected automatically. We applied the method to a case: REALvision Pro, a desktop slicing software. We ran two studies. Study 1 used ARUT. Study 2 used traditional in-person sessions as a benchmark. Results: Study 2 produced much richer information and uncovered more usability problems, including severe ones, than Study 1. The issues found in Study 2 were better supported because we could combine different data sources. In Study 1, two planned sources—critical-incident reports (users flagging major problems) and log data (automatically recorded actions)—did not work as intended and yielded no findings. The different setups also led to different types of problems being identified. Conclusion: The ARUT tool reduces resource use, scales well, and offers flexibility. However, in this case, in-person testing still delivers deeper insights and reveals more usability issues, which outweighs the benefits of the remote approach.
Folk forventer i stigende grad, at software er let at bruge. I dette projekt undersøgte vi en måde at teste brugervenlighed på uden at brugere og forskere behøver at være samme sted eller på samme tidspunkt. Vi udviklede og afprøvede asynkron fjernbrugertest (ARUT), hvor deltagere løser opgaver på deres egne computere, når det passer dem, mens data indsamles automatisk. Metoden blev afprøvet på en case: desktop-slicingsoftwaret REALvision Pro. Vi gennemførte to studier. Studie 1 brugte ARUT. Studie 2 brugte traditionelle fysiske tests som reference. Resultater: Studie 2 gav langt rigere information og afslørede flere brugervenlighedsproblemer, også alvorlige, end studie 1. Problemerne i studie 2 var bedre underbygget, fordi vi kunne kombinere forskellige datakilder. I studie 1 fungerede to planlagte kilder – rapporter om kritiske hændelser (når brugere selv markerer store problemer) og logdata (automatisk registrerede handlinger) – ikke efter hensigten og gav derfor ingen fund. De forskellige metoder førte også til, at der blev fundet forskellige typer problemer. Konklusion: ARUT-værktøjet sparer ressourcer, kan skaleres og giver fleksibilitet. Men i denne case giver fysiske tests stadig dybere indsigter og finder flere brugervenlighedsproblemer, hvilket vejer tungere end fordelene ved fjernmetoden.
[This apstract has been rewritten with the help of AI based on the project's original abstract]