Development and evaluation of the chatbot solution for Wikipedia editors
Authors
Gluza, Wioletta ; Turaj, Izabela Anna
Term
4. term
Publication year
2022
Submitted on
2022-01-02
Pages
71
Abstract
Wikipedia aims to increase diversity and inclusion and make knowledge available to everyone. Its 2030 goals include closing knowledge gaps and strengthening the foundations that make content more globally representative and accessible. This thesis examines how technology can support these aims by improving the experience of volunteer Wikipedia editors. We design and evaluate a chatbot intended to help editors get started (activation) and keep contributing over time (retention). The chatbot follows an atomic design approach, building the interface from small, reusable components. It is built with IBM Watson Assistant (a tool for creating conversational agents) as a full‑stack web application using React and Node.js, and hosted on Heroku (a cloud platform). To evaluate the solution, eleven participants completed think‑aloud sessions, verbalizing their thoughts while using the chatbot. They then rated usability through brief surveys: the Single Ease Question after tasks and the Chatbot Usability Questionnaire at the end. Qualitative feedback was analyzed using thematic analysis, revealing both positive and negative user experiences. Overall, the chatbot was seen as usable, informative, and friendly, but its “personality” – tone and style – needs further refinement.
Wikipedia arbejder for større mangfoldighed og inklusion og for at gøre viden tilgængelig for alle. Målene frem mod 2030 handler bl.a. om at lukke videnshuller og styrke det fundament, der gør viden mere globalt repræsentativ og tilgængelig. Dette speciale undersøger, hvordan teknologi kan støtte disse mål ved at forbedre oplevelsen for frivillige redaktører på Wikipedia. Vi udvikler og evaluerer en chatbot, der skal hjælpe redaktører med at komme i gang (aktivering) og blive ved med at bidrage (fastholdelse). Chatbotten er designet efter en “atomic design”-tilgang, hvor brugerfladen bygges af små, genbrugelige komponenter. Den er udviklet med IBM Watson Assistant (et værktøj til at bygge samtalerobotter) som en fuldstack-webapplikation i React og Node.js og hostet på Heroku (en cloud-platform). For at evaluere løsningen gennemførte elleve deltagere tænke-højt-tests, hvor de beskrev deres tanker, mens de brugte chatbotten. Derefter vurderede de brugbarheden via korte spørgeskemaer: Single Ease Question efter opgaver og Chatbot Usability Questionnaire efter testen. De kvalitative data blev analyseret tematisk for at identificere mønstre og viste både positive og negative brugeroplevelser. Samlet blev chatbotten vurderet som brugbar, informativ og venlig, men dens “personlighed” – tone og stil – bør forbedres yderligere.
[This apstract has been rewritten with the help of AI based on the project's original abstract]
Keywords