A Dual Edged Sword for Advancing Education for the youth with Double Diagnoses

Virtual Reality and Artificial Intelligence

Mohamoud, Nasir

4. term

Medialogy, Master

2023

2023-12-20

Unge med dobbeltdiagnose (to samtidige diagnoser) møder mange barrierer i almindelige læringsmiljøer. En lærer fra en døgnskole for teenagere med dobbeltdiagnose beskrev kognitive vanskeligheder, lavt fremmøde, lav motivation, psykiske udfordringer og eksamensangst som centrale hindringer. Samtidig fremhævede læreren, at det er vigtigt, at de unge har et formål og noget at sigte efter, når de afslutter et behandlingsforløb. Dette speciale undersøger, om et immersivt læringsmiljø, der kombinerer Virtual Reality (VR) og Artificial Intelligence (AI), kan støtte disse unge i engelsk og matematik, motivere dem, opbygge viden og forberede dem til nationale prøver. Der blev udviklet et virtuelt klasseværelse i Unity for at skabe en genkendelig og tryg ramme. AI‑delen skulle efterligne lærer‑lignende kommunikation, især for at støtte engelsklæring, ved hjælp af tekst‑til‑tale via Mimic3. Specialet beskriver designet af forsøgsopsætningen, baggrundstankerne og de tekniske komponenter, herunder integrationen af Mimic3. Før disse detaljer præsenteres, gennemgås hardware, software, scripts og miljø. Testen fulgte den samme struktur hver dag i en uge. Deltagerne brugte et VR Valve Index‑headset og to controllere til at interagere med opgaverne i klasselokalet, der var skabt med software som Unity, Blender, Fuse, Max 3D og Photoshop. Der blev truffet en vigtig designbeslutning om at inkludere en nødknap, og brugernes trivsel havde høj prioritet, så de til enhver tid kunne afbryde oplevelsen, hvis de blev utilpasse. Udgangspunktet var 21 unge, men engagementet svingede, og kun fire gennemførte alle trin i forsøget. I VR‑miljøet løste de tre matematikopgaver: et trekant‑puslespil til at træne rumlig tænkning og geometri, et spil med at forme kvadrater for at styrke visuo‑spatiale evner og geometri samt et ligningspuslespil, der trænede problemløsning og talforståelse. Læringsudbyttet blev vurderet med før‑ og eftertests, der var afstemt med VR‑indholdet, samt et spørgeskema. Til den statistiske analyse blev der anvendt JASP med Bayesianske t‑tests, Spearmans rangkorrelationskoefficient og Wilcoxon signed‑rank test. Resultaterne viste en statistisk signifikant forskel mellem før‑ og eftertest og i både matematik og engelsk (p = 0,049 < 0,05), hvilket understøtter specialets alternative hypoteser. Efter fem dages brug af VR‑forløbet havde de unge højere scores. Da kun få gennemførte hele forløbet, bør resultaterne betragtes som indledende, men de peger på, at VR og AI kan være lovende redskaber for denne målgruppe.

Young people with a dual diagnosis (two co‑occurring conditions) face many barriers in mainstream learning settings. A teacher from a residential school for teenagers with dual diagnoses highlighted key obstacles such as cognitive difficulties, low attendance, low motivation, mental health challenges, and exam anxiety. The teacher also stressed the importance of having a sense of purpose and clear goals after completing therapy. This thesis explores whether an immersive learning environment that combines Virtual Reality (VR) and Artificial Intelligence (AI) can support these youths in English and math, motivate them, build knowledge, and prepare them for national examinations. A virtual classroom was developed in Unity to create a familiar and comfortable setting. The AI component was designed to simulate teacher‑like communication, particularly to support English learning, using text‑to‑speech via Mimic3. The thesis explains the design of the experimental setup, the rationale behind it, and the technical components, including the integration of Mimic3. Before those details, the hardware, software, scripts, and environment are reviewed. Testing followed the same structure each day for a week. Participants used a VR Valve Index headset and two controllers to engage with classroom tasks built with software such as Unity, Blender, Fuse, Max 3D, and Photoshop. A key design choice was to include an emergency button, and user well‑being was prioritized so participants could exit the experience at any time if distressed. The initial sample included 21 youths, engagement varied, and only four completed all steps of the experiment. Within the VR environment, they completed three math activities: a triangle puzzle to practice spatial thinking and geometry, a square‑shaping game to strengthen visual‑spatial abilities and geometry, and an equations puzzle to support problem‑solving and number understanding. Learning was evaluated with pre‑ and post‑tests aligned with the VR content and a questionnaire. Statistical analyses were conducted in JASP using Bayesian t‑tests, Spearman’s rank correlation coefficient, and the Wilcoxon signed‑rank test. Results showed a statistically significant difference between pre‑ and post‑test scores and in both math and English (p = 0.049 < 0.05), supporting the study’s alternative hypotheses. After five days with the VR experience, participants had higher scores. Because only a few completed the full protocol, these findings should be seen as initial, but they indicate that VR and AI may be promising tools for this group.

[This summary has been rewritten with the help of AI based on the project's original abstract]

Download
View record in AAU Student Projects