The Ghosts of the Music Museum: A Markerless Augmented Reality Experience
Authors
Jakobsen, Jan Place ; Skjønnemann, Jonas
Term
4. term
Education
Publication year
2022
Submitted on
2022-05-25
Abstract
Dette speciale præsenterer den iterative designproces og evaluering af en markørløs augmented reality (AR) oplevelse til Dansk Musikmuseum. Målet var at udvikle en mobilapplikation, der kombinerer virtuelle performere med de udstillede instrumenter for at øge gæsternes forståelse og engagement. Systemet blev bygget i Unity med Vuforia Area Targets baseret på LiDAR-scanninger, som muliggør markørløs sporing og occlusion. Digitale musikere blev modelleret og animeret med kropsbevægelse optaget i en Rokoko Smart Suit samt ansigtsfangst via ARKit-blendshapes; skygger og dybdebaseret occlusion blev tilføjet for bedre rumlig integration. Lyd stammede fra museets bibliotek, nye optagelser af rekonstruktionsinstrumenter og udvalgte arkivvideoer. Prototypen blev udformet som en skattejagt, hvor besøgende finder performere placeret i udstillingerne. Der blev gennemført en blanding af kvalitative og kvantitative tests for at vurdere brugeroplevelse og sporingsstabilitet. En indledende gruppetest (fem deltagere) beskrev oplevelsen som intuitiv og underholdende og pegede på et ønske om længere musiksekvenser. Samlet peger resultaterne på, at applikationen kan fungere som en udstillingsforstærkning, mens kombinationen af flere Area Targets gav ustabil sporing, som kræver yderligere tekniske forbedringer.
This thesis presents the iterative design and evaluation of a markerless augmented reality (AR) experience for the Danish Music Museum. The aim was to develop a mobile application that combines virtual performers with real instruments to enrich visitors’ understanding and engagement. The system was built in Unity using Vuforia Area Targets derived from LiDAR scans to enable markerless tracking and occlusion. Digital musicians were modeled and animated from body motion captured with a Rokoko Smart Suit and facial capture using ARKit blendshapes; cast shadows and depth-based occlusion techniques were added to improve spatial integration. Sound was sourced from the museum’s library, new recordings of replica instruments, and selected archival videos. The prototype took the form of a treasure hunt in which visitors locate performers distributed across galleries. A mix of qualitative and quantitative tests was conducted to assess user experience and tracking stability. An initial group test (five participants) reported the experience as intuitive and entertaining and suggested longer performance sequences. Overall, the findings indicate that the application has potential as an exhibition enhancement, while combining multiple Area Targets produced unstable tracking that warrants further technical refinement.
[This summary has been generated with the help of AI directly from the project (PDF)]
Keywords
Documents