A master thesis from Aalborg University
Multimodal Looper: Interactive Visual Music through Gestures
Author(s)
Term
4. term
Education
Publication year
2023
Submitted on
2023-09-04
Pages
37 pages
Abstract
This thesis presents the first prototype of a multimodal looper, an embodied interface that enables musical improvisation through connecting body movements to audiovisual forms. With the aim of building an intuitive system, the initial research focused on music cognition through an embodied paradigm, where sound perception is integrated with visual and kinesthetic modalities; as well as widely observed crossmodal correspondences. This led to the development of a live-looping system with multimodal objects: sound fragments that are gesturally activated and visually represented ac- cording to their dynamics, motion trajectories, pitch and texture. The design and implementation focused on the development of the various modules of the live-looping system. The evaluation aimed to determine if the system effectively established crossmodal correspondences for an integrated multi- sensory experience through split testing method, where the participants experimented with a variation of the system with randomized crossmodal mappings along with the original version. Although the results revealed no significant difference between the two conditions, as well as a perplexing user experience, the qualitative findings proved to be quite valuable for future iterations.
Documents
Colophon: This page is part of the AAU Student Projects portal, which is run by Aalborg University. Here, you can find and download publicly available bachelor's theses and master's projects from across the university dating from 2008 onwards. Student projects from before 2008 are available in printed form at Aalborg University Library.
If you have any questions about AAU Student Projects or the research registration, dissemination and analysis at Aalborg University, please feel free to contact the VBN team. You can also find more information in the AAU Student Projects FAQs.