Multimodal Looper: Interactive Visual Music through Gestures
Author
Term
4. term
Education
Publication year
2023
Submitted on
2023-09-04
Pages
37
Abstract
This thesis presents the first prototype of a multimodal looper, an embodied interface that enables musical improvisation through connecting body movements to audiovisual forms. With the aim of building an intuitive system, the initial research focused on music cognition through an embodied paradigm, where sound perception is integrated with visual and kinesthetic modalities; as well as widely observed crossmodal correspondences. This led to the development of a live-looping system with multimodal objects: sound fragments that are gesturally activated and visually represented ac- cording to their dynamics, motion trajectories, pitch and texture. The design and implementation focused on the development of the various modules of the live-looping system. The evaluation aimed to determine if the system effectively established crossmodal correspondences for an integrated multi- sensory experience through split testing method, where the participants experimented with a variation of the system with randomized crossmodal mappings along with the original version. Although the results revealed no significant difference between the two conditions, as well as a perplexing user experience, the qualitative findings proved to be quite valuable for future iterations.
Documents