A master thesis from Aalborg University
Automatic sonification of video sequences through object detection and physical modelling
Author(s)
Term
4. Term
Education
Publication year
2017
Submitted on
2017-06-16
Abstract
The sounds generated by the objects that surround us is intrinsic to our life. We associate some specific sounds to the objects, its characteristics, and the actions that generate them, and we expect to hear the corresponding sound when we see that item. Similarly, one expect to see the corresponding object when we hear its sound. For example, on the street, when we hear a characteristic sound of a motor, we know that a car is approaching. There is, therefore, a particular relationship between the objects and the sound that they produce. In films, many of these sound effects are added in post-production, a method called "Foley". In this project, these sound effects will be generated automatically based mainly on one characteristic of the objects involved: their material. A system based on an object detector, an impact detector and a sound modeler will be presented. A perceptual evaluation in which the subjects will watch some videos and listen to the sounds predicted by the model will be performed. In this way, it will be tested if the sounds proposed by the model are played in accordance of what they expect to hear.
Documents
Colophon: This page is part of the AAU Student Projects portal, which is run by Aalborg University. Here, you can find and download publicly available bachelor's theses and master's projects from across the university dating from 2008 onwards. Student projects from before 2008 are available in printed form at Aalborg University Library.
If you have any questions about AAU Student Projects or the research registration, dissemination and analysis at Aalborg University, please feel free to contact the VBN team. You can also find more information in the AAU Student Projects FAQs.