Generative Lighting Design with Synesthesia

Tyroll, Simon Borst

4. Term

Lighting Design

2020

2020-09-28

89 pages

Et system til audiovisuelle oplevelser er udviklet med inspiration fra den menneskelige neurofysiologiske tilstand synestesi. Forskning tyder på, at synestesi kan være resultatet af en neural kortslutning mellem sensoriske systemer i hjernen, hvilket replikeres ved kortslutning af to kunstige sensoriske systemer. Det efterlignes ved brug af kunstige neurale netværk (ANN) til at replikere den menneskelige hjernes sensoriske systemer. En autoencoder (AE) bruges til at genkende strukturer i musik, mens et kompositionsmønsterproducerende netværk (CPPN) bruges til at generere visuelle kompositioner. Den uviklede AE finder mønstre i musikken der bruges til at skubbe til den uviklede CPPN hvilket skaber bevægelser i de mønstre den producerer. I et eksperiment med 30 deltagere blev systemet sammenlignet med et spektrogram og en CPPN, der producerede bevægelser baseret på Perlin-støj. Resultaterne viser, at det udviklede system adskiller sig selv i værste tilfælde og har potentiale til at blive brugt som et generativt designværktøj til dynamisk lysdesign. Et lysdesign blev lavet ved hjælp af systemet til at vise nogle forskellige kompositioner af lys og musik. Yderligere udvikling er oplagt for at frigøre systemets fulde potentiale ved hjælp af en cybernetisk tilgang til generativt design.

A system for audiovisual experiences is developed with inspiration from the human neurophysiological condition of synesthesia. Research suggests that synesthesia may be the result of a neural short circuit between the sensory systems in the brain, which is replicated by short circuiting two artificial sensory systems. It is emulated by the use of artificial neural networks (ANN) to replicate the sensory systems of the human brain. An autoencoder (AE) is used to recognise patterns in music, while a compositional pattern producing network (CPPN) is used to generate visual compositions. The AE extracts features from music and used to push the CPPN, which creates movements in the patterns it produces. In an experiment with 30 participants, the system was compared against a spectrogram and a CPPN producing movements based on Perlin noise. Results show that the developed system differentiates itself even in worst case conditions, and has potential for use as a generative design tool for dynamic lighting design. A lighting design was made using the system to show some different compositions of light and music. Further development is imperative to release the full potential of the system, by using a cybernetic approach to generative design.

Synesthesia ; Machine Learning ; Artificial Intelligence ; Artificial Cognition ; Generative Art ; Generative Design ; Augmented Design ; Feature Extraction ; CPPN ; Autoencoder ; Lighting Design ; Cybernetics ; NEAT ; HyperNEAT ; Genetic Algorithm ; Neuroevolution ; Evolutionary Algorithms ; Neural Networks ; Cross Modality ; Cross Modal Perception ; Perception ; Cross Modal Enhancement ; Neural Short Circuit

Download
View record in AAU Student Projects