Towards Augmented Cognition in Games : Psychophysiological signals and game events recognized through artificial neural network

Abstract

Gennem denne afhandling få skridt er blevet taget mod augmenterede kognition I relation til computerspil. Et eksperiment var udført hvor 31 personer spillede Super Mario mens deres Electroencephalogram og eye-tracking blev optaget. 22 forskellige spil begivenheder var fundet gennem analyse af spillet. Hver af disse fik relateret 3 sekunders fra hver af de optaget signaler. Hver persons dataset var set igennem for støj og fejl i signalet. Det ledte til en eksklusion af 18 ud af 31 testpersoner og 18 ud af 22 spil begivenheder. Gennem brugen af Independent Component Analysis blev signifikante elementer af EEG og eye-tracking optagelserne fundet og givet til et Artificial Neural Network (ANN), hvilket er lavet ved hjælp af Encog biblioteket. resultaterne fra det neurale netværk var signifikante. De var målt i ” accuracy” og analysere gennem ANOVA og Kruskal–Wallis ANOVA test. Der var signifikant forskel mellem det testede data fra EEG kanalerne i relation til alle spil begivenhederne (middelværdi 57.94% - 67.30%) den samme tendens gør sig gældende i relation to eye-tracking kanalerne, men kun i forhold til to af spil begivenhederne (middelværdi 57.78% -69.48%). Den samlede middel accuracy i forhold til diverse spil begivenheder basseret på EEG kanalerne viste ligeledes signifikant forskelle (middelværdi 59.27% - 65.97%). Den samlede middel accuracy basseret på eye-tracking viste den samme signifikante forskel (middelværdi 59.31% - 66.84%). Trods disse resultater er lovende kan de ikke måle sig med andre electroencephalogram identificerings studier. Det kan være grundet i at dataen kan være svære at genkende sammenlignet med den data studierne bruger, samt strukturen på det neurale netværk. Det er dog hensigten at optimere denne yderligere. For at konkludere denne tese så er det muligt at identificere psykefysiologiske mønstre relateret til spil begivenheder gennem en Artificial Neural Network.

In this thesis a few steps towards augmented cognition in games has been taken. By conducting an experiment where 31 subjects played Super Mario while Their Electroencephalogram(EEG) and Eye-tracking where measured. Through analysis of the game 22 different events were tracked. 3 second epochs of the psychophysiological signals were divided between the different experienced game events, through which one dataset per subject was created those were inspected for abnormalities, which led to an exclusion of 18 out of 31 subjects and 18 out of 22 game events. Through the use of Independent Component Analysis significant features in the EEG and Eye-tracking data were found and given to the Artificial Neural Network (ANN), which was devised through the Encog framework. The results from the ANN, measured as accuracy and analyzed through ANOVA test and Kruskal–Wallis ANOVA test, were promising. Showing significant differences between the trained models based on EEG channels in respect to all four game events (means = 57.94% - 67.30%). The eye-tracking data significant differences between its measurements in relation to two of the events were found as well (means = 57.78% -69.48%). Through the grand average of the accuracies in respect to the different events significant differences both in relation to the accumulated EEG accuracies (means = 59.27% - 65.97%) and the accumulated Eye-tracking accuracies were found (means = 59-31% - 66.84%). Albeit these results are promising they are not as prominent as other electroencephalogram recognition studies. This could be grounded in the very nature of the data, which can be harder to recognize and the structure of the ANN, which could be optimized further. Conclusively this thesis points towards that it is possible to recognize psychophysiological patterns related to game events through an ANN.

A master thesis from Aalborg University

Towards Augmented Cognition in Games : Psychophysiological signals and game events recognized through artificial neural network

[Mod Augmenterede Kognition i Computerspil: Psykefysiologiske signaler og spil begivenheder genkend gennem artificial neural network]