In this thesis a few steps towards augmented cognition in games has been taken. By conducting an experiment where 31 subjects played Super Mario while Their Electroencephalogram(EEG) and Eye-tracking where measured.
Through analysis of the game 22 different events were tracked. 3 second epochs of the psychophysiological signals were divided between the different experienced game events, through which one dataset per subject was created those were inspected for abnormalities, which led to an exclusion of 18 out of 31 subjects and 18 out of 22 game events. Through the use of Independent Component Analysis significant features in the EEG and Eye-tracking data were found and given to the Artificial Neural Network (ANN), which was devised through the Encog framework.
The results from the ANN, measured as accuracy and analyzed through ANOVA test and Kruskal–Wallis ANOVA test, were promising. Showing significant differences between the trained models based on EEG channels in respect to all four game events (means = 57.94% - 67.30%). The eye-tracking data significant differences between its measurements in relation to two of the events were found as well (means = 57.78% -69.48%). Through the grand average of the accuracies in respect to the different events significant differences both in relation to the accumulated EEG accuracies (means = 59.27% - 65.97%) and the accumulated Eye-tracking accuracies were found (means = 59-31% - 66.84%).
Albeit these results are promising they are not as prominent as other electroencephalogram recognition studies. This could be grounded in the very nature of the data, which can be harder to recognize and the structure of the ANN, which could be optimized further.
Conclusively this thesis points towards that it is possible to recognize psychophysiological patterns related to game events through an ANN.