Adversarial-learning on shallow-neural-networks

Moreaux, Marc Maurice Roland

4. term

Machine Intelligence, Master

2015

2015-06-07

57

Denne masterafhandling undersøger, om adversarial læring kan forbedre ydeevne og robusthed i lavdybe (shallow) neurale netværk sammenlignet med klassiske træningsmetoder. Med udgangspunkt i Goodfellow m.fl.’s metode til at generere og udnytte adversarielle eksempler reimplementeres tilgangen med fokus på at isolere effekten af selve adversarial læring. Arbejdet dækker en kortfattet gennemgang af neurale netværk, fremad- og bagudpropagation samt gradientnedstigning, og bruger et simpelt løbende eksempel til at motivere intuitionen. Eksperimentelt planlægges systematiske sammenligninger mellem klassisk og adversarial træning på ens arkitekturer, herunder variation af antal skjulte neuroner, størrelsen af perturbationer (epsilon), sammenligning med træning på støjende data og visualisering af lærte vægte. Undersøgelserne er tilrettelagt på tværs af tre datasæt: MNIST og CIFAR-10 (10-klasses billedklassifikation) samt UCI Covertype (tabulære data), for at vurdere om eventuelle fordele rækker ud over billeddomænet. Adversarielle eksempler forstås her som små, målrettede ændringer af input, der kan få en ellers præcis klassifikator til at fejle; målet er at lære modeller, der er mindre følsomme over for sådanne perturbationer og bedre generaliserer. Konkrete resultater og kvantitative sammenligninger præsenteres i senere kapitler og fremgår ikke af det medfølgende uddrag.

This master’s thesis investigates whether adversarial training can improve performance and robustness in shallow neural networks compared with conventional training. Building on the method of Goodfellow et al. for generating and exploiting adversarial examples, the approach is re-implemented to isolate the specific effect of adversarial learning. The thesis reviews core neural network concepts—forward and backpropagation and gradient descent—using a simple running example to build intuition. Experimentally, it proposes controlled comparisons between standard and adversarially trained models on identical architectures while varying the number of hidden units, the perturbation magnitude (epsilon), contrasting with training on noisy data, and visualizing learned weights. The evaluation is designed across three datasets—MNIST and CIFAR-10 (10-class image classification) and the UCI Covertype tabular dataset—to test whether any benefits extend beyond image tasks. Here, adversarial examples are small, targeted input perturbations that can cause misclassification; the aim is to train models that are less sensitive to such perturbations and that generalize better. Detailed results and quantitative findings are presented in later chapters and are not included in this excerpt.

[This summary has been generated with the help of AI directly from the project (PDF)]

adversarial-learning ; neural-networks

Download
View record in AAU Student Projects

Moreaux, Marc Maurice Roland:

• Text classification with generic Logistic-Regression classifier (2015)