Cooperation Among Autonomous Adaptive Agents II
Authors
Rydtoft, Michael Vengø ; Rasmussen, Henrik Skriver
Term
4. term
Education
Publication year
2003
Abstract
Denne afhandling undersøger, hvordan man kan udvikle samarbejdende adfærd hos autonome agenter med Strongly Typed Genetic Programming, en evolutionær metode der avler computerprogrammer under strenge typeregler. Vi arbejder med konkurrerende, flerobjektiv koevolution og bidrager med tre praktiske redskaber, der gør processen mere driftssikker: en mekanisme til at bevare diversitet og undgå for tidlig konvergens, en ændring af Competitive Fitness Sharing for at dæmpe støj i evalueringerne, og et objektivt mål til at følge fremskridt over tid. Vi afprøver metoderne i et udvidet “pursuit”-miljø—en rovdyr/bytte-simulation—med forskellige niveauer af kommunikation mellem rovdyrene. Vores første spørgsmål er, om samarbejde hjælper: I vores forsøg udvikler rovdyr, der kan kommunikere, både stærkere og mere robuste strategier mod svært bytte end rovdyr uden kommunikation. Vores andet spørgsmål er, hvordan man undgår afkobling (disengagement), en fejlsituation i konkurrerende koevolution hvor modstandere holder op med at udfordre hinanden meningsfuldt. Vi foreslår den indflettede tilgang (Interleaved Approach) og sammenligner den med metoder fra litteraturen, almindelig koevolution og enkeltpopulationsevolution. I dette miljø klarer den indflettede tilgang sig bedst. Samlet peger resultaterne på, at kommunikation kan forbedre udviklet teamwork, og at den indflettede tilgang hjælper med at fastholde produktiv konkurrence; vores tiltag til diversitet, støjreduktion og fremskridtsmåling understøtter stabile, gennemsigtige træningsforløb.
This thesis explores how to evolve cooperative behavior in autonomous agents using Strongly Typed Genetic Programming, an evolutionary method that breeds computer programs under strict type rules. We focus on competitive, multi-objective co-evolution and contribute three practical tools to make it more reliable: a diversity-maintenance mechanism to prevent premature convergence (getting stuck too early), a modification of Competitive Fitness Sharing to reduce evaluation noise, and an objective metric for tracking evolutionary progress over time. We evaluate these ideas in an extended pursuit environment—a predator–prey simulation—with different levels of communication between predators. Our first question is whether cooperation helps: in our experiments, predators that can communicate evolve strategies that are both stronger and more robust against difficult prey than predators that cannot communicate. Our second question is how to avoid disengagement, a failure mode in competitive co-evolution where opponents stop providing useful challenges. We propose the Interleaved Approach to prevent disengagement and compare it with methods from the literature, standard co-evolution, and single-population evolution. In this environment, the Interleaved Approach performs best. Together, these results indicate that communication can substantially improve evolved teamwork, and that the Interleaved Approach helps sustain productive competition; our diversity control, noise reduction, and progress metric support stable, interpretable training runs.
[This abstract was generated with the help of AI]
Documents