VM-Based Obfuscation for eBPF
Authors
Merstrand, Thormod Jakob Schøn ; Paludan, Andreas Hove ; Mariager, Mathias Holm
Term
4. term
Education
Publication year
2026
Submitted on
2026-06-11
Abstract
This thesis investigates how to make video game anti-cheat systems more secure by using eBPF on Linux as an alternative to traditional kernel module–based solutions. Kernel modules offer powerful capabilities for detecting advanced cheats but can introduce serious security vulnerabilities for users. eBPF provides a more controlled and verified way to run programs in kernel space, yet earlier work has shown that eBPF-based anti-cheat systems, including the authors’ own system TyrSecure, are vulnerable to reverse engineering. To address this, the thesis develops a virtualization-based obfuscation approach in which the core anti-cheat logic is executed as encrypted bytecode inside a custom bytecode interpreter implemented in eBPF. The design combines a tailored instruction set architecture with general obfuscation techniques and eBPF-specific measures, including decoy hooks, junk code, and XOR encryption. The implementation is evaluated through white-box tests comparing native eBPF programs with their obfuscated counterparts, showing that the virtualized programs run 8 to 21 times slower. However, black-box tests conducted in a realistic gaming environment indicate that the impact on game performance is negligible for both native and obfuscated programs. The thesis further discusses the achieved obfuscation quality, the broader applicability beyond anti-cheat scenarios, and challenges related to eBPF licensing requirements. It concludes that virtualization-based obfuscation is practically applicable to eBPF programs and can potentially strengthen eBPF-based anti-cheat solutions such as TyrSecure, but that more work on licensing, developer tooling, and improved obfuscation is needed before the system is ready for deployment.
Dette speciale undersøger, hvordan man kan gøre anti-cheat systemer til videospil mere sikre ved at bruge eBPF på Linux som et alternativ til traditionelle kernelmodul-baserede løsninger. Kernelmoduler giver stærke muligheder for at opdage avancerede cheats, men kan samtidig introducere alvorlige sikkerhedssårbarheder for brugeren. eBPF tilbyder en mere kontrolleret og verificeret måde at køre programmer i kernerummet på, men tidligere arbejde har vist, at eBPF-baserede anti-cheat systemer, som forfatternes eget system TyrSecure, er sårbare over for reverse engineering. For at imødegå dette udvikles en virtualiseringsbaseret obfuskering, hvor kernen af anti-cheat logikken afvikles som krypteret bytekode i en specialdesignet bytecode-fortolker implementeret i eBPF. Designet kombinerer en skræddersyet instruktionssæt-arkitektur med generelle obfuskeringsteknikker og eBPF-specifikke tiltag, herunder bl.a. decoy hooks, junk code og XOR-kryptering. Implementeringen evalueres ved white-box tests, der sammenligner oprindelige eBPF-programmer med deres obfuskerede modstykker, og viser en ydelsesnedgang på 8 til 21 gange for de virtualiserede programmer. Black-box tests, hvor systemet køres sammen med faktiske spil, viser dog kun marginal indflydelse på spillets ydeevne for både native og obfuskerede programmer. Specialet diskuterer derudover kvaliteten af den opnåede obfuskering, generel anvendelighed ud over anti-cheat scenarier samt udfordringer med eBPF’s licenskrav. Konklusionen er, at virtualiseringsbaseret obfuskering kan anvendes praktisk på eBPF-programmer og dermed potentielt styrke eBPF-baserede anti-cheat løsninger som TyrSecure, men at yderligere arbejde med licensafklaring, udviklerværktøjer og forbedret obfuskering er nødvendigt før en egentlig udrulning.
[This abstract has been generated with the help of AI directly from the project full text]
Keywords