Retour à la programmation
mercredi 24 mai  |  de 11:15 à 12:00

League Framework: A Reinforcement Learning Paradigm For Creating PvP Bots

The league framework is a game-agnostic system to automatically build bots for PvP games using league training and Reinforcement Learning (RL). The intuition behind league training is using RL to generate new bot behaviors by having bots fight each other to progressively improve. There are several benefits to the approach which we will discuss in the presentation. The first benefit is that bots generated by the league training framework are less exploitable than classical bots (BTs) since they’ve been exposed to many different adversaries. The system also allows for easy and quick scalability to many different NPC archetypes (classes, abilities, behaviors), which will become more and more useful as our games get more and more complex. Another benefit is in the context of live-service games, where we can quickly change the behavior of a certain character (or class) after balancing patches or quickly generate a behavior from scratch for a new character.

We will go over why For Honor is a good candidate for the league framework and what the production can gain from it. With our implementation, a new bot behavior could be created automatically for any character in the game in a matter of hours. Finally, we will present our findings from a playtest of these new bots.

Jeux vidéo
Conférence

Philippe MarcotteProgrammeur R&DUbisoft

Philippe a fait une maîtrise professionnelle en apprentissage automatique au Mila et un baccalauréat en génie logiciel à la Polytechnique de Montréal. Depuis, il travaille chez Ubisoft comme programmeur R&D à La Forge. Plus spécifiquement, il travaille dans l'équipe d'Apprentissage par Renforcement (RL). Il aide les scientifiques de La Forge à industrialiser les projets de recherche pour différentes productions et ouvrir la voie pour combiner RL et jeux vidéo.

Tarik AzzouniGameplay programmerUbisoft

With a passion for games, Tarik studied Computer Science with a specialization in computer games at Concordia University, focusing on gameplay and AI. At Ubisoft, Tarik started as Generalist Programmer on the Stadia port of Assassin’s Creed Origins, responsible for its online systems. Tarik currently works as a Gameplay Programmer on For Honor where he develops new heroes, the fight system, 3Cs, along with the Machine Learning mandate.