Using statistics and game knowledge to create better bots in Dota 2 - Sebastian Viro Malmö University
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Using statistics and game knowledge
to create better bots in Dota 2
Sebastian Viro
Malmö University
Bachelor Thesis in Computer Science, 15 credits
Department of Computer Science
System development
2021-06-01
Supervisor: José Font
Examiner: Reza Malekian
Abstract
Multiplayer online battle arena (MOBA) games and games in general offer developers
opportunities to design and test AI to further push research within the area. MOBA is a genre
that has seen an increase in popularity during recent years alongside a rapidly growing esports
scene.
The game's main purpose is ultimately to be enjoyed and played by human players, who often
come up with and are encouraged to create their own strategies in order to beat the game or
win out over their opponents.
This thesis will explore how mimicking human behavior in AI and using statistics from human
players can be beneficial to AI design. The AI will be developed in a framework similar to that
used by the Conference of Games for their Dota 2 5v5 AI competition.
Three different strategies will be created, each of which will mimic some aspect of human
behaviour, and then tested against the built-in AI in Dota 2. All the developed AI’s managed to
win with varying results. The results showed some support for the notion that human gameplay
and statistics can be beneficial to AI design. There are, however, difficulties with the complexity
of the game that is Dota 2.
Keywords: Dota 2, MOBA, AI, strategy, bots, games.
1
Contents
1 Introduction 3
1.1 Related work 4
1.2 Goal 5
2 Dota 2: Overview 6
2.1 Concepts in a game of Dota 2 8
3 Framework 9
3.1 Methodologies 10
4 Contributions 11
4.1 Strategy 1: Simple heroes 11
4.1.1 Hero introduction 11
4.1.2 Concept 1: Active Attack Modifiers 12
4.1.3 Concept 2: Using spells to land kills 12
4.1.4 Concept 3: Using spells to farm 13
4.2 Strategy 2: Jungle 13
4.3 Strategy 3: Io - Luna 14
5 Results 15
5.1 Strategy 1: Simple heroes 17
5.2 Strategy 2: Jungle 18
5.3 Strategy 3: Io - Luna 20
6 Limitations 21
6.1 Limitations from the game itself 21
6.2 Limitations with the framework 21
7 Discussion and Analysis 22
8 Conclusion and future work 24
References 26
Appendix 28
2
1 Introduction
In computer science the goal of AI is enabling computers to act and make decisions intelligently
[1]. Using games to help develop AI is not a new concept and can function as an ideal testbed
for AI research [2]. Some historic breakthroughs where AI has proven to be better than their
human counterparts include Kasparov’s defeat versus Deep Blue in a game of chess and [3]
AlphaGo beating Lee Sedol in Go [4][5]. In the world of Dota 2, OpenAI Five used reinforcement
learning techniques to develop their AI [6]. In 2019, they managed to beat the reigning world
champions in a series 2-0. They also let the community engage with the AI in competitive play,
with OpenAI Five winning 99.4% of over 7000 games.
MOBA (Multiplayer Online Battle Arena) is a genre that has seen an increase in popularity over
the last years [7], with games including Dota 2 and League of Legends. Dota 2 is developed and
published by Valve and is played through Steam, which is a platform to access games. The
game of Dota 2 has a vibrant scene with about 600 thousand unique players each day and is
one of the platform’s most played games [8]. There's also a lot of money involved in Dota’s
esports scene, mainly the annually held tournament known as The International. The last one
held in 2019 reached a prize pool of over $34 million [9]. This rise in popularity, as well as the
money involved in the professional gaming scene, serves as further motivation for the thesis. A
more detailed explanation of Dota and certain mechanics within the game is presented later in
the paper (Section 2).
The success of both AlphaGo and OpenAI show that AI can achieve superhuman performance
when using machine learning and reinforcement learning techniques. There is, however, a gap
when it comes to directly designing bots with pure inspiration from human players. Further
presentation and discussion regarding research done in the area of MOBA games and AI can
be found in the Related work (Section 1.1).
The purpose of games is to be played and enjoyed by human players, who will come up with
their own ideas and strategies on how to succeed in games. In this thesis, the goal is to develop
handcrafted bots that convey and mimic strategies used by human players, with the goal of
defeating the built-in AI. The purpose of this study is therefore to answer 2 research questions.
1. How can using gameplay from real players be beneficial to AI design?
2. How can using statistics from real players be beneficial to AI design?
In order to answer these questions, work will be conducted in a framework which was
developed by students at Malmö University [10]. This framework enables developers to design
their own bots and test them against Dotas own implemented bots in a real game of Dota 2.
This framework is named the Dota 2 5v5 framework (D25v5). This year CoG (Conference of
Games) will run the Dota 2 5v5 AI competition using the D25v5 framework [11][12]. The goal of
this competition will be to create a bot that beats the enemy in the shortest time possible.
Created bots will face off against the built-in AI. The AI developed in this thesis will also serve as
a baseline for the competition. Collecting data and acquiring a better understanding of how
3
humans play Dota 2 comes from both observing live games and watching replays of
professional Dota 2 matches. Third-party sites that collect statistics and information about
matches are also used to look at statistics to see how heroes (avatars that players control) had
performed at the end of a match in regards to different criterias.
In Section 2, Dota 2 will be described in further details, providing both the basics of the game
and more advanced concepts that are used in the design of the different strategies. In the
following Section 3, the framework is briefly introduced. In Section 4, the different strategies will
be presented and the rationale behind the chosen heroes explained. Section 5 will cover the
results of the strategies when faced off against the built in AI. In Section 6, limitations that arose
during the work process will be presented. Section 7 will analyze the result and discuss whether
using statistics and gameplay from real players is beneficial to AI design. Finally, Section 8
covers the final answers to the research questions as well as conclusion and possible future
work.
1.1 Related work
Different work has been conducted regarding certain elements of MOBA games, both when it
comes to AI research and other more specific elements of MOBA style games.
Regarding the research around bots in Dota 2 and other MOBA games, several previous works
have been conducted. The work in OpenAI Five was recently in the spotlight. They used
reinforcement learning techniques to develop bots that managed to beat professional players
[6]. This type of AI development falls under the machine learning category, which means it
improves by repeatedly playing against itself. In 2019, they managed to beat the reigning world
champions in a series 2-0. They also let the community engage with the AI in competitive play,
with OpenAI five winning 99.4% of over 7000 games.
Zhengxing Chen et al. [13] researched the drafting aspect, how the selection of heroes before
the game even starts can have a positive impact on a team’s chance to win a game. They
argued that this process of drafting is a challenging problem as there is a complex hero-to-hero
relationship to consider as some heroes play better against and with certain heroes. The
algorithm they developed by using Monte Carlo Tree Search (MCTS) proved to be more
successful than other types of drafting algorithms currently available. MCTS is a class of
heuristic search algorithms that can help identify almost optimal actions without having to
construct a complete search tree [13]. This study proves that team composition influences the
outcome of games. One limitation mentioned with their framework is that it lacks player specific
information, as some human players might be more proficient with certain heroes.
Michael Waltham and Deshen Moodley tested and analysed different Artificial Intelligence
techniques in MOBA games [14]. In their literature review, they present different techniques with
their advantages and disadvantages and finally decide on two techniques to review and
compare. These were “the Fuzzy state machine” (FuSM) and “Emotional behaviour tree” (EBT).
The FuSM incorporates something called “Fuzzy Logic”, which is an extension of conventional
boolean logic and accounts for truth values. One of the advantages is that it allows developers
4
to work with incomplete or partial data. The EBT tries to add emotions into their Non-Player
Characters (NPC’s) decision making and this is done by extending behaviour trees. In their
results it was clear that the FuSM was more effective in the game itself by managing to achieve
more objectives in the game. The “Emotional behaviour tree”, however, proved to be a more
“fun” opponent by its human opposition. In their discussion, they mentioned how success can be
measured when it comes to AI in games, if effectiveness or enjoyment by the human agents that
interact with the AI are more desirable. This shows that a variety of techniques can be used to
develop AI with different outcomes to both player enjoyment and effectivity.
One study performed by V. Araujo et. al. looked deeper into how they could supply better item
recommendations for players in League of Legends, which is another MOBA game, by using
data mining [15]. Data mining is the process of uncovering patterns and valuable information
from large data sets, transforming raw data into useful knowledge [16]. In their discussion, they
claim that the recommendation system in MOBA games has relevant challenges to face as the
importance of selecting proper items depends on the context of the game. For future work they
would like to have their recommended items evaluated and analysed by an expert with previous
knowledge about the game to have their results verified. This way they could make sure that
their recommendations are reliable and that they include context settings.
The related work around MOBAs has shown that there is a lot of interest when it comes to
different aspects of MOBA games. Work has been done when it comes to AI, however, the
techniques used usually fall under the machine learning category. As these techniques improve
and evolve when playing versus themselves there is a gap when it comes to understanding how
human skills and human game knowledge can be applied when creating new AI with the help of
a framework. The data collected from human players' gameplay was used both for the purpose
of item recommendations and drafting heroes for a better team composition. This begs the
question if it is possible to create a new AI by observing and analysing that same data. This gap
in knowledge, as well as the rising popularity of MOBA games, serves as motivation for the
relevance of this thesis.
1.2 Goal
Using the D25v5 framework the goal is to develop bots that beat the game’s own built-in bots in
a standard five versus five match. The developed bots’ addition to the framework will serve as
something future developers can compare with as a baseline. They will also serve as baselines
for the previously mentioned CoG competition. For instance, comparing which bot is fastest and
which has the most kills. The framework in itself can be further improved upon and feedback on
the framework will be thoroughly documented for future work.
The purpose of this thesis is to develop tailored bots that convey strategies that are used by
human players with the goal of beating the already built-in AI and to serve as a baseline for the
competition.
There are many sources with statistics regarding Dota 2 supplied by third parties, with
information regarding heroes, items and replays. How this data will be transformed into
5
information to be used in the design will be covered more in-depth later in the thesis. From this
information, bots will be designed and tested live against the AI. The goal will be to prove that
using statistics from real players is beneficial to AI design.
2 Dota 2: Overview
Dota 2 is a game in the MOBA (Multiplayer Online Battle Arena) genre that is developed and
published by Valve. The game has its roots as an old mod in Blizzard's game Warcraft 3. Dota 2
was released as a standalone game in July 2013 [17]. In a standard game of Dota 2, there are
two teams (Radiant and Dire) each consisting of five players engaging each other with the goal
of destroying a major building in the opponent's base, named “The Ancient''. Each player has
control of their own unique hero with a unique set of skills.
Heroes are the avatars that players control in a game of Dota 2, with each unique player
controlling one unique hero out of the currently 121 available. In order to grow in power, heroes
collect both gold and experience. Gold is the currency within the game and is used to buy items,
which are objects that enhance a hero's attributes or have unique active effects. These items
are stored in their inventory. After a hero has collected enough experience, they level up.
Heroes start at level 1 and can progress all the way to level 30. Heroes have one of the three
primary attributes, Strength, Agility and Intelligence. Each point that gets added to a hero’s
primary attribute also increases their attack damage. A hero also has health and mana, where
health represents the life force of a unit and if it reaches 0, the unit dies. Mana is primarily used
as a resource to cast abilities. Figure 1 shows the User Interface for a hero.
Figure 1, The hero Viper. Health Bar (green) and Mana bar (blue) in the middle. Above are his
abilities, where one can be selected at each level. The square in the right of the image is the
hero’s inventory. Screenshot taken from Dota 2.
Each hero has a selection of different abilities which are unique to that hero. These abilities
range from simple passive effects to powerful damaging spells. Most heroes have four abilities,
where three are more basic and the fourth is an ultimate ability. Some abilities cost mana to use
and some abilities have a cooldown, limiting how often a hero can use certain abilities.
Dota is played in real time with a map composed of 3 different lanes, Bottom, Mid and Top.
Figure 2 shows the map and location for each lane. A lane can be considered a path that
connects both bases.
6
Figure 2. Overview of the Dota 2 map, showing each lane. The color green represents the
Radiant team and the color red represents the Dire team. Screenshot taken from Dota 2.
Each lane is protected by two outer towers which also serve as rallying points on the map where
allies can teleport to in order to assist each other. In order to reach the enemy’s base, one side
has to advance and break through the enemy’s towers on the map. The base itself is protected
by a total of 5 towers. 3 of them are divided between the lanes and stand in front of buildings
called “Barracks”. The Ancient itself is protected by the last set of towers. One important note is
that towers are more powerful the closer they are to their ancient, meaning that the most outer
towers are the weakest whilst the towers guarding the ancient are the most powerful. As the
towers increase in power as one team progresses on the map, it is important that they grow
more powerful. Towers are classified between T1 and T4, with T1 being the furthest away from
the base, see Figure 3 below for a detailed view of each tower location.
Figure 3, showing tiers of the different towers for each side. O.P stands for Outpost which is a
structure that passively grants experience.
7
Killing enemy creeps, enemy heroes or enemies in the jungle (areas between the lanes) will
award the hero with both gold and experience, similar to games in the RPG genre. With gold,
heroes can purchase items that increase their power and as heroes get more experience, they
level up, unlocking new and more powerful abilities. Heroes usually fall under either the “carry”
or “support” role, with some heroes flexible enough to be played as either. A “carry” is a hero
that tends to get more farm priority in order to increase in power. Farm priority in this context
means that the carry player will prioritize killing creeps to acquire more gold. A “support” on the
other hand is often played to assist cores and tend to acquire less gold on the map. Supports
are also more prone to rely on their abilities to make differences in fights.
2.1 Concepts in a game of Dota 2
Below are some more advanced terms and concepts that are used within this thesis.
● Active attack modifier (AAM): Some heroes have spells that are classified as Active
attack modifiers and when used manually, they partly count as a spell cast. The benefit
of this is that, unlike regular attacks, they do not cause any creep aggro or tower aggro.
Normally when a hero attacks another enemy hero or a tower, creeps around the entity
attacked will aggro the attacker. This will cause them to attack the hero, and can make it
dangerous to play on the offensive. Players often use these Active attack modifiers to
harass and attack enemy heroes without the downside of aggroing creeps.
● Aggro: If a hero is attacked, nearby friendly units will attack nearby enemy heroes. This
applies to both creeps and towers. If a hero is attacked, its nearby friendly creeps will
attack nearby enemy heroes, prioritizing the closest enemies. Only enemies within the
creeps vicinity are considered, with a range of 500 for melee creeps, 600 for ranged
creeps and 800 for siege creeps. Towers have a list of priorities when choosing enemies
to attack. If nearby allied heroes are being attacked by enemy heroes, the tower will
target and attack the aggressive enemy if they’re in its attack range. Attack range for all
towers is 700. Attack range is the distance a unit can attack a target from.
● Cooldown: Most spells in Dota have something called cooldown, which is the time it
takes before a spell, ability or item can be used again. For instance, the spell Magic
Missile has a cooldown of 12 seconds at spell level 1. This means it takes 12 seconds
for the spell to be ready again after it has been cast.
● Farming: A term used to describe when a hero focuses on killing creeps rather than
enemy heroes.
● Kills: Killing an enemy hero counts as a kill. These are tracked in the game’s scoreboard.
● Jungle creeps: Neutral creeps found in specific locations in the jungle. These creeps
have no affiliation to either team and both teams can kill these creeps to gain gold, items
and experience.
8
● Jungling: Term used to specify that a hero is farming in the jungle.
● Lane Creeps: There are 3 types of lane creeps, Melee, Ranged and Siege. Lane creeps
spawn at the start of the game when the game timer reaches 00:00 and then once every
30 seconds. Lane creeps spawn at their respective barracks, one “creep wave” (group of
creeps) per lane. Melee creeps have more health than their ranged counterpart and are
the most numerous in a creep wave. Ranged creeps have less health than melee creeps
but instead have more damage and the ability to attack from a distance. Siege creeps
only spawn every tenth wave. They have the most health of all the creeps and their
attacks deal significant damage to buildings. Destroying an enemy’s barracks will cause
the creeps in that lane to become empowered. Destroying the melee barracks causes
melee creeps to become empowered and destroying the ranged barracks causes the
ranged creeps to become empowered. These creeps are called super creeps. When all
the enemy barracks are destroyed, creeps evolve even further into mega creeps which
are the strongest variant of creeps.
● Laning: The act of playing within their predetermined lane.
● Melee: A hero is classified as either melee or range which determines their attack range.
Attack range is from what distance a hero can attack an enemy. Melee heroes can only
attack enemies within their vicinity.
● Ranged: Ranged heroes have increased attack range and can attack from a distance.
● Teleport: A form of instantaneous movement between 2 points.
3 Framework
The framework used in this thesis has its roots as a framework developed for a 1v1 competition,
developed by Tobias Mahlmann and Jose Font [18]. The purpose of the framework was to let
participants submit their own AI controllers to “The Dota 2 Bot Competition”. Through their work
the aim is to foster research on AI techniques for real-time games. The framework made it
possible for an external bot to take control of a hero in a game of Dota 2. In the 1v1 solo mid
game mode, the bots fight each other until the first tower in the middle lane falls, determining
the winner. It worked by having the bot and framework run as a web service. The bot was
connected to the Dota 2 match with communication going both ways. The communication was
achieved by Hypertext Transfer Protocol (HTTP) POST calls with all the information regarding
the world, game state and actions transferred in JavaScript Object Notation (JSON) formatted
files. There were some limitations hindering an entire line up to be controlled by external bots,
as games could only be hosted with one hero played by the external controller and the rest were
controlled by the already built in AI. One of their future improvements listed exploring the
possibility of extending the framework to include 5v5 matches, with the mod connecting more
than one external bot at the time.
9Since Dota 2 is a team game, the previously mentioned framework was lacking the ability to test
teamwork and cooperation of multiple agents. With this motivation, Dennis Nilsson and Kalle
Lindqvist upgraded the framework developed by Tobias Mahlmann to be able to handle 5v5
matches [10]. Their work passed a number of iterations, for instance expanding the framework
to support 5v5 and going from Java web API to Python to enable support for future developers
to use Python's machine learning libraries. It is this upgraded framework that is being used for
both this thesis and the competition.
3.1 Methodologies
Two types of methods were used to collect data, observation and analysing data already
collected by third-party sites.
The goal of this thesis is to create bots that emulate player strategies, thus making observation
a relevant choice for collecting data. One of the upsides of collecting data through observation is
the ability to observe players in a natural setting [19]. Watching games and seeing how
proficient players make use of their heroes proved to be helpful. Dota 2 supports watching
games live and players can search for specific heroes that are currently played and sorts them
by their MMR (Matchmaking Rating). It is also possible to watch replays of past games. High
ranked and professional games were selected as these players are in the top percentile of the
playerbase, thus most likely being able to play the game and their heroes to their full extent.
To support and strengthen the findings gathered through observation, statistics were also looked
at from a quantitative viewpoint. Data looked at here were post-game stats, such as kills,
deaths, tower damage etc. This was done to strengthen the hypothesis of each strategy on how
they would perform. Another case where this was useful was for the lane selection. With the use
of Dota2protracker [20], it was possible to see in what lane each hero was mostly played. This
site collects data from Dota 2 games from games in a very high skill bracket, including
professional players when they play in public matches. This data was also used for two heroes
in the final strategy, with the purpose to answer the second research question regarding if using
statistics can be beneficial.
As previously mentioned, to collect data as inspiration for designing the bots, high ranked
games were looked at. In total, three different AI strategies were planned, where each one
would implement some behaviour observed in the games. The implementation of these
concepts and behaviour implemented into the AI will hopefully be able to provide an answer to
the first research question, regarding gameplay. Each strategy will be explained in depth in the
upcoming section. One example of observed behaviour was for Viper and his use of Active
attack modifiers (AAM) in the first strategy. Examples on how this hero makes the best use of
his AAM can be found in the Appendix, Section D. Another strategy where this method was
used was for the third strategy, Io - Luna. In the case of Io - Luna, reviewing games showed that
they often played away from their teammates with the ability to join the rest of the team if a fight
broke out with Io’s ultimate ability “Relocate”. Some examples with the game's references can
be found in the Appendix, Section E.
10The goal is to run tests on all different strategies by facing the built-in AI and then collect data
from the matches. This data will be analysed to see how well each strategy performed. The
different strategies will also be compared versus each other to see if some elements of human
gameplay were more valuable than others when they were implemented in the AI. The efficiency
of the select methods will be discussed in the Discussion and Analysis section.
4 Contributions
The contributions added to this framework are three different bots, each implementing their own
strategy with some aspect of their game plan trying to mimic that of human players. This section
will cover each strategy and the rationale behind them.
4.1 Strategy 1: Simple heroes
The heroes selected for this first line up were: Viper, Jakiro, Sniper, Luna and Vengeful Spirit.
The rationale behind selecting these heroes is that they are considered “simple” in their
gameplay and thus their playstyle should be easy to mimic in the developed AI. Four of the
heroes selected are a part of Dota 2’s new player experience, where new players are presented
with a limited pool of heroes to choose from. This limited hero pool only consists of 19 heroes
out of the total 121 heroes where 11 are considered “carries” and 8 are considered “supports''.
As they are part of the new player experience, they are well suited for new players trying the
game. In total, 3 different concepts were implemented and tried in this strategy. The purpose of
using 3 different concepts is to compare the results of how each concept performed versus the
opponent. The first lineup can be viewed in Figure 4.
Figure 4, showing the five heroes in the first strategy. From left to right: Viper, Jakiro, Sniper,
Luna and Vengeful Spirit. Screenshot taken from Dota 2.
4.1.1 Hero introduction
Viper is a ranged agility carry with potent abilities that deal damage over time and has the ability
to slow down enemy heroes. In this strategy he will be laning in the top lane together with
Jakiro, who is a ranged intelligence support with spells that can both stun and deal damage to
enemy heroes. This is also one of the few heroes that has an ability that works on enemy
buildings, namely “Liquid Fire”.
11Middle lane will be occupied by Sniper, who is a ranged agility carry and has one of the highest
attack ranges of all heroes. This means he can stand back from a far distance and attack his
enemies. His ultimate ability “Assassinate” deals a high number of damage and is suitable to
finish off low health targets. Mid laners almost always play the lane alone, without a partner.
Bottom lane will have Luna and Vengeful Spirit playing together. Luna is a ranged agility carry
that can stun, as well as quickly tear down multiple enemies with her bouncing glaives, traveling
and damaging enemies that are close to each other. A stun is an ability that renders the receiver
to be unable to move or perform actions. Her partner, Vengeful Spirit, is a ranged agility support.
In her toolkit she has a damaging stun and can both strengthen nearby allies as well as weaken
nearby enemies.
4.1.2 Concept 1: Active Attack Modifiers
The first concept tries to take full advantage of abusing the previously mentioned AAM.
Watching games from Viper showed that he would often win his lane by out-harassing the
enemy laner with his spell “Poison Attack”. He was able to output a high amount of damage
whilst taking little damage himself, as nearby creeps would not aggro him. It also proved hard
for the receiving hero to fight back, as “Poison Attack” stacks up the more he is hit, eventually
taking fatal damage. In most cases, the receiving hero would try to run back and outrange Viper
as soon as he was hit. For examples of this in action, see Appendix section D, page 32. The
goal was to mimic this behaviour in the top lane as Viper was paired with Jakiro, another hero
with an AAM.
The hypothesis is that these heroes will apply a considerable amount of pressure on the lane as
they can use these AAM to attack enemy heroes without creeps or towers interfering. With their
spells that deal damage over time, as well as slows enemies, it is believed that these heroes will
be able to also get kills on their lane.
4.1.3 Concept 2: Using spells to land kills
The bottom lane with Luna and Vengeful Spirit would focus on getting kills on enemy heroes, as
they both have high-damaging stuns in their arsenal. One example to demonstrate how effective
use of spells can be: at level 5 the enemy hero Phantom Lancer has 740 total health without
items. At level 5 both Luna and Vengeful Spirit have spells available that combine to deal 495
damage and can stun for a total of 2.4 seconds where the target can be further attacked without
the ability to respond or run away. Watching games showed that kills in the early part of the
game rarely comes from simply “auto-attacking” the enemy hero with basic attacks. Kills were
often accomplished by the hero using spells on their enemy together with their laning partner.
Observations and statistics gathered confirmed this, as it showed that it is rare for heroes to kill
enemy heroes on their own within the first 10 minutes of the game. Collected data of early game
kill participation can be found in the Appendix, Section C. Many combinations of spells were
successful, such as damaging spells paired with a stun or slow from its partner. During the lane,
the objective for Luna and Vengeful Spirit was to use their abilities synchronized in order to
12score kills. With the design of Luna and Vengeful Spirit heavily focused on this aspect, the belief
is that they would be the hero pairing involved in the most kills.
4.1.4 Concept 3: Using spells to farm
Findings from watching Jakiro games showed that the hero proved efficient when it came to
applying pressure to towers. The hero’s toolkit allows him to push out waves closer to the
enemy's tower with his first ability “Dragon’s Breath” and once at the tower hit it with his AAM
“Liquid Fire”. This meant that even if the hero was played in the support role, he could collect
some farm of his own and deal higher damage to the enemy’s tower than other supports.
According to statistics collected, Jakiro was slightly above third place for his team when it comes
to building damage, see the Appendix, Section A. This playstyle in addition to using AAM lead to
the hypothesis that this hero pairing would be the fastest in bringing down a tower.
4.2 Strategy 2: Jungle
This strategy introduces a new hero, one that is also a part of the suggested heroes in the “New
player experience”, namely Tidehunter, who is a strength melee carry. This hero is innately hard
to bring down, as he can shrug off stuns and other negative effects from himself, as well as
being able to lower the damage of enemies hit by his ability “Anchor Smash''. The hero from the
previous game Viper was replaced as he lacks the prowess that Tidehunter has when it comes
to farming the jungle.The entire lineup can be viewed in Figure 5 below.
Figure 5, showing the five heroes in the second strategy. From left to right: Tidehunter, Jakiro,
Sniper, Luna and Vengeful Spirit. Screenshot taken from Dota 2.
The second strategy implemented tries to take full advantage of the jungle, to gain more value
out of the map. As resources on the map are limited, having multiple heroes farming the same
lane can cause the heroes to become poor, as there are multiple heroes sharing the same
limited resource (creeps on the lane). By using the jungle, players can gain more value out of
the map by splitting up and farming in different areas. Jungle creeps also drop items that can be
used by heroes to further increase their power. In every single game looked at, the jungle was
farmed at some point in the game.
13Luna was to play in the jungle after the first bottom tower was dead and her ultimate ability was
on cooldown. The argument behind this is that she has the capacity to stay in the lane when her
ultimate ability is ready, as it provides the hero with a strong lane presence. The thought is that if
a fight breaks out when she is on the lane, she can turn the fight to her advantage with this
ability. If the ability has been expended, she is to fall back to the jungle, thus making her less
exposed to being attacked and killed by enemy heroes. The hypothesis is that she will be able
to get both farms and kills whilst present in lane and when her power spike decreases, waits for
it to come back while still getting farm in the jungle. The other hero chosen to jungle was
Tidehunter. The hypothesis behind the strategy was that since 2 heroes would spend some time
of the game farming in the jungle, and the rest each having their own lane to farm would lead to
the heroes acquiring a higher net worth than the previous strategy.
4.3 Strategy 3: Io - Luna
The last strategy looked at implementing the hero “Io”, a hero that is usually played close
together with a partner. This playstyle comes from the fact that Io is a hero that can greatly
improve the potency of one nearby ally. One important aspect of Dota is what kind of role each
hero fills in a lineup in regards to farm distribution. Heroes are defined either as a “carry” or
“support”, where the role of the carry is to acquire most of the farm available on the map which
translates into more items, making them more powerful. The support role usually entails as the
name suggests, “supporting” their carries to enable them to have the best game possible. The
hero Io enables a certain style of playstyle, with Io and its partner farming on one side of the
map with the rest of the team on the other side. This is enabled due to the hero's ultimate ability,
“Relocate”, teleporting Io and one ally to any point in the map. This strategy tried to mimic that
behaviour. The way they played in the lane was also altered to have Io focus more on
“supporting”. In other words, Io would prioritize hitting enemy heroes that were close and in very
rare cases try to get kills on enemy creeps, leaving more creeps to be killed by Luna. To aid in
how to create this hero pairing, statistics from high rated games were selected from the site
Dota2protracker [20]. Images showcasing the use of Io’s ultimate ability “Relocate” can be found
in the Appendix, Section E. The final lineup can be viewed in Figure 6.
Figure 6, showing the five heroes in the third and final strategy. From left to right: Viper, Jakiro,
Sniper, Luna and Io. Screenshot taken from Dota 2.
14The hypothesis was that Luna would be the most farmed hero on the map, as the bottom lane
duo closely mimicked the “carry” and “support” duo observed in real games. As both Luna and
Io now followed item and skill suggestions collected from high level games, this furthered the
anticipation of their success. The following of the suggestions are also used to see if statistics
from human players can be beneficial, in other words, help give an answer to the second
research question.
5 Results
This section will cover the result of each strategy in detail to see if it matched expectations when
it came to performance. As all three strategies use gameplay that is taken from observations
from human players, they all seek to collectively answer the first research question. All
strategies use concepts and gameplay from humans and the success of these implemented
concepts will be measured to show their potency in this regard. In strategy 3, the results
gathered in regards to two of the heroes, are used to help give an answer to the second
research question. Two heroes use statistics in this strategy and their results will be compared
to the rest of the strategies to show whether there was an improvement or not.
Comparison between the strategies in how the heroes performed will be conducted. A more
comprehensive discussion of the results can be found in the Discussion and Analysis section.
Statistics for specific games and strategies can be found in the Appendix section. Metrics
chosen to look at are presented below:
● Levels: Levels were tracked to see if there was a notable difference in the experience
earned between the strategies and heroes within the same match.
● Net worth: Net worth is calculated as how much gold the hero currently has in addition to
the value of the hero’s items. With gold being able to be acquired from multiple sources,
a hero’s net worth serves as a good indicator if the hero has been efficient in the game.
When comparing net worth for each hero between different strategies, the goal is to see
which strategy earned the hero the most gold.
● Kills: Kills will be tracked to see which strategy proved to be the deadliest for the enemy
team. Some strategies are drafted with a focus on killing enemy heroes, such as 2 of the
heroes in Strategy 1 (concept 2), thus tracking the number of kills for each hero will
prove if they are successful or not.
● Deaths: How many times each hero has died in a game. Will aid in determining which
strategy was most exposed by the enemy team.
● Assists: Assists are awarded to heroes that were involved in killing an enemy hero, but
not awarded to the hero that landed the killing blow.
15● KDA: KDA or Kill to Death ratio works by adding the number of kills and assists then
dividing them by the number of deaths. It can serve as a more impactful way to compare
for instance carries to supports, as it factors in both kills, assists and number of deaths.
● Tower timings: The time when towers fell was also recorded to see which lane was the
quickest to bring down their towers and if there was any difference between the
strategies. These tables contain the average time each of the outer towers fell.
● Average end score and game length: Noted to see how long each game was in order to
see which strategy won in the shortest time possible. Score was also noted to see how
even the teams were in terms of kills. Score in a game of Dota 2 refers to number of kills,
for instance a score of 20 - 12 means that Radiant killed 20 heroes and Dire killed 12.
165.1 Strategy 1: Simple heroes
Table 1
Average end score and game length over 5 games for strategy 1
Radiant 21.8
Dire 4.8
Length 28:07.2
Table 2
Average stats over 5 games for strategy 1
Hero Viper Jakiro Luna Sniper Vengeful
Level 12,4 13 14.2 14.4 12.4
Net worth 4,793 5,456.4 6,550.4 6,091 3,701.8
Kills 5.2 2.6 6.6 1.6 4.8
Deaths 0,4 0.2 1.2 0.2 2.8
Assists 2 3.4 6 0.6 5.6
KDA 6.6 6 10.66 2.2 4.386
Table 3
Average Tower timings for strategy 1
Tower Time
Top t1 9:27.6
Mid t1 12:48.8
Bot t1 10:35.4
Top t2 21:38.6
Mid t2 23:57.4
Bot t2 21:59.6
This section contributes to answering the first research question. Results from the first strategy
implemented show that the duo of Luna and Vengeful spirit were most proficient when it came to
killing enemy heroes as shown in Table 2. Luna averaged the most net worth of the course of 5
games whilst also topping the number of kills. This performance shows the strength that human
players would implement when laning, using spells together to acquire kills. As previously
discussed, their spells together are quite potent, and when the AI uses these abilities in a
17synchronized fashion it often leads to kills. What Contributes to Success in MOBA Games? An
Empirical Study of Defense of the Ancients 2, by Xia B, et. al. present that kills made by multiple
players is one of the key factors of the game result [21]. Table 1 shows the overall score and as
the score was in favor of the “Radiant” side (the side of the implemented AI), this allowed them
to carry out their programmed strategy to secure the win. This bot Sniper was a hero that was
playing alone for most of the game, as the other 2 lanes were working in pairs to push down
their respective towers. This led to Sniper being able to apply pressure on the middle lane and
getting his own farm mostly uninterrupted. Sniper also had the advantage of farming alone on
his lane, leading to a high value in net worth although he was least active in terms of kill
participation. Viper and Jakiro were consistent in bringing their top tower down first, see Table 3.
This was made possible by 2 major factors. Firstly, Vipers strong lane presence enabled the top
lane to scare the enemy heroes away from the lane or kill them. Secondly, Jakiro would push
the lane and apply damage to nearby enemy heroes with his spell usage, often pushing the
creep wave to the enemy’s tower. Here he would attack the tower with his ability “Liquid Fire”,
both damaging and debuffing the enemy tower. These factors were key in their success in
bringing down the top tower. The concepts implemented in the first strategy performed as
expected and with positive results versus the built-in AI. Given these results it shows support for
the notion that using gameplay from humans can be useful.
5.2 Strategy 2: Jungle
Table 4
Average end score and game length over 5 games for strategy 2
Radiant 11.8
Dire 7.2
Length 27:37.8
Table 5
Average stats over 5 games for strategy 2
Hero Tidehunter Jakiro Luna Sniper Vengeful
Level 12 10.8 12 13.6 11
Networth 5,088.2 3,259 5,101.6 5,362.8 3,045.8
Kills 2 1 3.4 2.4 2.4
Deaths 0.2 1 2.4 0.8 2.4
Assists 0.8 1.2 2.4 0.8 2.4
KDA 2.8 2.066 2.966 2.667 2.56
18Table 6
Average Towers timings for strategy 2
Tower Avg
Top t1 9:32.2
Mid t1 13:20.8
Bot t1 12:23
Top t2 x
Mid t2 20:24
Bot t2 17:26
This section contributes to answering the first research question. This strategy was trying to gain
more farm out of the map by jungling. Luna was chosen to only jungle when her ultimate ability
was on cooldown with the belief that she would be able to be strong on lane and secure kills
when it was available. This proved not to be the case, as her number of kills is only slightly
above her average deaths, see Table 5. Compared with the results from the first strategy, her
performance went down considerably in all measured aspects. Tidehunter, who instead spent
much more of his time in the jungle after the top tower had fallen, performed almost as
expected. As his time spent in the jungle was not dictated by his ultimate timing and was only
ended after he had reached a certain level, his kill participation was one of the lowest. His farm
managed to stay on par with the other cores in his team, only slightly behind. The timing of the
top tower kill was similar to the first strategy, although their potency in the lane in terms of kills
were quite lower. Tidehunter's skill build was adjusted to farm creeps and not heroes, resulting
in the top pairing landing fewer kills in the lane. As the overall strategy was to make use of the
jungle to enable heroes to hit a powerful timing before grouping up, this in turn affected Sniper.
He had heroes in his lane helping him push at an earlier stage, shown by the lower time taken
to kill the second mid tower as shown in Table 6. This meant that he had less alone time than
the previous strategy to farm creeps, causing his net worth to decrease. Although the average
end game score was much closer than the first strategy, this strategy managed to win in a
shorter time, see Table 4. Reasoning for this was after a predetermined point, the heroes would
group up and push down the middle tower. The second top tower was never killed which is
another reason for the lower game length. The results from this strategy was not as expected,
showing difficulties that can arise when trying to mimic the behaviour of human players in an AI.
195.3 Strategy 3: Io - Luna
Table 7
Average end score and game length over 5 games for strategy 3
Radiant 19,4
Dire 6.4
Length 26:04.6
Table 8
Average stats over 5 games for strategy 3
Hero Viper Jakiro Luna Sniper Io
Level 11.4 12,6 13,2 13,6 11,2
Networth 4,835 5,406 8,011 5,959.2 3,559.4
Kills 4 3,2 4,8 4,8 2,2
Deaths 1,2 2,4 0,2 1,6 1
Assists 2,4 2,6 4 2,4 5,8
KDA 5.6 2.85 8.8 5.7 7.6
Table 9
Average Towers timings for strategy 3
Tower Avg
Top t1 8:23.2
Mid t1 13:56
Bot t1 9:16.6
Top t2 17:03
Mid t2 20:10.4
Bot t2 21:26
This section contributes to answering the first research question as well as the second. This
strategy showed the potency of the Io-Luna combination as Luna was able to reach the highest
average net worth among all the strategies, shown in Table 8. She managed this whilst also
having the shortest matches as can be seen in Table 7. The fact that she was always backed-up
by Io led to her dying only once in all 5 games. These two heroes also applied statistics when it
came to items and skills into consideration in the design phase, a factor that could also be
20another reason behind their success. The fact that this hero pairing performed as well as they
did gives support to the second research question. Her skill build in this game was much more
farm oriented, as data collected suggested only skilling stun once. Her farming pattern after the
bottom tower was killed was also an upgrade from the previous versions, not only jungling but
also pushing out the bottom lane when creeps were nearby. This was the most successful
implementation of “human strategy”. The first tower in Io and Luna’s lane was also killed faster
than the other strategies, see Table 9 for tower timings. The other heroes were also not
negatively affected by this strategy, as their stats were similar to that of the first strategy. The
results for this final strategy shows support for both research questions, as they performed
better in many areas compared to the first two strategies.
6 Limitations
This section will cover limitations that arose during the development period, both in regards to
working with the game and the framework.
6.1 Limitations from the game itself
There have been some limitations working with Dota’s own API. One such case is regarding
items, more specifically newer items. The item “Crown” was added in patch 7.2 which was
released 2018-11-19. This item is a component that can be upgraded to five other items but the
problem is that this item lacks an “internal name”. This means it cannot be accessed by our
framework to be purchased as a stand alone item. The only way of acquiring items where
"Crown" is a component, is either by buying each component separately until all components
required for said item has been bought or if the item is purchased at once in its entirety for its
full cost.
Version 7.28 released in 2020-12-17 added 13 new items, all of them missing. Some of these
items are “final items”, meaning they are not part of components or used as a build up to other
items. This means that there is currently no way to acquire these items for the AI.
This lack of support from Valve's side is unfortunate, as it hinders the designing of AI from
keeping up with current item trends that players might use.
6.2 Limitations with the framework
Some limitations with the framework were discovered during the design process of the AI, where
the goal was to convert human strategies observed into behaviour in the AI. Items dropped from
jungle creeps and runes that were found on the map could not be interacted with. Both these
things were shown to be a big part of the gameplay in the observations but something that was
unable to be employed by the developed AI. Another problem that was unable to be solved was
regarding summoned minions by different heroes. One hero that was originally a candidate for
one of the strategies was the hero “Beastmaster”, as he was one of the heroes with the highest
win rate in both public and professional games at the time of data gathering. This hero’s
21playstyle revolves heavily around his ability to summon minions and controlling these to attack
enemy heroes or creeps. The framework however only supports the control of the 5 heroes
chosen at the start.
There were also limitations when it came to purchasing items, as the framework requires a hero
to be physically near the item shop. Normally, a player can purchase an item anywhere in the
map and have his items placed in his stash, which is an inventory back in the base. These items
can then be delivered to the hero via the use of the courier or the hero picking up the items
himself by travelling to his fountain. The courier is a mobile entity with the sole purpose of
buying and delivering items to heroes, with each hero having their own courier. Due to these
limitations, deviations had to be implemented that were different from how human players
usually purchase items and retrieve their items.
Finally, there were troubles with using “teleport scroll” to allow heroes to teleport around the
map. This once again forced deviations on how to design the AI as players frequently use these
“teleportation scrolls” to move around the map. The developed AI had to instead move around
the map by walking.
7 Discussion and Analysis
The purpose of this thesis was to determine if using game knowledge and statistics from human
players could be beneficial to AI design. Each of the different strategies employed some aspect
that was observed from games with human players and successfully managed to beat the
built-in AI. Given each strategy’s success in winning over the built-in AI, this result supports the
notion that mimicking human players is a valid strategy for designing AI. There were, however,
differences in performance between each strategy that requires further analysis. As Dota 2 is a
very complex game, changing how one hero acts can have an effect on another hero's
performance. This was shown in the second strategy, “Jungle”, as changes that were theorised
to be an improvement instead turned out to be detrimental. Four of the heroes in this strategy
suffered greatly, including a decrease in net worth and an increase in the number of deaths. The
decisions that human players make during a live game on which actions to take can be hard to
properly plan and implement in an AI. As data gathered suggested, the jungle is a huge part of a
match in Dota 2, but the decisions when to jungle and when to be present on a lane varies
greatly from game to game and can be hard to replicate in an AI.
In regards to the first strategy, which included 3 different concepts taken from observations, the
results were promising. All 3 of them delivered expected results, showing that mimicking human
behaviour can be successful. The fact that all of them were able to match their respective
hypothesis can to some extent be attributed to the statistics collected. The many different third
party sites with statistics were helpful in developing these hypotheses.
Regarding the second research question if using statistics from human players could be
beneficial, the results proved hard to analyse. The final strategy followed the most popular item
and skill build for high ranked games at the time for the heroes Luna and Io. One downside
22regarding this question that became clear after results were gathered was that two changes had
been done for the final strategy. The support - core mechanic during the lane between the 2
heroes was improved but Io’s and Luna’s item and skill build was also adjusted. As two metrics
were changed that both on their own could lead to improvements for the hero pairing, it is
difficult to tell the effects of each one individually. As Araujo et al. found in their paper, context
within a game matters greatly to determine the values of given items [15]. This translates to
some difficulties when trying to select items in advance in the design of the AI. This same
complexity can also impact what order certain heroes choose to level up abilities. However,
given the increase in performance for these 2 heroes in regards to the previous strategy, it is an
area that should be researched further. Another point that comes into critique when evaluating
Io’s performance is the fact the benchmark used to compare its performance was with another
hero entirely (Vengeful Spirit). It is possible to compare the strategies to each other and given
the results, declare which version is better. There is for thesis, however, no previous benchmark
for Io to compare if following item and skill builds from high ranked games were impactful.
There were also instances of large differences and large ranges between some of the results
collected for the same strategy. One example that stood out was for the third strategy. The
difference in net worth between Luna’s best and worst game was around 78%, with the best
game even being over a minute shorter. For future work it should be desired to run more tests
than what was performed in this thesis, as only having 5 games causes average values to suffer
more from outlying results. One theory for the cause of these differences between games is
believed to stem from the complexity of Dota 2, even if it is played by AI. Watching the
developed AI play versus the built in AI showed that there were differences between the games
in regards to which situations arose. Some of these situations were handled by the developed
AI, for instance winning a big fight between multiple heroes, and in other situations they lost.
Dota 2 also presents challenges in that it is an ever evolving game. A major patch was launched
during the development that came with a new hero, new items and map changes. In regards to
the work performed in this thesis, the map changes caused rework in the last 2 strategies.
These “balance patches” also mean that observations and data collected can become irrelevant
or outdated as players adapt new ways to play the game given these changes. Strong and
popular heroes can be changed to become weaker or have changes made to their abilities.
These changes cause the game to evolve as human players find new ways to play and optimize
their heroes and gameplay.
Regarding the methods used to collect data, being observation and looking at statistics, they
both proved to be useful. Through observation it was possible to directly collect inspiration for
strategies to implement. One downside of this approach is that it may require some previous
knowledge of the game. As the games analysed contained the highest percentile of players, it
does include a high level of gameplay and depth to the games, putting some requirements on
the observer. Data collection helped form hypotheses and strengthen these. One downside of
this data collection method is that it is entirely reliant on third parties for the data.
In regards to the sample pool when it came to matches and statistics, high ranked games were
selected. The argument behind this was since they are the top percentile of players, they have
23the most game knowledge and are most proficient with their selected heroes. In these matches,
their opposition is of equal caliber which is a much higher level than the built-in AI used as an
opponent in this thesis. Here could lie some difficulties as perhaps some behaviour and
gameplay observed might in some cases be considered excessive versus the lesser opponent
that was faced in this thesis.
8 Conclusion and future work
To conclude, the work performed in this thesis shows that AI design in MOBA games have
relevant challenges to face, due the complexity of the game itself, that require further
exploration. A variety of different strategies and concepts that humans use in their games were
able to be implemented with, for the most part, great results.
To answer the research questions:
1. How can using gameplay from real players be beneficial to AI design?
The thesis shows that implementing and mimicking plays that are used by human
players can be beneficial to AI design. All the different strategies were successful in
defeating the already built-in AI. Different strategies and different concepts proved to
have quite varying results when analysed. As such a small part of the game was
explored, with only 7 out of 121 heroes implemented, there are still opportunities for
further testing in order to increase the performance of the AI.
2. How can using statistics from real players be beneficial to AI design?
Given that the fastest strategy incorporated statistics from high ranked games and that
this hero pairing was better in many areas compared to the other strategies, there is
some support for this notion. However, as other research also has suggested, there can
be contextual differences between matches that determine how efficient using statistics
can be. Statistics were, however, useful when it came to generate hypotheses and
expectations for the AI. It also helped shorten the time spent during observations, as
observed behaviour could in some cases be verified much quicker by looking at
statistics. One example where this was helpful was for determining whether Jakiro was
an efficient hero when it comes to dealing building damage. As it stands, this thesis
shows some support, but ultimately lacks the ability to give a definitive answer to this
question.
For future work there are some areas that can be expanded upon. Firstly if the framework is
improved upon, it would open up new possibilities in what can be achieved in the developed AI.
As previously mentioned, many things that were observed could not be implemented. Should for
instance a courier be implemented it would decrease the downtime for the bots considerably, as
they can stay out on the map and be active instead of having to walk back to base to retrieve
purchased items. With less limitations from the framework, it can be possible to more closely
mimic human behaviour and gameplay.
24You can also read
