SERIOUS GAMES HUMANITARIAN USER RESEARCH - Lessons Learned
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
SPONSORED BY
SE RI O U S GA M ES
HUMANITARIAN USER RESEARCH
Imaginetic · Lessons Learned Simulations and Training
Serious Games: Humanitarian User Research
p. iii
that games are poor standalone learning tools,
and work better when used in coordination with
a variety of teaching methodologies.
Methodology
The research which informs this report took place
across six face-to-face workshops, conducted by
This report and the research which informs it are an experienced facilitator in January 2020. Three
intended to explore the potential applicability of workshops were held in Nairobi, Kenya, from 21-
serious games and games-based learning to the 23 January, and three were held in Amman, Jordan,
humanitarian sector, particularly in the context from 27-29 January. Each workshop was run with
of localization. Serious games are increasingly between 4 and 17 self-selected participants.
being adopted as training tools in other fields, Participants played through a selection of digital
but have seen slow adoption in the humanitarian and tabletop games, after which facilitated
sector (Brynen and Milante, 2013). debriefings took place. Participants were surveyed
before, during and after the workshops. See
Why should humanitarian training teams employ Section “2.1 Research Overview and Design” for
learning games? Students are more likely to a complete breakdown.
learn when they are interested in what they are
learning (Garris et al., 2002). Serious games are Key Findings
demonstrably effective learning tools. Games
E XEC UT IVE SUMMARY have been found to be particularly effective in: ◆ People demonstrated an ability to learn
promoting skill acquisition, knowledge retention, from games in the humanitarian context.
attitudinal change, supporting understanding Participants were able to recall lessons
of new concepts and ideas, shaping behaviour, even long after the fact, up to 45 days after
and improving context-based problem solving the workshops. Participants reported that
(Klabber, 2003; Mateas, 2003; Prensky, 2001; Ricci, engagement with games impacted their
Salas, & Cannon-Bowers, 1996). The capacity to behaviour at work, their approach to work,
safely fail in serious games is a key component and their relationships with beneficiaries
of their value as a learning tool. Failure has even 45 days after the workshops.
been identified as an integral part of learning ▷ See section “3.3 Effectiveness of Games-
(Anderson et al. 2018). Based Learning in Humanitarian Work”
Through “real life” problem solving, serious games
have the capacity to help humanitarian students
more deeply understand and critically engage
with important issues. Experiential Learning
Theory and Situated Learning Theory help explain
why this is the case. According to Experiential
Learning Theory (ELT), individuals learn most
from direct experience, active participation, and
Serious Games: Humanitarian User Research
visible feedback on the consequences of their
actions. Situated Learning Theory (SLT) likewise
suggests that people learn better when placed
in authentic contexts to perform actions that
parallel real world tasks, interacting with others
and applying knowledge. Games apply these
theories by creating experiences that reflect real-
life challenges, like how to manage employees
with different personalities, and allowing the
learner the opportunity to play through different
situations. However, available research suggests
p. ii
Serious Games: Humanitarian User Research
p. v
◆ Participants were eager to learn with ◆ Language ability was a serious determinant ◆ The additional time required for learning ◆ Only use a game when appropriate:
games. Even participants who were not of learning. Learning games are often games was cited by many participants as a Learning games are more expensive and
familiar with learning games were excited only available in English, and often rely potential challenge for implementation in time consuming to develop than traditional
to try. Pre-existing “game literacy” was not on detailed and complex instructions their offices. Wherever possible, learning learning tools. They should be employed
a determining factor in the effectiveness which must be well understood. Wherever games should be short and concise to thoughtfully to promote specific learning
of learning games. possible games should be translated into minimize operational overhead. outcomes.
▷ See section “3.2 Perception of Games- the language of the audience. ▷ See section 3.4.3: Potential barriers to
Based Learning in Humanitarian Work” ▷ See section 3.4.2: Potential barriers to learning: Time investment ◆ Get to the point quickly: The longer
learning: Language it takes a learner to unlock a learning
◆ People felt that games were better teaching ◆ Skepticism of terms such as “games” and outcome, the more likely it is for a learner
tools than PowerPoint lectures. 84.5% of “gaming” among managers was cited by to abandon the game before achieving
participants felt that learning games were participants as a potential barrier for the designer’s purpose. This is especially
more effective than PowerPoint slides implementation of learning games in relevant in contexts where participants
or lectures when learning the relevant existing training programs. are expected to engage with digital games
subject matter. ▷ See section 3.4.5: Potential barriers to independently.
▷ See section “3.3.5 “Game Literacy” is Not learning: Buy-in from Management
as Important to Games-Based Learning ◆ Pay attention to the user interface (UI)
as Expected” Digital games vs tabletop games and user experience (UX): Learning is
directly impacted by both user interface,
◆ Interest in games and self-reported This research included both digital and tabletop or UI (how the player interacts with the
learning from serious games was not learning games. Both were found to have game) and user experience, or UX (what
gendered: men and women were equally strengths, weaknesses, and trade-offs. the game makes the player think or feel).
excited, engaged, and learning. Age proved
to be a minor determinant of enthusiasm Digital games are harder to design well, more ◆ KISSS Principle – Keep it Simple in Scope
and engagement. expensive to produce and revise, but easier to and Small: Learning games are more
▷ See section 3.1: An Overview of Participant distribute and require less facilitation. Digital effective when they are small in scope, clear
Demographics games can be easily played repeatedly, on in intent, and aim to teach a limited number
demand and when convenient. They are best of learning outcomes. Unnecessarily large
◆ Debriefing, contextualization, and skilled for shorter, simpler lessons that benefit from projects or unintentionally complicated
facilitation are essential to the learning repetition. games can confuse learners.
process. Learning games should be
accompanied by other teaching material. Tabletop games are faster to design and ◆ Teaching the Reality rather than
The presence of a skilled facilitator revise, and can be corrected “on the fly” by an Teaching the Ideal: Humanitarian
was important to supporting learning. experienced facilitator. However, they are harder learning games need to be attentive
In particular, a structured debriefing to distribute and facilitate. Tabletop games are and responsive to real-life experiences
session was essential. For digital games, better for immersive one-time experiences that and data, rather than designed around
further study of how a debrief might focus on social interaction and complex problems. convenient assumptions. Learning games
be delivered through a learning app is ▷ See section 5, “Digital games vs tabletop should explore how mistakes and errors
recommended. games” for a detailed comparison. are made, rather than avoid difficult issues.
▷ See section 3.3.6: Briefing,
Contextualization, and Skilled Facilitation Best Practices in Learning ◆ Walk Before Running: Designing learning
are Essential to the Learning Process Game Design games is time consuming and difficult, and
Serious Games: Humanitarian User Research
applying them in humanitarian contexts
Potential Barriers to Learning In the course of this project, the research team is a new innovation. Starting with small,
collected several “good practices” for learning achievable projects will help build skill and
◆ Technological restrictions were a serious game design. These suggestions do not represent a minimize the cost of occasional errors.
limitation for learners. Digital learning consensus among all learning game designers, nor
games should be explicitly designed with should this list be considered exhaustive. However,
older technology and limited access to the these suggestions were supported by our research.
internet in mind. ▷ See Section 6: Best Practices in
▷ See section 3.4.1: Potential Barriers to Humanitarian Learning Games for a
Learning: Technological Challenges full discussion.
p. iv
CONTENTS
Serious Games: Humanitarian User Research
p.vii
1. Literature Review 6
1.1 What Are Games & What Kind of Games Exist? 6
1.2 Why Games? 10
1.3 Effects of Game Design on Game-Based Learning 13
1.4 Serious Games in Humanitarian Contexts 15
1.5 Goals and Structure of Serious Humanitarian Games 17
2. Research Methodology 23
2.1 Research Overview and Design 23
3. Research Outcomes 27
3.1 An Overview of Participant Demographics 29
3.2 Perception of Games-Based Learning in Humanitarian Work30
3.3 Effectiveness of Games-Based Learning
in Humanitarian Work 35
3.4 Potential Barriers to Learning 44
4. The McGill Workshop 50
5. Digital Games vs Tabletop Games 51
6. Best Practices in Humanitarian Learning Games 53
6.1 Why are we Using a Game? 53
6.2 “Get to the Point”: 54
6.3 The Genre and Structure of the Game
Must Reinforce the Learning Goals 55
Serious Games: Humanitarian User research
6.4 The Importance of the User Interface (UI)
and User Experience (UX) 55
6.5 The New “KISSS” Principle: Keep it Simple in Scope and Small 58
6.6 Teaching the Reality rather than Teaching the Ideal 58
6.7 Walk Before Running 59
p. vi
AP P E ND IC E S
Serious Games: Humanitarian User Research
p. ix
Figure 21: Worldwide Distribution of Android Versions (Active Users) 44
Appendix A: Game Features & Types 61 Figure 22: Android Version Use in Workshop Countries 45
Appendix B: Summary of Games Considered and Included in the Study 63 Figure 23: In What Language do You Play Games? 46
Appendix C: Training, Gaming and COVID-19 64 Figure 24: Enjoyment by Primary Natural Play Language 46
Appendix D: References 65 Figure 25: Impressions of Games Based Learning vs PPT Lecture by Natural Play Language 47
Appendix E: Further Reading 67 Figure 26: Self Assessed Learning by Natural Play Language 47
Appendix F: About the Authors 68 Figure 27: UI/UX Image 55
Figure 28: UNOCHA Image: Humanitarian Icons 56
FIG URE S
Figure 1: Demographics: Participant Nationality 26 TA BLES
Figure 2: Participant Level of Education and gender Identity 28 Table 1: Game Enjoyment by Gender Identity 30
Figure 3: Participant Age Distribution 29 Table 2: Average Enjoyment Over time 30
Figure 4: Participant Game Familiarity and Frequency of Play 30 Table 3: Role of Games 31
Figure 6: Would You Use a Similar Game in Your Personal Learning? 32 Table 4: Serious Games Awareness 32
Figure 7: Engagement of Gameplay 33
Figure 8: Perceived Learning 35
Figure 9: Skill Score Over Time 35
Figure 10: Skills Practiced During Workshop 36
Figure 11: Effectiveness of Interventions Over time 37
Figure 12: Understanding of Beneficiaries 37
Figure 13: Will the Lesson Affect Work (Post debrief) 39
Figure 14: Did the Workshop Influence Work (14 Day) 39
Serious Games: Humanitarian User Research
Figure 15: Did the Lessons Influence Work (45 Day) 39
Figure 16: Did the Lessons Learned Change Attitude Toward Served Communities 40
Figure 17: Enjoyment of Participants with No Prior Serious Games Experience 41
Figure 18: Self Assessed Learning by Participants with No Prior Serious Games Experience 41
Figure 19: How Important Was the Debriefing? 42
p. viii
Figure 20: Local Internet Accessibility 43
p. 2
AU THORS ’ REM A RKS
Serious Games: Humanitarian User research
“THE CREATION OF
SOMETHING NEW IS NOT
ACCOMPLISHED BY THE
INTELLECT BUT BY THE
PLAY INSTINCT.”
-CARL JUNG
Play.
Learning through play is as natural as breathing. learning. Herein, we explore just these issues,
Across the living world, development depends on examining prior research and conducting our own
play. Children’s socialization skills are learned study to inform the use of game-based learning
through mimicry and play at a young age. In the techniques, specifically in a humanitarian context.
animal kingdom, cubs and pups instinctively play
to learn the survival skills they will need in their Through direct, facilitated play workshops, we
adult life. Play is essential to development. explore front-line, local humanitarian workers’
attitudes toward games-based learning, and
Why, then, do we not play more? compare them to their preconceived ideas about
the endeavour. Then with contact touchpoints
I n c r e a s i n g l y , i n d u s t r i e s a n d s e c t o r s a r e two weeks and 45 days post workshop measure
recognizing the powerful nature of learning lessons learned, and knowledge retained.
through play in professional contexts, and the
pedagogical potential of games-based learning. As we seek novel educational delivery systems to
Professional gaming has been used in a military engage and challenge tomorrow’s humanitarian
context for two centuries, and filtered through leaders to become their best possible selves, we
myriad sectors by the end of the 20th century push them to develop the new ideas that will
and into the 21st. As technology has become drive the sector forward. This crucible of creative,
more accessible, and entertainment games insightful thought, if it is to be nurtured and
more popular, the envelope is being pushed encouraged, needs an environment where ideas
and learning games are coming to the forefront can thrive, be challenged and tested. Tomorrow’s
of educational innovations, crossing even more leaders need a safe-to-fail ecosystem of thought
boundaries. experimentation and creativity to evolve today’s
ideas into tomorrow’s solutions.
More and more research is emerging, reinforcing
the positive educational impact of games-based Tomorrow’s leaders need to play.
Serious Games: Humanitarian User Research
Tom Fisher & Matthew R. Stevens
p. 1
Serious Games: Humanitarian User research
p. 4
THE TEAM 01
M AT T H E W R . THOMAS JOHANNA
01 STEVENS 02 FISHER 03 REYNOLDS
02 03 Co-Author Co-Author Researcher
BRIANNA C AT H E R I N E STERLING
04 P RO C E V I AT 05 BENEDICT 06 PERKINS
04 05
Researcher Researcher Researcher
06 07
Serious Games: Humanitarian User Research
A L E JA N D R A
07
06 E S P I N O SA
Researcher · Editor
p. 3
p. 6
Serious Games: Humanitarian User research
employ Salen and Zimmerman’s definition as a
1. LI TERATU RE general guideline, while simultaneously including
REV I EW any activity which is commonly referred to and
accepted to be a game, understanding that game,
as a concept, is socially constructed.
Games can differ in purpose, such as games for
1.1 WHAT ARE GAMES & leisure versus games for learning. Games can also
WHAT KIND OF GAMES EXIST? differ in scale (gamification vs learning games
vs simulations), physical properties (analog vs
We are all familiar with the word “game”. The digital), and structure. These distinctions between
word brings to mind a variety of ideas and different types of games are important because
experiences, from competitive sports to boards they can influence potentially divergent costs,
and counters on a table, traditional past-times experiences and learning outcomes. Ultimately,
like backgammon and mancala to the latest digital deciding which type of game will best serve an
games. Some of us associate games with fun or instructor’s goals is about understanding the
excitement, others with boredom or confusion. target audience, objectives, and constraining
Our understanding of games is shaped by our factors.
personal experience and preferences, rather
than formal definitions. These preconceptions
often shape our receptiveness to the utility of 1.1.1 SCALE
serious games for learning, analysis, and research.
Unfortunately, these preconceptions are often Gamification
counterproductive; too much skepticism, and
we miss out on the unique benefits that serious Gamification is when game elements are applied
games can bring. Too much enthusiasm, and in non-game contexts. Common examples of
we risk applying games in inappropriate ways gamification in society include retailers’ points-
because we think they are “cool” or “fun”. Defining based rewards programs or organizations
terms--as best we can--is a starting point to providing publicly visible, socially desirable
formalizing the study and application of serious awards for achievements such as medals or online
games to humanitarian work. “badges” to staff as motivation. In educational
contexts, gamification is a teaching strategy that
Much like other forms of media, games come in uses “game-based mechanics, aesthetics, and
a wide array of forms and types, with different game thinking” to increase learner engagement,
features and characteristics. Because of this, motivation, learning, and problem solving
researchers, educators, and other practitioners (Kapp, 2012, p. 10). For instance, the concept of
have thus far struggled to agree on a precise and “unlockable content” is a game mechanism which
universal definition of “games”. One possible can be used in online learning modules to ensure
definition, suggested by Salen and Zimmerman learners have achieved a base-level understanding
(2006), describes games as “system[s] in which before they can move on to other content. The
players engage in an artificial conflict, defined value of gamification has been proven even
by rules, that results in a quantifiable outcome” among highly-skilled adult learners in continuing
Serious Games: Humanitarian User Research
(p.80). While broadly applied in the literature on professional development scenarios (Woolwine
games, this definition may extend to activities et al. 2019). Gamification can be useful in that
which are not considered a game such as trading it can often be less costly to incorporate game
stocks and debate; it also overlooks how games elements in current training than to develop a
might be cooperative. Others have defined games simulation or a serious game. That being said,
simply as a “structured form of play” (Flanagan gamification as a learning tool is mainly used by
2009; Yalowitz, 1995). However, this fails to
capture the complexity of games as compared to
puzzles or play-acting. Moreover, “play” implies
that fun is a defining characteristic of games,
which fails to encompass the serious games that
p. 5
generate meaning without being “fun”. We opt to
Serious Games: Humanitarian User Research
p. 8
professionals to enhance and regulate learning. scale such as learning the rules of airport security outcome such as a winner or loser, uncertainty, 1.1.2 ANALOG VS DIGITAL
Because it is not a fully developed game, it does (micro) to developing policy response to a global immediate feedback and assistance.
not come with some of its benefits such as an pandemic (macro). Games can be presented via a variety of media.
artificial environment under which players can In humanitarian training, simulations are Today, the most common games are digital
perform tasks by trial-and-error or engage in In this report and research, we are primarily commonly used to train and test staff skills. These (games played using an electronic device) and
large, dynamic interactions between multiple interested in the application of serious games to include applied emergency response activities analog (games that exist in the physical world).
people. Moreover, gamification as studied learning, with a specific focus on humanitarian such as search and rescue, safety procedures Digital games are delivered via a wide range of
so far is generally competitive (ie. inherently training in contexts of localization. in hazardous environments, or the deployment technological media, whether apps downloaded
non-cooperative) with perceived “winners” and of field medical equipment. Such simulations directly to mobile smartphones, web browser-
“losers”. It may therefore be less appropriate for Learning Games mimic reality via elaborate artificial settings based applications, computer games, console
training learners from organizational or ethnic and interaction with actors playing the role of games (such as Playstation, Xbox, or Nintendo), or
cultures which prize community, cooperation, Although used interchangeably, serious games injured persons, refugees, or kidnappers. The on other dedicated media. Analog games include
and consensus. can be differentiated from “learning games” in chief benefit of simulations is that simulations role-playing and tabletop games (games which are
that learning games aim to teach specific, often enable participants to behave almost exactly typically played on a table such as board games
Serious Games singular learning outcomes. Examples include as if they were “in the field” within a safe or card games) including Monopoly, Dungeons
“Muddy Motor Racing” which communicates environment without dangerous implications. and Dragons, and Poker.
“Serious” games differ from other games the imperfect driving conditions in many areas
by emphasizing knowledge production or where humanitarian services are delivered and
knowledge transfer over simple enjoyment and challenges individuals to come up with ways to
entertainment. While most games are valued for safely and efficiently dig the car out of the mud;
being “fun”, serious games focus on generation and “Immigration Nation” where players learn
of meaning beyond simple fun; this can include about immigration laws and what makes someone
education, analysis, the production of emotion, eligible vs. ineligible to enter and live in the United
the provoking of thought, etc. This is not to States. Learning games are a subset of serious
suggest that serious games and fun are mutually games. Learning games often possess a clear
exclusive, but that fun is not the primary goal and linear structure in which gamers progress
of the exercise. Serious games are often used through the material after demonstrating a
to educate, train, and inform the people who sufficient level of understanding.
play them.
In this research, we often interchange the
Serious games are often designed around a real- term “serious game” and “learning game”, This is advantageous as “on-the-job learning” is Is One More Effective Than the Other?
world problem, emulating an aspect of the real acknowledging that learning games are a subset considered the best method to consolidate new
world in an abstracted or simplified manner. Some of serious games. skills (Rumeser & Emsley, 2019), but can be very There is contention within the literature on
common applications might include projecting dangerous in high-risk environments. Similarly, games-based learning on differences between
a policy response during a crisis, workshopping Simulations it allows supervisors to identify individual staff analog vs digital presentation in facilitation of
supply-chain management for retailers or other members’ weak areas of knowledge so that learning. Kaufman & Flanagan (2013) tested the
businesses, examining military strategy in conflict, Simulations differ from serious games in that further instruction may take place following difference in impact of the same public health
or stress-testing emergency preparedness plans. the primary purpose of simulations is to provide the simulation. However, simulations cannot game in digital vs analog form. They found that
Serious game designers take a real issue and users with an experience that exactly models an be tailored to varying levels of capability. Thus, “that the digital version of the game, despite being
recreate it in a game setting so that users may aspect of reality such as a situation, scenario, beginners may find themselves unable to a nearly identical translation of the analog version,
“play” through the problem. “Serious games meet or environment. “A simulation is a dynamic and complete basic tasks within the simulation while proved significantly less effective at facilitating
their objectives by including rules, constraints and simplified model of reality and it is judged by more advanced users might find little challenge learning and belief change”. Cheng et al. (2015)
Serious Games: Humanitarian User Research
activities that closely replicate the constraints of its realism, by its correspondence to the system and a low-rate of failure. This prevents them similarly found that performance of reading
the real-world tasks that are being trained” (Ariffin which it represents” (Sauve et al. 2007). Serious from progressing past a certain point and acts comprehension (the desired learning outcome
et al. 2013). However, a good serious game is games, while similar, are often abstracted as an obstacle to further learning. Moreover, of their study) was significantly superior in the
more than applying game characteristics such as depictions of reality in which players reenact simulations, especially the highly realistic paper groups than the computer-based reading
scores and a narrative to a real-world problem. specific parts of an experience such as strategic ones, can be extremely expensive to produce groups. However, Cheng et al. (2015) also found
“It is the addition of pedagogy (activities that thinking, policy analysis, or procedure, without and maintain. Although research suggests that those with higher technological familiarity
educate or instruct, thereby imparting knowledge reproducing the experience as a whole, ie, in fidelity levels and knowledge transfer are not performed better - leading them to conclude that
or skill) that makes games serious” (Susi et. al physical space, with actual tools. Games maintain necessarily positively correlated (Feinstein and learning outcomes from digitally based media
2007). Serious games are not a particular genre essential attributes which separate them from Cannon 2002; Norman et al., 2002), this finding can be improved with sufficient training. It is our
of game. Serious games may come in the form a pure simulation, including: challenge, rules in itself undermines the perceived superiority opinion that the existing literature on digital vs.
of a role-playing game, puzzle, strategy game, that allow the determination of a quantifiable of simulations as tools for experiential learning. tabletop learning games favours tabletop gaming,
etc. Serious games address issues of different but we stress that it is not conclusive, and in
p. 7
1.2 WHY GAMES?
Serious Games: Humanitarian User Research
p. 10
particular fails to take into account the quality
of digital implementation and whether particular
learning goals might favour different media. In
addition, digital games impart other real-world 1.2.1 LEARNING THEORIES
benefits, such as quick global reach (where SUPPORTING THE USE OF GAMES
technology allows) and reduced reliance on a
facilitator. There is potential for reduced costs The increasing number and scale of complexity
if a digital game is carefully implemented and of multi-stakeholder issues in the humanitarian
properly scaled, but this is far from guaranteed; profession has given rise to new ways of thinking
see the following section for more on financial about knowledge dissemination and the need for
considerations. “a mutual learning loop framework that integrates
different learning theories, such as experiential
Financial Costs learning, adaptive management or transformative
learning.” (Lavell et al, 2015, p.1013) Through
There are different costs associated with the “real life” problem solving, serious games have
development and distribution of digital versus the capacity to help humanitarian students more
analog games. While digital games may be deeply understand and critically engage with
produced cheaply, poor-quality can suffer in a important issues. Experiential Learning Theory
multitude of ways: they may not be technologically and Situated Learning Theory help explain why
compatible, easy to learn, or even teach the this is the case.
intended lessons. Conversely, learning games can
be very expensive and time consuming to develop According to Experiential Learning Theory (ELT),
well. A good quality digital game can be expensive individuals learn most from direct experience,
to produce based on the inclusion of higher quality active participation, and visible feedback on
graphics, multiple players, and additional levels the consequences of their actions. Experience
and activities, notwithstanding the complexities provides the individual subjective meaning to
of developing an AI model capable of handling There is also a financial cost associated with in a structured debriefing session. Because abstract concepts and creates a concrete point
the permutations, nuance, and complexities of a playing digital versus analog games. A lack of analog games are more typically (although not of reference for testing the implications of ideas
humanitarian aid environment. This cost and time access to technology or applications necessary necessarily) facilitated by an instructor, including created during the learning process (Kolb, 2000).
requirement means that digital learning games to play a specific digital game (i.e. laptop, framing lectures, briefings and debriefing, Games foster experiential learning through
tend to be generalized to be broadly applicable smartphone, headphones, internet, a sufficient learning outcomes can be much more carefully features such as active participation where
to a wide range of contexts. graphics processing unit, flashplayer, etc.) poses shaped. players not only perform actions but directly
an obstacle to game play since there may be experience the effects of their choices. Games are
Analog games can be comparatively cheap and significant economic costs associated with Digital games, comparatively, may be played active forms of learning, as opposed to passive
quick to develop, and are simple to customize, acquiring these components. For instance, games alone. As such, they tend to offer a very lectures or videos, which do not require much
making them effective context-relevant learning designed for newer/state-of-the-art mobile predictable experience, easy for learners to action from the learner, aside from watching,
tools. For example, “print and play” games can phones, for example, do not function on older repeat if they are sufficiently motivated (as listening, and possibly taking notes. Situated
be transmitted electronically and easily printed or cheaper hardware. Thus, a target audience discussed previously in this report, repetition Learning Theory (SLT) likewise supports game-
and assembled in an office; all-in-one serious who cannot afford the newest iPhone will not can vastly boost retention of lessons). The option based learning as it posits that people learn better
game development kits help to quickly develop be able to partake in the game play. to run digital games alone, however, means that when placed in authentic contexts that parallel
scenario-specific games for any context and participants are not able to learn from each other real-world tasks, interacting with others rather
in any field office setting. Analog games tend Organizational Costs during and after the game. To our knowledge, than receiving knowledge that they are expected
Serious Games: Humanitarian User Research
to be more forgiving than digital games with there is no digital equivalent to an in-person to apply later. Games apply this theory by creating
respect to lower-quality game mechanics and Analog games often require the simultaneous debrief. In debriefing, lessons may be highlighted experiences that reflect real-life job issues,
components (Arnab et al., 2015), in part due to the presence of multiple players and a dedicated during peer discussion or with an experienced like how to manage employees with different
on-the-fly support of the facilitator. Component facilitator, so independent or take-home learning facilitator. Digital games are also less effective personalities, and allowing the learner the
costs for some analog games can prove to be a is not usually an option. Analog games are thus at modelling the dynamism of social or political opportunity to play through different situations.
barrier as well: while digital games are far more associated with greater coordination efforts and interactions in such a way that users can directly
expensive to develop, their distribution can be potential costs. observe and learn from them. The US Army, for
more cost effective--sometimes as simple as a example, has experienced this shortfall of digital
download--when compared to the component However, in-person multiplayer games allow games. Games such as UrbanSim have been
costs, manufacturing costs, and transport costs participants to learn from one another, both by created with much fanfare and large budgets,
for some analog games. directly observing how other participants react only to be shelved because they were found to
to decisions or actions, and sharing experiences be insufficient to meet training needs (Vogt, 2011)
p. 9Serious Games: Humanitarian User Research
p. 12
1.2.2 ENGAGEMENT: INTRINSIC involves many elements intrinsically related to a sufficient understanding is displayed or the do well and why:
VS. EXTRINSIC MOTIVATION the lesson. These elements which engage players emotional impact of losing the game reinforces
include: capturing and maintaining their attention, the learning outcomes better than traditional ◆ Decision-making: Decision-making in
People are more likely to learn when they are provoking thought and enabling curiosity, instruction methods and evaluations such as complex and high-stress environments is
interested in what they are learning (Garris et al., creating social connection, providing learners videos and quizzes. simulated by game mechanisms such as
2002). This is because they become intrinsically with an immediate sense of accomplishment and time pressure, competition, disruptors, and
motivated to learn--they are excited to learn for gratification, etc. Games’ unique characteristics Creating a safe-to-fail environment is useful not concentration of audio-visual stimuli. The
the sake of the lesson. According to Garris et are therefore particularly effective in harnessing only in training and preparing staff members numerous decision-making opportunities
al., intrinsic motivation (i.e. the learner engages individuals’ intrinsic motivations as compared more effectively, but is useful in developing presented in a single game enables players
in an activity because it is enjoyable) is found to other methods of teaching (Birk et al. 2016; better practices and new innovations. Games to practice and improve how they collect
to be a stronger driver of performance than Garris et al. 2002). make failure into a productive experience by and process information from their
extrinsic motivation (i.e. the learner engages in allowing players to take risks and deviate from surroundings.
an activity to satisfy a desired outcome, such regular procedure and try new things. The
as grades or financial gain). Intrinsic motivation 1.2.3 ENVIRONMENT: A SAFE SPACE TO FAIL ability to compare and contrast ideas, explore ◆ Retention: Repeated engagement with
encourages a higher degree of effort and longer- different methods, and elaborate on established interactive drill-and-practice games
term performance (Pinder 2011) as compared New employees inevitably make mistakes their approaches all encourages “outside-the-box” provides the repetition that may be needed
to extrinsic motivation; this is because extrinsic first time on the job, regardless of the extent of thinking. This process of experimentation for learners to memorize and retain certain
motivators (rewards that are tied to expectations preparatory training carried out in the classroom. is critical to successful innovation. In the types of content.
of gain or fear of loss such as benefits, money, In a humanitarian setting, these mistakes can humanitarian setting where problems are complex
stability, etc.) are motivating only to the extent have severe human costs in addition to other and continuously evolving, it is critical that ◆ Problem-Solving: In-game challenges
that an individual believes achieving the incentive undesirable outcomes such as negative social humanitarian practices effectively adapt with require students to think critically in
is useful and only up until the point that they experiences and fiscal loss. Because error is the issues. Games can provide the opportunity order to solve them and games’ use of
have attained the incentive. Individuals who are inevitable, it is beneficial to stakeholders in to test and develop new strategies to deliver, competition and rewards act as incentives
extrinsically motivated will thus be more likely humanitarian work that prospective employees be coordinate, and procure aid without posing harm to encourage students to overcome game
to put in the “bare minimum” in a bid to receive able to make mistakes in an environment where to relevant beneficiaries or stakeholders. problems.
their desired reward as quickly as possible. In a the impact of their errors will be mitigated. Games
humanitarian training setting, this may present in this case are an ideal training tool because they ◆ Collaboration: Games in which there are
itself as a trainee learning enough material to are able to artificially recreate aspects of the real 1.2.4 EFFECTIVENESS: PROVEN TOOLS teams or it is necessary to collaborate with
achieve the minimum score to pass an evaluation. world, recreating to some degree the environment FOR LEARNING AND ASSESSMENT other players in order to succeed can foster
Contrastingly, when individuals are intrinsically in which employees will be working. Thus, when and develop collaborative skills such as
motivated, they are more likely to fully endorse participating in a learning game, mistakes that Serious games have been demonstrably effective task coordination, clear communication,
and participate in the task and will persist in staff might otherwise have made while on the learning tools. While this is not an exhaustive list, and compromise.
performing the task beyond the point at which job can instead be made (and learned from) in games are found to be particularly effective in:
they are rewarded. This also increases the quality advance. Moreover, the feedback from the game promoting skill acquisition, knowledge retention, ◆ Empathy: Role-play in games allows
as well as quantity of what has been learned. allows instructors and students alike to identify attitudinal change, supporting understanding players to inhabit the lives and vicariously
and correct gaps or errors in students' knowledge. of new concepts and ideas, shaping behaviour, experience the disadvantages or
The method by which instruction is delivered has and improving context-based problem solving persecution faced by other people or
been shown to elicit student engagement during The capacity to safely fail in serious games is a (Klabber, 2003; Mateas, 2003; Prensky, 2001; groups. This can induce empathy by
training (Garris et al. 2002). In recognition of this, key component of their value as a learning tool. Ricci, Salas, & Cannon-Bowers, 1996). Games are challenging misconceptions; providing
organizations from a wide variety of sectors are Failure has been identified as an integral part also considered an effective form of assessment previously unrealized information; and
increasingly incorporating serious games in their of learning (Anderson et al. 2018). Sitkin (1992) as they require application of knowledge (as arousing emotion through immersive
training programs to help stimulate intrinsic argued that “failure typically represents an opposed to memorization and recital of facts) experience, encouraging players to relate
Serious Games: Humanitarian User Research
engagement toward company education. The exception that does not conform to expectations and provide real-time feedback (i.e. game to the group whose perspective they are
rationale guiding this new method of instruction and thus requires more active, deeper processing” progress, losing, in-game scores, etc.) on how embodying.
is that the feelings of excitement, challenge, and (p. 237). This is because when we encounter well knowledge is applied and skills are enacted
achievement which trainees experience while experiences that do not fit our preconceptions - allowing instructors to measure the level of an Although there is widespread consensus on
playing a learning game will encourage an intrinsic (i.e. trying something and failing), we are forced individual student’s ability. the anecdotal benefits of serious games for
motivation to learn, inspiring deeper engagement to adapt our understanding to account for learning, empirical research is limited. Conrad
with the material. Characteristics of games gaps in our knowledge. This process of critical Attributes of games that contribute to learning (2010), Connolly et al. (2012), and Marcos et al.
which make them engaging include competition, reflection has a stronger psychological impact include active participation, immediate feedback, (2016) highlight that there is insufficient research
challenge, rewards, pleasing aesthetics, surprise, than having gotten it right the first time. Games dynamic interaction, cultural context, competition, evaluating the effectiveness of games in learning.
etc. Part of the draw to learning games is the in particular promote better learning from the exchange of tacit and explicit knowledge, The majority of claims advocating games-based
simple expectation that games are fun, however, failure than other instructional methods. Game novelty, anthropomorphism, and goal direction. learning are based on anecdotal evidence of
p. 11
the sense of engagement that learners experience mechanics such as having to replay a level until Below is a list of learning outcomes that games their effectiveness and instructor judgement. The1.3 EFFECTS OF GAME DESIGN
Serious Games: Humanitarian User Research
p. 14
lack of evidence for the effectiveness of games designs where education greatly supplants game clarifying issues, and resolving any conflicts
for training purposes is further compounded ON GAME-BASED LEARNING aspects, users often find the game unplayable. For among participants. If an unseen design flaw in
by the lack of a well-developed methodology example, in their 2008 study, Virvou & Katsionis a tabletop game begins to result in unrealistic
for evaluating the effectiveness of games. The For serious games to achieve their intentions, found that not liking the game or finding the outcomes, an experienced facilitator can easily
assessment of serious games is largely hindered serious attention must be paid to game design. game unusable were fundamental stumbling “correct” the flow of the game via injects to re-
by their complexity, the difficulty in measuring Pre-existing empirical research indicates that blocks to learning. Issues related to likeability establish a user experience which mirrors that of
intangible variables (i.e. engagement), and a lack serious games are only effective if they are and usability such as poor game aesthetics, real life and demonstrates the desired learning
of organization among a fragmented community designed with specific instructional objectives in monotonous tasks, etc. can be alienating to outcomes.
of various stakeholders. In our review, a significantmind and the features of the game encourage players and undermine the key factors supporting
portion of the available studies evaluating the the achievement of those objectives (Hodent, game-based learning such as engagement and
effectiveness of games as learning tools used 2018). Features such as user interface design, motivation. Clearly it is necessary to strike a 1.3.3 EFFECTS OF PLAYER CHARACTERISTICS
a simple research design which produced teaching strategies, and the format and content balance between learning theory, game design,
subjective and short-term results such as asking of educational material influenced by the game and subject matter expertise. One of the most important considerations in
participants if, in their opinion, they had learned developers. The choices made by the game game design is the makeup and complexion of
something. Many studies lacked rigorous testing designer will have implications for the degree of the game’s target audience. Player characteristics,
methods such as a control or another group effectiveness of the game as a learning tool. Our 1.3.2 DEBRIEFING & FACILITATION generational and cultural differences, and
of comparison, pretests, and testing against review of game design as it affects the educational individual learning styles are just some of the
their declared objective. Our assessment of value of games finds that the major issues concern A school of thought posits that learning from factors that can affect players’ receptivity to any
the literature is corroborated by the findings of the apparent difficulty in balancing pedagogy games is only effective - or at the very least, given serious game.
Wangenheim & Shull (2009), Hayes (2005), and and entertainment, inadequate debriefing and more effective - with instructor support and
Bellotti et al. (2013). facilitation, and neglecting diverging player debriefing (Crookall 2014; Kolb, 2008; Kris, 2003; Familiarity with Games
characteristics. Effective games for training Thatcher, 1990). It is unreasonable to assume
Available research suggests that games are should be developed with their specific learning that no learning whatsoever occurs during the The effectiveness of games for educational
poor standalone learning tools. Learning objectives and target audience in mind. game itself. Rather, learning is strengthened purposes are affected by the playing audience’s
outcomes are most significant when games are and becomes more meaningful through the familiarity with games. In several studies,
used to complement, not utterly replace, other debriefing process. Debriefing is the process of participants who were considered inexperienced
instructional methods (Sadler et al. 2015; Schmitz 1.3.1 EDUCATION VS ENJOYMENT reflecting and discussing the game experience gamers did not learn as well as more experienced
et al. 2015; Hayes 2005; Virvour et al. 2005). to turn into learning. Participants often have g a m e r s ( R a v s y s e e t a l . , 2 0 1 6 ; V i r v o u a n d
Randel et al. (1992) reviewed 67 studies comparing T h e b i g g e s t c h a l l e n g e o f s e r i o u s g a m e s a limited picture of what happened; while Katsionis, 2008). There are several reasons for
the instructional effectiveness of games versus development is the integration of learning content playing the game, participants predominantly this, including: a dislike of games in general,
conventional classroom instruction. They found and pedagogy with core game mechanics. The observe only what their position allows them confusion leading to alienation, and an inability
that 56% showed no difference and 32% found prevailing inability to achieve a cohesive balance to. Post-game debriefing and reflection sessions to play the game properly. However, studies have
differences favouring games. Wangenheim & between enjoyment and education has limited help elucidate the learning material and place found that clear and concise instructions, simple
Shull (2009) reviewed some 21 studies in serious the effectiveness of games as learning tools. For the game-learning experience into a greater user interfaces, and lower game complexity
games for software engineering training and instance, in designs where the game mode is context. Serious games which address complex, can resolve issues that arise from non-gamers
found that a majority had a noticeable but a dominant, games can cause defective learning. multi stakeholder problems may trigger strong (Couceiro et al. 2013).
minor impact on learning. Hays (2005) review In our review, we found that where games were emotional reactions, provoke conflicts between
of 274 papers on instructional effectiveness of designed with an emphasis placed on being a “fun participants or misunderstandings. Finally, Generational & Cultural Differences
games likewise found that games can be effective game”, learning outcomes were either found to debriefing ensures all participants exit the
for learning specific subjects and skills such as be wrong or missing altogether. For example, in a session with the intended knowledge. It allows Generational and cultural differences can affect
math and collaboration, the majority of studies history simulation-game The Oregon Trail, Caftori instructors to identify missed, weak, or false willingness and understanding (technological
found no significant difference between games (1994) found that the educational objectives of learning outcomes based on the participants’ literacy) in terms of learning through games.
Serious Games: Humanitarian User Research
and conventional teaching. the games were missed by students because the self-reflection on what they have learned Older and younger workers “do not respond the
incorporated game design elements, such as from the game. At times, participants may not same way to training” (Grossman, 2008, p. 43)
One of the key limitations of games is the scores based on the hunting of animals and a time themselves be aware of the knowledge or skills and, as a result, game designers should build
inevitable simplification of circumstances, which is pressure to finish the trail, did not align with the that they have acquired. The debrief provides games to be as accessible as possible to a wide
inevitable in the recreation of complex scenarios. games’ learning objectives of teaching users about trainers with the opportunity to reiterate and range of people; it is not enough to assume that
Not all elements can be incorporated or recreated the terrain and wildlife on the Oregon Trail. It is reinforce desired learning outcomes. Likewise, players will be intrinsically motivated by the
and attempting to include too many details can be not enough to embed attractive characteristics instructional support helps learners to use the label of “game” or the promise of “fun” alone.
overwhelming for a player and result in making in the game because these can easily become game effectively, enabling gamers to focus on the Furthermore, little attention is given to issues
the game too convoluted to play. diversions from the real goals. It is necessary to instructional information rather than the rules of cultural diversity in serious games. Rasyse et
emphasize important information in the game and requirements of the game (Hays, 2005). The al. 2016 reviewed 45 serious games for disaster
and choose game mechanics that ultimately presence of a facilitator precludes a negative risk management and their respective literature
p. 13
support the instructional objectives. However, in experience by offering space to air emotions, and found cultural implications were distinctly1.
Serious Games: Humanitarian User Research
p. 16
absent; only a few mentioned “bridging the gaps there is extensive literature on the importance Telling ‘the story’: there is a tendency for understanding the complexity of
between people of different cultural and linguistic of engaging ethically in humanitarian work, in humanitarian response work to the topic and in terms of identifying
backgrounds to better manage risks”. This lack both for researchers and practitioners in represent refugees’ individual life stories solutions or learning particular
of cultural competency in game design can have the field (see for example, Clark-Kazak 2017; as a singular experience, essentializing skills. However, it is vital that serious
a significant impact on the player’s receptivity Hyndman, 2000; Malkki 2013). In her seminal the individual as a victim in a position games that engage with humanitarian
towards the method, as well as its effectiveness. piece “Can Humanitarian Work with Refugees of dependency. The foil of the unified contexts are centered on principles
Be Humane?” Harrell-Bond (2002) refers to the ‘refugee story’ is the solutions-based of respect and dignity, especially
need for a “rights-based humanitarianism” that humanitarian organization. When when representing people receiving
moves beyond the charity model and upholds designing and implementing serious humanitarian interventions. Ideally,
1.4 SERIOUS GAMES IN
the dignity and rights of the communities they games in humanitarian contexts, it is voices of members of the affected
HUMANITARIAN CONTEXTS hope to serve, moving away from the ‘victim’ important to problematize this binary. community should be directly involved
narrative framing of so many organizations. Who is telling the story, and to what in the development of the simulation.
Serious games remain an underused tool in These academic principles have recently been end? What mistakes do humanitarian
humanitarian training, especially when compared enshrined in the Sphere Standards and Core
to other fields dealing with similarly complex multi- Humanitarian Standard (2018).
stakeholder problems such as peacebuilding,
organizations make, and how can serious
games contribute to the identification,
description, and reduce the frequency
2. Avoid abstraction of voiceless
stakeholders: it is common in serious
game development that “voiceless”
medicine, disaster risk management, and security. The application of ethical principles in the and severity of those mistakes? Serious stakeholders are abstracted as a
Where serious games have been applied in the development of serious games should be games have the capacity to engage mechanical function. For example,
humanitarian context, they are more frequently foundational to their design, especially in the with the complexities of different in a military learning game on the
employed as advocacy and awareness-building context of humanitarian work where wildly experiences and consequently offer functioning of provincial reconstruction
tools for public consumption rather than training disparate power dynamics between ‘stakeholders’ up more nuanced solutions.Dignified teams in Afghanistan, the only played
tools. These advocacy games tend to be targeted and ‘beneficiaries’ already exist. Lessons Learned representation: unlike a purely fictional roles are military, government, and
at potential small-scale donors rather than Simulations & Training, which primarily works game, serious humanitarian games are humanitarian actors; local populations,
humanitarian workers themselves, and focus in refugee-response contexts, has developed a based on real-life and often ongoing displaced people, and militants are
on the building of general empathy and “feel- series of principles to consider in the design and crisis scenarios that are lived by real represented via the “flip of a card”
good” interest in humanitarian projects rather delivery of serious games with respect to forced people. There are clear benefits to (Mason and Patterson, 2013). While the
than specific learning about contexts or skills. displacement: experiential learning experiences, both game successfully teaches participants
Few serious games have been developed for about some stakeholders in the Afghan
or applied to humanitarian response, with little conflict, it fails to explore those who are
public data available on design, application, inarguably the most impacted by the
educational effectiveness, lessons learned, or ongoing violence. Abstracting voiceless
best practices. actors encodes their liminality into the
games’ approximation of reality. Not
only does this reinforce the assumption
1.4.1 ETHICAL CONSIDERATIONS that they are unimportant, it presents
these actors as irrational, unknowable,
Any discussion of serious gaming in or random, and misses an extremely
humanitarian contexts must be prefaced with valuable analytical opportunity to
an acknowledgement of the ethical considerations understand a scenario from that
of such a venture. Implementing serious games perspective.
on subjects of human or natural disasters, and
the resulting depiction of suffering, poverty,
3. Biases and assumptions: there is a risk
Serious Games: Humanitarian User Research
liminalization, exclusion from the national order, of ‘encoding’ biases and assumptions
and all associated hardships requires a deep into the design and implementation
sensitivity towareds the context. In this sector, of simulation exercises. These biases
we must be explicit in the difference between can be represented in a narrative
“gaming” for fun and serious games as powerful sense (how roles are described) or
training tools. Serious humanitarian games must a mechanical sense (how the game
be executed with a strict adherence to respect functions). Mechanical biases, where
for the people and communities affected by the the functioning of a serious game
topic being simulated. is internally coherent but does not
accurately represent the real-life
While there is little written about ethical process, is particularly dangerous as
p. 15
considerations in serious humanitarian games, participants can leave the game havingYou can also read
