Games as Neurofeedback Training for Kids with FASD
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Games as Neurofeedback Training for Kids with FASD
Regan L. Mandryk1, Shane Dielschneider1, Michael R. Kalyn1, Christopher P. Bertram2, Michael
Gaetz2, Andre Doucette1, Brett A. Taylor1, Alison Pritchard Orr2, Kathy Keiver2
1 2
University of Saskatchewan University of the Fraser Valley
Saskatoon, SK, Canada, S7N5C9 Abbotsford, BC, Canada, V2S 7M8
1-306-966-2327 1-604-504-7441
{firstname.lastname}@usask.ca {firstname.lastname}@ufv.ca
Figure 1. Columns show low, medium, and high levels of texture-based biofeedback. Rows show customizations of the same effect
for two different games: top) Static Sprite (cracks) over Portal 2, bottom) Static Sprite (mud) over Nail’d.
ABSTRACT Keywords
Biofeedback games help people maintain specific mental or
physical states and are useful to help children with cognitive Biofeedback, neurofeedback, games, FASD, ADHD.
impairments learn to self-regulate their brain function. However,
biofeedback games are expensive and difficult to create and are 1. INTRODUCTION
not sufficiently appealing to hold a child’s interest over the long Fetal alcohol exposure is the most prevalent cause of intellectual
term needed for effective biofeedback training. We present a impairment in the western world [17]. An accurate account of the
system that turns off-the-shelf computer games into biofeedback incidence of fetal alcohol spectrum disorder (FASD) is unknown
games. Our approach uses texture-based graphical overlays that but estimates range from 3 per 1000 live births to 10 per 1000
vary in their obfuscation of underlying screen elements based on children being affected by prenatal alcohol exposure [10], which
the sensed physiological state of the child. The textures can be translates to thousands of affected infants born each year in
visually customized so that they appear to be integrated with the Western Canada [3]. Children with FASD are often also
underlying game. Through a 12-week deployment, with 16 diagnosed with Attention Deficit Hyperactivity Disorder (ADHD)
children with Fetal Alcohol Spectrum Disorder, we show that our [2]; using biofeedback (BF) to train brain function self-regulation
solution can hold a child’s interest over a long term, and balances has been effective at reducing the symptoms of ADHD, and at
the competing needs of maintaining the fun of playing, while reducing differences of ADHD children from normative databases
providing effective biofeedback training. of elecroencephalography (EEG) [6,9].
Biofeedback training systems encourage a specific mental or
Categories and Subject Descriptors physical state in a user through a closed biofeedback loop. These
H5.2 [Information interfaces and presentation]: User Interfaces. - systems gather a child’s physiological state through sensing
Graphical user interfaces. hardware, integrate this state into a computer-based interactive
system, and present the feedback so that the child can work to
adjust their state. Biofeedback training systems often use games
Permission to make digital or hard copies of all or part of this work for for interaction because playing games is intrinsically motivating
personal or classroom use is granted without fee provided that copies are for most children. Biofeedback games work by altering the game
not made or distributed for profit or commercial advantage and that mechanics (i.e., rules and procedures) based on the child’s
copies bear this notice and the full citation on the first page. To copy physiology; however, traditionally, biofeedback games have not
otherwise, or republish, to post on servers or to redistribute to lists, been engaging enough to hold a child’s interest over the repeated
requires prior specific permission and/or a fee. sessions needed for effective training [14].
Interaction Design and Children’13, June 24–27, 2013, New York City,
New York, United States.
Copyright © 2013 ACM 978-1-4503-1918-8…$15.00.
Creating engaging biofeedback games remains difficult because motivating for many users and will encourage participation,
the game’s mechanics must be altered to create the biofeedback potentially resulting in improved training compliance [14];
loop. This means that each biofeedback game is a custom however, many biofeedback games (e.g., [20], [21]) might be
creation, which is both expensive and time consuming; choosing better described as interactive systems because they lack the
to play off-the-shelf games as biofeedback games is simply not uncertain and quantifiable outcome of a game [15].
possible. As a result, biofeedback games have two main problems.
First, they tend to be toy applications that don’t hold a child’s In biofeedback games, players must maintain a particular
interest in the long term, which is a problem because biofeedback physiological state to make progress, generally accomplished by
training requires repeated exposure to yield successful results adjusting the game’s mechanics (i.e., the rules and procedures of
[19]. Second, a child who wants to play a biofeedback game has play). For example, in a bowling game, the ball might roll toward
little choice over the game genre, and may not be motivated to the gutter as the player becomes stressed. The player responds by
play a game from a small selection with little appeal. self-regulating their sensed physiological state in the desired
direction (see Figure 2–left). Note that biofeedback games are not
In this paper, we present a system that turns off-the-shelf the same as ‘brain-training games’. Although both claim to help
computer games into biofeedback games. We propose to close the users improve cognition, biofeedback games require a closed loop
biofeedback loop by altering display graphics instead of game of physiological sensing and real-time feedback; brain-training
mechanics. We present a graphical overlay on top of a running games are mental exercises made fun with game-like elements.
game that obscures the underlying game when the child is not in
the desired physiological state (see Figure 1). Our system works 2.1.1 Commercial History
with off-the-shelf games, so children can choose games that they Industry manufacturers have investigated biofeedback gaming
like. The amount of obfuscation varies parametrically in real time, since the 1970s, when Thought Technology created CalmPrix1, a
and is driven by physiological sensors. Our graphical overlays can racing game that came packaged with an Apple II mouse modified
be chosen from an all-purpose set or be customized to be with galvanic skin response (GSR) electrodes. Other biofeedback
consistent with the visual style, theme, or genre of the game, so gaming systems include the unreleased Atari Mindlink2 in 1983,
that they appear to be integrated with the underlying game. In The Journey to Wild Divine3 in 2001, and the Nintendo 64
addition, the graphical effects are consistent with current abstract biosensor4 included in the Japanese version of Tetris 64 in 1998.
in-game visualizations that players are already familiar with using In 2005, Smart Brain Technologies5 released a home-based EEG
(e.g., tunnel vision representing poor in-game health). system. Recent biofeedback game devices include two popular
Because our solution is a novel approach to biofeedback, we consumer-grade EEG devices – the NeuroSky Mindset6 and the
investigate three main questions surrounding its efficacy. First, Emotiv EPOC7. These devices measure EEG via electrodes that
does altering display graphics instead of game mechanics work as are held in place by a headset; both include a variety of games. In
a biofeedback mechanism? Second, do graphical overlays ruin the particular, the NeuroSky features Focus Pocus8 – a 2012
fun of playing games? Third, can our system remain motivating biofeedback game designed specifically for children with ADHD.
over the long term needed for successful biofeedback training?
We answer these questions by deploying the system over 12 2.1.2 Academic History
weeks with 16 children with FASD, a target demographic for Early academic research in biofeedback games did not intend to
brain-based biofeedback training. We conclude by discussing how create therapeutic systems, but to create compelling play
our approach can be applied to interaction other than games, to experiences (see [12] for an overview). Sensed physiology can be
open biofeedback training to those who are uninterested in games indicative of a user’s emotional state, and if used to manipulate a
or who would prefer to integrate it into their day. play experience, could create engaging affective games [5]. Relax-
This research presents the first general solution for turning off- to-Win (2001) [1] involved racing dragons whose speed was
the-shelf software into biofeedback systems where the user controlled by GSR; BalloonTrip (2003) [16] also used GSR as a
chooses which physiological trait to train. We focus on games, game control; Brainball [8] is a game where a user’s EEG controls
leveraging the millions of dollars and years of development that a physical ball rolling on a table. There are also examples of
go into triple-A titles, and ensuring an engaging play experience systems that use software development kits to integrate
that will hold a child’s interest. Our low-cost system provides the biofeedback with existing games. In AlphaWoW [12], players
opportunity for biofeedback training – previously only available trigger their shapeshifting ability through EEG; in [4], some game
in clinics – to children directly in their homes. mechanics in Half-Life 2 (e.g., enemy spawn points, screen
shaking) are controlled with heart rate; and in AffQuake [13], the
player’s avatar jumps when startled and grows with player
2. RELATED WORK excitement. Finally, there are many examples of heart-rate control
Biofeedback training has been used to help patients with in games for the purpose of improving physical fitness (see [18]).
Asperger’s Syndrome [20], to reduce the frequency of seizures in
patients with epilepsy [6], and to improve the behaviour of These systems all use indirect physiological control [11], where
children with ADHD [9]. There is also evidence of successful players must work to change their physiological state through a
biofeedback training for children with tic disorder, autism,
schizophrenia, and learning disabilities (see [6]). In healthy 1
http://www.thoughttechnology.com/thewall2.htm
individuals, biofeedback has been used to improve working 2
http://www.atarimuseum.com/videogames/consoles/2600/mindlink.html
memory and attention [21]. 3
http://www.wilddivine.com/
4
http://nintendo.wikia.com/wiki/Bio_Sensor
5
http://www.smartbraintech.com/
2.1 Biofeedback Games 6
http://store.neurosky.com/products/mindset
Instead of providing biofeedback through simple graphical 7
http://www.emotiv.com/
8
feedback, games are used because they are intrinsically http://ballantinespr.com/News/NeuroSky/NeuroSky_Focus_Pocus.html
mediating mechanism. For example, to make the dragon race making it less enjoyable to play, and potentially impossible to
faster in Relax-to-Win, a user must reduce their GSR by relaxing. progress if there is enough obfuscation. Similar to traditional
Indirect control is also the principle used in biofeedback games; approaches, we vary the feedback depending on the user’s state;
however, the games presented here were not developed as the textures have different obscuring parameters (e.g., opacity,
therapy, but as novel systems that provide new forms of play. position, coverage) that vary continuously along a scale, providing
varying levels of obfuscation of the game display. Players want to
2.1.3 Biofeedback Games in Practice play with as little obfuscation as possible (see Figure 1),
Although biofeedback games used in clinical practice resemble motivating them to maintain the desired physiological state.
the simple systems presented here, they have seen some clinical Figure 2-left shows the traditional biofeedback loop, completed
success. Biofeedback games have been used to help people through modifying game mechanics. This approach requires that
manage stress, relax, focus, and manage anxiety see [6], and have each game be custom designed specifically with the biofeedback
seen particular success when with special populations – for loop in mind. With texture-based biofeedback (see Figure 2-right),
example, to help children with ADHD and anxiety learn to texture overlays obscure the screen to complete the biofeedback
manage their symptoms [6], or to help children with pelvic floor loop. This approach is agnostic to the underlying game, requiring
dysfunction improve voiding dysfunction [7]. no access to source code, thus can be used with any off-the-shelf
In research closer to ours, Pope and Palsson [14] created a game. In addition, the user’s physiological state is integrated into
hardware solution5 that worked with Playstation games by altering our system rather than into the game itself, so any physiological
the performance of a game controller based on a user’s EEG. In a system can be trained, regardless of game choice.
study comparing their game-based system to a traditional Our system has three main components; the biofeedback game
biofeedback system with children with ADHD, the authors found interface accesses two libraries: one senses a player’s
that both approaches resulted in improvements, but that children physiological state and the other renders the textures to the
and parents were happier with the game system [14]. Like our display. In the following sections, we describe these components.
research, the authors were interested in leveraging the motivation
provided by off-the-shelf games; however, our approach is not
limited to games played with a controller or on a Playstation, uses 3.1 Physiological-Sensing System
a software solution that is agnostic to the underlying game (their Current biofeedback games (e.g., Focus Pocus) integrate the
solution affected performance in the game), and is not limited to a sensor with the game mechanics, leaving users no decision over
single physiological sensor (they train a specific band of EEG). game choice or what aspect of their physiology they wish to train.
Our system separates the selections of sensor and game.
3. TEXTURE-BASED BIOFEEDBACK Our physiological sensing system is managed by a custom library
Biofeedback systems have two general requirements. First, they called SensorLib, which is a multi-threaded library written in C#
must sense a user’s physiological state; and second, they must that provides an interface for external third-party sensors. It
provide this sensed state to the user through a feedback handles the connection to the sensors, the data-buffering, digital
mechanism. Both of these requirements should occur without signal processing, and offers the data through a high-level .NET
delay, as close to real time as possible. Our biofeedback system interface. SensorLib aggregates third-party software development
had two additional requirements. First, as we wanted to engage kits (SDKs) into a single interface for ease of programming; new
children over the long term, our system had to work with off-the- sensors are easily added if the signal can be accessed via an SDK.
shelf games. Second, the computational resources needed to run
the system should not affect the performance of the game. 3.2 Texture-Rendering System
Our texture-rendering system (TextureLib) was built in C++ using
Microsoft DirectX 10 graphics libraries, and displays an overlay
window over any application. The overlay window is rendered
over top of other windows even when it does not have focus, is
transparent, and allows keyboard and mouse events to pass
through it so interaction still occurs with the applications running
“behind” the overlay. TextureLib renders visual representations in
the transparent overlay by making use of DirectX resources (DDS
files) and pixel shaders written in High Level Shader Language
(HLSL). Our software requires Windows 7 and a video card that
supports DirectX 10 (common in gaming computers).
3.2.1 Visual Appearance of Textures
The visual representations that we use are made up of an effect (a
Figure 2. Biofeedback loops: Left – Traditional loop; Right – pixel shader), and any number of resources (textures and
Our revised texture-based loop. colormaps) that can be edited using image editors. TextureLib
contains five pre-packaged effects that can be customized.
Together, these four requirements inspired our use of texture-
based overlays, rendered in real time in a transparent overlay on Tunnel Vision creates a semi-transparent texture with a definable
top of a user’s primary task of playing a game. Traditionally, encroachment area on the screen. The location and size of this
biofeedback games work by not allowing the user to progress area, the fade-in threshold for the texture to become opaque, and
unless they are in the desired physiological state. In our case, the the texture colour can be controlled. Note that the next four effects
textures obscure the graphics related to the user’s primary task, can be applied to Tunnel Vision, combining the effects.
Figure 3. Columns show progression of low to medium to high levels of texture-based biofeedback. Rows show customizations of
effects: 1) Tunnel Vision (vines) over Up, 2) Tunnel Vision (veins) over Hulk, 3) Fractal Noise (mist) over Homecoming, 4) Fractal
Noise (fire portal) over World of Warcraft.
Fractal Noise uses a noise texture to render semi-transparent with the theme of the underlying game. For example, rain falling
textures. Multiple octaves of a noise texture (e.g., Perlin noise) are in a golfing game may contribute to a better gameplay experience
used for variation. The colour and the opacity can be controlled. than using that same effect in an ice hockey game, where the
concept of rain inside an arena makes little sense. By visually
Waves fills the screen with drops that generate ripples. Generated customizing the effects, our system can appear to be integrated
with a 2D wave simulation, the resulting height field is rendered with the underlying game. However, the appearance of the effects
with specular lighting. The size, frequency, and coordinates of not need be as literal as falling rain on a golf course to appear to
drops and the size and decay of the ripples can be controlled. be integrated with the game. Effects that relate to the theme,
Static Sprite renders static 2D image sprites. The number, narrative, or world of the game may also contribute to a good
starting position (x, y coordinates), speed, acceleration, rotation experience. For example, a fiery portal that grows and shrinks to
speed, and size of the sprites can be controlled. In addition, reveal the underlying display may be effective for use with a
particle system parameters for sprites can be specified to create fantasy game, even though it has no literal meaning in the game.
visual representations such as explosions. Customizing the textures is a process can be done by a developer,
Animated Sprite renders animated 2D image sprites using a but also could be done by an end user with no programming
sprite sheet. The number, starting position, speed, acceleration, experience. Simply substituting a different image file or editing an
rotation, and size of the sprites can be controlled. image with standard image editing tools can customize existing
effects. For example, this approach can be used to change an
effect of mud splatters into slime splatters. We customized the
3.2.2 Game-related Texture Customizations five effects included in TextureLib to demonstrate a wide range of
Although our system can use any visual effect to provide visual appearances (see Figures 3 and 4). In the following
biofeedback, we feel that the experience of biofeedback games examples, we used each effect twice, with different graphical
might be improved if the graphical feedback is visually consistent resources, to show how the appearance can be changed.Figure 4. Columns show progression of low to medium to high levels of texture-based biofeedback. Rows show customizations of
effects: 1) Waves (droplets) over Spearfishing, 2) Waves (frost) over NHL ‘11, 3) Animated Sprites (spiders) over Crysis, 4)
Animated Sprites (particles) over NetRumble.
We used the Tunnel Vision effect to create vines, which grew over more generic (e.g., a flowing mist). Generic effects can be treated
the jungle setting in an adventure game, in a fairly literal as a separate interface element, specific to the biofeedback
customization. We also used the effect to generate pulsing veins, system. In this case, the textures would not be customized to
which we deployed over a game based on the Incredible Hulk, in match the game being played; however, our system would still
a less literal, but still thematically consistent visual representation. have the advantage of operating with off-the-shelf games. Casual
Fractal Noise was used to create a mist effect over a survival games are designed to be quick to set up and fast to play,
horror game, and a fiery portal over a fantasy roleplaying game. involving play times of only a few minutes. With a generic effect,
These effects are visually different, but require only a few a user could switch games a number of times within a single
changes to the graphical resources and parameters. We used the training session using our system; the visual effect would stay
Waves effect to create water droplets over a spearfishing game in present between games, and the biofeedback experience would be
a literal customization of the effect. This effect was also used to seamless throughout the training session.
create a frost effect, which we deployed over an ice hockey game.
We used the Static Sprite effect to create cracks that appeared in For more advanced users, parameter values of existing effects can
the screen over a first-person shooter game with puzzle-solving be programmatically changed at run-time by a client application.
aspects, where the world involves a lot of glass walls. This effect For example, this can be used to change a mist effect into a smoke
was also used to create mud splatter on the display in an outdoor effect by changing the colour and opacity values. Also, developers
racing game. Finally, we used the Animated Sprite effect to render can create new effects by implementing their own shaders.
spiders crawling over the screen during play of a first-person
shooter game that takes place in a jungle setting, and particle 3.3 Biofeedback Game Interface
explosions in a 2D space shooter game. The biofeedback game interface gathers the user’s physiological
state from the physiological sensing system (using SensorLib),
Some of the effects we created are fairly specific to certain games
and renders textures in the overlay (using TextureLib)
or environments (e.g., spiders on the screen), while others are
corresponding to the user’s state. Any sensor in SensorLib can beused to drive the biofeedback training, while any effect that can be Games. We provided a selection of games from which players
created with TextureLib can be used as feedback. To use a sensor could choose. Games were required to run in windowed mode,
not currently available, SensorLib would need to integrate the maximized to the screen (rather than full screen mode) to allow
sensor’s SDK, whereas creating a new visual effect does not the overlays to display properly. In addition, games could not
require changing TextureLib, but rather using the built-in contain objectionable content, including violence, sexuality, or
customization tools. To work with off-the-shelf games, our harsh language. Games were accessed via Steam, and initially
interface must be agnostic to the user’s underlying task, so the included NBA 2K10, Osmos, World of Goo, Bejeweled Twist,
selection and launching of the game occur outside of our interface and Blur. Part way through the deployment, we added additional
using the standard Windows interface. games, including Capsized and Plants vs. Zombies.
We created a simple graphical user interface that allows users Training. Users trained two or three times per week, in a research
control over their biofeedback training. A commercial system lab at the University of the Fraser Valley, with our system for 12
designed for an end user would hide some features; whereas, one consecutive weeks between October 2011 and April 2012. Each
designed for a trained biofeedback clinician would include more session lasted about 60 minutes, with 30-45 minutes of game play.
control. Our interface was developed for use in a series of Users chose which games they wanted to play; however, all
experiments, and resembles the features that would be presented players used the mist effect at obfuscation levels set by the
to a clinician. Users can select which aspect of their physiology experimenters, based on a standard pre-play calibration procedure.
they wish to train (e.g., range of EEG), determine normalization Obfuscation levels were set at the beginning of each training
values for the effect from a calibration procedure or accept values session so that thresholds could adjust with the players over time.
entered manually, preview and choose the effects and the
obfuscation levels, and view the signal strength for the training Measures. We asked users to fill out a survey with questions
hardware. In addition, we included an interface for saving log files related to the play experience after 12 weeks of training. All
for further processing throughout the evaluation of our system. questions were answered using a 3-point scale (yes-maybe-no)
suitable for the age of our users. In addition, we processed the log
files for each training session. EEG data was logged every 500ms,
4. TEST DEPLOYMENT and these data were aggregated over each session.
Because of the novelty of our approach, we were interested in
answering three questions through a test deployment:
4.2 Results
1. Does altering display graphics instead of game mechanics Does altering display graphics instead of game mechanics work
work as a biofeedback mechanism in games? as a biofeedback mechanism in games? Users generally agreed
2. Do graphical overlays ruin the fun of playing games? that they “wanted the mist to go away” (13-yes, 1-maybe, 1-no).
3. Can our system remain motivating over the long term needed Wanting to play without the obscuring textures is important to
for successful biofeedback training? motivate the progression of NF training. Players also agreed that
they were “able to control the mist to make it go away” (12-yes,
We chose to evaluate the system with children with FASD, as
3-maybe). That players felt in control over the textures suggests
fetal alcohol exposure is a prevalent cause of intellectual
that altering display graphics works as a biofeedback mechanism.
impairment in the western world [17], and children with FASD
experience symptoms similar to those with ADHD [2]. Users controlled the obfuscation of the mist with their Theta/low
Biofeedback training of brain function self-regulation (called Beta ratio, because children with ADHD have been shown to have
neurofeedback (NF) training) using EEG has been effective at elevated ratios as compared to population norms [19]. Figure 5
reducing the symptoms of ADHD [6,9]. Also, differences in EEG presents average ratios for the beginning and end of the
between children with ADHD and normative databases have been deployment. Because children participated at different times and
reduced with NF training. Specifically, those with ADHD exhibit for a different number of sessions, we performed a median split on
higher power in the Theta band of EEG (related to decreased the number of sessions and classified training as either beginning
attention and less retention of material) and lower power in the sessions or end sessions. Data for two children were removed due
low Beta band of EEG (related to increases in both hyperactivity to the connection with the Neurosky mindset being below
and impulsivity) [19]. NF training helped children with ADHD threshold (80%) for the majority of the sessions. Figure 5 shows
lower the ratio of Theta/low Beta activity, by either lowering that players were successful at lowering the ratio in the later
Theta activity or increasing low Beta activity [6,19]. training sessions as compared to the initial sessions. A paired-
samples t-test supports that this difference in average theta/low
We tested our system with 16 children (9 male) between the ages
beta ratio is significant (T13=2.16, pDo graphical overlays ruin the fun of playing games? Players (must be played in windowed mode) and content (no
agreed that “playing the biofeedback games was fun” (14-yes, 1- objectionable content), we also were limited by the timing of the
maybe). The NF textures did not fundamentally alter the training sessions. Because users played every few days for half an
gameplay experience so that it was no longer enjoyable, and fun hour, we offered casual games. Casual games are those that are
games remained fun to play when used as part of our system. easy to learn with limited instructions and simple rules and
However, the kids also agreed that they “would have preferred to controls; they have short play times and allow users to put the
play the games without the headsets and the mist” (14-yes, 1- game on hold, thus lending themselves well to our experiment
maybe). That kids wanted to play without NF is expected, and protocol [22]. These types of games also lend themselves to our
also fundamental to how the training system works; players need approach to biofeedback training, as the texture overlays do not
to prefer to play without obfuscation to help motivate training. present significant risk to players. For example, if a user lost a
game of Bejeweled during training, she could simply start a new
Players did not feel that “playing the games was challenging” (5- game. How our system would function with deep narrative-based
yes, 1-maybe, 9-no). It is good that the players did not feel games designed to immerse a player, which contain leveling tasks
excessively challenged by the games because the challenge of
or boss battles (e.g., Mass Effect, Uncharted) remains to be seen.
keeping the textures from obscuring the screens should be the Whether our system could work with time-critical collaborative
focus of the NF training activity. games (e.g., Team Fortress, World of Warcraft) also is unclear.
Can our system remain motivating over the long term needed for That our textures do not break the ‘fun’ offered by the casual
successful biofeedback training? Our test deployment shows that games tested in our deployment does not imply that the system
kids enjoyed playing the biofeedback games, even when asked at will work for all games. Commercial games are engineered to
the end of the 12-week period. Thus our texture-based provide a compelling and emotional experience for the player –
biofeedback system did not break the enjoyment of gameplay. In our intention is to help users self-regulate. There is a tension
fact, after a number of training sessions, players became bored between these goals that requires further investigation.
with the available selection of games and asked that we add more.
Once we did, the kids were again happy to play. As long as a child 5.2 Presentation of the Biofeedback Textures
is motivated to use the computer (i.e., through gameplay or other Our textures were abstract representations of brain state. A user
computer-based activities), our system will remain motivating. may understand that the opacity of flowing mist is indicative of
their brain state; however, it is possible that, in some applications,
5. DISCUSSION concrete visual representations may prove superior. Numerical
representations or progress bars are possible using TextureLib.
5.1 Summary of Findings Also, our textures were agnostic to the underlying interface of the
Our test deployment shows that kids enjoyed playing the user’s primary task. Although ours is a general solution that will
biofeedback games, wanted the mist to go away, and felt able to operate with off-the-shelf software in a black-box manner,
control the mist to make it disappear. Our texture-based TextureLib can be aware of interface elements. For example,
biofeedback system did not ruin the fun of games; rather it animated sprites could be directed toward a screen location, static
provided the opportunity for choice and variety so kids could sprites could reside over interface elements, or effects could
maintain enjoyment. When players became bored with the follow the mouse cursor. Consider a dynamic icon that resides in a
available selection of games, we added more. By giving choices, user’s system tray, or as part of a player’s interface in game.
we were able to retain the interest of the players over a long span Connecting a texture-based visual representation to an on-screen
of time (3 months). Had players tired of a traditional biofeedback or cursor location introduces new possibilities for biofeedback
game, with game mechanics that adjust to physiological state, training without requiring access to underlying applications.
there would have been no way to renew their interest in play,
Some games allow limited access to the game state through SDKs
short of building a new game or new game levels. Our test
(e.g., Valve Software’s (Portal, Team Fortress, Half-Life) Source
deployment also showed that obfuscating overlays have potential
Engine and SDK). With access to game state through an SDK,
as a feedback mechanism for biofeedback training; most users
biofeedback games could be highly customized to gameplay,
agreed that they were able to control the overlays. Also, the
appearing to be an integrated solution presented by the developer.
lowered ratios of Theta/low Beta activity in later play sessions
For example, different overlays could be presented at different
suggest that our biofeedback training approach holds promise.
locations (e.g., indoor / outdoor); overlays could be combined
with in-game information (e.g., in-game health pack could reduce
5.1.1 Interpretation of Results obfuscation); or overlays could follow in-game elements (e.g.,
Our results indicate the potential of obscuring overlays as a obscuring text bubble of a non-player character conversation).
biofeedback training system; we do not claim that our results
show successful NF training for kids with FASD. The data
gathered in our deployment are preliminary and representative of
5.3 Interaction Beyond Games
a small population. In addition, the EEG data used in our analyses Although we focus on biofeedback games in this paper, our
was gathered during play from a consumer electronics device, not system is general enough to operate over most computer-based
a multi-channel high-frequency EEG system with a standard tasks. The premise of biofeedback games is that the desire to play
electrode array. To determine whether our approach is successful motivates users to alter their physiological state in the preferred
in NF training, we are conducting a large-scale study involving direction. For children who do not enjoy playing games, our
pre- and post-testing using validated outcome measures. system can also be used over web browsers, such as Firefox or
Chrome, and thus function with any task that a user performs in a
web browser (e.g., search, chat, social sites, media viewing).
5.1.2 Generalizing Beyond Casual Games Alternatively, our system can be used with other off-the-shelf
We deployed our system alongside a selection of games from software (e.g., drawing application). Given that most consumer-
which players could choose. Limited primarily by technologylevel biofeedback training systems are game-based, our approach [4] Dekker, A. & Champion, E. Please Biofeed the Zombies:
creates possibilities for a new population of users by providing an Enhancing the Gameplay and Display of a Horror Game
opportunity for children uninterested in games to participate in Using Biofeedback. Proc. of DiGRA'07, 550-558.
biofeedback training. In addition, our system decouples the
[5] Gilleade, K., Dix, A., & Allanson, J. Affective Videogames
physiological sensing system from the activity being performed
and Modes of Affective Gaming: Assist Me, Challenge Me,
on the computer, so users can choose their training activity
Emote Me. Proc. DiGRA'05, 2005.
separately from their choice of which physiological trait to train.
So a child can train, for example, their brain activity, or their [6] Heinrich, H., Gevensleben, H., & Strehl, U. Annotation:
anxiety, by playing a game or chatting online with friends. Neurofeedback — Train your brain to train behaviour. J.
Child Psychology and Psychiatry, 48, (2007), 3!16.
There is still the opportunity for developers who wish to create
integrated biofeedback training solutions – where the sensing is [7] Herndon, C., Decambre, M., & McKenna, P. Interactive
directly integrated with the application – to do so. Our system computer games for treatment of pelvic floor dysfunction.
does not eliminate the prospect of integrated solutions; we simply The Journal of Urology, 166, (2001), 1893–1898.
provide a general-purpose solution that decouples sensing from
[8] Hjelm, S. Research + design: the making of Brainball.
activity, appealing to a broad range of tasks, domains, and users. interactions 10, 1 (2003), 26-34.
[9] Lubar, J.F., Swartwood, M.O., et al. Evaluation of the
6. FUTURE WORK AND CONCLUSIONS effectiveness of EEG neurofeedback training for ADHD in a
Our research demonstrates a new approach to biofeedback
clinical setting as measured by changes in T.O.V.A. scores,
training, where the physiological sensing is decoupled from the
behavioral ratings, and WISC-R performance. Biofeedback
user’s primary task. We show that this approach has potential
and Self Regulation, 20, (1995), 83–99.
through a test deployment. To investigate whether our system can
help children with FASD reduce their symptoms related to [10] May, P., & Gossage, P. (2001). Estimating the prevalence of
ADHD, we are conducting a large-scale study. Fetal Alcohol Syndrome: A summary. Alcohol Research &
Health, 25(3), 159-167.
We used casual games as the player’s primary task and are
presently exploring how the texture overlay system affects [11] Nacke, L.E., Kalyn, M., et al. Biofeedback game design:
gameplay with more immersive games with narrative, time- using direct and indirect physiological control to enhance
critical tasks, and longer playtimes. As part of this exploration, we game interaction. Proc. CHI '11, 103-112.
are focusing on how different presentations of the biofeedback [12] Nijholt, A., Plass-Oude Bos, D., & Reuderink, B. Turning
display balance the competing (and perhaps mutually exclusive) shortcomings into challenges: Brain-computer interfaces for
desires for engaging gameplay and effective biofeedback training. games. Ent. Computing 1, 2 (2009), 85-9.
Biofeedback games have the potential to help children self- [13] Picard, R.W. Affective Computing, MIT Press, Cambridge,
regulate their physiological function; neurofeedback games – 1997.
those that help users self-regulate their brain function – hold
promise for special populations, such as children with FASD. [14] Pope, A.T. & Palsson, O.S. Helping video games ‘‘rewire
However, choice in biofeedback games is limited, they are our minds”, NASA TR (2001).
difficult to create, and have little depth of play. We present a [15] Salen, K. & Zimmerman, E. Rules of play: Game design
solution for turning off-the-shelf games into biofeedback games. fundamentals. MIT Press, Cambridge, MA, 2003.
Our texture-based overlay solution for decoupling the sensing
from the game gives kids the choice of what aspect of their [16] Sakurazawa, S., Yoshida, N., Munekata, N. et al. A computer
physiology to train and which game to play. Our approach game using galvanic skin response, Proc. ICEC’03,
provides biofeedback training to children previously uninterested Pittsburgh, USA, 2003.
or unable, and creates enough choice in interaction to support the [17] Spohr, H.L., Willms, J., and Steinhausen, H.C. Prenatal
long-term and repeated use that is necessary for success. alcohol exposure and long- term developmental
consequences. The Lancet, 341, (1993), 907-910.
7. ACKNOWLEDGEMENTS [18] Stach, T., Graham, T.C.N., et al.. Heart rate control of
Thanks to NSERC and the GRAND and NeuroDevNet NCEs for exercise video games. Proc. GI '09, (2009), 125-132.
funding. Thanks to Bassam Khaleel and the UofS Interaction Lab.
[19] Thompson, M. and Thompson L. The Neurofeedback Book:
An Introduction to Basic Concepts in Applied
8. REFERENCES Psychophysiology. The Association for Applied
[1] Bersak, D., Mcdarby, G., et al. Intelligent biofeedback using Psychophysiology and Biofeedback, Colorado, USA, 2003.
an immersive competitive environment. Proc. of UBICOMP
2001 Workshop on Ubiquitous Gaming (2001). [20] Thompson, L., Thompson, M., & Reid, A. Neurofeedback
outcomes in clients with Asperger's Syndrome, App.
[2] Bhatara V, Loudenberg R, & Ellis R. (2006). Association of Psychophysiology Biofeedback, 35, (2010), 63-81.
attention deficit hyper-activity disorder and gestational
alcohol exposure: An exploratory study. Journal of Attention [21] Vernon, D., Egner, T., Cooper, N., et al. The effect of
Disorders, 9, 515–522. training distinct neurofeedback protocols on aspects of
cognitive performance. International Journal of
[3] Clarren, S., & Lutke, J. (2008). Building clinical capacity for Psychophysiology, 47, (2003), 75–85.
Fetal Alcohol Spectrum Disorder diagnoses in Western and
Northern Canada. Canadian Journal of Clinical [22] Casual Game White Papers | IGDA. International Game
Pharmacology, 15(2), e223-e237. Developers Association (2008).You can also read
