Personalizing User Engagement Dynamics in a Non-Verbal Communication Game for Cerebral Palsy
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Personalizing User Engagement Dynamics
in a Non-Verbal Communication Game for Cerebral Palsy
Nathaniel Dennler1 , Catherine Yunis2 , Jonathan Realmuto3 ,
Terence Sanger3 , Stefanos Nikolaidis1 , and Maja Matarić1
Abstract— Children and adults with cerebral palsy (CP) can participants over weakly embodied agents, such as computers
have involuntary upper limb movements as a consequence of (see review by Deng et al. [9]). Socially, we investigate how
the symptoms that characterize their motor disability, leading the feedback provided by the agent throughout the game
to difficulties in communicating with caretakers and peers. We
describe how a socially assistive robot may help individuals affects participant engagement. Understanding how robots
arXiv:2107.07446v1 [cs.RO] 15 Jul 2021
with CP to practice non-verbal communicative gestures using can effect engagement dynamics is an under-explored area
an active orthosis in a one-on-one number-guessing game. We of human-robot interaction (HRI) [10].
performed a user study and data collection with participants Both physical and social factors are investigated through a
with CP; we found that participants preferred an embodied user study of participants with CP. We found that participants
robot over a screen-based agent, and we used the participant
data to train personalized models of participant engagement preferred interacting with the SAR compared to a screen-
dynamics that can be used to select personalized robot actions. based agent but did not observe any significant differences
Our work highlights the benefit of personalized models in the in engagement levels between the two conditions, which
engagement of users with CP with a socially assistive robot and we attribute to individual differences in how participants
offers design insights for future work in this area. responded to the robot’s actions, as detailed in Section V-
I. I NTRODUCTION B. To explore user engagement further, we then developed
a probabilistic model for personalizing the robot’s actions
Cerebral palsy (CP) is one of the most prevalent motor
based on an individual participant’s responses to the robot,
disorders in children [1], affecting around 0.2%-0.3% of all
and show in simulation that this improves the users’ engage-
live births in the United States [2]. The main symptom of
ment levels compared to models that are not personalized.
CP is involuntary muscle contractions that lead to repetitive
Together, the results of this work indicate the promise of
movements [3] which can greatly affect a child’s ability to
personalized SAR for helping individuals with cerebral palsy
communicate with caregivers and peers [4]. This symptom
to practice non-verbal communication movements.
necessitates the use of active orthoses to facilitate proactive
communication and to aid in motor rehabilitation [5]. II. R ELATED W ORK
Retraining motor skills, however, requires repetitive and
Engagement is a key factor in measuring the quality
lengthy sessions to be effective [6]. In children especially,
of HRI scenarios [10], and in particular has been studied
this can lead to disengagement with the therapeutic ac-
extensively as means of maintaining user interest [11]. User-
tivity, negatively affecting functional outcomes. Thus, we
specific perceptual models to identify user engagement have
aim to facilitate engaging therapeutic activities through the
been explored in the context of the autism spectrum disorder
development of an engaging game between a participant
[12]–[14], where user behavior varies significantly due to
and a socially assistive robot (SAR) [7] [8]. The robot
personal differences. These personal differences are also
encourages the participant to perform (and therefore practice)
present in CP populations, where there is a great variance
non-verbal communicative gestures while providing social
between individuals in how motor function is impacted.
reinforcement as the participant makes progress in the game.
Consequently, there has been an emphasis on developing
This work explores the effect of both physical and social
personalized user models to facilitate interactions (see re-
factors of the interaction design on the ability to effec-
views by Clabaugh et al. [15] and Rossi et al. [16]). Person-
tively engage participants. Physically, we investigate the
alized interactions have shown promising results in various
embodiment of the agent that delivers the game. Several
domains, ranging from rehabilitation [17] to robot tutoring
recent works have described the positive effect of strongly
systems [18], by implementing robot action selection based
embodied agents, such as robots, on the engagement of
on personalized user models; however, few have studied
1 Nathaniel Dennler, Stefanos Nikolaidis, and Maja Matarić are from engagement dynamics.
the Computer Science Department at the University of Southern Cali- A review of several studies [19] concludes that SARs
fornia in Los Angeles, CA {dennler, nikolaid, mataric} have been effective for clinical populations diagnosed with
@usc.edu
2 Catherine Yunis is from the Biomedical Engineering Depart- CP, demonstrating that physical robots can elicit positive
ment at the University of Southern California in Los Angeles, CA responses from users with CP who are performing repetitive
cyunis@usc.edu physical exercise tasks. Robots as partners in game-like
3 Jonathan Realmuto and Terence Sanger and are from the Electrical
Engineering and Computer Science Department at University of California, therapeutic physical activities have been shown to create en-
Irvine in Irvine, CA {jrealmut, tsanger}@uci.edu gaging experiences for users and lead to increased motivationFig. 1: Study setup. The participants used non-verbal gestures to communicate with the robot or computer screen, depending
on the experiment condition. The external 3D camera collected real-time participant hand movement data used by the robot.
was controlled by a Beaglebone microprocessor [24], actu-
Robot Agent, Robot Agent, Screen Agent, Participant Choice,
Orthosis Off Orthosis On Orthosis On Orthosis On ated with a compressed air tank, and was attached to the
(Max 10 min) (Max 10 min) (Max 10 min) (Unlimited time)
participant with Velcro strips for facile donning and doffing
1+ min Robot Screen [5]. The orthosis was worn throughout the session and was
Break Rating Rating
not a manipulated variable in the experiment.
The participant’s thumb angle was measured by an RGBD
Fig. 2: Stages of the within-subject experiment design. camera and transmitted through a ROS network [25]. A
webcam placed on the table in front of the participant
captured and recorded the participant’s facial expressions.
[20]. The importance of engagement is emphasized in stud- An emergency stop button was provided to the participant
ies involving movement exercises for CP, and quantitative for terminating the interaction at any point.
measures of engagement are well-established for this context
[21]. Given the success of using SARs with this population, B. Interaction Design
we aim to understand how SARs can shape user engagement At the start of the session, each participant demonstrated a
in practicing repetitive exercises. thumbs-up and thumbs-down gesture to generate a baseline
for their individual range of motion. Next, the robot ex-
III. M ETHODS
plained the number-guessing game, telling the participant to
A. Study Setup think of a number between 1 and 50, and to communicate the
The study setup consisted of the participant sitting at a number secretly to the experimenter, by whispering or typing
table and facing a computer screen or a robot, both at eye- the number on an iPad. At each turn of the game, the robot
level, as shown in Figure 1. The experimenter was present in guessed the number and asked the participant if the guess
the room for safety and to collect verbal questionnaire data. was correct. The participant answered yes or no by making
Finally, the participant’s parent was located in the hallway a thumbs-up or thumbs-down gesture, respectively, using
outside, seated separately and not interacting with the study. the arm with the orthosis. If the robot guessed incorrectly,
We used the tabletop LuxAI QT robot [22], shown in it asked if the number was higher than the guess. The
Figure 1-a; the robot is 25 inches tall, has arms with three participant then answered thumbs-up if the number was
DOF arms and a head with two DOF with a screen face. higher, and a thumbs-down if the number was lower. The
The robot was modified to work with the CoRDial dialogue robot ensured that the number of thumbs-up and thumbs-
manager [23] that synchronizes facial expressions with text- down gestures were approximately equal by tracking the
to-speech. number of each and guessing higher or lower than the target
The screen condition used a standard 19” monitor that number to keep the counts balanced. The robot continued to
displayed the same simple animated face as the robot’s at guess numbers randomly from a range of decreasing size as
the same size, as shown in Figure 1-c, and used the same it narrowed in on the correct answer. Once the robot correctly
dialogue manager. The robot and screen agent used the guessed the number and the participant signalled with a
same voice, the same facial features, and the same facial thumbs-up, the robot asked to play the game again. The
expressions, as well as the same machine vision algorithm. participant responded with another thumbs-up or thumbs-
The strongly-embodied robot moved and gestured in the down gesture.
shared desk space with the participant, while the weakly- Every time the participant used a thumbs-up or thumbs-
embodied screen was stationary, as shown in Figure 1-b. down gesture to respond to the robot, the robot responded
The participants wore an orthosis shown in Figure 1-d with feedback that combined verbal, physical, and facial
that used fabric-based helical actuators to support the partic- action, based on the quality of the gesture and the history
ipant’s thumbs-up and thumbs-down gestures. The orthosis of the participant’s gestures. Specifically, the feedback was aclarifying utterance, an encouraging utterance, or a reward- TABLE I: Survey questions and associated factors of
ing utterance, accompanied with a corresponding physical Companionship (C), Perceived Enjoyment (PE), and
gesture and facial expression. Clarifying actions were given Perceived Ease of Use (PEU).
when the participant’s response was not legible. Encouraging Survey Question Factor
actions were given when the angle the participant’s thumb How much do you like playing with the {robot, screen}? C [27]
How much do you want to play again with the {robot, screen}? C [27]
made was near their personal baseline value. Rewarding How friendly is the {robot, screen}? C [27]
actions were given when the participant’s thumb angle ex- Is the {robot, screen} exciting? PE [28]
Is the {robot, screen} fun? PE [28]
ceeded their personal baseline value. All verbal, physical, and Does the {robot, screen} keep you happy during the game? PE [28]
facial components of these feedback actions were selected Is the {robot, screen} boring? (inverted) PE [29]
Is playing with the {robot, screen} easy? PEU [30]
randomly from a set of appropriate components for each Is communicating with the{robot, screen} easy? PEU [30]
action, to avoid repetition. Is the {robot, screen} useful when playing the game? PEU [30]
Is the {robot, screen} helpful when playing the game? PEU [30]
Is playing with the {robot, screen} hard? (inverted) PEU [30]
C. Study Design
The study used a within-subjects design shown in the
block diagram in Figure 2; the participants interacted with the successfully completed the study and were provided with
robot in a single session that lasted approximately one hour compensation for their time. This study was approved by
from the participants entering the room to their departure. the University of Southern California Institutional Review
The session was divided into four blocks, with periods of Board under protocol #UP-19-00185.
rest in between. The first three blocks lasted up to 10
minutes each and had the participant play as many games F. Measures
with the robot/screen as desired, while the final block was The participant preference of the embodiment (robot vs.
open-ended, with no fixed duration. Between blocks, the screen) was measured using a three-factor five-point Likert
participant rested for at least one minute or until they were scale, with questions from scales validated in previous works
ready to move to the next block to mitigate effects of [27]–[30]. The three factors were: perceived enjoyment,
muscle fatigue. The first block served as a practice block to companionship, and perceived ease of use.
familiarize the participant with the interaction. In that block, The participants’ engagement was quantified using identi-
the participant interacted with the robot while the orthosis cal criteria as in Clabaugh et al. [15], which also measures
was not powered and thus not assisting their movement. In engagement in a game-based interaction. The participant
the second block, the participant interacted with the robot was labelled as engaged if they responded to the robot’s
with the orthosis powered on. After the second block, the question, thought about the correct answer to the question,
experimenter verbally administered a questionnaire about the had a positive facial expression, and was looking at the
participant’s experience with the robot. In the third block, robot as seen in the auditory and visual data captured by the
the participant interacted with a computer screen with the camera. We represented the level of engagement as a binary
orthosis powered on. After the third block, the experimenter variable (engaged/not engaged), as measured by a trained
verbally administered a questionnaire on the participant’s annotator. To ensure consistency, a secondary annotator in-
experience with the screen-based agent. The fourth block was dependently annotated 10% of the videos selected at random.
optional, and the participant was given a choice of playing We measured inter-rater reliability with Cohen’s Kappa, and
with the robot, playing with the screen, or ending the session. achieved substantial agreement of k = .73, corresponding to
an agreement on 86% of videos, similar to other works in
D. Hypotheses engagement [15], [31].
Since strongly-embodied physical agents have been shown
IV. S TUDY R ESULTS
to increase engagement and positive outcomes in therapeutic
tasks [9], [19], the following hypotheses were tested: A. User Preference
H1: Users with CP will prefer the robot over the screen. Embodiment preference was determined by the difference
H2: Users with CP will be more engaged when interact- in ratings between corresponding questions for the robot
ing with the robot than the screen. and screen conditions. The specific questions are shown in
Table I. The combined responses for all factors are shown
E. Study Population in Figure 3. We found high internal consistency for all
We recruited 10 participants (3 female, 7 male) diagnosed factors: Perceived Enjoyment (α = .94), Companionship
with CP and having symptoms of dystonia in at least one (α = .91), and Ease of Use (α = .89). We evaluated
upper limb. The age range of the participants was 9-22 significance with a Wilcoxon Signed-Rank Test and found
years, with a median age of 15 years. The gender imbal- a significant preference for the robot over the screen in
ance is representative of the higher prevalence of males in factors measuring Companionship (Z = 9.0, p = .026) and
CP populations [26], and the large age range reflects the Perceived Enjoyment (Z = 23.5, p = .018). We found no
challenges of recruitment of this population. Half of the significant differences in Ease of Use (Z = 52.0, p = .399)
participants wore the orthosis on their left hand, and the other and attribute this to the fact that both embodiments used the
half wore the orthosis on their right hand. All participants same vision system, which suffered from perceptual errorsFig. 3: Participant responses to Likert-scale questions, grouped by measured construct.
(such as failing to detect the participant’s off-camera thumb
angle) about 20% of the time. We additionally note that many
of the responses showed no preference for either the robot
or screen due to the tendency of the participants to respond
with similar values for all questions. The results therefore
partially support H1, indicating that participants somewhat
preferred to interact with the robot over the screen.
B. Choice Condition
In the final study block, participants could choose to play
a game with the robot, play another game with the screen,
or stop playing and end the session. One participant chose to Fig. 4: Transition matrices of the two clusters found in the
play with the robot, five participants chose to play with the participant-based clustering method. Each matrix specifies
screen, and four participants chose to stop playing. While the probability of becoming engaged (E) or disengaged (D)
the sample size is too small to draw any conclusions, the at the next time-step, given the current state.
confounding factors include the possibility that participants
were too fatigued to continue playing or reluctant to require
work of the experimenter to exchange the embodiments. maximum likelihood estimation from the sequence of the
Several participants expressed these sentiments while doffing annotated engagement values.
the orthosis at the conclusion of the experiment. We next explored whether participants cluster in terms of
C. Engagement similar reactions to the robot’s actions. Previous work [32]
has shown that users can be grouped based on their prefer-
We found no significant differences in the participants’ ence on how to perform a collaborative task with a robot. We
engagement between the two conditions, which does not used a similar approach in the context of social interaction:
support H2; our post-hoc analysis shows that there were we clustered participants from the study based how their
individualized differences in how engagement changed in engagement changed in response to the robot’s actions.
response to the robot’s actions. We discuss those differences
We converted the transition matrices to vectors, then com-
next in the context of personalizing the interaction.
puted the distance between vectors using cosine similarity.
V. M ODELING E NGAGEMENT We then performed hierarchical clustering [33], by iteratively
We first explored whether there were differences across merging the two most similar vectors into a cluster. The
users in how engagement changed in response to the robot’s merged vector was formed by averaging the values of the two
actions. We modeled the evolution of engagement as a vectors. We selected the final number of clusters, so that each
Markov chain, where engagement is a binary state variable cluster contained at least two individuals. We transformed
that changes stochastically in discrete time-steps, after each the vector of each cluster back to a transition matrix that
of the robot’s actions. specified how engagement changed for participants of that
We define a transition matrix T that specifies how engage- cluster.
ment s ∈ S changes over time T : S × A → Π(S). Since We clustered participants at two different resolutions: 1)
the change depends on the robot’s action (clarify, encourage based on the user as a whole and 2) based on the users’
or reward), we parameterized the transition matrix by the response to each of the robot’s three possible actions. The
robot’s action a ∈ A. first clustering, based on each participant’s holistic response
to to robot actions, resulted in matching each participant
A. Learning Personalized Models to one cluster. We call this participant-level clustering. The
To learn personalized engagement models, the first step is second clustering, based on each participant’s responses to
to represent how likely a participant is to become engaged or each of the robot’s action separately, required three different
disengaged given a robot’s action. We captured this with the clustering iterations, one for each action, and resulted in
transition matrix of the Markov chain. For each participant having each participant matched to three clusters, one for
in the user study, we computed a transition matrix using each action. We call this action-level clustering.B. Cluster Identification
Using the participant-level method, we found two main
clusters, shown in Figure 4. In the first cluster, the encourage
action had a greater likelihood of causing the participant’s
next state to be Engaged (E) than the reward action. In
the second cluster we observed the opposite effect: the
probability of changing from Disengaged (D) to Engaged
(E) is lower for the encourage action than for the reward
action. We observe that the clarify action has a small effect
on changing a participant’s engagement state in both clusters.
Seven participants belonged to the first cluster, and three Fig. 5: Transition matrices of different clusters found in the
participants belonged to the second cluster. There were no action-based clustering method. Each matrix specifies the
clear factors that lead to the makeup of the participants in probability of becoming engaged (E) or disengaged (D) at
the clusters based on the background information collected the next time-step, given the current state.
in the study.
The action-level clustering method generated separate C. Personalizing Robot Actions
clusters for each action independently (Figure 5). Thus, a
If a robot knows the participant’s cluster and adapts
single participant is described as being a part of three action-
its actions to maximize engagement, to what extent does
level clusters. We observed three different types of matrices
this improve the participant’s engagement? Following prior
across the different actions:
work in simulating users based on personas [35], we show
• Type I indicates a high probability of the participant
the benefit of using a personalized engagement model by
becoming Engaged, regardless of their previous state. modeling users based on the data from the user study. We
• Type II has an approximately equal probability of
focus on the action-level clustering method, since it generates
becoming Engaged or remaining Disengaged, if the different clusters for each robot action, resulting in a higher
participant was previously Disengaged. resolution model than the participant-level clusters using the
• Type III features participants who are most likely to
same amount of data.
remain in the same state. We modeled users based on each participant’s transition
We found that the clarify action generated only Type II matrices described in Section V-A. At each timestep, we
and III clusters, since most participants’ engagement did have the user’s current engagement state and the set of
not change based on that action, with three participants the ground-truth transition matrices for each action from
belonging to the Type II cluster and seven participants the study. When the robot takes an action, the user’s next
belonging to the Type III cluster when conditioning on engagement state is sampled from the probability distribution
the clarify action. The reward action generated Type I and of the corresponding action and the current engagement state.
Type II clusters, since most participants became Engaged This process is repeated for 100 timesteps, as determined by
after a reward action. Three participants belonged to the the number of turns in the in-person study. We additionally
Type I cluster and seven participants belonged to Type II average our results over 100 runs for each participant to
cluster. This finding supports previous work [34] that showed mitigate the random effects of the simulation and converge
positive reinforcement improving participants’ engagement to the true mean of the engagement level over the course of
in computer-based animal guessing games. the simulation.
Only the encourage action generated clusters of all three Our strategy for selecting actions in the simulation is to
types. The encourage action had three participants that re- maximize the likelihood of the user becoming engaged based
sponded in alignment with the Type I cluster, five participants on the estimated user clusters. For instance, if a user is said
that aligned with the Type II cluster, and two participants that to be Type II for clarify, Type I for encourage and Type II
aligned with the Type III cluster. for reward, and the participant is currently disengaged (D),
We investigated whether the composition of the par- then the robot would choose the encourage action, since the
ticipants in each cluster was related to the demographic participant would become engaged (E) with probability 0.87
information we collected. Specifically, we analyzed cluster (compared to 0.48 for reward and 0.59 for clarify). These
composition with a multinomial logistic regression of age, estimated clusters, however, are distinct from the true clusters
gender, and handedness onto cluster type and found no used to simulate the users: for example, a user may truly be
significant differences in composition between any clusters associated with the type II cluster for encourage actions,
at either participant or action levels. Qualitatively, age was but we may erroneously select actions as if that user were
a minor component in the Encourage clusters; older ages associated with a type I cluster for the encourage action. We
appeared to be more associated with the Type II cluster additionally imposed a 0.2 probability of the robot taking a
(median age 16), whereas younger participants either fell clarify action no matter what state the participant was in to
into Type I (median age 14) or Type III (median age 10.5) account for incorrect or illegible gestures; similar to the rate
clusters. that was observed in the in-person study.(a) Correct User Inference vs. Random Selection (b) Incorrect User Inference vs. Random Selection (c) Comparison of all strategies
Fig. 6: Percentage of time that modeled users were engaged for different methods of robot action selection. Selecting actions
based on the correct user clusters (a) keeps users more engaged, however selecting actions on incorrect user models (b) has
an adverse effect. Considering the users as one group (c) performs similarly to the random baseline.
We computed two baselines: 1) the robot selects actions the lower-probability modes in which users interact with an
randomly, choosing encourage or reward action with equal implemented system through the use of qualitative tools such
probability; and 2) the robot selects actions to maximize as user personas [35]. These are especially helpful when
engagement based on the transition matrix computed from there are no apparent differences between the clusters.
the maximum likelihood estimate of all the participants. The Clustering user responses helps to reduce the design
second baseline, which we call the “impersonal strategy”, is space. Our clusters reveal three main types of responses
equivalent to having one cluster for every action; it does not to each action. Users found each action as either highly
account for individual differences. engaging (Type I), sometimes engaging (Type II), or having
Fig. 6 shows the average time the modeled users spent no effect on engagement (Type III). Interestingly, we did
being Engaged in the activity for each condition. A two- not see actions that were highly disengaging or caused
tailed paired t-test showed that modeled users spent signif- the participant to switch states with high probability. This
icantly more time being Engaged when the robot selected indicates that those types of interaction are less common,
actions given their cluster compared to taking random ac- and therefore less focus can be placed on considering how
tions (t(11) = 9.370, p < .001). However, if the robot those cases would affect the interaction.
had a flawed model of the user, the modeled users were Certainty in user cluster or persona is critical in person-
significantly less engaged over the course of the interaction alized algorithms. Our results show that misclassification of
compared to randomly selecting actions (t(131) = −4.692, a user’s cluster or persona leads to lower engagement than
p < .001). This result highlights the trade-offs that person- random selection. To build effective systems, it is critical to
alization may bring, especially in low-data scenarios. be certain that the inferred user’s classification is correct. A
The second baseline treated users as coming from one system that personalizes to a given user should consider the
cluster. The personalized strategy significantly outperformed risk of misclassification in selecting the level of certainty that
the impersonal strategy (t(11) = 4.984, p < .001). Further- is required to make decisions in an interactive scenario.
more, we cannot say that the impersonal strategy performed
any differently than randomly selecting actions (t(11) = VII. L IMITATIONS AND C ONCLUSION
−.656, p = .525). This shows the importance of algorithmic Our results are limited by our small sample size, resulting
design in interaction, and how incorrect assumptions can lead from the practical challenges of recruiting participants with
to data-driven models that are ineffective. CP. Applying our clustering approach with more participants
will likely result in more nuanced representations of user
VI. D ESIGN I NSIGHTS
engagement levels. Our method also carries the inherent
From the study conducted with a participants with CP, we limitations of Markov-based models: it does not account for
developed the following design insights that may be useful effects of the history of interaction, such as fatigue, or aspects
when designing personalized algorithms for end-users. of the interaction that are not modeled, such as participants
The distribution of interaction preferences is often un- getting distracted by other events in their environment.
balanced. The clusters formed from this study revealed Additionally, our user models are based on the assumption
skews in the number of participants, most commonly with that users can be associated with a previously known clas-
the majority cluster being composed of seven of the ten sification; they do not account for new, previously unseen
participants. This highlights the importance of understanding classifications. Implementation in a real-world setting wouldalso require correct inference of the user’s engagement level Conference on Computer Vision and Pattern Recognition Workshops
in real time, as well as accurate identification of the user’s (CVPRW). IEEE, 2019, pp. 217–226.
[15] C. E. Clabaugh, K. Mahajan, S. Jain, R. Pakkar, D. Becerra, Z. Shi,
classification. In fact, our user models show that incorrect E. Deng, R. Lee, G. Ragusa, and M. Mataric, “Long-term personal-
inference results in worse performance than random robot ization of an in-home socially assistive robot for children with autism
action selection. spectrum disorders,” Frontiers in Robotics and AI, vol. 6, p. 110, 2019.
[16] S. Rossi, F. Ferland, and A. Tapus, “User profiling and behavioral
This work introduces socially assistive robotics to the adaptation for hri: A survey,” Pattern Recognition Letters, vol. 99, pp.
context of communicative gesture practice for users with 3–12, 2017.
cerebral palsy. Our user study shows that participants with [17] A. Tapus, C. Ţăpuş, and M. J. Matarić, “User—robot personality
matching and assistive robot behavior adaptation for post-stroke re-
CP preferred to interact with a socially assistive robot habilitation therapy,” Intelligent Service Robotics, vol. 1, no. 2, pp.
compared to a screen-based agent. While we did not observe 169–183, 2008.
significant differences in user engagement overall, our post- [18] D. Leyzberg, S. Spaulding, and B. Scassellati, “Personalizing robot
tutors to individuals’ learning differences,” in 2014 9th ACM/IEEE
hoc analysis showed that there are nuanced differences International Conference on Human-Robot Interaction (HRI). IEEE,
between modes of participants: some participants became 2014, pp. 423–430.
more engaged after the robot gave encouraging feedback, [19] N. A. Malik, F. A. Hanapiah, R. A. A. Rahman, and H. Yussof,
“Emergence of socially assistive robotics in rehabilitation for children
while others responded better to rewarding feedback. We with cerebral palsy: A review,” International Journal of Advanced
show that understanding how a user will react to different Robotic Systems, vol. 13, no. 3, p. 135, 2016.
robot actions can be leveraged to design a more engaging [20] A. Brisben, C. Safos, A. Lockerd, J. Vice, and C. Lathan, “The
cosmobot system: Evaluating its usability in therapy sessions with
experience. children diagnosed with cerebral palsy,” Retrieved on, vol. 3, no. 25,
p. 13, 2005.
R EFERENCES [21] N. A. Malik, H. Yussof, F. A. Hanapiah, and S. J. Anne, “Human
robot interaction (hri) between a humanoid robot and children with
[1] M. Oskoui, F. Coutinho, J. Dykeman, N. Jette, and T. Pringsheim, “An cerebral palsy: experimental framework and measure of engagement,”
update on the prevalence of cerebral palsy: a systematic review and in 2014 IEEE Conference on Biomedical Engineering and Sciences
meta-analysis,” Developmental Medicine & Child Neurology, vol. 55, (IECBES). IEEE, 2014, pp. 430–435.
no. 6, pp. 509–519, 2013. [22] “Qtrobot: Humanoid social robot for research and teaching,” 2020.
[2] S. Winter, A. Autry, C. Boyle, and M. Yeargin-Allsopp, “Trends in the [Online]. Available: http://luxai.com/qtrobot-for-research/
prevalence of cerebral palsy in a population-based study,” Pediatrics, [23] E. Short, D. Short, Y. Fu, and M. J. Mataric, “Sprite: Stewart platform
vol. 110, no. 6, pp. 1220–1225, 2002. robot for interactive tabletop engagement. department of computer
[3] T. D. Sanger, “Toward a definition of childhood dystonia,” Current science, university of southern california,” Tech Report, 2017.
opinion in pediatrics, vol. 16, no. 6, pp. 623–627, 2004. [24] C. Long and J. Kridner, “Meet beagle: Open source computing,”
[4] M. J. C. Hidecker, N. Paneth, P. L. Rosenbaum, R. D. Kent, J. Lillie, 2019. [Online]. Available: https://beagleboard.org/
J. B. Eulenberg, K. CHESTER, JR, B. Johnson, L. Michalsen, M. Evatt [25] Stanford Artificial Intelligence Laboratory et al., “Robotic operating
et al., “Developing and validating the communication function clas- system.” [Online]. Available: https://www.ros.org
sification system for individuals with cerebral palsy,” Developmental [26] M. V. Johnston and H. Hagberg, “Sex and the pathogenesis of cerebral
Medicine & Child Neurology, vol. 53, no. 8, pp. 704–710, 2011. palsy,” Developmental Medicine & Child Neurology, vol. 49, no. 1,
[5] J. Realmuto and T. Sanger, “A robotic forearm orthosis using soft pp. 74–78, 2007.
fabric-based helical actuators,” in 2019 2nd IEEE International Con- [27] K. M. Lee, N. Park, and H. Song, “Can a robot be perceived
ference on Soft Robotics (RoboSoft). IEEE, 2019, pp. 591–596. as a developing creature? effects of a robot’s long-term cognitive
[6] J. A. Buitrago, A. M. Bolaños, and E. Caicedo Bravo, “A motor learn- developments on its social presence and people’s social responses
ing therapeutic intervention for a child with cerebral palsy through a toward it,” Human communication research, vol. 31, no. 4, pp. 538–
social assistive robot,” Disability and Rehabilitation: Assistive Tech- 563, 2005.
nology, vol. 15, no. 3, pp. 357–362, 2020. [28] J.-W. Moon and Y.-G. Kim, “Extending the tam for a world-wide-web
context,” Information & management, vol. 38, no. 4, pp. 217–230,
[7] D. Feil-Seifer and M. J. Mataric, “Defining socially assistive robotics,”
2001.
in 9th International Conference on Rehabilitation Robotics, 2005.
[29] M. Heerink, B. Krose, V. Evers, and B. Wielinga, “Measuring accep-
ICORR 2005. IEEE, 2005, pp. 465–468.
tance of an assistive social robot: a suggested toolkit,” in RO-MAN
[8] M. J. Matarić and B. Scassellati, “Socially assistive robotics,” in
2009 - The 18th IEEE International Symposium on Robot and Human
Springer handbook of robotics. Springer, 2016, pp. 1973–1994.
Interactive Communication, 2009, pp. 528–533.
[9] E. Deng, B. Mutlu, M. J. Mataric et al., “Embodiment in socially
[30] V. Venkatesh, “Determinants of perceived ease of use: Integrating con-
interactive robots,” Foundations and Trends® in Robotics, vol. 7, no. 4,
trol, intrinsic motivation, and emotion into the technology acceptance
pp. 251–356, 2019.
model,” Information systems research, vol. 11, no. 4, pp. 342–365,
[10] C. Oertel, G. Castellano, M. Chetouani, J. Nasir, M. Obaid,
2000.
C. Pelachaud, and C. Peters, “Engagement in human-agent interaction:
[31] O. Rudovic, J. Lee, M. Dai, B. Schuller, and R. W. Picard, “Personal-
An overview,” Frontiers in Robotics and AI, vol. 7, p. 92, 2020.
ized machine learning for robot perception of affect and engagement
[Online]. Available: https://www.frontiersin.org/article/10.3389/frobt.
in autism therapy,” Science Robotics, vol. 3, no. 19, 2018.
2020.00092
[32] S. Nikolaidis, R. Ramakrishnan, K. Gu, and J. Shah, “Efficient model
[11] O. Celiktutan, E. Sariyanidi, and H. Gunes, “Computational analysis learning from joint-action demonstrations for human-robot collabo-
of affect, personality, and engagement in human–robot interactions,” rative tasks,” in 2015 10th ACM/IEEE International Conference on
in Computer Vision for Assistive Healthcare. Elsevier, 2018, pp. Human-Robot Interaction (HRI). IEEE, 2015, pp. 189–196.
283–318. [33] D. Müllner, “Modern hierarchical, agglomerative clustering algo-
[12] S. Jain, B. Thiagarajan, Z. Shi, C. Clabaugh, and M. J. Matarić, rithms,” arXiv preprint arXiv:1109.2378, 2011.
“Modeling engagement in long-term, in-home socially assistive robot [34] B. J. Fogg and C. Nass, “Silicon sycophants: the effects of computers
interventions for children with autism spectrum disorders,” Science that flatter,” International journal of human-computer studies, vol. 46,
Robotics, vol. 5, no. 39, 2020. no. 5, pp. 551–561, 1997.
[13] O. Rudovic, J. Lee, L. Mascarell-Maricic, B. W. Schuller, and R. W. [35] A. Andriella, C. Torras, and G. Alenyà, “Learning robot policies
Picard, “Measuring engagement in robot-assisted autism therapy: A using a high-level abstraction persona-behaviour simulator,” in 2019
cross-cultural study,” Frontiers in Robotics and AI, vol. 4, p. 36, 2017. 28th IEEE International Conference on Robot and Human Interactive
[14] O. Rudovic, H. W. Park, J. Busche, B. Schuller, C. Breazeal, and R. W. Communication (RO-MAN), 2019, pp. 1–8.
Picard, “Personalized estimation of engagement from videos using
active learning with deep reinforcement learning,” in 2019 IEEE/CVFYou can also read
