The Crux of Voice (In)Security: A Brain Study of Speaker Legitimacy Detection - NDSS
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
The Crux of Voice (In)Security:
A Brain Study of Speaker Legitimacy Detection
Ajaya Neupane* Nitesh Saxena Leanne Hirshfield Sarah Elaine Bratt
University of California Riverside University of Alabama at Birmingham Syracuse University Syracuse University
ajaya@ucr.edu saxena@uab.edu lhirshf@syr.edu sebratt@syr.edu
Abstract—A new generation of scams has emerged that uses I. I NTRODUCTION
voice impersonation to obtain sensitive information, eavesdrop
over voice calls and extort money from unsuspecting human Voice is supposed to be a unique identifier of a person.
users. Research demonstrates that users are fallible to voice In human-to-human conversations, people may be able to
impersonation attacks that exploit the current advancement in recognize the speakers based on the unique traits of their
speech synthesis. In this paper, we set out to elicit a deeper voices. However, previous studies have shown that human-
understanding of such human-centered “voice hacking” based based speaker verification is vulnerable to voice impersonation
on a neuro-scientific methodology (thereby corroborating and attacks [30]. A malicious entity can impersonate someone’s
expanding the traditional behavioral-only approach in signifi-
voice by mimicking it using speech synthesis techniques. In
cant ways). Specifically, we investigate the neural underpinnings
of voice security through functional near-infrared spectroscopy particular, off-the-shelf speech morphing techniques can be
(fNIRS), a cutting-edge neuroimaging technique, that captures used to generate the spoofed voices of people of interest (vic-
neural signals in both temporal and spatial domains. We design tims) [30]. The attacker can then perform social engineering
and conduct an fNIRS study to pursue a thorough investigation of trickeries using an impersonated voice to fool the users into
users’ mental processing related to speaker legitimacy detection – accepting it as a legitimate one. These attacks may eventually
whether a voice sample is rendered by a target speaker, a different make the users reveal sensitive and confidential information,
other human speaker or a synthesizer mimicking the speaker. We which may hamper their security, privacy, and safety.
analyze the neural activity associated within this task as well as
the brain areas that may control such activity. Such voice imitation is an emerging class of threats, espe-
cially given the advancement in speech synthesis technology,
Our key insight is that there may be no statistically significant seen in a variety of contexts that can harm a victim’s reputation
differences in the way the human brain processes the legitimate and her security/safety [30]. For instance, the attacker could
speakers vs. synthesized speakers, whereas clear differences are publish the morphed voice samples on social media [17],
visible when encountering legitimate vs. different other human impersonate the victim in phone conversations [21], leave fake
speakers. This finding may help to explain users’ susceptibility voice messages to the victim’s contacts, and even launch man-
to synthesized attacks, as seen from the behavioral self-reported in-the-middle attacks against end-to-end encryption technolo-
analysis. That is, the impersonated synthesized voices may seem gies that require users to verify the voices of the callers [38],
indistinguishable from the real voices in terms of both behavioral
to name a few instances of such attacks.
and neural perspectives. In sharp contrast, prior studies showed
subconscious neural differences in other real vs. fake artifacts Given the prominence and rapid emergence of these threats
(e.g., paintings and websites), despite users failing to note these in the wild, it is crucial to understand users’ innate psy-
differences behaviorally.
chological behavior that governs the processing of voices
and their potential susceptibility to voice impersonation at-
Overall, our work dissects the fundamental neural patterns tacks. In this paper, we follow the neuroimaging methodology
underlying voice-based insecurity and reveals users’ susceptibility adopted in a recently introduced line of research (e.g., [8],
to voice synthesis attacks at a biological level. We believe that
this could be a significant insight for the security community
[32], [34]) to scrutinize the user behavior in the specific
suggesting that the human detection of voice synthesis attacks context of such “voice security”. Specifically, we study users’
may not improve over time, especially given that voice synthesis neural processes (besides their behavioral performance) to
techniques will likely continue to improve, calling for the design understand and leverage the neural mechanics when users
of careful machine-assisted techniques to help humans counter are subjected to voice impersonation attacks using a state-of-
these attacks. the-art neuroimaging technique called functional near-infrared
spectroscopy (fNIRS).
* Work The specific goal of this paper is to study the neural
done while being a student at UAB
underpinnings of voice (in)security, and analyze differences
(or lack thereof) in neural activities when users are processing
different types of voices. We examine how the information
Network and Distributed Systems Security (NDSS) Symposium 2019 present in the neural signals can be used to explain users’
24-27 February 2019, San Diego, CA, USA
ISBN 1-891562-55-X susceptibility to voice imitation attacks using synthesized
https://dx.doi.org/10.14722/ndss.2019.23206 voices (i.e., speaker legitimacy detection). Prior studies [25],
www.ndss-symposium.org [32]–[34] have shown that subconscious neural differences
TABLE I. S UMMARY OF R EAL -FAKE A NALYSIS OBSERVED IN
RELATED WORKS VS . OUR WORK
we do not obtain differences in the way the brains process
legitimate speakers vs. synthesized speakers, when subject to
Type of Artifacts Differences in Differences voice impersonation attacks, although marked differences are
Neural Activity in Behavioral seen between neural activity corresponding to a legitimate
Response
speaker vs. a different unauthorized human speaker. That is,
Websites under phish- Present Nearly absent the synthesized voices seem nearly indistinguishable from the
ing ( [32]–[34]) real voices with respect to the neurocognitive perspective.
Paintings [25] Present Nearly absent
This insight may serve well to explain users’ susceptibility to
Voices (our work) Absent Nearly absent
such attacks as also reflected in our task performance results
(similar to the task performance results reported in [30]).
Table I captures a summary of our work versus other real-
exist when users are subject to real vs. fake artifacts, even fake detection studies.
though users themselves may not be able to tell the two apart
behaviorally. Neupane et al. in their previous studies [32]– Since this potential indistinguishability of real vs. morphed
[34] found differences in neural activities at the right, middle, lies at the core of human biology, we posit that the problem
inferior, and orbitofrontal areas when users were processing is very severe, as the human detection of synthesized attacks
real and fake websites. Similarly, Huang et al. [25] found dif- may not improve over time with evolution, Further, in our
ferences in neural activation at similar areas when users were study, we use an off-the-shelf, academic voice morphing tool
viewing real and fake Rembrandt paintings. Higher activation based on voice conversion, CMU Festvox [19], whereas with
in the frontopolar area implicates the use of working memory the advancement in the voice synthesizing technologies (e.g.,
and cognitive workload. Lower activation in the orbitofrontal newer voice modeling techniques such as those offered by
area suggests that the users trust stimuli presented to them g Lyrebird and Google WaveNet [29], [41]), it might become
[15]. of the target speakers. even more difficult for users to identify such attacks. Also, our
study participants are mostly young individuals and with no
Based on these prior results, we performed our study with
reported hearing disabilities, while older population samples
the hypothesis that these and other relevant brain areas might
and/or those having hearing disabilities may be more prone to
be activated differently when users are listening to the original
voice synthesis attacks [16].
and fake voices of a speaker. We, therefore, set-up the fNIRS
headset configuration to measure the frontal and temporo- We do not claim that the rejection of our hypothesis nec-
parietal brain which overlaps with the regions reported in these essarily means that the differences between real and morphed
previous “real-fake detection” studies. The implications of the voices are absent conclusively – further studies might need to
neural activity differences, if present when processing real vs. be conducted using other neuroimaging techniques and other
fake voices, can be important as these differences could be wider samples of users. However, our work certainly casts
automatically mined and the user under attack could be alerted a serious doubt regarding the presence of such differences
to the presence/absence of the attack, even though the user may (in contrast to other real-fake contexts, such as paintings or
have himself failed to detect the attack (behaviorally). Neupane websites), which also maps well with our behavioral results,
et al. suggested such an approach in the context of phishing thereby explaining human-centered voice insecurity.
attacks [33]. Our study investigates the same line in the context
of voice synthesis attacks. In light of our results, perhaps the only feasible way to
protect users from such attacks would be by making them
The neuroimaging technique used in our study, i.e., fNIRS, more aware of the threat, and possibly by developing technical
is a non-invasive imaging method to measure the relative solutions to assist the users. Even though machine-based
concentration of oxygenated hemoglobin (oxy-Hb) and deoxy- voice biometric systems have also been shown to be vulner-
genated hemoglobin (deoxy-Hb) in brain cortex [11], [24], able to voice synthesis attacks [30], the security community
[26]. By examining the changes in oxy-Hb and deoxy-Hb, can certainly work, and has been working, towards making
we can infer the activities in the neural areas of interest. We such techniques more secure with advanced liveness detection
carefully selected fNIRS as our study platform as it has the mechanisms, which could aid the end users against voice
unique capabilities to provide spatially accurate brain activity synthesis based social engineering scams, whenever possible.
information better than the EEG (Electroencephalography)
and similar to that of fMRI (Functional Magnetic Resonance
Broader Scientific Significance: We believe that our work
Imaging) [26]. Therefore, we preferred fNIRS to ensure we
helps to advance the science of human-centered voice security,
capture the features from both temporal and spatial domains.
in many unique ways. It also serves to reveal the fundamental
Unlike fMRI, fNIRS also allows us to pursue the study in envi-
neural basis underlying voice-based security, and highlights
ronments with better ecological validity since the participants
users’ susceptibility to advanced voice synthesis attacks. Table
do not have to be in a supine position in the fMRI scanner
X gives a snapshot of all our results.
while making decisions.
Beyond the aforementioned novel contributions, one im-
Our Contributions: We design and conduct an fNIRS study portant scientific attribute of our work lies in recreating and
to pursue a thorough investigation of users’ processing of real revalidating the findings from the prior behavioral-only (task
and morphed voices. We provide a comprehensive analysis performance) study of voice insecurity reported in the literature
of the collected neuroimaging data set and the behavioral [30] through independent settings. Similar to [30], our results
task performance data set. Contrary to our hypothesis (and confirm the susceptibility of human users to voice imperson-
contrary to the case of website/painting legitimacy detection), ation attacks.
2
Security Relevance and Implications: Although our work such attacks. Shirvanian et al. [38] successfully launched man-
is informed by neuroscience, it is deeply rooted in computer in-the-middle attacks against “Crypto Phones”, where users
security and provides valuable implications for the security verbally exchange and compare each other’s authentication
community. We conduct a neuroimaging-based user study and codes before establishing secure communications, using mor-
show why attackers might be successful at morphed voice phed voice samples. Lewison et al. [28] proposed a block-
attacks. Many similar security studies focusing on human chain system in which face-recognition is combined with
neuro-physiology have been published as a new line of re- voice verification to prevent voice morphing attacks. Bai et
search in mainstream security/HCI venues, e.g., [8], [32]– al. [4] proposed the use of voices to authenticate certificates.
[34]. How users perform at crucial security tasks from a However, it can be subject to potential morphing attacks. In
neurological standpoint is therefore of great interest to the this light, it is important to understand why users may fail to
security community. identify the voice of a speaker under morphing attacks, and
inform the designers of automated speech recognition systems
This line of research followed in our work provides novel as to how users may distinctively process speakers’ voices.
security insights and lessons that are not possible to elicit via
behavioral studies alone. For example, prior studies [32]– Recently, several studies have reported the use of neu-
[34] showed that security (phishing) attacks can be detected roimaging to understand the underlying neural processes con-
based on neural cues, although users may themselves not be trolling users’ decision-making in security tasks [8], [32]–[34].
able to detect these attacks. Following this line, our work Specifically, Neupane et al. [34] analyzed brain signals when
conducted an fNIRS study to dissect users’ behavior under users were detecting the legitimacy of phishing websites and
voice impersonation attacks, an understudied attack vector. reading malware warnings using a state-of-the-art neuroimag-
Our results show that even brain responses cannot be used ing technique fMRI. Neupane et al. [32] followed up the study
to detect such attacks, which serve to explain why users are with EEG and eye-tracking to understand users’ neural and
so susceptible to these attacks. gaze metrics during phishing detection and malware warnings
tasks in a near-realistic environment.
II. BACKGROUND & P RIOR W ORK Relevant to our study is the fNIRS study of phishing
detection by Neupane et al. [33]. They used fNIRS to measure
In this section, we provide an overview on fNIRS system, neural activities when users were identifying real and phishing
and discuss the related works. websites, and built neural cues based machine learning models
to predict real and phishing website. Unlike this study, we use
A. fNIRS Overview the fNIRS system to measure the neural behavior in a different
application paradigm – voice security.
The non-invasive fNIRS technology has unique capabilities
in that it can provide spatially accurate brain activity informa- There are some neuroscience studies on voice recognition
tion in line with fMRI, but it can do so in an ecologically (e.g., [3], [6], [7], [20]). Belin et al. [6] studied the brain
valid experimental environments (not inside a scanner under areas involved in voice recognition. Formisano et al. [20] in
a supine posture). It is easy to set-up and robust to motion their fMRI study showed the feasibility of decoding speech
artifacts, and offers high spatial resolution [11], [24], [26]. The content and speaker identity from observation of auditory
basis of fNIRS is the usage of near-infrared light (700-900 nm cortical activation patterns of the listener. Similar to our study,
range), which can penetrate through scalp to reach the brain Bethman et al. [7] studied how human brain processes voices
cortex. Optical fibers are placed on the surface of the head for of a famous familiar speaker and an unfamiliar speaker. In
illumination while detection fibers measure light reflected back contrast to all these studies, our research focuses on the
[10], [40]. The differences in absorption spectra of the lights security aspect of voice impersonation attack and explains why
by oxy-Hb and deoxy-Hb allow the measurement of relative users fail to identify fake voices, and evaluates the privacy
changes in hemoglobin in the blood in brain. fNIRS provides issues related to the BCI devices.
better temporal resolution compared to fMRI and better spatial
resolution (approximately 5mm) compared to EEG. Based on C. The Premise of our Hypothesis
these attractive features, we have chosen fNIRS as a platform Prior researchers [25], [32]–[34] have also conducted stud-
to conduct our study reported in this paper. The hemodynamic ies to understand why people fall for fake artifacts (e.g., fake
changes measured by the fNIRS occurs at slow rate of 6-9 sec images, fake websites) analyzing their neural signals. Huang et
similar to fMRI, so the trial duration is relatively longer than al. [25] had conducted a study to understand how human brain
EEG [10]. reacts when they are asked to identify if a given Rembrandt
painting was a real or a fake. They reported differences in
B. Related Work neural activation at left and right visual areas when users were
viewing real and fake Rembrandt paintings (see Figure 2).
Voice spoofing attacks are a serious threat to users’ secu- Neupane et al. had also conducted studies [32]–[34] to analyze
rity and privacy. Previous studies have shown that attackers the neural activations measured using different neuroimaging
equipped with voice morphing techniques could breach ma- devices (e.g., EEG [32], fNIRS [33] and fMRI [34]) when
chine and human speaker verification systems [28], [30], [38]. users were subjected to phishing attacks. The users were asked
Mukhopadhyay et al. [30] attacked both machine-based and to identify if a given website was real or fake and their
human-based speaker verification system with morphed voice neural signals were concurrently measured when they were
samples and different speakers’ (other users’) voice samples. viewing these websites. They reported differences in the brain
They report that both human and machine are vulnerable to activations when users were processing real and fake websites.
3
detection task. We set-up our study to test the hypothesis that
the brain areas related to the real-fake decision making should
be activated differently when users are listening to the original
and fake voices of a speaker as well. We, therefore, set-up
the fNIRS headset configuration to measure the frontal and
temporo-parietal brain regions reported in these previous “real-
fake detection” studies. Next, we also try to automatically infer
the type of voice (real or morphed voice) a user is listening to
based on the fNIRS-measured neural signals.
D. Voice Synthesis
Voice synthesis is commonly used in text-to-speech sys-
Fig. 1. Activation in right middle frontal gyri (RMFG), right inferior frotal tems. The quality of the synthesized voice is judged by its
gyri (RIFG), left inferior parietal lobule (LIPL), left precentral gyrus, right intelligibility and its similarity to a targeted human voice. Tra-
cerebellum, and left cingulate gyrus is dependent on whether or not the ditionally, it is known to be challenging for a voice synthesizer
participant was viewing an image of a genuine website (real vs. fake) [34]. to produce a natural human speech (without noticeable noise)
TABLE II. B RAIN AREAS ACTIVATED IN PHISHING TASK [34] AND artificially and make it indistinguishable from the human voice
THEIR CORRESPONDING FUNCTIONS [18]. Additionally, voice synthesizers require a huge amount of
data from a target speaker to learn the phonemes and generate
Brain Areas Functions
a synthesized speech of the target speaker.
Orbitofrontal area Decision-making; judgment
Middle frontal, inferior Working memory However, newer techniques have emerged that may do a
frontal, and inferior much better job of synthesizing a voice. Voice morphing (also
parietal areas referred to as voice conversion and voice transformation) is one
Right cerebellum and Feedforward and feedback projections such technique to generate a more naturalistic (human-type)
left precentral gyrus voices with fewer samples. Voice morphing software takes
Occipital cortex Visual processing, search some samples of a speech spoken by a source speaker and
produces a sound as if it was spoken by the target speaker by
mapping between the spectral features of the source speaker’s
Specifically, Neupane et al. [34] revealed statistically sig- and the target speaker’s voice [42]. Previous research [30],
nificant activity in several areas of the brain that are critical and [38] has reported via behavioral studies that the morphed voice
specific to making “real” or “fake” judgments. For the websites attack may be successful against users with high probability.
those the participants identified as fake (contrasted with real), In our study, we followed a methodology described in [30] and
participants activated right middle, inferior, and orbital frontal used the CMU Festvox voice converter [19], an academic off-
gyri, and left inferior parietal lobule. The functions governed the-shelf voice morphing tool, to generate the morphed voices
by these areas are listed in Table II. On the other hand, when of the target speakers.
real websites were identified, participants showed increased
activity in several regions, such as the left precentral gyrus,
right cerebellum, and the occipital cortex (see Figure 1). E. Threat Model
Neupane et al. [33] also explored a feasibility of fake website
In our work, we study how access to a few voice samples
detection system based on fNIRS-measured neural cues, where
of a speaker can be used to launch attacks on human-based
they were able to obtain the best area under the curve of 76%.
speaker verification systems. We assume that an attacker has
collected a few voice samples previously spoken by the target
victim with or without her consent. The attacker can get such
voice samples from any public speeches made by the victim
or by stealthily following the victim and recording audio as
he speaks with other people. The attacker then uses the voice
samples to train a model of a morphing engine (we followed
the procedures mentioned in [30] for morphing engine), such
that the model creates the victim’s voice for any (new) arbitrary
speech spoken by the attacker. This morphed speech can now
be used to attack human-based speaker verification systems.
The attackers can either make a fake phone call, or leave voice
messages, or produce a fake video with the victim’s voice in
Fig. 2. Activation in the visual areas, calcarine sulcus (CS), is dependent on
whether or not the participant was viewing an image of a genuine Rembrandt
it and post it online.
(real vs. fake) [25].
F. Terminology and Attack Description
Similar to the tasks of identifying real and fake websites
or images, the speaker legitimacy detection task also involves Victim Speakers: These are the speakers whose voices were
real-fake decision making and hence we hypothesized that manipulated to launch voice impersonation attacks on partici-
similar areas should be activated in the speaker legitimacy pants in our study.
4
Familiar Speakers: In our study, we used the voice samples are listening to the morphed voice of a speaker compared to
of Oprah Winfrey and Morgan Freeman to represent the set of the original voice of the speaker.
familiar speakers. The reason for choosing these celebrities
in our case study is to leverage their distinct and unique To test this hypothesis, we used original, morphed, and dif-
voice texture and people’s pre-existing familiarity with their ferent voices for two types of speakers – familiar speakers and
voices. The participants were also asked to answer if they briefly-familiar speakers(see Section II-F). The participants
were familiar with the voice of these celebrities as “a yes/no were asked regarding their familiarity with these speakers’
question” before their participation. voices as “a yes/no question” before the experiment. During
the experiment, the participants were asked to identify the
Briefly Familiar Speakers: In our study, these are the speakers real (legitimate) and fake (illegitimate) voice of a speaker. We
whom the participants did not know before the study and were assumed the real voices may impose more trust compared to
only familiarized during the experimental task only to establish the fake voices. The failure of users in detecting such attacks
brief familiarity. The briefly familiar speakers represent a set would demonstrate a vulnerability of numerous real-world
of people with whom the users have previously interacted only scenarios that rely (implicitly) on human speaker verification.
for a short-term (e.g., a brief conversation at a conference).
Stimuli Creation: Voice Conversion: We followed the
Different Speaker Attack: Different speakers are the arbitrary
methodology similar to the one reported in [30] to create our
people who attempt to use their own voice to impersonate as
dataset. For familiar speakers, we collected the voice samples
the victim speaker. In our study, using a different speaker’s
of two popular celebrities, Oprah Winfrey and Morgan Free-
voice, replacing the voice of a legitimate speaker to fool the
man, available from the Internet. For unfamiliar speakers, we
participants, is referred to as the different speaker attack.
recruited participants via Amazon Mechanical Turk (MTurk).
Morphed Voice Attack: Attackers can create spoofed voice We asked twenty American speakers in MTurk to record the
using speech morphing techniques to impersonate the voice of speech of these two celebrities in their own voices. They were
a victim user, referred to as a morphed voice. In our study, the asked to read the same sentences the celebrities had spoken in
use of such a voice to deceive other users to extract their private the recorded audio and also to imitate the celebrities’ speaking
information is therefore called the morphed voice attack. style, pace and emotion. We fed these original voice samples of
the celebrities and the recorded voice samples from one male
Speaker Legitimacy Detection: In our study, this represents the and one female speaker among these twenty speakers to the
act of identifying whether the given voice sample belongs to CMU Festvox voice converter [19] to generate the morphed
the original speaker or is generated by a morphed engine. The voices of Morgan Freeman and Oprah Winfrey.
different speaker attack is used as a baseline for the morphing In line with the terminology used in [30], in our study, the
attack, since it is expected that people might be able to detect original recording of a victim speaker’s voice is referred to
the different speaker attack well. as a “real/original voice”. For each speaker, the fake voices
created using speech morphing techniques is referred as a
III. S TUDY D ESIGN & DATA C OLLECTION “morphed voice”, and recorded speech in other speaker’s
In this section, we present the design of our experimental voices is referred as a “different speaker”.
task, the set-up involving fNIRS, and the protocol we followed Experiment Design: We used an event-related (ER) design for
for data collection with human participants. our study. In ER design, each trial is presented as an event with
longer inter-trial-interval and we can isolate fNIRS response to
A. Ethical and Safety Considerations each item separately [37]. We familiarized the participants with
Our study was approved by our university’s institutional the voice a of victim speaker for 1-minute. We could not let the
review board. We ensured the participation in the study was participants replay the original audio during familiarization as
strictly voluntary and the participants were informed of the it would have made the experiment longer. As the experiment
option to withdraw from the study at any point in time. We gets longer, fatigue effects may set in, eventually affecting the
obtained an informed consent from the participants and made quality of the brain signals recorded.. After familiarization, we
sure they were comfortable during the experiment. We also fol- presented 12 randomly selected speech samples (4 in original
lowed the standard best practices to protect the confidentiality speaker’s voice voice, 4 in morphed voice and 4 in different
and privacy of participant’s data (task responses and fNIRS speaker’s voice and asked participants to identify legitimate
data). and fake voices of the victim speakers. The process was
repeated for four victim speakers (2 familiar speakers, and
2 briefly familiar speakers) randomly. Following [30], our
B. Design of the Voice Recognition Task
study participants were not informed about voice morphing
The study design for our voice recognition task followed to prevent explicit priming in the security task. In real life
the one employed in recent task performance study of speaker also, people may have to decide a speaker’s legitimacy without
verification [30]. However, unlike [30], our study captured knowing the voices may have been morphed.
neural signals in addition to the task performance data. We
The experiment started with the Firefox browser loading
designed our experiment to test the following hypothesis:
the instructions page (specifying the tasks the participants are
Hypothesis 1 The activation in frontopolar and temporopari- supposed to perform) for 30 seconds. This was followed by
etal areas, which covers most of the regions activated in previ- a rest page of 14 sec (+ sign shown at the center of a blank
ous studies (see Section II-C), will be high when participants page) during which participants were asked to relax. We next
5
Speaker Voice Real Good
Instruction Rest Rest
Familiarization Sample Or Fake? Bye
(30sec) (14sec) (8sec)
(60sec) (15sec) (5sec) (5sec)
Repeat loop for 12 voice samples
Repeat loop for 4 victim speakers
Fig. 3. The flow diagram depicts the presentation of trials in the experiment. The participants were familiarized with a speaker, and were asked to recognize
the short voice samples presented next as real or fake.
TABLE III. D EMOGRAPHICS
played a speech sample of a victim speaker for 60 sec, and
then showed a rest page for 8 sec. This was followed by Participant Size N = 20
12 trials, each 20 sec long. Each trial consisted of a voice Gender(%)
(corresponding to a fake/real voice of the speaker) played for Male 50%
15 sec, followed by a 5 sec long response page. The response Female 50%
page had a dialog box with the question, “Do you think the Age(%)
voice is of the original speaker?” and the “Yes” and “No” 18-22 20%
22-26 55%
buttons. The participants answered the question using mouse
27-31 20%
input. A rest page of 8 sec was loaded after each trial. Rest
31+ 5%
trials are considered as windows of baseline brain activity. The Handedness(%)
process was repeated for four speakers and the experiment Right-Handed 90%
ended with the goodbye note of 5 sec. The system recorded the Left-Handed 10%
participant’s neural data, responses and response time. Figure Background(%)
3 depicts the flow diagram. High School 10%
Bachelor’s 20%
C. Study Protocol Masters 55%
Doctorate 10%
Our study followed a within-subject design, whereby all Others 15%
participants performed the same set of (randomized) trials
corresponding to the voice recognition task.
Recruitment and Preparation Phase: We recruited twenty Our participant demographics is also well-aligned with prior
healthy participants from the broader university community neuroimaging security studies [7], [8], [32], [34]. Each partic-
(including students and staff) by distributing the study ad- ipant was paid $10 upon completion.
vertisements across our university’s campus. We asked the
participants about their familiarity with Morgan Freeman’s and Task Execution Phase: To execute the experiment, we used
Oprah Winfrey’s voices, i.e., if the participants have heard the two dedicated computers, one in which the experimental task
celebrities’ voices before and could recognize those voices, was presented and the task responses and the response times
along with participants’ age, gender, and educational back- were logged, namely stimuli computer, and the other in which
ground in the pre-test questionnaire. Of the 20 participants, fNIRS data measured during the experiment was recorded,
10 were male and 10 were female. All were English speaking namely data collection computer. We synchronized the data
participants in the age range of 19-36 years with a mean age of between these two computers by placing a marker in the fNIRS
24.5 years. Table III summarizes the demographic information data at the beginning of the task.
of our participants.
We recorded hemodynamic responses, the rapid delivery
Previous power analysis studies have found 20 to be an of blood to active neuronal tissues [23], in the frontal cortex
optimal number of participants for such studies. For instance, and the temporoparietal cortex on both hemispheres using
statistical power analysis of ER-design fMRI studies have fNIRS system developed by Hitachi Medical (ETG 4000) in
demonstrated that 80% of clusters of activation proved repro- our experiment. These areas cover the brain regions activated
ducible with a sample size of 20 subjects [31]. Both fMRI in previous studies [33], [34] (see Section II-C). We took
and fNIRS are based on hemodynamic response of the BOLD the measurement of each participant’s head to determine the
principle, so we assumed similar power analysis for fMRI best size of probe cap that would fit the participant. The
and fNIRS. Also previous fNIRS studies have shown that fNIRS optodes were then placed on participant’s head using
fNIRS measurements are reliable with 12 participants [35]. fNIRS probe cap which ensured standardized sensor placement
6according to the well established 10-20 system. The inter- and deoxyHb concentrations at the baseline (rest) and the task
optode distance was set to 30mm and the data was acquired performance is said to determine the location in the cortex of
at the frequency of 10Hz. Figure 4 depicts the experimental where activities are happening [27].
set-up used in our study.
The signal measured by each channel was related to the part
of the brain above which the channel was placed. These fNIRS
channels were virtually registered onto the stereotactic brain
coordinate system using Tsuzuki’s 3D-digitizer free method
[39]. This method allowed us to place a virtual optode holder
on the scalp by registering optodes and channels onto reference
brains. We then identified the brain area represented by each
channel and grouped these channels based on the majority
of the Brodmann Area [9] they covered. We considered
five broadman areas, namely, dorsolateral prefrontal cortex,
orbitofrontal gyrus, frontopolar area, superior temporal gyrus
and middle temporal gyrus, found to be activated in previous
studies [25], [33], [34] for real-fake artifacts to measure
differences in neural activities between real and fake voices.
These areas are referred as our regions of interest (ROIs).
As mentioned in III-B, the hypothesis of our study was that
Fig. 4. Experimental setup used in our voice impersonation attack study. The the areas related to decision making, trust, working-memory,
instructions were shown on the stimuli computer’s screen. Audio was played and familiarity will have different activations for real and fake
back through external PC speakers (not shown in the figure). voices as well, similar to other real-fake artifacts, like paintings
and websites. The oxy-Hb and deoxy-Hb measured by the
Calibration is needed to ensure the quality of the data channels in each group was then separately averaged for each
recorded is good. The participant was then moved to the stimuli trial.
computer for performing the speaker voice recognition task.
Next, the participants were instructed on the tasks they were
B. Statistical Tests
performing in our study. They were asked to use the mouse
to enter the responses and were requested to minimize the We used IBM SPSS [36] for the purpose of statistical
body movements during the experiment. The participants then analysis reported in our study. Kolmogorov-Smirnov test used
performed the task discussed in Section III-B. for normality detection revealed that oxy-Hb and deoxy-Hb
data were non-normal. Thus, Friedman’s test and Wilcoxon
In our study, we recorded the changes in the concentra-
Singed-Rank Test (WSRT) were used for measuring differ-
tion of oxygenated hemoglobin (oxy-Hb) and deoxygenated
ences in the means of different groups of trials underlying
hemoglobin (deoxy-Hb) through the fNIRS device, and task
our analysis. Following the correction methodology adopted
performance metrics (response and response times) while the
in [33], and since our analysis focused on pre-established
participants performed the tasks as the repeated measures of
ROIs per our hypotheses, the comparisons at each of the ROI
our study.
were considered separately and were corrected using Holm-
bonferroni correction [1]. We also report the effect size
√ of
IV. A NALYSIS M ETHODOLOGY WSRT which was calculated using the formula r = Z/ N ,
In this section, we provide the steps we followed to where Z is the value of the z-statistic and N is the number of
process the neural signals for our analysis. We also provide observations on which Z is based. Cohen criteria [12] reports
an overview of the statistical tests we conducted on our data. effect size > .1 as small, > .3 as medium and > .5 as large.
A. Neural Data Processing V. TASK P ERFORMANCE R ESULTS
Raw light intensity fNIRS data collected in the experiment To recall, in our experimental task, participants were asked
were processed to remove high-frequency noise and motion to answer if the voice trial played was of the “real” speaker or a
artifacts using Hitachi’s data acquisition software. A bandpass “fake” speaker. We had logged the participants’ responses and
filter saving the frequencies between .5 and .01 Hz was used response times during the experiment. A participant’s response
to attenuate signatures representing respiratory and cardiac was marked as correct if she had identified the original
fluctuations. We used the modified Beer-Lambert law [14] to speaker’s voice as real, and the other speakers’ (morphed and
transform the resulting light intensity data into relative con- different speaker) voice as fake. We then calculated the average
centration changes of oxygenated hemoglobin (oxy-Hb) and accuracy and response time (RTime) the participants spent on
deoxygenated hemoglobin (deoxy-Hb) [40]. We then designed providing answers for each type of trial. Accuracy is defined
an in-house software to extract the average oxy-Hb and average as the ratio of the total number of correctly identified instances
deoxy-Hb from each channel for each trial (15 second voice to the total number of samples presented to each participant.
sample) presented in the experimental task.
From Table V, we observe that the overall accuracy of
The hemoglobin is called oxy-Hb when it transports correctly identifying the voice of the speaker is around 64%
oxygen to cerebral tissue and is called deoxy-Hb when it which is only slightly better than the random guessing (50%).
releases oxygen after metabolism. The difference in oxyHb It seems highest for the original speaker and the lowest for
7TABLE IV. R EGIONS OF INTEREST (ROI): T HE BRAIN AREAS COVERED BY OUR F NIRS PROBE - CAP
# ROI Name Acronym Brodmann Area # Functionality
2 Dorsolateral Prefrontal Cortex DLPFC 9 Working memory, attention
3 FrontoPolar Area FPA 10 Memory recall, executive functions
7 Superior Temporal Gyrus STG 22 Primary auditory cortex, auditory processing
8 Middle Temporal Gyrus MTG 21 Recognition of known faces
9 Orbitofrontal Area OFA 11 Cognitive processing, decision making, trust
TABLE V. A LL S PEAKERS : ACCURACY (%), P RECISION (%), R ECALL (%) AND F- MEASURE (%) AND R ESPONSE T IME ( SECONDS )
Acc Prec Rec FM RTime
Trial
µ (σ) µ (σ) µ (σ) µ (σ) µ (σ)
Original 82.1 (16.6) 50.63 (12.5) 83.2 (16.1) 61.31 (9.0) 2.57 (0.5)
Morph 42.8 (24.1) 46.71 (19.7) 42.8 (24.1) 43.40 (19.8) 2.58 (0.5)
Different 67.2 (21.5) 47.82 (16.0) 68.1 (21.2) 55.82 (16.0) 2.51 (0.5)
Average 64.2 (11.5) 48.39 (15.3) 64.7 (20.7) 53.51 (15.0) 2.54 (0.5)
the morphed speakers. Also, we notice that the participants We ran Wilcoxon-Signed Rank Tests (WSRT) to evaluate
reported 58% of the morphed speakers and 33% of different the differences in mean oxy-Hb and mean deoxy-Hb at each
speakers as real speakers. This shows that the morphed speak- regions of Interest (ROIs) between the original trial vs. the rest
ers were more successful than the different speakers in voice trialVI. We found statistically significant differences in oxy-Hb
impersonation attacks. Our results are in line with the task at the dorsolateral prefrontal cortex, frontopolar area, superior
performance results of voice impersonation attacks reported temporal gyrus and middle temporal gyrus for the original
by Mukhopadhyay et al. [30]. speaker trial than the rest trial (such differences are listed in
Table VI, rows 1-4 and Figure 5(a)). Statistically significant
The Friedman’s test showed a statistically significant differ- differences in deoxy-Hb at the dorsolateral prefrontal cortex,
ence in mean accuracies across original, morphed and different frontopolar area, middle temporal gyrus and orbitofrontal area
speaker voices (χ2 (20)=17.71, pFig. 5. Activation regions with statistically significant oxy-Hb changes: (a) Original vs Rest; (b) Morphed vs Rest; (c) Different vs Rest.
TABLE VIII. N EURAL ACTIVATIONS : D IFFERENT S PEAKER VS . R EST
activation in superior temporal gyrus and middle temporal
# ROI-Type Hb-Type p-value Effect Size gyrus, related to auditory processing [43], shows that the
1 DLPFC oxy .007 .60 participants were carefully processing the voice of the speakers
2 FPA oxy .001 .71 to decide their legitimacy. This shows that the participants were
3 STG oxy .000 .83 actively trying to decide if the given voice was real or fake.
4 MTG oxy .004 .63
5 OFA oxy .042 .45
B. Speaker Legitimacy Analysis
6 DLPFC deoxy .027 .49
7 FPA deoxy .001 .73 Now we present the results of contrasting neural activations
8 MTG deoxy .000 .81 between the original speaker and fake speakers (i.e., the
9 OFA deoxy .046 .44 morphed speaker and the different speaker). These compar-
isons of the brain activities delineate the brain areas involved
in processing the voices of original and morphed speakers
and mean deoxy-Hb between the different speaker trial and (synthesized voices), and original and different speakers. To
the rest trial at various regions of interest revealed statistically recall, the different speaker scenario is used as a baseline to
significant differences in oxy-Hb at dorsolateral prefrontal study the morphed speaker scenario.
cortex, frontopolar area, superior temporal gyrus, middle tem-
poral gyrus and orbitofrontal gyrus for different speaker trial Contrast 1: Original Speaker vs. Morphed Voice: This anal-
than in the rest trial (Table VIII, rows 1-5 and Figure 5(c)). ysis provides an understanding of how the original speaker’s
It also rendered statistically significant differences in deoxy- voice and morphed speaker’s voices are perceived by the
Hb at dorsolateral prefrontal cortex, frontopolar area, middle human brain. To recall, our experimental task had four victim
temporal gyrus, and orbitofrontal area for the different speaker speakers. All these speakers were familiarized to participants
trial compared to the rest trial (Table VIII, rows 4-6). during the experiment. In this analysis, we examined the neural
activities when participants were listening to all original speak-
Interpretation: Listening, attention and decision-making are ers and all morphed speakers. For the same, we ran WSRT at
critical components of higher cognitive function. The detection different ROIs to evaluate the differences in mean oxy-Hb, and
of voice impersonation attacks in our study involves all these deoxy-Hb between original and morphed voice. However, we
elements in helping participants figure out the original and did not observe any statistically significant differences.
the fake voices of the speakers. The statistically significant
activity in the dorsolateral prefrontal cortex and the frontopolar Contrast 2: Original Speaker vs. Different Speaker: In this
area at the neural level (Tables VI, VIII and VII) is indica- analysis, we compared the neural metrics when participants
tive of the involvement of working memory and executive were listening to the voice of original speaker vs. the voice of
cognitive functions in this task. The previous studies [5], a different speaker. We hypothesized that the original speakers
[13] have found interaction among the dorsolateral prefrontal – since they were familiarized to participants – will produce
cortex, frontopolar area and orbitofrontal area to play a critical different neural activations than the different speakers.
role in conflict-dependent decision-making. The activation in For this analysis, we applied WSRT to contrast the mean
orbitofrontal area portrays that the participants were being oxy-Hb and mean deoxy-Hb at different ROIs between all
suspicious while identifying the different voices presented to the voices of original speaker and the voices of different
them. They were sometimes trusting the voice of the real speaker. It revealed statistically significant differences in oxy-
speaker as real and sometimes distrusting the voice as fake. Hb for original speaker than different speaker at superior
A study by Dimoka et al. [15] found that trust was associated temporal gyrus (p=.029) with medium effect size (r=.48).
with lower activation in orbitofrontal area. The activation in These differences are visualized in Figure 6.
orbitofrontal cortex is observed only when users are listening
to voice of different speaker indicating that they were being We also applied WSRT to contrast the neural activity
suspicious when they were asked to identify different speaker’s for original and different speakers’ voices only corresponding
voice sample. This level of suspicion is also reflected in the to the samples correctly identified by the participants per
behavioral results where participants were able to detect the their behavioral response. We observed statistically significant
different speakers much better than the morphed speakers. The differences at superior temporal gyrus (ptrials compared to rest trials on WSRT, we observed differ-
ences in oxy-Hb in dorsolateral prefrontal cortex, frontopolar
area, orbitofrontal area, middle temporal gyrus, and superior
temporal gyrus (all p-values less than pVII. N EURAL A NALYTICS : O RIGINAL VS . M ORPHED identifying the voice of morphed speaker vs. original speaker
C LASSIFICATION obtained was 53% (see Table IX). To improve the results, we
also evaluated the classification model with the best subset of
In the previous section, we observed that the neural activity features selected using correlation-based feature selection al-
in original vs. morphed speakers was not statistically signifi- gorithm [22], and the best F-measure we achieved was 56.2%.
cantly different. In this section, to further validate this result, We did not see statistically significant difference in these F-
we show that the machine-learning on neural data can also not measures when compared to those of human detection and
help classify these differences. random guessing. These results show that, similar to human
behavioral performance, even neural patterns and machine
A. Feature and Performance Metrics learning may not be successful to identify voice impersonation
For extracting features, we normalized the oxy-Hb and attacks.
deoxy-Hb data in each region of interest using z-score nor- TABLE IX. S PEAKER L EGITIMACY D ETECTION : P RECISION , RECALL
malization. Next, at each ROI, we computed maximum, mini- AND F- MEASURE ( HIGHLIGHTED BEST CLASSIFIER )
mum, average, standard deviation, slope, variation, skew, and Prec Rec FM
kurtoisis for the normalized oxy-Hb and deoxy-Hb data for
each trial as features. We computed these features separately RandomTree 49.5 (9.5) 48.8 (10.9) 48.9 (9.7)
Logistic 49.4 (11.2) 50.0 (11.9) 49.4 (10.9)
for the first and second half of each 15 second long task. We
J48 48.8 (10.7) 48.8 (12.1) 48.6 (11.2)
also separated the 15 seconds of data into 3 segments, and we NaiveBayes 46.7 (10.1) 43.8 (11.2) 44.8 (9.5)
took the average value of each of these 3 segments for the MultilayerPerceptron 50.0 (11.3) 48.8 (11.1) 49.0 (10.3)
oxy-Hb and deoxy-Hb datastreams. We had one feature vector LMT 48.3 (11.1) 54.4 (12.4) 50.8 (10.9)
for each trial. SimpleLogistic 49.4 (10.3) 59.1 (13.5) 53.2 (10.1)
To build classification models, we utilized off-the-shelf SMO 49.0 (12.6) 47.2 (12.9) 47.8 (12.0)
machine learning algorithms provided by Weka. To this end, RandomForest 47.1 (9.5) 52.8 (11.4) 49.6 (9.8)
we used 10-fold cross-validation for estimation and validation
of the models built on different algorithms including: Trees –
J48, Logistic Model Trees (LMT), Random Forest (RF) and VIII. D ISCUSSION AND F UTURE W ORK
Random Tree (RT); Functions – Neural Networks, Multilayer In this section, we summarize and discuss the main findings
Perceptron (MP), Support Vector Machines (SMO), Logistics from our work, outline the strengths/limitations of our study
(L) and Simple Logistic (SL), and Bayesian Networks – Naive and point to future work. Table X provides a summary of our
Bayes (NB). overall results.
As performance measures, we report the precision (P rec),
recall (Rec), F-measure (F M ) or F1 Score for machine Summary and Insights: The participants in our study showed
learning classification models. P rec refers to the accuracy increased activation in dorsolateral prefrontal cortex, frontopo-
of the system in rejecting negative classes and measures the lar cortex and orbitofrontal gyrus,the areas associated with
security of the proposed system. Rec is the accuracy of the decision-making, working memory, memory recall and trust
system in accepting positive classes and measures the usability while deciding on the legitimacy of the voices of speakers
of the proposed system. Low recall leads to high rejection of compared to the rest trials. They also showed activation in
positive instances, hence unusable, and low precision leads to superior temporal gyrus, which is the region that processes
high acceptance of negative instances, hence insecure. F M the auditory signals. Overall, these results show that the users
represents the balance between precision and recall. were certainly putting a considerable effort in making real vs.
fake decisions as reflected by their brain activity in regions
True positive (TP) represents the total number of correctly correlated with higher order cognitive processing. However,
identified instances belonging to the positive class while true our behavioral results suggests that users were not doing well
negative(TN) is the number correctly rejected instances of in identifying original, morphed and different speakers’ voices.
negative class. Similarly, false positive (FP) is the number of Perhaps the poor behavioral result was because the participants
negative instances incorrectly accepted as positive class and were unaware of the fact that the voices could be morphed (a
false negative (FN) represents the number of times the positive real-world attack scenario). Another reason could be that the
class is rejected. quality of voice morphing technology is very good in capturing
features of the original speaker/victim.
B. Speaker Legitimacy Detection Accuracies
We also analyzed the differences in neural activities when
Our statistical analysis of neural data related to original participants were listening to original voice and morphed voice
speaker, morphed speaker voices showed some, albeit minimal, of a speaker. However, we did not see any statistically signif-
significant differences in oxy-Hb and deoxy-Hb. Taking this icant differences in the activations in brain areas reported in
into account, we extracted features from these underlying previous real–fake studies [34]. Although the lack of statistical
neural differences and built a 2-class classifiers as discussed difference does not necessarily mean that the differences do not
in Section VII-A to identify original and morphed speakers. exist, our study confirms that they may not be present always,
In this classification task, the positive class corresponds to if at all present. Our task performance results also showed that
original speaker and negative class corresponds to the mor- people were nearly as fallible to the morphed voice as they
phed. For the speaker legitimacy detection, we observed that were to the real voices. The results show that the human users
none of the classifiers performed well. The best F-measure of may be inherently incapable of distinguishing between real and
11TABLE X. R ESULTS S UMMARY: N EURAL F INGERPRINTS OF S PEAKER L EGITIMACY D ETECTION , AND C ORRESPONDING TASK P ERFORMANCE AND
M ACHINE L EARNING (ML) R ESULTS .
Task Condition Activation Regions Task ML Implications
Result Result
Voice Trial vs Rest Original vs. Rest DPLFC; FPD; N/A N/A Regions associated with working
Trial STG; MTG; OFA memory, decision-making, trust,
Morphed vs. Rest DPLFC/ FPA; STG; N/A N/A familiarity, and voice processing
MTG are all activated during the
Different vs. Rest DPLFC; FPA; STG; N/A N/A experimental task.
MTG; OFA
Speaker Legitimacy Original vs. No difference 43% 53% Both users and their brains seem to
Detection Morphed fail at detecting voice morphing
attacks
Original vs. STG 55% N/A Brain differentiates original speaker
Different and different speakers voice
morphed voices. Consequently, they may need to rely on other the headset on might have affected the performance of some
external mechanisms to help them perform this differentiation of the participants. In real-world voice impersonation attacks,
(e.g., automated detection tools). Nevertheless, even current the participants are not explicitly told to identify the real and
voice biometric solutions have been shown vulnerable to voice fake voices of the speaker unlike our study. However, this
impersonation attacks [30]. could actually be seen as a strength of our work. Our results
show that the participants were unable to detect these attacks
In line to this, we built and tested an automated mechanism despite being asked explicitly, and hence the result in the
to identify original and morphed voice extracting features from real-world attack may be even worse, where the users have
neural data corresponding to each of the two types of voices. to make the decision implicitly. Also, the participants in our
We found that it could only predict the voice correctly with study were mostly young, so the results of the study may not
the accuracy slightly better than random guessing. This shows represent the success rate of voice spoofing attacks against
that both the explicit responses of the users and the implicit older people or people with declination in hearing ability (the
activity of their brains are indicative that morphed voices are attack success rates against such populations may actually
nearly indistinguishable from the original voices. This neuro- be higher in practice [16]). However, our sample size and
behavioral phenomenon serves to show that users would be diversity is well-aligned with those of prior studies (e.g., [33],
susceptible to voice imitation attacks based on off-the-shelf [34]). Also, our participants belonged to the broader university
voice morphing techniques. We believe this to be an important community, including students, employees, and others.
finding given the already emergence of voice imitation attacks.
More advanced state-of-the-art voice modeling techniques than One limitation of our study pertains to the number of trials.
the one we used in our study, such as those offered by Lyrebird We asked the users to identify forty eight voice samples, each
and Google WaveNet [29], [41], could make people even more presented for 16 seconds, in about thirty minutes of the study.
susceptible to voice imitation attacks. Although multiple long trials are a norm in neuroimaging
studies for desired statistical power [2], the users may not
This same finding also bears positive news in other do- have to face these many challenges in a short span of time in
mains. The fact that “listeners” can not differentiate between real-life. Another limitation pertains to the fNIRS device we
the original voice of a speaker from a morphed voice of the used for the study. fNIRS captures the brain activities mostly
same speaker, at both neural and behavioral level, seems to close to the cortex of the brain. So, it might have missed
suggest that current morphing technology may be ready to the neural activities in the inner core of the brain. The users’
serve those who have lost their voices. decision making process towards the end of 15 seconds might
In our study, the participants were not trained to look for also not be captured by fNIRS. It is an inherent limitation in
specific forms of fabrications in the fake voice samples. Such measuring the BOLD signal. Finally, the fact that the voices
an explicit training of users to detect fabrications could make of the speakers, Oprah and Freeman used in our study were
an impact on users’ ability to detect fabricated voices. Future distinctive and familiar to the speakers, might have confounded
studies should be conducted to analyze the effect of such the neural activity in the frontal cortex. Future studies might
training against users’ performance in identifying morphed be needed to explore a more realistic task set-up. Nevertheless,
voices. This will be an interesting avenue for further research. we believe that our work provides a sound first study of vocal
security from a neuro-physiological perspective with important
take-aways and insights that future studies can build upon.
Study Strengths and Limitations: In line with any other
study involving human subjects, our study had certain lim-
IX. C ONCLUDING R EMARKS
itations. The study was conducted in a lab setting. So, the
performance of the users might have been affected since they In this paper, we explored voice security through the lens
might not have been concerned about the security risks. Even of neuroscience and neuroimaging, running an fNIRS study
though we tried to emulate a real-world scenario of computer of human-centered speaker legitimacy detection. We dissected
usage in a lab setting and used a light-weight fNIRS probe the brain processes that control people’s responses to a real
caps designed for the comfort level, performing the task with speaker’s voice, a different speaker’s voice, and a morphed
12You can also read
