Alexa, Can I Trust You? - TSAPPS at NIST
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Alexa, Can I Trust You?
Hyunji Chung, Michaela Iorga, Jeffrey Voas, National Institute of Standards and Technology
Sangjin Lee, Korea University
Security diagnostics expose vulnerabilities and privacy threats that exist in
commercial Intelligent Virtual Assistants (IVA) – diagnostics offer the possibility
of securer IVA ecosystems.
Intelligent Virtual Assistants (IVAs) definition of a Whopper, pulled from
open a new world, a world where you the website Wikipedia [2]. Since the
can talk to a machine as if it were a website can be edited by users, the
human and the machine will perform definition had been changed and
the work you request. For example, “cyanide” was inserted as an
when you wake up, “Hey, what’s on ingredient in one version. Such kind of
my schedule for today?” Before you malicious information, if followed ad
leave the house, “Hey, what’s my litter am, can cause harm.
commute?” For dinner, “Hey, order
In this column, we urge readers to
one large size pepperoni pizza.” When
think about the potential security and
you go to sleep, “Hey, turn off the bed
privacy concerns of this technology.
room lights.” Ideally, such
For instance, (1) “Is my IVA secure?”,
conversations should be solely
(2) “Is it listening to my
between you and the device assisting
conversations?”, (3) “Where is my
you. But are they? Do you know?
voice data stored?”, etc. The fact that
Where is the trust?
IVAs are installed in private homes
IVAs may be new and mysterious to makes this a public-facing challenge,
some consumers, but they are in the and one that attracts instant media
market place today. Gartner has said attention when problems arise. To our
that the IVA market will reach $2.1 knowledge, security and privacy
billion by 2020 [1]. Voice assistants threats of these IVAs have not received
such as Google Home, Apple's Siri and enough attention.
Amazon's Echo devices have always
WHAT IS AN INTELIGENT
been susceptible to accidental hijack.
VIRTUAL ASSISTANT?
A Google ad during the Super Bowl
that used the phrase "OK, Google" Predecessors of Assisting IoT
reportedly set off people's home Devices
devices that began reciting theIoT devices for assistance are not new. devices in the proximity of the user, an
IoT devices for assistance have assistant of this type is powered by
evolved from half-century old chatbots artificial intelligence, and the “brain”
programmed to pass the Turing test 1 of the assistant is in a virtual place, e.g.,
(e.g., Eliza and Parry). A chatbot was a cloud. These devices are
a service that people interacted with in communicating with the virtual
writing via a chat interface. They assistant, sometimes by default, but
worked by examining a user's typed more often only when configured to do
comments and identifying known so, and have no embedded intelligence.
keywords. If a keyword was found, a We will employ, in this column, the
rule that transformed the user's term IVA-enabled device when
comments was applied, and the referring to such devices.
resulting sentence was returned [3].
Well known IVAs
Today’s newer versions of IVAs can
Table 1 summarizes known IVAs from
not only respond to voice commands,
major vendors such as Amazon, Apple,
but also can play music if asked,
Google and Microsoft [5]. (in
perform keyword searches, order items,
alphabetical order)
turn on lights, open garage doors, and
can even sustain conversations [4]. Table 1. Summary of best known IVAs
IVA-enabled
Defining IVA Vendor IVA devices
There are various terms for this (Endpoints)
category of IoT devices. They include, Echo, Dot, Tab,
Amazon Alexa
Fire Tablet
but are not limited to, Smart Assistant, iPhone, iPad,
Intelligent Personal Assistant, Digital Apple Siri
Mac
Assistant, Personal Virtual Assistant, Google
Any phone with
Virtual Assistant Bot, etc. Among Now &
Google Android, Google
Google
these terms we can recognize some Home
Assistant
common keywords: ‘smart’, ‘assistant’, Any PC with
Microsoft Cortana
‘intelligent’, and ‘virtual’. In this Windows
column, we employ the term
Intelligent Virtual Assistant (IVA), ‘IVAs’ has an agent programs running
because, even though the on ‘IVA-enabled devices’ (endpoints)
communication is facilitated by such as iPhone, iPad, Mac, Fire tablet,
1
The Turing test is a test, developed by Alan Turing in 1950, of a
machine's ability to exhibit intelligent behavior equivalent to, or
indistinguishable from, that of a human.Echo, Google Home, etc. The main (endpoints). Examples of the built-in
functionality, the “brain” of an IVA, is IVA include Siri (for Apple products)
housed as a cloud service that and Cortana (for Windows-based PCs).
processes voice data (converting Examples of the stand-alone IVA
voice-to-text, performing linguistic include Alexa (that uses Echo, Echo
context analysis, and providing Dot and Tab dedicated devices) and
answers to questions.) Google Assistant (that uses Google
Home dedicated device.) The
We divide IVAs into two types: (1)
remainder of this article focuses on
built-in IVA that use multi-purpose
security and privacy threat modeling
devices (endpoints) and (2) stand-
for stand-alone IVAs that are operating
alone IVA that use dedicated devices
in peoples’ homes.
Figure 1. IVA ecosystem
IDENTIFYING people’s actual voice unauthorized entities to
THREAT VECTORS sounds which are use the data to identify
OF IVAs Personally Identifiable individuals, to
Information (PII) [6], we maliciously obtain access
To identify ways to
extend the analysis to to systems that implement
secure IVAs, we begin by
include user privacy. IVA voice recognition, or
analyzing their security
vendors are already simply to process data
vulnerabilities. Then,
storing voice data, thus and construct voice
since IVAs handle
making it possible for artifacts that could beused to impersonate these (a.1) Endpoint1: an To utilize IVAs, the IVA-
individuals. These Alexa-enabled device enabled devices need to
scenarios are problematic. (Echo); run an agent program that
communicates with the
To identify the threat (a.2) Endpoint2: a
cloud services. Major
vectors, we have learned companion app that needs
vendors are providing this
how IVAs operate along to be installed on user’s
agent by integrating it
with their components device of choice;
into their operating
through a variety of
(b) The cloud side - the systems: for example, the
analysis methods such as
‘intelligent assistant’ latest versions of iOS and
voice command tests,
Alexa that operates in the OS X have the Siri agent
firmware analysis,
Amazon’s cloud installed by default.
network traffic analysis,
environment. Microsoft Windows 10
and application analysis.
To test Alexa, we asked has the Cortana agent as
By doing so, we can
Echo questions and got one of its default
unveil useful details
answers. We learned that processes. IVA agents
about IVA ecosystems.
all the requests sent to from Amazon and Google
Alexa (through Echo) are similar in principle
IVA ECOSYSTEMS were stored in a cloud in but use dedicated devices
text format and in such as Echo and Google
In general, IVAs consist Home.
recorded voice. All the
of multiple components
conversations and actual An interesting point here
in heterogeneous
voice recordings were relates to IVA-enabled
environments. As shown
accessible through devices – these
in Figure 1, there are two
Alexa’s companion app. ‘Endpoints1’ are stand-
user-side components: (1)
Performing packet alone products designed
companion applications,
analysis we discovered to only assist in the usage
and (2) IVA-enabled
what kind of data has of IVA services. Because
devices. One of the IVAs
been stored on the cloud these home-embedded
we studied was Amazon’s
side and how to get access devices need to be
Alexa ecosystem. The
to cloud-native data. In connected to the Internet
main components of this
addition, analysis of the to communicate with the
ecosystem are grouped
firmware and software of ‘intelligent assistant’, the
into two categories:
IVA-enabled devices vendors need to provide
(a) The client side – that helped us understand the convenient interfaces for
has 2 components: overall ecosystem. configuring them andmanaging activity history. important to note that A few examples are:
Amazon and Google are IVAs are expanding their opening a garage door or
providing companion features (often referred to unlocking a house door,
applications (apps or as ‘skills’) by allowing ordering a pizza, or
web-sites) for completing third-party entities to add utilizing a social network
these activities. It is also new compatible services. service by voice.
Case 1 Case 2
Packet sniffing Packet sniffing
Web proxy Cloud
24/7 voice
Unveiling … tomorrow I have recording
communication to go conference in
mechanism Firmware DC….
analysis
Remotely controlled
speaker
User Who are you? Compromised
IVA-enabled
device
User User IVA-enabled
Companion applications device
Case 3 Case 4
Cloud
Unwanted
ordering
Cloud Voice Conversation
Home Home
Door Stealing car “…He was
attack driving a
Malicious voice Unintentional voice record
Lexus in a way
commands she said was
dangerous…“
User
Adversary IVA-enabled IVA-enabled
device device
Figure 2. Four cases when the IVA turns rogue
ROGUE IVAs application-to-cloud or network traffic between
Endpoint2-to-cloud. client’s companion
Wiretapping the
(Figure 2 - Case1) application and the cloud
Internet
may expose user’s
On the left side of Figure
An IVA’s ecosystem security and privacy data.
2 - Case1, the cloud
network communication This is because
services may use
is divided into two parts: identifying network
encrypted connections to
(1) IVA-enabled device- communications helps
protect customer’s
to-cloud or Endpoint1-to- attackers understand
personal data. In this
cloud; and (2) companion overall operations of an
environment, sniffing theIVA ecosystem. For unencrypted packets, a status, talking to the
example, in laboratory man-in-the-middle attack assistant, listening to
environment, we used could take place. Even if music, ordering products
HTTPS interception tools, the image is not altered, or services, and so on [9].
to analyze requests and obtaining the firmware
Compromised IVAs
responses, and then image is an important
understand which APIs security concern because There are well-known
are used for sending and it provides a chance to cases of compromised
receiving data to and understand the internal home-embedded devices
from the IVA running in operations of a IVA- that were connected to the
the cloud. enabled device. Internet. Recently, DDoS
Furthermore, a malicious attacks against Dyn LLC
On the right side of the
attacker may be able to exploited vulnerabilities
Figure 2 - Case1, we
distribute modified of 10s of millions of
illustrate IVA-enabled-
firmware images [7]. The home-embedded devices
device-to-cloud or
rest of the communication such as webcams and
Endpoint1-to-cloud
between the IVA-enabled DVRs, infecting them
communication. Our
device and the IVA with Mirai botnet, and
analysis reveals that,
running in the cloud is turning these devices into
although most network
encrypted using HTTPS. an army of bots used to
traffic is encrypted, not
So, what about an attack Dyn’s systems.
everything may be sent
encrypted packet Because gateway devices
over a secure protocol.
(HTTPS)? There are used for the IVA
There may be
various existing studies ecosystem are also
unencrypted connections,
on classifying network embedded systems, there
including but not limited
traffic through scientific are similar possibilities
to, checking the current
approaches including for them to be
network connectivity
machine-learning compromised if they
status, transmitting the
algorithms [8]. Even contain security
firmware image upgrades,
though the traffic is vulnerabilities [10].
etc.
encrypted, various Figure 2-Case2 illustrates
In the first case, it is patterns including the vulnerability scenario
possible to detect the payload sizes and data of a 24/7 voice recording.
presence of IVA devices rates could be utilized for In general, IVAs are not
inside of a home network. identifying user’s always recording, but
Also, if firmware data is behavior such as turning always hearing. If an
transferred over on the device, the idleIVA-enabled device IVAs may be controlled Unintentionally
hears the ‘wakeup word’, by people pretending to recorded voice
the user’s voice is be in the proximity of the
The last scenario deals
recorded and transmitted IVA-enabled device,
with data privacy. Voices
to the IVA in the cloud. If while, in fact, they are
can be recorded by
the IVA-enabled device accessing a speaker
accident and transmitted
(Endpoint1) is positioned in the
to a cloud (Figure 2-Case
compromised by a proximity, in the house.
4). Because speech
malicious attacker, it can
Malicious voice recognition is not a
play the role of a virtual
commands perfect science, it is
spy. ‘Always-on’ voice
possible to eavesdrop on
recording in a user’s The third threat includes
private conversations
private location can allow malicious voice
unintentionally. The
all sounds or voices to be commands as shown in
potential for accidental
recorded and sent to an Figure 2- Case3. In voice-
recording means that
attacker in real time. This activated services, users’
users do not necessarily
is a privacy concern. voices may lead to
have complete control
dangerous outcomes.
An IVA-enabled device is over what audio gets
Some IVAs provide a
a remotely-controlled transmitted to the IVA in
voice training feature, but
speaker, similar to a smart the cloud [12].
it is difficult to perfectly
baby monitor. There was
recognize user’s voice, As more ‘things’ become
a recent case where a
tones and accents. connected to the Internet,
family living in
Therefore, an IVA could there is a growing need
Washington, spoke out
process requests and for better understanding
about the horrors they
answer for someone else of security and privacy
experienced while using a
or for a malicious person. threats from IVAs. Our
baby monitor inside their
If a malicious person can goal here is to provide an
3-year-old son's bedroom.
come close enough to the overview of IVA
Parents discovered that a
targeted IVA-enabled ecosystems and their
stranger had hacked into
device, he or she may be potential threat vectors,
their baby monitor and
able to fool the system and to explain four
was able to spy on their
into thinking that the real different cases involving
toddler and sometimes
owner is the person IVAs that turned rogue.
speaking disturbing
speaking.
messages into the device DISCLAIMER
[11]. In a similar way,Certain commercial [4] Wikipedia, “Virtual [8] T. Nguyen and G.
entities, equipment, or assistant (artificial Armitage, “A survey of
materials identified in this intelligence)”, techniques for Internet
document were used only https://en.wikipedia.org/ traffic classification
to adequately describe an wiki/Virtual_assistant_(a using machine learning,”
experimental procedure rtificial_intelligence). IEEE Communications
or concept. Such Surveys and Tutorials,
[5] Business Insider,
identification is not 2007.
“Why Amazon's Echo is
intended to imply
totally dominating — and [9] C. Gu, S. Zhang and Y.
recommendation or
what Google, Microsoft, Sun, “Real-time
endorsement by NIST,
and Apple have to do to encrypted traffic
nor is it intended to imply
catch up”, identification using
that the entities, materials,
http://www.businessinsid machine learning”,
or equipment are
er.com/amazon-echo- Journal of software, 2011.
necessarily the best
google-home-microsoft-
available for the purpose. [10] ORACLE+Dyn, Dyn
cortana-apple-siri-2017-1
Statement on 10/21/2016
REFERENCES
[6] E. McCallister and T. DDoS Attack”,
[1] Gartner Newsroom, Grance and K. Scarfone, http://dyn.com/blog/dyn-
“Gartner Says Worldwide “Guide to protecting the statement-on-10212016-
Spending on VPA- confidentiality of ddos-attack/.
Enabled Wireless Personally Identifiable
[11] The San Francisco
Speakers Will Top $2 Information (PII),” NIST
Globe, “Stranger hacks
Billion by 2020”, Special Publication 800-
family's baby monitor and
https://www.gartner.com/ 122, 2010.
talks to child at night”,
newsroom/id/3464317.
[7] “Exploring the http://sfglobe.com/2016/
[2] AP The Big Story: Amazon Echo Dot, Part 1: 01/06/stranger-hacks-
“How Burger King Intercepting firmware familys-baby-monitor-
revealed the hackability updates”, and-talks-to-child-at-
of voice assistants”, https://medium.com/@mi night/.
http://bigstory.ap.org/2d8 caksica/exploring-the-
[12] The Christian
036d742504890b2f9edc3 amazon-echo-dot-part-1-
Science Monitor, “What
f98c77ef intercepting-firmware-
do Alexa and Siri mean
updates-c7e0f9408b59
[3] Wikipedia, “Chatbot”, for privacy?”,
- .dyktzwphz
https://en.wikipedia.org/ http://www.csmonitor.co
wiki/Chatbot m/Technology/2017/0114/Devices-sprout-ears- What-do-Alexa-and-Siri- mean-for-privacy
You can also read
