The AI Manifesto The greatest challenges of AI application, and how we can conquer them - BlackBerry
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
ARTIFICIAL INTELLIGENCE IS THE FUTURE
The AI
Manifesto
The greatest
challenges of
AI application,
and how we can
conquer them.
IS SUE .0 1 SP RING 2 0 19
A lifetime to
build your career.
Five seconds
to lose it.
In cybersecurity time is precious.
Let our predictive AI prevent cyber attacks
and return you to a state of ZERO anxiety.
Turn five seconds into never.
Cylance.com/zero
Letter from
1.61803398875
the Editor
The universe is an amazing and sometimes incalculable phenomenon that few
of us can adequately appreciate, much less absorb. So, when we come across a
pattern, a universal, or a repeatable and replicable calculatable measure of it, we
are in awe…and many of us enthusiastically celebrate it!
1.61803398875
Sometimes known as the Golden Mean or the Golden Ratio, Phi is one such unique
phenomenon in the universe. Documented some 2,400 years ago by Euclid, Phi is
categorized as an irrational number similar to Pi and holds a secret for predicting
patterns in the universe. Observed in the chambered nautilus, falcon gyres,
rose petals, pineapple skin, sunflower centers, the Milky Way galaxy, and even
romanesco, the presence of Phi in the observable universe is undeniable.
In 2010, researchers even found the Golden Ratio in solid-state atomic particles
by applying a magnetic field at right angles to particles of cobalt niobate, which
yielded a magnetic resonance that showed a perfect ratio of 1.618.
Predicting the Future
The predictability of Phi is why we are here. We named this publication in its honor
because that is what artificial intelligence (AI) delivers: a universal pattern in the
observable universe that creates an algorithmic representation of that pattern to
allow for replication and, ultimately, predictability.
We hope you will allow us to take you on this journey of pattern and algorithmic
discovery to better the planet and those of us who so precariously dwell upon it.
Thank you for joining us!
Stuart McClure
Editor-in-Chief, Phi Quarterly
P H I • A R T I F I C I A L I N T E L L I G E N C E I S T H E F U T U R E • I S S U E I 1
Featured
ISSUE.01 SPRING 2019
EDITOR-IN-CHIEF
Contributors
Stuart McClure
Malcolm Harkins is the author of
EXECUTIVE EDITOR Managing Risk and Information
KC Higgins Security: Protect to Enable and a
trusted leader in the security
DEPUTY EDITOR
space. He has spent his career
Anthony Freed
helping CISOs and other execu-
MANAGING EDITOR tives understand information risk,
Natasha Rhodes security, and privacy issues and has served as an
instructor or board member at universities that include
RESEARCH EDITOR UC Berkley, UCLA, Carnegie Mellon, Arizona State, and
Kevin Livelli Susquehanna University. Malcolm lives in northern
California, works out compulsively before dawn, and
CREATIVE DIRECTOR
enjoys boating, cooking, and spending time with family
Drew Hoffman
and friends.
ART DIRECTOR
Aaron Zide John McClurg is a longtime secu-
rity executive and a global expert
PRODUCTION DIRECTOR in cyber counterintelligence. In
Patrick Huskey addition to holding senior execu-
tive roles within Dell, Honeywell,
PRODUCTION DESIGNER
Lucent, and the FBI, John also
Douglas Kraus
served as the co-chair of the U.S.
COPY EDITOR State Department’s Overseas Security Advisory Council.
William Savastano He has a degree in law and has completed doctoral
coursework in philosophical hermeneutics. John lives in
DIGITAL MANAGING EDITOR the Rocky Mountains and holds what are believed to be
Saren Sakurai global speed-reading titles.
EDITORIAL STAFF
Scott Scheferman’s thought
Kevin Clinton
leadership on AI and cybersecurity
Frankie Berry are highly sought after by execu-
RESEARCH STAFF tives seeking to address the
Jon Gross modern threat landscape, particu-
larly the velocity and automation
PROJECT MANAGER associated with complex attack
Donna Crawford campaigns. In his role as the senior director of worldwide
services at Cylance, he supports more than 100 consul-
SOCIAL MEDIA MANAGER
tants and managers across all industry practices. Scott
Joann Doan resides in Texas, enjoys fast Italian cars, produces live
hardware techno tracks, and won Kingpin’s first ever
Φ Phi Quarterly
DefCon badge-hacking contest…although he was
400 Spectrum Center Drive,
unaware there was even a contest underway.
Suite 900,
Irvine, California 92618 Sara Lofgren has been working in
+1-888-930-3858 computer security for over a
decade, with a focus on solving
enterprise security problems
through the union of technology,
For information regarding submissions,
people, and processes. Besides
subscriptions, advertising, or syndication,
malware, her other main areas of
please contact phiquarterly@cylance.com
interest include privacy, cryptography, and technology
regulations. Sara lives in Minnesota with four kids, two
2019 Cylance Inc. Cylance® and all associated logos and designs
©
are trademarks or registered trademarks of Cylance Inc. All dogs, a cat, and many rescue horses.
other registered trademarks or trademarks are property of
their respective owners. The opinions expressed in Phi are the
contributors’ own and do not reflect the views of Cylance.
2 P H I • A R T I F I C I A L I N T E L L I G E N C E I S T H E F U T U R E • I S S U E I
Contents IS SUE .0 1 SP RING 2 0 19
THE RACE IS ON 04
Artificial Intelligence in the Enterprise
TO CATCH A SPY 14
The Emergence of Artificial Intelligence
THE AI MANIFESTO 16
Understanding the Risks and Ethical
Implications of AI-Based Security
THREAT RESEARCH 26
Cat Versus Mouse: The Perennial Effort
To Catch Commercial Spyware
MALWARE SPOTLIGHT 32
How To Avoid a SamSam Ransomware Attack
CASE STUDY 40
Sydney Opera House and VMtech
Take on Cybersecurity
LIVING OFF THE LAND 42
Public Hacking Tools Get Their Day in the Sun
PUTTING THE “S” IN IOT 48
Prepare Today for the Security
Implications of a Connected World
OFF THE SHELF 52
CURB YOUR CURVES 56
P H I • A R T I F I C I A L I N T E L L I G E N C E I S T H E F U T U R E • I S S U E I 3
Artificial
Intelligence
in the Enterprise
RACE
4
IS ON P H I • A R T I F I C I A L I N T E L L I G E N C E I S T H E F U T U R E • I S S U E I
B Y " P H I
OVERVIEW
Artificial intelligence (AI) is one of the hottest topics
in today’s headlines. It powers natural language
E D I T O R I A L
recognition for voice-powered assistants like Siri
and Alexa, beats world-class Google Go players, and
enables hyper-targeted e-commerce and content
recommendations across the web on high-traffic
websites that include Target and Netflix.
But recently, leaders at organizations large and
small have been actively expanding the AI footprint
in their enterprises. Executives are trying to more
" S T A F F F"
fully comprehend what AI is and how they can use
it to capitalize on business opportunities by gaining
insight into the data they collect and engaging with
customers more productively to hone their compet-
itive edge. AI is the frontier of enterprise technology,
but there remain many misconceptions about what
it is and how it works. > > > >
P H I • A R T I F I C I A L I N T E L L I G E N C E I S T H E F U T U R E • I S S U E I 5
3
! 8%
say they will spend a quarter
to half of their IT budget on
AI over the next 12 months.
Part of the confusion stems from the fact gauge their understanding of and investment
that AI is an umbrella term that covers a range in AI. We asked a host of questions to find out
of technologies$—$including machine learning, where and how enterprises are using AI, what
computer vision, natural language processing, their future plans are, and what they think the
deep learning, and others$—$that are in various impact of AI will be on their organization.
stages of development and deployment. The
use of AI for dynamic market-based pricing Here are five key findings:
and targeted marketing has been spreading
through corporations for a while, but actual AI
computing where machines think like humans
1 AI moves the needle on security: The
survey found that 77% say they have
prevented more breaches following their
is still years in the future. The various possibil- use of AI-powered tools, and 81% say AI was
ities prompt a range of reactions from people detecting threats before their human security
who understand AI’s disruptive potential. teams could.
The research covered in this report focused
on artificial narrow intelligence (referred to
herein simply as AI$—$see The Three Practice
Areas on page 7) that is being targeted for
2 Organizations plan to increase AI
spend: Nearly all of the IT decision
makers surveyed said they are either currently
specific business cases in the enterprise, like spending on AI-powered solutions or planning
blocking malware and responding to intrusion to invest in them in the next two years. 60%
attempts by bad actors. already have AI in place.
Is enterprise AI just the next leader in the
series of recent new technologies all touted
as the holy grail of business innovation that
will leave companies without them in the
3 AI provides a competitive advantage:
87% of IT decision makers see AI-powered
technology as a competitive advantage for
dust of digital transformation? To answer this their IT departments, and 83% are investing
question, we partnered with Market Cube specifically in AI to beat competitors.
to commission a survey of more than 650
decision makers at large enterprises working
across major industries in the U.S. and Europe
and cross-functionally in the organization, from
4 AI lives up to its promise: Despite the fact
that 76% of respondents are concerned
that marketing hype will make it difficult to
middle management to the corner office, to evaluate AI-powered technologies, 86% say
6 P H I • A R T I F I C I A L I N T E L L I G E N C E I S T H E F U T U R E • I S S U E I
the AI they’ve used has lived up to its promises. have AI solutions already in production. This
Furthermore, 64% of IT decision makers expect percentage might seem high, but not if we
to see ROI from their investments in AI in fewer consider that data-driven IT departments
than two years. are often early adopters of new technologies
and are always looking for ways to optimize
5 Concerns for job retention don’t outweigh
opportunities: 68% of IT decision makers
say AI will make certain jobs obsolete, and 74%
processes and reduce costs.
Specifically, the survey reveals:
are concerned AI technology will replace jobs. • 60% already have AI in place
But, 93% say it will create new job opportunities, • 39% will spend 11% – 24% of their
and 80% believe AI will lead them to hire new IT budget on AI over the next 12 months
workers and retrain existing employees. • 38% will spend a quarter to half of their
IT budget on AI over the next 12 months
AI in the Enterprise
It appears we’ve finally reached a point where The survey shows that IT decision makers see
the use of AI is shifting from talk to action, AI as a way to stay competitive and feel they will
as companies have begun investing in AI in lose out if they don’t adopt it, particularly for
order to make better use of the data they IT and security departments. In addition, the
gather and the increased computing power competitive benefits AI provides can be seen
to which they have access. According to a across the organization:
recent McKinsey Global Institute Report, AI • 83% are investing specifically in AI to beat
entrepreneurial investments were between $26 competitors
billion and $39 billion a couple of years ago, a • 62% fear their competitors’ investments
three-fold increase over the previous three in the technology may pose a threat to
years. Research firm IDC predicts enterprise their business
spending on AI and cognitive computing will
grow to $46 billion by 2020.
Granted, most investment in AI comes from
big players like Google, Amazon, and other The Three Practice Areas
big tech firms, but the AI spending fever is
spreading. AI is used to forecast electricity As a field, artificial intelligence encompasses
demand at utilities, to train vehicles to become three distinct areas of research and practice:
chauffeurs and truck drivers, and to power
robots that pack and ship Amazon orders.
Netflix, for example, says the AI algorithm
behind its search-and-recommendations
1 Artificial superintelligence is the type popularized
in speculative fiction and in movies such as The
Matrix. The goal of this type of research is to produce
engine has saved it $1 billion in potential annual computers that are superior to humans in virtually
losses from canceled subscriptions. Early every way, possessing what author and analyst William
adopters tend to be technology, telecommuni- Bryk referred to as “perfect memory and unlimited
cations, and financial services firms that deploy analytical power.”
AI across technology groups and as a core part
of their business. One thing they all have in
common? All successful deployments have the
full support of executive leadership.
2 Artificial general intelligence refers to a
machine that is as intelligent as a human and
equally capable of solving the broad range of problems
that require learning and reasoning.
Investment in AI
The large enterprises that took part in our
survey are bullish on AI. Nearly all say they
are either currently spending on AI-powered
3 Artificial narrow intelligence exploits a
computer’s superior ability to process vast
quantities of data and detect patterns and relationships
solutions or planning to invest in them in that would otherwise be difficult or impossible for a
the next few years. A majority also say they human to detect, such as in the field of cybersecurity.
P H I • A R T I F I C I A L I N T E L L I G E N C E I S T H E F U T U R E • I S S U E I 7
6
! 0%
of IT decision makers
surveyed say they already have
AI-powered solutions in place.
• 87% see AI as a competitive advantage for place feels that the deployment has lived up to
their departments its promises. More than half expect to see ROI
• 79% believe AI will also benefit their from their investments in AI within 24 months,
security teams particularly in the areas of improved operational
• 75% think AI will benefit manufacturing efficiency, better business performance, and
and logistics automation of repetitive tasks.
• 74% believe AI will benefit their customer
service departments Perception of AI in the Enterprise
No study on AI would be complete without
So, which industries and departments are taking a look at how people think the technology
investing in AI? According to the survey, the might affect their jobs or their workforce. One
technology is primarily in use in the IT, security, of the biggest challenges to widespread
operations, and customer service areas, while adoption of AI is the perception that workers
manufacturing and logistics are fast becoming will be displaced. AI might require retraining
the top departments asking for it. As far as staff for a number of jobs, but it will result in
units within an organization, respondents say IT greater productivity and efficiency gains, and
departments lead adoption at 75%, followed by the potential for increased job satisfaction as
security teams at 48%, and operations at 39%. it will create vast new opportunities that will
As far as where respondents are feeling allow staff to use their brains for more critical
the most impact, IT, security, manufacturing, thinking and less monotonous, mundane, repet-
and logistics are the departments where AI itive tasks.
has changed the way they work the most. In In other words, the use of AI will change the
general, departments that traditionally deal nature of the work people do, moving it away
with data and analytics are best positioned to from menial tasks to more strategic functions.
take advantage of AI. Most survey respondents It will be used to parse through data about
say they are pleased with the results they’ve customers, operations, business activities, and
seen from their use of AI technologies. other processes that staff cannot compute or
While two-thirds of respondents say they manage manually. But, AI can’t operate on its
are concerned that marketing hype will make own or in a vacuum; it needs humans to create
it difficult to evaluate AI-powered technologies, the knowledge trees upon which it learns, and
nearly every respondent with an AI solution in to train and maintain it.
8 P H I • A R T I F I C I A L I N T E L L I G E N C E I S T H E F U T U R E • I S S U E IIn the next 12 months, what percentage of AI-powered
AI-poweredtechnology
technologyhas
haschanged
changedthe way
the way
these
thesedepartments
departmentsoperate.
operate.
your IT budget is your organization planning
to spend on AI-powered technology?
Somewhat Agree/ Neutral/ Somewhat Disagree/
Strongly Agree Not Sure Strongly Disagree
Spending 1–10%
HR
Spending 11–24% Finance
Sales
Spending 25–49% Marketing
Customer Service
Spending 50% or More
Operations
Manufacturing/Logistics
0 20 40 60 80 100%
Security
IT
Which departments are currently using
0 20 40 60 80 100%
AI-powered technology?
HR
Which departments are demanding more
Sales
AI-powered technology?
Finance
HR
Marketing
Finance
Manufacturing/Logistics
Sales
Customer Service
Customer Service
Operations
Marketing
Security
Operations
IT
Manufacturing/Logistics
0 20 40 60 80 100% Security
IT
0 20 40 60 80 100%
Job Creation
In the survey, concerns about job loss were • 80% say AI will lead them to hire new
heavily counterbalanced by expectations that workers and retrain existing employees
the technology will result in new opportunities, • 81% say AI will be a leading driver in
including more meaningful work for employees allowing technical employees to do more
and additional benefits throughout the organi- meaningful work
zation. Clearly the nature of some jobs within • 74% say AI will enable less technical staff
the enterprise will shift as a result of AI technol- to use technology more effectively
ogies, but most respondents predict new job
creation as a result too. Respondents with AI already in place report
numerous benefits from their use of it. 84% of
Specifically, the survey reveals: respondents say AI improved the overall quality
• 93% of respondents say AI will create new of employees’ work, and 80% believe that
types of jobs teams using AI have become more productive.
Meanwhile, 96% of respondents say they are
P H I • A R T I F I C I A L I N T E L L I G E N C E I S T H E F U T U R E • I S S U E I 9confident that AI-driven technologies will analytics experts who can help organizations
improve organizational efficiency, and 94% are make the most of AI technology. Our survey
confident AI will produce a quantifiable return results show that IT leaders are willing to
on investment. embrace the evolution of the workforce to more
Hiring rates are often an early indicator of strategic and analytical functions.
the health of the job landscape for emerging Enterprises are actively seeking employees
technologies, and we’re already seeing who have familiarity with AI to help build out
increased demand for data scientists and their capabilities$—$and job seekers are antic-
ipating that need. 64% of respondents say
that more candidates at every level are using
When do you anticipate seeing ROI from the
AI as a differentiator on their resumes and in
use of AI-powered technologies?
interviews. That is smart because 62% also
Already seeing ROI reported that these skills are a deciding factor
in the hiring process, and 61% say it is a critical
hiring factor for security teams. 62% are even
Less than 6 months
going so far as to ask candidates directly about
AI during the interview process.
6 months to 1 year
Security, Risk, and AI
Security is a strong application area where
1 to 2 years
AI can be used to help teams make quick
decisions and act on them. AI helps teams
3 to 5 years identify threats across an expanding attack
surface (including mobile, cloud services,
100% More than 5 years and the Internet of things) by automating
data aggregation across different file types,
0 20 40 60 80 100% mapping it back to compliance requirements,
and ruling out false positives.
Impact of AI-powered technology on your The technology is also being used to help
company’s hiring practices: companies assess risk and potential harm
to the business from specific threats using
internal security data and external data on
We have hired more employees
exploits, malware, and threat actors. In addition,
AI can automate remediation processes that
are used for incident reporting that can be
augmented by staff analysis to boost effec-
We have immediate needs to hire employees tiveness and reliability. AI is not just detecting
threats; it also stops attacks from executing
in the first place, entirely preventing future
incidents.
We have new hiring needs
Survey respondents reported that AI is
having a big impact on their security efforts.
70% say their security team is using AI in their
We are able to use our most technical threat-prevention strategies, and 77% say
workers more effectively
they have been able to prevent more breaches
since they began using AI-powered tools. 81%
of respondents say AI was detecting threats
Hiring requirements for line of business workers
now include technical literacy before their security teams could, 78% say
the technology has found threats humans
0 20 40 60 80 100% couldn’t see, and 77% believe it is impossible
for human-only cybersecurity teams to keep up
Yes Not sure No
10 P H I • A R T I F I C I A L I N T E L L I G E N C E I S T H E F U T U R E • I S S U E IARTIFICIAL INTELLIGENCE:
A POSITIVE FORCE IN THE ENTERPRISE
For security teams, AI is moving the needle:
70% say their security 77% say they have 81% say AI was 78% say the technology
team is using AI in their prevented more breaches detecting threats before has found threats humans
threat prevention strategies. following their use of their security teams could. couldn’t see.
AI-powered tools.
Organizations
60% say they already 40% said they are planning
are already have AI-powered to invest in them in the next
investing in AI, solutions in place. two years.
and this will
only increase:
AI is seen as 87% see AI-powered 83% are
technology as a competitive investing in
a competitive advantage for their IT AI to beat
advantage: departments. competitors.
AI brings productivity, meaningful work for employees:
80% believe that teams 81% say AI is critical 81% say AI will lead
using AI have become to the company's to more meaningful work
more productive. digital transformation. for employees.
Artificial intelligence is making inroads in enterprises as IT decision makers and other corporate leaders realize the
benefits it brings to productivity, digital transformation, employee work satisfaction, and for security in particular,
detecting and stopping threats. Companies that wait too long to adopt AI, or at least explore the possibilities with AI,
run the risk of losing to faster-moving competitors. With innovation, time is of the essence, and AI is happening now.
Survey conducted by Market Cube on behalf of Cylance.
P H I • A R T I F I C I A L I N T E L L I G E N C E I S T H E F U T U R E • I S S U E I 11How confident
How confidentare areyou
youabout
abouteach
eachofofthe
the with the threats. In other words, AI tools$—$and
following as it relates to AI technology?
following as it relates to AI technology? we believe native AI technologies have the
advantage here$—$are one of the most valuable
weapons in the threat-prevention arsenal.
AI will provide new job opportunities Importantly, AI doesn’t just make systems
in addition to displacing existing jobs smarter, it makes employees smarter too, by
enabling security and other workers to increase
skill levels. There are chatbot applications
My company is implementing AI correctly designed to help mentor junior security team
members to use specific technologies and
AI that adjusts the information it presents
AI will be a leading driver of our organization
based on user skill level and knowledge. As IT
hiring more highly skilled workers departments try to attract employees across a
broader range of skills, AI security products will
evolve to become more flexible in terms of the
AI will produce a quantifiable return on investment assumptions about the user’s background and
be more proactive about helping them learn.
Augmenting talent with robust AI solutions
can help close the technology skills gap. This
The broad use of AI-driven technologies will
improve our organizational efficiency talent shortfall, especially in cybersecurity, is
well documented and often remarked upon;
some analysts predict that by 2022, the
0 20 40 60 80 100%
global shortage of cybersecurity professionals
Very confident Not very confident is expected to reach 1.8 million. Our survey
Confident Not confident at all
respondents were optimistic that AI will help
Somewhat confident
solve that problem.
Specifically, the survey shows that 81% of
respondents believe that AI will help bridge the
Describe what you’re seeing in the
skills gap, and many have already seen their
Describe
hiring what you’re seeing in the hiring process:
process:
security teams do more analytical, contextual,
and highly skilled work as a result of their
investments in AI.
More candidates coming in qualified with
AI-specific credentials
The Future of AI in the Enterprise
Unlike other areas of IT spending, the AI
Experience and/or familiarity with AI
discussion is akin to cloud adoption because
is a critical hiring factor for security teams it involves executives at the highest levels of
the organization, including teams that lead the
strategy and transformation efforts organiza-
Specific questions about AI during new hire interviews tions require to gain competitive advantage.
Boards and C-suite executives are key stake-
holders in these conversations; their support is
required for AI initiatives to succeed.
Experience and/or familiarity with AI is
a deciding factor in the hiring process In addition, there seems to be no question
that AI is the next wave of digital transfor-
mation for most IT decision makers. 84%
More candidates (any level) using AI as a say AI-powered technology was part of their
differentiator in their resumes/interviews
digital transformation strategy, and 81% say
it’s critical for the success of those initiatives.
0 20 40 60 80 100% While companies may feel pressure to adopt
Yes Not sure No AI, they should realize that without a strong
12 P H I • A R T I F I C I A L I N T E L L I G E N C E I S T H E F U T U R E • I S S U E I81%
#
of respondents say AI was detecting
threats before their security teams
could, while 78% say the technology
has found threats humans couldn’t see.
digital foundation in place, the AI may be limited. you can observe how the software itself
As a result, the technology can serve as a processes data without human assistance.
forcing function. • Inquire about vendor data sources, the
While the value of AI is apparent to IT leaders, size of the data sets, data parameters, and
it’s not always easy to figure out which vendors system capacity.
to choose. 65% of respondents say that market • Ask about the algorithm being used,
noise around AI makes it difficult to under- including what data is encoded and
stand the difference between all the different decoded, how the neural network is
solutions when much of their marketing implemented, and other technical aspects
materials look and sound the same. Clearly, IT of the approach.
decision makers know AI will be important, and • Compile requirements for compatibility,
they know it can provide a strategic advantage, functionality, user experience, and price
but they don’t really know how or where to start. ahead of time.
What’s more, there are network effects with
AI, so scaling is exponential. In other words, Based on our survey responses, it’s clear that
the leaders of the pack, the first adopters, are enterprises are using AI to varying degrees
making sizable headway and their advantage is and that executives understand the benefits
immediately and increasingly defensible. it can provide for near-term and long-term
operational and market advantage. Enterprises
Evaluating AI Solutions would be wise to ramp their efforts to evaluate
Unfortunately, there is no standard how-to AI solutions now. Just as companies that
guide for choosing the best AI solution; technol- embraced early Internet and cloud opportu-
ogies vary substantially by application and nities saw positive impact to their business
industry. As with other technology investments, results, operational effectiveness, and market
there are a few simple rules of thumb that position, organizations that see AI as a
executives can use: strategic differentiator and support AI adoption
• Request customer references to find out will find themselves ahead of the curve instead
how their adoption is going and what the of behind it. Φ
pain points and challenges are, if any.
• Ask for a product demonstration and use
in-house data — ideally, choose a demo
that stands alone and not in the cloud, so
P H I • A R T I F I C I A L I N T E L L I G E N C E I S T H E F U T U R E • I S S U E I 13To Catch a
Spy: The Emergence of
Artificial Intelligence
F
olklore has it that during the agencies to build and maintain a viable insider
American Revolution, George threat program. No one seemed quite sure what
Washington was approached by the feds meant by “viable,” but I assumed, at
an enquiring member of the press a minimum, that a successful solution had to
who asked, “George! George! What involve the demonstrated use of analytical tools.
keeps you up at night?” It wasn’t the Conti- At the time, I was serving as the chief security
nental Congress, which even then seemed officer (CSO) at Dell. We leveraged the strength
challenged when it came to accomplishing of some big data analytics that allowed us to
anything. It wasn’t his troops either, although examine all forms of data, both structured
they were starving and freezing at Valley (Excel files) and unstructured (Internet traffic).
Forge. His reply? “Their spies…” Since that Within 12 months, we had tested and imple-
time$—$more than 240 years$—$we’ve gained mented our insider program. With that success
some useful tools that enable us to detect came my first glimpse of what the future might
early indicators that a trusted insider is at risk hold, my first inkling that, as stymied as our
of drifting over to the other side. But, despite profession had been in the world of reactive
these advances, the best that we seem to detection, proactive prevention rooted in
B Y
be able to do is catch the spies after they’ve artificial intelligence (AI) might just be possible.
already hurt us. Thomas Kuhn in his book, The Structure of
J O H N
In fact, it was while the U.S. was chasing Scientific Revolutions, describes the need for a
one such spy, Harold “Jim” Nicholson, that periodic refresh of society. He posits that over
an answer came to me: What we really have time, we need a profound change in our way of
is a big data problem. Previously, the early thinking. Kuhn challenges us to consider new
M C C L U R G"
indicators were distributed across too many paradigms and to change the rules of the game,
disparate silos for us to wrap our cognitively including letting go of accepted standards and
limited minds around. That fact didn’t stop the best practices.
U.S. government, in the wake of the Edward As I look at the paradigm shift that’s now
Snowden leak, from requiring all corporations available in the form of transformative technol-
with plans to continue to work with federal ogies, it occurs to me that what we’re up against
14 P H I • A R T I F I C I A L I N T E L L I G E N C E I S T H E F U T U R E • I S S U E Iin effecting this transition is a formidable and entrenched way of thinking. It’s comparable to what Copernicus himself faced almost six centuries ago, as he battled his Ptolemaic predecessors, disproving their belief that the earth was the center of the universe. The use and availability of AI has brought with it the dawning of a new era. We are witnessing a scientific revolution, the excitement of which hasn’t been felt in many years. I don’t think it’s an overstatement to say that AI delivers a new paradigm by putting the science back into security. AI focuses on prediction based on properties learned from earlier data; similarly, at the core of native AI security methodologies is a massively scalable data-processing brain capable of applying highly-tuned algorithmic models to enormous amounts of data in near real-time. A native AI approach to security fundamen- tally changes the way we understand and control cyber-based risks. Much like Kuhn’s model predicted, the security paradigm is shifting from that of “regular, outmoded strategies” to one of “security as a science,” and these cutting-edge technologies are the primary agents for that revolutionary change. Φ P H I • A R T I F I C I A L I N T E L L I G E N C E I S T H E F U T U R E • I S S U E I 15
AI
Manifes
THE
16 P H I • A R T I F I C I A L I N T E L L I G E N C E I S T H E F U T U R E • I S S U E IB Y
sto
M A L C O L M
H A R K I N S
Understanding the Risks and
PART 01
Ethical Implications of AI-Based Security
We live in a time of rapid technological change, where
nearly every aspect of our lives now relies on devices
that compute and connect. The resulting exponential
increase in the use of cyber-physical systems has
transformed industry, government, and commerce;
what’s more, the speed of innovation shows no
signs of slowing down, particularly as the revolution
in artificial intelligence (AI) stands to transform
daily life even further through increasingly powerful
tools for data analysis, prediction, security, and
automation.1
Like past waves of extreme innovation, as this
one crests, debate over ethical usage and privacy
controls are likely to proliferate. So far, the inter-
section of AI and society has brought its own
unique set of ethical challenges, some of which
have been anticipated and discussed for many
years, while others are just beginning to come to
light. For example, academics and science fiction
authors alike have long pondered the ethical impli-
cations of hyper-intelligent machines, but it’s only
recently that we’ve seen real-world problems start to
P H I • A R T I F I C I A L I N T E L L I G E N C E I S T H E F U T U R E • I S S U E I 17The Ethics of Computer-Based
Cybersecurity’s role Decisions
The largest sources of concern over the
in mitigating the practical use of AI are typically about the possi-
bility of machines failing at the tasks they are
ethical risks of AI use: given. The consequences for failure are trivial
when that task is playing chess, but the stakes
1 Prevent and mitigate harm to
mount when AI is tasked with, say, driving a car
or flying a jumbo jet carrying 500 passengers.
systems and services In some ways, these risks of failure are no
different than those in established technologies
2 Protect privacy by protecting data that rely on human decision-making to operate.
However, the complexity and the perceived
3 Enable AI-driven systems to be lack of transparency that underlie the ways AI
makes its decisions heighten concerns over
more accessible and transparent
AI-run systems, because they appear harder
4 Keep malicious AI in check
to predict and understand. Additionally, the
relatively short time that this technology has
been used more widely, coupled with a lack
of public understanding about how, exactly,
these AI-powered systems operate, add to the
fear factor.
surface, like social bias in automated decision- Consider a real-world example: Society has
making tools, or the ethical choices made by become accustomed to car accidents resulting
self-driving cars.2,$5 from human error or mechanical failure and, in
During the past two decades, the security spite of regulatory and technical improvements
community has increasingly turned to AI to reduce the danger inherent in car accidents,
and the power of machine learning (ML) to we now accept them without question as part
reap many technological benefits, but those of the overall risk of driving. Accidents caused
advances have forced security practitioners by AI failures, on the other hand, raise consid-
to navigate a proportional number of risks erably more public alarm than those caused by
and ethical dilemmas along the way. As the more traditional types of human or machine-
leader in the development of AI and ML for based failure.
cybersecurity, Cylance is at the heart of the The novelty of a computer making decisions
debate and is passionate about advancing that could have fatal consequences scares
the use of AI for good. From this vantage point, people, and a large part of that fear revolves
we’ve been able to keep a close watch on AI’s around how those systems balance ethical
technical progression while simultaneously concerns. Take, for instance, the furor over
observing the broader social impact of AI from the first known case of a driverless car killing
a risk professional’s perspective. a pedestrian.4,$8 The computer appears to have
We believe that the cyber-risk community determined too late that the car was about
and AI practitioners bear the responsibility to to hit a pedestrian, but could it have driven
continually assess the human implications the car off the road to avoid the collision? Did
of AI use, both at large and within security the computer favor its passenger’s safety
protocols, and that together, we must find ways over the pedestrian’s? What if it had been two
to build ethical considerations into all AI-based pedestrians? What if they were children? What
products and systems. This article outlines if the computer was faced with the choice of
some of these early ethical dimensions of AI colliding with one of two different pedestrians?
and offers guidance for our own work and that What would a human driver do differently from
of other AI practitioners.
18 P H I • A R T I F I C I A L I N T E L L I G E N C E I S T H E F U T U R E • I S S U E IP H I • A R T I F I C I A L I N T E L L I G E N C E I S T H E F U T U R E • I S S U E I 19
say, engine failure, but it would raise different
Ethical protections ethical considerations in terms of agency and
fault. Moreover, we would presumably be better
that must be built into able to quantify the risk of the accident being
repeated in a mechanical failure than in the
AI-driven security: case of a complex AI system.
Examples like these highlight the impor-
1
tance of ensuring that AI-dependent systems
Ensure effectiveness and provide are well-tested and built in ways that are
enough information to assess risk transparent enough to enable an adequate
assessment of risk by the end-users of those
2 Collect and use data based on systems.10 What that means in practice
informed consent depends to a large extent on the purpose for
which AI is being employed.
3 Avoid discriminatory or arbitrary
Careful attention needs to be given to the
potential harm that may result from failure at
restrictions a given task as well as to the complexity of the
4
system and the extent to which that complexity
Make logic transparent adds to uncertainty in estimates of the proba-
bility of failure. Risk professionals will need
to consider tradeoffs between transparency
and effectiveness, between transparency
and privacy, and between the possibility of
AI-based software when faced with that split- human override and overall effectiveness
second decision? of AI decisioning, all of which depend on the
Part of the alarm over this accident also contextual use of AI in any given setting.
results from fears that its cause affects
other autonomous vehicles and a wider array Privacy and Consent
of activities linked to AI. For example, did the AI’s rapid adoption and widespread use in
road conditions make this accident one that recent years also raises considerable privacy
no human or computer system could have concerns. AI systems increasingly depend on
avoided? Was it a flaw in the AI of this particular ingesting massive amounts of data for training
navigation system or in all AI-based navigation and testing purposes, which creates incen-
systems? The AI technology involved in a tives for companies not only to maintain large
driverless car is highly complex, making it more databases that may be exposed to theft, but
difficult to test than the car’s mechanical parts. also to actively collect excessive personal infor-
Do we know enough to adequately quantify the mation to build the value of those databases.5,%10
risks before this technology is rolled out on a It also creates incentives to use such data
global scale? in ways that go beyond that which the data’s
The fatal crash of Lion Air Flight 610 offers owner initially consented. Indeed, in complex
another instructive example. The crash appears AI systems, it may be hard to know in advance
to have been caused by a mechanical sensor exactly how any given piece of data will be used
error leading to the airplane’s computer in future.5
system forcing its nose down. The human These concerns are linked to the overall
pilots appear to have pulled the nose back proliferation and indefinite storage of captured
up repeatedly before losing control.9 The fact data, with an increasing percentage of this
that this incident involved a computer making data emitted like exhaust from cyber-physical
a flawed decision and removing control from systems such as the Internet of things (IoT).11,%12
the pilots raises concerns beyond those raised These fears are heightened exponentially by
by a purely mechanical failure. The tragedy the fact that AI derives the best value from
would be the same had it been the result of, large data sets, and is increasingly able to
20 P H I • A R T I F I C I A L I N T E L L I G E N C E I S T H E F U T U R E • I S S U E Idetect unique patterns that can re-identify in Broward County, Florida10,$15,$16 illustrates the data thought to be anonymized. Concerns are point. By comparing risk scores to defen- further ratcheted up by the increasing ability of dants’ subsequent conduct, Pro Publica cyber attackers to expose these large data sets showed not only how unreliable the scores that were supposed to be protected$—$a trend were, but also how biased they were against that goes hand-in-hand with the decreasing African Americans. The scores erroneously efficacy of traditional, signature-based security flagged African American defendants as solutions. future criminals at nearly twice the rate as it Such concerns add new dimensions to data falsely flagged European Americans defen- privacy laws that cybersecurity and risk leaders dants as such. Importantly, the flags occurred must consider as they help organizations even though the system did not explicitly ask navigate the onboarding of AI. The good news about race.16 in this case is that AI-powered technology can, In 2013, U.S. Immigration and Customs in fact, be used to enhance privacy, if installed Enforcement (ICE) began the nationwide use and correctly configured as part of a company’s of an automated risk assessment tool to help overall layered defense strategy. determine whether to detain or release non-cit- In contrast to other analysis tools, AI is often izens during deportation proceedings. It initially better suited to use and learn from properly recommended release in only about 0.6% of anonymized data. Feature hashing, when the cases.17 In 2017, ICE quietly modified the tool to data used to train a machine learning system make it recommend detention in all cases. This is first altered through a hashing algorithm,13,$14 came to light only through a Reuters investi- is an irreversible transformation that makes the gation of detention decisions in 2018. 4,$18 data worthless for analysis by humans but still The danger of these types of discriminatory readable by AI systems for pattern detection. and arbitrary AI usage is only heightened Feature hashing can make AI-based analysis with the spread of AI-based facial recognition more efficient by reducing the dimensionality tools in law enforcement and other settings, of the data, thus making the process more including classrooms and cars.4 A study by protective of privacy than it might otherwise be. Bias and Transparency Going back to the issue of ethics, the potential for AI systems to exacerbate social inequality through discriminatory or arbitrary decision- making (often caused by the use of limited data sets for training) has also become a recent source of public concern. 4,$10 As government agencies and courts increasingly turn to AI-based systems to aid and enhance human decision making, including life-altering decisions such as criminal sentencing and bail determinations, it has become apparent that existing social biases can unintentionally become baked into AI-based systems via their algorithms or in the training data on which these algorithms rely. It is also becoming apparent that some of these AI systems are being made intentionally biased to hide arbitrary or unjust results behind a veneer of objectivity and scientific rigor. A recent study by Pro Publica of AI-based risk assessment scores used for bail decisions P H I • A R T I F I C I A L I N T E L L I G E N C E I S T H E F U T U R E • I S S U E I 21
researchers at the ACLU and U.C. Berkeley security and risk professionals and AI practi-
found that Amazon’s facial recognition software tioners to create a bridge between various
incorrectly classified 28 members of Congress knowledge domains in order to enable and
as having arrest records. Moreover, the false support effective oversight activities.
positive rate was 40% for non-white members
compared to 5% for white members. The Malicious Use of AI
subfield of affect recognition raises even more Finally comes the dimension of ethical concern
concerns.4 that puts the most fear into the hearts of
One of the clear lessons to be taken from security professionals and the public alike: the
these examples is the importance of making use of AI for malicious purposes. The concerns
AI-based decision-making systems more start with the attacks on benign AI systems
transparent to the end-user or administrator for malicious purposes, but extend into the
charged with purchasing, installing, and strategic use of AI by attackers to subvert cyber
supervising these systems. Information about defenses.
algorithms and training data should be available By gaining access to an AI-based system$—$or
for inspection on demand, and systems should even to the data on which such a system is
be able to objectively record and display the trained$—$an attacker can potentially change
logic patterns behind their decisions.10 In the way it functions in harmful ways. A world in
addition, regular auditing is clearly important, which everything from cars to heart implants to
as built-in biases may only become apparent power grids relies on AI and are connected to a
as systems are used and the data they collect network is one in which cyber attacks become
and store expands. Such audits will require increasingly life-threatening. Additionally, when
AI determines the flow of personalized news
and other information, malicious actors can
undermine societal trust in government and
media on a grand scale$—$a scenario that is
all-too-common today.
One of the largest public concerns
surrounding the release of any powerful new
technology is that once Pandora’s box has been
opened, whether that invention is for the good
of mankind or engineered to cause its detriment,
there is no putting that new technology back in
the box. Once it is out there in the wild, it is here
to stay, and whether it will make society better
or worse can only be determined by careful
and consistent monitoring over time. AI-based
security technology has now reliably proven
itself to be more effective than traditional
technology (such as antivirus products that
rely on human-generated signatures), but so
long as security practitioners have access to
that cutting-edge technology, so too do people
with malicious agendas.
Preventing the malicious use of AI requires
security professionals to double down on their
commitment to the fundamentals of security,
ensuring the confidentiality, integrity, and
availability, or CIA, of AI-based systems. Again,
such commitments will require greater levels
of transparency into the application of AI at the
22 P H I • A R T I F I C I A L I N T E L L I G E N C E I S T H E F U T U R E • I S S U E Ialgorithmic and code level, to ensure that future of proliferating automated attacks, advances
growth happens in an open and accountable in malware production and distribution, and
fashion. Additionally, as risk professionals the increasingly vulnerable attack surfaces
examine systems for the kinds of problems of organizations that rely on cloud computing
noted above, such as operational failure, and networks with numerous endpoints, the
privacy, and algorithmic bias, they’ll need to unchecked and often unregulated growth in the
consider how threat actors distort or amplify technology sector over the last few decades
the risks to achieve their own ends. has created ever more cybersecurity vulnera-
Security professionals must also remember bilities by exponentially expanding the attack
that threat actors continually look for ways to surface of globally connected companies, while
leverage their own personal application of AI providing malicious actors with increasingly
to boost the effectiveness of their attacks. The powerful tools.
rise of AI-based cyber attacks like DeepLocker Fortunately, most security practitioners
further undermine traditional cybersecurity recognize that AI-fueled cyber attacks can be
methods, making it hard to imagine adequate best thwarted by AI-powered security and are
defenses that do not themselves rely on AI. continually updating their defenses to meet
this challenge. It is also fortunate that leaders
Risks in AI-Driven Cybersecurity in cybersecurity, such as those at Cylance, have
Back in the late 1890s when the first steam- acknowledged that effective cybersecurity
powered motor cars chugged around the for automated systems needs to be driven by
streets at a top speed of 12 miles per hour, AI in order for the defenders to stay one step
nobody would have suspected that just a few ahead of the attackers at all times and provide
decades later, their descendants would make real-world AI-based solutions for security
the horse-drawn carriage obsolete. practitioners to deploy in their environments.
In contrast, long before the global spread and Reducing risk in AI adoption thus requires
integration of AI into all walks of life, security advances in AI-based cybersecurity, coupled
professionals recognized that traditional cyber- with the expansion and adoption of that
security solutions were becoming increas- technology across many industry and
ingly ineffective and antiquated. In the face government sectors, to take it into more
P H I • A R T I F I C I A L I N T E L L I G E N C E I S T H E F U T U R E • I S S U E I 23widespread use.6 Attackers who themselves industry users be given enough information
use AI-based tools to manipulate AI-based about the ways their security is implemented
cybersecurity to, for example, recognize benign and how it has been tested, in order to make
code or behavior as malicious, damage both the informed decisions about their level of risk in
system that security tool was protecting and granting access to that data.
the public reputation of AI. In other words, a
practical first step to securing the very future of Building Ethically-Grounded
AI entails first ensuring that AI-based cyberse- Cybersecurity Organizations
curity systems and any training data that they The risk of AI-based cybersecurity technology
use are themselves secure. making unethical decisions is unlikely
While so much of the ethical oversight to be nearly as large as when AI is used to
of AI depends on transparency within the classify malicious real-word activity, such
security ecosystem, AI-based cybersecurity as is occurring right now in China through
is yet another area in which transparency may a controversial experimental social credit
conflict to some extent with the effectiveness system designed to classify people based on
of the solutions. The advantages of making their personal and public data.23 Nonetheless,
code open in this context may be outweighed by AI-based cybersecurity has the potential to
the risk of subsequent exploitation by malicious exclude individuals or groups from accessing
actors; likewise, where training and testing computer systems in discriminatory or arbitrary
data are supplied, there are obvious privacy ways, most importantly in ways the individuals
concerns around making that data open, as themselves may not fully understand.
we discuss below. The stakes in cybersecurity The same lessons that apply to other
efficacy demand that IT admins and similar AI-based systems in this regard therefore also
24 P H I • A R T I F I C I A L I N T E L L I G E N C E I S T H E F U T U R E • I S S U E Iapply to AI-based cybersecurity: That which is References
not 100% transparent is open to unintentional
flaws and misuse. At the same time, AI-based 1 M. Harkins, “The Promises and Perils of Emerging
Technologies for Cybersecurity: Statement of Malcolm
cybersecurity also has the capacity to make
Harkins,” 2017.
other AI-based decision-making systems
2 “The AI Now Report: The Social and Economic Implications of
more secure, thus protecting them from Artificial Intelligence Technologies in the Near-Term,” 2016.
malicious attacks. 3 A. Campolo, M. Sanfilippo, M. Whittaker, and K. Crawford, “AI
AI-driven cybersecurity can be used to Now 2017 Report,” 2017.
enhance privacy for both individuals and 4 M. Whittaker, K. Crawford, R. Dobbe, G. Fried, E. Kaziunas, V.
corporations, but it also creates incentives for Mathur, S. M. West, R. Ricardson, J. Schultz, and O. Schwartz,
“AI Now Report 2018,” 2018.
the creators of such systems to collect and
5 I. A. Foundation, “Artificial Intelligence, Ethics and Enhanced
use data without informed consent, so the
Data Stewardship,” 2017.
inclination to behave badly must be countered
6 Cylance, “The Artificial Intelligence Revolution in
at all times by organizational and technical Cybersecurity: How Prevention Achieves Superior ROI and
safeguards. The risk of discriminatory or Efficacy,” 2018.
arbitrary decisions made by AI will always be 7 Cylance Data Science Team, Introduction to Artificial
Intelligence for Security Professionals. Cylance, 2017.
present as a result of the self-learning capabil-
ities of such systems, and thus they will always 8 A. Smith, “Franken-algorithms: the deadly consequences of
unpredictable code,” The Guardian, August 30, 2018.
require regular human audits to ensure that
9 J. Glanz, M. Suhartono, and H. Beech, “In Indonesia Lion
individuals and groups are not excluded from Air Crash, Black Box Data Reveal Pilots’ Struggle to Regain
system use or privacy protections. Control,” The New York Times, November 27, 2018.
At the end of the day, our call to action is 10 Committee on Oversight and Government Reform, “Rise of
clear: AI plays a vital and beneficial role in the Machines,” Washington, D.C., 2018.
society and in security, but deploying it in 11 U.N. Global Pulse, “Big Data for Development: Challenges &
Opportunities,” 2012.
the real world requires careful attention to
detail on the part of those who deploy it and a 12 O. Tene and J. Polonetsky, “Big Data for All: Privacy and
User Control in the Age of Analytics,” Northwest. J. Technol.
careful balance of openness and transparency Intellect. Prop., vol. 11, p. xxvii, 2012.
on the part of those who create and supply it. 13 K. Weinberger, A. Dasgupta, J. Attenberg, J. Langford, and A.
While AI-driven security can mount a highly Smola, “Feature Hashing for Large Scale Multitask Learning,”
effective defense against cyber attacks as February 2009.
part of a layered defense strategy, care needs 14 J. Attenberg, K. Weinberger, A. Smola, A. Dasguptaa, and
M. Zinkevich, “Collaborative spam filtering with the hashing
to be taken at all times to ensure that systems trick,” Virus Bulletin, November 2009.
and training data are sufficiently transparent
15 J. Angwin, J. Larson, S. Mattu, and L. Kirchner, “Machine
to allow users and administrators to make Bias,” Pro Publica, May 2016.
informed decisions about acceptable risk levels. 16 J. Larson, S. Mattu, L. Kirchner, and J. Angwin, “How We
Although many of the points outlined here Analyzed the COMPAS Recidivism Algorithm,” 2016.
are largely technical guidelines, they depend on 17 M. Nofferi and R. Koulish, “The Immigration Detention Risk
Assessment,” Georget. Immgr. Law J., vol. 29, 2014.
the creation of accountability structures and an
ethics-focused organizational culture to ensure 18 M. Rosenberg and R. Levinson, “Trump’s catch-and-detain
policy snares many who call the U.S. home,” Reuters,
that they are implemented effectively.21,$22 June 20, 2018.
In the next installment of the AI Manifesto, 19 United States Government, “AI, Automation and the
we will look at the ways organizations can Economy,” no. December 2016.
hold themselves accountable for better cyber 20 D. Acemoglu and P. Restrepo, “The Race between Man and
risk assessments and better overall attack Machine: Implications of Technology for Growth, Factor
Shares, and Employment,” Am. Econ. Rev., vol. 108, no. 6, pp.
defenses. Φ 1488–1542, June 20, 2018.
21 M. Harkins, Managing Risk and Information Security, Second.
Aspen, 2016.
22 M. C. Gentile, “Giving Voice to Values,” Stanford Soc. Innov.
Rev., 2018.
23 Rogier Creemers (via China Law Translation), “Planning
Outline for the Establishment of a Social Credit System
(2014-2020),” 2015.
P H I • A R T I F I C I A L I N T E L L I G E N C E I S T H E F U T U R E • I S S U E I 25You can also read
