Unknown Threats: The Achilles Heel of Email Security - March 2020 - Bitdam
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Unknown Threats:
The Achilles Heel of
Email Security
March 2020
Contents
Executive Summary 2
Motivation 3
Study Description 4
The Results 6
Discussion and Implications 10
Appendix: Technical Challenges Resolved During The study 12
About BitDam 17
Executive Summary
All enterprises rely on email security products to protect their email. Unfortunately, malicious
files and links regularly bypass the leading email security products, which leaves enterprises
vulnerable to email-based attacks including Ransomware, Phishing and malware leading to
Data Breaches. Our experience at BitDam, and the proliferation of threat variants fueled by
automation, led us to suspect that email security products struggle to detect threats they
encounter for the first time (aka Unknown Threats). Therefore, we have conducted an empirical
study aimed at measuring how well these products handle Unknown Threats at First Encounter.
Our findings show that email security systems, such as Microsoft’s Office365 ATP, G-Suite
Enterprise and ProofPoint TAP, have a high miss rate of 20-40% for unknown threats at
first encounter. In fact, up to 45% of threats bypass at least one of these leading products.
Furthermore, it takes them 10-53 hours to start protecting against the threats they miss. This
Detection Gap means that enterprises are continually unprotected against unknown threats,
allowing successful Ransomware, Phishing and Data Breaches. We describe the root cause
of this inherent limitation of email security systems, and suggest threat-agnostic protection
technologies (that do not require knowledge about threats) as an effective remedy. BitDam’s
Advanced Threat Protection (ATP) solution is threat-agnostic, has a very low miss rate of
unknown threats, significantly reducing the risk of successful email-based attacks.
Current data from the continuous study is available here. contact@bitdam.com 2
Motivation
Most cyber-attacks start with an email bearing a malicious file or link. Enterprises use an email
security product to protect their email, but are still vulnerable to email-based attacks including
Ransomware, Phishing and data breaches. As threat actors continue to develop their attacks
to be stealthier, malicious files bypass the leading email security products on a regular basis,
landing in mailboxes.
BitDam provides an Advanced Threat Protection (ATP) solution that blocks content-borne
threats (files and links) on multiple enterprise collaboration channels (email, cloud drives,
messaging). As part of our work with customers we have seen first-hand that email security
is bypassed by malicious content. First, being installed as a last line of defense, our solution
detects malicious attachments that slip through various Secure Email Gateways (SEGs).
Additionally, when customers try our Breach & Attack Simulation (BAS) tool for simulating email
attacks, the results are consistent – all the email security products score poorly and are unable
to block the majority of the samples sent during the BAS tests.
It should also be noted that increased use of automation allows attackers to create many
variants (mutations) for a malware or malicious file, potentially inundating email security
products with new unknown threats. As these products rely on threat data for detection, a
significant increase in the numbers of unknown threats may impair their efficacy.
Our own experience described above, together with the proliferation of threat variants,
suggests that the problem with the leading email security products is that they are struggling
when it comes to detecting Unknown Threats, i.e., threats they encounter for the first time. Thus,
we decided to conduct an empirical study aimed at measuring just how well email security
systems handle unknown threats at first encounter. The unknown threats we use are fresh, live
samples of malicious files. We measure Miss Rate at First Encounter and Time To Detect (TTD)
for unknown threats. This type of analysis is not generally available to potential customers of
popular email security solutions.
The findings show that email security systems, such as Microsoft’s Office 365 ATP, G-Suite
Enterprise and ProofPoint TAP, have a Miss Rate at First Encounter in the range of 20-40% for
unknown threats. Their Time To Detect is in the range of 10-53 hours on average. This observed
Detection Gap means that enterprises are unprotected against many specific unknown threats
every day, explaining the increasing success of Ransomware, Phishing and Data Breaches. We
believe that the handicap of email security with regard to unknown threats is inherent, and
stems from their complete reliance on knowledge about a threat. If they already ‘know’ a threat
than they can detect it; if not, then they go through a process designed to turn the Unknown
into a Known. This process, described in the Discussion section takes a significant amount
of time to run its course, creating a window of opportunity for the attacker and risk to the
enterprise.
In this paper we will describe the study and how it is being conducted. We will present the
data, and discuss its implications. Finally, we will suggest an explanation for what we see as a
fundamental limitation of ‘knowledge-driven’ email security systems, and propose how ‘threat-
agnostic’ technologies can overcome it to provide protection against all unknown threats.
contact@bitdam.com 3Study Description
Our empirical study has been in progress since October 2019. Thousands of ‘fresh’ malicious
file samples, taken from various sources, have been sent to mailboxes protected by Office
365 ATP, G-Suite enterprise, and ProofPoint TAP. The fresh samples are considered Unknown
Threats, for which we want to measure Miss Rate and Time To Detect.
In this study we use real functional (live) email addresses in an internet environment that
are fully protected by the security products mentioned above. The study uses the following
process:
Continuously Obtain fresh samples of malicious files
Qualify the samples: validate that they are malicious; modify them sufficiently so that they
will not be blocked when we send them out
For each sample: Send an email containing it (the malicious item) to the mailboxes
protected by the security products covered in this study
Monitor all mailboxes to log which sample made it to the inbox
If a sample is not detected the first time it is sent (i.e., at first encounter), Re-send it until it
is eventually detected
Collect all the data needed for measuring the Miss Rate at First Encounter and the Time
To Detect (TTD)
Analyze the data per email security product
� Qualifying the samples
The files we send are chosen from live traffic that we have detected and flagged as malicious.
The file types cover any non-executable files that cause malicious code to run on the device
(Ransomware, password stealers, data breaches, APT and so on). These are mainly Office files
and PDF files.
Due to a concern that False Positives (FPs) might lead to private data being exposed, we first
verify that the file is on a public source before sending it out. A small percentage of the files has
to be verified manually by our research team.
We also verify that each file was uploaded to the public source recently. Our goal is to
minimize the time between when the sample is first seen in the wild and the time it is sent
to the target mailbox. The reasoning behind this is that vendors such as, Google, Microsoft,
Proofpoint, can see trends before anyone else, and could potentially be able to examine them
before we are able to receive them.
contact@bitdam.com 4� Configuration of target mailboxes
In general, we do not control the exact configuration1 of the mailboxes we send the samples to,
since they are configured by the customer. This ensures that our study is based on real-world
scenarios.
An Office ATP mailbox is configured as appropriate for an organization with reasonable security
control. We use instances in Europe and in the USA. All mailboxes are configured such that
malicious attachments are replaced with a text message stating the attachment name and the
reason for blocking it.
A G-Suite Enterprise mailbox is configured with all advanced options selected, including the
sandbox option in pre-delivery mode. Although all mailboxes are configured to “pre-delivery
and discard” of the email message, some messages do get through to the inbox, with the
attachment marked as “virus found”. We handle these attachments by comparing our send
log to the Google receive logs. These marked attachments constitute less than 10% of the
attachments in the inbox.
� Measuring TTD (time to detect)
In order to measure how long it takes for an email security solution to start blocking a malicious
file that it had previously missed, we re-send the sample until it is blocked, or a timeout expires.
Our resend schedule is as follows:
In the first 4 hours re-send the sample every 30 minutes
For the next 20 hours re-send the sample every 2 hours
For the next 7 days re-send the sample every 6 hours
After 7 days we stop re-sending the sample
In the Appendix we discuss some technical challenges that we faced during this study, and how
we managed to overcome them.
x1- Example of Office 365 ATP Safe Attachments policies contact@bitdam.com 5The Results
Our study of the efficacy of email security products in detecting unknown threats at first
encounter is in progress. So far, we have been running the study for almost 5 months, targeting
mailboxes protected by O365 ATP and G-Suite Enterprise. Running against ProofPoint TAP has
started in late January. During this period, we have sent many thousands of ‘fresh’ malicious file
samples to multiple mailboxes protected by each security product. Below we provide charts
for the two main indicators, Miss Rate at First Encounter and TTD, based on the data collected
during this period.
The results of the study are summarized in Figure 1. The detailed results for each email security
product are given below.
Miss Rate Time (h)
40.0% 60
53.3
35.0%
50
30.0%
40
25.0%
31.6
20.0% 30
15.0%
20
10.0%
10.1 10
5.0%
0.0% 0
ProofPoint OATP G Suite
Average Time to Detect
Figure 1. Overall Study Results
� Office 365 ATP
The charts below cover the period October 23rd 2019 – March 11th 2020. The miss rate during
this period is about 25%. TTD average is about 53 hours. About 20% of unknown threats take 4
days or more to be detected.
contact@bitdam.com 6O365 ATP Miss Rate (average 25.3%)
Miss Rate
60%
50%
40%
30%
20%
10%
0%
-
-
-
-
-
-
-
-
-
-
-
19
19
19
19
19
19
20
20
20
20
20
20
20
20
20
20
20
20
20
20
20
20
0/
3/
7/
1/
5/
9/
2/
6/
9/
3/
8/
/2
/0
/1
/0
/1
/2
/1
/2
/0
/2
/0
10
11
11
12
12
12
01
01
02
02
03
O365 ATP Time To Detect (average 53.3h)
#Threats
160
140
120
100
80
60
40
20
0
≤6 [6, 12] [12, 18] [18, 24] [24, 30] [30, 36] >36 Time (h)
Figure 2. Office 365 ATP Miss Rate and Time To Detect for Unknown Threats
� G Suite Enterprise
The charts below cover the period November 5th 2019 - March 11th 2020. The Miss rate during
this period is 35.4%. TTD average is about 32 hours. About 10% of unknown threats take 3 days
or more to be detected.
contact@bitdam.com 7G Suite Miss Rate (average 35.4%)
Miss Rate
80%
70%
60%
50%
40%
30%
20%
10%
0%
-
-
-
-
-
-
-
-
-
-
19
19
19
19
19
20
20
20
20
20
20
20
20
20
20
20
20
20
20
20
3/
7/
1/
5/
9/
2/
6/
9/
3/
8/
/0
/1
/0
/1
/2
/1
/2
/0
/2
/0
11
11
12
12
12
01
01
02
02
03
G Suite Time To Detect (average 31.6h)
#Threats
500
400
300
200
100
0
≤6 [6, 12] [12, 18] (18, 24] [24, 30] [30, 36] >36 Time (h)
Figure 3. G Suite Miss Rate and Time To Detect for Unknown Threats
� ProofPoint TAP
The charts below cover the period January 26th 2020 - March 11th 2020. The Miss rate during
this period is 22.6%. TTD average is about 10 hours.
contact@bitdam.com 8ProofPoint Miss Rate (average 22.6%)
Miss Rate
60%
50%
40%
30%
20%
10%
0%
-
-
-
-
-
-
-
20
20
20
20
20
20
20
20
20
20
20
20
20
20
3/
6/
2/
9/
6/
1/
8/
/2
/2
/0
/0
/1
/0
/0
02
01
02
02
02
03
03
ProofPoint Time To Detect (average 10.1h)
#Threats
30
25
15
10
5
0
≤6 [6, 12] [12, 18] [18, 24] [24, 30] [30, 36] >36 Time (h)
Figure 4. ProofPoint Miss Rate and Time To Detect for Unknown Threats
� Exposure Time Intervals
The chart below shows time intervals when O365 ATP is ‘blind’ to selected unknown threats
it does not detect at first encounter. Although this is a relatively small sample of threats, it is
notable that seven different unknown threats breeze through email security on Nov. 4th, ten on
November 6th, six on Nov. 12th etc.
contact@bitdam.com 9Detection Gaps for Missed Threats
YCIp.csv
ea2b.xlsm
2964.doc
e3ef.xls
23d4.doc
dbbd.doc
d940.xls
ba2a.xls
5a98.doc
3d7b.xls
6491.xls
6b98.xls
2d2a.rtf
d791.xlsm
f896.xlsm
65f0.doc
27d5.xls
254d.doc
InfoLeaker.doc
62ba.xlsm
10/31/2019 11/2/2019 11/4/2019 11/6/2019 11/8/2019 11/10/2019 11/12/2019 11/14/2019
Figure 5. ‘No Detection’ time intervals for various samples
� Discussion and Implications
The findings described in this document suggest that leading email security products have
a very high miss rate for unknown threats. Considering that email security products depend
heavily on threat data, this should not come as a big surprise. Figure 6 is an abstraction of the
process used by email security vendors to handle unknown threats. When they encounter the
first sample of an unknown threat, they usually ‘notice’ it through a reputation service or threat
hunting (GAP 1). It then takes them some time to qualify the sample as malicious (GAP 2), and
then they may need to apply some changes to their product to be able to detect the sample
(GAP 3). This abstraction applies to vendor products using signatures, sandboxing, as well as
machine learning.
contact@bitdam.com 10Reputation |
Threat hunting
GAP 1 GAP 2 GAP 3 Detection
First encounter Qualifying maliciousness (i.e. Training ML | Adding a
not a marketing campaign) mechanism to detect it
Figure 6. Common process for handling unknown threats
The following description by Palo Alto Networks ties this time-related data dependency to
recent changes in attackers’ practices:
“Attackers use automation to move fast and deploy new threats at breakneck speeds… A
next-generation security platform rapidly analyzes data, turning unknown threats into known
threats, creating an attack DNA, and automatically creating as well as enforcing a full set of
protections...”.
Currently there are many more unknown threats than in the past. The industry’s best bet
for handling them is to use data to turn them into known threats as quickly as possible. The
problem is that as a result, Time To Detect (TTD) becomes hours or even days, which presents
an unacceptable risk for the customers.
Since data-driven threat detection technologies fail to provide protection against unknown
threats due to their inherent dependency on data, they must be augmented by a different
type of technology in order to provide better email security. We believe that the answer lies in
threat-agnostic detection technologies. If a detection technology is independent of data about
the threats it identifies, then it should potentially provide excellent protection against unknown
threats. It should probably provide similar protection against known and unknown threats.
The BitDam ATP solution is built on top of a unique threat-agnostic detection engine. BitDam’s
technology uses a model of ‘clean’ execution flows of the application used to open (render) a
file or a link (e.g., Office apps, Acrobat Reader, Chrome, etc.). This model consists of whitelisted
CPU-level code execution flows of the application. A model is built for each application by
tracing the execution of the application on benign files. When a new file is scanned, BitDam
compares the execution flows extracted while running the application opening the file to the
whitelisted flows included in the model for this application to calculate a verdict.
BitDam’s model-driven threat detection technology at the heart of BitDam ATP allows it to
reach extremely high detection rates for unknown threats at first encounter. It’s TTD is zero, so
full protection power is available at all times. BitDam is able to correctly identify all the unknown
threats missed by the email security products we cover in our study. This makes BitDam ATP a
natural choice for augmenting current email security products and considerably reduce the risk
customers face today from their incoming email.
contact@bitdam.com 11Appendix: Technical Challenges Resolved
During The study
Challenge 1: Sending the emails containing the
samples
Sending emails containing malicious attachments is not a trivial task. A naïve approach of
simply trying to send them using common services such as Google, Yahoo and others, ends up
being blocked fairly quickly when an attempt to attach the file is made. Using email services
that are not scanning for malicious attachments (like SendGrid2), results in an account freeze in
less than one day.
Since reputation services are effective against known attacks, and since BitDam isn’t obtaining
the samples before Gmail or O365 do, we needed to apply some changes to the files. The
changes don’t alter the essence of the attack - it includes the same IP addresses, dropped files,
sandbox evasion techniques and so on.
There are two major methods we employ in order to modify the files, thus avoiding being
blocked before even sending the email.
� Method #1
Changing the hash of the file by adding benign data to it. For example, for RTF files, adding
spaces to some of the streams. The malicious content in the file is left intact.
This method alone is effective for logical exploits that one can find in RTF files, but is ineffective
for macro attacks (due to entropy calculations done on the macro itself by the email security
vendor).
20
bitdamluc...
15
Count
10
5
0
20
david.bs@bi...
15
Count
10 Success
5 Failed
0
20
g.thompson...
15
Count
10
5
0
03:00 05:00 07:00 09:00 11:00
Figure 7. Sending files with hash changed
x2 - World’s largest cloud-based email delivery platform contact@bitdam.com 12� Method #2
Changing the static signature of a macro:
1. Adding comments consisting of random words. Changing the macro’s entropy usually
results in the file not being blocked on sending. This method works well, and about 90%
of the malicious files are sent successfully.
15
bitdamluc...
10
Count
5
0
Success
15 Failed
g.thompson.....
10
.
5
0
14:00 17:00 20:00 23:00
Figure 8. Sending files with comments in the macro
2. Converting the code of each macro function to a base64 string and adding a code to
decompress this code. This method works for 99% of the samples, as static analysis can’t
detect if there is any issue with the macro.
10
bitdamluc...
Count
5
0
Success
Failed
10
g.thompson......
Count
5
0
17:00 20:00 23:00
Figure 9. Sending files with base64 function contents
contact@bitdam.com 13Challenge 2: Measuring Miss Rate
We send a few hundred malicious files each day to each target mailbox. For an O365 ATP
protected mailbox, due to the way it is configured, we can tell which emails were blocked and
view their eml headers where O365 writes the end to end latency of the scanning. We noticed
that there are mainly two ranges of scanning times, below 5 seconds and above 2 minutes.
All the files with scanning time below 5 seconds, were detected. In addition, inside the
text file that replaced them, those files had a signature stating the malware type, i.e.
“a02b332b78ede5d03490d4db96ebad85fca7fc2f.rtf O97M/CVE-2017-11882.AX!eml”
which means it is the logical exploit marked as CVE-2017-11882, that is known as triggering
“eqendt32.exe”.
We can deduce that these files were already known because of two main reasons:
1. The scanning time was below 5 seconds, which shows it didn’t go through a dynamic
analysis solution (since dynamic analysis seems to take 2 minutes or more)
2. The malware type is descriptive and accurate, which means some sort of a signature
mechanism was applied.
Hence, we assume that files with the low scanning latency are known threats and were detected
using a reputation service. Due to the strong mechanism of permutation we apply to macros,
the modified files were almost never detected – they had long scanning time with detonation.
On the other hand, RTFs with CVE-2017-11882 are almost always detected due to a weak
permutation mechanism (only basic hash change).
For files with a high scan time, we can deduce that they were unknown threats, hence had
to go through the complete scanning process that includes a dynamic analysis\detonation
mechanism. Only these files were included in the calculation of detection rates.
In the next step, we looked at the macros in each file we sent and went through long scanning
(aka – unknown). We noticed that there are many files with different hashes but with the exact
same macro (Figure 10). Since the same macro will result in the same IOCs (IP addressed,
dropped files hashes, etc.), it doesn’t make sense to consider these files as different, and once
you are able to detect one of them you should be able to detect all these similar threats.
Lastly, we noticed that some macros are “similar” to others, meaning that they are the same
macro but slightly modified. See Figure 10 which has a similar macro to the one in Figure 11.
The changes we have seen include:
Changes of strings\comments
Changes in variable names
Addition of benign macro functions\lines
As in the case of the copied macros, the similar macros will also result in the same IOCs, thus
were not included in the calculation of the overall result.
contact@bitdam.com 14Figure 10. File macro (hash: 516d29bed93c78cd03a2ba8f2704050572954371, 8ccf648775188207f47fc4b7e7758ad13837b6a8)
contact@bitdam.com 15Figure 11. Similar macro (hash: c1883e56b58a2027d25ea6184f5de9393a140981)
contact@bitdam.com 16About BitDam
BitDam is a pioneer in cyber defense, securing enterprise email (Office 365, G-Suite, MS
Exchange), cloud drives (OneDrive, G-Drive, Dropbox, Box etc.) and other collaboration tools
from ransomware, malware, and phishing.
Unlike the alternatives that give a “grace period” to unknown cyberthreats, BitDam’s patented
attack-agnostic technology stops malicious files and links at first encounter with unprecedented
detection rates. Independent of feeds, reputation and intelligence services, BitDam’s cloud-
based Advanced Threat Protection (ATP) detects never-seen-before attacks of any type,
providing a remarkably higher detection rate and empowering organizations to collaborate
safely.
Recognized by Frost & Sullivan for its technology leadership, BitDam’s award-winning
ATP solution is utilized by hundreds of thousands of end-users and deployed by leading
organizations in Europe and the US, with a proven record of detecting threats that other
security solutions fail to uncover.
contact@bitdam.com 17You can also read
