Small business endpoint protection - APR - JUN 2021 - SE Labs
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
small business Endpoint protection APR - JUN 2021 selabs.uk info@SELabs.uk @SELabsUK selabs.uk/newsletter blog.selabs.uk
SE Labs tested a variety of anti-malware (aka ‘anti-virus’; aka ‘Endpoint security’) products
from a range of well-known vendors in an effort to judge which were the most effective.
Each product was exposed to the same threats, which were a mixture of targeted attacks
using well-established techniques and public email and web-based threats that were
found to be live on the internet at the time of the test.
The results indicate how effectively the products were at detecting and/or protecting
against those threats in real time.
2 Small Business Endpoint Protection April - June 2021
MANAGEMENT
Chief Executive Officer Simon Edwards
Chief Operations Officer Marc Briggs
CONTENTS
Chief Human Resources Officer Magdalena Jurenko
Chief Technical Officer Stefan Dumitrascu
Introduction04
Testing team
Nikki Albesa Executive Summary 05
Zaynab Bawa
Thomas Bean 1. Total Accuracy Ratings 06
Rory Brown
Solandra Brewster
Small Business Endpoint Protection Awards 07
Liam Fisher
Gia Gorbold
Jeremiah Morgan
2. Threat Responses 08
Joseph Pike
Sarah Sargeant 3. Protection Ratings 10
Dave Togneri
Dimitrios Tsarouchas 4. Protection Scores 12
Stephen Withey
Liangyi Zhen 5. Protection Details 13
IT support
6. Legitimate Software Ratings 14
Danny King-Smith
Chris Short
6.1 Interaction Ratings 15
Publication
Sara Claridge 6.2 Prevalence Ratings 16
Colin Mackleworth
6.3 Accuracy Ratings 16
Website selabs.uk
Twitter @SELabsUK 6.4 Distribution of Impact Categories 17
Email info@SELabs.uk
LinkedIn www.linkedin.com/company/se-labs/
7. Conclusions 17
Blog blog.selabs.uk
Phone +44 (0)203 875 5000
Post SE Labs Ltd,
Appendix A: Terms Used 18
55A High Street, Wimbledon, SW19 5BA, UK
Appendix B: FAQs 18
SE Labs is ISO/IEC 27001 : 2013 certified and
BS EN ISO 9001 : 2015 certified for The Provision Appendix C: Product Versions 19
of IT Security Product Testing.
Appendix D: Attack Types 20
SE Labs is a member of the Microsoft Virus Information
Alliance (VIA); the Anti-Malware Testing Standards
Organization (AMTSO); and the Messaging, Malware
and Mobile Anti-Abuse Working Group (M3AAWG).
AMTSO Standard Reference:
selabs.uk/amtso21q2
Document version 1.0 Written 16th July 2021
© 2021 SE Labs Ltd
3 Small Business Endpoint Protection April - June 2021
INTRODUCTION
Endpoint protection needs your attention
How to get out more than you put in
Technology gives us tools to achieve sometimes amazing things. But no If you were going to buy a new security fence, burglar alarm or CCTV
matter how advanced, it usually requires humans to make an effort. system you would research the various options and consider how to
deploy it in your very specific situation. The same should follow for
We’ve all heard the terms, “fire and forget” or “plug and play” but these are endpoint protection systems. What do you really need? Are the
usually marketing dreams. In security, as with any other area, you can buy tools, candidates basically capable? And can you get the best out of them in
but you need to understand how to use them if you are going to succeed. your environment. We hope this report helps you answer some of these
important questions.
Adding security to a business or home isn’t (or shouldn’t be) a box-ticking
exercise. General advice or even official standards highlight ‘endpoint security’ If you spot a detail in this report that you don’t understand, or would
as a must-have. Having seen plenty of good and bad deployments, we advise like to discuss, please contact us via our Twitter or LinkedIn accounts.
that choosing an endpoint security product is just the start. SE Labs uses current threat intelligence to make our tests as realistic
as possible. To learn more about how we test, how we define ‘threat
You need to understand not only the strengths and weaknesses of endpoint intelligence’ and how we use it to improve our tests please visit our
security, but the specific abilities of the products you are considering. And your website and follow us on Twitter.
specific needs. One size does not fit all.
This test report was funded by post-test consultation services provided
Sometimes understanding your tools can save you money. They might be able by SE Labs to security vendors. Vendors of all products included in this
to do things you hadn’t realised previously. Buying new tools and add-ons can report were able to request early access to results and the ability to
become expensive, particularly if they are not necessary. dispute details for free. SE Labs has submitted the testing process behind
this report for compliance with the AMTSO Testing Protocol Standard v1.3.
Your own expertise (or your team’s) is also important. A good basic endpoint To verify its compliance please check the AMTSO reference link at the
security product might be good enough if you have thought through other bottom of page three of this report or here.
security layers on your network. The latest and greatest AI-powered start-up
rock star product might not do everything it claims, and almost certainly won’t
be everything you need. And if you need a PhD to use it, so much the worse.
4 Small Business Endpoint Protection April - June 2021
Executive Summary
Product Names
It is good practice to stay up to date with the latest version of your chosen Endpoint ● The endpoints were generally effective at handling
security product. We made best efforts to ensure that each product tested was the very general threats from cyber criminals…
latest version running with the most recent updates to give the best possible outcome. Most products were largely capable of handling public
web-based threats such as those used by criminals to attack
For specific build numbers, see Appendix C: Product Versions on page 19. Windows PCs, tricking users into running malicious files or
running scripts that download and run malicious files. Most
were 100 per cent effective.
● .. and targeted attacks were prevented in nearly all cases.
Only Bitdefender failed to provide 100 per cent protection in
this part of the test, missing one attack.
● False positives were not an issue for the products
Most of the endpoint solutions were good at correctly
Executive summary classifying legitimate applications and websites. With two
Protection Legitimate Total exceptions they allowed all of the legitimate websites
Accuracy Accuracy Accuracy
and applications.
Products Tested Rating (%) Rating (%) Rating (%)
Kaspersky Small Office Security 100% 100% 100%
● Which products were the most effective?
McAfee Endpoint Security 100% 100% 100%
Products from Kaspersky, McAfee, Microsoft and Sophos
Microsoft Defender Antivirus (enterprise) 100% 100% 100% achieved 100% in this test. Products from FireEye, ESET,
Sophos Intercept X 100% 100% 100% Bitdefender and Webroot achieved extremely good results
FireEye Endpoint Security 98% 100% 99% due to a combination of their ability to block malicious URLs,
handle exploits and correctly classify legitimate applications
ESET Endpoint Security 96% 100% 99%
and websites. They all achieved AAA awards.
Bitdefender Small Office Security 98% 97% 97%
Webroot SecureAnywhere Endpoint Protection 94% 97% 96%
Products highlighted in green were the most accurate, scoring 85 per cent or more for Total Accuracy.
Those in yellow scored less than 85 but 75 or more. Products shown in red scored less than 75 per cent.
For exact percentages, see 1. Total Accuracy Ratings on page 6.
5 Small Business Endpoint Protection April - June 2021
1. Total Accuracy Ratings
Judging the effectiveness of an Endpoint security
total accuracy RATINGS
product is a subtle art, and many factors are at play
Total Accuracy Total
when assessing how well it performs. To make things Product Rating Accuracy (%) Award
easier we’ve combined all the different results from
Kaspersky Small Office Security 1,136 100% AAA
this report into one easy-to-understand graph.
McAfee Endpoint Security 1,136 100% AAA
Microsoft Defender Antivirus (enterprise) 1,136 100% AAA
The graph below takes into account not only each
product’s ability to detect and protect against threats, Sophos Intercept X 1,135 100% AAA
but also its handling of non-malicious objects such as FireEye Endpoint Security 1,127 99% AAA
web addresses (URLs) and applications. ESET Endpoint Security 1,122 99% AAA
Bitdefender Small Office Security 1,106 97% AAA
Not all protections, or detections for that matter, are
Webroot SecureAnywhere Endpoint Protection 1,089.5 96% AAA
equal. A product might completely block a URL, which
stops the threat before it can even start its intended
series of malicious events. Alternatively, the product
might allow a web-based exploit to execute but
prevent it from downloading any further code to the
target. In another case malware might run on the target
for a short while before its behaviour is detected and its
code is deleted or moved to a safe ‘quarantine’ area for
Kaspersky Small Office Security
future analysis. We take these outcomes into account
when attributing points that form final ratings. McAfee Endpoint Security
Microsoft Defender Antivirus (enterprise)
For example, a product that completely blocks a threat
is rated more highly than one that allows a threat to run Sophos Intercept X
for a while before eventually evicting it. Products that
FireEye Endpoint Security
allow all malware infections, or that block popular
legitimate applications, are penalised heavily. ESET Endpoint Security
Bitdefender Small Office Security
Categorising how a product handles legitimate objects Total Accuracy
Webroot SecureAnywhere Endpoint Protection Ratings combine
is complex, and you can find out how we do it in protection and
6. Legitimate Software Ratings on page 14. 0 284 568 852 1,136 false positives.
6 Small Business Endpoint Protection April - June 2021
Small Business Endpoint Protection Awards
The following products win SE Labs awards:
● Kaspersky Small Office Security
● McAfee Endpoint Security
● Microsoft Defender Antivirus (enterprise)
● Sophos Intercept X
● FireEye Endpoint Security
● ESET Endpoint Security
● Bitdefender Small Office Security
● Webroot SecureAnywhere Endpoint Protection
7 Small Business Endpoint Protection April - June 2021
2. Threat Responses
Full Attack Chain: Testing every layer of detection and protection
Attackers start from a certain point and don’t stop steps leading to actually stealing data or causing damage or theft has been achieved, then similarly
until they have either achieved their goal or have some other form of damage to the network. the product may be denied a chance to
reached the end of their resources (which could demonstrate its abilities in behavioural detection
be a deadline or the limit of their abilities). If the test starts too far into the attack chain, such and so on.
This means, in a test, the tester needs to begin as executing malware on an endpoint, then many
the attack from a realistic first position, such as products will be denied opportunities to use the Attack stages
sending a phishing email or setting up an infected full extent of their protection and detection The illustration below shows some typical
website, and moving through many of the likely abilities. If the test concludes before any ‘useful’ stages of an attack. In a test each of these should
Attack Chain: How Hackers Progress
Figure 1. A typical attack starts with
an initial contact and progresses 01 02 03 04 05 06
through various stages, including
reconnaissance, stealing data and
causing damage.
PDF
Figure 2. This attack was initially
successful but only able to progress 01 02 03 04 05 06
as far as the reconnaissance phase.
PDF
Figure 3. A more successful attack
manages to steal passwords but 01 02 03 04 05 06
wholesale data theft and destruction
was blocked.
PDF
8 Small Business Endpoint Protection April - June 2021be attempted to determine the security represented by steps two through to seven below. be that their goal is persistent presence on the
solution’s effectiveness. This test’s results We broadly categorise these stages as: Access systems to monitor for activities, slowly steal
record detection and protection for each of (step 2); Action (step 3); Escalation (step 4); information and other more subtle missions.
these stages. and Post-escalation (step 5).
In figure 3. the attacker has managed to
We measure how a product responds to the In figure 1. you can see a typical attack running progress as far as stage five. This means that
first stages of the attack with a detection and/ from start to end, through various ‘hacking’ the system has been seriously compromised.
or protection rating. Sometimes products allow activities. This can be classified as a fully The attacker has a high level of access and has
threats to run but detect them. Other times successful breach. stolen passwords. However, attempts to exfiltrate
they might allow the threat to run briefly before data from the target were blocked, as were
neutralising it. Ideally they detect and block In figure 2. a product or service has interfered attempts to damage the system.
the threat before it has a chance to run. with the attack, allowing it to succeed only as
Products may delete threats or automatically far as stage 3, after which it was detected and The table below shows how a typical way
contain them in a ‘quarantine’ or other safe neutralised. The attacker was unable to progress in which security testers illustrate attackers’
holding mechanism for later analysis. through stages 4 and onwards. behaviour. It is largely the same as our images
above, but more detailed.
Should the initial attack phase succeed we then It is possible that attackers will not cause
measure post-exploitation stages, which are noticeable damage during an attack. It may
MITRE Example Attack Chain Details
Initial Access Execution Privilege Escalation Credential Access Discovery Collection Command and Control Exfiltration
Spearphishing via Command-Line File and Directory
Input Capture Input Capture Data Encoding
Service Interface Discovery
Powershell Process Discovery Exfiltration Over C2
Bypass UAC
Channel
Spearphishing
Scripting OS Credential Dumping Data from Local System Data Obfuscation
Link System Information
Discovery
User Execution
C2
Spearphishing Link Scripting Bypass UAC Input Capture System Information Input Capture Data Encoding Exfiltration Over C2
Discovery Channel
9 Small Business Endpoint Protection April - June 20213. Protection Ratings
The results below indicate how effectively the alerts the user, who may now take steps to secure target system is successful this score is applied.
products dealt with threats. Points are earned the system. Examples of successful commands include listing
for detecting the threat and for either blocking current running processes, exploring the file system
or neutralising it. Rating Calculations and so on. If the first command is attempted and
We calculate the protection ratings using the the session is terminated by the product without
Detected (+1) following formula: the command being successful the score of
If the product detects the threat with any degree Neutralised (see above) will be applied.
of useful information, we award it one point. Protection Rating =
(1x number of Detected) + Action (-1)
Blocked (+2) (2x number of Blocked) + If the attacker is able to exfiltrate a document from
Threats that are disallowed from even starting (1x number of Neutralised) + the target’s Desktop of the currently logged in user
their malicious activities are blocked. Blocking (1x number of Complete remediation) + then an ‘action’ has been successfully taken.
products score two points. (-5x number of Compromised)
Escalation (-2)
Complete Remediation (+1) The ‘Complete remediation’ number relates to The attacker attempts to escalate privileges to NT
If, in addition to neutralising a threat, the product cases of neutralisation in which all significant Authority/System. If successful, an additional two
removes all significant traces of the attack, it traces of the attack were removed from the target. points are deducted.
gains an additional one point.
These ratings are based on our opinion of Post-Escalation Action (-1)
Neutralised (+1) how important these different outcomes are. After escalation the attacker attempts actions
Products that kill all running malicious processes You may have a different view on how seriously that rely on escalated privileges. These include
‘neutralise’ the threat and win one point. you treat a ‘Compromise’ or ‘Neutralisation without attempting to steal credentials, modifying the file
complete remediation’. If you want to create your system and recording keystrokes. If any of these
Persistent Neutralisation (-2) own rating system, you can use the raw data from actions are successful then a further penalty of
This result occurs when a product continually 5. Protection Details on page 13 to roll your own one point deduction is applied.
blocks a persistent threat from achieving its aim, set of personalised ratings.
while not removing it from the system.
Targeted Attack Scoring
Compromised (-5) The following scores apply only to targeted attacks
If the threat compromises the system, the and are cumulative, ranging from -1 to -5.
product loses five points. This loss may be
reduced to four points if it manages to detect Access (-1)
the threat (see Detected, above), as this at least If any command that yields information about the
10 Small Business Endpoint Protection April - June 2021Email security
Protection RATINGS
services
protection
Product Protection Rating Protection Rating (%)
Kaspersky Small Office Security 400 100%
McAfee Endpoint Security 400 100%
Microsoft Defender Antivirus (enterprise) 400 100% Which services from
Sophos Intercept X 399 100%
well-known vendors are
FireEye Endpoint Security 391 98%
the most effective?
Bitdefender Small Office Security 390 98%
ESET Endpoint Security 386 96%
Webroot SecureAnywhere Endpoint Protection 377 94%
Average 98%
Kaspersky Small Office Security
McAfee Endpoint Security
Microsoft Defender Antivirus (enterprise)
Sophos Intercept X
FireEye Endpoint Security
Bitdefender Small Office Security Protection Ratings
are weighted to
ESET Endpoint Security show that how
products handle
Webroot SecureAnywhere Endpoint Protection threats can be
subtler than just
Download
0 100 200 300 400 ‘win’ or ‘lose’.
now!
selabs.uk/essp2020
11 Small Business Endpoint Protection April - June 20214. Protection Scores
This graph shows the overall level of protection,
Protection scores
making no distinction between neutralised and
Product Protection Score
blocked incidents.
Kaspersky Small Office Security 100
For each product we add Blocked and Neutralised McAfee Endpoint Security 100
cases together to make one simple tally. Microsoft Defender Antivirus (enterprise) 100
Sophos Intercept X 100
Webroot SecureAnywhere Endpoint Protection 100
Bitdefender Small Office Security 99
FireEye Endpoint Security 99
ESET Endpoint Security 98
Kaspersky Small Office Security
McAfee Endpoint Security
Microsoft Defender Antivirus (enterprise)
Sophos Intercept X
Webroot SecureAnywhere Endpoint Protection
Bitdefender Small Office Security
FireEye Endpoint Security Protection Scores
are a simple count
ESET Endpoint Security of how many times
a product protected
0 25 50 75 100 the system.
12 Small Business Endpoint Protection April - June 20215. Protection Details
These results break down how each product Protection DETAILS
handled threats into some detail. You can see
Product Detected Blocked Neutralised Compromised Protected
how many detected a threat and the levels of
Kaspersky Small Office Security 100 100 0 0 100
protection provided.
McAfee Endpoint Security 100 100 0 0 100
Products sometimes detect more threats than Microsoft Defender Antivirus (enterprise) 100 100 0 0 100
they protect against. This can happen when Sophos Intercept X 100 100 0 0 100
they recognise an element of the threat but Webroot SecureAnywhere Endpoint Protection 100 87 13 0 100
aren’t equipped to stop it. Products can also FireEye Endpoint Security 99 99 0 1 99
provide protection even if they don’t detect
Bitdefender Small Office Security 100 97 2 1 99
certain threats. Some threats abort on
ESET Endpoint Security 98 97 1 2 98
detecting specific Endpoint protection software.
Kaspersky Small Office Security Blocked
Neutralised
McAfee Endpoint Security
Compromised
Microsoft Defender Antivirus (enterprise)
Sophos Intercept X
Webroot SecureAnywhere Endpoint Protection
FireEye Endpoint Security
Bitdefender Small Office Security
This data shows in
ESET Endpoint Security detail how each
product handled
0 25 50 75 100 the threats used.
13 Small Business Endpoint Protection April - June 20216. Legitimate Software
Ratings
These ratings indicate how accurately the products LEGITIMATE SOFTWARE RATINGS
classify legitimate applications and URLs, while Product Legitimate Accuracy Rating Legitimate Accuracy (%)
also taking into account the interactions that each
ESET Endpoint Security 736 100%
product has with the user. Ideally a product will
FireEye Endpoint Security 736 100%
either not classify a legitimate object or will classify
Kaspersky Small Office Security 736 100%
it as safe. In neither case should it bother the user.
McAfee Endpoint Security 736 100%
We also take into account the prevalence Microsoft Defender Antivirus (enterprise) 736 100%
(popularity) of the applications and websites used Sophos Intercept X 736 100%
in this part of the test, applying stricter penalties for Bitdefender Small Office Security 716 97%
when products misclassify very popular software
Webroot SecureAnywhere Endpoint Protection 712.5 97%
and sites.
To understand how we calculate these ratings,
see 6.3 Accuracy Ratings on page 16.
ESET Endpoint Security
FireEye Endpoint Security
Kaspersky Small Office Security
McAfee Endpoint Security
Microsoft Defender Antivirus (enterprise)
Sophos Intercept X
Bitdefender Small Office Security Legitimate Software
Ratings can indicate
Webroot SecureAnywhere Endpoint Protection how well a vendor
has tuned its
0 184 368 552 736 detection engine.
14 Small Business Endpoint Protection April - June 20216.1 Interaction Ratings
It’s crucial that anti-malware Endpoint products
None Click to Allow Click to Allow/Block Click to Block None
not only stop – or at least detect – threats, but that (allowed) (default allow) (no recommendation) (default block) (blocked)
they allow legitimate applications to install and run
Object is Safe 2 1.5 1 A
without misclassifying them as malware. Such an
error is known as a ‘false positive’ (FP). Object is Unknown 2 1 0.5 0 -0.5 B
Object is not Classified 2 0.5 0 -0.5 -1 C
In reality, genuine FPs are quite rare in testing. In our
Object is Suspicious 0.5 0 -0.5 -1 -1.5 D
experience it is unusual for a legitimate application
Object is Unwanted 0 -0.5 -1 -1.5 -2 E
to be classified as ‘malware’. More often it will be
classified as ‘unknown’, ‘suspicious’ or ‘unwanted’ Object is Malicious -2 -2 F
(or terms that mean much the same thing). 1 2 3 4 5
We use a subtle system of rating an Endpoint’s
approach to legitimate objects, which takes into
account how it classifies the application and how
it presents that information to the user. Sometimes
the Endpoint software will pass the buck and interaction ratings
demand that the user decide if the application is None None
safe or not. In such cases the product may make a Product (allowed) (blocked)
recommendation to allow or block. In other cases, ESET Endpoint Security 100 0
the product will make no recommendation, which FireEye Endpoint Security 100 0
is possibly even less helpful. Kaspersky Small Office Security 100 0
McAfee Endpoint Security 100 0
If a product allows an application to install and
Microsoft Defender Antivirus (enterprise) 100 0
run with no user interaction, or with simply a brief
Sophos Intercept X 100 0
notification that the application is likely to be safe,
it has achieved an optimum result. Anything else Bitdefender Small Office Security 99 1
is a Non-Optimal Classification/Action (NOCA). Webroot SecureAnywhere Endpoint Protection 99 1
We think that measuring NOCAs is more useful Products that do not bother users and classify most applications correctly earn more points than
than counting the rarer FPs. those that ask questions and condemn legitimate applications.
15 Small Business Endpoint Protection April - June 20216.2 Prevalence Ratings 6.3 Accuracy Ratings
There is a significant difference between an We calculate legitimate software accuracy
LEGITIMATE SOFTWARE PREVALENCE
Endpoint product blocking a popular application RATING MODIFIERS ratings by multiplying together the interaction
such as the latest version of Microsoft Word and Impact Category Rating Modifier and prevalence ratings for each download
condemning a rare Iranian dating toolbar for and installation:
Very High Impact 5
Internet Explorer 6. One is very popular all over the
High Impact 4
world and its detection as malware (or something Accuracy rating = Interaction rating x Prevalence
Medium Impact 3
less serious but still suspicious) is a big deal. rating
Conversely, the outdated toolbar won’t have had Low Impact 2
a comparably large user base even when it was Very Low Impact 1 If a product allowed one legitimate, Medium impact
new. Detecting this application as malware may be application to install with zero interaction with the
wrong, but it is less impactful in the overall scheme Applications were downloaded and installed user, then its Accuracy rating would be calculated
of things. during the test, but third-party download sites like this:
were avoided and original developers’ URLs
With this in mind, we collected applications of were used where possible. Download sites will Accuracy rating = 2 x 3 = 6
varying popularity and sorted them into five sometimes bundle additional components into
separate categories, as follows: applications’ install files, which may correctly This same calculation is made for each legitimate
cause anti-malware products to flag adware. application/site in the test and the results are
1. Very High Impact We remove adware from the test set because it summed and used to populate the graph and table
2. High Impact is often unclear how desirable this type of code is. shown under 6. Legitimate Software Ratings on
3. Medium Impact page 14.
4. Low Impact The prevalence for each application and URL is
5. Very Low Impact estimated using metrics such as third-party
download sites and the data from Alexa.com’s
Incorrectly handling any legitimate application will global traffic ranking system.
invoke penalties, but classifying Microsoft Word as
malware and blocking it without any way for the
user to override this will bring far greater penalties
than doing the same for an ancient niche toolbar.
In order to calculate these relative penalties, we
assigned each impact category with a rating
modifier, as shown in the table above.
16 Small Business Endpoint Protection April - June 20216.4 Distribution of 7. Conclusions
Impact Categories
Attacks in this test included threats that affect the Products from Kaspersky, McAfee, Microsoft,
Endpoint products that were most accurate in wider public and more closely targeted individuals Sophos and Webroot protected against all of the
handling legitimate objects achieved the highest and organisations. You could say that we tested public and targeted attacks. Bitdefender stopped
ratings. If all objects were of the highest prevalence, the products with ‘public’ malware and full-on all of the targeted attacks but missed one public
the maximum possible rating would be 1,000 (100 hacking attacks. We introduced the threats in a threat, while products from ESET and FireEye
incidents x (2 interaction rating x 5 prevalence realistic way such that threats seen in the wild on stopped all targeted attacks but missed one or two
rating)). websites were downloaded from those same public threats each.
websites, while threats caught spreading through
In this test there was a range of applications with email were delivered to our target systems Most of the products handled the legitimate
different levels of prevalence. The table below as emails. applications correctly, with no mistakes at all.
shows the frequency: Bitdefender’s and Webroot’s blocked one each.
All of the products tested are well-known and
LEGITIMATE SOFTWARE CATEGORY FREQUENCY should do well in this test. While we do ‘create’ The leading products from Kaspersky, McAfee,
Prevalence Rating Frequency threats by using publicly available free hacking Microsoft, Sophos, FireEye, ESET, Bitdefender and
Very High Impact 32 tools, we don’t write unique malware so there is Webroot all win AAA awards.
High Impact 32
no technical reason why any vendor being tested
should do poorly.
Medium Impact 16
Low Impact 12
Consequently, it is not a shock to see most
Very Low Impact 8
products handle the public threats very effectively.
With only a handful of threats missed, all-round
performance was excellent. Targeted attacks were
also handled well by all, although not perfectly.
17 Small Business Endpoint Protection April - June 2021Appendices
APPENDIX A: Terms Used APPENDIX B: FAQs
TERM MEANING
A full methodology for this test is available from our website.
● The products chosen for this test were selected by SE Labs.
The attack succeeded, resulting in malware running
● The test was unsponsored.
unhindered on the target. In the case of a targeted attack,
Compromised ● The test was conducted between 7th April to 10th June 2021.
the attacker was able to take remote control of the
system and carry out a variety of tasks without hindrance. ll products were configured according to each vendor’s recommendations, when
●A
The attack was prevented from making any changes to
such recommendations were provided.
Blocked
the target. alicious URLs and legitimate applications and URLs were independently located
●M
and verified by SE Labs.
When a security product misclassifies a legitimate
False positive application or website as being malicious, it generates a argeted attacks were selected and verified by SE Labs.
●T
‘false positive’. alicious and legitimate data was provided to partner organisations once the test
●M
The exploit or malware payload ran on the target but was
was complete.
Neutralised
subsequently removed. E Labs conducted this endpoint security testing on physical PCs, not virtual machines.
●S
he web browser used in this test was Google Chrome. When testing Microsoft
●T
Complete If a security product removes all significant traces of an
Remediation attack, it has achieved complete remediation. products Chrome was equipped with the Windows Defender Browser Protection
browser extension (https://browserprotection.microsoft.com). We allow other
Target The test system that is protected by a security product.
browser extensions when a tested product requests a user install one or more.
A program or sequence of interactions with the target that
Threat is designed to take some level of unauthorised control of
What is a partner organisation? Can I become one to gain access to the threat
that target.
data used in your tests?
Security vendors provide information to their products in Partner organisations benefit from our consultancy services after a test has
an effort to keep abreast of the latest threats.
Update been run. Partners may gain access to low-level data that can be useful in
These updates may be downloaded in bulk as one or more
files, or requested individually and live over the internet. product improvement initiatives and have permission to use award logos, where
appropriate, for marketing purposes. We do not share data on one partner with other
partners. We do not partner with organisations that do not engage in our testing.
I am a security vendor and you tested my product without permission.
May I access the threat data to verify that your results are accurate?
We are willing to share a certain level of test data with non-partner participants
for free. The intention is to provide sufficient data to demonstrate that the results
are accurate. For more in-depth data suitable for product improvement purposes we
recommend becoming a partner.
18 Small Business Endpoint Protection April - June 2021APPENDIX C: Product Versions
The table below shows the service’s name as it was being marketed at the time of the test.
Product versions
Vendor Product Build Version (start) Build Version (end)
Bitdefender Small Office Security 25.0.14.68 25.0.21.78
ESET Endpoint Security 8.0.2028.0 8.0.2028.0
FireEye Endpoint Security 32.30.13 33.46.0
Kaspersky Small Office Security 21.2.16.590 21.3.10.391
McAfee Endpoint Security 10.7.0.2421 5.7.2.162
Microsoft Defender Antivirus (enterprise) 1.335.850.0 1.341.377.0
Sophos Intercept X 2.0.20 2.18.2
Webroot SecureAnywhere Endpoint Protection 9.0.30.75 9.0.30.75
19 Small Business Endpoint Protection April - June 2021APPENDIX D: Attack Types
The table below shows how each product protected against the different types of attacks used in the test.
ATTACK TYPES
Product Web-Download Targeted Attack Protected
Kaspersky Small Office Security 75 25 100
McAfee Endpoint Security 75 25 100
Microsoft Defender Antivirus (enterprise) 75 25 100
Sophos Intercept X 75 25 100
Webroot SecureAnywhere Endpoint Protection 75 25 100
Bitdefender Small Office Security 75 24 99
FireEye Endpoint Security 74 25 99
ESET Endpoint Security 73 25 98
20 Small Business Endpoint Protection April - June 2021SE Labs Report Disclaimer 1. The information contained in this report is subject to change and revision by SE Labs without notice. 2. SE Labs is under no obligation to update this report at any time. 3. SE Labs believes that the information contained within this report is accurate and reliable at the time of its publication, which can be found at the bottom of the contents page, but SE Labs does not guarantee this in any way. 4. All use of and any reliance on this report, or any information contained within this report, is solely at your own risk. SE Labs shall not be liable or responsible for any loss of profit (whether incurred directly or indirectly), any loss of goodwill or business reputation, any loss of data suffered, pure economic loss, cost of procurement of substitute goods or services, or other intangible loss, or any indirect, incidental, special or consequential loss, costs, damages, charges or expenses or exemplary damages arising his report in any way whatsoever. 5. The contents of this report does not constitute a recommendation, guarantee, endorsement or otherwise of any of the products listed, mentioned or tested. 6. The testing and subsequent results do not guarantee that there are no errors in the products, or that you will achieve the same or similar results. SE Labs does not guarantee in any way that the products will meet your expectations, requirements, specifications or needs. 7. Any trade marks, trade names, logos or images used in this report are the trade marks, trade names, logos or images of their respective owners. 8. The contents of this report are provided on an “AS IS” basis and accordingly SE Labs does not make any express or implied warranty or representation concerning its accuracy or completeness.
You can also read
