Independent Scoring of MITRE ATT&CK™ Evaluation Validates FireEye Endpoint Security as the Most Effective EDR Solution

FireEye Endpoint Security delivered the highest efficacy scores, highest number of behavior-based detections, and provided the most relevant context in the 2018 MITRE ATT&CK^TM assessment, announced on Feb. 13, 2019. In this first iteration of MITRE’s Adversarial Tactics, Techniques and Common Knowledge (ATT&CK) framework evaluation of endpoint detection and response (EDR) solutions, MITRE tested 56 ATT&CK techniques using 136 individual procedures. This iteration of the evaluation leveraged a sophisticated adversary emulation of APT3, a China-based threat group that researchers have attributed to China's Ministry of State Security.

MITRE, an independent not-for-profit organization, created a globally-accessible knowledge base of adversary tactics and techniques. The ATT&CK framework is used as a foundation for the development of specific threat models, and methodologies in the private sector, government, and the broader cyber security community.

This is the first open technical evaluation of EDR vendors. FireEye is proud to have participated in MITRE’s public evaluation, strongly believes in the value of transparent assessments and applauds MITRE’s unbiased test methodology.

Scoring Methodology

MITRE’s evaluation is a detailed capability assessment of each solution’s ability to detect and respond to techniques used by APT3. The MITRE assessment does not provide any quantitative scoring of the solutions that were evaluated.

To offer context on how FireEye Endpoint Security solution compared to the other solutions evaluated by MITRE, FireEye used the quantitative scoring criteria published by Josh Zelonis of Forrester.

Using these criteria, FireEye Endpoint Security solution receives a best-in-class showing in all key categories evaluated.

FireEye Has the Highest Efficacy Score Amongst all Vendor Participants

MITRE evaluated 56 ATT&CK techniques with 136 test procedures. There are cases where a vendor can have multiple detections for the same test procedure/step (e.g. Telemetry and General Behavior). Efficacy is a measure of how many detections a vendor had, making sure the results are not skewed for multiple detections for the same procedure/step based on the scoring methodology above.

FireEye has the highest efficacy score of any of the evaluated solutions and the result is a testament to our unique innovation cycle. Our deep understanding of threats is derived from the most robust threat intelligence capabilities in the world combined with insights gathered from over 200,000+ hours yearly of hands-on client-facing attack investigation and response. This unique combination allows for rapid innovation and development of enhanced capabilities in FireEye Endpoint Security.

FireEye Has the Highest Number of Behavior-Based Detections Amongst all Participants

Behavior-based detections identify the exact nature of malicious activity and provides the context that defenders need to fully understand to detect and respond appropriately. They are the most reliable and resilient way of detecting adversary activity.

FireEye identified the highest number of behavior-based detections of any solution taking part in the MITRE ATT&CK evaluation. This was because of FireEye Endpoint Security’s capability to detect the most sophisticated attacks showcasing relevant context, impactful telemetry and the most critical alerts. Developed by the world’s best frontline responders, FireEye Endpoint Security’s response capability offered unparalleled visibility into the threat.

FireEye believes that technology alone cannot solve all security challenges and the best security posture includes technology, intelligence and expertise. FireEye Managed Defense allowed us to showcase the industry’s best detection and response experts, alongside the largest global cyber threat intelligence capability, which harnesses machine, campaign, adversary and victim intelligence gained on the frontlines of the world’s most consequential cyber attacks.

FireEye Has the Highest Number of Enrichment Detections Amongst all Participants

Enrichment detections capture data and enrich it with additional information such as a rule name, labels, tags or ATT&CK tactics or techniques that assist in a user’s analysis of the data, and directly map to the ATT&CK technique under test. FireEye’s enrichment of alerts with relevant ATT&CK techniques provides the most relevant context and additional information that enables effective alert triage and accelerates response. FireEye had enrichment detections for 70 of the 136 tested procedures; the most of any tested vendor.

Detection Differentiators: Only Vendor to Share Full Detection Logic of an Alert, and Highlighting the Ability to Create Custom Security Content

FireEye Endpoint Security was the only vendor to share and showcase the detection logic of an alert. This gives analysts and SOC teams the context to determine the validity of an alert and to understand the reasoning for the detection. The following example from the evaluation highlights our ability to detect file-less attack techniques, showcases our full detection logic and provides the deepest context to our customers.

(Screenshot redacted. Full details available in the product.)

FireEye Endpoint Security allows customers to create and upload their own security content, in addition to the default content in Endpoint Security. To showcase this capability, FireEye leveraged custom ATT&CK-specific security content, alongside production content, during the evaluation. FireEye has released the ATT&CK security content to our FireEye Market. These custom rules can be used to augment detection and as examples for customers interested in creating their own rules.

FireEye Managed Defense: A Force Multiplier

FireEye not only provides robust detection of attacker activity, it also investigates and responds to the activity to reduce the impact of cyber attacks through FireEye Managed Defense. FireEye successfully showcased this force multiplier during the MITRE evaluation. FireEye Managed Defense is a managed detection and response (MDR) service that combines industry-recognized security expertise, FireEye technology and unparalleled knowledge of attackers to identify threats early and help reduce the consequences of a breach. FireEye Managed Defense rapid response capability delivers quick containment of the impact of the threat and provides detailed reporting and analysis on the investigation. The innate focus on delivering real answers to security challenges and being an extension of a customer’s existing security operations is transformational for any enterprise. To effectively defend against today’s sophisticated attacks, organizations need proactive, advanced threat detection and response services.

Day 1 Report (Steps 1-10 in the ATT&CK evaluations)

Day 2 Report (Steps 11-20 in the ATT&CK evaluations)

View the full reports for more details.

MITRE ATT&CK Evaluation Timeline

FireEye confirmed participation in MITRE’s evaluation as part of the rolling admission cohort on Aug. 16 2018. FireEye’s evaluation commenced on Oct. 29, 2018, and finished on Nov. 1, 2018, well before the first cohort results were released on Nov. 30, 2018. FireEye, therefore, had no visibility into any results of competing vendors before our participation.

FireEye is in full support of MITRE’s collaborative, open product evaluation process, how it will help inform organizations about the action adversaries take, and most importantly, how those behaviors affect security efficacy. FireEye’s strong showing demonstrates FireEye’s deep understanding of what it takes to protect our users, and our pedigree of having world-class expertise responding to breaches, the best threat intelligence, and best-in-class product capability.

FireEye Endpoint Security: Finding the Needle in the Haystack

FireEye Endpoint Security provides the most robust endpoint security solution combining FireEye technology, expertise and intelligence, to defend against today’s cyber-attacks. FireEye uses four engines in Endpoint Security to prevent, detect and respond to threats, as well as providing extensive investigative and threat hunting capabilities. To prevent common malware, Endpoint Security uses a signature-based endpoint protection platform (EPP) engine. To find threats for which a signature does not yet exist, MalwareGuard uses machine learning seeded with knowledge from the frontlines of cyber-attacks. Exploit Guard, a behavior-based analytics engine, stops exploits and threats from common attacks like phishing. Endpoint detection and response (EDR) capabilities are enabled through a real-time events engine that uses current, frontline intelligence to identify advanced threats. This defense in depth strategy helps protect enterprises by both preventing and reducing detection time of attacks. Native forensic capabilities and the ability to rapidly search EDR data and operating system artifacts at enterprise scale empower analysts and investigators to efficiently search for compromise, determine the scope of attacks and resolve incidents.

Additional third-party validations as an Approved Business Product from AV-Comparatives, and certification from Virus Bulletin, showcases FireEye Endpoint Security’s leading position in the market, and our commitment to independent third-party testing.

View details from MITRE on FireEye Endpoint Security’s results. Register for our upcoming webinar to learn more about the MITRE results. Additionally, request a free 30-day evaluation of FireEye Endpoint Security, and learn more about FireEye Managed Defense.