MITRE's Center for Threat Informed Defense (CTID) members discuss Summiting the Pyramid project, analyzing adversary behaviors in cyber, trade-offs in attack analysis, making analytics robust, evasion and false positives in detection, balance between precision and recall, and encouraging user feedback.
The Summiting the Pyramid project by MITRE's CTID provides a unified method of grading detection efficacy, allowing for improved detection programs in organizations.
Comprehending adversary behaviors is crucial for enhancing detection practices and building stronger defenses.
Going beyond surface-level detections and focusing on behaviorally-focused analytics can improve detection effectiveness and the understanding of different levels of the pyramid of pain and attack framework.
Deep dives
Analyzing cyber defenses and creating robust detections
The podcast episode focuses on creating robust and well-thought-out detections. The guests, Steve and Roman from MITRE's Center for Threat and Form Defense, discuss the summoning the pyramid project and its goal of improving understanding and implementation of robust analytics. The project breaks down analytics into different components, such as observables and telemetry sources, to evaluate their robustness. They use a 2D model to map the levels of analytics and event robustness, allowing for a more precise scoring and categorization of analytic effectiveness. By considering factors like data sources, behavior, and sensor robustness, defenders can develop more effective and resilient detection strategies.
The importance of understanding adversary behavior
The podcast emphasizes the significance of comprehending adversary behaviors and utilizing that knowledge to enhance detection practices. Steve and Roman highlight the value of focusing defensive resources on techniques and behaviors that have proven to be persistent and consistent across different adversaries. By targeting the fundamental behaviors that adversaries employ, defenders can build stronger defenses and improve their return on investment in terms of detection and hunting.
Analyzing the pyramid of pain and attack framework
The podcast delves into the pyramid of pain and attack framework, exploring its strengths and limitations. It discusses the need to go beyond surface-level detections based on tools, hashes, and IP addresses to more behaviorally focused analytics. The project's focus on understanding the different levels of the pyramid, such as tooling, behaviors, and techniques, helps defenders make informed decisions about detection effectiveness. Additionally, it highlights the importance of data sources and event robustness in enhancing detection capabilities.
The importance of analyzing file play
Analyzing the manner in which a file came into being can provide valuable insights into file play. Adversaries often invoke existing functionality and rely on registry keys or files to carry out their actions. By understanding these follow-on effects and leveraging built-in functions, defenders can develop a better approach to detect and mitigate file play.
Balancing recall and precision in analytics
Recall and precision are two common measurements in analytics. Recall refers to the ability to detect all the bad things, while precision measures how accurately all detected instances are bad. Increasing recall can lead to more false positives, impacting precision. Defenders can utilize a combination of analytic approaches to optimize both recall and precision, such as first identifying all instances and then applying an analytic to detect specific variations of interest.
The Summiting the Pyramid project by MITRE's Center for Threat Informed Defense (CTID) released in September 2023, allowing for a unified method of "grading" detections for efficacy. Two of the pivotal members of that project join Jared and Luke to talk about how it came to be, and how it can be used to further the detection program of any organization!