Sorting the false positives from the true positives: Ask any safety operations middle skilled, they usually’ll inform you it is one of the vital difficult facets of creating a detection and response program.
As the quantity of threats continues to rise, having an efficient method to measuring and analyzing this sort of efficiency information has turn into extra essential to a company’s detection and response program. On Friday on the Black Hat Asia convention in Singapore, Allyn Stott, senior workers engineer at Airbnb, inspired safety professionals to rethink how they use such metrics of their detection and response applications — a subject he broached final yr’s Black Hat Europe.
“On the finish of that discuss, numerous the suggestions I obtained was, ‘That is nice, however we actually need to understand how we will get higher at metrics,'” Stott tells Darkish Studying. “That is an space the place I’ve seen numerous struggles.”
The Significance of Metrics
Metrics are essential in assessing the effectiveness of a detection and response program as a result of they drive enchancment, scale back the influence of threats, and validate funding by demonstrating how this system lowers threat to the enterprise, Stott says.
“Metrics assist us talk what we do and why individuals ought to care,” Stott says. “That is particularly vital in detection and response as a result of it’s totally obscure from a enterprise perspective.”
Probably the most essential space for delivering efficient metrics is alert quantity: “Each safety operations middle I’ve ever labored in or ever walked foot in, it is their main metric,” Stott says.
Figuring out what number of alerts are coming in is vital however, by itself, remains to be not sufficient, he provides.
“The query is at all times, ‘What number of alerts are we seeing?'” Stott says. “And that does not inform you something. I imply, it tells you what number of alerts the group receives. But it surely does not truly inform you in case your detection and response program is catching extra issues.”
Successfully leveraging metrics could be advanced and labor-intensive, including to the problem of successfully measuring risk information, Stott says. He acknowledges that he has made his share of errors in relation to engineering metrics to evaluate the effectiveness of safety operations.
As an engineer, Stott routinely evaluates the effectiveness of the searches he conducts and the instruments he makes use of, in search of to get correct true- and false-positive charges for detected threats. The problem for him and most safety professionals is connecting that info to the enterprise.
Implementing Frameworks Correctly Is Important
One in all his largest errors was his method in focusing an excessive amount of on the MITRE ATT&CK framework. Whereas Stott says he believes it gives essential particulars on risk actors’ completely different risk methods and actions and organizations ought to use it, that does not imply they need to apply it to all the things.
“Each approach can have 10, 15, 20, or 100 completely different variations,” he says. “And so having 100% protection is sort of a loopy endeavor.”
In addition to MITRE ATT&CK, Stott recommends utilizing the SANS Institute’s Looking Maturity Mannequin (HMM), which helps describe a company’s present threat-hunting functionality and gives a blueprint for enhancing it.
“It provides you the power to, as a metric, say the place you are at so far as your maturity at this time and the way the investments you are planning to make or the tasks you are planning on doing will enhance your maturity,” Stott says.
He additionally recommends utilizing the Safety Institute’s SABRE framework, which gives threat administration and safety efficiency metrics validated with third-party certifications.
“Relatively than take a look at throughout the entire MITRE ATT&CK framework, you are truly engaged on a prioritized checklist of methods, which incorporates utilizing MITRE ATT&CK as a instrument,” he says. “That manner, you are not simply taking a look at your risk intel but in addition at safety incidents and threats that will be essential dangers for the group.”
Use of those tips for metrics requires buy-in from CISOs, because it means gaining organizational adherence to those completely different maturity fashions. Nonetheless, it tends to be pushed by a bottom-up method, the place risk intelligence engineers are the early drivers.
