Gold Nuggeting: A Critical Step in Vulnerability Remediation Prioritization in 2020
January 6, 2020
Focusing on a vulnerability report example, this blog post introduces the concept of outstanding network asset detection, or what we call Gold Nuggeting, a critical step in vulnerability remediation prioritization.
Finding interesting network devices is a fundamental part of the vulnerability prioritization process employed by Delve. While many different factors are taken into account when making predictions about the importance of a given vulnerability (see “Prioritization is for Everyone”), the evaluation of the underlying asset’s appeal to a potential adversary is a critical step. Yet, this is a non-trivial problem.
Using our expertise in AI combined with our experience in pen testing, Delve tackled this challenge with the help of unsupervised machine learning methods, the details of which can be found in our latest white paper, Automating Intuition: Applying Machine Learning to Outstanding Network Asset Detection.
First, some background from the world of penetration testing. Enterprise networks house thousands of devices (IoT devices, servers, laptops, etc.), some of which present particularly ripe targets for bad actors. Over the course of years of experience, the best pen testers can very quickly find these priority assets, the ones best suited to compromise or to collect valuable information, and from which to launch a successful attack.
When first evaluating the results of network scans, typically an Nmap, a prime vulnerability report example. The experienced pen tester - or intruder - will quickly get a “feel” for the type of network he is currently in. During this critical early stage of an attack, the intruder is attempting to understand the relationship between the network devices he is looking at, or the underlying structure in the data, all of which is highly context-dependent. It is only when he understands the overall context of the network that he can start “digging for gold.”
Understanding context is a very human and intuitive thing to do. The context of a network device includes many factors, for example, the relationship between other devices in the asset’s surroundings, the actual or typical use of a workstation or a server, the surface of attack of the network, the business line in which this asset is used, etc.. So, our goal is to find an automated method to filter out interesting and outstanding network assets given their context.
Unsupervised learning is a class of machine learning algorithms that specifically use the underlying structure of the data to solve a task. This is in contrast to supervised learning, where the model is trained to optimize on the prediction given a supplied label for each example, say tagged faces in photos or pictures of cats.
Anomaly detection is an archetypal unsupervised learning task. By encoding each network device with it’s interesting characteristics and properties into a numerical representation, we are able to run efficient anomaly detection models in order to sort out the outstanding assets, or Gold Nuggets, from each network, given the specific context of the asset.
Gold Nuggeting is just one part of the Contextual Prioritization process (see "Introduction to Contextual Prioritization," but the power of the idea seemed so interesting to us that we decided to share the process used internally into a free, open source, stand-alone tool called Batea. Given the simple XML output of an Nmap scan, Batea allows you to filter the Gold Nuggets in your own network, helping you prioritize vulnerability remediation efforts, for example. For a more technical dive into the workings of Batea, download our previously-mentioned white paper: Automating Intuition: Applying Machine Learning to Outstanding Network Asset Detection.
To try Batea right now using our trained model: https://delvesecurity.com/batea/
To train your own model or to contribute to the project: https://github.com/delvelabs/batea
How Does Identifying Outlier Assets Reduce Vulnerability Risk?
As discussed previously in this blog, experienced pen testers (and their cyber criminal counterparts) look to identify assets on a network that stand out in some way. Why? Because experienced pen testers know that these assets are often the softest targets, the easiest to compromise. Thus, understanding which of your own assets are the juiciest targets is like a political operative doing opposition research on their own candidate so they are prepared for attacks opponents will launch on the candidate’s weaknesses. Another way to put it is obtaining a copy of the opposing team’s playbook. Once you know what their game plan is - how they intend to exploit your weaknesses - you can work to address those weaknesses.
Thus, if you know which assets are most attractive to an attacker, you can prioritize the remediation of vulnerabilities on those assets, shoring up the defenses of the very assets that are most likely to be the first attack candidates on the network. By doing so, you’ve effectively reduced your organization’s vulnerability risk. Prioritizing vulnerability remediation is not just one objective of a vulnerability management program, it’s THE primary objective. So a key element of vulnerability remediation solutions is prioritizing the remediation of vulnerabilities most likely to expose the organization to attack, and one of the most critical ways to do so is remediating vulnerabilities on outlier assets, or the ones most likely to draw the attention of bad actors.
How do You Prioritize Vulnerability Remediation?
The key to effectively and efficiently prioritizing vulnerability remediation is to account for as many variables as possible, and therefore paint a picture of the vulnerability in its context on the network. There was a time when security and IT practitioners relied on a vulnerability CVSS score to aid in their vulnerability triage efforts, largely because it was the only vulnerability metric available. Most knew the CVSS score was inadequate, and specifically because it was fixed. The CVSS score of a given vulnerability is a constant, irrespective of the network it is on or the specific location on the network. This makes the CVSS score a starting point, at a minimum, not a vulnerability remediation solution.
Of late, a more sophisticated technology has been employed to help prioritize vulnerability remediation. Specifically, predictive exploitability has been marketed as a panacea in the vulnerability remediation community, and there’s no question that it can help. The problem is that it’s been pitched as a cure-all. Predictive exploitability is the process of evaluating a newly published vulnerability and using machine learning techniques to predict whether or not an exploit is likely to be published for that vulnerability at some point in the future. This is, without question, a good piece of information to have, but on its own, it’s flawed in many respects. First, the predictive exploitability score is delivered as a probability. So, if you believe a vulnerability has a 99% chance of having an exploit published for it, you would likely prioritize that vulnerability. If the vulnerability is 1% likely to have an exploit published, you would de-prioritize it. But how about 85%? 64%? 75%? What is the cut-off point at which you can worry about that vulnerability at a later time? Also, let’s assume that you choose 90% as your threshold, and that 95% of your network vulnerabilities score less than 90 on the exploit prediction analysis, and you feel comfortable moving those 95% of your vulnerabilities to the back-burner. You’re still left with 5%, which doesn’t sound like many, but with many networks hosting hundreds of thousands or even millions of vulnerabilities, that 5% can still represent thousands of vulnerabilities that are highly likely to be exploited. In that case, where do you start? Which of the 5% is the most dangerous, poses the highest risk to the organization.
What is the Best Vulnerability Prioritizing Solution?
It’s rare that a complex question can be answered with one word, but in this case, it’s possible. The word? Context. Prioritizing vulnerability remediation tasks has to be based on a comprehensive evaluation of each vulnerability’s context. The vulnerability itself, the asset on which the vulnerability resides, the network environment in which the asset resides, the importance of the asset to the organization, and external threat factors like the likelihood the vulnerability will have an exploit published for it. Each of the 5 categories should take into account multiple factors, with the total exceeding 3 dozen or more.
Moreover, since the network environment changes so frequently, this comprehensive, three hundred and sixty degree view of a vulnerability’s context should be recalculated on a regular basis. Clearly, to achieve this, an automated system must be implemented. Only this kind of meaningful, context-based prioritization will give an organization the confidence in the resulting remediation priorities to impact operations, and ultimately maximize vulnerability risk reduction for a given level of remediation resource expenditure.
Thus, when it comes to vulnerability remediation solutions, only those that include a context-based prioritization element will provide a meaningful outcome.
What is the Key to an Effective Vulnerability Remediation Solution?
Any fans of the classic sitcom MASH know the meaning of the word - and the importance of - triage. In the show, the medical would periodically be inundated with wounded soldiers, far too many to treat simultaneously. Thus, the first step in the process was to decide which soldiers were hurt so badly they needed to be treated immediately, while some, although serious, could wait. Under normal circumstances, the ones “that could wait” would be treated immediately, but in that situation, everything is relative.
Vulnerability remediation solutions are similar. Most organizations have vulnerabilities on their networks far too numerous to remediate immediately, so the IT operations and security teams must decide which ones can wait, and which pose the highest risk to the organization. Just like in the MASH analogy, there may be some serious vulnerabilities that have to be put on hold because they’re not AS serious as some others, and in a perfect world they’d all be addressed quickly. As IT and security professionals know all too well, however, there’s no such thing as a perfect world.
Thus, the key to an effective vulnerability remediation solution is a sophisticated, credible, comprehensive vulnerability prioritization solution that the entire organization trusts to guide it to the correct remediation plan, the one that will maximize vulnerability risk reduction.