Vulnerability Management Automation = Good or Bad and for Whom?
Vulnerability Management Automation = Good or Bad and for Whom?
Do we believe "highly automated security services" are a good thing? Where does automation work and where does it fall short?
The Good:
Scale
Security Automation can deliver thousands of assessments, on-demand and scale to extremely large estates which require vulnerability management on a regular basis.
"Low hanging fruit" can be easily detected but at times Risk can be inaccurate which affects prioritization.
Automation still needs to be tuned such that its production safe and does not negatively affect the asset being assessed.
Automation can be challenging in relation to authenticated assessments and even more so when multi-factor authentication is used by the asset.
Security Automation can deliver thousands of assessments, on-demand and scale to extremely large estates which require vulnerability management on a regular basis.
"Low hanging fruit" can be easily detected but at times Risk can be inaccurate which affects prioritization.
Automation still needs to be tuned such that its production safe and does not negatively affect the asset being assessed.
Automation can be challenging in relation to authenticated assessments and even more so when multi-factor authentication is used by the asset.
Metrics
Frequent or on-demand assessments via automation can assist in the provision of ongoing metrics.
We can measure TTR (Time To Remediation), Identify most common vulnerabilities, Assist with Root-cause analysis to help focus on prevention.
The caveat is that is automation delivers inaccurate results metrics will suffer and prioritization will also be less effective (More on this below).
Visibility
Automation used for asset profiling on a continuous basis is very effective.
Detection of change on an on-going basis delivers visibility, assuming you are profiling an organisations entire estate with no blind-spots.
Asset visibility is still a simple but undervalued aspect of cyber security and vulnerability management; If its not on the "Radar" we don't know about it.
Frequent or on-demand assessments via automation can assist in the provision of ongoing metrics.
We can measure TTR (Time To Remediation), Identify most common vulnerabilities, Assist with Root-cause analysis to help focus on prevention.
The caveat is that is automation delivers inaccurate results metrics will suffer and prioritization will also be less effective (More on this below).
Visibility
Automation used for asset profiling on a continuous basis is very effective.
Detection of change on an on-going basis delivers visibility, assuming you are profiling an organisations entire estate with no blind-spots.
Asset visibility is still a simple but undervalued aspect of cyber security and vulnerability management; If its not on the "Radar" we don't know about it.
The Bad:
Accuracy
Automation is still not very good at delivering accurate results. This can be via false positives, False negatives or Risk Context which does not help with vulnerability prioritization or time wasted in validating issues highlighted by the automated system.
When I say "Risk Context" were talking about the fact that a vulnerability does exist but is it truly exploitable? and what is the business risk of the discovered issue?
Risk Context is core to vulnerability prioritization which affects effort spent which affects focus on what matters and ultimately the effectiveness of a vulnerability management program.
We did a survey over the summer at Information Security Europe where we got 300 people to respond (Thanks!). 60% of respondents said they spend on average over 3 hours per day validating vulnerabilities. Lets consider this for a minute:
On a 7.5 hour day that 40% (3 hours) of a staff members time validating false positives!
On a salary of €50K per year that's €20K per year making sure the tools used by the organisation used to detect vulnerabilities are real!!
- I'm sure the 40% could be better spent training technical staff on preventative measures, secure coding, SDLC security improvement etc?
Automation is still not very good at delivering accurate results. This can be via false positives, False negatives or Risk Context which does not help with vulnerability prioritization or time wasted in validating issues highlighted by the automated system.
When I say "Risk Context" were talking about the fact that a vulnerability does exist but is it truly exploitable? and what is the business risk of the discovered issue?
Risk Context is core to vulnerability prioritization which affects effort spent which affects focus on what matters and ultimately the effectiveness of a vulnerability management program.
We did a survey over the summer at Information Security Europe where we got 300 people to respond (Thanks!). 60% of respondents said they spend on average over 3 hours per day validating vulnerabilities. Lets consider this for a minute:
On a 7.5 hour day that 40% (3 hours) of a staff members time validating false positives!
On a salary of €50K per year that's €20K per year making sure the tools used by the organisation used to detect vulnerabilities are real!!
- I'm sure the 40% could be better spent training technical staff on preventative measures, secure coding, SDLC security improvement etc?
Asymmetric Warfare: Using Automation Alone
Its got to be considered that reliance on tools/automation alone to defend against experienced skilled attackers is a loosing battle. Automation just wont win. Humans are by nature curious and can find the most obscure issue which could result in a vulnerability. Many exploitable vulnerabilities are in relation to issues automation just cant detect very well.
Issues such as Business Logic and Authorization issues are not suited for detection via automation because automation does not have intelligence or is context-aware.
Automation may find *some* issues quickly but humans are capable of detecting and exploiting complex attacks based on breaking a systems logic albeit more slowly.
Human speed is not conducive to keeping pace with software development and its expensive (but less expensive than a breach). We can't rely on humans to defend our systems anymore. Penetration testing/Bugbounty alone wont secure the internet, its too slow (but coverage is deep) and too expensive (consultants and bug bounty's cost $$$).
To keep pace with change we need a combination of both. Technology which augments human expertise, removes the boring & repetitive tasks, provides us with scale but expertise is used when required. (sound familiar? *cough* edgescan *cough*).....
Business logic / Complex logic:
Automation for speed, Humans for depth. We need to combine both to effectively provide adequate assessment coverage. Automation is great a discovering "Technical Issues" but woeful at "Logical" vulnerability detection. Attackers take time to do both and that's why we see a continuous increase in breaches in the news on a daily basis.
So the bottom line is "When using Automation alone to defend against a human adversary the human will always win ", did you ever see the Terminator movies, yes the human folks prevail in the end....
Its got to be considered that reliance on tools/automation alone to defend against experienced skilled attackers is a loosing battle. Automation just wont win. Humans are by nature curious and can find the most obscure issue which could result in a vulnerability. Many exploitable vulnerabilities are in relation to issues automation just cant detect very well.
Issues such as Business Logic and Authorization issues are not suited for detection via automation because automation does not have intelligence or is context-aware.
Automation may find *some* issues quickly but humans are capable of detecting and exploiting complex attacks based on breaking a systems logic albeit more slowly.
Human speed is not conducive to keeping pace with software development and its expensive (but less expensive than a breach). We can't rely on humans to defend our systems anymore. Penetration testing/Bugbounty alone wont secure the internet, its too slow (but coverage is deep) and too expensive (consultants and bug bounty's cost $$$).
To keep pace with change we need a combination of both. Technology which augments human expertise, removes the boring & repetitive tasks, provides us with scale but expertise is used when required. (sound familiar? *cough* edgescan *cough*).....
Business logic / Complex logic:
Automation for speed, Humans for depth. We need to combine both to effectively provide adequate assessment coverage. Automation is great a discovering "Technical Issues" but woeful at "Logical" vulnerability detection. Attackers take time to do both and that's why we see a continuous increase in breaches in the news on a daily basis.
So the bottom line is "When using Automation alone to defend against a human adversary the human will always win ", did you ever see the Terminator movies, yes the human folks prevail in the end....
Comments
Post a Comment