May 18, 2009, 10:32 AM — It's safe to say that vulnerability assessment tools have become commonplace within most security teams' toolboxes. As security programs mature, they often begin to look at ways to automate tasks that are mundane and repetitive. [Related: How to Handle Security Patches With Sanity]
These applications have become better at identifying common mistakes within Web applications, patch management, configurations, systems and database hardening.
But with the proliferation of vulnerability assessment products and services, we have begun to create a different problem. [Related: The Failure of Security Investments]
Any organization that maintains a reasonably sized infrastructure or Web presence can easily end up with many different applications, services and tools to maintain and monitor their vulnerabilities. These tools include V.A. scanners to identify security bugs within applications, databases, hosts and networks.
Vulnerability management programs may also employ software-as-a-service (SaaS) solutions to assist in vulnerability identification through both automated tools and manual testing.
Static source code analysis tools add to the internal store of vulnerabilities. Want more data? How about adding the results of your penetration tests?
This vulnerability data may include Web application vulnerabilities -- technical vulnerabilities missed by VA scanners, social engineering exploits through a lack of processes or awareness and logic flaws.
A Mountain of Data
As we begin to find out, in some cases, maturity can bring complexity and more data! But more data is just the tip of the iceberg. How does a CISO connect all of this data? How does management understand what issues and bugs should be prioritized when conducting remediation?
Once prioritized, how do we then migrate these bugs to our bug -- tracking, change -- management and trouble ticketing systems?
Your problem is not only managing the mountain of data you're sitting on, it now includes managing all of this data described in different ways -- managing vulnerability assessment reports that contain overlapping bugs or false positives. Identifying your bugs and problems are no longer the primary issues. You now have to do something about them.
In order to get these vulnerabilities closed, the security teams need to start sorting and moving this data around and getting the appropriate issues in front of management, developers and engineers.
You've taken the step to add more tools to your management arsenal to eliminate the mundane and repeatable tasks only to have your team stuck with enough mundane and repeatable tasks to occupy a small army of security professionals!
So when taking on this problem, I set out with six basic requirements:
- Schema normalization: All vulnerability data needs to be described in the same manner to compare apples to apples across host, network, application and database vulnerabilities.
- Use existing standards (where possible): Don't reinvent the wheel.
- Connect the data: If we don't do this, we're not solving the problem! The primary purpose of our solution is to eliminate these silos of data, reporting and tracking.
- Define our metrics: I thought this might be one of the easier requirements; after all, we already have some vulnerability metrics as part of our current program. Lo and behold, as I drilled deeper into this, I discovered once this data is tied together it gives us additional views and metrics to track and measure our success.
- Useful reporting: Once I have centralized, normalized and correlated this data, I have the ability to crack open the database and sort this any way I see fit.
- Keep it simple: Enough said. [Related: A New Hope for Software Security?]
Enter the Security Content Automation Protocol (SCAP)
What's SCAP, you ask? Until last year I hadn't heard of it either. I was struggling with some development I had taken on myself to help address the problem.
It became a personal project of mine to go out and build something to solve this ever-growing issue.
I was making some progress on a caffeine-fueled weekend development bender when reality hit me in the face.
I've been successful at building some automated connectors to "move" much of the data, but how on earth was I going to describe this data in common terms and normalize the vulnerabilities?
That week I happened to send out a "tweet" on Twitter describing my new, painful reality when a friend and follower of mine, Mike Smith (@rybolov on Twitter), responded with, "You need SCAP!"
I looked it up and found that SCAP is part of the Information Security Automation Program and is made up of a collection of existing standards. These standards include some that many of us are already familiar with, such as the Common Vulnerabilities and Exposures (CVE) and the Common Vulnerability Scoring System (CVSS). Additionally, it includes the Common Platform Enumeration (CPE), a standard to describe a specific hardware, OS and software configuration.
This is helpful for enumerating assets, giving you your baseline information to apply all of this data; the Common Configuration Enumeration (CCE), very similar to CVE but dealing with misconfiguration issues; the Open Vulnerability and Assessment Language (OVAL) to provide schemas that describe the inventory of a computer, the configuration on that computer and a report of what vulnerabilities were found on that computer; and Extensible Configuration Checklist Description Format (XCCDF), a description language to help you apply your technical policies and standards to your scanning tools.
That's a heck of a lot of acronyms. Let's see how this helps me in building a real solution.
As a head of a vulnerability management program as discussed earlier, I am sitting on data from application security assessment tools, host and network scanners, and database vulnerability and configuration scanners.
In reality, this includes multiple products and services for application security, as well as multiple tools for host and network assessments.
I set out by taking advantage of APIs when available from the assessment tool providers as well as XML data feeds. Utilizing the code I've just written to automate the movement of the data, I now need to map this information to a normalized schema, taking advantage of the SCAP standards. This is a big deal!
I now have a common way to describe the vulnerabilities. I can eliminate duplicates that reference the same CVE on the same platforms.
I can score many of these utilizing CVSS, which not only gives me a common scoring formula, it is now being utilized by audit standards such as the PCI DSS, which is very helpful in my world of e-commerce.
Connecting the Dots
Once I have all of my vulnerability information stored in a centralized data store, I can create reporting and metrics that give management a view into our security vulnerability state across all applications, hosts, networks, databases, etc. This centralized and normalized data also gives the CISO and technology management the ability to prioritize security-bug-fixing work.
From there I can now build connectors into my remediation systems, such as bug trackers and trouble ticketing systems, closing the time from identification to remediation dramatically.
In the end I hope to address this heaping data issue giving security teams the ability to once again automate the mundane and repeatable, and at the same time accelerate the "time to close" gap that organizations often suffer from.
I'm not quite there yet, but I'm getting awfully close. Care to help me out?
Ed Bellis is CISO of Orbitz.
