January 30, 2013, 4:18 PM — IBM today rolled out a tool it says can cull massive terabytes of data, including email -- to help customers detect external attacks aimed at stealing sensitive information or insider threats that might reveal corporate secrets.
The tool, called IBM Security Intelligence with Big Data, is built on top of two core IBM products: the IBM enterprise version of open-source Hadoop database with analytics tools known as InfoSphere BigInsights, plus the IBM QRadar security event and information management (SIEM) product that IBM obtained when it acquired Q1 Labs back in 2011.
At its heart, IBM Security Intelligence with Big Data -- IBM thinks 500 terabytes cluster size would be a likely starting point -- would collect and analyze data at high speed data that would include packet-capture data, security-event information from firewalls and other gear, and analyze a stream of content that might include anything from raw email to scrapped SharePoint content, among other business information. The idea is to pull from this voluminous stream the clues that indicate a company is under attack or has been compromised and how.
[ RELATED: Could data scientist be your next job?
IBM's CTO Sandy Bird said the technology is most likely to first be adopted by large companies with data scientists on staff. He acknowledged there's still a lot to be learned about which analytical models and patterns will be the most successful in threat detection. IBM Security Intelligence with Big Data can be theory be applied to cloud-based services, but its starting point is likely to be deployment near the enterprise data center where massive amounts of data are the moist easily accessed for it to work.
The tool is already being deployed in some large corporations and governments. Mark Clancy, chief information security officer at financial firm Depository Trust & Clearing Corporation, said the bank is using IBM's technology to get real-time security awareness. "We need to move from a world where we 'farm' security data and alerts with various prevention and detection tools to a situation where we actively 'hunt' for cyber-attackers in our networks."