See it, protect it, control it: A roundtable discussion
Network World recently asked blogger Alan Shimel, co-founder and managing partner of The CISO Group (see his blog here), to host a roundtable discussion with representatives from three sectors of the security community: a practitioner, an analyst and a vendor. The wide-ranging conversation touched on everything from the state of threats today to the failure of risk management, the need to share information and a massive attack suffered by the user.
SHIMEL: Hi everyone, this is Alan Shimel, co-founder and managing partner of The CISO Group, and welcome to a Network World Roundtable on Security. The topic of our roundtable is "See it, Protect it, Control it: Advanced Security Intelligence to Outsmart Attackers." We're lucky to be joined by a fantastic group of folks today: Kevin Kerr, the chief information security officer, or CISO, at Oak Ridge National Laboratory, one of the leading research labs in the world; Richard Stiennon, a former Gartner analyst and now the chief analyst at IT-Harvest; and Adam O'Donnell, Sourcefire's chief architect from the company's Cloud Division.
[ FREE DOWNLOAD: 68 great ideas for running a security department ]
GIVEAWAYS: Free security tools out there for the asking
Kevin, you're in the front lines of this war we're waging against cybersecurity attackers, so I'm going to start with you. Have you seen a sea change recently in the kinds of attacks, the kinds of methods attackers are using?
KERR: I think so. They used to knock on your front door or come through the window or over the wall. Nowadays they're relying more on social engineering to try to get someone who's inside the fortress to let them in, whether through phishing or malware or something like that. So they're trying harder to avoid detection in the hopes that they can get one little foothold and, once they're in, then it's fun time for them.
SHIMEL: But the security industry hasn't been sitting on its hands. Adam, how has the industry responded?
O'DONNELL: The game has definitely become far more challenging, not only because attackers now have a profit motive, but because nation-states are involved and willing to break into a system at any cost. In some ways the industry's technologies have become equivalent to a seatbelt, something you absolutely have to have to help be able to protect you, but they're not going to be able to safeguard every situation you get into. In order to address the more challenging threats, both nation-state attackers or from a committed individual or group trying to get into a network, we need to start using technologies that can be modified for your specific environment, something that gives you control over the threats that your specific network is seeing and also gives you visibility into what may have come in in the recent past. [also see: "What is an 'advanced persistent threat,' anyway?"]
SHIMEL: Richard, you're in the catbird's seat here in that you speak to people like Kevin on the one hand, and on the other hand speak to suppliers like Adam. What is your take? Have we passed into a new era of threats that demand a new era of solutions, or are we dealing with a lot of hype here?
STIENNON: Actually, no, I think we're under-hyped. I was just at a meeting where someone voiced the opinion that the security industry is broken, that it doesn't address the new threats. And I had to object and say, no, it was just built for something different. Back in the early days of mainframes, the primary purpose of security was protecting data from users that had access to it and we had pretty good security against targeted attacks. Then we went through an era of random attacks, hackers looking for anything to attack just for the fun of it, which built into a great industry for countering viruses. And then we had attackers using the network and we came up with an industry to counter worms.
But things have changed dramatically in the last three or four years, as targeted attacks recognize the value of certain pieces of information, whether it be a data store of credit card information or design information for the F-35 Joint Strike Fighter. And the attackers, as Kevin pointed out, have realized that the easiest way in is through one of these open doors that's not guarded.
So yes, we're in a completely new realm, but the industry is responding. The cutting-edge vendors I see are starting to be information managers. When they catch an attack against one client they quickly anonymize it and pull it into their cloud so the rest of their customers can look for similar indicators, whether it is as simple as an IP address or the type of malware used or the source domain of the emails. And that's the big difference here. We're starting to recognize these threat actors.
SHIMEL: Regarding the idea that the security industry is broken, that's something I've heard as well. It's this pessimism about not only the security industry but the security profession, almost that we're shoveling sand against the tide here. Kevin, you're out there where the ocean meets the sand, do you feel that sense of pessimism?
KERR: Yes and no. I'm a realist. If someone says they can protect me 100%, they're either ignorant or lying. You need a multitude of things to protect yourselves. When we were attacked last year it was a phish that came in. We got about 750 phishes. We had about 50-some-odd people see them and one person who clicked, and that one system was not running appropriately at the time and the malicious perpetrator got a foothold on their box, was able to grab credentials, and then started to move across our network. So they weren't even using malware when they broke in last year, they were using authorized credentials and a zero day to walk in. And it was because of that we didn't initially see what was going on. It was later detection in the network that led us to the realization that something wasn't right. At that point it was a game of cat and mouse or, as we like to say, Whac-A-Mole, as we tried to keep up with them. When we realized we couldn't keep up tit-for-tat, we decided to disconnect from the network to prevent them from exfiltrating data. [Also see: "Advanced persistent threats force IT to rethink security priorities"]
SHIMEL: Disconnecting from the network certainly isn't a long-term solution. Adam, what can Kevin do short of disconnecting?
O'DONNELL: Depending upon the asset you're trying to protect, disconnecting from the network can be a reasonable solution for a specific instance. If you are protecting the nation's nuclear assets, it makes sense. But obviously not everyone can go about that. If Kevin had tools on hand that allowed him to say, "OK, this attack happened, can I identify every single place this person went, every single system the person touched, and scope the problem?" And then respond within that scope, he might have been able to react without having to take the network offline. Tools that gave visibility into what the attacker did after the attack happened would be critical for that situation.
SHIMEL: OK. That brings up another thing that I'd like to throw at Richard. I've heard this discussed as the positive security model versus the reactive model. But to me it boils down to this nugget: the realization that we may not be able to stop everything that gets in. Part of the security practitioner's role is to understand when something happens, figure out how it happened and prevent it from causing any more harm. Richard, what do you think about that?
STIENNON: A lot people who have been under the sorts of attacks Kevin describes are saying you can't stop everything so our only hope is to detect and get them before they exfiltrate the data. And I've come around to that. It flies in the face of traditional "stop everything, be preventive and not reactive," but it's a new level of reactive. This isn't coming in Monday morning and looking at your IDS logs and going, "Oh, no." This is eyes-on-the-screen-100%-of-the time going, "Whoops, somebody just opened an attachment and infected his machine and a remote-access Trojan has been downloaded and it's starting to scan my network." Or, "Oops, the guy already jumped to the active directory server and is consuming all of my identities, we have to do something now."
And if you haven't caught it by then -- by then it might be all over the place -- it's either shut off the network and cut yourself off or find every little bit and segment of the code left behind. You've got to be able to find them all, shut them down and clean them up before you turn your network back on.
KERR: Let me build on that. With us, once they got in and got some credentials they moved from that box to a server, and then across to another, and as they did they kept gathering credentials, and eventually they got our domain credentials, which at that point is pretty much game over. And because they were moving across our network, they were touching quite a few boxes -- somewhere in the low hundreds -- and they created so many back doors that every time we closed one they opened another. And that's when we realized we couldn't stop this from happening and couldn't stop the data from moving off.
Unfortunately, we didn't have enough IDS, IPS and other monitoring tools to see what they had touched, so it was a risky decision to disconnect from the network. Some of my sister labs were attacked at a later date by similar entities and they decided not to disconnect because they were able to see more of what happened and where they were, and I guess had also learned some lessons from us.
So what you do is obviously dependent on the risk. Today we have a better picture of what's going on in our network. We've re-architected to provide better monitoring, to see better what's going, so we can disconnect 20-30 machines versus 20,000.
SHIMEL: The kind of assets you guys are talking about at Oak Ridge are national and strategic and you can't afford to risk them getting out. So disconnecting as a means of stopping the information from being exfiltrated is certainly viable, not a long-term solution, but faced with what you were faced with, what else could you do? But Kevin, it's not just about having more IDS and IPS, is it? It's having the plan in place about what to do when this happens, and I'm sure part of re-architecting is putting in place procedures and processes in case this sort of thing happens again, right?
KERR: Correct. Just to give it some context, as a CISO I had started at Oak Ridge about two months before this happened. So I was still learning the lab and when this happened I asked for our Incident Response Plan and someone reached up on the shelf, blew all the dust off of it and gave it to me and basically it was how to address a Trojan or a virus on a system. It had nothing to do with how to deal with an advanced persistent threat, and it had nothing to do with how and where to disconnect or anything like that. So there were a lot of lessons learned really quickly about how to react, and unfortunately it was a lot of ad hoc, fly by the seat of the pants stuff.
SHIMEL: Adam, what are companies like Sourcefire doing to help Kevin and those like him in these frontline situations?
O'DONNELL: Sourcefire is very much behind the "See it, Control it" idea. And that means giving visibility into any kind of connection or threat that comes into the network, as well as giving the user the ability to control the threat. We have structured all of our products and technologies along those lines. So if you have an attack that comes in through the network, you would see it on your IDS/IPS. If it gets over to the host side, you would use our host technologies to see what files were introduced, what files those files introduced, and what systems those files talked to.
To address Kevin's issue, we believe that the threats are somewhat unique to each network, so we want to come up with tools that allow specific network operators to address their specific threats. So we make these things configurable and adaptable and definable by each product owner, we give customers full access so they can generate their own rules and signatures because we believe that waiting on a vendor to address a specific threat inside of a network, especially one that's seen by a government entity or a private organization that does not want to share that threat, is essential to allowing our customers control threats.
SHIMEL: Richard, would it help if companies like Sourcefire, Symantec, McAfee and the others shared information about attacks so there would be a global threat response in the cloud? Is the security industry mature enough for that? [also see: "Startup envisions CISO collective to share cyberattack information"]
STIENNON: It's not nearly mature enough to share that information, and the attacks are so targeted it wouldn't be completely effective anyway. I know one defense contractor that, for every attack they shut down, tries to tie that back to indicators they either detected themselves or had pre-knowledge of from the Defense Industrial Base Information Sharing Network, which is easily the most mature. And information from that network only helped them stop about 20% of the attacks, so there's always going to be this need for internal situational awareness.
SHIMEL: OK. Let me throw a question out to all three of you. Are whales -- and I don't mean that in a derogatory way -- are whales like the Department of Defense, like an Oak Ridge National Lab, an exception to the rule that demands customized solutions, or do smaller shops need the same kinds of solutions?
O'DONNELL: I believe that the magnitude of threat is going to be a function of the value of the resources targeted. If it is all the credentials for a popular cloud provider or blueprints for the next F-35 modification or something else that has a monetary value that's hard to quantify, the attackers are going to throw everything they have into it. They are going to come up with custom exploits, they're going to use highly trained individuals and they're going to spend a good bit of time and be patient until they get that data.
That's a very different threat than something you're going to see if you are running a single server with no credit card data. Now, does that mean that you're never going to face the kind of attack that someone like Oak Ridge or a large government entity would face? Absolutely not. But it does mean that if you are an Oak Ridge or a Lockheed Martin or Nasdaq people are going to bring their A game and you need to have really well-trained people, as well as top-notch technologies they can use to respond to the threat.
STIENNON: I agree. There's some security from obscurity if you really have nothing worth stealing. The trouble is you can't conclude that. I'm starting to see people succumbing to attacks where they're just a channel to the real target, say a bank in Australia being attacked when the attackers are ultimately after the bank's mining resource clients. Adam mentioned Nadsdaq. So Nasdaq's Director's Desk website was injected with malware and Nasdaq wasn't the target. It was the users of the Director's Desk.
So that's the trouble with concluding that, "Hey, we don't have anything so we're not going to see this level of attack." And then the response is, there's no way a law office can afford to get the processes in place that a Lockheed or a major research lab should be building. But there will be service providers that will start offering that, and ultimately we're going to see tools that reach all the way down to the home office.
KERR: Going back to the question about the need to share information, I think we definitely need to be interconnected, and I think the cloud's a good place to share information. After all, we're all interconnected in one way or another. We share data with outside entities to process our purchase orders and they send us files and things, so if they get infected that's a back door into us. We've seen various entities have trust relationships between corporations, and that's a tunnel from one to another. One of the first things we did before we came back online was disable every single trust relationship we had with everybody, so 1) we didn't hurt them, because the last thing I want to do is to be blamed for infecting somebody else, and 2) I didn't want anybody else coming into me that I couldn't see through a trust relationship. So we need to be interconnected and share this information in the cloud. I think that's the way we need to be going.
O'DONNELL: We believe that data sharing among our products is essential and everything that we've been announcing with the FireAMP technology, as well as the IDS/IPS technology, is heavily leveraged to sharing data between those two products. Sharing outside of organizations and sharing across different technologies is something that will take many, many years because people are concerned that by identifying threats they've been exposed to, they may be giving additional information back to an attacker. So you need to have technologies that allow people to address a threat without sharing it, but still share the data if they choose to. Otherwise you'll hear the people saying, "Well, if I install this product and it mandates sharing, I can't use the product, because I can't actually tell the public what threat I'm experiencing, but I still need to have some mechanism of combating it."
SHIMEL: Agreed. Now let me switch gears a bit. In the security industry we have a tendency to go for the shiny new trinket and the latest and greatest. Richard, you're a dean of this industry, is that a good thing, a bad thing or a non thing?
STIENNON: Definitely don't go buy the shiny new technology and then figure out how to use it. Start with understanding the threats. Actually a bigger issue is the move away from so-called risk management procedures, which are all based on identifying assets, determining their vulnerabilities and then stack ranking them. You're never going to get that done, right, you'll still be doing that 10 years from now. It is better to start recognizing the threats and then build up defenses against each particular threat. For the most part the tools are there, but a lot of them are from very young companies.
SHIMEL: Fantastic. Kevin, you can only talk about what you're allowed to talk about, but how does an organization like Oak Ridge Laboratory go about evaluating security solutions, how do you look at an up-and-comer versus well-established companies for new kinds of solutions?
KERR: One thing we like is the openness of being able to look at what they're doing behind the curtains. If you come to me with a magic box and says it can do X, Y and Z and you're not willing to show me how it works, you're not going to get much further. I don't need to know all the secret sauce, obviously, but I want to know why it's doing what it's doing and, not only that, if it can be integrated with the tools I have.
I'm going to be the first to admit that we have some wonderful shelfware here that we bought and stuck on our network because I didn't have enough resources or didn't have enough money to buy services, it's never been fully implemented for the capability it offers. We're actually in the process of downsizing some of our tools and trying to end up with two or three that provide a wider swath of visibility into our network, because my objective is to see as much as I can with the tools we have, baseline things, and then allow my experts to be able to drill down based on the wider swath.
SHIMEL: OK guys, we're coming up on the end here. Any final advice to share?
STIENNON: My advice is to throw out your current risk management regime and start over by looking at the three common threat vectors: We've got the hactivists (with Anonymous being one of the most obvious examples), we've got cybercriminals and we've got nation-states. Then strive to understand the methodologies and the targets that each of those will go after, and then look at your current defense regime and see if it's anywhere close to being ready to counter those.
And learn lessons from people like Kevin who have lived through this. Because if you have not seen the types of attack that Kevin has experienced, you're in deep, deep trouble, because you are experiencing them.
SHIMEL: Adam, your advice?
O'DONNELL: Easy. You can't go and address the threats you are facing nowadays by buying a black box from a security vendor and having them say, "Trust me, we got this." You need technologies that give you visibility into the attacks you're seeing right now, as well as visibility into seemingly innocuous behavior. You need to be on both the host and the network, and you need to be able to tie those two data points together. You also need to have a plan in place that lets you control and address some new attack once it comes in. Because as Kevin said, when D-Day comes and you don't have these tools in place, you're not going to be able to respond.
SHIMEL: Fantastic. And last but not least, Kevin?
KERR: You can't just do one thing. Technology helps, but understanding risk and threats is a big piece because you can't go and buy the latest two zillion dollar tool. You're working in a very hostile environment, and you've got to figure out how to detect [the bad guys], contain them and then stop them from getting stuff out. And be willing to reach out for help when the stuff hits the fan. That was one thing we weren't afraid of and I thank management for backing us on this from a security perspective. We reached out to other national labs, we reached out to industry, and within 24 hours we had 30 people on-site from various entities helping us to figure out what was going on.
Read more about wide area network in Network World's Wide Area Network section.