September 06, 2006, 12:41 PM — David Geer recently spoke with Gene Kim, CTO of Tripwire and co-founder of the IT Process Institute, an independent research organization that exists to support the membership of IT audit, security, and operations professionals. Following is an edited transcript of that conversation. You may also listen to the original interview here, or visit our Podcast Center for more audio interviews.
Hello, I'm David Geer, and we're talking with Gene Kim about the IT Controls Benchmark survey conducted by Kim and researchers from Carnegie Mellon, Florida State, and the University of Oregon, and published by the Institute. For this survey, 98 respondents from a wide range of company sizes and industries were queried with 97 questions related to IT controls. Forty-three percent of respondents were directors, vice presidents, or C-level managers. The purpose of the survey was to demonstrate how IT organizations can begin to move from good to great in handling IT control issues.
David Geer: In general, why is this survey important?
Gene Kim: This survey is important because I think we're starting to test the medication that we're doling out. In the manufacturing world, there was a big breakthrough in the decision sciences around automotive manufacturing when the lean manufacturing researchers out of MIT basically benchmarked every major automotive manufacturing plant in the world and they found out, wow, high performance exists. They have one-half the floor space, one-half the defects, one-half the inventory, one-half the cycle time, and they've called these the high performers. And they went out and captured and codified what they did. And so what the IT Controls Performance Study is all about is really trying to replicate that same methodology and figure out what is it exactly that the high-performing IT organizations are doing, and figure out more specifically what is it that the medium and low performers aren't doing that's keeping them from being high performers.
Geer: The two top controls in this survey that were most universally present in the high performers and yet virtually absent in everyone else, including 87% of the rest of the respondents, were monitoring systems for unauthorized changes and having defined consequences for intentional unauthorized changes. What surprised you about these results, Gene?
Kim: You just really put your finger on one of the two big surprises that came out of the study, which was that of the 63 controls and the six ITIL processes that we tested, what we were looking for was, is there a smoking gun that says there are a handful of things that management is focusing on in the high performers that none of the medium and low performers are. In other words, is there a small set of things that might be keeping the low and medium performers from becoming high performers?
High performers are doing two things that medium and low performers aren't, which is do you monitor a system for unauthorized change? And the second one is, do you have defined consequences for intentional unauthorized change? So the reason we think this is important is that when you take a look at ITIL process frameworks or COBIT control frameworks, they're full of many good ideas, but you can't do all of them. So if you can't do all of them, where do you start? Where do you have the highest rate of return? Where do you get the biggest bang for the buck? And these are things that these descriptive frameworks don't really give us a lot of guidance to management on.
So what we did in this survey was really tried to create the tool so that management can focus on what is most important -- which controls and behavior and processes really lead to the most improvements in IT effectiveness and IT efficiency.
