Complexity of IT systems will be our undoing

By John Dix, Network World |  Networking, Complexity

Other people's research. The Communications of the ACM in '07, for example, found that systems significantly under a million dollars have a better than 75% chance of success. $2 million to $3 million it drops down to 50% to maybe 40% success. $10 million plus it's under 10% success.

OK. So what do we do about it?  

Well, the obvious answer is not to do big systems, do small systems. And to some extent, that's the approach I advocate. You've probably seen the anticomplexity patent we recently got for the Simple Iterative Partitions (SIP) methodology. That patent was interesting because it's the first patent that's ever been granted for a methodology to simplify either business or IT systems, and we actually tackle both since you can't simplify IT unless you simplify the business process on which IT is based. 

As you break a big system down into a number of smaller systems, you reduce the functionality, the complexity, and the cost of those smaller systems. So in theory you're getting your system size down to a reasonable size that yields at least 75% success rate. Unfortunately as you minimize the complexity of the system by breaking it down, you increase the complexity of those system interdependencies.

So, on the one hand, you need to break down big systems into small systems to reduce the complexity. On the other hand, as soon as you do that you increase the intersystem dependencies, which increases the complexity. So you're in a no-win situation. 

Our methodology uses a mathematical approach to finding the best possible balance between those two tradeoffs, that is, between making the system small and minimizing functionality related complexity and keeping the system large and minimizing dependency related complexity.

How do you apply a mathematical model to this kind of stuff? Walk me through the approach.

We are actually already using mathematical models without realizing it. For example, Service-Oriented Architectures (SOA) are mathematically described as partitions, which is part of set theory. But typically we build SOA without understanding how partitions behave mathematically. So we don't use a mathematical approach to find the best possible partitions and then translate those partitions into SOAs. Instead we design our SOAs through decompositional design. Decompositional design is highly arbitrary. It is a process that is mathematically defined as irrational. There are literally trillions of ways of decomposing a problem and the vast majority of these are sub optimal. So many large SOAs end up in the failure bin.

Originally published on Network World |  Click here to read the original story.
Join us:






Ask a Question