Building roads versus building skyscrapers

June 29, 2008 —

What do you do for a living? If you answered "I am a programmer", or , "I am an application designer" or "I am an IT architect" or "I am an engineer" or "I am a tester" I would reply "Me too."

How odd. How can I be all of those things? You are thinking the same thing I suspect. Maybe you do all of the above too - and more? How odd!

Imagine a similar conversation in the road construction industry. I ask "Hello, what do you do?" and you reply "I am the architect, and the designer, oh and I decide on all the materials and um, I build chunks of the road myself and then do all the testing." How distinctly odd that would be.

I have been noodling the whys and wherefores of this. It would be easy to discount the whole thing as a simple consequence of the relative youth of computing. One day, perhaps, building software will be broken down along the same lines as any other industrial scale engineering project. Designers will design and architects will architect and builders will build and it will all be happily, cleanly segmented.

I don't think so. At least not if continue on our current trajectory. Here is how I have tried to explain the core of the problem to myself . . .

Imagine building a road. In the physical world I can pour concrete and put up fences and lay asphalt bit by bit, mile by mile. As long as there is an overall plan in place, it's all good. This mile of road is very much the same as the next mile of road. All we need to do is make sure they are all joined up.

In the software world however, I cannot write functions and create modules and design screens bit by bit, mile-of-code by mile-of-code. Every line, every dialog box every data item, needs to be considered in terms of the overall project. Otherwise, there is no way I will end up with something that scales, that is easily modified, that has 5-nines availability etc. etc. There is an uncomfortably high degree of design smarts and architecture smarts required all the way down to the code cutting itself.

Coding is coding but coding done well is also design and also architecture and also fault tolerance and also testing . . .

How odd.

I think I have a better metaphor. Also from the construction world. Software design is like building a skyscraper. Every bolt, every block needs to be put in place from the bottom up with a view to withstanding load that does not exist yet. Every part is built on every other part. Dependencies exist all the way down to the ground. You cannot just knock down a wall or build on extra rooms as it suits you. You need to wonder what impact every bit of the skyscraper has on every other bit. There are dependencies from top to bottom.

In the construction industry, the main enemy is gravity. Gravity comes already load-balanced for you by mother nature. If you build sideways load (gravity) is distributed. However, if you build upwards it is not distributed at all. In the vertical, you need to give gravity your full attention always. That is fundamentally why motorway building is much more likely to be a "loosely coupled", decomposable activity than skyscraper building.

The software application equivalent of gravity is users. At Internet scale, the load on your application is like a mile high skyscraper - and your application is the foundations and the bottom floor. Every nail, every joint needs to be "programmed" to support the load above.

Software development happens in the foundations and on the first floor of the skyscraper. Even the humbler coders in the corner have to be thinking "one-mile-high". Otherwise your application is dust.

I don't see this changing until such time as coding has built-in insurances for all the "ilities" that distinguish an outhouse from a mile-high sky scraper. We need to find a way to cut programmers loose without them (or you) having to worry about all the "ilities" - without which the code will not stand.

We have a long way to go. Till then, the good programmers will be - unavoidably - part programmer, part designer, part architect, part tester.