Illusions of chaos, illusions of calm
As luck would have it, I have spent the last few months oscillating between a number of development projects using wildly different technology stacks. Keeping all the details of any one development project in your head is hard. Keeping the details of multiple projects and switching between them rapidly is very hard. If the various projects are on wildly different technology stacks, the problem is even harder. I'm feeling a bit frazzled as a result and looking forward to a few days off around the Christmas break.
My particular scenario involves a commercial software stack in one project and an open source stack in another. It has been interesting to stand back a little and compare the dynamics of the two projects. The area that has been most striking is the deep equivalencies in the complexities in both environments.
I say 'deep' because on the surface open source looks more chaotic than commercial software which appears calm and ordered by comparison. On the face of it, open source software is an intensely chaotic primordial soup. A constantly evolving mish-mash of technical brilliance co-existing with the not-so-brilliant; of ego clashes and flame wars. High drama executed 24x7, fully visible to anyone who wants to look. Like all primordial soups, a quick glance leaves you with the impression that chaos truly reigns.
On the face of it, closed-source software is much more ordered. Applications proceed in an orderly fashion, progressing from concept through to implementation. Applications have nice tidy version numbers that proceed from 1 upwards or perhaps by year, upwards. Ordered, safe, predictable. The complete opposite of chaos.
I won't go in to the reasons why the latter simply isn't the way the world actually works but trust me, the world doesn't work that way. Here I would like to concentrate on one of the reasons why closed-source software would appear to progress in a calmer fashion than open source software.
The key word I think is 'brand'. A brand is a device used to great effect to create 'sameness' in software. The industry is overflowing with examples of applications that are all branded as 'Application X' but in reality, are totally different depending on the year you are talking about, the version you are talking about, and so on.
The reasoning behind brands is clear and obvious and sound and I have nothing against them whatsoever. However, at an engineering level, they are largely meaningless. The file formats between two versions of Application X may be totally different. The APIs totally different. In extreme cases (of which there are quite a few), every single line of code between two versions of Application X are different. Every developer has had experience of a conversation that goes something like this 'we have Application X, you have Application X what could possibly go wrong'? Like I said, the world does not work like that.
In the open source world, brand control is less pronounced. When the time comes for a new version of an application to be radically different from a previous version, the option typically exists for the new, radically different application to carve out its own identity as a brand new application in cyberspace. Even if it stays within the fold as it where, the open nature of the discussions will make the differences between versions explicit rather than implicit. You won't need to look very hard to see the differences and appreciate their implications at an engineering level.
In open source, the apparent chaos is on the surface. In closed source, the apparent chaos is still there. It is just a level further down. You need to scratch the surface a bit harder in order to see it. In both cases, the chaos is an illusion. I don't think we really know how this works but both models can and do produce great software. Is this because both models work or because, at some level not yet fully understood, the models are really the same? I suspect the latter.