How cloud can give back to open source

Debian is using cloud resources, why not other FLOSS projects?

There's been a lot of talk about open source and the cloud, which makes sense given the proliferation of open source technologies that actually comprise cloud computing.

But what about open source in the cloud? Can cloud computing and storage provide open source projects less expensive access to computing resources?

The question seems to be answered by a revelation today from Debian GNU/Linux developer Lucas Nussbaum, who has outlined how Debian archive services are now completely hosted on Amazon Web Services.

Nussbaum, who is an assistant professor at Université de Lorraine, is involved in the Debian QA and Release Management team and responsible for rebuilding every Debian package from source in order to "identify packages that do not build anymore due to changes in other packages (compilers, interpreters, libraries). It is also a good way to stress-test all packages that are involved in building other packages," Nussbaum .

The use of cloud computing to perform these rebuilding tasks is not exactly new… Nussbaum is also a systems engineer on the Grid'5000 high-performance computing research platform and had been using Grid'5000 to rebuild and test packages since 2007.

But in late 2011, Nussbaum reported, Amazon offered the Debian Project a grant to host Debian's QA services on AWS. For the tech minded like me, here's the specs:

"On the technical level, 50 to 100 EC2 spot instances are started, and then controlled from a master instance using SSH. On build instances, a classic sbuild setup is used. Logs are retrieved to the master node after rebuilds, and build instances are simply shut down when there are no more tasks to process. Several tasks are processed simultaneously on each instance, and when they fail, they are retried again with no other concurrent build on the same instance, to eliminate random failures caused by load or timing issues. All the scripts are designed to support other kind of QA tests, not just rebuilds."

This tweaked my attention, if only because services like AWS and other public and private clouds could be a real boon to open source projects who always seem to be hunting around for computing resources.

This doesn't affect distributions like Red Hat and SUSE Linux, , of course; nor their community-run counterparts Fedora and openSUSE; or Canonical's Ubuntu distribution--the commercial interests behind these distros and any projects they might be running can afford their own infrastructures.

But for projects like Linux Mint or Mandriva, which are popular yet without commercial sponsorship (or in the case of the latter, their commercial sponsorship is weak), I can see cloud services offering a powerful alternative for storage and computational needs. After all, if cloud can help the latest super-neato-Silicon Valley startup get on its feet, why not a struggling open source project? That cloud computing fits perfectly with the distributed development model that's so pervasive in FLOSS projects is just icing on the cake.

Of course, there's the very real matter of costs to take into account. Debian's QA team received a grant from Amazon, which is cool, but not every fledgling open source project has that advantage. Perhaps infrastructure as a service providers like Amazon, Citrix, and Rackspace could donate more space to qualifying projects.

Or perhaps one of the foundations that permeate the open source community can obtain a discount rate of services for participating members. Not to put my former employers on the spot, but I could see the Linux Foundation doing something like this, because if Linux needs apps, then what better way to help get apps than providing cheap access to testing servers in a public cloud?

This is an important mission to undertake. Even though the costs of computing resources are far less than that of human time and effort, getting FLOSS projects access to more computing resources can only help them flourish.

Read more of Brian Proffitt's Zettatag and Open for Discussion blogs and follow the latest IT news at ITworld. Drop Brian a line or follow Brian on Twitter at @TheTechScribe. For the latest IT news, analysis and how-tos, follow ITworld on Twitter and Facebook.

Insider: How the basic tech behind the Internet works
Join the discussion
Be the first to comment on this article. Our Commenting Policies