June 09, 2011, 12:59 PM — Cloud computing is famously scalable – but only horizontally.
IBM is trying to change that with new cloud-systems software designed to connect high-performance-computing resources to make them accessible through a single interface in the same way horizontally-scaled systems are with typical cloud software.
It carries a name with Cloud in it, but the software itself looks a lot more like cluster controls than what most people mean by "cloud."
You don't hear as much about HPC as you did a few years ago, when the only efficient way to get a lot of compute power in the same place was to scale vertically – add enough processing power, memory and bandwidth to a single machine that could do the work of many.
With virtual-server and cloud software, it's now more efficient and less expensive to keep letting many servers do the work of many, but make them all divvy up the workloads and resources neatly. The end result was similar to what customers could get from specialized clusters, grids or even HPC systems, at much lower cost.
As closely as clusters or clouds can mimic some of the performance of HPC systems, they can't do it all.
The Air Force custom-built a graphics-analysis supercomputer out of almost 2,000 PlayStations because commodity level general-purpose servers couldn't crunch the visual data quickly enough.
Engineering apps, mining or geographic modeling software, some scientific software and other apps too resource-intensive to run well on standard servers – even clustered – still force many companies to by HPC systems.
IBM's HPC Management Suite for Cloud is designed to at least let those companies avoid wasting expensive HPC resources, by combining them into a single cloud system in the same way horizontally scaled servers are, but with much higher processing potential.
It's not virtualization, though. Running a hypervisor – which amounts to an additional operating system, though a small one – adds computational overhead HPC apps can't afford.
So IBM's approach is to make workload management much more aggressive and much more sophisticated than previously. It allows HPC server farms to split up computational tasks and distribute them among all the farms, to run on the servers with the most capacity.
By that description, it isn't really cloud software at all.
It's cluster-management software that's able to reach out to more pools of system resources than typical load balancers.