IBM pitches HPC load balancing as high-performance cloud

Cloud computing is famously scalable – but only horizontally.

IBM is trying to change that with new cloud-systems software designed to connect high-performance-computing resources to make them accessible through a single interface in the same way horizontally-scaled systems are with typical cloud software.

It carries a name with Cloud in it, but the software itself looks a lot more like cluster controls than what most people mean by "cloud."

You don't hear as much about HPC as you did a few years ago, when the only efficient way to get a lot of compute power in the same place was to scale vertically – add enough processing power, memory and bandwidth to a single machine that could do the work of many.

With virtual-server and cloud software, it's now more efficient and less expensive to keep letting many servers do the work of many, but make them all divvy up the workloads and resources neatly. The end result was similar to what customers could get from specialized clusters, grids or even HPC systems, at much lower cost.

As closely as clusters or clouds can mimic some of the performance of HPC systems, they can't do it all.

The Air Force custom-built a graphics-analysis supercomputer out of almost 2,000 PlayStations because commodity level general-purpose servers couldn't crunch the visual data quickly enough.

Engineering apps, mining or geographic modeling software, some scientific software and other apps too resource-intensive to run well on standard servers – even clustered – still force many companies to by HPC systems.

IBM's HPC Management Suite for Cloud is designed to at least let those companies avoid wasting expensive HPC resources, by combining them into a single cloud system in the same way horizontally scaled servers are, but with much higher processing potential.

It's not virtualization, though. Running a hypervisor – which amounts to an additional operating system, though a small one – adds computational overhead HPC apps can't afford.

So IBM's approach is to make workload management much more aggressive and much more sophisticated than previously. It allows HPC server farms to split up computational tasks and distribute them among all the farms, to run on the servers with the most capacity.

By that description, it isn't really cloud software at all.

It's cluster-management software that's able to reach out to more pools of system resources than typical load balancers.

IBM isn't offering many details on how it works, but does say it developed the approach to create a super-cluster to be used by more than 3,000 IBM engineers working on the design of its latest-generation of Power7 processors.

It won't ship until the third quarter of this year, at a price IBM hasn't announced. The Register reports sources who estimated it would cost about $700 per node.

Sounds like exactly the kind of thing a very narrow slice of resource-hungry apps need – a market that will get smaller and smaller as more typical cloud and virtualization apps get more efficient at distributing and allocating their own resources.