May 21, 2003, 10:51 AM — In the world of High Performance Computing (HPC), Beowulf clustering stands in a class of its own.
Beowulf is an approach to building a supercomputer by means of clustering commodity off-the-shelf (COTS) computers that are interconnected with a local area network technology like Ethernet and running programs optimized for parallel processing.
It is possible to assemble a collection of commodity hardware components and freely available software packages in a day and have them executing real-world applications. Traditionally, Beowulf is a class of Pile-of-PCs that leverages on low cost mass-market systems that support Unix-like operating systems at low or no cost and for which source code is readily available.
The benefits of a Beowulf cluster is that it allows even small research firms to build their own supercomputers. In addition to possible cost savings, building their own supercomputers is very much a learning investment in itself for the researchers as well as making them less dependent in the future on particular hardware and software vendors. A Beowulf cluster can upgrade and evolve as off-the-shelf technology evolves.
The development of this technology is due to the development of the causal computing market such as office automation, home computing, games and entertainment which provided system designers with cost-effective components. The COTS industry provides fully assembled subsystems such as microprocessors, motherboards, disks and network interface cards for which mass market competition has driven the prices down and reliability up for these subsystems.
Coupled with the development of standards for interoperability of subsystems which has generated an open market of COTS, it is possible to customize different versions of Beowulf or just maximizing cost advantages. Beowulf developers often choose the Linux operating system and use standard message passing protocols between the computers within the cluster for cost advantages.
This presents an opportunity for parallel computer vendors to provide a low entry-level cost to parallel systems, expanding the role of parallel computing and the number of people capable of using parallel computers, creating a larger customer base for parallel computers in the long term. In the taxonomy of parallel computing, a Beowulf cluster is placed somewhere between a massively parallel processor (MPP) and a network of workstations (NOW) that is clustered for the purpose of load balancing.
The original Beowulf cluster was developed in 1994 at the Center of Excellence in Space Data and Information Sciences (CESDIS), a contractor to the U.S. National Aeronautics and Space Administration (NASA) at the Goddard Space Flight Center in Greenbelt, Maryland. Thomas Sterling and Don Becker built a cluster computer that consisted of 16 Intel DX4 processors connected by channel-bonded 10M bps Ethernet. Their success led to the Beowulf Project, which fosters the development of similar COTS clusters. A number of such clusters have been developed in universities and research groups around the world.
While the original Beowulf cluster was designed to squeeze out additional computing power and life span out of spare equipment to maximize cost advantage, nowadays, while cost is still an advantage, researchers are creating Beowulf clusters out of new hardware devices instead of just spare equipment.
"People in the past were either used to keeping older hardware or they re-cycle hardware. Today, Beowulf clusters are part of any researchers' arsenal of tools, so people buy new hardware to implement Linux Beowulf clusters," explained Lawrence Liew, manager of Singapore Computer Systems' Linux Competency Centre (SCS-LCC). "Treat Beowulf clusters as a cost-effective supercomputing solution. It does not make sense to run on old P2 clusters as your code will probably run faster on a single 2.8GHz P4 then on a cluster of P2s," said Liew. "So people buy latest, faster CPUs they can find to build clusters."
Locally, the Singapore-MIT Alliance (SMA) at the National University of Singapore, together with the SCS-LCC, recently commissioned a new Beowulf cluster based on Intel Itanium 2. The SMA is a science and engineering education and research collaboration between three engineering research universities, namely, National University of Singapore (NUS), Nanyang Technological University (NTU) and Massachusetts Institute of Technology (MIT).
