How to use Facebook's open sourced data center design to cut costs

Facebook's state of the art data center in Oregon uses 38% less power and costs 24% less to run that its older data centers. These figures are astounding, and they should certainly make any cost-conscious CIO sit up and take notice.

What's the secret behind these savings? The company honored its hacker roots by custom-designing both the data center itself, and the servers (and management tools) inside it, from the ground up. It's akin to what Google has been doing for the past 10 years or so, but the good news is that--unlike Google--Facebook has not kept what it has achieved and how it has achieved it a secret at all.

Slideshow: 8 Data Center Lessons From Facebook

In fact the reverse is true: following the open-source software model, Facebook turned its server, server management, storage, rack, electrical, cooling and overall data center designs over to the Open Compute Project (OCP) Foundation, a not-for-profit with some heavyweight names on its board including Intel, Rackspace, Goldman Sachs and Arista Networks as well as Facebook. The OCP Foundation has formalized Facebook's designs into OCP specifications, and these specifications are freely available and anyone can contribute improvements to them.

Pick a Vendor, Any Vendor

One of the key ideas behind the development of the OCP specifications has been to avoid the "gratuitous differentiation" between different vendors' products, according to Frank Frankovsky, Facebook's vice president of hardware design and supply chain, and founding board member of the OCP. (The phrase "gratuitous differentiation appears to be a mantra used by everyone involved with the OCP, and Frankovsky is no exception.)

"Buying off-the-shelf servers, for example, is not economical as they include features that most organizations don't need," says Frankovsky. "That leads to extra costs and wasted electricity, and there are issues like it not being possible to manage HP boxes in the same way as Dell boxes: proprietary management wreaks havoc in environments like ours." Instead of being supplied by established hardware vendors like Dell or HP, OCP servers are often supplied directly by manufacturers in the Far East who are willing to supply them at wafer-thin margins.

The overall theme of the OCP then is to slim the server hardware down to what's actually needed to carry out the sort of large-scale computing that Facebook is involved in, and run it off a highly efficient electrical system with a minimum amount of power transformations. Further power and cost savings are achieved by housing it all in a data center with a high efficiency cooling system.

Facebook has proven that its designs work in its purpose-built data center in Oregon, but the vast majority of enterprises have far smaller computing environments and are unlikely to embark on a new data center build. So the question you're probably asking is this: can regular small and medium sized businesses, or even large enterprises that don't carry out computing on the same scale as Facebook, benefit from OCP designs? And if so, how?

"We certainly expected initially that our designs would appeal only to companies doing large scale computing like us, or companies like banks that have huge IT operations," says Frankovsky. "But it turns out that there is almost no limit to who can benefit. I would argue that the OCP is applicable to everyone, and who wouldn't want to be more efficient and eliminate gratuitous differentiation?"

The reason that anyone can benefit is that you don't have to build a new data center to achieve some of the cost savings: You can pick and choose which OCP specifications to adopt. That means that even if you are using colocation facilities you could put in OCP servers and power supplies and make significant power and cost saving.

"We achieved a 38% reduction in energy usage in a newly built data center, but putting OCP servers in a colocation you could expect energy savings of at least half that," says Frankovsky. The news is even better when it comes to overall cost savings: Frankovsky says he believes that even if you are responsible for a fairly small IT environment, you could still match Facebook's 24% cost reduction. That's because while Facebook was already getting heavy discounts on its hardware due to its scale, you probably aren't. That means the cost savings from switching to cheaper OCP gear--minus the gratuitous differentiation--will be all the more significant.

Smaller Enterprise Benefit From OCP

But Digital Realty Trust CTO Jim Smith says that its existing cooling systems are already very efficient, even if they can't quite match OCP standards. "Smaller customers can pick their own OCP servers, and we can offer them between 75% and 85% of the benefits that Facebook gets at its data center," he says.

Smith says that typical customers for Digital Reality's OCP environments will be small-to-midsized companies or large, non-technology intensive companies that require between 1 and 3 megawatts of capacity. So far the company is in talks with potential customers, but has yet to sign any up.

One reason for this may be that companies are not yet sure that "open source" hardware design will really work, and prefer to take the safe route of buying from one of the established big name vendors, Frankovsky says. But he expects that will change quickly. "Think of Linux in the early years, and how unsure people were of the risks," he said "But think of the speed of development in the open source software market, and then envision that in hardware."

To help you take advantage of OCP, there is a growing list of "OCP solution providers" such as Avnet, ZT Systems and Hyve Solutions (owned by Synnex, a business process services company), while QCT (owned by Quanta, a Taiwanese original design manufacturer (ODM) which makes many of Dell's servers) and Wiwynn (owned by Wistron, another Taiwanese ODM) have been set up to sell OCP hardware direct to end users.

HP and Dell have also announced new, server designs (codenamed "Coyote" --based on HP's ProLiant platform and Intel's Xeon E5-series processors--and "Zeus," respectively) which will be compatible with OCP's Open Rack specification, and VMware has announced that it will certify its vSphere virtualization platform to run on OCP iron.

OCP solution provider Hyve Solutions takes standard OCP servers, adds cabling, drives and networking gear, and then ships them out as OCP racks. Steve Ichinaga, the company's general manager, says Hyve has seen huge interest and significant sales volumes of OCP servers ever since Facebook announced the savings it had made by specifying them.

"Prior to this people bought what was available--maybe Dell or HP servers. But we have seen a flood in terms of big names coming to us looking for servers that are less expensive and consumed less power, but which can be customized," said Ichinaga.

Storage, Servers and Beyond

But some day will OCP encompass other areas, like networking or perhaps mobile devices? "You never know..." is all that Frankovsky will say at this stage.

Paul Rubens is a technology journalist based in England. Follow everything from CIO.com on Twitter @CIOonline, on Facebook, and on Google +.

Read more about data center in CIO's Data Center Drilldown.

This story, "How to use Facebook's open sourced data center design to cut costs" was originally published by CIO.

Insider: How the basic tech behind the Internet works
Join the discussion
Be the first to comment on this article. Our Commenting Policies