Users lament state of Ethernet

By , Network World |  Networking, Ethernet

SAN JOSE -- Vendors have a lot more work to do to make Ethernet ready for large-scale data center duty.

According to users at the Ethernet Technology Summit here this week, Ethernet could be a lot greener and "fabric-friendly." This shows that recent efforts by vendors and standards organizations -- such as the Data Center Bridging work by the IEEE and Cisco's next-generation Nexus platforms -- to ruggedize and reduce the power consumption of Ethernet switches and routers for data center applications are still incomplete.

100 Gigabit Ethernet: Bridge to Terabit Ethernet

"The biggest, baddest switch you can buy today is still too small -- woefully too small," said Donn Lee, an engineer at Facebook. "We have to lash together huge arrays of 10G links to scale. Switches are not built for being in a fabric."

Facebook is the fourth largest Web site in the world after Google, Microsoft and Yahoo, Lee said. And it is still growing: The social networking site jumped from 50 million users to 400 million users in the past two years, he said.

But that growth has not come without some Ethernet pain points, Lee said. Facebook has warehouse-sized data centers running racks and racks of commodity PCs -- between 20 and 40 per rack -- all running quad core processors.

And with each PC equipped with a 1Gbps Ethernet NIC, there could be as much as 40Gbps coming out of each server cabinet.

"We have to use 10G and a lot of 10G" to connect to aggregation switches and then backhaul traffic into the core of a single 700,000 square-foot data center, Lee said. That's one of the reasons Facebook said it needs 100G and even Terabit Ethernet now. 

But 100G Ethernet and Terabit Ethernet won't be around for a while. So Facebook has to settle for 10G as its data center fabric.

"It's a pain point we see with Ethernet," Lee said. "How do we build an Ethernet network to support tens of thousands of servers and thousands of 40G [cabinets]? It's not easy because the switches aren't big enough."

Some of the bigger Ethernet switches on the market support 500-plus 10G ports, but only 25% or less of them at line rate. That still doesn't meet Facebook's needs.

Originally published on Network World |  Click here to read the original story.
Join us:






Spotlight on ...
Online Training

    Upgrade your skills and earn higher pay

    Readers to share their best tips for maximizing training dollars and getting the most out self-directed learning. Here’s what they said.


    Learn more

Answers - Powered by ITworld

Ask a Question