January 19, 2010, 1:22 PM — In many data centers, latency and jitter are the most important metrics. Even the small amounts of delay (latency) or delay variation (jitter) introduced by a switch can have a profound impact on application performance. This is different than with general enterprise switching and routing, in which speed-of-light propagation times dwarf the latency added by any given switch or router.
In contrast, devices in the data center are only a few meters apart, or less, so every microsecond counts. Also, there's often a simple business driver involved: The more transactions an organization can process in a given unit of time, the more revenue it can expect to realize.
To help reduce latency and jitter, some switches (Arista, Blade, and Cisco) use so-called cut-through switching. A cut-through device begins forwarding a frame after examining only the first 12 bytes of its Ethernet header. In contrast, a store-and-forward switch caches the entire frame before making a forwarding decision.
The Dell, Extreme and HP switches all used store-and-forward mode in these tests. Extreme's Summit x650 can be configured in either mode; Extreme's engineers opted to use store-and-forward switching in order to achieve lower latencies.
Cut-through designs typically deliver lower latency, but there are tradeoffs. The biggest issue is that cut-through switches will forward corrupted frames, since they don't wait to see if the checksum at the end of each frame is valid. A router or other store-and-forward device will keep corrupted frames from leaving the data center, but such traffic could be a problem inside the data center, especially in large broadcast domains.
Another possible concern is that relatively low latency often means relatively small buffers. This isn't a problem when moving traffic between pairs of ports operating at the same speed, but speed mismatches between ports (say, gigabit and 10G Ethernet) or congestion from many-to-one traffic patterns could cause frame loss earlier than with store-and-forward devices.
Some vendor data sheets claim lower latencies than those we measured. Those claims may be valid, but they're not necessarily the most meaningful numbers for end-users (see related story "Lies, Damned Lies and Latency").
A cut-through design clearly paid benefits for the Arista and Blade switches, which delivered far lower latency across all frame sizes than their competitors. Blade's G8124 wins bragging rights with the lowest unicast latency – 750 nanoseconds with 64-byte unicast frames – but both the Arista and Blade devices consistently posted numbers around 800 nanoseconds in other tests.