The one problem with LANs is that they're, well, local. A LAN doesn't traditionally extend beyond the physical boundaries of a data center, or at least a corporate campus. For many applications and services this isn't a problem, and WAN connectivity between data centers and campuses does the job just fine. However, not all services are created equal, and certain functions simply can't be pushed through a traditional routed WAN. For instance, you can't migrate a running VM from one data center to another and have it maintain network connectivity.
Or can you?
Last week, Cisco walked me through a demonstration of Cisco OTV (Overlay Transport Virtualization), a novel approach to connecting remote data centers at layer 2 while skipping some of the pitfalls normally associated with such an endeavor. The tech is deceptively simple -- elegant, in fact -- but as with any cutting edge technology, there are some gotchas.
[ If you could design a blade server system entirely from scratch, it might look a whole lot like Cisco's Unified Computing System. See "How Cisco reinvents the data center" and "InfoWorld review: Cisco UCS wows." ]
At its core, Cisco OTV is simply a way for far-flung Cisco Nexus 7000 switches to share MAC address tables. Normally, if you have two or three data centers, for example, each exists as a layer-2 island with its own set of VLANs, spanning-tree, and so forth. Extending one of those networks into another data center generally runs into issues related to broadcast storms, spanning-tree loops, and other problems that aren't generally at issue within a local switched LAN but can be disastrous if propagated across expensive and lower-bandwidth WAN links. In short, it's generally more trouble than it's worth. That's where OTV comes in.
No LAN is an islandThe implementation is quite simple: A switch running at each data center has a trunked interface to the local switched LAN and plays on all VLANs relevant to the data center extension. On the other side is a link to the WAN transport to all of the other data centers. That WAN link could conceivably be any flavor, but it will need to be OC-12 or better to make good use of OTV. With a few commands, a pseudo interface is created on the switch, and a group access address range is specified. At that point, the switch begins receiving MAC table updates from the other participating switches and transmitting its own. It also then begins responding to requests for remote MAC addresses it's learned on the local LAN segment, essentially proxying those addresses.
When an OVF switch receives a frame destined for another data center, it encapsulates it in a normal IP packet and transmits it over the WAN to the data center where that destination MAC resides. On the receiving end, the local OVF switch strips the encapsulation and drops the frame on the appropriate VLAN as if nothing ever happened. The sending and receiving hosts never know that that they are in different datacenters, or that a WAN link was involved at all.
The underlying table information and routing transport for this scenario is a pretty neat adaption of existing technology. Cisco is leveraging some of the capabilities of the IS-IS (Intermediate System to Intermediate System) routing protocol to make this happen, although the IS-IS configuration is completely under the covers. It really is only about five commands to add a data center to the mix, although the necessary configuration of the Nexus 7000 switches might be a bit more involved.
The upshot is that even though the overlay transport is transparent to the ends of the connection, there's no fear of spanning-tree looping as each site maintains a distinct spanning-tree topology and BPDUs aren't forwarded across the WAN. The OVF functions as a gatekeeper for the frames that should remain local while forwarding those that should be allowed to pass.
When databases fly In the demo I saw, Cisco used OTV to migrate a loaded SQL Server VM from one VMware ESX host to another over a simulated WAN, with the hosts residing at different data centers the equivalent of 400km apart (4ms latency). The VM migrated over in about 30 seconds or so without losing the connection with the client load... with one catch. Although the VM definitely moved, the virtual disk didn't. (Moving an 8GB VMDK through an OC-12 would take far longer than 30 seconds, and such a trip isn't really feasible for a VM under load anyway.) In the demo, Network Appliance's FlexCache technology bridged this gap, enabling the VM disk to remain in the original data center while keeping the delta at the new data center. Naturally, this isn't a scenario that lends itself to a permanent migration, but it might prove useful in some load-balancing and global distribution scenarios.
It's important to note that the established connections to migrated VMs continue along their original data paths. Even though the VM ends up running at the remote data center, the existing TCP connections to that server must still pass through the initial data center to maintain the consistency of the connection. New connections could be rerouted to the remote data center, but an existing connection cannot. This could add significant latency and bandwidth consumption to the WAN links if not monitored. It should also be noted that current technologies put a distance damper on any effort like OTV, since VMotions on links with greater than 4ms latency can get problematic really fast. This roughly translates to 400km of physical separation. This isn't a limitation of OTV, but it's still a constraint.
Needless to say, Cisco's OTV isn't a technology that many companies need. However, to those that do, it's quite compelling. OTV isn't immediately ready to handle intercontinental data center linking, but it could certainly be used to connect data centers in New York City and Washington, DC, or anywhere within a 250-mile radius.
Although those distance limitations are the result of current data transport technologies, the framework is there to support anything coming down the pike. Once it's feasible to achieve 4ms latencies across a 2,500 mile link, OTV will be ready. As such, it goes a long way toward allowing geographically disparate data centers to play in the same pool while greatly reducing the chance of Layer-2 boogymen compromising the network. It's an important step in localizing remote computing resources.
- How Cisco UCS reinvents the data center
- InfoWorld review: Cisco UCS wows
- VMware vSphere 4: The once and future virtualization king
- Exclusive review: HP BladeSystem Matrix
This story, "First glimpse: Cisco OTV," was originally published at InfoWorld.com. Follow the latest developments in virtualization, networking, and cloud computing at InfoWorld.com.
Read more about virtualization in InfoWorld's Virtualization Channel.
This story, "First glimpse: Cisco OTV" was originally published by InfoWorld.