The self-evident truth about fiber infrastructure is you can never have enough. Spare fiber in your infrastructure today will run out tomorrow. The result is fiber exhaust, which occurs when demand exceeds the availability on your backbone fiber.
The most obvious solution for fiber exhaust is to install additional fiber. But the costs associated with pulling fiber can be substantial, and saying "no" to those who want new applications on your infrastructure can be a career-ending move, while removing older applications on your network to make room for the newer ones won't win you any popularity contests.
Take heart. Wave division multiplexing (WDM) can rescue you. Prior to WDM, a single light beam was used per fiber strand. By contrast, WDM uses many different colors (called lambdas or channels). Each lambda carries an individual optical signal providing the same bandwidth per channel (approximately 2.4G bit/sec with most of today's fiber) as a single light stream.
The more channels, the more bandwidth you can get out of a fiber pair. Generally, WDM provides up to eight channels per fiber pair. This is equivalent to adding seven additional dark fiber strands. Even better news comes in the form of dense wave division multiplexing (DWDM), which supports up to 40 channels today and promises even more tomorrow.
Like many enterprise networks, West Virginia University's fiber infrastructure teeters on the edge of fiber exhaust. Therefore, the university's Advanced Network Applications Lab was charged with evaluating DWDM equipment.
While some established vendors bowed out of our evaluation, Alcatel, Cisco, iTouch Communications (formerly NBase/Xyplex) and Nortel Networks provided us with DWDM shelves. DWDM products are called shelves because they are not really routers or switches. They operate more like intelligent optical patch panels.
Nortel's OPTera Metro 5200 earned the Network World Blue Ribbon Award, because it offered more features, was more resilient and had enhanced management over its competitors. However, the other three boxes were not that far behind the OPTera -- they performed well and provided decent management capabilities.
DWDM may be new to the enterprise scene, but carriers have used it for quite some time to provide a cost-effective cure for fiber exhaust in their own networks. The maturity of these products demonstrate that fact.
Enterprise test criteria
DWDM shelves are strictly Layer 1 devices, meaning they don't really route or switch data packets. For this reason, they can't be measured by the same performance metrics that we would apply to a router or switch. Measurements like packets per second, buffer allocations, back plane speed and processor performance are not relevant in the optical world. Therefore, our tests measured each product's performance based on the following criteria:
Can the product transport the various network protocols?
How well does each product survive unexpected outages?
What kind of redundancy or resiliency does each product offer?
The enterprise features on these Layer 1 devices fall into two primary categories: management and suitability for use in a corporate network environment.
Physical layer enterprise management has far fewer complexities than is typically found in Layer 2/Layer 3 switches and routers because there is much less to manage. The shelf either works or doesn't. Enterprise management features should, at the very least, let you check the status of the devices and the optical cards in the shelf via a command line interface using telnet, serial port access or both. Support for SNMP management information bases is also imperative, so systems management stations can monitor the performance of the DWDM devices. The ability to access IP transports from the shelf management using ping utilities is also a plus.
All the devices we tested have these base-line management capabilities, but the Nortel OPTera Metro 5200 wins points for its additional Web-based management interface and extensive management tool set.
Enterprise feature sets are not extensive with a Layer 1 device either, but are roughly equivalent to what you might find on a first-rate electronic patch panel. The ability to loop back for testing an optical link is important, as is the ability to automatically switch fiber routes in the event of a failure. Indicators such as LEDs that offer information on the shelf status at a glance are also important features. All the shelves we tested came with an abundance of status indicators, and all but the iTouch product can do loop-back and reroute paths.
As with any network device, DWDM shelves need to offer some form of security. Because these are Layer 1 devices, there isn't much likelihood that someone could intrude on them as they could, for example, a router. However, because they offer SNMP- and IP-based management, some security is necessary. No one wants the management of these devices hacked and their services disrupted.
Each shelf supports password protected telnet and command line access. SNMP community strings can be changed from the default. In the case of the OPTera Metro 5200, which uses Web-based management, additional security is offered. Nortel uses Secure Sockets Layer encryption along with a security certificate to authenticate the management PC.
The Cisco Metro 1500 and Alcatel Optinex 1690 use a key to turn power on and off. However, they did nothing to prevent disruption by unauthorized personnel pulling their power plugs, so we saw the keys of dubious value.
As mentioned earlier, all shelves tested have SNMP management capability in addition to telnet and serial port configuration.
The Cisco Metro 1500 does not support a Web interface or IOS. IOS, Cisco's proprietary operating system, used to manage its Layer 2/3 devices. The interface is a combination command line and graphical user interface (GUI) set up through a telnet or serial connection. This recalls the days when some interfaces tried to imitate a GUI with a text screen -- usually not very successfully. While the GUI in the Metro 1500 gets the job done, we couldn't get the arrow keys to work in selecting a menu item through telnet or a serial port connection.
Alcatel's Optinex and Cisco's Metro 1500 provide a user command line interface for management purposes. The commands are a bit arcane, but there is a decent online help system. Through the command line, it is possible to check the status of individual cards, turn lasers on and off, and turn loop-back on or off.
Because the Nortel OPTera Metro is more sophisticated than other products we tested, there is more to manage. The preferred way to manage the OPTera is with a Web-based browser interface.
This Java interface places a heavy requirement on your PC. We attempted to manage the shelves with a 166-MHz Pentium laptop. However, the browser was slow while waiting for the various dialog boxes to fill in. Later we moved the Java tool to a 500-MHz Pentium III workstation, and performance was significantly better.
Once installed, the Web interface is powerful, flexible and easy to use. It provided sophisticated management of nearly every item in the shelf and also any connected shelves.
Management of the iTouch WDM44 is handled through Megavision, an optional GUI, or an included command line driven interface available through serial port or telnet. The command line interface is straightforward. Typing a question mark will bring up a list of commands applicable to the section you are in at the time.
Reliability and redundancy
With multiple backbone services running through the DWDM gear, this is one piece of equipment you simply do not want to go down. Therefore, we ran a suite of tests on each product to see if and how we could disrupt it.
The Cisco Metro 1500, the Nortel OPTera Metro 5200 and the Alcatel Optinex 1690 all survived the module pull test with flying colors. Only the service we expected to fail did. In all cases, the disrupted services returned typically within 10 seconds after we reinserted the pulled line card. In no case did a line card removal cause any ripple effect to any other service simultaneously running over any of the shelves.
Simulating card failures produced some interesting surprises. As expected, when a line card was pulled it caused a corresponding loss of whatever service was connected to that line card. However, when we pulled any other of the optical cards, the shelves kept trucking merrily on. At first this caught us off guard. (Usually when you pull a major card out of a switch or router, the device dies.) Then we realized we were operating in the optical, not the electrical domain. Optical cards do not need power to continue to pass signals.
The Metro 1500s were configured with redundant point-to-point connections between the shelves. Pulling the fibers in use resulted in an immediate transfer to the back-up fiber. Only on one occasion did we notice a reaction due to disruption of the fiber. The Fast Ethernet connection reported a single packet drop (roughly equivalent to a 1-second outage). Not bad at all.
Because the Nortel OPTera Metro shelves had a redundant ring configuration, testing fiber failures between the shelves required a slightly more complex approach than with the other shelves connected on a point-to-point basis. Running in protected mode, we forced all three transports (ATM, Fast Ethernet and Gigabit Ethernet) in the same direction on the ring (west to east). We then disconnected that part of the ring. This forced all the transports to the other side of the ring (east to west.) We counted the number of pings lost during the switchover from one side of the protected ring to the other. Fast Ethernet lost one ping, ATM lost four but Gigabit Ethernet lost none. We repeated this test by reconnecting the west to east ring and disrupting the east to west ring. In this case, Fast Ethernet and ATM lost one ping each while Gigabit Ethernet lost none. In each case ATM and Fast Ethernet suffered brief outages, while Gigabit Ethernet never noticed a thing.
Nortel support said Version 3.0 of the OPTera Metro code had a timing issue with the subrate multiplexing card. Because we were using the subrate multiplexing card as inputs for ATM and Fast Ethernet, but not for Gigabit Ethernet, that explained the dropouts. We were told that Nortel would fix the timing issue in the next version of the OPTera code.
There wasn't much we could do to "damage" the iTouch WDM44. We couldn't simulate module failures because everything is hard-wired in the shelf. Because there was no link redundancy provided in the WDM44 shelves we received, simulating DWDM link failure didn't make sense. We could fail the redundant power supplies by pulling its power cords. The shelf was unaffected by the loss of any one power supply. Interrupting individual protocols caused no effect to any other protocol running in the shelf.
None of the products had a measurable loss of pings for any of the transports during the 72 hours we ran simultaneous and continuous pings. This indicated that all the shelves are likely to perform reliably in production service.
Installation and configuration
The Cisco Metro 1500 installation was straightforward. Simply mount the shelves and connect the fiber jumpers between the appropriate modules. The Metro1500 asks for time and date, physical location and standard IP network information for management purposes.
The Alcatel Optinex installed and configured like its Cisco Metro 1500 sister.
Nortel sent an installation team to rack the equipment and an Advanced Deployment Group to configure the OPTera Metro 5200. The customer in most cases has little involvement, other than as an interested spectator, in the installation and configuration of the system.
Nortel takes a different slant on configuration. Instead of purchasing specific input cards depending on the transport and bandwidth, Nortel provides user configurable input cards. There are only two card types to select from based on laser wavelength, 850 nm or 1,410 nm. Once you have the correct wavelength card to match the laser on the subtending equipment side of the optical connection, you decide what the card will be when it is goes into service using the management software. Input cards can then be changed on the fly. For example, if your ATM connection is migrating from OC-3 to OC-12, the change on the OPTera side is simply a selection change on the interface menu.
Like the DWDM shelves, there is little to configure with the iTouch WDM44 shelves. The shelf can be given an IP address for management purposes. The device is a snap to install and connect to your subtending equipment.
We had great difficulty getting Alcatel's gear to work in our test scenario. The Optinex would not respond to the edge devices in our test setup. Eventually an engineer from ADVA Optical, an Alcatel reseller partner, determined that the line cards shipped to us had the wrong configuration. Alcatel sent one shelf configured with single-mode channel cards, while the other had multimode channel cards. The only way to tell this was to closely compare the part numbers of the channel cards. But even this wasn't a sure thing. When Alcatel sent us replacement cards, these cards also failed to communicate with the edge equipment. Alcatel thought the replacement cards might be mislabeled, so it made one more attempt to supply us with properly configured cards. The third attempt proved to be the charm.