West Virginia auditor blasts Cisco, state for 'oversized' router buy

West Virginia wasted millions in federal grant money when it purchased 1,164 Cisco routers for $24 million in 2010, a state audit concluded.

A report issued this month by the West Virginia Legislative Auditor found the state used a "legally unauthorized purchasing process" when awarding the router contract, paid for with federal stimulus funds, to Cisco. The auditor also found Cisco "showed a wanton indifference to the interests of the public" in recommending the investment in its model 3945 branch routers, the majority of which were "oversized" for the requirements of the state agencies using them, the report stated.

[PRICEY STUFF: Country's largest 4-year university expels Cisco, saves $100 million]

"The Legislative Auditor believes that the Cisco sales representatives and engineers had a moral responsibility to propose a plan which reasonably complied with Cisco's own engineering standards," the report states. "(The auditor) concludes that the majority of Level 3 Cisco model 3945 routers which were purchased are unnecessary."

Cisco declined to comment on the report, instead referring all inquiries to the state of West Virginia.

West Virginia could have saved almost $8 million had the scope of the purchase been scaled to the requirements of the state's libraries, schools and state police, the report states. Smaller, less expensive routers could have been used in the state's 172 libraries, resulting in a savings of $2.8 million; in state police facilities, for a savings of $1 million to $1.4 million; and in 368 schools with enrollments of less than 500, for a savings of $3.68 million.

The report recommends that West Virginia's State Office of Technology look into possibly redeploying any Cisco 3945 routers to more appropriate public facilities to maximize their usage; and/or trading out unnecessary features and modules on the 3945s which have yet to be deployed for the $80,000 of VoIP modules necessary to run the state police's VoIP system.

The report also recommends the state's purchasing division determine whether the Cisco representatives involved with the sale are liable for debarment under state code.

The findings come shortly after California State University concluded that Cisco's bid $100 million more than the winning $22 million bid to refresh the university system's 23 campus network. Indeed, the West Virginia report cited the CSU findings in its own conclusions.

And in 2004, Cisco was found to have written portions of an $8 million bid to upgrade San Jose City Hall's network. The contract was withdrawn by city officials after the discovery.

The West Virginia auditor found that the State Office of Technology used a purchasing process which is unauthorized by West Virginia statute or legislative rule to purchase the Cisco 3945 routers under the Broadband Technology Opportunity Program (BTOP) grant. The Office of Technology used a "Secondary Bid Process" on an existing contract approved by the State Purchasing Division, instead of a competitive bid process open to non-Cisco vendors, as required by law.

In addition to using the legally unauthorized secondary bid process, the contract used to purchase the routers was inappropriate, the report stated. The 2007 IPT07D Contract with Verizon Network Integration was a statewide contract for a "simple expansion" of telephony; however, the 1,164 Cisco routers have the capabilities to support between 814,000 and 1.39 million VoIP telephones, significantly more than the state needs, the report concluded.

"The 2010 branch router purchase was inconsistent with the specific requirements of the purchasing statute," the report states. "Had this purchase been competitively bid, the Purchasing Division would have administered the bidding process and provided contract management. In addition to exempting purchases from Purchasing Division scrutiny, the secondary bid process, as applied to the BTOP router purchase, appears to have the effect of limiting competition for future purchases to only those vendors on the original contract."

The original IPT07 contracts specifically requested bids for "Cisco name brand product(s), "or equal," the report states. The release document for the 2010 purchase, however, stated that the "IPT07SWC [statewide contract] specifies Cisco equipment only."

"Accordingly, it appears that, in addition to limiting the pool of bidders to those holding an IPT07 contract, only those bidders who could provide Cisco products were eligible to bid on the (2010) release," the report states.

The approval for the purchase should have been made by the West Virginia Purchasing Division and not the State Office of Technology, the auditor's report states. And it should have been open for competitive bid, the report says.

"The process used to purchase the Cisco routers was not equal, fair or consistent with the intent of the purchasing statute," the report states. "Vendors of non-Cisco branch routers such as HP, Brocade, Juniper and Alcatel-Lucent were not given notice or any opportunity to bid on the statewide expansion of broadband."

And because the secondary bid process excluded all other branch router makers, the state cannot be certain that it received the best possible price for the routers, including the lowest possible price Cisco was willing to offer through its partner vendors, the report states. It then cited the CSU situation and the May 2011 Purdue University Hansen Cluster project in which Cisco won a competitive bid by discounting its gear 76%.

"While the Legislative Auditor cannot say with certainty that a more open competitive bid process, required wherever possible by West Virginia law, would have resulted in a lower cost for the branch routers purchased by the BTOP grant, it is a basic principal of our free-market economy that competition between vendors leads to lower prices and higher quality products," the report states. "Yet, instead of using an open competitive bid process to attempt to receive the lowest possible price and highest quality product, the State of West Virginia simply relied on Cisco's goodwill."

Cisco CEO John Chambers is a native of West Virginia.

Jim Duffy has been covering technology for over 25 years, 21 at Network World. He also writes The Cisco Connection blog and can be reached on Twitter @Jim_Duffy.

Read more about lan and wan in Network World's LAN & WAN section.