The abrupt crash of Pilot Network Services recently raises anew questions about outsourcing critical network services -- particularly services as sensitive as security.
As the shell-shocked survivors of the Pilot implosion survey the damage, they say that not only would they not trust Pilot again -- should it ever emerge from its current financial situation -- but they also are now not willing to throw their fate in with any of the smaller companies providing managed security services. This sentiment is potentially bad news for smaller companies such as Counterpane Riptech and OneSecure, which operate data centers aimed at providing managed security services.
Pilot's demise, which came without warning from founder and CEO Marketta Silvera, left customers scrambling to retrieve their Web servers from Pilot and locate new ISPs. One Pilot customer, Providian Financial, stepped forward to hire a half dozen Pilot employees to keep Pilot's Alameda, Calif., facility running. And Providian took on the job of staying in touch with Pilot's telecommunications providers to ensure network circuits connecting Pilot's Alameda data center and elsewhere would not be shut down.
Maintaining circuits
As of last week, Pilot's circuits were being maintained on a day-to-day basis. Service providers SBC Communications, AT&T, Sprint and WorldCom were using the opportunity to woo the stranded Pilot customers. But by all accounts, it was Providian that helped keep the centers open.
"Our interest is to keep Providian's Web site and secure Internet access up and running, although other Pilot customers may benefit from our efforts," says Konrad Alt, senior vice president.
Providian was one of the more fortunate Pilot customers that received a phone call from a Pilot employee warning that Pilot was apparently going out of business. Several customers have pointed out they were contacted by ex-employees, as Pilot had fired its approximately 200 workers through a mass e-mailing on April 24.
"They gave everyone 15 minutes to leave the building," says Len Cibelli, a former Pilot sales director. "It was deplorable. It's insane and incomprehensible."
"We really didn't get direct notification through the company," adds Jon Hertzog, CIO at Rand, a three-year Pilot customer. "The sales agent, who's no longer an employee, had the decency to let us know."
Rand has a contract with Pilot for Internet access, plus firewall and intrusion-detection services, which stipulates that neither can terminate the arrangement without a 90-day notification.
Financial trouble
"We were watching their financial situation and thought they'd at least give us a month's notice. But now we're scrambling around like crazy," Hertzog says. For fiscal year 2000, Pilot reported $31.9 million in revenue with a net loss of $21.7 million.
According to several Pilot customers who asked that their names not be used, Pilot filed for bankruptcy last week, but the filing could not be confirmed by press time.
Silvera, who hasn't responded to requests for explanation, has been "missing in action" the last two weeks during Pilot's crisis, according to several customers.
During all of this, Pilot's Web site continues to blink with the message: "Securing the future of e-business," with no indication of the company's woes.
In some cases, Pilot's ex-employees stayed on duty and were said to be running data centers or helping Pilot customers through the crisis. "They are now doing a very good job of keeping us in the loop with daily conference calls," says Bruce Bartolf, vice president of IS at architectural firm Gensler & Associates, which expects to move 27 of the company's servers to another provider's hosting center within two weeks. He has not been able to secure high-speed local connections in San Francisco and New York to support the service in-house due to apparent backlog.
Bartolf says he has emergency low-speed Internet-only backup that can be used if necessary.
Other Pilot customers include Sovereign Bancorp, The Washington Post, The Los Angeles Times, PeopleSoft, the Gap, GE Capital, Altera, Newsweek, Fremont Bank and Memorial Health Services. Some were scrambling to find alternate hosting centers and if possible, the kind of managed security services Pilot offered, such as firewall, antivirus, VPN and intrusion detection.
Like other Pilot customers, Hertzog was pleased with Pilot's managed security services, which cost the company about $120,000 per year. "There aren't a super lot of alternatives out there," he says, noting it's expensive to mimic the same security services through in-house staff on a round-the-clock basis.
One Pilot customer who shared information anonymously notes his company had lost Pilot security services but not Internet access and was in daily contact with Providian over the fate of the Pilot data centers. The customer also says his company had just paid Pilot more than $75,000 for the next 12 months' service.
For some observers, Pilot's failure came as no surprise. In a report called "Surviving the managed security services shakeout," Gartner last month warned it was a "no-brainer" to predict that more than half of the start-up managed security providers would fail.
More than $1 billion in venture capital has poured into these start-ups so far, Gartner says. But many just won't have the partnering agreements with larger companies to build national sales, or will neglect development of the complex monitoring technologies needed to run these kinds of data centers once their funding fails. When the managed security providers do fail, it will be "quickly and loudly," Gartner stated.
"At this point, I think we see the long-distance providers, the monopoly [local exchange carriers] and companies like IBM as partners for this," says one Pilot customer, who asked that his name not be used. "For us, Genuity is also a lead candidate for these types of security services."