It was an e-commerce manager's worst nightmare. The OshKosh B'Gosh Inc. online store appeared open to customers -- they could place orders -- but the orders went nowhere. The communications link between the clothing retailer and the company hosting its Web site had gone down.
But the nightmare was just beginning. The Oshkosh, Wis.-based company struggled for several days to re-establish contact with its Web server at Digex Inc. in Cupertino, Calif. "The [Digex] facility was secure, and they wanted a two-day notice before anybody could get into it," says Jon Dell'Antonia, CIO at OshKosh. "But how are you going to notify them two days before you have a failure?"
Oshkosh B'Gosh CIO Jon Dell'Antonio says he had a disastrous first experience with an ASP.
Dell'Antonia's problem was further complicated because OshKosh's outsourcing contract was with Sunnyvale, Calif.-based application service provider (ASP) Pandesic LLC, which had, in turn, subcontracted with Digex for the hosting site and servers. And it was a fourth party -- OshKosh's telecommunications carrier -- that needed to get into the Digex site to repair equipment.
"It was like the Three Stooges and the Keystone Cops combined," Dell'Antonia says. "If I went through the whole litany, you'd be rolling on the floor laughing. But we were not laughing at the time."
The lessons are clear for information technology managers: If you don't ask the right questions up front, you risk paying the price later. Fortunately, managers contemplating moving to an ASP can learn from the experiences of veteran users.
Measure of Success
The Motley Fool Inc. in Alexandria, Va., outsources the running of its payroll, budgeting and other financial systems to USinternetworking Inc. (USi) in Annapolis, Md. Kevin Book, senior director of technology at The Motley Fool, says his own IT experts "went on-site and really put them through the wringer," especially on issues of security, system availability, capacity for growth, data redundancy and technical support.
But The Motley Fool's interest in USi's technology went only so far, Book says. "On their hardware and software platform, we were relatively agnostic," he says. The bottom line for all the questions put to USi was "availability," and The Motley Fool's contract with USi contains quantitative service-level agreements (SLA) and penalties, he says.
If the ASP industry were a mature one, users would just need to specify service levels in their contracts and let it go at that, says Audrey Apfel, an analyst at Gartner Group Inc. in Stamford, Conn. "But it's an immature market. Half the ASPs are going to fail, so you better spend time placing a good bet," she says.
Apfel says customers often concentrate on the A in ASP -- the application that will be provided - but they should focus more on the S -- the service component. "The thing that's important is the people," she says. "Ask them how they hire, train and retain staff. When they have turnover, how do I know that expertise I need is not walking out the door?"
Apfel says performance guarantees and penalties are important, "but if there's a metric in your SLA you can't audit, it's not very useful." Audit tools could include users' own record keeping, reports from the ASP or automated, real-time Web monitors.
Jay Robertson, vice president of data center operations at USi, says users should ask, "You say you are going to provide 99.9%, but how are you going to measure it, and what does it include?"
Business vs. Technology
Another mistake some users make is bringing their application specialists but not their technical or operations people to meetings with prospective ASPs. When that happens, USi recommends that the potential customer bring more technically savvy people to the next session, Robertson says.
Some users forgo the nitty-gritty technical analysis and simply count on being able to leverage their clout with the ASP during the course of service.
Clifford Macaylo is the supply-chain vice president at Fischbach & Moore Electric LLC in New Providence, N.J., which has contracted with Cephren Inc. in Palo Alto, Calif., for procurement and project-management application services. He says he doesn't insist on performance guarantees in his contracts, because most ASPs are so new to the game, they will do anything to get and hold on to customers. "It's easy for me to get extremely top-level service," he says.
But didn't he check out Cephren in advance regarding things such as security and data backup? "I have to assume that with today's encryption and firewalls, that if they can't figure it out, I sure as hell can't figure it out," he says.
"Far and away, the most important thing is to get reference accounts," says Art Williams, an analyst at Giga Information Group Inc. in Cambridge, Mass. Next comes the suitability of the application and the financial viability of the ASP; technical considerations are way down the list, he says.
ASPs "tend to be long on technical depth and short on business sense," Williams says. "The field is full of techies who really know what they are doing." He advises, however, that customers ask for a user advocate -- "a single person within the ASP who's measured on keeping the customer happy."
Like Fischbach & Moore, eVinyard Inc. in Portland, Ore., is similarly blase about technical details such as an ASP's backup, recovery and security procedures. "These things are a basic entry point," says Michael Osborn, vice president of sales and technology at eVinyard.
EVinyard is another Pandesic alumnus in search of a new ASP, and the online wine merchant is likely to go with Intel Corp.'s Intel Online Services (IOS), Osborn says. The ASP is attractive, he says, because of its "world-class network-operations centers," its intimate knowledge of SQL -- on which eVinyard's applications are based -- and the around-the-clock on-site presence of Intel technicians and representatives of major hardware and software vendors. Osborn says that by clustering redundant computers and disk drives, IOS can offer a 100% uptime guarantee.
But what's most compelling to Osborn about the service, he says, is Intel's ability to tailor its problem response to customer specifications. "They will do anything we say in response to a logged activity such as an error message," he says. "For example, we may just want them to notify us when a server is down -- just make a phone call. In other cases, we may want them to do everything they can to get it back up."
This kind of flexibility is critical in a rapidly growing and evolving electronic business, Osborn says. "We know their procedures, and they can be modified collaboratively. It's the ability for us to access -- and help build -- the run book for every discrete process," he says.
Dell'Antonia at OshKosh B'Gosh says he will make sure he thoroughly understands his next ASP's run book. "Something we'll be asking for this time is for them to define staffing levels in the [hosting] facility and how they really operate it," he says. "Do you actually have people there physically all the time, and how quick can we get someone in there if it's a problem you are not going to work on?" And he says he will insist on the right to make surprise visits to the ASP's data center.
Dell'Antonia says he will also ask more probing questions about the spare parts and equipment that are kept on-site, the average time it takes to make repairs and the location of repair people.
Gartner Group's Apfel says that's precisely the right approach. "The biggest pitfall is assuming the ASP offers what you need, when what they are offering is whatever they cooked up in the bar the night before," Apfel says. "They are going to try things. ASPs' business models are changing week by week."
Apfel advises users to carefully examine and document their needs before going to an ASP. "It doesn't matter what's in the contract they hand you," she says. "Make sure you have your own independent list of exactly what you need."