December 15, 2000, 11:35 AM — Some Java application server vendors say they have been hampered by hundreds of flaws in test programs that Sun provides to make sure these products use Java correctly and work with each other.
Some vendors gripe about wasting weeks struggling with unexpected bugs, while others report only minor problems. The products being tested form the foundation of enterprise Web commerce sites.
Sun engineers patched an unknown number of the almost 6,000 test programs, and 771 went on an "exclusion list," meaning vendors don't have to run these tests to win the coveted "Java-compatible" certification for their application servers.
Rollouts not hurt
While the problem has ruffled feathers, the vendors and industry experts say it should not hurt product rollouts.
"[The Java 2 Enterprise Edition specification] is a big animal," says Randy Heffner, an analyst with Giga Information Group. "It's a major standard that can be [interpreted] in different ways. It's normal that vendors will have trouble implementing it."
But the glitches are frustrating for Sun, which released Java 2 Enterprise Edition (J2EE) with great fanfare one year ago. J2EE specifies behaviors, services and APIs as part of a programming model for Java applications on servers.
By all accounts Sun has responded quickly in making fixes to some tests, excluding others, and keeping the test process moving along. Sun engineers are putting the finishing touches on the next version of the J2EE specification, and this time Sun plans a beta test for the new code and the modified programs to test it.
ART Technology Group, BEA, Hitachi and iPlanet are shipping application servers certified as having passed the test suite. Iona, Silverstream and Sybase have announced their products have passed the tests but are not yet shipping.
The test tempest is likely to make enterprise users wary.
"People claim they're J2EE-certified, but I'm going to [certify their products] through my own evaluation testing," says John Amos, director of financial systems at McKessonHBOC, a San Francisco drug distributor, and a BEA user. "There's a difference between marketing and reality, and it's a question of where the two shall meet."
To ensure that a Java component will run the same on any application server, Sun engineers wrote two sets of Java code.
One is their own version of the J2EE APIs and services -- called the reference implementation -- for which the source code is available to each licensee.
The second is a group of 5,824 tests, called the compatibility test suite, which a vendor uses to check that its product will correctly store an Enterprise JavaBean, make a call to a database through the Java Database Connectivity interface, and so on.