To find out, we examined seven PaaS solutions based on the top concerns we hear from customers:
Key differentiators. What's the special sauce? What can you get from vendor X that you can't get from the others?
Lock-in. Once you get on, how easy is it to get off?
Security. Are important security standards (PCI, SAE, etc.) supported?
Reference customers. Who are they marketing to and who is not a good fit? Are there any "keynote" deployments?
In addition to posing these questions to each vendor, we subjected each PaaS offering to a simple test.
The trouble is that all of the tutorials and getting-started documentation seems to be aimed at people working on green field applications. Most of us spend the majority of our time working on existing applications. The big money is getting the legacy to run in our new cloudy world. We wanted to see how easy or difficult it would be to port a legacy application to the cloud.
Our legacy app, called Granny's Addressbook (aka Granny), is a training exercise we use at my company to teach the Java-based Spring framework. The rationale: Chances are if you're comparing PaaS, you aren't a Microsoft shop. If you're not a Microsoft shop, then statistically speaking you're a John Doe with 2.3 kids and your application is in Java. If your application is in Java, then it is probably written in the Spring framework. This is basic market math, so we compared the process of deploying Granny on each of the PaaS clouds.
If you'd like to try this yourself or examine our code, you can download the Granny's Addressbook WAR file, find the Granny source code on GitHub, and follow our steps to deploying Granny to each PaaS (with screen images).
Before diving into the details, we'll give you an executive summary of our results. CloudBees and VMware's Cloud Foundry proved the easiest for deploying our legacy app. CloudBees shined with a built-in CI (continuous integration) tool, while Cloud Foundry's IDE integration was seamless and wonderful. Heroku was a distant third, but probably would have fared better if Granny had been written in Ruby.
Google App Engine has the best SLA but also a high risk of lock-in. Red Hat's OpenShift was a bit disappointing, but we expect the kinks will get worked out as it exits preview status. It likely would've been more impressive if Granny were a Java EE application instead of a Spring app. Red Hat and VMware had the best answers with regards to lock-in.