EXCEPTION: SaaS mission-critical applications often thrive in public cloud. A mission-critical software-as-a-service (SaaS) application often lives its entire life cycle in the public cloud. The main reason SaaS applications stay in the cloud forever is because in most cases they are green-fielded in the cloud and constantly rely on the elasticity of IaaS to continually grow and contract the environment as usage patterns swing. SaaS apps usually follow a similar pay-as-you-go billing pattern and it's most cost-effective to match infrastructure costs with revenue.
RULE: Highly integrated, business-critical apps should stay where they are. Any highly integrated business app, like your ERP or your finance applications, should stay where they are because it would be nearly impossible to move the application without causing problems that would grind business processes to a halt. Your ERP is your logistics system and you can't get more integrated than that. Any application that would halt operations if you moved it wouldn't be worth moving.
EXCEPTION: Only rarely will you have an exception to this rule. One exception might be that you don't have an ERP solution yet and you're looking to start one. As long as you aren't already integrated into the business, it can make sense to start off in the cloud so you have the benefits from the start and don't have to worry about the mobility of it later. It's possible that if you're young and looking for integrated solutions, than going to a SaaS solution will make sense. The key question is, "Have you already done the integration?" If not, then you can do a cloud-based SaaS because it's probably better in the long run.
RULE: Bursting into the public cloud doesn't work. There's a general understanding that you can have your application live in your internal environment and when it needs more room you can "burst" those extra resources into the public cloud. It's an idea of bursting between clouds and owning the base while renting the burst. That idea isn't widely in use because it hasn't been successful yet. The required technologies are not as agile as they need to be to create the desired efficiencies.