I'll push and you pull. The mashup approach to application integration

April 06, 2007, 4:29 PM — The phenomenon of mashup applications[1] on the web intrigues me. The applications themselves are fascinating enough but on top of that you have the richly painted canvas of reaction and opinion from the IT community. In a scene reminiscent of a busy Hieronymus Bosch[2] painting, many voices speak at once. Many reactions take place all at the same time, evidencing delight, horror, exacerbation in equal measure.



Commonly heard phrases include:



'Mashups are just toys.'



'Mashups will change the world and you greybeards just cannot see it.'



'Mashup technique is all well and good for simple projects but you could not do a real enterprise application like that. What about X and Y and Z...'



'Hey! Buy my 8 week old company now before your competitors get their hands on this game-changing application.'



In the midst of all the noise and haste I detect a very useful architectural pattern at work. A pattern which has broader applicability than many think. A pattern which may have a significant impact on the shape of application integration technology. A pattern which may ultimately be the key reason why 'Web Services/SOA' as currently formulated by the industry at large remains a niche rather than dominant force. But I'm getting ahead of myself. Best to back up the truck here and make an argument rather than just present tentative conclusions.



Application Integration is all about getting macro-level things done with a combination of micro-level applications. When we talk about how to go about implementing such a thing, we use very human-oriented language. We talk about application A 'sending' a customer update to application B. We talk about application C notifying application D that an order has shipped, and so on.



The language we use naturally follows how a people-oriented process would work. Bob would proactively tell Jill about the updated customer information. Jack would proactively notify Sally about the order shipment, and so on. This language naturally translates into computer-speak using a push-centric design. i.e. application A sends a message to application B. It is a short hop from here to the classic design patterns of enterprise application integration in which messages flow over wires from sender to receiver. All the usual problems arise: how to ensure that messages arrive, how to avoid duplicates, how to ensure that messages have not changed in transit etc. etc.



There is an alternative formulation of these business processes we could use. We could speak in terms of Application B asking application A periodically for customer updates. We could speak in terms of Application D periodically asking Application A if any orders have shipped and so on. We tend not to express things this way as it is not very human. In the real world, Bob, Jill, Jack and Sally are very unlikely to arrange their working lives this way. This is a pull-centric design.



Now here is the kicker. The web - with all its concomitant bits'n'bobs from XML to RSS/Atom to AJAX - is an extremely good platform for pull-centric design. On the Web, if you try to pull some piece of information and something goes wrong, well you just pull again and again until you get it or give up. Nothing fancy. Just brutish repetition. Something machines are extremely good at. If you want to look at information from yesterday, you just go to the URL that contains yesterday's information. Nothing fancy. Just a simple naming convention that includes dates in URLs.



On the other hand, push-centric design is really hard to implement - not just on the Web, but anywhere. It is complex. It is expensive. If often doesn't scale well even if you get it to work...



Which brings us to the point I want to make. The mashup phenomenon is all about pull-centric design. The mashup phenomenon begs a very pertinent question: can all application integration scenarios that we naturally express in push-centric language, be implemented with pull-centric designs? Is there a limit to what you can do with a mashup - something that would force you to a push-centric approach instead? If every time A needs to send something to B why not get A to publish it, with an RSS/Atom feed instead. Then B can just poll the published information as it sees fit. How far can we push (no pun) that idea?



'Quite far', is, I think, the answer. Maybe 'all the way' is the real answer. If so, the real story behind the mashups phenomenon is the pull-centric design that underlies it.



If you are thinking that this makes REST look more and more attractive, especially the plain-old-GET bits, then I'm right behind you.