Monday, November 30, 2009

Musings with John Schlesinger

John Schlesinger is an event thinker par excellence. So whenever I get the chance, I visit him in London to validate some thinking - or just to spend time with a terrific guy! So on a recent trip to London the subject turned to the rise of event thinking and the downplaying of the traditional SOA patterns. Of course the SOA traditions are being reborn to encompass the events brigade, but because SOA is so broadly and imprecisely defined that's perfectly OK. The SOA hype is over, long live the SOA hype. But that's perhaps a topic for another time.

The key observation from my lunch with John was one I had suspected, but was not able to frame properly. With a few well chosen sentences John had it for me.

This is all concerned with orchestration and control. So (deep breath), here goes. Where an event is raised and that event is to be processed by some subscriber, any intent to orchestrate the handling of the event by the subscriber results in a massive increase in complexity. (Roger Sessions will love this!). Naively one starts to think you have the "OK/Not OK pair" of possible responses. But then the "Not OK" responses blossom out of control. We have situations where the "Not OK" response must result in the retransmission of the event (and how does that happen?) and other cases where it must not. We have cases where the originator of the event has to interpret the behavior of the recipient. That sounds like some awfully nasty coupling to me. So instead of thinking that one has the "OK/Not OK" duality from the recipients view point actually what you have is the"OK/{set of lots of possible not OKs which the sender has to know about} multi-trality. In short that's just crappy design!

Thanks John


John Schlesinger said...

You should be scared of orchestration, very scared. Chris' post just gets you from complacent to worried. Here are two more steps.

If I orchestrate another service and it fails (that is, I roll back the activity) then I need to compensate all previous activities. That means that I have gone from a one-way message (no orchestration) to a message with multiple responses, and now from there to multiple services - the straight through service and all the ways (yes, there may be more than one) it has to be compensated.

You should now be scared of orchestration.

The next step is to think about compensation after an activity that took more than a day to fail (for instance, if a manager has to review a decision). Now you have to undo the previous activity even though overnight processing has spread your mistake through the enterprise.

We have now gone from understanding nothing of the service depending on our event, to having to understand its internals and all the other systems that it relates to. You should be very scared by now.


Roger Sessions said...

One way to deal with the problem of failure within an orchestration without breaking encapsulating is to use a compensatory transaction manager (not to be confused with a transaction processing monitor.)

As a general rule, most of the problems that occur within orchestration are because of granularity problem. When web services are made too fine grain (a common problem), the result is an explosion of complexity.

Richard Veryard said...

Orchestration may be scary, as John suggests, but that doesn't mean we should run away from orchestration.

Roger attributes most of the orchestration problems to granularity, but (in the general case) we cannot fix granularity without understanding the orchestration requirements in the first place.

There is a story about complexity and orchestration in my work with Philip Boxer. See our paper Metropolis and SOA Governance. See also the Asymmetric Design blog.

dave.hollander said...

"This is all concerned with orchestration and control". It seems to me the complications are from not aligning our services with the real business processes. Events are great at notification and feedback which when used appropriately should diminish the complications of orchestration and leave compensation under control of the resource owner as a response to a notification.

Orchestrated compensatory transactions makes me very scared.