SOA and ESB's - cue backlash

In a pretty funny article titled "ESB-oriented architecture: The wrong approach to adopting SOA", Bobby Woolf of IBM points out, rather sanely, that if you think that an ESB is some magic wand you can wave at your project Harry Potter style, mumble a SOA incantation and expect a fully formed SOA architecture to emerge then you deserve everything that is coming to you. I couldn't agree more. An ESB is a useful tool, but it isn't magic. If you think you merely have to deploy an ESB to SOA-ify your project, then you are putting the cart before the horse.

Repeat after me "There are no Silver Bullets".

SOA is an architectural approach, you need to concentrate on building services, services that align with business needs. The promise of SOA is business agility. It may be that an ESB will help you realise your vision quicker, since they are fundamentally useful, but you need to have the vision first. Then you can choose the right tool for the right job.

This is much the same message being put forward by Jim Webber on Guerilla SOA. Define what is it that you want to do and choose the right tool for the right job. It may be that you need an ESB, it may not. Definition will at the very least help you choose between ESB products.

This is also the message coming from Cape Clear: The End of Big SOA. Over the last years we have seen increasingly bloated and complex products and perhaps more disturbingly increasingly bloated RFI's. I've lost count of the number of RFI's that specify requirements for technologies I know that customers don't understand and more importantly will never, ever use. So good riddance to this trend. Part of the attraction of SOA was a move to simplicity. And the industry is now re-asserting this message.

Also the refocusing on SOA and not on the ESB, will hopefully shift the focus back to producing services. ESB's mix a couple of features:

• connectivity (email, HTTP, FTP, File, JMS etc, etc).
• mediation (transformation, routing, micro-flow).
• service-creation
• service-orchestration
  

Message-Oriented Middleware (MOM) typically refers to message-queue systems such a JMS. However, ESB's do enable a superset of MOM, which can accept messages in any format,from any transport and route and transform them. This is related to Jim Webber's MEST acronym which stands for Message Transfer. This is supposed to be a message analogy to REST. Wouldn't MT have been a more appropriate acronym (but then it wouldn't rhyme with REST I suppose).
These are the connectivity and mediation parts of the ESB (the bus parts). They are a generally useful tool. You can use them to:

• build an MO-system
• mechanism for versioning services.
• mediate different clients of services.

However, many people when they think of an ESB, focus only on these features rather than service-creation and service-orchestration. This is wrong. SOA is about creating services, the connectivity and routing part is important but should be considered secondary to services. Or put another way, routing / mediation is important, but you need something to route / mediate to.

So clarify your vision, define your business process, define your services, identify your clients and data formats. Then choose the right tool for the job. Most of the time you'll choose an ESB to help you implement the project, but this is not ESB-first. That would be like saying "I want to create a Real-time Flight Simulator! Step 1: I need a Java compiler" - you will most likely, but Step 1, is define the problem to be solved.

Powered by ScribeFire.

Is BPEL universal?

Jeff Davis blogs about some of what he sees as the short-comings with BPEL: What's wrong with BPEL?. It is a mixed-bag - somethings are correct, some aren't. Here are they points:

1. BPEL only works with SOAP-based services. This isn't true. With BPEL 1.1 spec, BPEL required WSDL as a means to specify a partner's API. Now WSDL can support SOAP bindings, but other bindings exist (e.g. WSDL-HTTP). So at a spec-level SOAP wasn't a prescription. However, in practice many BPEL 1.1 engines only supported SOAP-bindings. With BPEL 2.0 even this dependency on WSDL has been removed. - BPEL 2.0 doesn't require WSDL, and it already didn't require SOAP. The standard has deliberately and in my view correctly moved away from the WSDL/SOAP heritage. BPEL is a process workflow language about manipulating XML documents - so now it requires only XML Schema. I think over the next year many of the existing BPEL tools will support creating partner descriptions from REST+XSD, WADL and POX. The next version of Cape Clear supports inclusion of REST-style services directly in BPEL (no SOAP required). We've also greatly simplified connecting to JMS, Email, FTP, etc from BPEL with or without SOAP.
   
2. Passing of data between activities requires XML and WSDL constructs. Again BPEL 2.0 removes the WSDL part. You only need XSD knowledge. Admittedly this is confusing for developers - so I think this is mostly a valid point.. Looking forward, good tool support can help. BPEL 2.0 also supports expression languages. I think innovative companies will provide good Java-XML bindings automatically to make them amenable to Javascript or Groovy manipulation.
3. BPEL is a closed standard. This is the point that if you use an proprietary extension then you are no longer vendor neutral. While this point is true in the general sense, many BPEL engines provide the ability for developers to define XPath extension functions to Java which can be called from within BPEL. This is a reasonably indepdendent extension mechanism. Also, any Specification can be called a closed standard from this point of view. So should we abandon standards?
   
4. BPEL requires deployment for testing. This is true of some products (it is true of Cape Clear). Not sure I agree with the "impacts developer productivity". I think productivity can be enhanced through good tools: viewers, debuggers and test frameworks. For example Cape Clear allows developers to use Buckminster to automatically build, deploy and test BPEL or Java Web Services from within a continuous-build system.
   
5. BPEL is confusing - I agree with this point. There are lots of new things to learn. I'll get back to this point at the end.
6. BPML as a solution to BPEL/Process modelling - only makes things more brittle. I agree, but my advice would be: don't use BPML as a means of specifying BPEL. BPEL is a programming language, to gain its power, you need to use it directly.
   

So as I said a mixed bag, some right, some wrong. Jeff mentions that JBoss JBPM may be a better fit than BPEL for Java shops (which is interesting given his point 3 above). I obviously wouldn't agree :-) However, I think a more interesting issue is whether BPEL is a universal tool, can it/ should it be applied to all integrations involving Web Services. No it shouldn't. There is no such thing as a universal tool - there are no silver bullets. BPEL is incredibly powerful and has very well defined semantics, but it has a sweet-spot: orchestrating stateful web services together. If that is not what you are doing, for example, you are using BPEL as a stateless route-n-transform mediation, you can do this in BPEL, but if you are a Java shop, you can quite easily do this in Java. Many of the ESB's on the market (including Cape Clear) will help you do this with no BPEL required. So pick the appropriate tool to fit your circumstances.