Anyone interested in the history of BPM technology (brief as it is) should not miss Ismael Ghalimi’s recounting of it, “Why All This Matters.” As a seminal figure in that history, his discussion of the relationship between BPMN and BPEL, the two important standards in BPM, is especially notable. Neither standard is perfect. But while BPMN has succeeded in the BPMS world in spite of its shortcomings, BPEL’s shortcomings have largely confined it to the SOA/integration space, where “business-empowerment” does not have especially high priority. And in spite of the fact that BPEL was originally conceived by IBM and Microsoft as an Intalio/BPML-killer - Ismael does not say that directly, but I will - his post insists that BPEL remains central to BPM’s (and Intalio’s) larger mission.
The context for this history is a geeky article and comment thread on infoQ that generally mocks Ismael’s continued fixation on BPEL. Unlike the stacker-bashing posts that appear periodically from the BPMS pureplays worried about IBM and Oracle entering their turf, this one is technical, interesting here and there, but I think mostly beside the point. And Ismael’s own post fails to address the question why BPMN has succeeded in the marketplace while BPEL has not. (He basically characterizes non-BPEL vendors as stuck in some benighted workflow past, but that doesn’t explain why, for example, both SAP and Oracle are moving to “native” BPMN engines for BPM and edging BPEL into the SOA/integration box.)
Here’s my take on it. The original goal of BPMN, as the design surface for BPML (and later, in Intalio’s case, for BPEL), was business-empowerment. With BPMN the process model would no longer be just a “business requirements” sketch but - with implementation attributes added by IT - a diagrammatic representation of the executable solution. This idea is still twisted and mocked by many in the developer and enterprise architecture community, but it is in fact BPM’s “winning idea.”
Even though they need training to use it effectively, business analysts find BPMN’s notation familiar and comfortable. A key reason for that is it is graph-oriented like a flowchart, not block-oriented like a structured program, or more to the point, like BPEL. For example, you can loop back to some previous step with a BPMN gateway, but BPEL places severe restrictions on that. What that means is you can draw BPMN diagrams that cannot be mapped to BPEL, or at least to “roundtrippable” BPEL that preserves the BPMN view after even minimal tweaking in the BPEL environment.
It’s a square peg in a round hole. Vendors have tried to address it mostly by restricting the BPMN you can draw to block structures supported by BPEL - gateway branches must rejoin, no arbitrary loopbacks, restrictions on exception flows from attached events, etc. Really smart tools impose fewer restrictions than dumber tools, but they all have some. So the question becomes, is this still BPMN or a graphical BPEL designer? On this point, the marketplace has mostly voted that such restrictions make it less usable by business.
On the other hand, BPEL provides something that BPMN does not, which is standardized semantics. The issue is frequently mischaracterized as BPMN semantics being “vague.” In most cases they are not vague at all. The problem is they are not required to be implemented by a BPMN-”compliant” engine. Attached events? Event gateways? Oh yeah, we don’t support those. Or, as the vendors would put it, “our customers haven’t told us those are a high priority.” Ridiculous! With BPEL you don’t have that luxury. You can add extra stuff, but if you don’t support the semantics in the spec you’re not BPEL. Period.
So it’s a stalemate for now. But it could change if OMG, and in particular the BPEL backers in OMG, wanted it to. OMG could define a compliance spec for BPMN 2.0 - the elements, attributes, and flow patterns that MUST be supported to claim BPMN support. They have made some murky noises about it, but I haven’t seen anything concrete yet. Since IBM and Oracle are driving the bus there, it would make sense that that subset would have a defined mapping to BPEL, but it would allow non-BPEL engines to be compliant as well.
That would be a healthy thing for the industry. But I’m not holding my breath.
October 25th, 2008 at 1:29 pm
…I may be more used to specifications for computer languages (like FORTRAN, COBOL) than business process but I suggest that the specification for BPMN 2.0 itself should be what vendors comply with and not a separate compliance statement. Compliance statements from vendors will then effectively state what they have not achieved (yet). Purchasers and consultants can then judge whether the level of compliance of a particular product is sufficient for there purpose.
As BPMN is unlikely to stop at 2.0, a really supportive vendor should not only be compliant with the currently approved specification but also support proposals for future versions, so that practitioners can work with the new ideas while they work through the acceptance process of a standard…more at
David
October 25th, 2008 at 4:21 pm
Yes I agree, not a separate document but part of the BPMN 2.0 spec. It is unrealistic for many reasons to expect any single vendor (much less several) to support ALL of BPMN, so unlike BPEL it would be necessary to identify a subset.
October 27th, 2008 at 8:12 am
Thanks for writing this, I agree with everything you say.
Both BPML and BPEL were flawed from the beginning because of the idea that one must “translate” the notation into a different executable form. This concept of needing to translate to another form came from years of experience in software engineering with standard programming languages, including software modeling with things like UML. When these software engineering experts approached the business process domain, they treated it like any other software problem, and applied the old approaches to it. This mistake might be forgiven given the number of people doing software engineering and the momentum of the approaches there.
The business process domain is not simply another programming problem — something I have been pointing out for more than 15 years. When people are involved, it changes the nature of the problem, and profoundly effects the solution. One rather clear difference, is that when I run a typical software program as a user, I have no need to know how the software works on the inside. But for an effective organization to be able to cope with external contingencies, to be able to adapt and respond to changes, the internal state of a running business process must be visible. This is not the only difference, but this one difference drives (though a line of reasoning I can’t include here) to the resolution that the diagram drawn should be directly executable. You should not transform the diagram from BPMN to any other representation at all. It is not that BPEL is flawed, it is the the transformation of BPMN to anything that is the root of the problem.
We must have BPMN that can be executed directly. This is the What you draw is what you execute approach to supporting business processes. This would be done by clarifying the BPMN spec, and providing the conformance criteria that you have called for.
I do find it annoying that there are still some people who defend the “software engineering” approach to business process with so much sophistry that it clouds the entire marketplace.
October 28th, 2008 at 1:40 am
Hi Bruce,
Nice to see that our article around BPMN and BPEL is generating a lot of discussions last days
At Bonita and Orchestra teams side we have been working with both XPDL and BPEL standards for a while. For the last two years, we decided to concentrate our efforts on building a generic and extensible process engine (The Process Virtual Machine) which is process language agnostic. This technology allows us to support different process languages and so we are supporting already XPDL and BPEL (JPDL to come from Jboss guys).
However, we will not invest at front on generating BPEL from a BPMN definition because we really think that this is not the way to go: the transformation is hard to implement (most of the vendors are doing it wrong btw), the round trip is more than difficult and the synchronization between analysts and developers is lost.
I see XPDL2 much more convenient to store a BPMN definition (easy transformation). This simplifies the analyst’s vs developer’s collaboration including round trips.
The only missing thing would be the execution related information that is not included in XPDL2 file.
So my point is, when you talk about:
“OMG could define a compliance spec for BPMN 2.0 - the elements, attributes, and flow patterns that MUST be supported to claim BPMN support”
In that case XPDL2 spec seems closer to this target than BPEL; just flow patterns should be added, XPDL2 already provides a mapping with BPMN elements. Why do you not consider XPDL as ?
Another point would be why not use BPMN+XPDL+BPEL as the standard for BPM ? meaning, BPMN to XPDL2 transformation (tooling side) + to leverage the BPEL semantic wherever XPDL is under-specified (runtime side).
Best regards,
Miguel Valdes
October 28th, 2008 at 1:46 am
Erratum:
“In that case XPDL2 spec seems closer to this target than BPEL; just flow patterns should be added, XPDL2 already provides a mapping with BPMN elements. Why do you not consider XPDL as ?”
would be
“In that case XPDL2 spec seems closer to this target than BPEL; just flow patterns should be added, XPDL2 already provides a mapping with BPMN elements. Why do you not consider XPDL as a basis for that rather than BPEL?”
best regards,
Miguel
October 28th, 2008 at 8:48 am
Miguel,
From a technical perspective I agree XPDL could be used for model portability. In fact I have done work with Robert Shapiro and colleagues there on model portability and that work is actually incorporated in XPDL 2.1. You should check it out. But from an execution standpoint XPDL is more like EDI than BPEL, i.e. a convenient container for transport but requiring private agreement between sender and receiver to know what the content means. It was never intended to be a standardized execution language.
–Bruce
October 28th, 2008 at 10:02 am
I agree with your statement Bruce. My point is: why not just clearly define the contrat between sender and receiver exchanging a XPDL2 file ?
Basically, if we agree on how a XPDL2.1 definition should be executed in a BPM runtime we made it ! so what about taking the BPEL execution semantic as the specification for the runtime ?
or in other words, to leverage BPEL semantic wherever XPDL is under-specified.
regards,
Miguel
October 28th, 2008 at 10:35 am
OK I understand now, but politically XPDL is hopeless for that. Standardized BPMN execution semantics is not a goal of WfMC. It is a goal of OMG in BPMN 2.x - although they move too slowly and their specs tend to sink under their own weight - and it is also a goal of the new BPMlab shadow-BPMI Ismael is trying to set up. If they could focus on the BPMN side and not the BPEL side, that might be an effort you could contribute to. Wouldn’t that be an ironic twist to the whole thing!
October 28th, 2008 at 7:59 pm
@kswenson good points. At a slightly higher level, one of the biggest educational points that I have worked on with customers is to help them understand that the diagram doesn’t need to be “coded up” - that the diagram IS the process … Now, there might be code to aide execution of a process, but the code doesn’t drive the PROCESS it drives some detail within the process (a call to a webservice within an implementation step, or a user interface, or whatever..) but all things PROCESS live inside the model. It has its challenges but it sure is nice to not be worrying about roundtripping…
@bruce - my suggestion for a spec starter… we write a nice BPMN model that uses a reasonable percentage of the BPMN spec-defined model elements. Compliance would come at two levels:
1. can i MODEL the specification model (ie, can i draw it in this tooling with standard components). By this definition of compliance, a visio stencil may do the trick…
2. can the target environment EXECUTE the model.
Next, additional models could be provided that describe “illegal” models. If you model these in a compliant environment, it should, in some fashion, warn you that the model is not correct (either by throwing an informative error when you run/test the model, or by showing validation errors at design time).
I’ll post some ideas on our blog, and linkback here…
October 28th, 2008 at 8:22 pm
Scott,
I like #1, not as interested in #2. But compliance isn’t can I DRAW it in tool B, but can tool B import or load compliant xml serialization created by tool A (and understand it, continue to edit it, etc.) We can copy drawings today. What we need is interchange between tools.
–bruce
October 28th, 2008 at 9:04 pm
I would argue that today, you cannot even DRAW the same model in all the tools out there, the same way. To my mind, we haven’t even reached that level of “compliance” … and its actually important.
We really CAN’T have import/export compliance until the spec comes out for storage. I’m not sure the goal is interchange between tools - that’s useful for consultants, less useful for customers of BPMS tools. Its useful, but its like writing database-portable code - useful to customers, but not as useful as it is to software consultants who switch databases often. Consultants switch tooling depending on the customer and want to bring models with them quickly. Customers switch tooling much less often. (note: they still generally WANT interchange, so please don’t read it as me saying they don’t want it)
That said, what I have heard a lot from customers is that they really want to be sure that they’re getting BPMN. That the tooling will faithfully represent the visio (or other ) models they might already have drawn.
Finally, you have to have the drawing part compatible in order for the interchange to work… and someone has to draw these models to begin with, designed to have reasonable coverage of the whole spec…
So, step one is to draw a compliance model anyway.
October 29th, 2008 at 1:40 am
#Bruce:
My team and myself we are indeed open to work with OMG guys, or with any other consortium/organization to go further on BPMN execution semantics.
However, I currently see some of those efforts, i.e BPMLab, as just a promotion channel of BPEL focused on one BPM provider products and components
#sfrancis:
The interchange format between BPMN tools is already available: XPDL2.
best regards,
Miguel Valdes
November 2nd, 2008 at 1:58 am
Microsoft just announced the “M” language under the slogan - models will not succeed if they can’t be read in a text editor. Is it that simple - BPMN needs a textual representation, not just the boxes and arrows?
I know M is not comparable to BPEL, but from a programmer/code perspective.
November 6th, 2008 at 9:34 am
#Miguel - problem is, the vendors don’t all export XPDL. ok, that’s only part of the problem. but its a big one. we want to test vendors for compliance to the BPMN spec. we still need to draw that compliance model (the model that illustrates compliance) to make sure vendors are really allowing authors to draw BPMN diagrams.
Secondly, it would greatly improve the utility of this compliance model if it had storage (e.g. XPDL, BPEL, BPMN 2.0) that is portable or at least “agnostic”. I think XPDL might meet the need of being agnostic, but practically isn’t portable due to lack of support, and lack of vendor commitment to future support. BPMN 2.0 (in whatever form) seems to have a better chance of getting this support from vendors… and BPEL has reasonably good vendor support, but doesn’t have a high-fidelity two-way mapping between BPMN and BPEL. (for purpose of compliance, the mapping needs to be two-way or “native” imho…)