Archive for October, 2006
The Forrester Wave for Business Process Modeling tools has been released, and you can see it here courtesy of ProForma. The ‘clear leader’ is still IDS Scheer ARIS, but ProForma and MEGA have narrowed the gap. ProForma actually leads on the ’strength of offering’ axis, while ARIS leads on the ’strategy’ axis. Making the list as strong performers are EMC (ProActivity), iGrafx, IBM, Telelogic, and Casewise.
Now that I’m getting into process modeling via the BPMN training, this is a vendor space I need to begin tracking more closely.
October 26th, 2006
For those of you who follow enterprise content management, the latest Gartner magic quadrant has been made available by one of the “winners,” EMC. The link, via the EMC site, is here. EMC just noses out IBM, whose entry includes both the DB2 Content Manager family and FileNet. Rounding out the Leader quadrant are OpenText (including Hummingbird acquisition) and Stellent. Hovering nearby are Vignette and Interwoven in the Visionary quadrant, and Hyland in the Challenger quadrant. Microsoft and Oracle, who seem to get all the press buzz, are farther back.
Most people don’t know it, but EMC’s Documentum Process Suite is an excellent BPMS, with ECM integration built in out of the box. If you’re interested in content-centric BPM, check out my free report on Documentum Process Suite in the 2006 BPMS Report series.
October 25th, 2006
This Thursday October 26 I am doing a webcast on BPM for IBM as part of their 2-day ”virtual jam” on BPM and SOA. Interactive Q&A follows, and a chance to find out more about how IBM is putting the puzzle pieces together. Click here to register.
October 23rd, 2006
Yeesterday IBM briefed analysts on their latest round of BPM and related “business services” announcements, all part of their broad push on SOA. They start with a message we can all nod our heads about: Services are the building blocks for business processes. But connecting the dots is not that simple.
The first half of the briefing talked about a lot of new tools for building and managing business services, along with a major initiative on the part of IBM Global Business Services to build them for you, based on a single well-defined approach used globally across IBM: “one registry, one repository, one methodology, one set of tools across IBM”. The second half talked about some minor tweaks to the BPM offering, with engagements based on a very different approach — an “open methodology based on best practices” that lets customers start anywhere (i.e. modeling, BAM, automation) and do just as much as they want. IBM obviously has a clearer vision for what business services are and how they should be built than they do for business processes.
IBM describes a business service as “a business function whose execution can be adapted at runtime based on business policy and user context.” That sounds interesting in its own right, but they didn’t elaborate on the adaptation piece. The key ingredients are business metadata (describing the policy), reference business models (standard representations of elements needed to select and tailor the behavior), and the business function (the underlying services selected to do the job).
IBM announced a new Business Services Fabric as their strategic platform for building and managing business services, plus some industry-specific prebuilt SOA assets based on the Fabric. The Fabric includes existing components such as WebSphere Process Server and WebSphere Integration Developer, and adds a bunch of new ones, including:
- Dynamic Assembler, a scalable “service personalization and semantic services engine”
- Business Services Repository, layered on WebSphere Services Registry and Repository, supporting business services metadata
- Subscriber Manager, controlling “entitlement”
- Performance Manager, providing visibility and monitoring of running services
- Governance Manager, to “visualize hierarchies and dependencies to monitor business SLAs”
- Composition Studio, an Eclipse plug-in for visually modeling and managing business services metadata and policies
- Industry-specific services content, including Healthcare Payor Pack and Insurance P&C Pack
The Business Services Repository “leverages and extends” the WebSphere Service Registry and Repository. The WebSphere Service Registry and Repository enables search and publishing of record metadata such as service location, endpoint definitions, lifecycle state, and service usage parameters, along with technical metadata associated with infrastructure implementation - WSDL, XSD, WS-Policy, etc. The Business Services Repository contains business-related metadata and industry contextual data for use in dynamic service selection and assembly, such as subscription policies, business policies, industry semantics, and industry metamodels.
Using these repositories and the rest of the Fabric, IBM Global Business Services is launching a major effort to build large-scale “composite business services” - essentially customizable industry solutions based on SOA - in a few SOA Solution Centers, and distribute them globally through its army of 90,000 global business consultants. To give you an idea, the composite business service called Insurance Rate Quote Issue Auto - described as “one of the smaller ones” - has a price tag of $900K and includes 1000 hours of custom “configuration.” So these aren’t out-of-the-box solutions.
The briefing then shifted to BPM, which IBM defines as “a discipline combining software capabilities and business expertise to accelerate process improvement and facilitate business innovation.” As in the case of business services, when IBM talks about “process knowledge” and “industry expertise”, they mean theirs not yours.
On the BPM side, enhancements to WebSphere Business Modeler seem to be minor tweaks:
- Better integration with Visio, PowerPoint, and ARIS
- Improved simulation input validation and troubleshooting
- A simplified business format for developing KPIs and business metrics
The new version of Modeler apparently does not yet support the WebSphere Service Registry and Repository or Business Services Repository (or possibly any other part of the Fabric). In the Q&A session, those links were described as something to look for in the future. I would hope so.
Enhancements to WebSphere Business Monitor appear to be more substantial but mostly in allowing it to be used for BAM outside the context of BPM. The new Monitor will be able to aggregate and respond to events conforming to IBM’s CBE standard, regardless of their source. Previously I believe Monitor just listened for Process Server events. Also the monitoring models are expanded to include input from Modeler, WID, or custom events, instead of just Modeler.
Process Server enhancements emphasize improvements to human workflow plus dynamic selection of service endpoints on the ESB (along with configuring mediation logic). The human workflow enhancements were described as follows:
- Graphical process view
- Group work
- Ad hoc follow-up tasks and sub-tasks
- Remote client support
- Web Service interface for tasks
No details were given, and the term BPEL4People was not mentioned. FileNet came up only as follows (I’m paraphrasing here): The deal closed yesterday, and we’re positioning it as content-centric BPM, and we’ll have more to say at a later date.
The strength of the announcements is clearly the Fabric, along with what appears to be a massive effort in IBM Global Business Services (and presumably ISV partners as well) to use it in building industry solution components. The “one registry, one repository, one methodology across IBM” idea should really accelerate SOA by providing a concrete blueprint used on a large-scale. I only wish the company would apply a similar kind of thought leadership on the BPM side by advocating more forcefully for a specific BPMS-oriented methodology in which modeling leads directly to an executable process with performance monitoring, and by creating well-defined touchpoints in the tools and methodology linking BPM to SOA and the Business Services Fabric. It’s coming, but clearly not yet.
October 18th, 2006
This week OMG is putting on a 4-day workshop in San Francisco on Building a Service Oriented Architecture with BPM and MDA. Judging from the two sessions I attended, it’s really a good program, kind of an IT perspective on BPM. Yesterday I got the chance to meet Stephen White, one of the principal authors of the BPMN spec, and hear about what it is, how it works, and where it’s going… directly from the source.
In terms of what it is and how it works, I was surprised at the number of new things I learned in his session… since I have been back and forth over that spec many times in preparing my BPMN training. For example:
- Tasks in BPMN have a Type attribute, with defined values Service (the default), Send, Receive, Script, User, Manual, Reference, Abstract, or None. Depending on the value of the Task Type, certain message flows to or from the task are prohibited, and certain other attributes are required. According to Stephen, any real BPMN tool should enforce these validation rules. I am sure many do not.
- A message intermediate event in normal flow can either send or listen for a message event. You can’t tell from the diagram (unless a message flow is drawn). This might be legal but it seems bad practice. I think less ambiguous to use a Send or Receive Task instead.
- A sequence flow is “not a control flow” because the target activity could also be gated by other conditions, such as data flow. I dunno about this one. All simulation engines assume a sequence flow is a control flow.
- The exception flow from an error intermediate event can loop back to the same activity, essentially retrying it after the error handler. BPEL doesn’t allow this; following the fault handler, BPEL re-enters the process flow after the activity (scope) where the fault was caught.
These might seem in-the-weeds details but it shows how complicated even a seemingly simple standard like BPMN can be to understand.
As far as where BPMN going, I was a bit underwhelmed. Rumors of an imminent BPMN 2.0 last summer were nowhere close to the truth. In spring of 2007 we should see BPMN 1.1, which makes minor tweaks to the current spec, cleaning up the semantics of details such as Link events and Exclusive Merge gateways. BPDM and a schema for BPMN? Don’t hold your breath. When asked about this at his session, Stephen weakly offered WfMC’s XPDL 2.0. That’s not OMG’s official position, obviously. But to me, OMG’s lack of urgency on this issue is unconscionable. What about some kind of compliance guidelines, such as a list of MUST-SUPPORT objects and attributes? Today any tool that has rounded rectangles and swimlanes claims to be BPMN. Stephen was vague on this, but it sounded like a possibility. However, OMG will not be enforcing or certifying compliance.
BPMN 2.0 — the version that brings in BPDM, choreography, etc. — is still about two years away! The RFP should go out in the spring.
October 17th, 2006
Re yesterday’s post, Yi Gao of eClarus wanted to get this image in his response, so I’m posting the following note on his behalf.
From Yi Gao, Oct 13, 2006:
In eClarus Business Process Modeler 1.0 that was released in June, we could translate the BPMN models with synchronization links. We had some samples that show the capability even though we did not explicitly categorize the models in our website.
The simply structured BPMN models defined as Class 3 can be translated to “readable BPEL” using context-free pattern matching. The technique is similar to computer language parsing based on a finite number of production rules. However, with synchronization link, the context-free pattern matching is not sufficient.
Let me use the following diagram to explain. Without the sequence flows in red and gateway G5 and G6, it can be translated using the pattern matching algorithm.
1) A-B-C-D-E => BPEL Sequence “P1”
2) H-I => BPEL Sequence “P2”
3) G3-(P2 | J)-G4 => BPEL Flow “P3”
4) F-P3-K => BPEL Sequence “P4”
5) G1-(P1 | P4) => BPEL Flow “P5”
(click image for a clearer view)
With the synchronization links, if you try to use the same technique, you need define an infinite number of patterns (production rules) since there are infinite ways to synchronize your activities. However, if you can identify those synchronization links, you can solve the problem by imaging they do not exist first, then adding the BPEL links to flow “P5” after pattern matching translation.
We have an algorithm to identify those synchronization links by analyzing the flows. It is much more sophisticated than described in the example above. And it will be incrementally enhanced to cover more cases.
The mapping algorithm is no longer context free. In other words, you cannot figure out G3, H, G6, I, J, and G4 can be mapped to BPEL flow without looking at a larger scope. So flow analysis is necessary.
In BPEL 1.1, the semantics is clear on “Suppress Join Failure” and “Dead Path elimination”. But it is not that specific in BPMN 1.0. So we intentionally disabled the translation for some models until the specification is clarified in the future. But the algorithm itself is capable if BPMN can align with BPEL on those concepts.
Our goal is to solve real business integration problems. Some patterns in the research papers are very meaningful from academic point of view, and the conclusion can provide guidance for commercial implementations. However, some in Class 1 can be easily solved by adding BPEL extensions, including discrimination pattern, multiple-instance loop, etc. If there is a real need in today’s business, they will be added to BPEL implementations, and even future BPEL specifications. Oracle BPEL extension “flowN” and “exec” are good examples.
October 13th, 2006
eClarus today declared they’ve cracked a tough nut in the BPMN-BPEL roundtripping problem.
If you follow that topic, you probably know that Yi Gao of Seattle startup eClarus and Marlon Dumas of Queensland University of Technology are basically the two guys in the world who know what they’re talking about. Way back in February, when I was still blogging on Ismael’s site, I posted on the roundtripping problem. You can find that thread here on BPMS Watch or here at IT|Redux.
In the comment thread, Marlon responded to my post by describing some of his group’s academic work on the question.
We have recently defined a BPMN-to-BPEL translation that can deal with BPMN graphs with arbitrary topology. In this translation, if the original graph is well-structured, the resulting BPEL code follows the structure of the original graph and everything works well. But if the BPMN graph is not structured, the translation generates rather convoluted BPEL code and the way back to BPMN would be far from easy. For those interested, see this technical report….
There are classes of models for which a reversible BPMN-to-BPEL-and-back translation is possible. We can define three such classes of models:
1) The class of “BPMN models without livelocks”: If a BPMN model (with any topology) is not “divergent”, we can translate it to BPEL using BPEL’s “event handler” feature. You don’t want to read the resulting code though, because it’s just a bunch of event handlers throwing message events at each other. This is pretty much what we show in this paper… where we explain what is meant by a model without livelocks and we give an example of a livelock.
2) The class of “Synchronising BPMN models”: We are currently working on another paper … that contains an algorithm which can determine whether a BPMN model can be translated into BPEL structured activities and control links (without using event handlers, which is what causes the mess). This algorithm could be used for the purpose that you describe: we could allow modellers to write their models as they wish, but on the background we run the algorithm and the algorithm will come back and tell you: Yes, this BPMN model can be translated to “readable BPEL”, or no it can’t.
3) The class of “Structured BPMN models”. Structured BPMN models (where every “splitting” gateway has a corresponding “joining/merging” gateway) can be easily translated into BPEL process definitions containing only BPEL structured activities. An algorithm for determining whether a BPMN process model is structured or not is not difficult to write. Some academic folks from Austria and Denmark have written such an algorithm in another paper written in “runic script”. They do not consider the full spectrum of BPMN constructs, but it could be a point of departure for tool implementors….
For the second and third classes of BPMN models mentioned above, reversible translations are possible, while for the first class of BPMN models, a reversible translation would be very difficult. Note that the first class of models is larger than the second, and the second is larger than the third.
The second class of models is fairly large, and my hunch is that it includes almost any meaningful BPMN diagram that people will be drawing out there. So in summary, there are good hopes that in a not-too-distant future, there will be some reversible BPMN-to-BPEL translations that generate readable BPEL code and cover fairly large classes of models. But there is a bumpy road ahead.
In April, eClarus announced that their tool supported roundtripping for a wide range of models, and provided a link to Yi Gao’s white paper. At that point they admitted to having more work to do. Today Yi Gao claims to have solved Marlon’s Class 2:
In the first release of eClarus Process Modeler we have solved the Class 2 practically, even though the translation can be fine tuned.
Class 1 can be solved using simple pattern matching.
With synchronous links, simple pattern match is not enough. We use a flow analysis technique (we call it “Static Flow Token Analysis”) to identify those links. And our tool can translate them to “readable BPEL” if it can be. If you are interested, we can discuss more offline.
Since not too many people are still following a thread dormant since last March, I’m re-posting here in hopes of advancing the discussion. Marlon has in the past sounded skeptical about eClarus’s approach. I for one am very interested in what eClarus has done and Marlon’s response.
October 12th, 2006
[This is next week's BPMS Watch column on BPMInstitute.org]
A central promise of BPMS is that process improvement can be projected and optimized in advance of implementation, using process modeling’s simulation capability. By including simulation analysis, process modeling tools can not only define the structure of the proposed to-be process but project its expected ROI. For that reason, nearly all BPMS offerings today include some form of simulation tool. But are these tools really fulfilling the promise? Not yet, in my view. Let’s look at what they do, and what’s still missing.
First, let’s deconstruct the promise. As a management discipline, BPM asserts that measures of process improvement should be quantified in the process model and then measured in the implementation. A key assumption of process modeling is that by making reasonable estimates of a few parameters for each step in a business process, a business analyst can estimate the values of key performance indicators even without knowing the technical details of the process implementation.
Those KPIs typically represent one of three types of measurements: times, costs, or some quality metric such as defects. Incremental changes to the process model will generally make some KPIs better and make others worse, and tradeoffs between cycle times, process costs, and defects are to be expected. Thus, the “optimum” process design and configuration is a matter of business judgment.
In process modeling, each activity is assigned to a defined resource, which could be a system or a human task role or workgroup. In BPMS, simulation analysis most often comes into play when a pool of workers performs tasks from a shared queue. In that analysis, the two critical simulation parameters are the average task duration and the hourly cost of the assigned resource. If you configure more resources for the task, backlogs (and hence cycle time) will decrease but costs will increase.
In addition, process models describe branch points in the flow. At each such branch, some of the instances follow one path, and some follow other paths. Thus a third basic simulation parameter is the fraction of instances following each path out of a branch point in the flow. Add to this a fourth parameter, the fixed costs of a process activity (other than those captured in the resource cost), and you’ve pretty much exhausted the capabilities of most BPMS simulation tools.
The first question to ask is what value a simulation engine adds to simple static analysis, such as you could do with Excel or even pencil and paper. Let’s say Task A takes 15 minutes and you have on average 20 instances an hour. It doesn’t take simulation analysis to know that, on average, 5 workers can handle the volume and that adding more will decrease occasional backlogs but add some cost and unutilized capacity.
But if you know not only the average volume of process instances but their profile – for example, the fact that 40% of the weekly volume is generated at 8am on Monday morning – you can get a more detailed picture using simulation analysis. For example, what is the maximum expected backlog at Task A, when does it occur, and what are its effects on service level agreements for the process as a whole? These results require dynamic analysis that only a simulation engine can provide.
Unfortunately, that added value requires simulation capabilities that frequently do not exist in BPMS modeling tools, such as the ability to model instance creation profiles (not just volume) and resource schedules (shifts, holiday calendars, time required for other activities). Also, the simulation model should be able to account for resource contention among multiple concurrent processes; only a few of them can.
Thus, problem number one for simulation in BPMS is the inability to adequately describe the dynamic nature of instance creation and task performance, making it often little better than simple static analysis.
The second problem is that simulation in BPMS does not account for differences among process instances unless they result in different paths in the flow diagram. This is perhaps the fundamental distinction between real discrete event simulation engines and their “lite” counterparts in BPMS.
Consider, for example, an order handling process. Even if a step in the process is independent of who the customer is or the content of the order, the pool of resources available to handle it (and their associated costs), as well as the task’s duration and fixed costs, may be customer- or order-dependent. Moreover, the path followed at any branch point in the flow may be customer- or order-dependent as well, not – as BPMS assumes – a random probability.
This can be a slippery slope, since you don’t want to require defining an entire data model (plus business rules) to do process modeling and simulation. But simulation models should provide at least a few user-defined instance properties and allow them to be referenced in simulation parameters. Instance properties in the simulation would also allow the results to be used for activity-based costing as well as for modeling the tradeoff between time, cost, and defects. For example, an instance might be rerouted to a secondary resource pool with a higher defect rate in order to meet a service level agreement. Instance properties allow simulation to capture quality metrics as well as the costs of correcting defects downstream.
A third problem, usually connected to the process models themselves, concerns incorporation of exception handling into the simulation. What happens, for example, when a customer cancels or changes the order while it is being processed? What happens when a system error occurs and a “business transaction” has to be unwound through an exception process and compensation? In traditional process modeling, these conditions were generally omitted from the process diagram, often considered implementation details. But if you believe the old saw that 80% of the cost comes from 20% of the work – the exceptions – you see that approach cannot lead to an accurate simulation result.
New modeling standards such as BPMN include exception conditions as first-class constructs in the model – represented by so-called intermediate events. Today, few BPMS vendors support these constructs, and even fewer attempt to include them in simulation. An intermediate event attached to Task A means if the event occurs, Task A is terminated immediately and the exception flow emanating from the BPMN event symbol is triggered. The event could be attached to a subprocess or even to the process as a whole. The simulation parameters for intermediate events would need to include a probability of occurrence, and perhaps a time of occurrence (mean and distribution). Moreover, the simulation engine would need to follow the event-handling logic of the BPMN model – which most simulation models cannot do today.
For most BPMS vendors, modeling and simulation were an afterthought, since they are primarily in the runtime business. To be fair, striking the right balance is hard. Process modeling is supposed to be one of those anyone-can-do-it steps, and BPMS vendors don’t want to create a huge learning curve to get started. But to fulfill the core promise of simulation – projecting KPIs resulting from implementing the to-be process – BPMS needs to beef up the tool in these three areas: instance volume and resource availability profiles, instance properties, and handling of events.
October 9th, 2006
