Archive for December, 2006
webMethods, who at the beginning of this year couldn’t even break into the BPM analysts’ magic circle/wave/whatever, ends 2006 taking top honors in the Forrester Wave for Integration Centric BPM. For you non-subscribers, you can get the report from the webMethods website. webMethods has put a lot into its new version of the offering, part of the Fabric 7.0 suite. In addition to a real SOA platform under the covers - a big part of why Forrester liked it - webMethods has put in some BPM features that will knock your socks off. A unified Eclipse design environment with separate perspectives for business analysts and develoers, no-code drag-and-drop design, including a full-Ajax slick task UI, an enterprise metadata library - result of the Cerebra acquisition, fully integrated Blaze Advisor for business rules management (they OEM it so it’s truly part of the BPMS), and what has to be the best BAM - by far - in the BPM space. Maybe even in the non-BPM BAM space as well.
I’m hoping to add webMethods - along with some others who did well in the Forrester Wave, like Oracle and Tibco - to the 2007 version of my BPMS Report series on BPM Institute.
December 22nd, 2006
A couple months ago I posted about the deficient simulation capabilities of most process modeling tools. More recently I’ve been working with ITP Commerce - the tool provider for my upcoming BPMN training - on enhancing their simulation features to address the use cases that figure most prominently in process analysis. It’s been a really instructive exercise, and I’m still struggling through it. I’m interested in hearing about the experience of BPMS Watch readers who have used simulation successfully. From my own process of thinking about the problem, here’s what I’ve come up with.
First of all, I haven’t found any books on process modeling or tutorials from modeling tool vendors that talk about how to actually use simulation analysis in a practical way. (The tool vendors tell you how to set the parameters and click the buttons, but that’s another thing entirely.) So I’ve had to imagine general classes of use cases based on the kind of analysis that process consultants talk about, the kind of simulation that the tool vendors seem to assume, and what users imagine they are getting from simulation.
I’ve come up with 3, and I’m building my BPMN training around the particular modeling and simulation patterns associated with each one. Call them simulation use cases 1, 2, and 3. I’ll post on Use Case 1 today, and follow with the others in the next day or two.
Use case 1, which I think is the most important, focuses on the general type of “process improvement” expected from scrutiny of the modeled as-is process (whether using simulation or not). Usually the key metric is end-to-end cycle time.
Use case 2, which maps more or less to what the modeling tool vendors are offering, assumes a production workflow scenario where bottlenecks are based on contention for resources, and are addressed by optimally configuring staffing and shift calendars at specific worksteps. The metrics analyze the tradeoffs between cost and cycle time for various configuration scenarios.
Use case 3 represents a capability users think they are getting from simulation analysis, but almost never are: activity-based costing (ABC). Activity-based costing is a technique of fairly assigning fixed or indirect costs to different classes of work, in order to calculate the “true” cost of each type of work.
The important BPMN features that simulation must support are different for each use case, as are the key metrics of their respective simulation reports.
Use Case 1: The problem is the current process takes too long. The problems can usually be traced to things such as handoff delays between organizations, inefficient ping-ponging between participants, sequential activities that could be done in parallel, manual tasks that could be automated, and a preponderance of batch activities, ranging from overnight mainframe runs to interoffice mail pickup and delivery. In books on process modeling, such as Sharp and McDermott’s Workflow Modeling, the basic difference between the as-is and the proposed to-be process is cleaning up problems like these.
Simulation issues for use case 1 include:
- Activity duration, active time vs total time. In some simulation tools, activity duration is a single parameter. But an approval that occupies 2 minutes of a manager’s time might still on average take 3 days to complete. If you assign the Approve activity a mean duration of 3 days, is the simulation engine going to interpret that as 3 days of active time, so you would need to artificially configure lots of managers in order to avoid contention for that resource? Or, the simulation tool might allow you to disable contention for resources for the model, avoiding that problem. For instance, ITP Commerce Process Modeler lets you do that, but that disables the resource’s shift calendar as well. I don’t want my simulation engine assuming the manager is Approving at 4am. Process Modeler solves the problem by providing 2 independent duration parameters for an activity - a Wait time that does not consume resources, and an Active time (after the Wait) that does. It preserves the shift calendar and effectively solves the problem.
- Handoff delays. You can model this in three ways. Either build it into the Wait time of the activity following the handoff, as described above, or if your tool doesn’t support Wait time, create a dummy activity with a zero-cost resource to model the delay. Alternatively, use a timer intermediate event in the sequence flow, set to wait for a specified duration. Of course, your simulation engine needs to support timer events.
- Batching delays. For once-daily occurrences, such as batch computer runs or Fedex shipments, you can again use a timer intermediate event, but setting a specfic time rather than a duration for the timer. If your simulation engine handles it (Process Modeler does), it’s a really handy one. What about batching that occurs more than once a day? Such as the interoffice mail cart, that arrives at 9am and 3pm. For that you can use the event-based gateway with a timer event on each leg set to a specific time of day. Again, if your simulation engine supports it (Process Modeler does), that one works great.
- Resource shift calendars. Even though this use case rarely involves modeling of resource contention, the cycle time does depend on the working hours of various staff roles. The simulation tool should specify shift calendars for each role or group, i.e. days of the week and hours of the day that the resource is available to perform tasks. Without that, the cycle times won’t be right.
- The metric you’re usually looking for is the mean cycle time, plus some cycle time statistics. Process Modeler gives me mean and maximum cycle time, and with a little custom Excel work I can get more, such as a histogram of cycle times. I’d really like the histogram out of the box, but maybe that’s being greedy.
December 18th, 2006
Books on process modeling generally warn against getting bogged down in detail. They tend to recommend a top-down methodology that starts with a big-picture end-to-end view and drills down just as far as you need for your modeling purpose. In preparing my Process Modeling with BPMN training I stumbled across a pretty good recipe for how to do this in Worflow Modeling by Sharp and McDermott - one of the only books on the topic I can recommend. My training overlaps slightly with the information-gathering phase that Sharp and McDermott talk about, but mainly focuses on how to put that knowledge into BPMN, so you can share it, run it through a simulation engine, or even generate an executable implementation.
Subprocesses are a very handy concept in BPMN. Besides providing a natural way to draw a condensed top-down view with drill-down to any level of detail, BPMN subprocesses also determine the boundaries of when an event can be received - a change to an order in process, for example. But I find that one really basic property of BPMN subprocesses gets in the way of top-down modeling, as described by Sharp and McDermott, or anyone else for that matter. That’s the basic fact that a subprocess in BPMN has one entry point and one exit point. Let me illustrate the problem.
Here is a familar diagram of an order handling process in BPMN - three subprocesses in a sequential flow. But when you drill down, you typically want to say that if Check Credit fails, you terminate the order. Something like this:
Unfortunately the diagram above is not legal in BPMN! A sequence flow cannot cross a subprocess boundary. Hmmm. Let’s look at the options. I’m not crazy about any of them, but I’m hoping BPMS Watch readers will help me recommend one as the “best”.
The first option, shown above, is to separate the top-level subprocesses with gateways. But to show the semantics you have to use a task to set the end state of the subprocess that is tested in the gateway. That means business analysts have to think like programmers. I don’t like it.
The second option, shown above, is to use Link events. You’ve never heard of Link events? They’re in the spec. A Link End event is a GoTo to a Link Start event or Link Intermediate event with matching LinkId in another subprocess of the same parent process. That’s what we have here. In fact, each of the top-level subprocesses can have a Link End that jumps to the End subprocess to kill the order. So I kind of like it, except that I heard a talk by Steve White who said OMG didn’t like it and they were going to replace it with something else in BPMN 1.1. The other thing is that my Process Modeler tool’s simulation engine doesn’t handle it today. But if you all like this, maybe they’ll put that in.
The third option uses an Error event throw-and-catch. Programmers get it, but I’m not sure business analysts would. Besides, I don’t think of bad credit as an error, but maybe that’s just me. So if the credit is bad, an Error Intermediate event in normal flow throws the error, and the attached Error Intermediate event catches the error, terminates the subprocess and triggers the exception flow, which terminates the order process. So that works, but I don’t like the idea of asking analysts to deal with this throw-catch business. Also, I’m not sure my simulation engine works with it yet, but again… they might, if this is the best way to do it.
So what do you think? Don’t be shy, all comments welcome.
December 15th, 2006
Recently a BPMS Watch reader wrote with some questions about modeling ITIL processes in BPMN. ITIL is an industry standard framework for managing IT services. It describes a number of interrelated business processes, including Change Management and Release Management. These processes are linked, but not in a one-to-one correspondence - one release does not correspond to one change - and it thus provides an interesting real-world example of the 1:N problem in BPMN, which I also call the batching/debatching problem.
In BPMN (or any orchestration language), an instance of the process represents one “thing.” But sometimes that one thing splits into N independent instances of some subprocess, or you may have N instances that are collected at some point into a batch - 1 thing - that continues from there. In the ITIL Change Management process, a request for change (RFC) can represent either a Business RFC or a Technical RFC. A Business RFC is ultimately resolved into a set of Technical RFCs that must be implemented and tested before being deployed in the Release process. In other words, if an instance of the Change Management process represents a Businesss RFC, that instance splits into N instances of the same Change Management process for a Technical RFC. (You could simplify this by saying Change Management for a Business RFC is a different process than for a Technical RFC, but that’s not how ITIL - or perhaps just my correspondent’s company - has modeled it.)
The resulting diagram is shown here. A couple things are worthy of note here. One is the use of paired message events (send and receive) to communicate with both the Release process (not shown here) and other instances of the Change Management process. The pairing is indicated by the label of the event. For example, the trigger for the Change Management process is an RFC event. But note in the ForEach subprocess, each instance of this multi-instance subprocess generates an RFC event, so this model is actually recursive! Also, the end event for the Change Management process is RFC Status, and each instance of the ForEach subprocess waits for the RFC Status event for the Change Management instance that subprocess instance started. It makes your head spin a bit, but it works.
In a choreography diagram, you could actually draw 2 pools of Change Management, I guess, showing the message flows between them represented by these paired events.
The second interesting thing (to me) is the use of the multi-instance subprocess to solve the 1:N problem. In BPMN, an activity (task or subprocess) can optionally be defined as a simple (While) loop, a multi-instance (ForEach) sequential loop, or a multi-instance (ForEach) parallel loop. The last of these is what is indicated here by the parallel bars (VCR “pause” symbol) at the bottom of the activity. It means generate N instances of this subprocess, and the activity is not complete until all N instances complete. In this case each instance represents a Technical RFC generated from a Business RFC, and the subprocess spawns a new instance of Change Management for the Technical RFC and waits for RFC Status.
Note, by convention I only use a message intermediate event in sequence flow to mean “wait for the event” - the BPMN spec also allows it to be used as “send the event”, but I prefer to use a Send task for that. In this diagram, to make the event pairings clearer, the Send task is followed by a message end event in parallel with the continuation sequence flow.
For my BPMN training I am beginning to think of a number of these batching/debatching patterns and how to model them properly in the notation. Now if I can only get my simulation engine to work with it…
December 7th, 2006
[This is next week's BPMS Watch column at BPM Institute.]
When you get started in BPM, the first step invariably is documenting your current, or as-is, process. You can gather the facts from process participants and process owners in a variety of ways – in a group, putting yellow stickies on the wall, or in separate interviews. But eventually you face the challenge of reducing that collected knowledge into a structured, semantically precise yet intuitively understandable, diagram – a process model.
The majority of process “models” in the wild are simply Visio diagrams, applying shapes and lines as the individual modeler sees fit. That simplifies the task for the individual modeler, but diagrams without precise process semantics tied to a structured notation have limited value. They are not immediately understood by others without a detailed accompanying narrative, making it harder to achieve the number one goal of process modeling, which is shared understanding of how the process works. Moreover, these unstructured models are merely descriptive, and cannot be used to project process performance using simulation analysis. For that you need both a structured notation and a methodology to use it correctly. Nor can they be used to fulfill BPM’s “Third Wave” promise, that a model of a new and improved process can be used to actually generate an executable implementation.
All right, you accept that your old seat-of-the-pants Visio diagrams are less than optimal as process models, so you look for a “real” modeling tool. You quickly find that there are a multitude of them available, each with its own diagramming notation, usually some variant of a flowchart or swimlane diagram. These tools tend to fall into one of two categories. Full business process analysis (BPA) tools, such as IDS Scheer ARIS or ProForma’s ProVision, make use of a welter of interlinked models, of which the diagrammatic process model is just one. A variant of this category, blending into the more IT-oriented enterprise architecture toolset – tools such as Casewise Corporate Modeler or Telelogic System Architect – also makes the process diagram just a small piece of a much larger modeling effort.
Products in this category are rich and complex, and cost thousands of dollars per seat. They do more than just descriptive modeling, but typically include KPI definition and simulation analysis (sometimes linked to actual business activity monitoring), and even generation of skeleton executable BPEL from the model. They are generally offered with extensive training, on both the tool and its accompanying methodology.
The other category is typified by the modeling tools built into most BPM Suites. They are much more stripped down and focus attention on a single model, represented by the process diagram. They are essentially “free” – although with most of them, you have to buy the BPMS first! They typically require no training, and have no real methodology for how to use them properly from a business perspective. Most of them provide simulation analysis as well as generation of executable artifacts used in the BPMS design environment. Because they are tied to the inner workings of the BPMS process engine and its associated execution language, they tend to omit notational patterns that the process engine does not support and add non-standard features of the BPMS environment. Thus even those tools that claim to be based on modeling standards usually implement just a subset of the standard plus some non-standard elements.
Today the term “process modeling standard” has come to mean the Business Process Modeling Notation from OMG. Unlike the BPA tools, BPMN concentrates process understanding in a single model, the business process diagram. To me, that’s much more effective for shared understanding than 12 interlinked models. Like most structured modeling notations, BPMN is based on traditional swimlane diagrams, but adds an innovative twist enabled by today’s IT system architectures: the ability to respond to business events. An event is a signal from some external system or process that something has occurred, or possibly a signal sent by the process to another process or even to another part of the same process. While process flow in BPMN is mostly based on the traditional flowcharting paradigm – after Activity A ends, start Activity B – it also includes a symbol for events. The event symbol can be used in many ways: pause and wait for a particular event, or interrupt an ongoing activity if the event occurs, or both signal and respond to an error or timeout.
In modern service-oriented architecture, events – in the form of SOAP messages – are the way business services communicate with each other. By including events as a standard feature of business processes, BPMN allows processes to interact more flexibly with the external environment, and react in real time to exception conditions. BPMN-based process models thus allow the business analyst to describe behavior unavailable with traditional flowcharting, such as how to handle a change to an order “in flight” – when is that change event valid or invalid, and how is it handled, including the semantic precision required to do simulation analysis.
Unfortunately, events are the part of BPMN that most modeling tools leave out. BPA vendors say they’re too complex for business analysts to understand. (Probably they would also break those vendors’ simulation engines.) BPMS offerings, particularly those coming from a workflow heritage, also may have limited engine support for events, so they too tend to leave them out of the modeling tool. A few have stepped up to the challenge – Appian, Lombardi, and Cordys, in particular. Savvion and Tibco have added partial event support to their downloadable BPMN tools, as well.
An important downside of BPMN is that it has strict rules but no standard methodology. Each tool vendor tends to ignore the rules and assume its own methodology, based on the subset of BPMN that it implements. The BPA vendors just support a limited piece of BPMN, and the BPMS vendors’ methodology emphasizes building the executable implementation, which is beyond where most people starting out in BPM want to go, at least immediately. That means that while BPMN takes a quantum leap beyond traditional process modeling, business analysts are left in the dark as to how to use it correctly and effectively.
I see huge interest among the BPM community in process modeling with BPMN. The range of available tools is large and continually increasing, and they are much less expensive than full BPA. But there is also a dearth of educational material on how to use it, including both adherence to the rules defined in the BPMN specification – something the tool vendors themselves don’t understand very well – and a best practices methodology for mapping the yellow stickies on the wall into the notation. That’s why I’ve been developing a training course on just this subject, which will be offered through BPM Institute as well as through other channels in 2007.
For that training I wanted to find a tool with the following qualities: a) supports full BPMN, including events and business transactions; b) provides simulation analysis, including support for events; c) independent of a particular vendor’s BPMS; and d) free, at least for use in the training. I found one in Process Modeler 4 from ITP Commerce, which is an add-in to Visio. It can even generate BPEL, but that’s not the focus of the training. We’re still working on the final touches of the simulation parameters and reports, but getting ready for a formal launch around the end of the year.
I’m hoping that with a critical mass of business analysts understanding how to use full BPMN correctly and effectively, tool support for the standard will improve and become even more widely available. And at this stage of BPM market evolution, that’s exactly what we need.
December 6th, 2006
