What actually means edBPM?

[rainer93138]

What actually means edBPM?

This question was just raised by Paul Vincent in connection with our edBPM/U-CEP workshop this Monday:

http://tibcoblogs.com/cep/2010/12/14/edbpm-really-means-dynamic-agile-event-driven-business-processing/

and I answered (what I would like to repeat here ):

"edBPM as a term was allegedly introduced by Gartner some time after our CEP symposion 2006 and our CEP/BPM/BAM panel there at Hawthorne/NY as far as Opher told me when I was asked 2007 to define a bit the term for the Springer Encyclopedia Database Systems http://www.springer.com/computer/database+management+&+information+retrieval/book/978-0-387-49616-0.

Actually edBPM means two different platforms, BPM and CEP, which run in parallel whereby a concert of collaborating processes is managed by the real-time analysis of an event cloud or event streams and where another event as a result of this Complex Event Processing may be generated and be inserted to a process or more processes whose process flow may be dynamically influenced according to a taxonomy which we sketched e.g. in the paper http://www.citt-online.com/downloads/62750370.pdf. My 2 cents:-)

This is quite more complex than to model and manage a single business process, even between different partners and enterprises on the basis of swimlanes and pools. We also distinguish there between deterministic and non-deterministic approaches for use cases like fraud management where we have to react on so far unknown, but suspicious event patterns which must trigger accordant processes to manage such situations adequately and in near time.

All such challenges are not addressed by BPMN in order to model such use cases and not by BPEL in order to make such use cases executable.

This was actually the job of table-1 at our workshop http://forum.complexevents.com/viewtopic.php?f=13&t=278 because especially regarding U-CEP enhancements we need advanced modelling and execution approaches for complex dynamics for the fields of cell biology, epigenetics, brain research, global disaster or emergency management etc...."

and Paul answered

"... I hadn’t realised edBPM was a Gartner term - its not one I’ve heard them using recently. The key factor is the business modelling part - which is why BPM is synonymous with BPMN, the modeling notation. And why some customers are using CEP technologies to better model event-oriented business models, usually where they are real-time, dealing with large event rates, and prefer to model aspects like lifecycles (eg state models) over activities (eg human workflow tasks). Of course the approaches are complementary, and used together… see http://tibcoblogs.com/cep/2010/03/04/how-does-cep-fit-into-bpm-and-soa-environments/ for example..."

(BTW: I would like to suggest to discuss Paul's ideas together with our sketch of a taxonomy mentioned above)

And as luck would have it, today also Opher has started a thread on his blog where he got an answer from Alessandro Margara so far

http://epthinking.blogspot.com/2010/12/revisiting-epn.html

and I remembered a discussion we had in the beginning of this year together with Hans Gilde and Peter's/Opher's EPIA book, as mentioned in that thread.

As Opher has already announced today, we'll certainly continue this interesting subject of
"Grand challenges for modelling of Complex Dynamics and for execution platforms of edBPM and U-CEP"
http://forum.complexevents.com/viewtopic.php?f=13&t=278

[opher]

Hi Rainer. About the origin of the term edBPM and how Gartner called it, see here:
http://epthinking.blogspot.com/2010/12/ ... -from.html

[rainer93138]

"Event-driven Business Process Management" (edBPM) is now included in the Gartner hype cycle since 2010:-)
Because I'm not rich enough to be a subscriber of Gartner reports, I got the hint from a customer or a former CITT-sponsor:

Hype Cycle for Business Process Management, 2010
David W. McCoy, Michele Cantara
Gartner Research, 27 July 2010

There, Jim Sinur has analyzed edBPM and sees it "on the rise" and benefit rating as "high" and he lists as "Sample Vendors: Active Endpoints; IBM; Inubit; Microsoft; Oracle; Pegasystems; Progress Software; Software AG; Tibco Software; Whitestein Technologies".

That's the reason why we've called edBPM as already "commodity", e.g. when we announce our edBPM/U-CEP workshops like
viewtopic.php?f=13&t=313

The report is also a nice source of definitions, what edBPM or "Process Registry and Repository" or "Dynamic BPM" and around 20 more terms actually mean. Such definitions are helpfull if we organize edBPM related workshops where accepted submissions should be consistent with the definitions.

The report is also very helpful when we have to develop a BPM roadmap for an enterprise.

[opher]

Right. I found the Gartner report and it does mention event-driven BPM. It is down the rising slope.

Opher

[PVincentTIBCO]

Opher - do you mean it is not up the lowering slope?

[rainer93138]

This is a proposal if “Your Global-Enterprise” would start an edBPM-based project. Then you must typically write a Request for Quotation and you could start with Chapter-1 as follows (perhaps it helps you thinking about):

1. Introduction
1.1 Purpose of the RfQ

The global aim of „your Global-Enterprise“ is to monitor and manage the global enterprise in real time. This means that the enterprise has to be changed from a structural organization to a process organization. Therefore the business of „your Global-Enterprise“ shall be based on

- a better transparency through all the business processes of the National Companies (NatCo’s). Although all the NatCo’s use similar processes, each NatCo has its own process variants because of different reasons like legal conditions, etc.
- a better flexibility in order to react on new business challenges, to add new NatCo’s, new products and services, to add new suppliers, to react on legal changes and to change business processes accordingly, etc.
- a better quality when new or changed products and services have to be delivered better, faster, cheaper and more flexibly.

To achieve these goals „your Global-Enterprise“ is in the process of standardizing and streamlining its Business Process Management (BPM) in the context of correspondingly standardizing and streamlining its Business Support Systems (BSS) and Operations Support Systems (OSS) architecture and application portfolio as well (see term definitions http://en.wikipedia.org/wiki/Operations_support_system). As part of this process, „your Global-Enterprise“ has for instance selected “Your CRM-Vendor” as the standard for CRM and as the main application for retail order entry. As a consequence a lot of the kernel processes of „your Global-Enterprise“ are today started and managed by such applications like CRM or ERP which are “blackbox systems” and cannot be monitored and flexibly managed inside in order to install Performance Indicators for realtime BAM and to react on problems or errors which have to be traced down until to the technical causes and have to be solved in neartime. So far the business processes are driven more isolated and in a proprietary way of CRM and ERP systems or of other “Commercials Of The Shelf” (COTS).

A new department is founded to realize a concept of Business Process Integration (BPI). The main challenges are to

- analyze and document all the processes of „your Global-Enterprise“ starting from the high-level value chain and the kernel processes down to the basic (sub) processes, avoid process variants, make processes reusable and adaptable or customizeable between NatCo’s
- make the process models executable driven by a standards-based BPMS and complemented by the needed technical services, communication styles and parameter details
- instrument all processes and services with sensors for BAM views, their (Key) Performance Indicators ((K)PI) and drill down-features
- monitor and manage all the business processes by a console or automatically according to defined event patterns and in advance modelled situations
- make the todays, more isolated processes collaborative, e.g. between order-, billing- and fraud detection- or fraud management-processes.

This long-term challenges shall be achieved stepwise.

In the current project, „your Global-Enterprise“ intends to standardize a so-called “closed loop” from analyzing and modeling to the execution and the monitoring or managing of the processes in a single standards-based BPMS. “Closed loop” means in this context that all models of the different components have to be orthogonal (free combination of independent models or concepts) and any changes in a model or in a component are also performed and managed automatically in all other components without any lost of semantic information.

Within this context of Business Process Integration for all NatCo’s, „your Global-Enterprise“ is conducting a corresponding Request for Quotation (RfQ) to assess the capability of Business Process Management Suites (BPMS) for its Next Generation Business and IT vision.

Your „Global-Enterprise“ invites the vendors to participate in this RfQ. It takes place on „your Global-Enterprise“ Trading Platform (https://www.esourcing-yourglobalenterprise.com).

The purpose of the current RfQ is to select a vendor / product who best meets the requirements as
described in this document. We expect a product proposal that
• meets the BPI key requirements (functional and non-functional)
• covers as many requirements as possible OOTB
• enables „your Global-Enterprise“ to close the remaining gaps through limited customization
• has an architecture which is sufficiently open to allow for these customizations in a well-defined manner
• includes an end-to-end (e2e) solution concept.

The vendor should therefore address the following points:
• Provide a product based solution to the enabling requirements as described
• Estimate gaps between requirements and solution, describe how to close these gaps
• Develop a high level design describing how to integrate the product in this closed loop
• Clearly identify any areas of risks, provide mitigating actions
• Provide a PoC by demonstrating the product capabilities according to reference use cases given by „your Global-Enterprise“

Purpose of this document is
• to describe the expectations of „your Global-Enterprise“ regarding timeline, content and format of vendor responses
• to provide requirements, a rough solution description and Use Cases (Scenarios) which will serve as a basis for the assessment
• to describe the set-up for the aforementioned PoC in a compact and concise way. Commercial aspects will not be addressed in this paper.

The choice / standardization of a system integrator (SI) is not in scope of the RfQ. The SI choice will be addressed after a product standardization, i.e. after the RfQ has been completed.

Primary target groups for this document are vendor representatives as well as „your Global-Enterprise“ representatives who are involved in this project…
(then follow your RfQ-template)

We will discuss and demonstrate this at our edBPM/U-CEP workshop in October http://forum.complexevents.com/viewtopic.php?f=13&t=313

[rainer93138]

Proof of Concept & “anti-fake stress test” – (ed)BPM assessment

In the PoC a typical, appropriate scenario of “your Global Enterprise” has to be realized or implemented by the invited vendors. How to define or to describe and to set up a PoC is well known. A special job to avoid “special implementations” or so called “fakes” for a special challenge is a kind of a stress test which should be demonstrated by the vendors in the final live presentation, unprepared. Because the term “stress test” is already defined in a different context in computer science (test of the robustness of software by testing beyond the limits of normal operation, http://en.wikipedia.org/wiki/Stress_testing_(software)), we call it the “anti-fake stress test”.

Some challenges for the „edBPM closed loop“

Of course there are a lot of criteria regarding the appropriateness of a (ed)BPM platform which have to be solved and evaluated (I will discuss such a catalogue of evaluation criteria and how to evaluate them in a later posting). Especially if the PoC is not implemented under the observation of “your Global Enterprise” but offsite in the labs of the vendors, “your Global Enterprise” should “stress” the final presentation of the PoC. The following challenges might be typical situations when “your Global Enterprise” has rolled out BPM or even edBPM to all of its National Companies (NatCos), when the processes are living and had to be changed and adapted according to new situations and challenges – very often already after some days or weeks or months, again and again, more or less, depending on your business domain like automotive, finance or telecommunication:

- Insert an additional human task in an implemented and working business process
- Move a process step to a different position in the business process model
- Change a service or its signature (parameters) at a process step
- Change a Key Performance Indicator, its threshold and the accordant, triggered action (KPI as an aggregation of Performance Indicators, and add a new event type and its measurepoint(s))

Of course there are a lot of more which you cannot check during a final live presentation. A very helpful overview of so-called non-functional criteria and of their evaluation – especially regarding the newly added CEP part of edBPM - was recently presented with the tutorial of Opher Etzion, Tali Yatzkar-Haham, Ella Rabinowitz and Inna Skarbovsky (IBM Haifa Research Lab) about “non functional properties of event processing”, DEBS-conference, Yorktown Heights, July 2011 (http://www.slideshare.net/opher.etzion/ ... processing). The criteria like performance, scalability, availability, failover, security, usability, etc. are well known from JEE application servers or from former TP- or (CORBA) object monitor middleware platforms and now to be adapted for CEP and edBPM. To evaluate them in an edBPM PoC would not be possible because of the effort and timeframe, but it must be done later in a prototyping phase or you can check it already on a first basis by papers, reference visits, or certifications like SPEC (http://www.spec.org/benchmarks.html) during the assessment. There is also an accordant recent project BICEP (see Pedro Bizarro’s paper http://wenku.baidu.com/view/a51e383a376 ... fadd0.html). The publishing of the results of comparative benchmark tests is normally forbidden, and there is also a reason why it is not done mostly: we don’t need the strongest, but which is strong enough for our needs. For that we must know our needs, like number and kind of processes and their concurrent instances and so on. There is still something to do in the future regarding (ed)BPM.

What actually is a “(ed)BPM closed loop" at all?

A so-called "closed loop" as a directed graph (means with an arrow at the end of the loop, in the sense of a one-way road) meets only a special situation in BPM: if we model a business process the first time, we can start from the beginning, e.g. with the current toolset ARIS 7.2 – in Germany ARIS (http://de.wikipedia.org/wiki/IDS_Scheer) was a kind of a de-facto standard since around 15 years - for modelling the value chain on levels 1 to 4 to find the kernel processes and then to detail each down to the Event-driven Process Chain (EPC) level.


Figure from Johannes Rost: Vergleichende Evaluation von Workflow-Management-Systemen. Dipl.-Arbeit 2009, Universität Hamburg. http://swt-www.informatik.uni-hamburg.d ... s_Rost.pdf. “Closed loop” as a directed graph between Analysis&Design, Technical Design, Execution&Monitoring without cycles and iterations inside is replaced by a cyclic iterative procedure today, as shown in this diploma thesis as State-of-the-Art.

An EPC model is not executable and must be “enriched”. Or we can convert the EPC model to a BPMN model in order to do so. There are very different BPMN versions 1.0 from 2002, 1.1 from January 2008 and 1.2 from January 2009, the current BPMN version is 2.0 from January 2011 which contains a lot of more specifications, also execution semantics and a BPMN-to BPEL mapping, on a first level, not really final. BPMN 2.0 is not yet supported by some vendors. You must trust their roadmaps, if so. And of course, there are already known challenges for a BPMN 2.x release (http://www.saperionblog.com/tag/bpmn-request/ by Martin Bartonitz) which must be solved proprietarily by a vendor so far. But anyway, in the case of such a complex platform we would always buy a vendor lock-in and cannot wait another decade.

Now we have the EPC model as a BPMN model with the same details and this might raise the question why then EPC is needed at all? In Germany we might assume that “historical” reasons might be important in the case of existing ARIS adopters, with a lot of EPC models from the last years (mostly outdated in the meantime, BTW), who should still be supported and who should not loose these models when they try to make them executable. Is this a challenge for “your Global Enterprise” at all, especially regarding the non-German countries? So, ask them first, perhaps.

Then we have to add the technical details in the BPMN model to make it executable. This is proprietary, especially in the case of BPMN 1.x which does not contain execution semantics, etc. or BPEL mappings.

At last we can add measurepoints and Performance Indicators in the processes and we can on the basis of them define KPI's and show them in BAM dashboards.

This is the so-called closed loop which is actually an "old" or well known idea from publications of A.-W. Scheer and colleagues respectively IDS Scheer AG (since 2009 acquired by Software AG) or as seen from theses like this mentioned above, which is already enhanced for the challenges we must meet today.

The reality is different however

In real situations and in the sense of the “Added Value” which we want to buy with an additional complexity of the software stack - what has to be justified against our procurement department – we want - for instance - to shorten the implementation time of business processes and the time to market. Therefore we want to test the technical model and the BAM views and we want to discuss the test with the stakeholders or process owners in a single onsite or online session. We would have the chance only once to bring all stakeholders from different locations and countries into a single session, because this is the actual and time-consuming challenge, and a problem of the “BPM-governance”-strategy of “your Global Enterprise” (what we will discuss in a later posting). If it would not work in this session, the stakeholders would never attend again the next time, probably.

In such a session we would have to change the process and measurepoints, etc. in some iterations. This changed technical model cannot automatically be synchronized in the EPC or in all the ARIS levels up to the value chain, if so. For instance, there are around 10 start event types in BPMN, around 10 intermediate event types, around 10 end event types – I have not counted them anymore, the last time when I did it for my BPM lecture around 2007/8 I counted 27. Anyway, they cannot be mapped to EPC and vice versa, so if we change a model in EPC or BPMN, we loose semantic information. There is a relatively new, upcoming BPMN community with all the BPMN specialists, also from University of Potsdam/HPI (http://en.bpmn-community.org/forum/24/) and of course the fora of Bruce Silver, Jakob Freund (http://www.camunda.com/), Frank Leymann/U Stuttgart for BPEL mapping and execution problems and so on. There is also some scientific literature about this subject, e.g. Willi Tscheschner/University Potsdam http://bpt.hpi.uni-potsdam.de/pub/Publi ... C2BPMN.pdf, but related to BPMN1.1 and a one-way transformation from EPC to BPMN, in the meantime outdated because of the changes in BPMN2.x).

Make realistic scenarios...

Imagine the situation that “your Global Enterprise” would introduce a new product and would design a process model for the advertisement and the introducing and delivery processes. The vision (and the “added value” of BPMS or edBPM) is to bring together all stakeholders for this new product in ONE session, from all needed countries etc., and to model all stuff in one day, test and change and simulate as long as all stakeholders agree with the process at the end of the day – what today takes weeks and months. Our students or team members and I described this already during this very nice workshop, not yet really edBPM, but based on BPM+real-time BAM already, in Ireland February 2008, our preferred way to learn together http://www.citt-online.com/index.php?id ... 1&id4=more

For that you need a BPMS or an actually already existing edBPM platform that is flexible enough to automatically synchronize all resources in all the areas.

Because this is perhaps not possible today based on standards, “your Global Enterprise” needs a platform provider who is willing to develop these challenges as a work-around as long as there are no standards or available product features right now, but who is stable enough to work with “your Global Enterprise” in a longer time horizon and to develop a concrete roadmap when these features will be available.

...and start today

So, “your Global Enterprise” could achieve the goal stepwise in the next years. The way to that goal would already be a big win-win situation for both sides and help “your Global Enterprise” to standardize and automate its business processes already today, and learn a lot about your business and its processes - a better insight in the global event cloud and the “smart dust” of “your Global Enterprise”.

[rainer93138]

edBPM – centralized or decentralized? Another RfQ challenge

Another challenge to be described in the RfQ is the problem of how to deploy or install the (ed)BPM platform components and all their different servers – depending on the platform vendor - in the network topology of “your Global Enterprise” (http://en.wikipedia.org/wiki/Network_topology), but including its WAN topology (http://en.wikipedia.org/wiki/Wide_area_network). Every NatCo (National Company) has its own back bone (central LAN), but not all of them use the same LAN, sub-LANs and LAN operating systems (e.g. Microsoft Windows Server 2003, Microsoft Windows Server 2008, UNIX, Linux, Mac OS X, and Novell NetWare, see http://en.wikipedia.org/wiki/Network_operating_system). We should decide on which LAN OS the PoC has to be implemented, especially if a PoC is developed in the labs of the vendors, whereby the one is using Windows Server 2003, the other Windows Server 2008 and the next is using Linux.

Another problem might be the machines connected to the LAN’s like mainframes, UNIX-midframes or MS Windows workstations based on different operating systems. (ed)BPM is always based on a Enterprise Service Bus (ESB) middleware which is again based on another middleware, deeper in the software stack like application servers, Object Request Brokers (CORBA), Transaction Processing-monitors, IBM CICS, Tuxedo, etc. As we know from these middleware platforms, it’s very challenging to deploy them in heterogeneous clusters, even some of the code is written in Assembler sometimes because of performance reasons and is machine dependent. This means that we must know the infrastructures of all the NatCo’s, and we have to decide, which infrastructures or topologies should be supported, and we must ask the vendors, what is supported. Another question is then how performant and scalable the (ed)BPM platform would work in such an infrastructure, but we have already addressed this in the former posting. And not at last, such challenges of the RfQ have a tremendous impact regarding licensing models and prizes of course.

The following figures show the more “profane” (from an U-CEP point of view ) problems we have to solve right now. Because we cannot start in the reality on a “green field”, we are first faced with the situation, that some of the (kernel) processes are already implemented and deployed as proprietary solutions of CRM or ERP systems or of other Commercials of the Shelf (COTS) – as black box systems, which cannot - without further ado - be instrumented by measurepoints or Performance Indicators needed for BAM. And not all NatCo’s and your Central IT use the same systems and/or they have different business process variants because of different legal situations in the countries, etc.



We have to face the (ed)BPM vendors with the requirement, whether they are able to support a centralized (more easily to operate) solution or whether their solution could also be deployed in a decentralized infrastructure. In the case of a centralized solution it must be decided how to “harmonize” the "legacy systems" (ERP, CRM, other COTS or applications). In that case all processes would run at the Central IT (CIT), and this should first carefully be discussed with all the NatCo’s - normally in a BPM Center of Excellence according to the BPM-governance strategy of “your Global Enterprise” (see an upcoming standardizing initiative from Open Group http://www.bptrends.com/publicationfile ... man-v5.pdf, “BPM-governance” will be discussed in a next posting).



In a decentralized solution, only the repository with all reference business processes is centrally installed at the CIT, if any exists. This repository would be the basis for all the customized or adapted process variants, which have to be deployed in the infrastructure of the NatCo’s. That way, each NatCo would control its own business processes, and this might help to a better acceptance. What is a reason, which could be discussed under strategical viewpoints.
So, we should first gather all the advantages and disadvantages of each solution, not only technical ones.



We shall extrapolate this discussion from business processes to the Future and Emerging Technologies (FET) of our U-CEP sketch for the European Commission’s Flagship initiative and what it means, if we connect Body Area Networks (BAN), Wearable Technologies, BioSensors, Selftracking and all the Human Enhancement Technologies as new services at our edBPM/U-CEP workshop in October 2011(viewtopic.php?f=13&t=313 and viewtopic.php?f=13&t=303).

[PVincentTIBCO]

The good news is that pretty much all the BPM solutions out there have standardised (SOA type) interfaces at runtime, and designtime can be abstracted to a high level process description [*1]. So an edBPM overall solution (if indeed that is a goal) can indeed communicate with existing business processes.

However, there are 2 main roles for edBPM today:
1 - high performance business automation, where "high performance" is aligned to event processing (both in throughput/latency AND capability), and "business automation" includes decision rules and actions.
2 - end-to-end business monitoring (a.k.a. BAM), where monitoring includes systems, processes and applications

Notes:
[*1] There was some hope that a "meta" process standard such as OMG BPDM would allow the abstraction of different process types. IMHO the last version of BPDM failed at this (it did not seem to be able to abstract non-orchestration models like rule-defined processes) although the main reason for its lack of popularity is that the BPM community has been focussing on BPMN2. It remans to be seen if BPMN2 actually provides sufficient modelling capabilities for edBPM or CEP - personally I doubt it - or whether the BPMN community moves on to cover things like ECA rules...

IBM Research presented an interesting paper at DEBS11 last week on "business modelling", which aligned with their submission for the OMG Case Management standard, using ECA rules and a non-standard state representation model. This aligns with what TIBCO have been deploying with TIBCO BusinessEvents and TIBCO BPM in the AFF solution (goals, rules, states, and processes) primarily targeting telco fulfillment solutions. One can easily imagine these sorts of capabilities (ECA rules, advanced state models etc) merging into a BPMN3 standard to cover edBPM quite nicely...

[rainer93138]

Thanks for interesting comment, Paul.
As I mentioned above: nonetheless that we need a new notation or a next release of BPMN 3.x, we can and should start our edBPM projects already today and the platforms should be mature enough in the meantime, also according to the Gartner Hype Cycle report mentioned above - of course always a bit enhanced by proprietary add ons. This is similar to SQL-based database management systems of the last decades, which have done a great job, despite of the fact that every DBMS provider has added its proprietary enhancements in order to make the DBMS faster or better, etc.

The next BPMN release - if we use the same procedure of standardizing as always - will take around 5 years, what is the usual timeframe for a major release, when the number before the dot is changed.