Event Processing Glossary, a new version

[PVincentTIBCO]

All analytics algorithms are pretty specific about what they do, what they look for and what problems they are good for. Even algorithms that search for a variety of patterns have a specific set of patterns that they can possibly find. Meaning that there is no algorithm called "find some patterns" that looks at your data and locates all the best patterns for your convenience. Now you could go ahead and talk about analytics on events, the way IBM is talking about Intelligent Event Processing. But we should all keep in mind that there is really no such thing (in the sense of a concrete thing or idea) as general "analytics" and so this terminology is only applicable in a very broad sense.

Well, I guess this is why EPTS *might* want to strawman the terminology for event processing usage

For example, I refer to event analytics as any analytic method that identifies a new process, rule, query etc, or combination thereof, from the viewing of events as they arrive from an event cloud. Compare and contrast to conventional analytics that view only historic events or data... what I really want in an EP world is to continually assess incoming events to derive new information.

It could be that this area is too new or novel (or a different domain) for EPTS to consider. For example, in TIBCO we might use rules to identify changes to rule parameters or continuous queries. These rules can be event driven (ie execute statistics including S+ operations on historic records vs the new event information) to determine what changes need to take place to rules. This is a subtype of "analytics" and is particular to CEP (event driven, and creating event-processing rules or queries / changes to these). But is this sufficiently different from creating a decision tree from a set of data to justify new terminology?

On the other hand, as Tim Bass keeps reminding us, for CEP application areas in security, identifying new event patterns is just as important as detecting the instances of these patterns. So it would seem to be a relevant area...

[hgilde]

Yeah, I guess that when it comes to the glossary, I am not clear on the goal. I mean, we could go back and forth parsing out the ontology of analytics until the cows come home (event type: arrival of cows; location: home). But without knowing why such a thing is being done, I don't know how to choose between any two reasonable approaches.

[DLuckham]

Hans,
Am I misunderstanding something in this thread?

I guess that when it comes to the glossary, I am not clear on the goal

My original request for suggestions said:

The Glossary is intended to define a small set of commonly used basic terminology for event processing.

I dunno how to put it any differently!
At present "event analytics" is not in the glossary.
But Paul's suggested usage:

For example, I refer to event analytics as any analytic method that identifies a new process, rule, query etc, or combination thereof, from the viewing of events as they arrive from an event cloud

would seem reasonable. One question is whether the term is "commonly used"?

[hgilde]

Ok, then I don't mind Paul's description and I also don't know of it's common use. Before the glossary, I also don't know of any common use of the term complex event either. So that goes to show how much I know about commonly used terms. Even if Paul's term is not commonly used, should it be included in the glossary? That's the kind of decision that I don't know how to give input on - one opinion says to include it because it might help clarify things in the future, another says not to include it until people start to use it.

Of course, there are plenty of commonly used terms from the worlds of ML, statistics, etc. But I'm thinking you don't want to start including those in the glossary.

[Tim Bass]

Hi Tim, I'm not sure if you checked, but the google search you referenced above seems to be all academic papers. Nonethless I concur that there are many OR techniques that are yet to be "productized" in general CEP software technology...

Cheers

Hi Paul,

I am not following what you are trying to say, Paul. First, you seem to be saying that because I quickly did a Google search with a set of keywords and most of the hits are academic... has some meaning in this discussion context? Normally, there is both a lot of research and then application in a field. Yes, there is a lot of research, mountains higher than what exists in the space of the buzzword airwaves of 'CEP'... so does all this prior art have some context? Does the lack of formal methods in CEP buzzword land have context? I hope you are not implying, as it seems when I read your reply, that years of knowledge is somehow inferior to a book, a buzzword or two, and a lot of marketing?

At your other post with the phrase 'event analytics'..... you asked my opinion.

My opinion is that complex event processing is the same as event analytics, so conjuring up another term to support a weak, confusing, and meaningless (the current) definition of CEP, is not going to help. As I mentioned, the foundation of the house is weak, so piling on more on top of it is not really going to help.

My other is that almost all of the prior art exists that covers what CEP is trying to market and accomplish, so you don't need new terms, you simply need to use the prior art and knowledge that already exists. It is precisely this seemingly "we invented real-time detection" attitude (a marketing view) that hurts the credibilty of CEP/EP.

Basically, some folks are just tossing out 30 decades of prior art in real-time detection theory for the sake of a buzzword with all the marketing positioning. This is that old "not invented here" that frustrates most of us, from time to time.

This glossary is an exercise that supports marketing some software, not furthering the state-of-the-art of processing complex events. Your idea of what is "on topic" and what is "off topic" proves my point above. (Laughing)

Yours faithfully, Tim

[Tim Bass]

Deleted.

[Tim Bass]

Deleted

(Something is strange on this board. When I edit a post, it creates a new message. Then, I can't delete the old ones.)

[Tim Bass]

Hi Tim: I like complex event above, but complex event processing above seems to need a lot of work (e.g. identifying meaningful events within the event cloud ... surely if the event cloud is just messages, then this is just filtering, not abstraction? etc)

Dear Paul,

Hint: Study the CEP/EP reference architecture that TIBCO published .....

Also, please note that engineers generally use the term "messages" to represent the entity that transmits information, in this particular context, an event-object. The same is true in biology, as the event and sensory information is called "messages". The brain processes "messages" from the rest of the body. Hopefully, doing more than just filtering!! LOL.

Cheers.

[PVincentTIBCO]

...My opinion is that complex event processing is the same as event analytics, ...

Hi Tim: just to be sure, you are saying that, in your opinion, CEP is about the identification of new event pattern TYPEs, not new event pattern INSTANCES?

For the record, its my opinion that both instance detection and new type detection are covered, with the current emphasis on the instance detection in current technologies, and the "event analytics" part being an area of potential future research for general-purpose CEP tools. Of course, I could be wrong and it could be that everyone else agrees with you on this! However, as far as I know, the existing EPTS definitions do not cover this area... hence the suggesting for the terminology extension.

PS: TIBCO's CEP technology, BusinessEvents, uses production rules and dynamic queries for primarily "complex event instance detection". However, both rule parameters and indeed whole queries (defining set-based facts from observed events) can be adjusted by other rules and processes inside a BusinessEvents application, providing a form of "event analytics". Conventional predictive analytics models such as those provided by TIBCO Spotfire S+, SAS and SPSS can also be used to define for example decision trees that can be imported into BusinessEvents, although for the most parts these are still identifying patterns from existing "types" rather than whole "new types" of complex events. Neural nets can also be used, but again these generally need "training" against known result sets, which means they are less "predictive" than one might expect.

Cheers

[PatternStorm]

There are doubtless things that ought to be added, so here's your chance!

The following terms,

Situation
Context
State
Transition

in no particular order, but often going hand in hand, have generated not a few number of situations of confusion, disagreement and missunderstanding in many event processing discussion situations in the blogosphere. Arguably this situation may be caused by the lack of precise definitions for these terms under the context of event processing which leaves no option but to interpret these terms as defined in other contexts such as natural language, formal automata theory, situation theory, etc.


Do you think we should be providing a precise definition of these terms in the Glossary?

Thank you very much!

Regards,
PatternStorm