What are your predictions for CEP in 2009?

[hgilde]

What say ye?

I doubt that SMB's will want to work with events in real-time. I think that the farthest they will want to go in this area is Splunk for log analysis.

First, real-time processing adds significant risk over stored data. More stuff can go wrong and the system will need more care and feeding. That means more staff, and the SMB IT staff (a few people) doesn't want to deal with this.

Second, a SMB already needs a database, so they have someone marginally skilled in databases. They want to leverage this skill set by putting everything through the database. They don't want to get into the new skill sets required for real-time processing.

Third, stuff just doesn't move as fast in an SMB. They do not run into the cases where database hardware is too expensive to process their fast data flows.

Just my $.02.

[hgilde]

Sure would like to see an example of these, even a small example?

I'm probably not the best person to talk about patterns because I have not been researching them. But I will give it a shot to get things started. This is based on conversations over the past year, and different folks may see things from different perspectives.

The motivation for patterns comes from the customer. The customer needs help understanding how to use this new technology and how it fits in to their world. As you (David) wrote recently, the CEP market is sales driven and so may not align directly with customer needs. Each vendor cares only to train the customer how to use *their* product. But if you talk to the folks on the ground, in pre-sales support and professional services, they will confess that they would love to see customers having a better understanding of EP, even if that meant more awareness of competitors. From this, I interpret that a good push for patterns will meet success.

Ask an architect about what patterns they use in their real-time infrastructure and they'll talk about the bus, processing agents, services and maybe (if they are masochistic) distributed transactions. These are architectural patterns and they give limited options for expressing the logic of a business application.

But think about the event meta-language that Opher has mentioned before. This is probably full of patterns. You can first describe your specific logic needs using these patterns, then look at available software and judge how well it works with these patterns.

Or here are some lower level idiom ideas:
* Windowed batch processing - gather a set of events over some window (time, count, whatever) and then trigger a process over them (say a regression function).
* Windowed incremental processing - maintain some state for a window and incrementally modify that state for each event, but don't keep all events (maybe find the mean of something over a time window).
* User managed state - state that is maintained over time that is modified by the code of the application (rather than the state that is inherent to, say detecting a declared pattern).

I can take your logic needs and discuss it in terms of these idioms. There are many, many more and the ones that I give here may not even make a final list. Some can be pulled in from existing patterns, some are fairly specific to real-time processing.

Then there are patterns for certain vertical markets (pardon me for the focus on finance here):

* Real-time updated, locally queryable order/positions book. Many trading applications must, at a fundamental level manage their books of positions and open orders. And it pretty much works the same way for all applications. So this is a pattern that can be discussed and can be coded against.
* Patterns for FIX processing (well known in FIX circles).

Again here, there are plenty of patterns for the industries that use lots of events. Of course they are specific to certain applications, but are very worth talking about for people in those areas.

As I said, I have not thought this out thoroughly. But I do know that everyone I know on the front lines of building EP applications like the idea of patterns in some form.

[marco]

If you look at all (not so many...) companies that claim that they do some kind of event processing I think I can see a trend...

Those that seems to do fine are those not selling the technology, but instead focusing selling solutions to a particular problem. From what it seems, there are problems to solve on Wall Street which are a good match for CEP.

I predict that those that will survive 2009 are those that talk more about solving problems for the financial sector than talking about event processing. I would not be surprised that the marketing messages will talk less about event processing and more about solutions for a variety of financial problems in the end of 2009 for those companies that make it.

If we read this in early 2010 I think that the web pages of several of todays companies are not presenting themselves as CEP companies but "financial solution providers", or whatever the correct term is.