PMML - The Predictive Modeling Markup Language - is the primary XML format for describing predictive analytic models so that a modeling tool can share a model with either another modeling tool or, more usefully, with a deployment environment. The folks over at KDNuggets recently ran a poll asking their readers about their use of the standard (the results are available here). While use of PMML is still pretty limited, with half the responders saying they don’t use it at all, I was encouraged to see a solid core deploying most of their models using it (15%). There were some interesting comments, too, showing the usual range from people who don’t like XML, to those who prefer deployment direct to specific code to those who like the idea of a standard but have concerns about the specific.

As someone who believes strongly that models are most valuable when both deployed and mixed with declaractive rules-based logic (for policy, regulations etc), I hope PMML will continue to gain ground. I am encouraged both by the number of modeling tools that generate it (including SAS, SPSS, Fair Isaac,IBM, R - open source) and rules-based environments that can comsume if (Fair Isaac’s Blaze Advisor, Pegasystems PegaRules and Zementis ADAPA, Chordiant Decision Management).

Any readers got comments on PMML? Anyone using it out there? I’d love to hear from you.

Steve Cranford of PwC wrote an interesting piece called Bringing Order to Chaos (brought to my attention by Alan over at Tibco) that made me think. Steve’s focus is on the next software suite for enterprises (something he calls an Intelligent Business Performance Platform) consisting of business intelligence, business process and business rules. Reading this it seemed to me very like the concept of dynamic business applications seen recently from Forrester, though with a somewhat different focus.

While I agree that organizations need a platform for running their business more intelligently and more “by the numbers”, I think the components of such a platform are not BI, BPM and BRM but:

• Performance management
  Using reporting and especially event-based dashboards to actively and effectively monitor ongoing business performance.
• Process management
  To manage and improve the workflow and processes that deliver value.
• Decision management
  To combine policies, regulations and analytic insight into continuously improving decisions.

I don’t like to use “BI” as a component as it is such a broad term and we are mostly talking about dashboards/monitoring and data mining/analytics rather than reporting. I also think business rules is too broad as it is a useful component of many elements (something Steve recognizes later in the article when he talks about all the different things involved). Splitting out the combination of data mining/analytics/business rules/optimization (decision management) makes it clearer, I think, what is required. After all I need to run efficient processes that rely on effective decisions and can be monitored reliably.

I also liked the drivers Steve outlined - a move to real-time, increasingly extended enterprises and a flat world though I have a longer list that I use when discussing the need for smarter systems. I would also add that it is not just about efficiency - actually also about effectiveness.

Steve sees this new platform having many of the characteristics I see with real-time data feeds, better use of data with data mining and analytics not reporting, dynamic dashboards for performance management, business control and all built on a service-oriented platform. Indeed I see decision services as crucial in all of this. Like Steve I also believe that this approach does not require a “rip and replace” mindset but allows you to add value to existing systems - essentially making existing systems smarter (hence the book title).

A final comment. Unlike Steve I don’t think such a platform requires a heroic effort. Lots of rules vendors have integrated analytics with rules, the best way to integrate Business Process and Decision Management integration is pretty well established and several vendors have integrated pieces of the puzzle well. Most of the rest can be done using standard service interfaces, portals etc. The products are, mostly, ready. Whether companies are is another question…

I got an interesting series of questions from a reader that seemed to me to justify a longish post. The initial question was quite harmless looking:

Can you give a clue as to what software engineering approach you use/recommend for EDM, but especially business rules that non-IT staff can alter safely?

But the whole thing got more interesting as they explained some follow-up items:

[is this] ‘just’ putting non-developer friendly interfaces on the process business rules inherent in MDD [Model Driven Development] or something completely different?

Has all this effort been put into business rules engines because of the current limitations of MDD approaches: ie they are a stop gap or an ultimate end in themselves?

Is the software engineering approach of the business rules engine a temporary convenience or something fundamental?

Very interesting questions. Taking on the software engineering approach question first I would refer everyone to Enterprise Decision Management and the Software Development Lifecycle, a post I wrote not that long ago on the topic. Personally I think the Agile approach works especially well as I said in my recent Application Development 2.0 post. To repeat an article I wrote on agile business rules, I find the use of business rules to be completely complementary to the four tenets of Agile development:

• Tenet 1: Individuals and interactions over processes and tools.
  One of the key interactions is between developers and domain experts. The use of busines rules facilitates this conversation.
• Tenet 2: Working software over comprehensive documentation
  Business rules can deliver working software that is easier for domain experts to read and manipulate making it more “self-documenting” and lessening the pressure for documentation.
• Tenet 3: Customer collaboration over contract negotiation
  The fact that both developers and domain experts can read and understand business rules allows true collaboration over the implementation of business logic.
• Tenet 4: Responding to change over following a plan
  Business rules deliver business agility by making the actual code you write easier to change both during the project, and after it.

So, to summarize: It does not really matter what approach you use (they can all work well) but Agile or Agile-like approaches will be highly compatible with using business rules. In particular, fast iterations and test-driven development work well with the kinds of short-cycle changes that business users make themselves.

Now on to the Model Driven Development(MDD)/Model Driven Architecture (MDA) questions.

I regard business rules as a part (and a very important one) of a Model-Driven Development approach or a Model-Driven Architecture (MDD/MDA) as they allow you to specify business logic without the nitty-gritty of code. No approaches to MDD/MDA manage business logic well today - they mostly assume it will still be coded by developers. As long as designers and developers are embedding business rules in use cases, statecharts, class models etc then MDD/MDA will not deliver on its promises. In the context of MDD/MDA a model cannot be limited to things that can be diagrammed - a model-driven approach must include a declarative language for business decisions - something like the current business rules languages. After all, improved business/IT cooperation requires a shared language and when it comes to the details of business logic I have yet to see anything better than business rules. Business rules allow for IT departments and business users to truly work together. As one of the key drivers for MDD is the need to build systems that can cope with faster business change and to do so by improving business/IT alignment, business rules would seem to be ideal. Too many MDD/MDA approaches seem more worried about tracing requirements to code (which won’t fix the problem).

Without a truly declarative language to describe the business logic of business decisions, MDD/MDA is just another technical approach for generating code - a new form of CASE (Computer Aided Software Engineering) tools based on UML. MDD/MDA removes the burden of lots of programming from developers but still expects them to deliver the code that implements interesting things such as business logic and algorithms. But business logic and algorithms require business verification and business users hate code. Adding business rules to the MDD/MDA mix would address this. Furthermore, while projects using MDD/MDA have shown a reduction in time from problem identification to working code, it is still code and the IT department is still making the change rather than the business. Again, business rules would help with this. Finally, some of the benefits stated for MDD/MDA seem simple to someone used to a Business Rules Management System (BRMS) or even of a Business Process Management Suite (BPMS) - easy change of logic or process steps without regenerating the whole application for example - so business rules and business process would seem natural components of MDD/MDA.

Current MDD/MDA approaches do have limitations because they don’t handle business logic the way a business rules approach does (or business process the way a business process approach does). This was not the motivation for current business rules products (they mostly pre-date MDD/MDA being widely discussed) but they remain a hugely beneficial and complementary approach for anyone doing MDD/MDA. I don’t see anything on the horizon that makes me think of them as stop-gap but I would never be rash enough to say that ANYTHING was “fundamental”.

I think the effective mangement and automation of business decision making is now and will continue to be critical for the next generation of systems - helping to make current systems and infrastructure (and models) “smart (enough)”. For now there is nothing better than business rules for this.

Earlier this week I posted Application Development 2.0 in which I addressed what I see as some of the issues with current development practices and tried to explain why I think a declarative, business rules approach is essential. This (and some blog posts around the blogosphere) made me think about the mismatch I see when talking to Architects - most have heard of business rules, few use them. Part of the challenge, I think, is that those of us who “believe” are perceived as fringe thinkers. It is easy to see why - we must a fringe as everyone else is writing procedural logic! Beyond this, though, are serious questions about business rules:

• What are the benefits that justify the additional investment?
• Why wouldn’t I just code the rules?
• Will the rules perform, integrate and “play nice” with the rest of my system?

and so on. When asked why he believes, Chris Berg of ILOG listed Encapsulation, Simplicity, Visibility, Collaboration and Shared Execution (SOA) as the reasons to believe in BRE. In a post on my ebizQ blog recently I listed a similar set of reasons:

• The separation of decision logic from mechanical implementation gives you more flexibility to make changes with minimal or zero impact on basic systems operation.
• Business rules are more understandable to business-level people, leading to better business/technical cooperation, reduced implementation times, and fewer opportunities for interpretation errors.
• Business rules are easily segmented into groups for control over functional interaction and management.
• Business rules management systems have interactive testing, execution flow, cross-reference tools, and reporting features to aid in development, testing, and documentation.
• Business rules management systems have predefined rule replacement features to handle system updates without interrupting service to application users.
• Business rules can have explicit times and dates when they should go into and out of effect.
• Rule management templates can be created to let users update, view, or create rules in a controlled manner.
• The rule engine can quickly look through large sets of rules, finding the proper ones to apply based on case-specific conditions.

Now, there are those who believe that the purpose of business rules is replacing developers with business people - development without developers. I am not one of these people as I believe the value in business rules is in allowing collaboration between two groups with different, but valid perspectives. Yet this difference of perspective often results in serious problems when developing complex systems. Business users are at the mercy of regulations, court rulings and business policies that must be enforced. They must also respond to an ever-changing business and competitive environment. This often means they can’t stabilize their requirements or explain them easily to developers – legal or business jargon does not always map well to Java code! Meanwhile the developers typically don’t understand the regulations or business environment well enough and so can’t develop necessary applications, systems and updates fast enough. As an Illustration let’s take the example of a healthcare benefits system designed to who is eligible for what. First, the business Perspective

• Business users don’t need to manage objects, interface to existing systems, package up and deploy working code – IT still owns this process
• IT can write technically complex rules in a syntax that business users can validate
• Knowledgeable users can write regulations as rules using a simple, English-like syntax
• IT can develop templates to constrain rules that business users want to change – templates prevent problems but allow for a high degree of flexibility
• IT can do architectural design, integration etc. ensuring that the resulting application can be supported and integrated easily
• Business user rule maintenance frees up future development time that would be tied up with system updates

In fact, business rules offer as much to developers (perhaps) as to business people. By allowing non-technical folks to collaborate, developers will find it easier to build applications that make their users happy and so increase the rate of positive feedback. By pushing the nits of maintenance to the business users they will find more time to work on “cool” stuff.

This post was prompted by a number of posts I saw today:

• The Holy Grail - Application Development without Developers
• I Want To Believe Redux
• Business Rules Engine - I WANT TO BELIEVE

My friend Steve has been using the ILOG Rules for .NET product (which you can now download for an extended trial as I discussed here) and has written a nice little review of it - Lab test: ILOG Rules for .Net meshes well with Microsoft. There’s a lot to like in the .Net version of ILOG’s product and I, like many others, am waiting to hear what IBM’s plans for it will be when it finishes acquiring ILOG.

Our article on The Nature of Operational Decisions was just published - enjoy!

Ann All had a post on Agile development brings IT, business together that had the great phrase “application development 2.0″. In the article she mentioned some very worthy objectives for this 2.0 version of application development. Here they are, paraphrased slightly.

• Encourage close collaboration between developers and end users
• Involve users in quality assurance processes
• Don’t use traditional programming languages
• Stress simplicity
• Emphasize frequent releases
• Users, not developers, should determine new features

I will come back to the bit about “Use dynamic scripting languages like Ruby, Python and Perl” in a moment. Back to the list.

I am struck by the inherent contradiction between this list and traditional development technologies - code, to be blunt. How can we expect close collaboration between developers and end users if the developers are using a language (Java, C#, Perl) that the end users cannot read? How can users do quality assurance on code they can’t read? Perhaps users can be the drivers for new features, but wouldn’t it be better if they could actually so something about the features they want? How frequent can releases be if the code must go through the usual QA/test/deploy sequence?

The answer, I think, lies in “Don’t use traditional programming languages” though it is not “Use dynamic scripting languages like Ruby, Python and Perl”. As I noted before, this very focus on programming and programmers is a barrier to progress and replacing one procedural language with another won’t help because, as Ira Fuchs put it:

Programming languages today remain syntactic, abbreviated, and procedural, as opposed to semantic, verbose, and declarative

Instead of continuing to use procedural gobbledegook that no user is going to understand, it is time to move the business logic that drives our systems in to something more useful - business rules. Looking at Ann’s list in this context we see a different perspective:

• Encourage close collaboration between developers and end users
  CHECK - users can actually read and write business rules, allowing them to be equal participants in the specification of business logic.
• Involve users in quality assurance processes
  CHECK - ditto for checking that the rules really do what is needed because users can look at the rules and say things like “Yup, that’s what the regulation says” or “That’s our policy”.
• Don’t use traditional programming languages
  CHECK - use a declarative, verbose and semantically rich business rules language instead
• Stress simplicity
  CHECK - the ratio of business rules to lines of code is often very high, making the same functionality much simpler to specify
• Emphasize frequent releases
  CHECK - business users can make changes to rules that can be tested in isolation and rapidly deployed, enabling “releases” of business logic daily if necessary.
• Users, not developers, should determine new features
  CHECK - in fact users, not developers, BUILD the new features when those features are about the business logic in the system.

So, a business rules approach should be part of Application Development 2.0. Stop writing code, start specifying business rules.

Lastly Ann correctly highlights Agile techniques as a component in Application Development 2.0 and this is entirely compatible with the use of business rules. Indeed, I wrote an article for InfoQ on this topic - rules and agile - some time ago but it is still worth a read. It is also interesting to consider the point at which Application Development 2.0 might not really be application development any more.

Occasionally people ask me if I know someone who might be interested in a particular job so I have decided, for now, to post such requests on the blog so you can all see them. If I get too many then I will come up with something else. Here’s the first:

Marketelligent is a Bangalore-based consulting firm providing a range of analytics services to global clients and across various domains - consumer banking, insurance, telecom, retail, manufacturing, travel, etc. Services provided include simple MIS and reporting all the way toadvanced analytics, including segmentation, forecasting and predictive models.

We are looking for a senior-level executive to lead the Business Development/Sales function in the US.

Details: marketelligent_business-development

We have a number of speaking engagements in the next few months and I thought I would highlight them on the blog today. Arranged in date order…

• August 6th, 10am Pacific Time, “How to leverage scorecards for accurate risk management”
  Webinar, multiple speakers including James, sponsored by ILOG
• August 19th, 8:30am Pacific Time, Reinventing BI with Real-Time Analytics: Embedding Analytics into Business Processes
  Presentation, Neil, part of the TDWI Executive Summit
• September 21st-23rd, InfoCentricity Xeno User Conference
  James will be speaking, details TBD
• October 8th, 3pm European Time, “Decision Services: A pattern for smarter service-oriented systems”
  Presentation, James, part of the SOA Symposium, Amersterdam.
• October 12th-16th, Teradata Partner Conference
  Neil will be participating in a panel discussion and/or speaking, details TBD
• October 26th, 2:30 Eastern Time, “Succeeding as a Decision-Centric Organization”
  Tutorial, James and Neil, part of the EDM Summit
• October 29th, 9:00 Eastern Time, “Competing on Decisions”
  Keynote, James and Neil, part of the EDM Summit

Drop us a line if you are going to be attending any of these and want to meet.

Frank posted some great comments on Here’s how to get started with decision management the other day and made me think about this, often very severe, problem. As Frank put it:

How do you overcome the moral fear some organizations have when they realize 40-80 percent performance improvements come at 40-60 percent less personnel; so if operational budgets and manager salaries are dependent on the number of subordinate employees, then it is in the best interest for some to resist EDM; especially in state public health organization.

This is a real problem with adopting EDM for sure and one I have seen personally in collections organizations, for instance, where the size of the department is a key measure of the power/success/income of managers. In these kinds of circumstances there really is little chance of success without changing these measures. As in all things, the act of measuring something changes it. Measuring managers on the difference between their spend and their income or output is essential if you are to get them on board with the changes that EDM will bring.

Often it is not worth tackling this kind of problem first. Instead I often recommend that decisions currently not being taken at all or ones that are already being automated poorly be targeted first rather than the replacement of manual decision making. These decisions may not have the bang for the buck that replacing manual decisions do but they help establish that the approach works without having to deal with the whole empire thing.

Frank goes on to make another good point:

Public health activities are driven by funding and politics.  A great example is Bio-Terror; there is only one difference between a bio-terror event and any other outbreak or chemical spill — bio-terror is intentional and except for criminal issues, responses are the same: quarantine, clean up, etc.; but states get lots of additional money for bio-terror and create whole sub-organizations with their own information systems…all existing to collect their OWN specific data and report those data by their OWN standards (if such standards exist at all). [It] may be impossible to introduce EDM in these organizations since information sharing is loss of power

Here the problem is not just one of empire size but of the number of little empires involved and the threat to those silos of information sharing. Tackling this problem is also beyond the scope of introducing EDM but it need not be tackled first. While shared information is clearly more useful and more likely to result in better decisions and analytics, it is not essential. Plenty of effective decision management solutions do not use cross-silo data. Indeed connecting decisions across silos is one of the later stages of adopting EDM. The “E” is not meant to imply a need to do enterprise-wide decision management so much as to imply enterprise ownership of decisions - making them explicit (Explicit Decision Management). So the decisions within each silo can be attacked using EDM and can show good results, even if some decisions would require integration of the silos.

EDM is a great approach but it needs the same kind of organizational readiness and flexibility that any new development approach does and it can be limited by the politics and performance measurement systems that are in place. You can, and should, “think global, act local” and get started with a focused EDM project because success will make the necessary organizational change easier.