In a recent blog post, I talked about the need for many ways to represent business rules depending on your task at hand. That sparked some excellent comments that make a perfect introduction for this follow-up post on Rule Va
In a recent blog post, I talked about the need for many ways to represent business rules depending on your task at hand. That sparked some excellent comments that make a perfect introduction for this follow-up post on Rule Validation.
How do you know that this one extra rule will not blow up your decisioning???
- Will it blow up my code and result in software exceptions?
- Will it blow up my existing logic and result in decisions I don’t want?
In this post, I will explore different series of activities that protect you from those unwanted blasts in your Production systems.
Unavoidable QA
Quality Assurance is the lowest level of safety net that your IT police will not let you get around — and that is for your own good!
The objective here is basically to make sure that the changes do not cause issues at runtime. In particular, you want to focus on:
- Historical transactions that are representative of what is to be expected
- Test cases developed out of requirements
- Malformed data that may cause ungraceful handling
You may encounter issues that make your code “explode” to keep the analogy, or they may cause your code to “not detonate”. The latter here refers to systems that hang for way too long. Runtime Performance delays can be as serious an impediment as a quick blow up.
The good news is that QA is a fairly mature discipline and most organizations have invested in at least partially if not totally automated tests for both categories. Your IT organization is likely to have a good handle on those — how to detect them at least.
The key question: Is it the logic you want?
This is the most interesting question. Yaser, who used ILOG and FICO in the past, was obviously primarily concerned about that very question. How do you know that your decisioning logic is still sound after you add one little business rule into the mix… I suggest a series of steps to validate that it is:
- Does it do what you want it to do? That semantic question can only be looked at by the experts that codified it. In real-life, we have seen business users or business analysts grab one test case and walk through the rule, one finger on the screen, one finger on the paper sample. You may have invested in more automated unit case testing… but the business owners may still want some oversight regardless so that nothing falls through the cracks.
- Does it clash with some logic already there? This is where automated Verification complemented by expert oversight can help. The goal is to identify logic flaws like gaps and overlaps. Tools can help find those but there are many cases where the “anomalies” are intended. For example, you will likely not issue insurance policies for drivers under the age of 16… But what happens to the underwriting and rating logic? Will you bound the condition to the age bracket 16-25 or just refer to the “less than 25″ youngsters? Although equivalent if you red-flag the applications upstream in your logic, the latter may give you a false sense of comfort if and when the driving age limit is lowered to 14… Is the same logic applicable? Are they similar in risk / behavior? Will you find all the business rules that discriminate against (young) age to make the necessary modification then? Or should you create that segmentation now, instructing the system that super-youngsters (anyone less than 16 in our example) are not covered yet? Regardless of what is most applicable to your business, you probably want someone to look at those identified gaps and overlaps to confirm that they are intended… or not!
- How does it affect past transactions? I may be stating the obvious here: if you add a rule, you clearly expect some transactions to be handled differently. Step 1 focuses on whether you are handling those correctly. I suggest here to look at ALL the transactions that are affected and making sure they are ALL meant to be. In other words, this is the hunt for false positives. Although it is obvious, I have seen many projects that do not focus on those until they get problems in Production.
- How does it affect KPIs? I made the argument on my past post on Factual Logic. Changes in trends, changes in metrics are unavoidable when you change your decisioning logic. Think of them as your health indicators. You want to monitor that they are healthy before your systems become sick. *** an ounce of Prevention is worth a pound of Cure ***
Validation does not stop at the time you deploy your business rules though. You may have made assumptions while developing them, that may be worth monitoring, validating as well. If business conditions deteriorate, or dramatically improve (let’s remain optimistic), your strategies may not be as optimized as they were when you deployed them. The projections based on your historical data would have to be reassessed. This is why validation should turn seamlessly into monitoring once it reaches Production.