So to follow up on Part 1 of our Key Risk Indicator network or system which can be found here [LINK], perhaps the most important consideration we should ponder on is what do we actually do with the KRI data we capture?
In reality, why bother tracking Key Risk Indicators if we aren't going to do anything with them? Yet, so many risk managers do go through this very exercise without bringing it to full conclusion.
In this article I would like to describe at a top level what we can use our KRI data for and outline what holds us back from that objective.
Before leaping into the modelling aspects of Key Risk Indicators or factors as we refer to them in a model, we need to ask ourselves a very important question here; what exactly do we use KRI data for?
One answer may go along the following lines: To give us forward looking signals or early warning signs on potential risks we are facing in our business. Okay then, we've built a KRI database to capture this data and we can connect this data to risks we have identified in Part 1 of this article [LINK], what's next?
If we think about the way a Key Risk Indicator describes a risk, it could sit on either side of a specific threat. It may be a formative indicator or precisely it expounds the causal factors that lead to a risk before the event happens. It can also be a reflective indicator and furnish us with potential evidence that a risk event has actually occurred.
These things are much better explained with an example: A fire alarm sounding gives us an inkling that a fire might be taking place and is a fantastic exemplification of reflective indicator. Alternatively, the number of times we have mismanaged fire hazards would be a good case for a formative indicator or an indicator that describes what randomly might happen if we handle fuels without care.
8 KRI Realms to Traverse
One answer may go along the following lines: To give us forward looking signals or early warning signs on potential risks we are facing in our business. Okay then, we've built a KRI database to capture this data and we can connect this data to risks we have identified in Part 1 of this article [LINK], what's next?
If we think about the way a Key Risk Indicator describes a risk, it could sit on either side of a specific threat. It may be a formative indicator or precisely it expounds the causal factors that lead to a risk before the event happens. It can also be a reflective indicator and furnish us with potential evidence that a risk event has actually occurred.
These things are much better explained with an example: A fire alarm sounding gives us an inkling that a fire might be taking place and is a fantastic exemplification of reflective indicator. Alternatively, the number of times we have mismanaged fire hazards would be a good case for a formative indicator or an indicator that describes what randomly might happen if we handle fuels without care.
The two things we seek
Along with formative and reflective indicator positions around a risk, there are at least two key findings we could possibly want from Key Risk Indicators.
The first is a translation of a data signal into information.
The second insight we can gain from KRIs is estimating how much something grows or decays by indirectly observing a reflective or formative indicator that describes it.
Again this is all explained much more clearly with examples: We see smoke and it follows there could be a fire is a very simple binary example, you see clouds and so you conclude that it might rain as a consequence is another obvious indicator. Alternatively, we have identified that there were fifteen errors in a processing department, we can then use this error count to dimension how big our losses may be for the month of processing.
This all sounds like basic simple stuff doesn't it but random factors or variables of which Key Risk Indicators belong, have some additional considerations which make the process of variable measurement a little more cumbersome.
The first is a translation of a data signal into information.
The second insight we can gain from KRIs is estimating how much something grows or decays by indirectly observing a reflective or formative indicator that describes it.
Again this is all explained much more clearly with examples: We see smoke and it follows there could be a fire is a very simple binary example, you see clouds and so you conclude that it might rain as a consequence is another obvious indicator. Alternatively, we have identified that there were fifteen errors in a processing department, we can then use this error count to dimension how big our losses may be for the month of processing.
This all sounds like basic simple stuff doesn't it but random factors or variables of which Key Risk Indicators belong, have some additional considerations which make the process of variable measurement a little more cumbersome.
Establishing these formative or reflective relationships between the Key Risk Indicators and their associated risk events seems logical but we need to take a look at the statistical constraints before we go any further.
When modelling anything, it is important to comprehend the limitations of a model so that the model can be sensitive to these additional structures or factors.
With Key Risk Indicators there are eight unique axioms or postulations that come immediately to mind as important for consideration. We need to identify whether our KRI data is suffering from one or more of these constraints and then adjust our models accordingly, at the very least we need to state the model's limitations and error.
Let's have a look at these eight KRI modelling twists:
[Axiom-1] A key risk indicator may present us with false positives and it is possible that our fire alarm may sound but no fire is present.
[Axiom-2] By considering the flip side of the first axiom, it follows that a key risk indicator may also turn up false negatives, a place where there is a fire but the alarm didn't catch it. A worrying consideration but a reality we need to accept, controls can fail!
[Axiom-3] Beyond the obvious, risk is usually driven by the intertwining of many factors not just one variable in our KRI data ensemble alone. These factors will have causal relationships which can be very complex beyond what we can possibly imagine, that we can be nearly sure of. For example, we can have two risk factors which drive an event but alone these factors may be innocuous or harmless by themselves. However, when the factors are combined together, well we might be sitting on an explosive mix.
[Axiom-4] Causal factors that drive risk are themselves randomly occurring phenomena that may lead to a risk actualization or outcome or just zip, a null event. The dependency for an outcome from a specific factor is unlikely to be 100%, very few things in life are 100% when precision or risk are in scope for measurement.
[Axiom-5] We must also keep in mind that correlation doesn't translate directly to causation, this is very important. Precisely, just because something exists doesn't mean to say that it actively drives the second condition, something might be no more than a by-product of another formative condition and a condition we aren't aware of or monitoring.
[Axiom-6] Extending on more deeply with axiom 5, we could also be dealing with a factor that adds a positive feedback loop to the risk system, a risk catalyst if you prefer. It might go the other way too but either way, there can be cross correlation effects between different factors that cancel out a risk or add to it. KRIs shouldn't always be viewed in a negative light alone or simply taken as having effects/affects that add to or take away from a risk. A factor can even add to a risk in one circumstance but also take from it in a different concentration or situation!
[Axiom-7] We should ponder on this idea of 'concentration' we made mention to in Axiom 6 and in a little more detail because most causal factors have a dose response to risk. Let me explain with another straightforward example: You are unlikely to feel dizzy from one drop of wine, two bottles well that is a different story. So then, what is the tolerance of a risk factor before it presents a condition, where is this saturation point, where is the tipping point in a system, what is fragile and where is this conversion threshold are all questions which Key Risk Indicators can potentially answer?
[Axiom-8] The final modelling constraint is perhaps the most frustrating to statistically work on. Random factors can have time lag! That is, a causal factor may be present for a long time as a resident pathogen before it activates and becomes a living threat. We can be nearly sure that this additional time delay factor across a single univariate KRI signal is probably going to be random too, it may cluster and more often than not it has seasonality.
If we bring all of these arguments into play we are likely to have a pretty nonlinear response to risk when we are just observing a single Key Risk Indicator or factor alone. Okay then, so our Key Risk Indicator model needs to be sensitive to these potential modelling paradoxes and in Part 4 of this article set we will fly in the more complex modelling parts required to address these marginal considerations.
Before all of this however, we need to walk through a simple exercise to bring our KRI database into operation. In Part 3 of this KRI article series we are going to look at sample data in the database and how we can model a very simple single Key Risk Indicator against a registered risk and front to back.
When modelling anything, it is important to comprehend the limitations of a model so that the model can be sensitive to these additional structures or factors.
With Key Risk Indicators there are eight unique axioms or postulations that come immediately to mind as important for consideration. We need to identify whether our KRI data is suffering from one or more of these constraints and then adjust our models accordingly, at the very least we need to state the model's limitations and error.
Let's have a look at these eight KRI modelling twists:
[Axiom-1] A key risk indicator may present us with false positives and it is possible that our fire alarm may sound but no fire is present.
[Axiom-4] Causal factors that drive risk are themselves randomly occurring phenomena that may lead to a risk actualization or outcome or just zip, a null event. The dependency for an outcome from a specific factor is unlikely to be 100%, very few things in life are 100% when precision or risk are in scope for measurement.
[Axiom-5] We must also keep in mind that correlation doesn't translate directly to causation, this is very important. Precisely, just because something exists doesn't mean to say that it actively drives the second condition, something might be no more than a by-product of another formative condition and a condition we aren't aware of or monitoring.
[Axiom-6] Extending on more deeply with axiom 5, we could also be dealing with a factor that adds a positive feedback loop to the risk system, a risk catalyst if you prefer. It might go the other way too but either way, there can be cross correlation effects between different factors that cancel out a risk or add to it. KRIs shouldn't always be viewed in a negative light alone or simply taken as having effects/affects that add to or take away from a risk. A factor can even add to a risk in one circumstance but also take from it in a different concentration or situation!
[Axiom-7] We should ponder on this idea of 'concentration' we made mention to in Axiom 6 and in a little more detail because most causal factors have a dose response to risk. Let me explain with another straightforward example: You are unlikely to feel dizzy from one drop of wine, two bottles well that is a different story. So then, what is the tolerance of a risk factor before it presents a condition, where is this saturation point, where is the tipping point in a system, what is fragile and where is this conversion threshold are all questions which Key Risk Indicators can potentially answer?
[Axiom-8] The final modelling constraint is perhaps the most frustrating to statistically work on. Random factors can have time lag! That is, a causal factor may be present for a long time as a resident pathogen before it activates and becomes a living threat. We can be nearly sure that this additional time delay factor across a single univariate KRI signal is probably going to be random too, it may cluster and more often than not it has seasonality.
If we bring all of these arguments into play we are likely to have a pretty nonlinear response to risk when we are just observing a single Key Risk Indicator or factor alone. Okay then, so our Key Risk Indicator model needs to be sensitive to these potential modelling paradoxes and in Part 4 of this article set we will fly in the more complex modelling parts required to address these marginal considerations.
Before all of this however, we need to walk through a simple exercise to bring our KRI database into operation. In Part 3 of this KRI article series we are going to look at sample data in the database and how we can model a very simple single Key Risk Indicator against a registered risk and front to back.
Other articles in this series
This blog is just one chapter in a series of articles on 'What to do with KRIs?'. There will be four other chapters published on this KRI topic that deal with different aspects of Key Risk Indicators. These chapters have been listed below:
Part 1 - KRI Metadata Structure [LINK]
Part 2 - Eight KRI realms to transverse << This article
Part 3 - Front to back operation of the KRI system [Not published yet]
Part 4 - Modelling the KRI realms [Not published yet]We trust you will enjoy the ongoing dialogue and the practical examples that are going to follow.
No comments:
Post a Comment