When assessing Project & Programme Risk the most commonly used approach is to assess the Impact and Likelihood (or probability) for each potential risk event and draw a Risk Map as shown below.
This is a good way to visualise the project risks that have been identified for deeper analysis and review. By multiplying the risk impact and probability you can come up with a risk rating that can be used to compare and prioritise different types of risks for the development of mitigation plans.
The challenge with this approach is that the categorisation of both the risk impact and the probability are estimations based on currently available information and knowledge. The situation will change over the lifetime of the project as more information is gained.
So it is good practice to repeat the assessment at regular intervals throughout the duration of the project to update the mitigation actions and define new ones.
One method we developed to provide further visibility into the risk assessment process is to look at the level of uncertainty in each specific risk.
A risk, which has been clearly identified and understood, would have a low level of uncertainty. An example of this could be the risk of finding a suitable project manager with the right experience and skills, which could delay the project start. This may be a significant risk but it has a low level of uncertainty since it is clearly defined and is a common situation that has a clear solution – find the right project manager.
On the other hand, a risk that is not so well-defined or for which there is very little experience available will carry a high level of uncertainty. For example, the implementation of a new or unproven technology usually carries a high level of uncertainty due to lack of prior experience. So the estimate of risk impact and risk probability are limited by the lack of experience and could be much higher or lower than assumed.
Technology projects and programmes introduce changes over a defined timeline. Uncertainty arises because we don’t fully know the outcomes of these changes or whether there may be external factors that affect the project during its lifecycle.
Uncertainty in the outcome of technology projects comes primarily from a lack of knowledge and experience of similar situations. No two projects or programmes are exactly the same.
Risk occurs when those uncertain outcomes may have a negative effect on achieving the project goals. In a standard risk assessment, we assign an impact and likelihood (or probability) to each risk event.
The former US Secretary of Defence, Donald Rumsfeld, captured uncertainty well when describing unknown unknowns:
"There are known knowns; there are things we know that we know. There are known unknowns; that is to say, there are things that we now know we don’t know. But there are also unknown unknowns – there are things we do not know we don’t know."
To better quantify uncertainty in risk assessments, we have defined an additional qualifier for each risk in terms of the level of uncertainty as shown below:
This is, in effect, an assessment of the level of knowledge and experience that we have with the specific risk event.
With this new information, it is now possible to plot the risk rating and the level of uncertainty in a map as shown below.
The project risk & uncertainty map shows on one axis the risk rating (impact x probability) and on the other axis the level of uncertainty as described in the previous table.
When used in combination with the standard Risk Map, the Risk & Uncertainty Map helps to draw attention to those risks with a high level of uncertainty.
Project teams and their management usually focus their mitigation efforts primarily on the risks with the highest risk rating. The trouble with this approach as one recent study1 found is that it is the unpredictable events – the black swan risks – that have the largest impact in terms of project cost and schedule overrun.
This study also identified that the longer the project, the higher the risk of cost overrun, due to the increased uncertainty when starting the project.
By focusing also on those risks with a high level of uncertainty ensures that project teams and their management put in place plans to reduce the level of uncertainty over time.
The following diagram shows how risks can be mitigated using this model:
Reducing the level of uncertainty can be achieved in several ways with the objective being to increase knowledge. This may be achieved by bringing in additional expertise, casting the net wider to learn from other projects which have had similar situations, proof of concept testing, or other forms of early testing.
For the cases where the risk rating is high and the level of uncertainty is high (upper right quadrant), it may highlight the need to rescope the project objectives and requirements to eliminate this risk completely.
We applied this approach recently to assess the programme risks on a new long term multi-country IT Integration programme with complex stakeholder dependencies to help the customer develop their transition plan for the new contract.
This is a better approach but it remains a snapshot of risks and uncertainty at one point in time and it deals only with those risks which you have foreseen. Over the project lifecycle the risk severity can increase, decrease or new risks can materialise based on new information or events. The solution is to perform assessments at regular intervals to update mitigation plans and create new ones.
Creating a Project Risk & Uncertainty Map, to complement the existing Risk Map will help to build a better project risk profile and help to prevent the black swans.
1Delivering large-scale IT projects on time, on budget, and on value, McKinsey, October 2012
