2022-03-24

There's Lies, Damn Lies, and Maturity Assessments

On the immaturity of maturity assessors

The emperors new clothes

 

Over the course of my career I have been the assessor as well as the assessee of maturity assessments revolving around Security, Architecture and/or Operational Maturity of IT/OT.

Having made the mistake myself of painting a better picture of As-Is being interviewed, as well as experiencing others painting a picture that was obviously far from reality I learned the hard way that "If it isn't documented, it doesn't exist". While there's certainly value in feeling pride in your work, it shouldn't stand in the way of getting as precise a maturity measurement as possible.

After all, it is embarrassing to start at say ML-3 then, after spending a lot of resources, being at 2 just because the initial assessment was overly optimistic. In reality that is not going to happen, so everyone will just lie, painting a better and better picture, never having the time to address the actual root of the issue. As we all know it is hard work keeping up appearances.

So here's my issue with this. Several assessments that I have seen are in the category of overly optimistic.

There could be 2 reasons this is happening; 1) The Assessor is caving to pressure and/or just don't want the customer to look (too) bad to be able to sell more hours, or 2) The Assessor isn't competent.

Or in short, 1) Lack of integrity, or 2) Incompetence.

Based on what I've seen the worse option (Lack of Integrity) is the main issue here - For the sake of better security postures and providing real value to customers it would be great to see assessors taking the hard discussions to be able to provide direct and honest information in order to enable senior executives to make fact-based decisions.

I suggest starting with demanding supporting information and if that is not made available within hours, it doesn’t exist and maturity is <1.