Measurement is such a vital part of understanding if you are doing a good job that everyone should be doing it. There are however obstacles and limitations facing measurement in complex organisations, that get in the way.
People resist measurement
People present the main obstacle to organisational measurement. Sometimes they actively go out of their way to thwart, subvert or manipulate measurement systems for perceived protection or personal gain. Sometimes people are just hard to measure because they’re so inherently complex and changeable.
Many people tend to view the whole idea of measurement with fear and suspicion. The most common question, fired from beneath knitted brows tends to be, “Why do you want to do that?”
One of the many reasons people behave this way is because measurement is almost always a precursor to some sort of change. There’s a huge amount written about why people resist change, and what to do about that. .
Where are you going with all that data?
Organisations struggling with measurement are often focusing at the wrong end, on the data collection itself. This can result in complex, unwieldy processes to amass piles of very prescriptively defined data that aren’t being used for anything very important.
These piles of data are often nurtured and protected from disturbance or exploitation by gate keepers. Like sphinxes, these guardians seek to confuse the unwary and will gleefully waste hours discussing statistical validity, non-random samples, longitudinal issues, and other ‘statistrivia’, to scare people away from using the data pile to make decisions.
This sort of behaviour gives measurement a bad name and, as indicated above, people often feel threatened by the whole concept of measurement anyway, so this makes it all too convenient to just forget about the whole thing and go back to comfortable fumbling around in the dark.
It’s usually better to start at the other end: what are we trying to achieve? Then see if any metrics already captured in the financials, or HR, or IT systems are pointing in the right direction to enable better decision making.
Rules of thumb for measurement
- There is no such thing as 100% certainty.
- Human beings always make decisions based on imperfect or incomplete data – so get used to it.
- Some data is better than no data.
- Approximate data is better than no data.
- The quality of the decision making process is more important than making sure the data is accurate to 15 decimal places.
- It’s easier, faster and cheaper to use data you already have than to create a new measurement system.
- Combine measures of quantity with measures of quality, to make sure you know what you’re measuring.
This last point may seem obvious, but it’s not. For example, lots of IT service desks use the number of jobs closed as a measure of performance. This would be great if ‘closed job’ = ‘problem solved’ = ‘satisfied customer’.
But this is not true, because the IT service desk staff usually determine when a job is closed, not the customer. So this provides an incentive for staff to close off all jobs as fast as possible, even though the problem isn’t solved.
When the customer complains, the service desk just opens another job, the cycle repeats, and the service desk staff look good, because they’re smashing the ‘closed jobs’ KPI.
So measuring quantity by itself is not enough. You also need an accompanying measure of quality to understand what’s actually being measured.