Tuesday, August 29, 2006

The "What isn't easily measurable, doesn't exist" Rule

In one of my earlier posts, What Is a Software Architect, I touched upon the effect of governance in software organizations. Specifically, IT governance promises more than it delivers.

In principle, it sounds great. The reality is today few if any organizations can accurately track enough information to deliver these promises. Governance requires a great deal of information, and taking action without the necessary information is likely to have negative effects, rather than positive. Every decision made at a higher level negates the decision making power of those closer to the reality of the situation. Those closer to ground usually have masses of data that are not trivially quantifiable or reducible.

Gathering information is still valuable, but it requires a flexible process to make use of it. The data is not accurate enough that it can replace hands on techniques. At best, the data can be useful to augment hands on techniques, but this requires the data to be available to those individuals with the best position for decision-making. Contrary to popular belief, these individuals are not always those highest in an organizational structure. Except for some extraordinary decisions, those in the best position are actually those lowest in the organizational structure.

One example of misapplication is when IT governance drives performance reviews. While unbiased, measurable performance reviews are attractive, attempting to perform one using incomplete or irrelevant data is a mistake. For example, IT governance often uses metrics like process compliance as a key measurement, because they are easy to measure. However, since the relationship between process compliance and performance is very weak, performance reviews will often reward poor performance, and fail to reward stellar performance.

At organizations treating process compliance as a metric of performance, the opposite is likely true. Processes always have flaws, and individuals with a drive for performance will use processes when they help achieve results, and avoid them when they harm results. Individuals strictly interested in a favorable performance review, will strictly apply processes with a degree of paranoia that harms the performance of the entire organization.

The desire to drive up process compliance, and thus increase information accuracy is a natural instinct among those involved in governance, because these are the metrics upon which they are judged. It is easy for them to ignore the side effects, such as wasted bureaucratic efforts, delays to real progress, and other general dysfunctions. The side effects are outside of their purview, and highly difficult to measure. These two factors lead to application of the “What isn’t easily measurable, doesn’t exist” rule.

This simple mistake is responsible for a huge quantity of the bad business decisions. Many trends, such as poor computing resources, poor work environments, adversarial management-employee relationships. Quite possibly, it is second only to general short sightedness as the most common source of business mistakes. Quite often the two are related as the more distant a future event becomes, the less measurable it becomes.

So, what can you do about this? You could try to use the arguments of this essay with management, in order to gain greater control, but that is likely to be a difficult battle, because that requires managers to place more trust in their employees. Trust is not an easily won quality. It is especially hard when you figure that trust is not based solely upon your qualities, but is heavily based upon managers’ trust in themselves to pick the good and bad apples.

Another tactic is to try your best to quantify the unquantifiable. You’ll likely have some very imprecise results. But even those extremely rough values are more accurate than the “zero”, assigned to them today. It’s important when you do this that you make the imprecision clear. Unfortunately, even when you take all reasonable measures to communicate this there are likely to be misinterpretations, and this more than anything is the reason why most people are afraid to attempt to quantify these values.

Neither path is without risk, and it’s quite possible that successfully following either will gain little or no recognition, so it’s no wonder that it’s uncommon to find champions who try to address these problems. More often than not, they progressively decline until the point where only natural selection can fix them. But since the issues are endemic to the average corporation, natural selection can take a very long time.