Ben Allfree :: Painless Programming

I have a valuable viewpoint toward custom software. To best describe it, I should talk about measuring custom software quality.

Quality measurements feed into the support framework for many decisions. Getting accurate information early is perhaps the best way to optimize the efficacy of your decisions, and examining the people involved is the best way to find indicators of quality.

Counting defects is the simplest way to measure quality, but it certainly is not the most accurate. I will leave it as an exercise for you to do your own research into quality engineering and management, but suffice it to say that measuring defects in custom software doesn’t provide great indicators of quality. You will find yourself differentiating custom software engineering firms based on dissimilar defects in dissimilar projects – apples and oranges, so to speak. The single largest factor governing the output quality of your custom software firm is in the approach taken by management. In the majority of cases I have reviewed, the people involved made the project a success or failure, and most catastrophes came down to mismanagement instead of engineering incompetence.

To improve the accuracy of your custom software quality measurement, examine how the firm handles the unexpected problems that are bound to arise in a field that combines research with results. Put another way: well-managed firms will expect problems and react gracefully to them. That is very different from a manufacturing mentality where zero defects have increasingly become the standard expectation. Consumers demand zero defects in finished products, even after heavy use, yet I am telling you that good software engineering managers plan for defects in your custom software project. It sounds contradictory.

Planning for defects in your product is contradictory until you realize that most custom software projects begin too soon, before the research is finished. I have yet to see an architecture that was researched and tested so thoroughly that it contained no conceptual errors, yet our errors in understanding are the largest, ugliest, most expensive types of bugs. By comparison, fixing a simple defect in workmanship is trivial.

Consider the author’s view stated in this article. In it, he asserts that a software construction method termed Test-Driven Development is faster than any other approach. He goes on to suggest that the uninformed software engineer masses still consider it good technique to plan for bugs. While I agree that testing work along the way may yield higher quality software and perhaps even higher productivity during coding (”construction”), I would argue that Test-Driven Development is in fact a methodology built around the expectation that problems will occur and that it is therefore contradictory to suggest that Test-Driven Development eliminates them. It is simply the name chosen to describe what I contend is the key to quality: handling unexpected problems gracefully.

Since the best measure of quality is to examine how the firm handles the unexpected problems caused by conceptual disconnects, we must then turn our attention to people rather than technology. This is almost like saying I prefer attitude over ability, which is true, but it is not the whole story. I look for attitudes that lead to great abilities: curiosity, persistence, calmness. I think of these as attitude characteristics because they are constantly developing.

The ability to respond gracefully is particularly problematic for even talented solo developers. They do not have a network of cross-functional people to help devise and test hypotheses, and Internet search tools just can not replace the immediate feedback and productivity of other minds challenging your assumptions.

To illustrate this point, I like to tell the story of an embedded systems project I worked with a very skilled electrical engineer. He provided the circuitry design skills, and I contributed the operating system design skills. Each of us was particularly strong in our function and had only a limited understanding of the other’s skills. Though we were mere beginners in each other’s fields, we were able to spot obvious errors and conceptual gaps that would have otherwise sent us into long experiments that ultimately revealed a frustrating “duh” mistake. Simple mistakes like miswired chips, inverted logic, forgetting to initialize registers, or processing data out of order can often cost hours of wasted time alone when the error would be immediately obvious to everyone but the author.

No doubt, the up-front costs of teams seem more expensive, especially if your product life is short and the niche is small. The conclusion is usually wrong, but only a needs assessment can find the real answer. After you decide the size of team you need, even if it is a team of one, look for the team with the right attitude.