Software Quality and Value - A mathematical examination

Authors Note: Before I begin, I want to point out that a lot of the components of these equations are very difficult to objectively define. That is because quality and value are very subjective topics. These equations are meant more as a thinking tool than a way to determine an exact value. That said, if you take the time to push this out to its logical conclusion, you can actually apply numbers to these formulas and come up with some pretty interesting figures that relate to your project.

Let's begin with Value. Value is a concept we are all familiar with. The things we build or buy have value; our car or bicycle, our meal or beer. We also value things and people based on our judgements about them. Ask someone to explain why something has value to them, though, and you are likely to get a vague answer somewhere between 'I just like it' and a set of personal judgements about the quality of said item.

There will be a thread, though. Value is practically always defined as a relationship between Quality and Cost. This relationship can be (over)simplified to the following equation:

In order for that to have any meaning, though, we need to be able to quantify it. This is not exactly easy to do. There are no units of measurement for Value or Quality. They exist only as perception, and even then are only meaningful when related to something else.

Quality

There is no unit of measurement for Quality. Quality is a vague thing. It is often hard to describe and can be difficult to nail down. I have, however, come up with a definition of quality which I find works in nearly every application of the term. My definition of Quality is "suitability for its purpose." A thing is generally said to be of excellent quality when it performs its purpose flawlessly. When a thing is said to be of low quality, it generally performs its purpose sub-optimally, or not at all. This definition clearly defines Quality as a range and for the range to be useful, an upper and lower numeric bound must be defined.

To keep our concept of Quality grounded, we will use a range of 0 to 1. When judging software using this definition and range of Quality, we will say that a piece of software that performs every aspect of its purpose exactly the way it is intended has a Quality of one (1). A Quality of zero (0), by contrast, does not perform its intended task at all. Obviously, when developing software, the ideal target is always a Quality of 1. Keep this in mind, we will use it later.

Value

Now let's explore the next element of the equation, value. Value is another hard to define concept. There is no unit of measurement that is generally defined for value. It is again a range and is somewhat arbitrary. In order for our results to be meaningful we need to define a range. When we speak of the value of a thing, we may say it has no value, or that it has high value. So, we will use a range that is in keeping with the way we usually use the word. We will say that if a thing has no value, its value is 0. Let us say, for the purposes of this subject that a value of 1 is a 'neutral value' that is neither exceedingly good, nor exceedingly bad. Anything that has a value that is higher than one (1) is of 'high value.'

Cost

Now that we have workable definitions and ranges for Quality and Value, we only have one component of our initial equation left to define: cost. Generally speaking, cost can be quantified in terms of money, or time, or a relation of the two. When developing software, cost is most often quantified using money. The time element of cost can usually be converted to money by multiplying the total hours required by the hourly rate of the developer or developers. Every working developer knows this equation all too well.

So now we have a working definition of cost as well. We have to apply it to our original equation. In order for it to work, however, we have to remember that value has a perceptual element as well. Remember that value is judged based on our concept of what something's cost should be according to some ideal.

Ideal cost

So what is ideal cost? You already know the answer to this. If the actual cost is what it really takes to get a set of features done, then the ideal cost would be the minimum required to get those features working. Let's use a web application as an example. We know that each application has business logic that is unique and specific to that application, it also has presentation code that will be specific to that application. In an ideal world, we would only have to write the code to define that behavior. So our ideal cost is what it would take to write just the business logic and presentation portion of our application.

All of this translates to an equation that looks like this:

With this equation, you have a working model for determining value. With our target Quality as 1, as actual cost approaches ideal cost, our value approaches 1, or 'good value.' If actual cost is less than ideal cost, value increases, and likewise if actual cost is more than ideal cost, your value goes down.

The problem with this equation right now is that it only works if we always use our target of quality level of 1, or highest quality. As any working developer knows, this highest level of quality is rarely attained. Unfortunately, this returns us to the question, how do you define quality with regard to software? We've given it a range, but it currently has no meaning. In order for our overall equation to mean anything, quality must be more clearly defined.

Software Quality Revisited

There are many ways you can judge a code's quality, but many of them are subjective. Here we want to focus on as objective measures as possible. Different people have different preferences, but ultimately there are some objective areas that can be used nearly universally. For our purposes there are two areas that define the quality of a chunk of code.

Functionality

First and most simply is functionality. Functionality simply answers the question 'does this code actually provide the feature required in its entirety?' Functionality is easily determined if you have a complete specification. You can think of functionality as a percentage. Take the implemented number of components of a given feature and divide it by the total components of that same feature according to the spec.

Any professional developer with even a few projects under his belt would agree that overall code quality tends to suffer when there is an incomplete (or worse, nonexistent) spec. It is mathematically impossible to solve the value equation if required functionality remains an unknown.

Note that often people define many other areas to consider when evaluating software quality, for example: reliability, scalability, portability, usability, etc. When it comes down to it, though, each application will have different requirements for each of these areas. In this respect, it is clearer to think of these requirements as feature requirements and thus they fall under the functionality heading in terms of analysis.

Maintainability

The second part of software quality is maintainability. Software rarely is written and never modified, so for any real code quality judgement you have to look at how easy it will be for the code to be maintained and modified down the road. In my experience code's maintainability can be defined as a combination of three elements: documentation, clarity of code and verifiability of functionality.

Documentation we are all familiar with. It is the easiest element of code quality to control, as the author of the code can easily write what the code does in plain language. Sadly, it is all too often neglected. The level of documentation required can be somewhat subjective depending on the reader, but for our purposes we are talking about documentation required for the software to be well maintained. This includes documentation of the intended function, as well as the details of how the code functions. Note that this includes both formal documentation and code comments. We can quantify documentation quality by comparing the information required to maintain the code with the information actually present in the form of documentation. This will vary from developer to developer, but in practice it comes down to something akin to code coverage testing. In my opinion, if the general purpose of the code is defined, and every function or method is documented in terms of what it expects as arguments, what state (if any) it affects while running, and what it produces as a return value, the documentation requirement has been met. This, like functionality, can be taken as a percentage.

Clarity of code is an assessment of the code itself. Code clarity answers the question 'How easy is it to determine what the code is doing?' Are variables well named? Are clever but hard to understand solutions eschewed for clearer, if more verbose, solutions? When hard to understand solutions are unavoidable, are there enough code comments to decipher the associated code? All of this boils down to the question, 'When a change is required to the code, can you easily determine where and how to make that change?' This is a difficult one to quantify, but it is not unreasonable to treat it as a 0/1, yes or no proposition. If another reasonably competent developer (who is unfamiliar with the code) can read it and understand it without requiring additional explanation or a long code spelunking process, then clarity of code is achieved, otherwise it is not.

Verifiability of functionality is the ability to objectively verify that a piece of code does what it is supposed to do. Generally speaking, there is one way to verify functionality: unit tests. In order to achieve maintainable software, you have to be able to tell whether the maintenance change you just did damaged the code. In order to do that, you need to have set inputs and expected outputs and you need to have automated tests that verify that the code produces the expected outputs (and side effects) when a given set of inputs are provided.

If you can not easily verify this for each piece of code your maintenance will be significantly more troublesome. The clearest (and quite significantly discussed) method of evaluating this is 'code coverage' analysis of your tests. Taking your code coverage as a percentage is sufficient for this element.

Complete definition of Quality

So, the quality of a given set of code or feature can be said to be the average of these four elements: functionality, documentation, code clarity and tests. If you think of each of these elements as a percentage, or a fraction of 1, then our quality equation becomes:

Which makes our Value equation:

Boiling it down

We now have equations that give us a way to gauge both the quality of software and its value, but what do we do with them? Well, to begin, anyone in charge of the completion of a software project now has a way to evaluate the completeness of a piece of software. We also have a way to compare competing software packages more objectively. There are likely always to be factors to consider outside of those we have covered. That said, however, at minimum, the quality equation gives us a baseline to evaluate all packages. If the values are significantly skewed at the quality level to start, evaluation of additional factors may be irrelevant.

As developers it is easy to at first be put off by these figures and equations. It can seem like a recipe for more slapping around by managers who don't understand what development really entails. I think, however, that this can have the opposite effect. It provides an unbiased and clear set of criteria for evaluating code quality. It allows you to base your code creation process on factors that do not change, and it provides a clear business reason for defining specifications ahead of time.

These formulas provide a way to explain a target for software development that makes our jobs easier. It also provides a way to explain why the all too common practice of 'We need it now, I don't care how ugly it is' ultimately leads to low quality software and an unmanageable disaster of maintenance.

Obviously, these can only be useful if you can apply them. Next time, I will be covering some of the more interesting implications of these equations. In the meantime, I'd love to hear your experiences with development process, selecting software and modules, and what you've found useful for improving software quality. Got a thought to share? Sound off in the comments.