Raising and Sustaining Software Engineering Quality
In the summer of ’99, I interned at one of India’s largest automobile manufacturers at the time, where I learned essential lessons. This one is about quality.
One morning, I saw a group of executives in factory gear gathered around a workstation. They were intently peering at a machine. Its computer reported that the quality control measure had fallen below a high threshold of less than one in a million defects. There was something wrong with the machine or its coolant. It was disrupting the assembly line throughput. The executives were worried by the factor this would affect the overall manufacturing target.
I have spent the better part of the last two decades in software development. Still, I have not seen the same level of clarity towards quality. It remains a fuzzy concept that even developers in the same team may not agree with completely. When an executive identifies software engineering quality as a problem, it is usually because of a significant incident that risks their reputation.
Why is software engineering quality still a hot-button issue that vexes organizations?
Producing Software is a Creative Process
Software is the interface between a machine and humans that desire specific behavior from the machine. This behavior exists in a problem space, while the software exists in a solution space. For most products, the behavior is a moving target. The Kano Model presents a compelling argument — a feature that may be groundbreaking in the beginning becomes an expectation after a while and later becomes something that annoys customers in its absence. Only a connoisseur would buy a black and white TV today.
The process of creating software is evolving as well. At its earliest maturity level, programming was unstructured. Jumping around one program location to another was considered ordinary and necessary. Then Dijkstra showed that structured programming with conditionals and loops was better for program structure. Object-oriented programming, functional programming, actor-based concurrency, and cloud-native development have all contributed to this evolution.
When both the problem space and solution space are evolving, any activity to produce a solution will be complex. All complex phenomena have essential and incidental complexity. Software Engineering can devise models to handle the essential complexity but can’t correctly handle incidental complexity.
This inability to completely model a complex set of activities is why producing software will probably always be a creative activity.
How can we measure the quality of a creative process?
Picture Good Quality
Artists mostly think of the Mona Lisa as great art, not just good art. They can tell you many reasons for their high regard.
There is a parallel with Software Engineering. Suppose you asked various groups of developers to describe good quality software. You are likely to hear common attributes such as Availability, Interoperability, Security, Performance, Testability, Usability, and Modifiability. How many quality attributes are there in all? I have counted 75 in a book that called it a partial list.
So step one is to get all your top developers together and ask them to draw a picture of good quality Software Engineering. You may want to try a design thinking exercise to develop a set of attributes. If you need to get started quickly, you can begin with Availability, Security, Performance, and Modifiability. These four attributes tie to financial objectives most closely and are likely to get executive buy-in with reasonable explanations.
Build a Gate Check
Software Engineering borrows heavily from the lessons learned in manufacturing. It manifests itself as the DevOps culture that binds all business functions into a single value chain. A value chain rejects upstream work that does not meet the quality threshold at a workstation.
Organizations should build quality into the value chain as a gate check. How to decide the thresholds at which the gate check passes? If you don’t want to disrupt the business (sometimes you may have to), you can set the thresholds at their current values.
How do you measure the attributes to get their current values? The more heterogeneous your environment, the more difficult it is to create a one-true-way of measurement. I would recommend for each attribute a simple rubric with yes/no questions. You could split up your developers by each attribute to frame the questions that the larger group can discuss and come up with the score.
Bake into Organization Culture
A gate check is not sufficient to encourage the right behavior, just as law enforcement is not enough to promote civility.
It is crucial for developers to intrinsically care about quality and not pay lip service by following a checklist. Involving them in a design thinking exercise to develop the quality attributes they care about will get you early buy-in. More importantly, you will understand what they already care about. The more they care about, the less you have to convince them!
Patrick Lencioni says that values must be reinforced by human systems such as hiring, compensation, rewards, and promotions. The conversations among leaders while making such decisions create a cultural awareness about quality.
Connect to the Business
Encourage teams to come together with their Product Manager to discuss how the quality attributes affect the team’s product. If there are systemic problems in the product and the team sees a correlation with specific quality attributes, they can create a goal to improve those attributes.
Developers need to be empowered to ask the business about these attributes. They need to be supported by Product Managers (and Owners) because they know all too well the long-term consequences of ignoring the agreed-upon thresholds.