Most Software Is Never “Done”
On some nature of software development and heuristics for gauging impact of technical debt
Preliminaries
Developers often justify attention to quality by justifying through the need for proper professionalism. But this moralistic argument implies that this quality comes at a cost — dooming their argument. The annoying thing is that the resulting crufty code both makes developers’ lives harder, and costs the customer money. When thinking about internal quality, I stress that we should only approach it as an economic argument. High internal quality reduces the cost of future features, meaning that putting the time into writing good code actually reduces cost.
Your 6 person team with a consistent autodeploy loop would take 24 people to do the same amount of work, if it took days to deploy their changes. Your 10 person team that ships in weeks would need 80 people.
At cost to the company of approx 200k per engineer, that’s $3.6 million in the first example and $14 million in the second example. That’s how much your neglect of internal tools and kneejerk fear of autodeploy might be costing you.
Even ignoring usability, engineering progress slows to a halt once a project becomes too complex. Each new line of code added to an application has a chance of interacting with every other line. The bigger an application’s codebase, the more bugs are introduced whenever a new feature is built. Eventually, the rate of work created from new bugs cancels out the rate of work done from feature development. This is known as “technical debt” and is the main challenge in professional software development.
…
Building good software involves alternating cycles of expanding and reducing complexity. As new features are developed, disorder naturally accumulates in the system. When this messiness starts to cause problems, progress is suspended to spend time cleaning up.
Software Is Never Done
An underappreciated aspect of software development that is not found in most other disciplines is that the product is an ongoing endeavor and any work done on it has effects on subsequent efforts. A writer can write a boring book and a factory can produce a batch of physical products of low quality. They are done, finished. The next story, product design, or batch can be produced independently of previous results. Software is unlike this.
The delivery of software — putting software into users' hands — is like manufacturing but the development of software is everything but. It is more like constantly changing the structure of a house while people are living in it. We have to be concerned about the change of rate/cost of change on top of completion of the immediate requests and not affecting ongoing lives.
But where do we stop? It is not practical to consider the infinite number of hypothetical scenarios. This brings us to useful but very demanding metrics like lead time, cycle time, and deployment frequency. These terms have more precise definitions but they are used loosely here. The gist is that if once upon a time it took 30 minutes to reliably and confidently put the smallest changes into users' hands and now it takes 15 more, it’s an indication that it’s probably wise to address the increase in time and nip it. Efforts for big changes are hard to quantify but they are just a series of smaller changes. This is where technical excellence, techniques, and practices come in.
We might think these are things that end users don’t see. But actually often they do. Users do not just need feature A in the quickest time. Implicit is also the expectation and demand to also to provide features B, C, D, and many more changes in the fastest time. They can compare between competitors. If you have a stake in software, you have to be concerned about its internal qualities.
It is a challenge to quantify “technical debt” and its impact but we can start with heuristics. Don’t let the cost of change bloat — there’s no absolute figure to go for but everyone should be able to feel the friction and act on it. Developers should be able to say “Compared to last quarter, I seem to be taking twice the amount of time to understand and make changes to this part of the code and the entire pipeline is taking 15 minutes more for a single commit” and then spend time to reduce it. The well-known DORA has research on these metrics and their associations with organization performance.
“Developers” often complain about the quality of code slowing down development but the expectation to go faster is always there. Software development is not repetitive work, how can we know if expectations are unreasonable or mismatched? Lead time and cycle time are measurable, observable, and actionable (though in many places impossible because of waterfall, yada yada…). The very best simply outperform the rest in orders of magnitude. We don’t make code more readable because we like it to be pretty. It’s because we expect many future changes to be related and cheaper to make. Conversely, we don’t “improve” on areas where no savings in time is expected.
Of course, this is all contextual. We do not need to care about the cost of change for software that is not expected to change but that is usually not the case. The cost of software development can be deceptively low because in comparison to physical products immediate and early changes can be observed very quickly. But this is where the challenge lies — keeping the cost of change low because software is expected to change frequently and indefinitely.
The same concerns will apply in the context of houses if in this imaginary world, changing building structures and installations are not costly and are expected every other day. Firms that are able to make changes quicker reliably and confidently will be more competitive. In present reality, unlike physical products, the lower limit of lead time for software development can be very low. Not taking advantage of this has been shown to cost much more. On top of that is great opportunity cost. Software development is more like a marathon than sprints. Most software is never done. Keep them easily modifiable.
