My five-step process for eliminating technical debt in your (big) projects

I was conversing with a colleague today and we were lamenting the ever-growing pile of technical debt in a project we were both working on. Technical debt is not a new concept in software, in fact it’s probably one of the oldest, but what’s less discussed (I feel) is how to solve it. It’s simple enough to add // KLUDGE or // FIXME comments to your code and address them later on, or wait for a more senior engineer to get un-busy enough to help you out. But, this is an untenable solution on larger projects with a veritable mountain of technical debt that has built up over the years.

I’ve spent some time thinking about this recently and I’ve come up with a framework that makes sense in the context of 1) large applications that are 2) under active development and are 3) uptime-critical that 4) have a very long backlog of technical debt that is actively hampering ongoing development efforts. And, as if there weren’t already enough acronyms in this industry, I decided to create one more: I call this framework PURGE.

• P – Prevent It
• U – Understand It
• R – Review It
• G – Group & Grade It
• E – Eliminate It

In the following sections I will briefly discuss each aspect of the framework, and then at the end I’ll explain why I think it’s especially effective for larger projects.

Prevent It

An ounce of prevention is worth a pound of cure, it’s a phrase I’ve heard many times throughout my life but surprisingly rarely in the context of software development (probably because SWEs usually use the metric system, lol). The adage rings true here, though: the best way to address technical debt is to prevent it from happening in the first place. This is not always possible, of course, and in fact I personally am comfortable with about a 5% technical debt percentage in code that I produce, so long as it’s not security- or privacy-critical code.

Depending on your language and/or stack (mine is PHP, as many of you know) there are many options for preventing technical debt from occurring, but they broadly fall into three categories:

1. Code Linters like PHP_CodeSniffer (AKA phpcs)
2. Static Analyzers like PHPStan and Psalm
3. High-Quality Code Reviews

These tools are (in my opinion) very easy to integrate with any project that uses Composer, no matter your PHP version. And, with Continuous Integration (CI) pipelines being at an all-time high for popularity, there really is no reason not to run these tools on every commit/push of your project. Coding with these tools may sometimes prove cumbersome, but they absolutely do prevent bugs and technical debt from accruing when you’re not looking.

And, if you’re not already doing thorough code reviews in 2023, let me quote from this article I found on paddle.com:

One of the easiest ways to identify technical debt is to review the code. You can look for code smells or bad coding practices that can lead to maintenance issues in the future. Code smells are indicators that the code may have a design problem or may be difficult to maintain.

What is a code smell? A “code smell” is a term used to describe any characteristic of the code might have issues that could lead to problems in the future. Some common examples of code smells include overly complex code, duplicated code, long methods or functions, and excessive use of comments. Code smells are often subjective, and what one developer considers a code smell may not be considered a problem by another developer.

From “The SaaS leader’s guide to technical debt: business impact + how to identify and reduce it” on paddle.com

Understand It

As mentioned previously, technical debt as a whole is unavoidable. Sometimes when coding, we don’t know what we don’t know and so we make decisions that we later have to pay for. However, I think it’s important to review these decisions on a regular basis so that we can understand where the technical debt comes from. This is especially important for avoiding the Sunk-Cost Fallacy that I have seen far too many junior engineers and product managers commit.

When doing this exercise, I like to utilize the 5 Whys Exercise (straight from Sakichi Toyoda’s playbook). I find this really helps get to the root of the problem. Here are some sample questions one might ask when evaluating the first of the five why’s behind an instance of technical debt:

• “Why did we not research open-source solutions before writing this component in-house?”
• “Why did we not search our own codebase before re-inventing this wheel?”
• “Why did we not document that algorithm X should be done the Y way and not the Z way?”

It’s incredibly important not to place blame during these discussions. Some people call these “blameless retrospectives” and I like that attitude. Everyone, from juniors to seniors to principals, makes mistakes and introduces defects and debt sometimes. There is no place for ego here.

Review It

One of the biggest “cringe moments” I always encounter on big projects with a lot of technical debt (and this finger is pointed squarely at my own chest, do not fret) is when tech debt gets added to a list where it essentially goes to die. You might call this the “One Day List” (as in, we’ll do it one day) or the “Profitable List” (we’ll do it when we’re profitable) but you should really call it what it is: sweeping it under the rug. Yikes. If you make a list of technical debt and then never review it, it might as well not exist.

A colleague pointed out to me once that the biggest reason why technical debt accrues is not lack of desire to address it, or the push to go to market (well, maybe that one a little bit), but rather a lack of visibility into the growing balance sheet of tech debt. It’s super easy to say “we’ll deal with that later” when you know later is never going to come. It speaks to a lack of accountability. But, what many people forget, again myself solidly included, is that there is always a price to pay with these things. It may come three months from now, or it may come when you’re nearing your IPO (looking at you, reddit with your terrible UIs).

Don’t just make lists, review them on a regular basis with your entire team. Then, and only then, can everyone start getting a handle on what needs to be done.

Group & Grade It

So, you’ve got your list and you’ve (perhaps miraculously) gotten the Sales department to hold off on some new feature requests for a period of time so as to allow you to address some of your debt. How do you start? I recommend a two-step approach:

1. GROUP your codebase into major areas where debt is concentrated
2. GRADE each area by the severity of the technical debt contained within

These groupings might be along the natural “fault lines” of your code, e.g. the Models vs. the Controllers, or they may be more along the lines of features e.g. “User Login Debt” vs. “Custom Reporting Debt”. The groupings may be more subtle still, but the key here is to keep them as small and focused as possible. These groupings are going to breakdown into MRs eventually, and you certainly want contributions of this type to be small and focused to make reviewing and merging the changes easier and worry-free.

Seriously, if you’re not physically nodding your head right now then I encourage you to go back and re-read the previous paragraph, it’s that important that its message resonates with you. An example of not following this guideline is one we see all too often with junior engineers: “Oh this codebase is using a mix of tabs and spaces? Lemme just helpfully run a code fixer over the whole thing and submit that as one huge merge request! My boss will be so happy!” Likely not, friend, sorry. A better strategy would to group the messy files and call it the “Style Fixes” group and grade it low severity.

Your grading scale can be whatever makes sense, I use LOW/MED/HIGH or sometimes A through F.

Eliminate It

If you’re at this step in your tech-debt-purging process and you’re feeling confident, you ABSOLUTELY SHOULD BE! It takes a lot of work to get here when using this framework. By this point, you’ve done all the following:

• Attempted to prevent technical debt from occurring in the first place with automated tools like linters and manual tools like code reviews
• Discussed and understood where the tech debt comes from so you can be on the lookout for it in the future
• Set up a regular review of your technical debt so that you can constantly re-evaluate it, identify patterns, and otherwise stay in control of the list instead of letting it control you
• Divided up your codebase into the areas needing the most attention and made a plan for addressing the key issues in each area

Now…it’s time…to call the Ghostbusters. Just kidding, it’s all on YOU and your team now. It’s time to roll your sleeves up and dig in, one section at a time, addressing the most critical tech debt first and making sure you make small, test-covered changes (where possible) in an effort to clean up the mess that Rob the Manager made last summer when he wrote that login page himself. This can be a slow dredge at first, BUT once the ball really gets rolling it can be a very exciting time on a project. Just approach this work with the same excitement commitment to diligence as you approach feature development (easier said than done, but I believe in you) and you’ll be making huge strides.

Review

Here’s the PURGE framework again, for quick reference:

• P – Prevent It
• U – Understand It
• R – Review It
• G – Group & Grade It
• E – Eliminate It

I’ll be honest, this framework is geared towards larger projects with 5+ engineers more than little projects with one or two. Why? Because, when it’s just two engineers personally I feel way more comfortable ripping stuff out indiscriminately on a Thursday night and putting it back together in a cleaner fashion, finally typing git push at 3AM that Friday morning. But, this is just not tenable for most enterprise-grade software projects. For those, we have not only our fellow engineers, but also stakeholders to whom we are accountable. We must move carefully in order to avoid breaking things (even if our job is to move fast and break things, we shouldn’t BREAK break things).

If you or your organization are facing down a sizeable backlog of technical debt, I do recommend trying my framework to see what kind of footholds you are able to get on the mountain. I think you (and your team, and your stakeholders, and ultimately your customers) will be very pleased, and if you meet with success I encourage you to comment here and let me know!