What even is Staff Engineering? Building apps to last

This is part 2 of my series on Staff Engineering. Here's part one if you haven't read and are interested.

In part 1 of this series, we discussed the broad role of a Staff Software Engineer. Today, we’re going to start looking at the specifics. What does a Staff engineer's "everyday" look like? What are their most powerful tools? The first of the more specific points I want to write about is one I'm straight robbing from my long-time Staff+ mentor and fellow WillowTree employee, Kevin Conner.

When I first started as a Staff Engineer, you can imagine that I had many questions about what my shiny new job looked like. My first position in the new role would be to take Kevin's place as tech lead on a 2-year-old project.^[1] Having been on this project with Kevin for most of its life, I already had plenty of firsthand experience watching them do incredible work. To give me an even stronger start on the project, Kevin graciously met with me one-on-one several times to give me tips and pointers on how to be a good Lead Engineer. One of their first and strongest pieces of advice? "Build the app as though it's still going to be around in 10 years."

Now for those who don't know, 10 years is an eternity in the tech space. I've actually said the same about just 3 years in the tech space. Don't believe me? 3 years prior to me writing this sentence (so aaaallll the way back in May 2020), drive-up pickups at stores and restaurants barely existed. Hybrid/remote working was still considered a benefit enjoyed by just the fringe of society, and was likely going away forever as soon as quarantine ended. The term "AI" was mostly a joke that euphemistically meant "the best those poor, dumb computers can do."

“But wait!” you yell, determined to disprove me (or at least determined to give me a chance to make my point). “The 'rona sped some of those things up! This wasn’t a typical 3 years of tech progress!” That’s a fair point. But how about this one: Twitter was a respectable social media platform just 7 months ago. I'll go ahead and mark that one down as making my point for me.^[2] The TLDR is that building an app to last 10 years is absurd.

But that was Kevin's point. The app we were building for our client would almost certainly be long-retired by the time 10 years had passed, but it wouldn't be because the codebase itself forced the issue. This approach was especially pertinent for the client in question because we were in the middle of rebuilding their entire web presence from scratch. So how does one take on a task as formidable as the nigh-impossible? In this case, the first and most important answer to that question is modularization. And here's where we start getting our first look into some of the more important tools in a Staff Software Engineer’s belt.

Modularization

(So, I handed Gary^[3] this article and asked it for editing advice, and it very condescendingly told me to include definitions of each of these topics before diving into my talk about them. So as petty revenge, I made it give me those definitions itself.)

Gary’s definition of modularization: "Modularization in the context of software engineering refers to the process of subdividing a computer program into separate sub-programs or modules. Each module represents a separate function or feature of the program, contributing to the overall functionality of the application.”

One of a Staff engineer's primary roles will be to ensure that their codebase is properly modularized. So what does "properly" mean? Do you remember when [insert latest tech fad here] was first introduced, and we all lost our collective minds and put it into every project ever? Then 6 months later, we all blinked like we'd just woken up from a really weird dream, and went "but…why??" But, then we went ahead and did it all over again the next time a new technology appeared? Of course you remember, it just happened last month. The lesson here is: don't do that with modularization in your codebase. Modularizing every little thing for the sake of it is worse than not doing it at all.

One good strategy is to ensure that each piece of your given tech stack can be cut out of your app as quickly and cleanly as possible, and replaced with another, comparable piece. For example - and this is especially pertinent for web devs who live and work in a world where libraries seem to breed of their own volition - you should be able to rip out your given analytics solution and replace it with another by just changing a couple of files.^[4] A refactor of your entire codebase should never be necessary when changing out one framework or technology for another. And for those of you getting ready to give me an "Actually…," I do mean never. Even if you're making an enormous change that will affect a large portion of your codebase, e.g. swapping React out for Vue.js, there is no (good) reason your entire codebase should change. Things like helper functions that perform common logic, analytics solutions, and logging solutions are all things that should remain untouched during that transition. So, at the risk of making this word mean absolutely nothing by using it yet again, properly modularizing your codebase will ensure that you can remove deprecated or otherwise outdated libraries with as little effort as possible. And the day when some piece of your codebase is deprecated will come and it’ll likely be soon.^[5]

Another excellent daily tool in the Staff Engineer's belt is abstraction. Bear with me here because we're about to get....abstract.

Sorry not sorry. I’m a dad, get used to it.

Abstraction

Gary’s definition of abstraction: “In software engineering, abstraction is a principle that enables handling complexity by hiding unnecessary details and exposing only the essential features of a concept or an object. This makes it possible for developers to work with complex systems by focusing on a higher level of functionality without being overwhelmed by the underlying details.”

One of the age-old adages for us Software Engineers is to "keep things DRY," with DRY standing for "Don't Repeat Yourself." When you first start as a bright-eyed and bushy-tailed Junior Engineer, this adage is strictly literal: do not, under any circumstances, type the same line of code more than once. I once legitimately got annoyed that I kept having to call the same function in a bunch of different places and wondered, for just a second, if there was a way I could shorten the one-line function call. (Thankfully, that second passed quickly and I then proceeded to sit down and wonder if I was okay. I'm clearly not, but that's beside the point here.)

As a Staff engineer, this gets a lot less literal and a lot more abstract. Are there boilerplate chunks of code or patterns that you or your engineers are having to type out by hand (or, more likely, having to copy-pasta) every time? Write a helper function that takes the work out of that. One good example of this is from the first project I tech-led (mentioned at the top of the article). We had a pattern in place for unit tests that would allow us to quickly and easily mock React’s useState functions. The problem I noticed is that we’d have to write a ternary at best, and some complicated if-then logic at worst, every time we used a state variable in a component, to make it easily mockable for unit testing. Eventually, this turned into stumbling around the codebase until we found just the right use case we needed in one of our dozens of other components, and copy-pasting the syntax into the new file.

This made parsing component files unnecessarily complex, and writing them a painfully long process. It was a perfect spot for some abstraction, so I wrote a function that handled all of the possible use cases and made it simple — via a combination of variable names, TypeScript types, and comments — to figure out how to get the use case you needed out of my helper function. At this point, state variables went from a complicated mess of logic to a function call not much more complex than calling useState directly!

Doing this kind of abstraction correctly makes your codebase easier for developers to use, easier to read, and makes it (sometimes literally) infinitely cheaper to change the logic you’re abstracting away into one single function. Without these things, your codebase will quickly get “too messy” and everyone who works on it will quickly get the “let’s rewrite this” bug, which usually damages a codebase’s ability to stick around for 10+ years.

Addressing Tech Debt

Gary’s definition of tech debt: “Technical debt, often referred to as ‘tech debt,’ is a concept in software development that reflects the implied cost of additional rework caused by choosing a quick or easy solution now instead of using a better, but more demanding, approach that would take longer. Technical debt can be intentional or unintentional.”

The last tool in a Staff’s belt that we’ll discuss here today is tech debt. Or I guess the tool would be addressing tech debt. Setting up a codebase for long-term success is hugely beneficial, but you will never set everything up perfectly on the first try. This just isn’t how the world works. Requirements change, your app underperforms in production, users get mad that feature X isn’t available, and the focus needs to shift. It happens. And when it happens, tech debt is accrued.

So let’s talk about tech debt. For starters, it’s scary. No one wants to be in debt. But how much time should you dedicate to addressing tech debt? I’ve heard people throw out numbers (”30% of your time each sprint should be used to address tech debt”), but once again, this feels entirely unrealistic to me. We live in the real world, where deadlines, money, prod issues, etc. all factor into how much time you can reasonably spend on addressing tech debt. I’ve found that the best strategy for me is playing things by ear. Product development tends to come in waves.^[6] Use the dips in between those waves to address some of that tech debt.

The other part of this equation is that not all debt is equal. One piece of tech debt might be a couple of dollars that you put on your credit card a couple of months ago, while another might be more akin to all of your student loan debt. Those are not the same and you should not expect to tackle the first with the same ferocity or urgency as the second. Some things can’t or shouldn’t wait, and others will probably be okay sticking around with a //TODO: Remove above it for all eternity. That’s just software engineering in the real world.

So there’s part two for you. If I’ve done my job correctly, you now have a solid grasp on some of the more important tools you’ll need as a Staff Software Engineer. In my next post, we’ll discuss another one of those tools and continue our deep dive into a Staff Software Engineer’s day-to-day existence. Until next time, you stay classy.