Is it really complex or is it just unfamiliar?

At work, we’re taking it upon ourselves to rewrite and/or replace a large number of systems that we currently use and rely on. There are tools that have been provided to us by other teams and simply put, we don’t like them. This had me starting to think about why I don’t like them, or you can listen to why other people say that we shouldn’t use the tools. Many times, what you’ll hear is that the solution is really complex, and doesn’t suit our needs. I’ve been guilty of this. I’ve called systems complex, but as we have been building the replacements, I’m seeing some of the complexities coming back.

I’m not new to the idea that systems are complex because they needed to be. In general, I actually find the idea of doing a rewrite as a sign of engineering immaturity. I sometimes will tell myself that if I was a really great engineer, that I would be able to work in any environment, and that it’s a skill issue if I think a system is too complex. I firmly believe in Tesler’s Law but applied to even more contexts.

The law of conservation of complexity, also known as Tesler’s Law, or Waterbed Theory, is an adage in human–computer interaction stating that every application has an inherent amount of complexity that cannot be removed or hidden. Instead, it must be dealt with, either in product development or in user interaction.

Sometimes, these systems are complex because they have to be. I feel this way when I think about systems like Kubernetes. People will tell me that Kubernetes is overly complex, but when I ask them to elaborate on what they find complex, they give me answers like: “Why can’t you just have more storage? Why do I have to get this persistent volume thing?”.

Funny enough, I think the people who feel most that Kubernetes is complex are the ones who are more of developers and less the type of person who does operational work. Deploying software is just really hard. There are a lot of considerations.

Configuration is hard. If your software can take 4 different boolean configurations, that means that there’s 2 to the power of 4 possible ways for your software to be configured. All you have is 4 configurations and that’s 16 versions of your application that needs to be scheduled.

At what point are you actually just unfamiliar with the system? When you write the software yourself, it becomes easier to learn about each piece because you’re probably writing each piece. Or at the least, you’re reviewing the pieces as it goes into the system. You have so much more opportunity to gather and build the context of the development history. Having context is really a blessing. I like what I call “software archaeology”. Trying to figure out why a system made certain choices is really fascinating to me. Sometimes I wonder if studying software should be considered its own discipline entirely.

I rewatch Simple Made Easy every few years. Studying the definitions and histories of words is also really interesting to me. Figuring out what words mean is really hard. Yet again, the context that you hear the word is usually how people learn about words. How often are we going through and actually researching the words that we use?

When we’re faced with a new system that we have to work in, I now always like to ask, is it complex because it actually has to be? Is it complex because I’m missing context? Or perhaps it’s only complex because it’s unfamiliar? How can I decide that for myself? Where are the lines?

Since I’m an operations heavy engineer, I’ve always found that configuration is a big source of complexity. Backwards compatibility is another one. Having to maintain backwards compatibility naturally implies some kind of cruft or rust in your system’s bones. Software is so commonly additive over time. I can’t think of many times where I’ve removed software. Doing a rewrite doesn’t count because you’re not removing anything so much as you’re trying to replace everything wholesale. State management is another big source of complexity.

When I think of a few of these dimensions for complexity, it helps me frame my mental model for analyzing or determining this trade off of sincere complexity versus unfamiliarity. The answer is almost always that there’s a legitimate reason for the complexity and that the people who complain about the complexity only believe it to be unnecessarily complex because they don’t have the history or context that’s been built up.