Simplicity is not the solution to Software Complexity
Software just keeps getting more complex. As it gets more complex, it becomes harder to maintain. Fixing bugs begins to feel like exchanging old bugs for new ones. There is a clamor to simplify the code. But is simplification the answer? And, what does simplification mean?
A software system is composed of interrelated elements each with different properties. As the number of elements, relationships and properties increase the system gets progressively more complex. If your application has 1000 different components and those components are actually used, there isn’t much you can do about reducing the number of components.
If a financial software solution deals with many types of securities can you simplify by reducing the number of securities or their interrelationships or the properties that distinguish them? If a medical device requires FDA certification, can you reduce the steps or the artifacts that are required for the certification? If the real world is inherently complex, the software we develop to model it will be complex.
For sure, there is accidental complexity that results from the implementation whether it be inadequate design or tools that are hard to use and understand. How do you separate accidental complexity from inherent complexity? And, regardless of where the complexity comes from, how do you deal with it?
In my experience, there is only one good answer to these questions. You need to organize to deal with inherent complexity. You also need to organize to discover the accidental complexity be it clutter or be it design-debt.
Specialize to Distribute Complexity
As the number of parts of a system increases it becomes harder and harder to keep all the details in our heads. No matter how brilliant you are, you aren’t going to know about the details of all the js files or libraries that are in your Nodejs backend (substitute with your favorite frontend or backend framework). At some point it becomes impossible and we feel that the system has become “complex”. Most software that we used today is maintained by teams of developers and is too complex for one person to understand fully.
The key approach to dealing with complexity is decomposition. We group the elements of the problem into parts and let people specialize in those parts. This is no different from how the complexity of the human body is dealt with by doctors with different specializations. This is a recursive process – as the complexity of a part increases, we decompose further.
The key issue is how to decompose. If you have hundreds of restful APIs how do you group those APIs so that each of the groups is easier to understand? If you have a large number of React components, what is the best strategy to group them? If you have numerous services how do you organize the code that is specific to a service and the code that is shared?
The answer to these questions is what an organization is about.
Organize to Tame Complexity
A good organization tames complexity by dividing it up:
- It allows different parts to be developed independently most of the time. Of course, if the shared parts are changed it will affect all the impacted parts. On the other hand, if the common parts are fairly stable then you might find yourself in a super productive team. A decomposition that combines loose coupling between cohesive parts is the goal.
- The skills needed for each of the parts are different. For instance, the team that works on the front end can master CSS while the backend team can build expertise in databases.
Note that the overall complexity of the system hasn’t decreased. Instead, it is now distributed over multiple teams so that each team (and, ultimately, each developer) deals with reduced complexity. Of course, all this is predicated on an organization that actually allows this to happen.
What about accidental complexity? This complexity is often a result of sub-optimal design. Actually, a good organization is also a great way to uncover poor design. Depending on how bad the design is, the desired decomposition will force you to see and fix the unwanted couplings. Furthermore, poor design in different parts of the decomposition can be dealt with by teams who have a deeper understanding of that part. Often changes improvements can be made without affecting the entire project.
Without good code organization and a team that reflects it, development is like blind men trying to make sense of an elephant. Do you know how your software is organized? When a change is to be made, is it clear what to change and what will be affected by that change? If not, every change will be like a blind man feeling around an elephant.
