The Importance of Managing Complexity

Introduction

We write software to make our lives easier; to reduce the complexity and chaos. The need to reduce complexity is at the very center of software development. Unfortunately, we tend to mirror the complexity of the real world in our software and this leads to many problems. As Bruce Schneier in his essay A Plea for Simplicity: You can’t secure what you don’t understand said, “the worst enemy of software security is complexity.” In this blog post, we will talk about the importance of managing complexity and how to reduce complexity in software.

The Importance of Managing Complexity

Fred Brooks, in his paper “No Silver Bullet: Essence of Accidents of Software Engineering”, explains there are two types of issues that introduce complexity, the essential and the accidental properties. Essential properties are the properties that a thing must have to be that thing. Accidental properties are the properties a thing happens to have, but they are not really necessary to make the thing what it is. Accidental properties could also be called incidental, discretionary, or optional properties. Brooks argues that difficulties from accidental properties have been addressed with higher-level languages, the introduction of IDEs, hardware enhancements, etc.

This leaves the problem of essential difficulties, and essential difficulties are harder to solve. You are now interfacing with the complex, chaotic real world where you must identify all the dependencies and edge cases for a problem. As software addresses larger and larger problems, interactions between entities become more complex, thus increasing the difficulty of creating new software solutions. The root of all essential difficulties is complexity.

Conceptual errors (specifications, design, and testing) are much more damaging than syntax errors. C.A.R Hoare, the inventor of the quicksort algorithm, stated in the talk The Emperor’s Old Clothes, “there are two ways of constructing a software design: one way is to make it so simple that there are obviously no deficiencies, and the other is to make it so complicated that there are not obvious deficiencies.” Steve McConnell in Code Complete states that software’s primary technical imperative is to manage complexity. When software projects fail for technical reasons, one reason is often uncontrolled complexity (i.e. things are so complex that no one really knows what is going on). Edsger Dijkstra pointed out that no one’s skull can contain a whole program at once; we should try to organize our programs in such a way that we can safely focus on one part at a time.

Brooks paints a pessimistic picture of the future stating there is no “silver bullet” in either technology or management technique that promises even an order of magnitude improvement in productivity, reliability, or simplicity. While there is no silver bullet, there are things that can be done to reduce complexity.

How to Reduce Complexity

When designs are inefficient and complicated they can be broken down into three categories:

1. A complex solution to a simple problem
2. A simple, incorrect solution to a complex problem
3. An inappropriate, complex solution to a complex problem

As stated above, the most important technical goal is managing complexity. In terms of software architecture, complexity can be reduced by dividing a system into a series of subsystems. The goal is to break a complicated problem into simpler, easier to manage pieces. The less subsystems are dependent on each other the safer it is to focus on one area of complexity at a time. McConnell gives 14 recommendations on how to conquer complexity:

1. Dividing a system into subsystems at the architecture level so your brain can focus on a smaller section of the system at one time
2. Carefully defining class interfaces so you can ignore the internal workings of the class
3. Preserving the abstraction represented by the class interface so your brain doesn’t have to remember arbitrary details
4. Avoiding global data, because global data vastly increases the percentage of the code you need to juggle in your brain at any one time
5. Avoiding deep inheritance hierarchies because they are intellectually demanding
6. Avoiding deep nesting loops and conditionals because they can be replaced by simpler control structures.
7. Avoiding gotos because they introduce nonlinearity that has been found to be difficult for most people to follow
8. Carefully defining your approach to error handling rather than using an arbitrary number of different error-handling techniques
9. Being systematic about the use of built-in exception mechanisms, which can become nonlinear control structures that are about as hard to understand as gotos if not used with discipline
10. Not allowing classes to grow into monster classes that amount to whole programs in themselves (Lattix Architect has a number of metrics that can help with this)
11. Keeping routines short
12. Using clear, self-explanatory variable names so your brain doesn’t have to waste cycles remembering details like “i stands for the account index, and j stands for the customer index, or was it the other way around?”
13. Minimizing the number of parameters passed to a routine, or, more importantly, passing only the parameters needed to preserve the routine interface’s abstractions
14. Using conventions to spare your brain the challenge of remembering arbitrary accidental differences between different sections of code

Conclusion

The importance of managing complexity can be seen when we start looking at other people’s source code. Depending on the complexity of the software, this can be a daunting task. As stated in Code for the Maintainer, “always code and comment in such a way that if someone a few notches junior picks up the code, they will take pleasure in reading and learning from it.” Lattix Architect has helped hundreds of companies reduce the complexity of their source code by visualizing, optimizing, and controlling their software architecture.