Cyclicality and Bugs

18 Jun 2013
Posted by Neeraj Sangal

Metrics have an obvious charm. If we could measure the quality of a system then we could track it and act as soon as it starts to degrade. But can we even hope to come up with a metric that works across something as complex as the software that runs the Mars Rover or something as simple as the software that plays the game of Tic-Tac-Toe?

Remember that the metric(s) we seek is not likely to be an individual metric such as file size, or the number of paths within a method, or even the count of bugs filed against each component. Useful as these individual metrics may be, what we want is something that is a predictor of the overall system quality. A metric that, if monitored, will help us manage the overall quality as the system evolves.

Indeed there are a number of system metrics to consider. They are graph theoretic. They come under names such as: Cohesion, Coupling, Cyclicality, Normalized Cumulative Dependency, Propagation Cost, Stability etc. But, how do we know if they are a good predictor of system quality? Research is beginning to catch up. New research shows a clear correlation between the cyclicality of your code and how buggy it is.

Cyclicality Matters

Interestingly, the evidence comes not from some dyed-in-the-wool computer science guru but from astute observers whose work is rooted in business and management. It’s a trio of business school professors collaborating across the Altantic Ocean: Manual Sosa, Tyson Browning and Mihm Jurgen. They are skilled at statistics and experts at teasing apart and verifying correlations.

And their conclusion is: Cyclicality of your code has a bearing on how buggy it is.

This conclusion may not be a surprise to many software engineers; and yet, it is a big deal because now we have large scale empirical evidence that demonstrates it. The researchers examined more than a hundred releases for various open source projects. They conclude that cyclicality of code and the presence of bugs in it are correlated. Their research goes deeper into the nature of cyclicality as well. The size of the cycle, the centrality of the component in the cycle, and the lack of encapsulation of the cycle all have an impact on the quality. They also present interesting results about “hubs” which are generally good until there are “overdone”.

You can peruse a highly readable article that summarizes the results of the research. You can also delve deeper into articles [7] and [9] at this link for the original work.

And then there is the question of “why”. Why do the bugs in code increase if cyclicality increases? The answer is not, nor is it likely to be, a mathematical theorem. Instead, the answer lies in how our brains function and how we think. I believe that cycles, particularly large cycles, make it harder for us to think about abstractions in a coherent way. This is also why architecture is so valuable. The systems we design and maintain are less prone to errors when we can think about them in ways that makes them understandable and maintainable.

Postscript: There is additional research that has arrived at largely the same conclusion. It’s from MIT in a doctoral thesis by Dan Sturtevant. Dan is a seasoned software engineer with a PhD in Systems Engineering. His pioneering work examined cyclicality using techniques that go well beyond traditional static code analysis. Dan’s work suggests that not just the bugginess of code but even employee turnover may have something to do with large scale cyclicality! Companies struggling with woes related to their software systems might consider giving him a call (Dan is at Harvard these days).

Posted by Neeraj Sangal

There is a major new book out: Design Structure Matrix Methods and Applications. The authors, Steven D. Eppinger and Tyson R. Browning, may not be household names, but they are rock stars in the world of DSMs. This book contains a clear and lucid introduction to DSMs. However, what makes this book so compelling is its rich collection of examples.

The book contains 44 real world examples as varied as NASA's Mars Pathfinder, BP’s LNG Terminal, LLBean’s Software Code Base, or Luftfahrt’s Airport Security System. These examples drive home the generality and simplicity of DSMs. I am also proud to note that the LLBean analysis was done using Lattix tools.

The book groups these examples into 4 categories:

  • Product Architecture
  • Organization Architecture
  • Process Architecture
  • Multidomain Architecture

While the first three applications have existed for some time, the application of DSMs to link and analyze multiple domains is relatively new. This is a natural progression; and, it raises interesting new possibilities. For instance, we know that different stake holders have different ways of looking at the architecture of a complex system. We often choose to think of them as separate views. Yet, these views are related. Can a system be designed without the designer being cognizant of how it will be deployed? The multi-domain DSMs may provide an effective mechanism to capture those relationships.

Lattix 7.0 Profiles

02 Dec 2011
Posted by Neeraj Sangal

We released Lattix 7.0 last month and its been a hectic time.

Let's take a quick glance at history. Lattix 6.0 was released last year and was followed up by a point release every month, each adding significant new features and usability.

We now have the platform for the next round of improvements. I will discuss various new features of Lattix in my upcoming blog postings. Today we will talk about the concept of a Profile, new in Lattix 7.0.

Lattix 7.0 sports 4 different profiles:

What is Profile?

A profile is a configuration of Lattix that comprises of related modules, tools, scripts, and default settings to support a common usage pattern.

Consider just a few examples (these are just examples, each profile has many more capabilities):

  • You are doing C/C++ development. If you are working on a mobile device, you may also have an interest in understanding how your Java application uses JNI to call into C/C++ based services. If your code is strongly object oriented, you may be interested in combining source(cpp) and header(hpp) into a single abstraction.
  • You are applying Lattix to Java applications. It is highly probable that you also care about Spring, Hibernate, EJBs, Hibernate and other frameworks. You may even have an interest in databases.
  • Your primary environment is a .NET based enterprise. You want to understand how managed code interacts with unmanaged code. You may also want to understand how the schemas, tables, stored procedures of your SQL Server are organized.
  • You are an expert in DSMs. You use Lattix for its powerful restructuring and impact analysis capabilities. Your data is in Excel. You are used to different conventions for dependencies and ordering.

A profile makes Lattix easier to use and gives visibility to a host of related capabilities that are built into Lattix.

What if your use of Lattix does not correspond to one of these profiles? In that case, you always have the ALL profile, which makes all the modules, tools and scripts available. In fact, the intrepid users can create their own profile or alter an existing profile.