What Does Control Flow Really Look Like? Eyeballing the Cyclomatic Complexity Metric

Assessing the understandability of source code remains an elusive yet highly desirable goal for software developers and their managers. While many metrics have been suggested and investigated empirically, the McCabe cyclomatic complexity metric (CC) --- which is based on control flow complexity --- seems to hold enduring fascination within both industry and the research community. However, the CC metric also has obvious limitations. For example, it is easy to produce example code that seems trivial to understand yet has a high CC value; at the same time, one can also produce "spaghetti" code with many GOTOs that has the same CC value as a well-structured alternative. In this work, we explore the causal relationship between CC and understandability through quantitative and qualitative studies, and through thought experiments and discussion. Empirically, we examine eight well-known open source Java systems by grouping the abstract control flow patterns of the methods into equivalence classes and exploring the results. We found several surprising results: first, the number of unique control flow patterns is relatively low; second, CC often does not accurately reflect the intricacies of Java control flow; and third, methods with high CC often have very low entropy, suggesting that they may be relatively easy to understand. These findings appear to challenge the widely-held belief that there is a clear-cut causal relationship between understandability and cyclomatic complexity, and suggest that at the very least CC and similar measures need to be reconsidered and refined if they are to be used as a metric for code understandability.