Study finds that refactoring doesn’t improve code quality

New research finds that, contrary to popular belief, restructuring software code to be more maintainable and efficient may not be worth the time and effort

A Six Million Dollar Man action figure strong superhero
Credit: flickr/Mike Mozart

Refactoring software, that is, restructuring existing source code to make it more readable, efficient, and maintainable, is something all developers do every now and again. Of course, the implicit assumption behind refactoring is that the benefits (time and headaches saved in the future) outweigh the costs (time and effort spent now). However, new experimental research suggests that this may not be the case and that software code quality may not be improved much, if at all, by refactoring.

The study was done by researchers in Sri Lanka and recently published in the International Journal of Software Engineering & Applications titled An Empirical Evaluation of Impact of Refactoring On Internal and External Measures of Code Quality. The goal was to test whether  common refactoring techniques resulted in measurable improvements in software quality, both externally (e.g., Is the code more maintainable?) and internally (e.g., Number of lines of code).

The researchers selected a small-scale application (about 4,500 lines of C# code) used by the academic staff at the University of Kelaniya for scheduling events and managing online documents for evaluation. 10 common refactoring techniques were applied to the code (e.g., Replace Type Code with Subclasses, Replace Conditional with Polymorphism).

To measure the effect of refactoring on external measures of software quality, the researchers selected computer science students from the University of Kelaniya who knew object oriented programming and C#. Half the students evaluated the refactored code for analyzability and changeability while the other half evaluated the un-refactored code.

Execution time and resource utilization of the refactored and un-refactored code were also measured by running each code base 1,000 and averaging the results.

Internal measure of software quality were calculated for the refactored and un-refactored code using Visual Studio. These metrics were: Maintainability Index, Cyclomatic Complexity, Depth of Inheritance, Class Coupling, and Line of Code.

Here are the main conclusions made by the researchers:

Refactoring doesn’t make code easier to analyze

Study participants actually scored the refactored code lower for analyzability, but the difference wasn’t statistically significant.

Refactoring doesn’t make code easier to change

Again, it actually took participants longer to fix bugs in the refactored code, but, again, the difference wasn’t statistically significant.

Refactoring code doesn’t make it run faster

Refactored code had a longer mean execution time than non-refactored code, but the difference wasn’t statistically significant.

Refactoring code doesn’t result in lower resource utilization

Based on 1,000 runs of both code bases, there was no evidence that refactored code consumed less memory than non-refactored code.

There is some evidence that refactored code is more maintainable

The one result that was in refactoring's favor was that the refactored code had slightly higher Maintainability Index score (+4%) than the non-refactored code. But the authors argue that this index, being a composite measure, is of questionable quality. However, using the other internal measures of quality, the refactored code performed either the same (Depth of Inheritance) or worse (Cyclomatic Complexity, Class Coupling, Lines of Code) than the non-refactored code.

The authors acknowledge some might question these results, since they are all based on a small sample size, a small-scale application and fixed set of refactoring techniques. The mileage you get out of refactoring may obviously vary. Still, if you’re not a fan of refactoring, these results may just give you a good reason to not even bother.

ITWorld DealPost: The best in tech deals and discounts.