I aim to help software developers efficiently manage bugs. For my thesis, a step-by-step procedure that effectively predicts buggy code changes (the Bug Prognosticator), produces bug fix suggestions (Fix Suggester), and utilizes human feedback is presented. Each of these steps can also work in isolation, but combining them allows more effective management of bugs. These techniques are tested on many open source and a large commercial project.
The Bug Prognosticator explores feature selection techniques and classifiers to improve results of code change bug prediction. The optimized Bug Prognosticator is able to achieve an average 97% precision and 70% recall when evaluated on eleven projects, ten open source and one commercial.
The Fix Suggester uses the Bug Prognosticator and statistical analysis of keyword term frequencies to suggest unordered fix keywords to a code change predicted to be buggy. The suggestions are validated against actual bug fixes to confirm their validity. The Fix Suggester is able to achieve 46.9% precision and 38.9% recall on its predicted fix tokens. This is a reasonable start to the tough problem of predicting the contents of a bug fix.
To improve the efficiency of the Bug Prognosticator and the Fix Suggester, active learning is employed on willing human participants. Developers aid the Bug Prognosticator and the Fix Suggester on code changes that machines find hard to evaluate. The developer's feedback is used to enhance the performance of the Bug Prognosticator and the Fix Suggester.
Sunghun Kim, Thomas Zimmermann, Rahul Premraj, Nicolas Bettenburg, Shivkumar Shivaji: Predicting method crashes with bytecode operations. ISEC 2013
Shivkumar Shivaji, E. James Whitehead Jr., Ram Akella, Sunghun Kim: Reducing Features to Improve Code Change-Based Bug Prediction. IEEE Trans. Software Eng. 39(4) 2013
Sunghun Kim, Shivkumar Shivaji, E. James Whitehead Jr.: Kenyon-web: Reconfigurable web-based feature extractor. ICPC 2009
Shivkumar Shivaji, E. James Whitehead Jr., Ram Akella, Sunghun Kim: Reducing Features to Improve Bug Prediction. ASE 2009
Ahmed Radwan, Akmal Younis, Mauricio Hernandez, Howard Ho, Lucian Popa, Shivkumar Shivaji, and Sawsan Khuri, "BioFederator: A Data Federation System for Bioinformatics on the Web," Proc. AAAI Sixth Int. Workshop on Information Integration on the Web (IIWeb) 2007.
Rosa Badia, Gargi Dasgupta, Onyeka Ezenwoye, Liana Fong, Howard Ho, Sawsan Khuri, Yanbin Liu, Steve Luis, Anthony Praino, Jean-Pierre Prost, Ahmed Radwan, Seyed Masoud Sadjadi, Shivkumar Shivaji, Balaji Viswanathan, Patrick Welsh, and Akmal Younis, "High Performance Computing and Grids in Action, chapter Innovative Grid Technologies Applied to Bioinformatics and Hurricane Mitigation," pp. 436-462, IOS Press, ISBN 978-1-58603-839-7, Amsterdam, 2008.