Enhancing software fault prediction with deep neural networks: An empirical analysis of error-type metrics

Phung, Khoa; Ogunshile, Emmanuel; Aydin, Mehmet E

Abstract

In the context of software quality assurance, Software Fault Prediction (SFP) serves as a critical technique to optimise costs and efforts by classifying software modules as faulty or not, using pertinent project characteristics. Despite considerable progress, SFP techniques seem to have hit a "performance ceiling", mainly due to the limitations of small-scale datasets from public repositories and the challenge of selecting the most appropriate software metrics for each unique application domain. Additionally, traditional machine learning techniques have been the mainstay for fault-proneness prediction, leaving the potential of more advanced methodologies, such as Deep Neural Networks (DNNs), largely unexplored. This paper addresses these gaps by introducing an innovative approach for fault-proneness prediction through the application of DNNs trained with Error-type metrics on industrial open-source software projects. Error-type metrics, with their application-agnostic nature and proven capabilities in improving prediction performances, are leveraged to facilitate broader informational content in the training data and overcome the "performance ceiling". The empirical results reveal that DNN models, trained with Error-type metrics, have shown significant performance improvements of up to 40% in terms of AUC and ROC when compared to models built using the conventional CK metrics. Notably, our proposed methodology also demonstrates superior performance when compared to state-of-the-art DNN models, even those that leverage the sophisticated self-attention mechanism, with our approach surpassing them by up to 17.86%.

Citation

Phung, K., Ogunshile, E., & Aydin, M. E. (in press). Enhancing software fault prediction with deep neural networks: An empirical analysis of error-type metrics.

• Conference Name: International Conference on Applied Artificial Intelligence (AICONF’23)
• Conference Location: Pitt Rivers Museum, the University of Oxford, England, UK
• Start Date: Oct 5, 2023
• End Date: Oct 6, 2023
• Acceptance Date: Jul 29, 2023
• Deposit Date: Sep 29, 2023
• Series ISSN: 1613-0073
• Keywords: Error-type Metrics; Software Metrics; Deep Neural Network; Software Fault Prediction
• Public URL: https://uwe-repository.worktribe.com/output/11143041
• Publisher URL: https://iaicf.net/publications/