Explainable Artificial Intelligence to Improve Interpretability in Predictive Mutation Testing

Abstract

Predictive Mutation Testing (PMT) has been developed to alleviate the significant computational demands of traditional mutation testing by employing machine learning models to predict whether a test suite can detect artificially introduced code changes, known as mutants. Despite advances in predictive accuracy, the interpretability of these models has been limited, restricting insights into why specific predictions are made. This study investigates the application of explainability methods, specifically SHAP and LIME, to enhance the interpretability of PMT models. Using a comprehensive dataset of over 4 million mutants sourced from real-world Java projects, we analyze the contributions of various features to model predictions. Key findings indicate that features related to test execution, such as the number of times a mutation is executed or the number of tests from the test suite covering the mutation, play crucial roles in accurately detecting mutants. Additionally, code complexity correlates with higher mutant survival rates. The influence of the mutation operator highlights that different mutation types inherently vary in detection difficulty. This work not only demonstrates the effectiveness of SHAP and LIME in understanding model decisions but also provides actionable insights for improving test case design.