Performance mutation testing: Hypothesis and open questions

Abstract

Performance bugs are common, costly, and elusive. Performance tests aim to detect performance bugs by running the program with specific inputs and determining whether the observed behaviour is acceptable. There not exist mechanisms, however, to assess the effectiveness of performance tests. Mutation testing is a technique to evaluate and enhance functional test suites by seeding artificial faults in the program under test. In this new idea paper, we explore the applicability of mutation testing to assess and improve performance tests. This novel approach is motivated with examples and open questions.