A Case Study on Combining Model-based Testing and Constraint Programming for Path Coverage

Abstract

This paper within the advances in software testing is related to the generation of test suites and black box testing. Test coverage is a popular technique to evaluate test quality: a test suite that can exercise a large part of the paths in a program will provide higher confidence that the program will work as expected. One way to obtain test suites with high path coverage is through model-based testing, and specifically using a diagram, which represents the various states in the program and their possible transitions. Using model-based testing, it is possible to obtain a set of paths that achieves a set of coverage criteria. Constraint programming approaches can then be used to turn each path into a test case with its program inputs and oracle, by solving the input and output constraints collected along the path. In this paper, we present a case study on this combination of techniques, by developing a state diagram that represents a popular piece of software, and using it to derive a set of test cases that achieve path coverage through constraint programming. Specifically, two tools have been used: GraphWalker (to obtain a set of paths through the program) and MiniZinc (to obtain test inputs by solving the constraints along the paths). We have obtained good results on the cat GNU Coreutils utility, especially in terms of ease of understanding the logic of the software through the GraphWalker diagram and its visual execution, as well as the agility in obtaining the restrictions to be solved to achieve path coverage.