Efficient Regression Testing Based on Test History: An Industrial Evaluation
Author
Summary, in English
Due to changes in the development practices at Axis Communications, towards continuous integration, faster regression testing feedback is needed. The current automated regression test suite takes approximately seven hours to run which prevents developers from integrating code changes several times a day as preferred. Therefore we want to implement a highly selective yet accurate regression testing strategy. Traditional code coverage based techniques are not applicable due to the size and complexity of the software under test. Instead we decided to select tests based on regression test history. We developed a tool, the Difference Engine, which parses and analyzes results from previous test runs and outputs regression test recommendations. The Difference Engine correlates code and test cases at package level and recommends test cases that are strongly correlated to recently changed packages. We evaluated the technique with respect to correctness, precision, recall and efficiency. Our results are promising. On average the tool manages to identify 80% of the relevant tests while recommending only 4% of the test cases in the full regression test suite.
Department/s
Publishing year
2015
Language
English
Publication/Series
2015 IEEE International Conference on Software Maintenance and Evolution (ICSME)
Full text
- Available as PDF - 261 kB
- Download statistics
Document type
Conference paper
Publisher
IEEE - Institute of Electrical and Electronics Engineers Inc.
Topic
- Other Engineering and Technologies
Conference name
International Conference on Software Maintenance and Evolution
Conference date
2015-09-29
Conference place
Bremen, Germany
Status
Published
ISBN/ISSN/Other
- ISBN: 978-1-4673-7532-0