Indirect Effects in Evidential Assessment: A Case Study on Regression Test Technology Adoption

Background: There is a need for ecient regression testing
in most software development organizations. Often the
proposed solutions involve automation. However, despite
this being a well researched area, research results are rarely
applied in industrial practice. Aim: In this paper we aim
to bridge the gap between research and practice by providing
examples of how evidence-based regression testing
approaches can be adopted in industry. We also discuss
challenges for the research community. Method: An industrial
case study was carried out to evaluate the possibility
to improve regression testing at Sony Ericsson Mobile Communications.
We analyse the procedure undertaken based
on frameworks from the evidence based software engineering,
EBSE, paradigm (with a focus on the evidence) and
automation literature (with a focus on the practical eects).
Results: Our results pinpoint the need for systematic approaches
when introducing a new tool. Practitioners and researchers
need congruent guidelines supporting the appraisal
of both the evidence base and the pragmatic eects, both
direct but also indirect, of the changes. This is illustrated
by the introduction of the automation perspective.