Near Failure Analysis Using Dynamic Behavioural Data

Automated testing is a safeguard against software regression and provides huge benefits. However, it is yet a challenging subject. Among others, there is a risk that the test cases are too specific, thus making them inefficient. There are many forms of undesirable behaviour that are compatible with a typical program’s specification, that however, harm users. An efficient test should provide most possible information in relation to the resources spent. This paper introduces near failure analysis which complements testing activities by analysing dynamic behavioural metrics (e.g., execution time) in addition to explicit output values. The approach employs machine learning (ML) for classifying the behaviour of a program as faulty or healthy based on dynamic data gathered throughout its executions over time. An ML-based model is designed and trained to predict whether or not an arbitrary version of a program is at risk of failure. The very preliminary evaluation demonstrates promising results for feasibility and effectiveness of near failure analysis.