Self-adaptive Testing in the Field: Are We There Yet?

Abstract

Testing in the field is gaining momentum, as a means to identify those faults that escape in-house testing by continuing the testing even while a system is operating in production. Among several approaches that are proposed, this paper focuses on the important notion of self-adaptivity of testing in the field, as such techniques clearly need to adapt in many ways their strategy to the context and the emerging behaviors of the system under test. In this work, we investigate the topic by conducting a scoping review of the literature on self-adaptive testing in the field. We rely on a taxonomy organized in a number of categories that include the object to adapt, the adaptation trigger, its temporal characteristics, realization issues, interaction concerns, the classification of field-based approach, and its impact/cost. Our study sheds light on self-adaptive testing in the field by identifying related key concepts and key characteristics and extracting some knowledge gaps to better guide future research.