Welcome

Welcome to Reks home page. The Reks project targets at supporting refactoring-aware regression test selection during software development process.

Abstract

Regression testing checks that recent project changes do not break previously working functionality. Although important, regression testing is a costly when changes are frequent. Regression test selection (RTS) optimizes regression testing by running only tests whose results might be affected by a change. Traditionally, RTS collects dependencies (e.g., files) for each test and skips the tests, at a new project revision, whose dependencies did not change. Existing RTS techniques do not differentiate behavior-preserving transformations (i.e., refactorings) from other code changes. As a result, tests are run more frequently than necessary.

We present the first step toward a refactoring-aware RTS technique, dubbed Reks, which skips tests affected only by behavior-preserving changes. Reks defines rules to update the test dependencies without running the tests. To ensure that Reks does not hide any bug introduced by the refactoring engines, we integrate Reks only in the pre-submit testing phase, which happens on the developers' machines. We evaluate Reks by measuring the savings in the testing effort. Specifically, we reproduce 100 refactoring tasks performed by developers of 37 projects on GitHub. Our results show that Reks would not run, on average, 33% of available tests (that would be run by a refactoring-unaware RTS technique). Additionally, we run 27 refactoring types on ten open-source projects. The results, based on 74,160 refactoring tasks, show that Reks would not run, on average, 16% of tests (max: 97% and SD: 24%). Finally, our results show that the Reks update rules take negligible time.

Refactorings

We evaluated Reks by measuring savings in the testing effort. Specifically, we reproduced 100 refactoring tasks performed by open-source developers of 37 projects on GitHub. Our results show that Reks would not run, on average, 33% (max: 100% and SD: 33%) of available tests; these tests would be run by a refactoring-unaware RTS technique.

Table 1. Refactorings From Open-Source Projects

Contact

Please send comments or questions to Kaiyuan Wang.