Welcome to Reks home page. The Reks project targets at supporting refactoring-aware regression test selection during software development process.
Regression testing checks that recent project changes do not break previously working functionality. Although important, regression testing is a costly when changes are frequent. Regression test selection (RTS) optimizes regression testing by running only tests whose results might be affected by a change. Traditionally, RTS collects dependencies (e.g., files) for each test and skips the tests, at a new project revision, whose dependencies did not change. Existing RTS techniques do not differentiate behavior-preserving transformations (i.e., refactorings) from other code changes. As a result, tests are run more frequently than necessary.
We present the first step toward a refactoring-aware RTS technique, dubbed Reks, which skips tests affected only by behavior-preserving changes. Reks defines rules to update the test dependencies without running the tests. To ensure that Reks does not hide any bug introduced by the refactoring engines, we integrate Reks only in the pre-submit testing phase, which happens on the developers' machines. We evaluate Reks by measuring the savings in the testing effort. Specifically, we reproduce 100 refactoring tasks performed by developers of 37 projects on GitHub. Our results show that Reks would not run, on average, 33% of available tests (that would be run by a refactoring-unaware RTS technique). Additionally, we run 27 refactoring types on ten open-source projects. The results, based on 74,160 refactoring tasks, show that Reks would not run, on average, 16% of tests (max: 97% and SD: 24%). Finally, our results show that the Reks update rules take negligible time.
We evaluated Reks by measuring savings in the testing effort. Specifically, we reproduced 100 refactoring tasks performed by open-source developers of 37 projects on GitHub. Our results show that Reks would not run, on average, 33% (max: 100% and SD: 33%) of available tests; these tests would be run by a refactoring-unaware RTS technique.
Project | Commit | Refactoring Type |
---|---|---|
Pure Refactorings (the commit only contains refactoring changes) | ||
truth | 53f457b7 | Rename Method |
truth | 95018e20 | Rename Method |
truth | 23d88f9d | Rename Class |
truth | da634db4 | Rename Method |
truth | 95938102 | Rename Class |
truth | 742cbeb4 | Encapsulate Field |
errorprone | 8774176e | Rename Class + Move Class |
auto | 75a9cee0 | Rename Class |
oldauto | 6aedf2fd | Rename Package |
jimfs | 1e39f6a5 | Rename Method |
jimfs | 118fd4ec | Rename Method |
re2j | cbf6cce7 | Rename Class |
re2j | 1cd2113d | Remove Unused Method |
openrtb-doubleclick | a0a02631 | Rename Class |
openrtb | fb85f7c1 | Rename Class + Rename Method |
mug | 6c68aa81 | Rename Method |
mug | 94e1e0bf | Rename Local + Inline Method |
mug | fd5acc53 | Rename Method |
mug | 646f5ee0 | Rename Method |
MOE | c59435b7 | Rename Package + Rename Class + Move class |
MOE | 56e5463d | Rename Class |
closure-templates | 40cf35b2 | Rename Method |
closure-templates | c2b2169e | Rename Package + Rename Class + Rename Method |
closure-templates | 0ed335c3 | Rename Class |
closure-templates | e01bd363 | Rename Class + Rename Method + Rename Local |
closure-compiler | dbd00b98 | Rename Class |
closure-compiler | b9f92fc1 | Rename Method |
closure-compiler | b70f1d80 | Rename Class |
closure-compiler | 13d3f477 | Rename Class |
closure-compiler | 4a7d2609 | Rename Method |
closure-compiler | 58b1dea7 | Rename Method |
closure-compiler | 9e9565d5 | Rename Local |
closure-compiler | f4a119be | Rename Field + Rename Method |
compile-testing | 3c8eb886 | Rename Package |
compile-testing | 224c3266 | Rename Package |
commons-codec | 63687e70 | Rename Field |
commons-codec | f282e951 | Rename Local |
commons-codec | 89de3df8 | Rename Field + Rename Method |
commons-codec | d61d1cf0 | Rename Field + Rename Method |
commons-compress | 3e3a1388 | Rename Class |
commons-compress | 2b5ba89b | Rename Field |
commons-compress | 9f418190 | Rename Method |
commons-compress | ce13559c | Move Method |
commons-compress | 3e45dc86 | Rename Field |
commons-compress | f3a5bf68 | Move Method |
commons-crypto | ad81d236 | Rename Class + Rename Method |
commons-crypto | 1d854e99 | Rename Method |
commons-crypto | 69889c4a | Rename Class |
commons-crypto | 1a33867f | Move Field |
commons-crypto | afa3d61e | Rename Class |
commons-dbcp | 8c1a7f37 | Rename Class |
commons-io | 24330e77 | Rename Field + Rename Local |
commons-io | cf971eb0 | Rename Method |
commons-io | 508f4664 | Rename Method |
commons-io | 94afbe46 | Move Class |
commons-csv | a72c71f5 | Rename Class + Rename Field + Rename Local |
commons-csv | aa0762d5 | Rename Field + Rename Local |
commons-csv | 040c2606 | Rename Method + Rename Local |
commons-csv | 7ac5dd3e | Rename Method |
commons-configuration | 25227140 | Rename Class |
commons-configuration | 3410aed6 | Rename Class |
commons-configuration | 60e0cbe8 | Move Class |
wicket | 9ec235f9 | Rename Class |
wicket | 431fba86 | Rename Class |
commons-math | 346a81d7 | Rename Interface |
commons-math | 3fd9cf17 | Rename Method |
commons-pool | a6246167 | Move Field |
commons-pool | b954d153 | Rename Field |
commons-pool | 15067e86 | Rename Field |
commons-pool | 6d9eadeb | Rename Field |
byte-buddy | f1dfb66a | Pull Up Method |
bitcoinj | a6601066 | Move Attribute + Move Method |
bitcoinj | 95bfa406 | Extract Method + Introduce Parameter |
bitcoinj | 2fd96c77 | Extract Method |
camel | 9f319029 | Extract Interface |
checkstyle | a07cae0a | Inline Method |
docx4j | 1ba36143 | Rename Package |
graylog2-server | f05e86c4 | Move Attribute |
metrics | 4c6ab3d7 | Extract Method |
robovm | 1ef86e69 | Rename Package |
sonarqube | 06fa5718 | Move Class |
commons-collections | 80b08f8c | Move Method + Rename Method |
commons-collections | ed085d06 | Rename Class |
commons-lang | a49ea7a6 | Rename Local |
commons-lang | bcb33ec1 | Rename Class |
commons-lang | d9bd12dc | Rename Class |
commons-lang | 3efd6ccd | Extract Constant |
commons-lang | 8a82d7b7 | Rename Class |
commons-lang | f5a83bb9 | Extract Method |
commons-lang | f416eb20 | Rename Field |
commons-lang | b168f291 | Move Class |
commons-lang | a7530eae | Rename Method |
Mixed Changes (the commit contains both refactoring changes and non-refactoring changes) | ||
singularity | 46f65138 | Pull Up Attribute |
camel | 5e08a9e8 | Extract Method |
jline2 | 80d3ffb5 | Extract Method |
robovm | 7837d0ba | Extract Method |
druid | 87f3f814 | Extract Method |
sonarqube | 091ec857 | Move Class |
truth | d4f564ba | Rename Method |
jimfs | 5ad28d5e | Rename Class |
Please send comments or questions to Kaiyuan Wang.