A Practical Guide to Select Quality Indicators for Assessing Pareto-Based Search Algorithms in Search-Based Software Engineering

Shuai Wang1, Shaukat Ali1, Tao Yue1,2, Yan Li3, Marius Liaaen4

1Simula Research Laboratory, Oslo, Norway; 2Department of Informatics, University of Oslo, Oslo, Norway; 3Beihang University, Beijing, China; 4Cisco Systems, Oslo, Norway
Contact: {shuai, shaukat, tao}@simula.no, lyadeng79@gmail.com, marliaae@cisco.com

For implementation:

As mentioned in [2], all the six selected Pareto-based search algorithms and the eight quality indicators are implemented based on jMetal [1]. The source codes related with the six Pareto-based search algorithms and eight quality indicators can be downloaded from jMetal’s website [1].

For the experiment data:

Due to the confidential issues from our industrial partners, we cannot share the original data for the industrial case studies (e.g., test case id, fault detection capability value for each test case, requirement description and stakeholder familiarity). But we provide the data (objective function values) by running the six Pareto-based search algorithms 100 times for each industrial problem (as mentioned in Figure 2 in the paper). Thus, for each case study (9 in toal for the three indsutiral problems), 100 files are included for each algorithm and each file consists of the data related with the values of the defined objectives for each run.

Data related with the experiment

Download The data related with Test Suite Minimization (TM) problem;
Download The data related with Test Case Prioritization (TP) problem;
Download The data related with Requirement Allocation (RA) problem.