Details
| Original language | English |
|---|---|
| Article number | 100040 |
| Journal | Expert Systems with Applications: X |
| Volume | 8 |
| Early online date | 19 Aug 2020 |
| Publication status | Published - Nov 2020 |
Abstract
Data extracted from software repositories is used intensively in Software Engineering research, for example, to predict defects in source code. In our research in this area, with data from open source projects as well as an industrial partner, we noticed several shortcomings of conventional data mining approaches for classification problems: (1) Domain experts’ acceptance is of critical importance, and domain experts can provide valuable input, but it is hard to use this feedback. (2) Evaluating the quality of the model is not a matter of calculating AUC or accuracy. Instead, there are multiple objectives of varying importance with hard to quantify trade-offs. Furthermore, the performance of the model cannot be evaluated on a per-instance level in our case, because it shares aspects with the set cover problem. To overcome these problems, we take a holistic approach and develop a rule mining system that simplifies iterative feedback from domain experts and can incorporate the domain-specific evaluation needs. A central part of the system is a novel multi-objective anytime rule mining algorithm. The algorithm is based on the GRASP-PR meta-heuristic but extends it with ideas from several other approaches. We successfully applied the system in the industrial context. In the current article, we focus on the description of the algorithm and the concepts of the system. We make an implementation of the system available.
Keywords
- Explainable artificial intelligence, Human-in-the-loop, Interpretable artificial intelligence, Multi-objective, Rule mining, Set cover
ASJC Scopus subject areas
- Engineering(all)
- Computer Science(all)
- Computer Science Applications
- Computer Science(all)
- Artificial Intelligence
Cite this
- Standard
- Harvard
- Apa
- Vancouver
- BibTeX
- RIS
In: Expert Systems with Applications: X, Vol. 8, 100040, 11.2020.
Research output: Contribution to journal › Article › Research › peer review
}
TY - JOUR
T1 - GIMO
T2 - A multi-objective anytime rule mining system to ease iterative feedback from domain experts
AU - Baum, Tobias
AU - Herbold, Steffen
AU - Schneider, Kurt
PY - 2020/11
Y1 - 2020/11
N2 - Data extracted from software repositories is used intensively in Software Engineering research, for example, to predict defects in source code. In our research in this area, with data from open source projects as well as an industrial partner, we noticed several shortcomings of conventional data mining approaches for classification problems: (1) Domain experts’ acceptance is of critical importance, and domain experts can provide valuable input, but it is hard to use this feedback. (2) Evaluating the quality of the model is not a matter of calculating AUC or accuracy. Instead, there are multiple objectives of varying importance with hard to quantify trade-offs. Furthermore, the performance of the model cannot be evaluated on a per-instance level in our case, because it shares aspects with the set cover problem. To overcome these problems, we take a holistic approach and develop a rule mining system that simplifies iterative feedback from domain experts and can incorporate the domain-specific evaluation needs. A central part of the system is a novel multi-objective anytime rule mining algorithm. The algorithm is based on the GRASP-PR meta-heuristic but extends it with ideas from several other approaches. We successfully applied the system in the industrial context. In the current article, we focus on the description of the algorithm and the concepts of the system. We make an implementation of the system available.
AB - Data extracted from software repositories is used intensively in Software Engineering research, for example, to predict defects in source code. In our research in this area, with data from open source projects as well as an industrial partner, we noticed several shortcomings of conventional data mining approaches for classification problems: (1) Domain experts’ acceptance is of critical importance, and domain experts can provide valuable input, but it is hard to use this feedback. (2) Evaluating the quality of the model is not a matter of calculating AUC or accuracy. Instead, there are multiple objectives of varying importance with hard to quantify trade-offs. Furthermore, the performance of the model cannot be evaluated on a per-instance level in our case, because it shares aspects with the set cover problem. To overcome these problems, we take a holistic approach and develop a rule mining system that simplifies iterative feedback from domain experts and can incorporate the domain-specific evaluation needs. A central part of the system is a novel multi-objective anytime rule mining algorithm. The algorithm is based on the GRASP-PR meta-heuristic but extends it with ideas from several other approaches. We successfully applied the system in the industrial context. In the current article, we focus on the description of the algorithm and the concepts of the system. We make an implementation of the system available.
KW - Explainable artificial intelligence
KW - Human-in-the-loop
KW - Interpretable artificial intelligence
KW - Multi-objective
KW - Rule mining
KW - Set cover
UR - http://www.scopus.com/inward/record.url?scp=85090830401&partnerID=8YFLogxK
U2 - 10.1016/j.eswax.2020.100040
DO - 10.1016/j.eswax.2020.100040
M3 - Article
AN - SCOPUS:85090830401
VL - 8
JO - Expert Systems with Applications: X
JF - Expert Systems with Applications: X
M1 - 100040
ER -