Details
| Original language | English |
|---|---|
| Title of host publication | EASE '23: Proceedings of the 27th International Conference on Evaluation and Assessment in Software Engineering |
| Pages | 329-338 |
| Number of pages | 10 |
| ISBN (electronic) | 979-8-4007-0044-6 |
| Publication status | Published - 14 Jun 2023 |
Publication series
| Name | ACM International Conference Proceeding Series |
|---|
Abstract
To address this problem, we combine the already well-established method of expert evaluation with goal-oriented heuristics. Goal-oriented heuristics are heuristics that are grouped with respect to the goals that the explanations are meant to achieve.
By establishing appropriate goal-oriented heuristics, software engineers are enabled to evaluate explanations and identify problems with affordable resources. To show that this way of evaluating explainability is suitable, we conducted an interactive user study, using a high-fidelity software prototype. The results suggest that the alignment of heuristics with specific goals can enable an effective assessment of explainability.
Keywords
- explainability, metrics, software evaluation, heuristics, explainable systems, Metrics, Software Evaluation, Heuristics, Explainability, Explainable Systems
ASJC Scopus subject areas
- Computer Science(all)
- Software
- Computer Science(all)
- Human-Computer Interaction
- Computer Science(all)
- Computer Vision and Pattern Recognition
- Computer Science(all)
- Computer Networks and Communications
Cite this
- Standard
- Harvard
- Apa
- Vancouver
- BibTeX
- RIS
EASE '23: Proceedings of the 27th International Conference on Evaluation and Assessment in Software Engineering. 2023. p. 329-338 (ACM International Conference Proceeding Series).
Research output: Chapter in book/report/conference proceeding › Conference contribution › Research › peer review
}
TY - GEN
T1 - A Means to what End? Evaluating the Explainability of Software Systems using Goal-Oriented Heuristics
AU - Deters, Hannah Luca
AU - Droste, Jakob Richard Christian
AU - Schneider, Kurt
N1 - Funding Information: This work was funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Grant No.: 470146331, project softXplain (2022-2025).
PY - 2023/6/14
Y1 - 2023/6/14
N2 - Explainability is an emerging quality aspect of software systems. Explanations offer a solution approach for achieving a variety of quality goals, such as transparency and user satisfaction. Therefore, explainability should be considered a means to an end. The evaluation of quality aspects is essential for successful software development. Evaluating explainability allows an assessment of the quality of explanations and enables the comparison of different explanation variants. As the evaluation depends on what quality goals the explanations are supposed to achieve, evaluating explainability is non-trivial. To address this problem, we combine the already well-established method of expert evaluation with goal-oriented heuristics. Goal-oriented heuristics are heuristics that are grouped with respect to the goals that the explanations are meant to achieve.By establishing appropriate goal-oriented heuristics, software engineers are enabled to evaluate explanations and identify problems with affordable resources. To show that this way of evaluating explainability is suitable, we conducted an interactive user study, using a high-fidelity software prototype. The results suggest that the alignment of heuristics with specific goals can enable an effective assessment of explainability.
AB - Explainability is an emerging quality aspect of software systems. Explanations offer a solution approach for achieving a variety of quality goals, such as transparency and user satisfaction. Therefore, explainability should be considered a means to an end. The evaluation of quality aspects is essential for successful software development. Evaluating explainability allows an assessment of the quality of explanations and enables the comparison of different explanation variants. As the evaluation depends on what quality goals the explanations are supposed to achieve, evaluating explainability is non-trivial. To address this problem, we combine the already well-established method of expert evaluation with goal-oriented heuristics. Goal-oriented heuristics are heuristics that are grouped with respect to the goals that the explanations are meant to achieve.By establishing appropriate goal-oriented heuristics, software engineers are enabled to evaluate explanations and identify problems with affordable resources. To show that this way of evaluating explainability is suitable, we conducted an interactive user study, using a high-fidelity software prototype. The results suggest that the alignment of heuristics with specific goals can enable an effective assessment of explainability.
KW - explainability
KW - metrics
KW - software evaluation
KW - heuristics
KW - explainable systems
KW - Metrics
KW - Software Evaluation
KW - Heuristics
KW - Explainability
KW - Explainable Systems
UR - http://www.scopus.com/inward/record.url?scp=85162207646&partnerID=8YFLogxK
U2 - 10.1145/3593434.3593444
DO - 10.1145/3593434.3593444
M3 - Conference contribution
T3 - ACM International Conference Proceeding Series
SP - 329
EP - 338
BT - EASE '23: Proceedings of the 27th International Conference on Evaluation and Assessment in Software Engineering
ER -