Details
| Original language | English |
|---|---|
| Pages (from-to) | 861-893 |
| Number of pages | 33 |
| Journal | Journal of Artificial Intelligence Research |
| Volume | 64 |
| Publication status | Published - 16 Apr 2019 |
| Externally published | Yes |
Abstract
Good parameter settings are crucial to achieve high performance in many areas of artificial intelligence (AI), such as propositional satisfiability solving, AI planning, scheduling, and machine learning (in particular deep learning). Automated algorithm configuration methods have recently received much attention in the AI community since they replace tedious, irreproducible and error-prone manual parameter tuning and can lead to new state-of-the-art performance. However, practical applications of algorithm configuration are prone to several (often subtle) pitfalls in the experimental design that can render the procedure ineffective. We identify several common issues and propose best practices for avoiding them. As one possibility for automatically handling as many of these as possible, we also propose a tool called GenericWrapper4AC.
ASJC Scopus subject areas
- Computer Science(all)
- Artificial Intelligence
Cite this
- Standard
- Harvard
- Apa
- Vancouver
- BibTeX
- RIS
In: Journal of Artificial Intelligence Research, Vol. 64, 16.04.2019, p. 861-893.
Research output: Contribution to journal › Article › Research › peer review
}
TY - JOUR
T1 - Pitfalls and Best Practices in Algorithm Configuration
AU - Eggensperger, Katharina
AU - Lindauer, Marius
AU - Hutter, Frank
N1 - Funding information: and Joaquin Vanschoren. We also thank the anonymous reviewers for their valuable feedback. The authors acknowledge funding by the DFG (German Research Foundation) under Emmy Noether grant HU 1900/2-1. K. Eggensperger additionally acknowledges funding by the State Graduate Funding Program of Baden-Württemberg.
PY - 2019/4/16
Y1 - 2019/4/16
N2 - Good parameter settings are crucial to achieve high performance in many areas of artificial intelligence (AI), such as propositional satisfiability solving, AI planning, scheduling, and machine learning (in particular deep learning). Automated algorithm configuration methods have recently received much attention in the AI community since they replace tedious, irreproducible and error-prone manual parameter tuning and can lead to new state-of-the-art performance. However, practical applications of algorithm configuration are prone to several (often subtle) pitfalls in the experimental design that can render the procedure ineffective. We identify several common issues and propose best practices for avoiding them. As one possibility for automatically handling as many of these as possible, we also propose a tool called GenericWrapper4AC.
AB - Good parameter settings are crucial to achieve high performance in many areas of artificial intelligence (AI), such as propositional satisfiability solving, AI planning, scheduling, and machine learning (in particular deep learning). Automated algorithm configuration methods have recently received much attention in the AI community since they replace tedious, irreproducible and error-prone manual parameter tuning and can lead to new state-of-the-art performance. However, practical applications of algorithm configuration are prone to several (often subtle) pitfalls in the experimental design that can render the procedure ineffective. We identify several common issues and propose best practices for avoiding them. As one possibility for automatically handling as many of these as possible, we also propose a tool called GenericWrapper4AC.
UR - http://www.scopus.com/inward/record.url?scp=85065255539&partnerID=8YFLogxK
U2 - 10.1613/jair.1.11420
DO - 10.1613/jair.1.11420
M3 - Article
AN - SCOPUS:85065255539
VL - 64
SP - 861
EP - 893
JO - Journal of Artificial Intelligence Research
JF - Journal of Artificial Intelligence Research
SN - 1076-9757
ER -