Details
| Originalsprache | Englisch |
|---|---|
| Fachzeitschrift | Transactions on Machine Learning Research |
| Jahrgang | 2023 |
| Ausgabenummer | 6 |
| Publikationsstatus | Elektronisch veröffentlicht (E-Pub) - 5 Juni 2023 |
Abstract
about generalization tasks. We confirm the insight that optimal behavior in cRL requires context information, as in other related areas of partial observability. To empirically validate this in the cRL framework, we provide various context-extended versions of common RL environments. They are part of the first benchmark library, CARL, designed for generalization based on cRL extensions of popular benchmarks, which we propose as a testbed to further study general agents. We show that in the contextual setting, even simple RL environments become challenging - and that naive solutions are not enough to generalize across complex context spaces.
Zitieren
- Standard
- Harvard
- Apa
- Vancouver
- BibTex
- RIS
in: Transactions on Machine Learning Research, Jahrgang 2023, Nr. 6, 05.06.2023.
Publikation: Beitrag in Fachzeitschrift › Artikel › Forschung › Peer-Review
}
TY - JOUR
T1 - Contextualize Me – The Case for Context in Reinforcement Learning
AU - Benjamins, Carolin
AU - Eimer, Theresa
AU - Schubert, Frederik Günter
AU - Mohan, Aditya
AU - Döhler, Sebastian
AU - Biedenkapp, André
AU - Rosenhahn, Bodo
AU - Hutter, Frank
AU - Lindauer, Marius
PY - 2023/6/5
Y1 - 2023/6/5
N2 - While Reinforcement Learning ( RL) has made great strides towards solving increasingly complicated problems, many algorithms are still brittle to even slight environmental changes. Contextual Reinforcement Learning (cRL) provides a framework to model such changes in a principled manner, thereby enabling flexible, precise and interpretable task specification and generation. Our goal is to show how the framework of cRL contributes to improving zero-shot generalization in RL through meaningful benchmarks and structured reasoning about generalization tasks. We confirm the insight that optimal behavior in cRL requires context information, as in other related areas of partial observability. To empirically validate this in the cRL framework, we provide various context-extended versions of common RL environments. They are part of the first benchmark library, CARL, designed for generalization based on cRL extensions of popular benchmarks, which we propose as a testbed to further study general agents. We show that in the contextual setting, even simple RL environments become challenging - and that naive solutions are not enough to generalize across complex context spaces.
AB - While Reinforcement Learning ( RL) has made great strides towards solving increasingly complicated problems, many algorithms are still brittle to even slight environmental changes. Contextual Reinforcement Learning (cRL) provides a framework to model such changes in a principled manner, thereby enabling flexible, precise and interpretable task specification and generation. Our goal is to show how the framework of cRL contributes to improving zero-shot generalization in RL through meaningful benchmarks and structured reasoning about generalization tasks. We confirm the insight that optimal behavior in cRL requires context information, as in other related areas of partial observability. To empirically validate this in the cRL framework, we provide various context-extended versions of common RL environments. They are part of the first benchmark library, CARL, designed for generalization based on cRL extensions of popular benchmarks, which we propose as a testbed to further study general agents. We show that in the contextual setting, even simple RL environments become challenging - and that naive solutions are not enough to generalize across complex context spaces.
KW - contextual RL
KW - cMDP
KW - generalization
U2 - 10.48550/arXiv.2202.04500
DO - 10.48550/arXiv.2202.04500
M3 - Article
VL - 2023
JO - Transactions on Machine Learning Research
JF - Transactions on Machine Learning Research
SN - 2835-8856
IS - 6
ER -