Details
| Original language | English |
|---|---|
| Title of host publication | 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2024 |
| Publisher | Institute of Electrical and Electronics Engineers Inc. |
| Pages | 4965-4972 |
| Number of pages | 8 |
| ISBN (electronic) | 979-8-3503-7770-5 |
| ISBN (print) | 979-8-3503-7771-2 |
| Publication status | Published - 14 Oct 2024 |
| Event | 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2024 - Abu Dhabi, United Arab Emirates Duration: 14 Oct 2024 → 18 Oct 2024 |
Publication series
| Name | IEEE International Conference on Intelligent Robots and Systems |
|---|---|
| ISSN (Print) | 2153-0858 |
| ISSN (electronic) | 2153-0866 |
Abstract
Domain Randomization (DR) is commonly used for sim2real transfer of reinforcement learning (RL) policies in robotics. Most DR approaches require a simulator with a fixed set of tunable parameters from the start of the training, from which the parameters are randomized simultaneously to train a robust model for use in the real world. However, the combined randomization of many parameters increases the task difficulty and might result in sub-optimal policies. To address this problem and to provide a more flexible training process, we propose Continual Domain Randomization (CDR) for RL that combines domain randomization with continual learning to enable sequential training in simulation on a subset of randomization parameters at a time. Starting from a model trained in a non-randomized simulation where the task is easier to solve, the model is trained on a sequence of randomizations, and continual learning is employed to remember the effects of previous randomizations. Our robotic reaching and grasping tasks experiments show that the model trained in this fashion learns effectively in simulation and performs robustly on the real robot while matching or outperforming baselines that employ combined randomization or sequential randomization without continual learning. Our code and videos are available at https://continual-dr.github.io/.
Keywords
- continual reinforcement learning, Domain randomization, robotic manipulation, sim2real transfer
ASJC Scopus subject areas
- Engineering(all)
- Control and Systems Engineering
- Computer Science(all)
- Software
- Computer Science(all)
- Computer Vision and Pattern Recognition
- Computer Science(all)
- Computer Science Applications
Cite this
- Standard
- Harvard
- Apa
- Vancouver
- BibTeX
- RIS
2024 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2024. Institute of Electrical and Electronics Engineers Inc., 2024. p. 4965-4972 (IEEE International Conference on Intelligent Robots and Systems).
Research output: Chapter in book/report/conference proceeding › Conference contribution › Research › peer review
}
TY - GEN
T1 - Continual Domain Randomization
AU - Josifovski, Josip
AU - Auddy, Sayantan
AU - Malmir, Mohammadhossein
AU - Piater, Justus
AU - Knoll, Alois
AU - Navarro-Guerrero, Nicolás
N1 - Publisher Copyright: © 2024 IEEE.
PY - 2024/10/14
Y1 - 2024/10/14
N2 - Domain Randomization (DR) is commonly used for sim2real transfer of reinforcement learning (RL) policies in robotics. Most DR approaches require a simulator with a fixed set of tunable parameters from the start of the training, from which the parameters are randomized simultaneously to train a robust model for use in the real world. However, the combined randomization of many parameters increases the task difficulty and might result in sub-optimal policies. To address this problem and to provide a more flexible training process, we propose Continual Domain Randomization (CDR) for RL that combines domain randomization with continual learning to enable sequential training in simulation on a subset of randomization parameters at a time. Starting from a model trained in a non-randomized simulation where the task is easier to solve, the model is trained on a sequence of randomizations, and continual learning is employed to remember the effects of previous randomizations. Our robotic reaching and grasping tasks experiments show that the model trained in this fashion learns effectively in simulation and performs robustly on the real robot while matching or outperforming baselines that employ combined randomization or sequential randomization without continual learning. Our code and videos are available at https://continual-dr.github.io/.
AB - Domain Randomization (DR) is commonly used for sim2real transfer of reinforcement learning (RL) policies in robotics. Most DR approaches require a simulator with a fixed set of tunable parameters from the start of the training, from which the parameters are randomized simultaneously to train a robust model for use in the real world. However, the combined randomization of many parameters increases the task difficulty and might result in sub-optimal policies. To address this problem and to provide a more flexible training process, we propose Continual Domain Randomization (CDR) for RL that combines domain randomization with continual learning to enable sequential training in simulation on a subset of randomization parameters at a time. Starting from a model trained in a non-randomized simulation where the task is easier to solve, the model is trained on a sequence of randomizations, and continual learning is employed to remember the effects of previous randomizations. Our robotic reaching and grasping tasks experiments show that the model trained in this fashion learns effectively in simulation and performs robustly on the real robot while matching or outperforming baselines that employ combined randomization or sequential randomization without continual learning. Our code and videos are available at https://continual-dr.github.io/.
KW - continual reinforcement learning
KW - Domain randomization
KW - robotic manipulation
KW - sim2real transfer
UR - http://www.scopus.com/inward/record.url?scp=85216474476&partnerID=8YFLogxK
U2 - 10.48550/arXiv.2403.12193
DO - 10.48550/arXiv.2403.12193
M3 - Conference contribution
AN - SCOPUS:85216474476
SN - 979-8-3503-7771-2
T3 - IEEE International Conference on Intelligent Robots and Systems
SP - 4965
EP - 4972
BT - 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2024
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2024
Y2 - 14 October 2024 through 18 October 2024
ER -