TY - GEN

T1 - Continual Domain Randomization

AU - Josifovski, Josip

AU - Auddy, Sayantan

AU - Malmir, Mohammadhossein

AU - Piater, Justus

AU - Knoll, Alois

AU - Navarro-Guerrero, Nicolás

N1 - Publisher Copyright: © 2024 IEEE.

PY - 2024/10/14

Y1 - 2024/10/14

N2 - Domain Randomization (DR) is commonly used for sim2real transfer of reinforcement learning (RL) policies in robotics. Most DR approaches require a simulator with a fixed set of tunable parameters from the start of the training, from which the parameters are randomized simultaneously to train a robust model for use in the real world. However, the combined randomization of many parameters increases the task difficulty and might result in sub-optimal policies. To address this problem and to provide a more flexible training process, we propose Continual Domain Randomization (CDR) for RL that combines domain randomization with continual learning to enable sequential training in simulation on a subset of randomization parameters at a time. Starting from a model trained in a non-randomized simulation where the task is easier to solve, the model is trained on a sequence of randomizations, and continual learning is employed to remember the effects of previous randomizations. Our robotic reaching and grasping tasks experiments show that the model trained in this fashion learns effectively in simulation and performs robustly on the real robot while matching or outperforming baselines that employ combined randomization or sequential randomization without continual learning. Our code and videos are available at https://continual-dr.github.io/.

AB - Domain Randomization (DR) is commonly used for sim2real transfer of reinforcement learning (RL) policies in robotics. Most DR approaches require a simulator with a fixed set of tunable parameters from the start of the training, from which the parameters are randomized simultaneously to train a robust model for use in the real world. However, the combined randomization of many parameters increases the task difficulty and might result in sub-optimal policies. To address this problem and to provide a more flexible training process, we propose Continual Domain Randomization (CDR) for RL that combines domain randomization with continual learning to enable sequential training in simulation on a subset of randomization parameters at a time. Starting from a model trained in a non-randomized simulation where the task is easier to solve, the model is trained on a sequence of randomizations, and continual learning is employed to remember the effects of previous randomizations. Our robotic reaching and grasping tasks experiments show that the model trained in this fashion learns effectively in simulation and performs robustly on the real robot while matching or outperforming baselines that employ combined randomization or sequential randomization without continual learning. Our code and videos are available at https://continual-dr.github.io/.

KW - continual reinforcement learning

KW - Domain randomization

KW - robotic manipulation

KW - sim2real transfer

UR - http://www.scopus.com/inward/record.url?scp=85216474476&partnerID=8YFLogxK

U2 - 10.48550/arXiv.2403.12193

DO - 10.48550/arXiv.2403.12193

M3 - Conference contribution

AN - SCOPUS:85216474476

SN - 979-8-3503-7771-2

T3 - IEEE International Conference on Intelligent Robots and Systems

SP - 4965

EP - 4972

BT - 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2024

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2024

Y2 - 14 October 2024 through 18 October 2024

ER -