Details
| Original language | English |
|---|---|
| Pages (from-to) | 1061-1070 |
| Number of pages | 10 |
| Journal | Procedia Computer Science |
| Volume | 192 |
| Early online date | 1 Oct 2021 |
| Publication status | Published - 2021 |
| Event | 25th KES International Conference on Knowledge-Based and Intelligent Information and Engineering Systems, KES 2021 - Szczecin, Poland Duration: 8 Sept 2021 → 10 Sept 2021 |
Abstract
Multi-agent reinforcement learning allows a team of agents to learn how to work together to solve complex decision-making problems in a shared environment. However, this learning process utilises stochastic mechanisms, meaning that its use in safety-critical domains can be problematic. To overcome this issue, we propose an Assured Multi-Agent Reinforcement Learning (AMARL) approach that uses a model checking technique called quantitative verification to provide formal guarantees of agent compliance with safety, performance, and other non-functional requirements during and after the reinforcement learning process. We demonstrate the applicability of our AMARL approach in three different patrolling navigation domains in which multi-agent systems must learn to visit key areas by using different types of reinforcement learning algorithms (temporal difference learning, game theory, and direct policy search). Furthermore, we compare the effectiveness of these algorithms when used in combination with and without our approach. Our extensive experiments with both homogeneous and heterogeneous multi-agent systems of different sizes show that the use of AMARL leads to safety requirements being consistently satisfied and to better overall results than standard reinforcement learning.
Keywords
- Assurance, Assured multi-agent reinforcement learning, Multi-agent reinforcement learning, Multi-agent systems, Quantitative verification, Reinforcement learning, Safe multi-agent reinforcement learning, Safety-critical scenarios
ASJC Scopus subject areas
- Computer Science(all)
- General Computer Science
Cite this
- Standard
- Harvard
- Apa
- Vancouver
- BibTeX
- RIS
In: Procedia Computer Science, Vol. 192, 2021, p. 1061-1070.
Research output: Contribution to journal › Conference article › Research › peer review
}
TY - JOUR
T1 - Utilising Assured Multi-Agent Reinforcement Learning within Safety-Critical Scenarios
AU - Riley, Joshua
AU - Calinescu, Radu
AU - Paterson, Colin
AU - Kudenko, Daniel
AU - Banks, Alec
PY - 2021
Y1 - 2021
N2 - Multi-agent reinforcement learning allows a team of agents to learn how to work together to solve complex decision-making problems in a shared environment. However, this learning process utilises stochastic mechanisms, meaning that its use in safety-critical domains can be problematic. To overcome this issue, we propose an Assured Multi-Agent Reinforcement Learning (AMARL) approach that uses a model checking technique called quantitative verification to provide formal guarantees of agent compliance with safety, performance, and other non-functional requirements during and after the reinforcement learning process. We demonstrate the applicability of our AMARL approach in three different patrolling navigation domains in which multi-agent systems must learn to visit key areas by using different types of reinforcement learning algorithms (temporal difference learning, game theory, and direct policy search). Furthermore, we compare the effectiveness of these algorithms when used in combination with and without our approach. Our extensive experiments with both homogeneous and heterogeneous multi-agent systems of different sizes show that the use of AMARL leads to safety requirements being consistently satisfied and to better overall results than standard reinforcement learning.
AB - Multi-agent reinforcement learning allows a team of agents to learn how to work together to solve complex decision-making problems in a shared environment. However, this learning process utilises stochastic mechanisms, meaning that its use in safety-critical domains can be problematic. To overcome this issue, we propose an Assured Multi-Agent Reinforcement Learning (AMARL) approach that uses a model checking technique called quantitative verification to provide formal guarantees of agent compliance with safety, performance, and other non-functional requirements during and after the reinforcement learning process. We demonstrate the applicability of our AMARL approach in three different patrolling navigation domains in which multi-agent systems must learn to visit key areas by using different types of reinforcement learning algorithms (temporal difference learning, game theory, and direct policy search). Furthermore, we compare the effectiveness of these algorithms when used in combination with and without our approach. Our extensive experiments with both homogeneous and heterogeneous multi-agent systems of different sizes show that the use of AMARL leads to safety requirements being consistently satisfied and to better overall results than standard reinforcement learning.
KW - Assurance
KW - Assured multi-agent reinforcement learning
KW - Multi-agent reinforcement learning
KW - Multi-agent systems
KW - Quantitative verification
KW - Reinforcement learning
KW - Safe multi-agent reinforcement learning
KW - Safety-critical scenarios
UR - http://www.scopus.com/inward/record.url?scp=85116918514&partnerID=8YFLogxK
U2 - 10.1016/j.procs.2021.08.109
DO - 10.1016/j.procs.2021.08.109
M3 - Conference article
AN - SCOPUS:85116918514
VL - 192
SP - 1061
EP - 1070
JO - Procedia Computer Science
JF - Procedia Computer Science
T2 - 25th KES International Conference on Knowledge-Based and Intelligent Information and Engineering Systems, KES 2021
Y2 - 8 September 2021 through 10 September 2021
ER -