Details
| Original language | English |
|---|---|
| Title of host publication | Deep Learning for Knowledge Graphs 2023 |
| Subtitle of host publication | Proceedings of the Workshop on Deep Learning for Knowledge Graphs (DL4KG 2023) co-located with the 21th International Semantic Web Conference (ISWC 2023) |
| Publication status | Published - 2023 |
| Event | 2023 Workshop on Deep Learning for Knowledge Graphs - Athens, Greece Duration: 6 Nov 2023 → 10 Nov 2023 |
Publication series
| Name | CEUR Workshop Proceedings |
|---|---|
| Publisher | CEUR Workshop Proceedings |
| Volume | 3559 |
| ISSN (Print) | 1613-0073 |
Abstract
In reinforcement learning (RL) an agent usually learns the specifics and rules of the environment via interaction. This limits the agent in taking the best action only from the current observation and past experience. Therefore, providing relevant external knowledge for RL agents, as well as incorporating learned knowledge in the RL process can be a critical part of agent’s successful training in real-world tasks. We propose a method, an architecture and experimental results for injecting expert knowledge in the form of RDF knowledge graphs (KGs) into the RL processes, showing how knowledge consumption increases sample efficiency. Furthermore, we investigate the scalability of our approach concerning the complexity of the underlying task showing injection of KGs is beneficial to the solution of more complex RL tasks. For experimental evaluation we used the Minigrid environment, which is a standard benchmark for RL. For this environment, we designed an ontology and generated a KG, that promotes reusability and interoperability across heterogeneous data of the environment. We show that adding knowledge to the agent’s learning process improves sample efficiency and the benefits increase with the complexity of the environment.
Keywords
- Knowledge Graphs, Knowledge Injection, Reinforcement Learning, State Representation
ASJC Scopus subject areas
Cite this
- Standard
- Harvard
- Apa
- Vancouver
- BibTeX
- RIS
Deep Learning for Knowledge Graphs 2023: Proceedings of the Workshop on Deep Learning for Knowledge Graphs (DL4KG 2023) co-located with the 21th International Semantic Web Conference (ISWC 2023). 2023. (CEUR Workshop Proceedings; Vol. 3559).
Research output: Chapter in book/report/conference proceeding › Conference contribution › Research › peer review
}
TY - GEN
T1 - Knowledge Graph Injection for Reinforcement Learning
AU - Wardenga, Robert
AU - Kovriguina, Liubov
AU - Pliukhin, Dmitrii
AU - Radyush, Daniil
AU - Smoliakov, Ivan
AU - Xue, Yuan
AU - Müller, Henrik
AU - Pismerov, Aleksei
AU - Mouromtsev, Dmitry
AU - Kudenko, Daniel
N1 - Funding Information: *Corresponding author. †R. W. appreciates funding from Federal Ministry of Education and Research (BMBF) under iDOKS + FKZ 16SV8765 $ wardenga@infai.org (R. Wardenga); lk@metaphacts.com (L. Kovriguina); zeionara@gmail.com (D. Pliukhin); daniil.radyush@gmail.com (D. Radyush); smol.ivan97@gmail.com (I. Smoliakov); xue@l3s.de (Y. Xue); hmueller@l3s.de (H. Müller); alekceu444@gmail.com (A. Pismerov); d.muromtsev@gmail.com (D. Mouromtsev); kudenko@l3s.de (D. Kudenko) Funding Information: R. W. appreciates funding from Federal Ministry of Education and Research (BMBF) under iDOKS + FKZ 16SV8765.
PY - 2023
Y1 - 2023
N2 - In reinforcement learning (RL) an agent usually learns the specifics and rules of the environment via interaction. This limits the agent in taking the best action only from the current observation and past experience. Therefore, providing relevant external knowledge for RL agents, as well as incorporating learned knowledge in the RL process can be a critical part of agent’s successful training in real-world tasks. We propose a method, an architecture and experimental results for injecting expert knowledge in the form of RDF knowledge graphs (KGs) into the RL processes, showing how knowledge consumption increases sample efficiency. Furthermore, we investigate the scalability of our approach concerning the complexity of the underlying task showing injection of KGs is beneficial to the solution of more complex RL tasks. For experimental evaluation we used the Minigrid environment, which is a standard benchmark for RL. For this environment, we designed an ontology and generated a KG, that promotes reusability and interoperability across heterogeneous data of the environment. We show that adding knowledge to the agent’s learning process improves sample efficiency and the benefits increase with the complexity of the environment.
AB - In reinforcement learning (RL) an agent usually learns the specifics and rules of the environment via interaction. This limits the agent in taking the best action only from the current observation and past experience. Therefore, providing relevant external knowledge for RL agents, as well as incorporating learned knowledge in the RL process can be a critical part of agent’s successful training in real-world tasks. We propose a method, an architecture and experimental results for injecting expert knowledge in the form of RDF knowledge graphs (KGs) into the RL processes, showing how knowledge consumption increases sample efficiency. Furthermore, we investigate the scalability of our approach concerning the complexity of the underlying task showing injection of KGs is beneficial to the solution of more complex RL tasks. For experimental evaluation we used the Minigrid environment, which is a standard benchmark for RL. For this environment, we designed an ontology and generated a KG, that promotes reusability and interoperability across heterogeneous data of the environment. We show that adding knowledge to the agent’s learning process improves sample efficiency and the benefits increase with the complexity of the environment.
KW - Knowledge Graphs
KW - Knowledge Injection
KW - Reinforcement Learning
KW - State Representation
UR - http://www.scopus.com/inward/record.url?scp=85178606909&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:85178606909
T3 - CEUR Workshop Proceedings
BT - Deep Learning for Knowledge Graphs 2023
T2 - 2023 Workshop on Deep Learning for Knowledge Graphs
Y2 - 6 November 2023 through 10 November 2023
ER -