TY - GEN

T1 - KnowlyBERT

T2 - Hybrid Query Answering over Language Models and Knowledge Graphs

AU - Kalo, Jan-Christoph

AU - Fichtel, Leandra

AU - Ehler, Philipp

AU - Balke, Wolf-Tilo

PY - 2020

Y1 - 2020

N2 - Providing a plethora of entity-centric information, Knowledge Graphs have become a vital building block for a variety of intelligent applications. Indeed, modern knowledge graphs like Wikidata already capture several billions of RDF triples, yet they still lack a good coverage for most relations. On the other hand, recent developments in NLP research show that neural language models can easily be queried for relational knowledge without requiring massive amounts of training data. In this work, we leverage this idea by creating a hybrid query answering system on top of knowledge graphs in combination with the masked language model BERT to complete query results. We thus incorporate valuable structural and semantic information from knowledge graphs with textual knowledge from language models to achieve high precision query results. Standard techniques for dealing with incomplete knowledge graphs are either (1) relation extraction which requires massive amounts of training data or (2) knowledge graph embeddings which have problems to succeed beyond simple baseline datasets. Our hybrid system KnowlyBERT requires only small amounts of training data, while outperforming state-of-the-art techniques by boosting their precision by over 30% in our large Wikidata experiment.

AB - Providing a plethora of entity-centric information, Knowledge Graphs have become a vital building block for a variety of intelligent applications. Indeed, modern knowledge graphs like Wikidata already capture several billions of RDF triples, yet they still lack a good coverage for most relations. On the other hand, recent developments in NLP research show that neural language models can easily be queried for relational knowledge without requiring massive amounts of training data. In this work, we leverage this idea by creating a hybrid query answering system on top of knowledge graphs in combination with the masked language model BERT to complete query results. We thus incorporate valuable structural and semantic information from knowledge graphs with textual knowledge from language models to achieve high precision query results. Standard techniques for dealing with incomplete knowledge graphs are either (1) relation extraction which requires massive amounts of training data or (2) knowledge graph embeddings which have problems to succeed beyond simple baseline datasets. Our hybrid system KnowlyBERT requires only small amounts of training data, while outperforming state-of-the-art techniques by boosting their precision by over 30% in our large Wikidata experiment.

KW - Knowledge graphs

KW - Language models

KW - Query answering

UR - http://www.scopus.com/inward/record.url?scp=85096523582&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-62419-4_17

DO - 10.1007/978-3-030-62419-4_17

M3 - Conference contribution

SN - 9783030624187

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 294

EP - 310

BT - The Semantic Web – ISWC 2020 - 19th International Semantic Web Conference, 2020, Proceedings

A2 - Pan, Jeff Z.

A2 - Tamma, Valentina

A2 - d’Amato, Claudia

A2 - Janowicz, Krzysztof

A2 - Fu, Bo

A2 - Polleres, Axel

A2 - Seneviratne, Oshani

A2 - Kagal, Lalana

ER -