Details
| Originalsprache | Englisch |
|---|---|
| Titel des Sammelwerks | SemEval 2021 - 15th International Workshop on Semantic Evaluation, Proceedings of the Workshop |
| Herausgeber/-innen | Alexis Palmer, Nathan Schneider, Natalie Schluter, Guy Emerson, Aurelie Herbelot, Xiaodan Zhu |
| Herausgeber (Verlag) | Association for Computational Linguistics (ACL) |
| Seiten | 364-376 |
| Seitenumfang | 13 |
| ISBN (elektronisch) | 9781954085701 |
| Publikationsstatus | Veröffentlicht - 2021 |
| Extern publiziert | Ja |
| Veranstaltung | 15th International Workshop on Semantic Evaluation, SemEval 2021 - Virtual, Bangkok, Thailand Dauer: 5 Aug. 2021 → 6 Aug. 2021 |
Publikationsreihe
| Name | SemEval 2021 - 15th International Workshop on Semantic Evaluation, Proceedings of the Workshop |
|---|
Abstract
There is currently a gap between the natural language expression of scholarly publications and their structured semantic content modeling to enable intelligent content search. With the volume of research growing exponentially every year, a search feature operating over semantically structured content is compelling. The SemEval-2021 Shared Task NLPCONTRIBUTIONGRAPH (a.k.a. ‘the NCG task’) tasks participants to develop automated systems that structure contributions from NLP scholarly articles in the English language. Being the first-of-its-kind in the SemEval series, the task released structured data from NLP scholarly articles at three levels of information granularity, i.e. at sentence-level, phrase-level, and phrases organized as triples toward Knowledge Graph (KG) building. The sentence-level annotations comprised the few sentences about the article’s contribution. The phrase-level annotations were scientific term and predicate phrases from the contribution sentences. Finally, the triples constituted the research overview KG. For the Shared Task, participating systems were then expected to automatically classify contribution sentences, extract scientific terms and relations from the sentences, and organize them as KG triples. Overall, the task drew a strong participation demographic of seven teams and 27 participants. The best end-to-end task system classified contribution sentences at 57.27% F1, phrases at 46.41% F1, and triples at 22.28% F1. While the absolute performance to generate triples remains low, in the conclusion of this article, the difficulty of producing such data and as a consequence of modeling it is highlighted.
ASJC Scopus Sachgebiete
- Informatik (insg.)
- Theoretische Informatik und Mathematik
- Informatik (insg.)
- Angewandte Informatik
- Mathematik (insg.)
- Theoretische Informatik
Zitieren
- Standard
- Harvard
- Apa
- Vancouver
- BibTex
- RIS
SemEval 2021 - 15th International Workshop on Semantic Evaluation, Proceedings of the Workshop. Hrsg. / Alexis Palmer; Nathan Schneider; Natalie Schluter; Guy Emerson; Aurelie Herbelot; Xiaodan Zhu. Association for Computational Linguistics (ACL), 2021. S. 364-376 (SemEval 2021 - 15th International Workshop on Semantic Evaluation, Proceedings of the Workshop).
Publikation: Beitrag in Buch/Bericht/Sammelwerk/Konferenzband › Aufsatz in Konferenzband › Forschung › Peer-Review
}
TY - GEN
T1 - SemEval-2021 Task 11
T2 - 15th International Workshop on Semantic Evaluation, SemEval 2021
AU - D’Souza, Jennifer
AU - Auer, Sören
AU - Pedersen, Ted
N1 - Funding Information: We thank the anonymous reviewers for their comments and suggestions. This work was co-funded by the European Research Council for the project ScienceGRAPH (Grant agreement ID: 819536) and by the TIB Leibniz Information Centre for Science and Technology.
PY - 2021
Y1 - 2021
N2 - There is currently a gap between the natural language expression of scholarly publications and their structured semantic content modeling to enable intelligent content search. With the volume of research growing exponentially every year, a search feature operating over semantically structured content is compelling. The SemEval-2021 Shared Task NLPCONTRIBUTIONGRAPH (a.k.a. ‘the NCG task’) tasks participants to develop automated systems that structure contributions from NLP scholarly articles in the English language. Being the first-of-its-kind in the SemEval series, the task released structured data from NLP scholarly articles at three levels of information granularity, i.e. at sentence-level, phrase-level, and phrases organized as triples toward Knowledge Graph (KG) building. The sentence-level annotations comprised the few sentences about the article’s contribution. The phrase-level annotations were scientific term and predicate phrases from the contribution sentences. Finally, the triples constituted the research overview KG. For the Shared Task, participating systems were then expected to automatically classify contribution sentences, extract scientific terms and relations from the sentences, and organize them as KG triples. Overall, the task drew a strong participation demographic of seven teams and 27 participants. The best end-to-end task system classified contribution sentences at 57.27% F1, phrases at 46.41% F1, and triples at 22.28% F1. While the absolute performance to generate triples remains low, in the conclusion of this article, the difficulty of producing such data and as a consequence of modeling it is highlighted.
AB - There is currently a gap between the natural language expression of scholarly publications and their structured semantic content modeling to enable intelligent content search. With the volume of research growing exponentially every year, a search feature operating over semantically structured content is compelling. The SemEval-2021 Shared Task NLPCONTRIBUTIONGRAPH (a.k.a. ‘the NCG task’) tasks participants to develop automated systems that structure contributions from NLP scholarly articles in the English language. Being the first-of-its-kind in the SemEval series, the task released structured data from NLP scholarly articles at three levels of information granularity, i.e. at sentence-level, phrase-level, and phrases organized as triples toward Knowledge Graph (KG) building. The sentence-level annotations comprised the few sentences about the article’s contribution. The phrase-level annotations were scientific term and predicate phrases from the contribution sentences. Finally, the triples constituted the research overview KG. For the Shared Task, participating systems were then expected to automatically classify contribution sentences, extract scientific terms and relations from the sentences, and organize them as KG triples. Overall, the task drew a strong participation demographic of seven teams and 27 participants. The best end-to-end task system classified contribution sentences at 57.27% F1, phrases at 46.41% F1, and triples at 22.28% F1. While the absolute performance to generate triples remains low, in the conclusion of this article, the difficulty of producing such data and as a consequence of modeling it is highlighted.
UR - http://www.scopus.com/inward/record.url?scp=85132941197&partnerID=8YFLogxK
U2 - 10.18653/v1/2021.semeval-1.44
DO - 10.18653/v1/2021.semeval-1.44
M3 - Conference contribution
AN - SCOPUS:85132941197
T3 - SemEval 2021 - 15th International Workshop on Semantic Evaluation, Proceedings of the Workshop
SP - 364
EP - 376
BT - SemEval 2021 - 15th International Workshop on Semantic Evaluation, Proceedings of the Workshop
A2 - Palmer, Alexis
A2 - Schneider, Nathan
A2 - Schluter, Natalie
A2 - Emerson, Guy
A2 - Herbelot, Aurelie
A2 - Zhu, Xiaodan
PB - Association for Computational Linguistics (ACL)
Y2 - 5 August 2021 through 6 August 2021
ER -