Details
| Originalsprache | Englisch |
|---|---|
| Titel des Sammelwerks | Findings of the Association for Computational Linguistics: EMNLP 2022 |
| Seiten | 4636-4648 |
| Seitenumfang | 13 |
| Publikationsstatus | Veröffentlicht - Dez. 2022 |
| Veranstaltung | 2022 Findings of the Association for Computational Linguistics: EMNLP 2022 - Abu Dhabi, Vereinigte Arabische Emirate Dauer: 7 Dez. 2022 → 11 Dez. 2022 |
Abstract
Adding interpretability to word embeddings represents an area of active research in text representation. Recent work has explored the potential of embedding words via so-called polar dimensions (e.g. good vs. bad, correct vs. wrong). Examples of such recent approaches include SemAxis, POLAR, FrameAxis, and BiImp. Although these approaches provide interpretable dimensions for words, they have not been designed to deal with polysemy, i.e. they can not easily distinguish between different senses of words. To address this limitation, we present SensePOLAR, an extension of the original POLAR framework that enables word-sense aware interpretability for pre-trained contextual word embeddings. The resulting interpretable word embeddings achieve a level of performance that is comparable to original contextual word embeddings across a variety of natural language processing tasks including the GLUE and SQuAD benchmarks. Our work removes a fundamental limitation of existing approaches by offering users sense aware interpretations for contextual word embeddings.
ASJC Scopus Sachgebiete
- Informatik (insg.)
- Theoretische Informatik und Mathematik
- Informatik (insg.)
- Angewandte Informatik
- Informatik (insg.)
- Information systems
Zitieren
- Standard
- Harvard
- Apa
- Vancouver
- BibTex
- RIS
Findings of the Association for Computational Linguistics: EMNLP 2022. 2022. S. 4636-4648.
Publikation: Beitrag in Buch/Bericht/Sammelwerk/Konferenzband › Aufsatz in Konferenzband › Forschung › Peer-Review
}
TY - GEN
T1 - SensePOLAR
T2 - 2022 Findings of the Association for Computational Linguistics: EMNLP 2022
AU - Engler, Jan
AU - Sikdar, Sandipan
AU - Lutz, Marlene
AU - Strohmaier, Markus
N1 - Funding Information: Sandipan Sikdar was supported in part by RWTH Aachen Startup Grant No. StUpPD384-20.
PY - 2022/12
Y1 - 2022/12
N2 - Adding interpretability to word embeddings represents an area of active research in text representation. Recent work has explored the potential of embedding words via so-called polar dimensions (e.g. good vs. bad, correct vs. wrong). Examples of such recent approaches include SemAxis, POLAR, FrameAxis, and BiImp. Although these approaches provide interpretable dimensions for words, they have not been designed to deal with polysemy, i.e. they can not easily distinguish between different senses of words. To address this limitation, we present SensePOLAR, an extension of the original POLAR framework that enables word-sense aware interpretability for pre-trained contextual word embeddings. The resulting interpretable word embeddings achieve a level of performance that is comparable to original contextual word embeddings across a variety of natural language processing tasks including the GLUE and SQuAD benchmarks. Our work removes a fundamental limitation of existing approaches by offering users sense aware interpretations for contextual word embeddings.
AB - Adding interpretability to word embeddings represents an area of active research in text representation. Recent work has explored the potential of embedding words via so-called polar dimensions (e.g. good vs. bad, correct vs. wrong). Examples of such recent approaches include SemAxis, POLAR, FrameAxis, and BiImp. Although these approaches provide interpretable dimensions for words, they have not been designed to deal with polysemy, i.e. they can not easily distinguish between different senses of words. To address this limitation, we present SensePOLAR, an extension of the original POLAR framework that enables word-sense aware interpretability for pre-trained contextual word embeddings. The resulting interpretable word embeddings achieve a level of performance that is comparable to original contextual word embeddings across a variety of natural language processing tasks including the GLUE and SQuAD benchmarks. Our work removes a fundamental limitation of existing approaches by offering users sense aware interpretations for contextual word embeddings.
UR - http://www.scopus.com/inward/record.url?scp=85149902132&partnerID=8YFLogxK
U2 - 10.48550/arXiv.2301.04704
DO - 10.48550/arXiv.2301.04704
M3 - Conference contribution
AN - SCOPUS:85149902132
SP - 4636
EP - 4648
BT - Findings of the Association for Computational Linguistics: EMNLP 2022
Y2 - 7 December 2022 through 11 December 2022
ER -