On the Subjectivity of Emotions in Software Projects: How Reliable are Pre-Labeled Data Sets for Sentiment Analysis? (Summary)

Publikation: Beitrag in Buch/Bericht/Sammelwerk/KonferenzbandAufsatz in KonferenzbandForschungPeer-Review

Organisationseinheiten

Forschungs-netzwerk anzeigen

Details

OriginalspracheEnglisch
Titel des SammelwerksSoftware Engineering 2023
UntertitelFachtagung des GI-Fachbereichs Softwaretechnik
Herausgeber/-innenGregor Engels, Regina Hebig, Matthias Tichy
Herausgeber (Verlag)Gesellschaft fur Informatik (GI)
Seiten67-68
Seitenumfang2
ISBN (elektronisch)9783885797265
PublikationsstatusVeröffentlicht - 2023
Veranstaltung2023 Fachtagung des GI-Fachbereichs Softwaretechnik, Software Engineering 2023 - 2023 Conference of the GI Software Engineering Division, Software Engineering 2023 - Paderborn, Deutschland
Dauer: 20 Feb. 202324 Feb. 2023

Publikationsreihe

NameLecture Notes in Informatics (LNI), Proceedings - Series of the Gesellschaft fur Informatik (GI)
BandP-332
ISSN (Print)1617-5468

Abstract

Social aspects (e.g., the sentiment of developers) are important for software development. In order to automatically analyze sentiments, sentiment analysis tools use machine learning methods that require data sets labeled according to emotion or polarity. As these labeled data sets strongly influence the tools' accuracy, we investigate whether the labels match developers' perceptions. For this purpose, we conducted an international survey with 94 participants who labeled 100 statements. We compare the median as well as every single participant's perception with the labels. The results show that the median perception of all participants coincides with the predefined labels for 62.5% of the statements, and that the difference between the single participant's ratings and the labels is even worse. This summary refers to the paper with the title “On the subjectivity of emotions in software projects: How reliable are pre-labeled data sets for sentiment analysis?” [He22b]. It was published in the Journal of Systems and Software (JSS) in 2022 peer-reviewed.

ASJC Scopus Sachgebiete

Zitieren

On the Subjectivity of Emotions in Software Projects: How Reliable are Pre-Labeled Data Sets for Sentiment Analysis? (Summary). / Herrmann, Marc; Obaidi, Martin; Chazette, Larissa et al.
Software Engineering 2023 : Fachtagung des GI-Fachbereichs Softwaretechnik. Hrsg. / Gregor Engels; Regina Hebig; Matthias Tichy. Gesellschaft fur Informatik (GI), 2023. S. 67-68 (Lecture Notes in Informatics (LNI), Proceedings - Series of the Gesellschaft fur Informatik (GI); Band P-332).

Publikation: Beitrag in Buch/Bericht/Sammelwerk/KonferenzbandAufsatz in KonferenzbandForschungPeer-Review

Herrmann, M, Obaidi, M, Chazette, L & Klünder, J 2023, On the Subjectivity of Emotions in Software Projects: How Reliable are Pre-Labeled Data Sets for Sentiment Analysis? (Summary). in G Engels, R Hebig & M Tichy (Hrsg.), Software Engineering 2023 : Fachtagung des GI-Fachbereichs Softwaretechnik. Lecture Notes in Informatics (LNI), Proceedings - Series of the Gesellschaft fur Informatik (GI), Bd. P-332, Gesellschaft fur Informatik (GI), S. 67-68, 2023 Fachtagung des GI-Fachbereichs Softwaretechnik, Software Engineering 2023 - 2023 Conference of the GI Software Engineering Division, Software Engineering 2023, Paderborn, Deutschland, 20 Feb. 2023.
Herrmann, M., Obaidi, M., Chazette, L., & Klünder, J. (2023). On the Subjectivity of Emotions in Software Projects: How Reliable are Pre-Labeled Data Sets for Sentiment Analysis? (Summary). In G. Engels, R. Hebig, & M. Tichy (Hrsg.), Software Engineering 2023 : Fachtagung des GI-Fachbereichs Softwaretechnik (S. 67-68). (Lecture Notes in Informatics (LNI), Proceedings - Series of the Gesellschaft fur Informatik (GI); Band P-332). Gesellschaft fur Informatik (GI).
Herrmann M, Obaidi M, Chazette L, Klünder J. On the Subjectivity of Emotions in Software Projects: How Reliable are Pre-Labeled Data Sets for Sentiment Analysis? (Summary). in Engels G, Hebig R, Tichy M, Hrsg., Software Engineering 2023 : Fachtagung des GI-Fachbereichs Softwaretechnik. Gesellschaft fur Informatik (GI). 2023. S. 67-68. (Lecture Notes in Informatics (LNI), Proceedings - Series of the Gesellschaft fur Informatik (GI)).
Herrmann, Marc ; Obaidi, Martin ; Chazette, Larissa et al. / On the Subjectivity of Emotions in Software Projects : How Reliable are Pre-Labeled Data Sets for Sentiment Analysis? (Summary). Software Engineering 2023 : Fachtagung des GI-Fachbereichs Softwaretechnik. Hrsg. / Gregor Engels ; Regina Hebig ; Matthias Tichy. Gesellschaft fur Informatik (GI), 2023. S. 67-68 (Lecture Notes in Informatics (LNI), Proceedings - Series of the Gesellschaft fur Informatik (GI)).
Download
@inproceedings{264d1e1e6ff74937920e7ebb76927449,
title = "On the Subjectivity of Emotions in Software Projects: How Reliable are Pre-Labeled Data Sets for Sentiment Analysis? (Summary)",
abstract = "Social aspects (e.g., the sentiment of developers) are important for software development. In order to automatically analyze sentiments, sentiment analysis tools use machine learning methods that require data sets labeled according to emotion or polarity. As these labeled data sets strongly influence the tools' accuracy, we investigate whether the labels match developers' perceptions. For this purpose, we conducted an international survey with 94 participants who labeled 100 statements. We compare the median as well as every single participant's perception with the labels. The results show that the median perception of all participants coincides with the predefined labels for 62.5% of the statements, and that the difference between the single participant's ratings and the labels is even worse. This summary refers to the paper with the title “On the subjectivity of emotions in software projects: How reliable are pre-labeled data sets for sentiment analysis?” [He22b]. It was published in the Journal of Systems and Software (JSS) in 2022 peer-reviewed.",
keywords = "communication, development team, polarity, Sentiment analysis, software projects",
author = "Marc Herrmann and Martin Obaidi and Larissa Chazette and Jil Kl{\"u}nder",
year = "2023",
language = "English",
series = "Lecture Notes in Informatics (LNI), Proceedings - Series of the Gesellschaft fur Informatik (GI)",
publisher = "Gesellschaft fur Informatik (GI)",
pages = "67--68",
editor = "Gregor Engels and Regina Hebig and Matthias Tichy",
booktitle = "Software Engineering 2023",
address = "Germany",
note = "2023 Fachtagung des GI-Fachbereichs Softwaretechnik, Software Engineering 2023 - 2023 Conference of the GI Software Engineering Division, Software Engineering 2023 ; Conference date: 20-02-2023 Through 24-02-2023",

}

Download

TY - GEN

T1 - On the Subjectivity of Emotions in Software Projects

T2 - 2023 Fachtagung des GI-Fachbereichs Softwaretechnik, Software Engineering 2023 - 2023 Conference of the GI Software Engineering Division, Software Engineering 2023

AU - Herrmann, Marc

AU - Obaidi, Martin

AU - Chazette, Larissa

AU - Klünder, Jil

PY - 2023

Y1 - 2023

N2 - Social aspects (e.g., the sentiment of developers) are important for software development. In order to automatically analyze sentiments, sentiment analysis tools use machine learning methods that require data sets labeled according to emotion or polarity. As these labeled data sets strongly influence the tools' accuracy, we investigate whether the labels match developers' perceptions. For this purpose, we conducted an international survey with 94 participants who labeled 100 statements. We compare the median as well as every single participant's perception with the labels. The results show that the median perception of all participants coincides with the predefined labels for 62.5% of the statements, and that the difference between the single participant's ratings and the labels is even worse. This summary refers to the paper with the title “On the subjectivity of emotions in software projects: How reliable are pre-labeled data sets for sentiment analysis?” [He22b]. It was published in the Journal of Systems and Software (JSS) in 2022 peer-reviewed.

AB - Social aspects (e.g., the sentiment of developers) are important for software development. In order to automatically analyze sentiments, sentiment analysis tools use machine learning methods that require data sets labeled according to emotion or polarity. As these labeled data sets strongly influence the tools' accuracy, we investigate whether the labels match developers' perceptions. For this purpose, we conducted an international survey with 94 participants who labeled 100 statements. We compare the median as well as every single participant's perception with the labels. The results show that the median perception of all participants coincides with the predefined labels for 62.5% of the statements, and that the difference between the single participant's ratings and the labels is even worse. This summary refers to the paper with the title “On the subjectivity of emotions in software projects: How reliable are pre-labeled data sets for sentiment analysis?” [He22b]. It was published in the Journal of Systems and Software (JSS) in 2022 peer-reviewed.

KW - communication

KW - development team

KW - polarity

KW - Sentiment analysis

KW - software projects

UR - http://www.scopus.com/inward/record.url?scp=85150068781&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85150068781

T3 - Lecture Notes in Informatics (LNI), Proceedings - Series of the Gesellschaft fur Informatik (GI)

SP - 67

EP - 68

BT - Software Engineering 2023

A2 - Engels, Gregor

A2 - Hebig, Regina

A2 - Tichy, Matthias

PB - Gesellschaft fur Informatik (GI)

Y2 - 20 February 2023 through 24 February 2023

ER -

Von denselben Autoren