Details
| Original language | English |
|---|---|
| Title of host publication | 15th ACM International Conference on Web Search and Data Mining (WSDM 2022) |
| Editors | K. Selcuk Candan, Huan Liu, Leman Akoglu, Xin Luna Dong, Jiliang Tang |
| Pages | 66-74 |
| Number of pages | 9 |
| ISBN (electronic) | 9781450391320 |
| Publication status | Published - 15 Feb 2022 |
Abstract
In this paper, we analyze comparative questions and answers. At least 3%∼of the questions submitted to search engines are comparative; ranging from simple facts like "Did Messi or Ronaldo score more goals in 2021?'' to life-changing and probably highly subjective questions like "Is it better to move abroad or stay?''. Ideally, answers to subjective comparative questions would reflect diverse opinions so that the asker can come to a well-informed decision. To better understand the information needs behind comparative questions, we develop approaches to extract the mentioned comparison objects and aspects. As a first step to answer comparative questions, we develop an approach that detects the stances of potential result nuggets (i.e., text passages containing the comparison objects). Our approaches are trained and evaluated on a set of 31,000∼English questions from existing datasets that we label as comparative or not. In the 3,500∼comparative questions, we label the comparison objects, aspects, and predicates. For 950∼questions, we collect answers from online forums and label the stance towards the comparison objects. In the experiments, our approaches recall∼71% of the comparative questions with a perfect precision of∼1.0, recall∼92% of subjective comparative questions with a precision of∼0.98, and identify the comparison objects and aspects with an F1 of∼0.93 and∼0.80, respectively. The stance detector fine-tuned on pairs of objects and answers achieves an accuracy of∼0.63.
Keywords
- Answer stance detection, Comparative questions, Comparison objects and aspects, Question intent understanding
ASJC Scopus subject areas
- Computer Science(all)
- Computer Networks and Communications
- Computer Science(all)
- Computer Science Applications
- Computer Science(all)
- Software
Cite this
- Standard
- Harvard
- Apa
- Vancouver
- BibTeX
- RIS
15th ACM International Conference on Web Search and Data Mining (WSDM 2022). ed. / K. Selcuk Candan; Huan Liu; Leman Akoglu; Xin Luna Dong; Jiliang Tang. 2022. p. 66-74.
Research output: Chapter in book/report/conference proceeding › Conference contribution › Research › peer review
}
TY - GEN
T1 - Towards Understanding and Answering Comparative Questions
AU - Bondarenko, Alexander
AU - Ajjour, Yamen
AU - Dittmar, Valentin
AU - Homann, Niklas
AU - Braslavski, Pavel
AU - Hagen, Matthias
N1 - Funding Information: This work has been partially supported by the DFG through the projects “ACQuA” and “ACQuA 2.0” (Answering Comparative Questions with Arguments; grants HA 5851/2-1 and HA 5851/2-2) as part of the priority program “RATIO: Robust Argumentation Machines” (SPP 1999). Pavel Braslavski acknowledges funding from the Ministry of Science and Higher Education of the Russian Federation (project 075-02-2021-1387). We are also thankful to Ekaterina Shir-shakova and Jonas Hirsch for their help with the data annotation and code development.
PY - 2022/2/15
Y1 - 2022/2/15
N2 - In this paper, we analyze comparative questions and answers. At least 3%∼of the questions submitted to search engines are comparative; ranging from simple facts like "Did Messi or Ronaldo score more goals in 2021?'' to life-changing and probably highly subjective questions like "Is it better to move abroad or stay?''. Ideally, answers to subjective comparative questions would reflect diverse opinions so that the asker can come to a well-informed decision. To better understand the information needs behind comparative questions, we develop approaches to extract the mentioned comparison objects and aspects. As a first step to answer comparative questions, we develop an approach that detects the stances of potential result nuggets (i.e., text passages containing the comparison objects). Our approaches are trained and evaluated on a set of 31,000∼English questions from existing datasets that we label as comparative or not. In the 3,500∼comparative questions, we label the comparison objects, aspects, and predicates. For 950∼questions, we collect answers from online forums and label the stance towards the comparison objects. In the experiments, our approaches recall∼71% of the comparative questions with a perfect precision of∼1.0, recall∼92% of subjective comparative questions with a precision of∼0.98, and identify the comparison objects and aspects with an F1 of∼0.93 and∼0.80, respectively. The stance detector fine-tuned on pairs of objects and answers achieves an accuracy of∼0.63.
AB - In this paper, we analyze comparative questions and answers. At least 3%∼of the questions submitted to search engines are comparative; ranging from simple facts like "Did Messi or Ronaldo score more goals in 2021?'' to life-changing and probably highly subjective questions like "Is it better to move abroad or stay?''. Ideally, answers to subjective comparative questions would reflect diverse opinions so that the asker can come to a well-informed decision. To better understand the information needs behind comparative questions, we develop approaches to extract the mentioned comparison objects and aspects. As a first step to answer comparative questions, we develop an approach that detects the stances of potential result nuggets (i.e., text passages containing the comparison objects). Our approaches are trained and evaluated on a set of 31,000∼English questions from existing datasets that we label as comparative or not. In the 3,500∼comparative questions, we label the comparison objects, aspects, and predicates. For 950∼questions, we collect answers from online forums and label the stance towards the comparison objects. In the experiments, our approaches recall∼71% of the comparative questions with a perfect precision of∼1.0, recall∼92% of subjective comparative questions with a precision of∼0.98, and identify the comparison objects and aspects with an F1 of∼0.93 and∼0.80, respectively. The stance detector fine-tuned on pairs of objects and answers achieves an accuracy of∼0.63.
KW - Answer stance detection
KW - Comparative questions
KW - Comparison objects and aspects
KW - Question intent understanding
UR - http://www.scopus.com/inward/record.url?scp=85125794689&partnerID=8YFLogxK
U2 - 10.1145/3488560.3498534
DO - 10.1145/3488560.3498534
M3 - Conference contribution
SP - 66
EP - 74
BT - 15th ACM International Conference on Web Search and Data Mining (WSDM 2022)
A2 - Candan, K. Selcuk
A2 - Liu, Huan
A2 - Akoglu, Leman
A2 - Dong, Xin Luna
A2 - Tang, Jiliang
ER -