TY - GEN

T1 - Towards Understanding and Answering Comparative Questions

AU - Bondarenko, Alexander

AU - Ajjour, Yamen

AU - Dittmar, Valentin

AU - Homann, Niklas

AU - Braslavski, Pavel

AU - Hagen, Matthias

N1 - Funding Information: This work has been partially supported by the DFG through the projects “ACQuA” and “ACQuA 2.0” (Answering Comparative Questions with Arguments; grants HA 5851/2-1 and HA 5851/2-2) as part of the priority program “RATIO: Robust Argumentation Machines” (SPP 1999). Pavel Braslavski acknowledges funding from the Ministry of Science and Higher Education of the Russian Federation (project 075-02-2021-1387). We are also thankful to Ekaterina Shir-shakova and Jonas Hirsch for their help with the data annotation and code development.

PY - 2022/2/15

Y1 - 2022/2/15

N2 - In this paper, we analyze comparative questions and answers. At least 3%∼of the questions submitted to search engines are comparative; ranging from simple facts like "Did Messi or Ronaldo score more goals in 2021?'' to life-changing and probably highly subjective questions like "Is it better to move abroad or stay?''. Ideally, answers to subjective comparative questions would reflect diverse opinions so that the asker can come to a well-informed decision. To better understand the information needs behind comparative questions, we develop approaches to extract the mentioned comparison objects and aspects. As a first step to answer comparative questions, we develop an approach that detects the stances of potential result nuggets (i.e., text passages containing the comparison objects). Our approaches are trained and evaluated on a set of 31,000∼English questions from existing datasets that we label as comparative or not. In the 3,500∼comparative questions, we label the comparison objects, aspects, and predicates. For 950∼questions, we collect answers from online forums and label the stance towards the comparison objects. In the experiments, our approaches recall∼71% of the comparative questions with a perfect precision of∼1.0, recall∼92% of subjective comparative questions with a precision of∼0.98, and identify the comparison objects and aspects with an F1 of∼0.93 and∼0.80, respectively. The stance detector fine-tuned on pairs of objects and answers achieves an accuracy of∼0.63.

AB - In this paper, we analyze comparative questions and answers. At least 3%∼of the questions submitted to search engines are comparative; ranging from simple facts like "Did Messi or Ronaldo score more goals in 2021?'' to life-changing and probably highly subjective questions like "Is it better to move abroad or stay?''. Ideally, answers to subjective comparative questions would reflect diverse opinions so that the asker can come to a well-informed decision. To better understand the information needs behind comparative questions, we develop approaches to extract the mentioned comparison objects and aspects. As a first step to answer comparative questions, we develop an approach that detects the stances of potential result nuggets (i.e., text passages containing the comparison objects). Our approaches are trained and evaluated on a set of 31,000∼English questions from existing datasets that we label as comparative or not. In the 3,500∼comparative questions, we label the comparison objects, aspects, and predicates. For 950∼questions, we collect answers from online forums and label the stance towards the comparison objects. In the experiments, our approaches recall∼71% of the comparative questions with a perfect precision of∼1.0, recall∼92% of subjective comparative questions with a precision of∼0.98, and identify the comparison objects and aspects with an F1 of∼0.93 and∼0.80, respectively. The stance detector fine-tuned on pairs of objects and answers achieves an accuracy of∼0.63.

KW - Answer stance detection

KW - Comparative questions

KW - Comparison objects and aspects

KW - Question intent understanding

UR - http://www.scopus.com/inward/record.url?scp=85125794689&partnerID=8YFLogxK

U2 - 10.1145/3488560.3498534

DO - 10.1145/3488560.3498534

M3 - Conference contribution

SP - 66

EP - 74

BT - 15th ACM International Conference on Web Search and Data Mining (WSDM 2022)

A2 - Candan, K. Selcuk

A2 - Liu, Huan

A2 - Akoglu, Leman

A2 - Dong, Xin Luna

A2 - Tang, Jiliang

ER -