Details
| Original language | English |
|---|---|
| Title of host publication | Advances in Information Retrieval - 39th European Conference on IR Research, ECIR 2017, Proceedings |
| Editors | Claudia Hauff, Joemon M. Jose, Dyaa Albakour, Ismail Sengor Altingovde, John Tait, Dawei Song, Stuart Watt |
| Publisher | Springer Verlag |
| Pages | 484-492 |
| Number of pages | 9 |
| ISBN (print) | 9783319566078 |
| Publication status | Published - 2017 |
| Event | 39th European Conference on Information Retrieval, ECIR 2017 - Aberdeen, United Kingdom (UK) Duration: 8 Apr 2017 → 13 Apr 2017 |
Publication series
| Name | Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) |
|---|---|
| Volume | 10193 LNCS |
| ISSN (Print) | 0302-9743 |
| ISSN (electronic) | 1611-3349 |
Abstract
Inspecting text to affirm the occurrence of an event is a nontrivial task. Since events are tied to temporal attributes, this task is more complex than merely identifying evidence of entities acting together and thus defining the event in a document. Manual inspection is a typical solution, although it is an onerous task and becomes infeasible with an increasing scale of documents. Therefore, the task of automatically determining whether an event occurs in a document or corpus, named as event validation, has been recently investigated. In this paper, we present a dataset for benchmarking event validation methods. Events and documents are coupled in pairs, whose validity has been judged by human evaluators based on whether the document in the pair contains evidence of the given event. In contrast to the notion of relevance considered in available datasets for event detection, validity judgments in this work strictly consider whether a document reports an event within its timespan as well as the number of event participants reported in the document. These requirements make the generation of manual validity judgments an onerous procedure. The ground truth, made of multiple judgments for each pair, has been acquired through crowdsourcing.
Keywords
- Crowdsourcing, Evaluation, Event detection, Event validation, Human computation
ASJC Scopus subject areas
- Mathematics(all)
- Theoretical Computer Science
- Computer Science(all)
Cite this
- Standard
- Harvard
- Apa
- Vancouver
- BibTeX
- RIS
Advances in Information Retrieval - 39th European Conference on IR Research, ECIR 2017, Proceedings. ed. / Claudia Hauff; Joemon M. Jose; Dyaa Albakour; Ismail Sengor Altingovde; John Tait; Dawei Song; Stuart Watt. Springer Verlag, 2017. p. 484-492 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10193 LNCS).
Research output: Chapter in book/report/conference proceeding › Conference contribution › Research › peer review
}
TY - GEN
T1 - Justevents
T2 - 39th European Conference on Information Retrieval, ECIR 2017
AU - Ceroni, Andrea
AU - Gadiraju, Ujwal
AU - Fisichella, Marco
N1 - Publisher Copyright: © Springer International Publishing AG 2017.
PY - 2017
Y1 - 2017
N2 - Inspecting text to affirm the occurrence of an event is a nontrivial task. Since events are tied to temporal attributes, this task is more complex than merely identifying evidence of entities acting together and thus defining the event in a document. Manual inspection is a typical solution, although it is an onerous task and becomes infeasible with an increasing scale of documents. Therefore, the task of automatically determining whether an event occurs in a document or corpus, named as event validation, has been recently investigated. In this paper, we present a dataset for benchmarking event validation methods. Events and documents are coupled in pairs, whose validity has been judged by human evaluators based on whether the document in the pair contains evidence of the given event. In contrast to the notion of relevance considered in available datasets for event detection, validity judgments in this work strictly consider whether a document reports an event within its timespan as well as the number of event participants reported in the document. These requirements make the generation of manual validity judgments an onerous procedure. The ground truth, made of multiple judgments for each pair, has been acquired through crowdsourcing.
AB - Inspecting text to affirm the occurrence of an event is a nontrivial task. Since events are tied to temporal attributes, this task is more complex than merely identifying evidence of entities acting together and thus defining the event in a document. Manual inspection is a typical solution, although it is an onerous task and becomes infeasible with an increasing scale of documents. Therefore, the task of automatically determining whether an event occurs in a document or corpus, named as event validation, has been recently investigated. In this paper, we present a dataset for benchmarking event validation methods. Events and documents are coupled in pairs, whose validity has been judged by human evaluators based on whether the document in the pair contains evidence of the given event. In contrast to the notion of relevance considered in available datasets for event detection, validity judgments in this work strictly consider whether a document reports an event within its timespan as well as the number of event participants reported in the document. These requirements make the generation of manual validity judgments an onerous procedure. The ground truth, made of multiple judgments for each pair, has been acquired through crowdsourcing.
KW - Crowdsourcing
KW - Evaluation
KW - Event detection
KW - Event validation
KW - Human computation
UR - http://www.scopus.com/inward/record.url?scp=85018671536&partnerID=8YFLogxK
U2 - 10.1007/978-3-319-56608-5_38
DO - 10.1007/978-3-319-56608-5_38
M3 - Conference contribution
AN - SCOPUS:85018671536
SN - 9783319566078
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 484
EP - 492
BT - Advances in Information Retrieval - 39th European Conference on IR Research, ECIR 2017, Proceedings
A2 - Hauff, Claudia
A2 - Jose, Joemon M.
A2 - Albakour, Dyaa
A2 - Altingovde, Ismail Sengor
A2 - Tait, John
A2 - Song, Dawei
A2 - Watt, Stuart
PB - Springer Verlag
Y2 - 8 April 2017 through 13 April 2017
ER -