Chatterbox: Conversational interfaces for microtask crowdsourcing

Publikation: Beitrag in Buch/Bericht/Sammelwerk/KonferenzbandAufsatz in KonferenzbandForschungPeer-Review

Autoren

  • Panagiotis Mavridis
  • Owen Huang
  • Sihang Qiu
  • Ujwal Gadiraju
  • Alessandro Bozzon

Organisationseinheiten

Externe Organisationen

  • Delft University of Technology
Forschungs-netzwerk anzeigen

Details

OriginalspracheEnglisch
Titel des SammelwerksACM UMAP 2019
UntertitelProceedings of the 27th ACM Conference on User Modeling, Adaptation and Personalization
Herausgeber/-innenGeorge Aneglos Papadopoulos, George Samaras, Stephan Weibelzahl, Dietmar Jannach, Olga C. Santos
ErscheinungsortNew York
Seiten243-251
Seitenumfang9
ISBN (elektronisch)9781450360210
PublikationsstatusVeröffentlicht - 7 Juni 2019
Veranstaltung27th ACM Conference on User Modeling, Adaptation and Personalization, UMAP 2019 - Larnaca, Zypern
Dauer: 9 Juni 201912 Juni 2019

Abstract

Conversational interfaces can facilitate human-computer interactions. Whether or not conversational interfaces can improve worker experience and work quality in crowdsourcing marketplaces has remained unanswered. We investigate the suitability of text-based conversational interfaces for microtask crowdsourcing. We designed a rigorous experimental campaign aimed at gauging the interest and acceptance by crowdworkers for this type of work interface. We compared Web and conversational interfaces for five common microtask types and measured the execution time, quality of work, and the perceived satisfaction of 316 workers recruited from the FigureEight platform. We show that conversational interfaces can be used effectively for crowdsourcing microtasks, resulting in a high satisfaction from workers, and without having a negative impact on task execution time or work quality.

ASJC Scopus Sachgebiete

Zitieren

Chatterbox: Conversational interfaces for microtask crowdsourcing. / Mavridis, Panagiotis; Huang, Owen; Qiu, Sihang et al.
ACM UMAP 2019: Proceedings of the 27th ACM Conference on User Modeling, Adaptation and Personalization. Hrsg. / George Aneglos Papadopoulos; George Samaras; Stephan Weibelzahl; Dietmar Jannach; Olga C. Santos. New York, 2019. S. 243-251.

Publikation: Beitrag in Buch/Bericht/Sammelwerk/KonferenzbandAufsatz in KonferenzbandForschungPeer-Review

Mavridis, P, Huang, O, Qiu, S, Gadiraju, U & Bozzon, A 2019, Chatterbox: Conversational interfaces for microtask crowdsourcing. in GA Papadopoulos, G Samaras, S Weibelzahl, D Jannach & OC Santos (Hrsg.), ACM UMAP 2019: Proceedings of the 27th ACM Conference on User Modeling, Adaptation and Personalization. New York, S. 243-251, 27th ACM Conference on User Modeling, Adaptation and Personalization, UMAP 2019, Larnaca, Zypern, 9 Juni 2019. https://doi.org/10.1145/3320435.3320439
Mavridis, P., Huang, O., Qiu, S., Gadiraju, U., & Bozzon, A. (2019). Chatterbox: Conversational interfaces for microtask crowdsourcing. In G. A. Papadopoulos, G. Samaras, S. Weibelzahl, D. Jannach, & O. C. Santos (Hrsg.), ACM UMAP 2019: Proceedings of the 27th ACM Conference on User Modeling, Adaptation and Personalization (S. 243-251). https://doi.org/10.1145/3320435.3320439
Mavridis P, Huang O, Qiu S, Gadiraju U, Bozzon A. Chatterbox: Conversational interfaces for microtask crowdsourcing. in Papadopoulos GA, Samaras G, Weibelzahl S, Jannach D, Santos OC, Hrsg., ACM UMAP 2019: Proceedings of the 27th ACM Conference on User Modeling, Adaptation and Personalization. New York. 2019. S. 243-251 doi: 10.1145/3320435.3320439
Mavridis, Panagiotis ; Huang, Owen ; Qiu, Sihang et al. / Chatterbox : Conversational interfaces for microtask crowdsourcing. ACM UMAP 2019: Proceedings of the 27th ACM Conference on User Modeling, Adaptation and Personalization. Hrsg. / George Aneglos Papadopoulos ; George Samaras ; Stephan Weibelzahl ; Dietmar Jannach ; Olga C. Santos. New York, 2019. S. 243-251
Download
@inproceedings{46d10477be1349b7ac9fff6c68edd86c,
title = "Chatterbox: Conversational interfaces for microtask crowdsourcing",
abstract = "Conversational interfaces can facilitate human-computer interactions. Whether or not conversational interfaces can improve worker experience and work quality in crowdsourcing marketplaces has remained unanswered. We investigate the suitability of text-based conversational interfaces for microtask crowdsourcing. We designed a rigorous experimental campaign aimed at gauging the interest and acceptance by crowdworkers for this type of work interface. We compared Web and conversational interfaces for five common microtask types and measured the execution time, quality of work, and the perceived satisfaction of 316 workers recruited from the FigureEight platform. We show that conversational interfaces can be used effectively for crowdsourcing microtasks, resulting in a high satisfaction from workers, and without having a negative impact on task execution time or work quality.",
keywords = "Chatbots, Conversational agents, Microtask crowdsourcing",
author = "Panagiotis Mavridis and Owen Huang and Sihang Qiu and Ujwal Gadiraju and Alessandro Bozzon",
note = "Acknowledgements: This research has been supported in part by the Amsterdam Institute for Advanced Metropolitan Solutions with the AMS Social Bot grant, by the Dutch national e-infrastructure with the support of SURF Cooperative (grant e-infra170237), and by the Erasmus+ project DISKOW (grant no. 60171990). We also thank all the anonymous crowdworkers from F8 who participated in our experiments.; 27th ACM Conference on User Modeling, Adaptation and Personalization, UMAP 2019 ; Conference date: 09-06-2019 Through 12-06-2019",
year = "2019",
month = jun,
day = "7",
doi = "10.1145/3320435.3320439",
language = "English",
pages = "243--251",
editor = "Papadopoulos, {George Aneglos} and Samaras, {George } and Stephan Weibelzahl and Dietmar Jannach and Santos, {Olga C.}",
booktitle = "ACM UMAP 2019",

}

Download

TY - GEN

T1 - Chatterbox

T2 - 27th ACM Conference on User Modeling, Adaptation and Personalization, UMAP 2019

AU - Mavridis, Panagiotis

AU - Huang, Owen

AU - Qiu, Sihang

AU - Gadiraju, Ujwal

AU - Bozzon, Alessandro

N1 - Acknowledgements: This research has been supported in part by the Amsterdam Institute for Advanced Metropolitan Solutions with the AMS Social Bot grant, by the Dutch national e-infrastructure with the support of SURF Cooperative (grant e-infra170237), and by the Erasmus+ project DISKOW (grant no. 60171990). We also thank all the anonymous crowdworkers from F8 who participated in our experiments.

PY - 2019/6/7

Y1 - 2019/6/7

N2 - Conversational interfaces can facilitate human-computer interactions. Whether or not conversational interfaces can improve worker experience and work quality in crowdsourcing marketplaces has remained unanswered. We investigate the suitability of text-based conversational interfaces for microtask crowdsourcing. We designed a rigorous experimental campaign aimed at gauging the interest and acceptance by crowdworkers for this type of work interface. We compared Web and conversational interfaces for five common microtask types and measured the execution time, quality of work, and the perceived satisfaction of 316 workers recruited from the FigureEight platform. We show that conversational interfaces can be used effectively for crowdsourcing microtasks, resulting in a high satisfaction from workers, and without having a negative impact on task execution time or work quality.

AB - Conversational interfaces can facilitate human-computer interactions. Whether or not conversational interfaces can improve worker experience and work quality in crowdsourcing marketplaces has remained unanswered. We investigate the suitability of text-based conversational interfaces for microtask crowdsourcing. We designed a rigorous experimental campaign aimed at gauging the interest and acceptance by crowdworkers for this type of work interface. We compared Web and conversational interfaces for five common microtask types and measured the execution time, quality of work, and the perceived satisfaction of 316 workers recruited from the FigureEight platform. We show that conversational interfaces can be used effectively for crowdsourcing microtasks, resulting in a high satisfaction from workers, and without having a negative impact on task execution time or work quality.

KW - Chatbots

KW - Conversational agents

KW - Microtask crowdsourcing

UR - http://www.scopus.com/inward/record.url?scp=85068045274&partnerID=8YFLogxK

U2 - 10.1145/3320435.3320439

DO - 10.1145/3320435.3320439

M3 - Conference contribution

AN - SCOPUS:85068045274

SP - 243

EP - 251

BT - ACM UMAP 2019

A2 - Papadopoulos, George Aneglos

A2 - Samaras, George

A2 - Weibelzahl, Stephan

A2 - Jannach, Dietmar

A2 - Santos, Olga C.

CY - New York

Y2 - 9 June 2019 through 12 June 2019

ER -