Fair ranking: a critical review, challenges, and future directions

Research output: Chapter in book/report/conference proceedingConference contributionResearchpeer review

Authors

  • Gourab K. Patro
  • Lorenzo Porcaro
  • Laura Mitchell
  • Qiuyue Zhang
  • Meike Zehlike
  • Nikhil Garg

Research Organisations

External Research Organisations

  • Universität Pompeu Fabra (UPF)
  • Competition and Markets Authority (CMA)
  • Accenture Plc
  • Max Planck Institute for Software Systems (MPI SWS)
  • Cornell Tech
  • Zalando Research
  • Indian Institute of Technology Kharagpur (IITKGP)
View graph of relations

Details

Original languageEnglish
Title of host publicationFAccT '22
Subtitle of host publication2022 ACM Conference on Fairness, Accountability, and Transparency
PublisherAssociation for Computing Machinery (ACM)
Pages1929-1942
Number of pages14
ISBN (electronic)9781450393522
Publication statusPublished - 20 Jun 2022
Event5th ACM Conference on Fairness, Accountability, and Transparency, FAccT 2022 - Virtual, Online, Korea, Republic of
Duration: 21 Jun 202224 Jun 2022

Publication series

NameACM International Conference Proceeding Series

Abstract

Ranking, recommendation, and retrieval systems are widely used in online platforms and other societal systems, including e-commerce, media-streaming, admissions, gig platforms, and hiring. In the recent past, a large "fair ranking"research literature has been developed around making these systems fair to the individuals, providers, or content that are being ranked. Most of this literature defines fairness for a single instance of retrieval, or as a simple additive notion for multiple instances of retrievals over time. This work provides a critical overview of this literature, detailing the often context-specific concerns that such approaches miss: the gap between high ranking placements and true provider utility, spillovers and compounding effects over time, induced strategic incentives, and the effect of statistical uncertainty. We then provide a path forward for a more holistic and impact-oriented fair ranking research agenda, including methodological lessons from other fields and the role of the broader stakeholder community in overcoming data bottlenecks and designing effective regulatory environments.

Keywords

    Algorithmic Impact Assessment, Exposure, Fairness, Ranking, Recommendation, Strategic Behaviour

ASJC Scopus subject areas

Cite this

Fair ranking: a critical review, challenges, and future directions. / Patro, Gourab K.; Porcaro, Lorenzo; Mitchell, Laura et al.
FAccT '22: 2022 ACM Conference on Fairness, Accountability, and Transparency. Association for Computing Machinery (ACM), 2022. p. 1929-1942 (ACM International Conference Proceeding Series).

Research output: Chapter in book/report/conference proceedingConference contributionResearchpeer review

Patro, GK, Porcaro, L, Mitchell, L, Zhang, Q, Zehlike, M & Garg, N 2022, Fair ranking: a critical review, challenges, and future directions. in FAccT '22: 2022 ACM Conference on Fairness, Accountability, and Transparency. ACM International Conference Proceeding Series, Association for Computing Machinery (ACM), pp. 1929-1942, 5th ACM Conference on Fairness, Accountability, and Transparency, FAccT 2022, Virtual, Online, Korea, Republic of, 21 Jun 2022. https://doi.org/10.48550/arXiv.2201.12662, https://doi.org/10.1145/3531146.3533238
Patro, G. K., Porcaro, L., Mitchell, L., Zhang, Q., Zehlike, M., & Garg, N. (2022). Fair ranking: a critical review, challenges, and future directions. In FAccT '22: 2022 ACM Conference on Fairness, Accountability, and Transparency (pp. 1929-1942). (ACM International Conference Proceeding Series). Association for Computing Machinery (ACM). https://doi.org/10.48550/arXiv.2201.12662, https://doi.org/10.1145/3531146.3533238
Patro GK, Porcaro L, Mitchell L, Zhang Q, Zehlike M, Garg N. Fair ranking: a critical review, challenges, and future directions. In FAccT '22: 2022 ACM Conference on Fairness, Accountability, and Transparency. Association for Computing Machinery (ACM). 2022. p. 1929-1942. (ACM International Conference Proceeding Series). doi: 10.48550/arXiv.2201.12662, 10.1145/3531146.3533238
Patro, Gourab K. ; Porcaro, Lorenzo ; Mitchell, Laura et al. / Fair ranking : a critical review, challenges, and future directions. FAccT '22: 2022 ACM Conference on Fairness, Accountability, and Transparency. Association for Computing Machinery (ACM), 2022. pp. 1929-1942 (ACM International Conference Proceeding Series).
Download
@inproceedings{72b6682135b04f2c9b292c62795b99e9,
title = "Fair ranking: a critical review, challenges, and future directions",
abstract = "Ranking, recommendation, and retrieval systems are widely used in online platforms and other societal systems, including e-commerce, media-streaming, admissions, gig platforms, and hiring. In the recent past, a large {"}fair ranking{"}research literature has been developed around making these systems fair to the individuals, providers, or content that are being ranked. Most of this literature defines fairness for a single instance of retrieval, or as a simple additive notion for multiple instances of retrievals over time. This work provides a critical overview of this literature, detailing the often context-specific concerns that such approaches miss: the gap between high ranking placements and true provider utility, spillovers and compounding effects over time, induced strategic incentives, and the effect of statistical uncertainty. We then provide a path forward for a more holistic and impact-oriented fair ranking research agenda, including methodological lessons from other fields and the role of the broader stakeholder community in overcoming data bottlenecks and designing effective regulatory environments.",
keywords = "Algorithmic Impact Assessment, Exposure, Fairness, Ranking, Recommendation, Strategic Behaviour",
author = "Patro, {Gourab K.} and Lorenzo Porcaro and Laura Mitchell and Qiuyue Zhang and Meike Zehlike and Nikhil Garg",
note = "Funding Information: The authors would like to thank Francesco Fabbri, Jessie Finocchiaro, Faidra Monachou, Ignacio Rios, and Ana-Andreea Stoica for helpful comments. This project has been a part of the MD4SG working group on Bias, Discrimination, and Fairness. This research has received funding under European Research Council (ERC) Marie Sklodowska-Curie grant (agreement no. 860630) for the project No- BIAS, Spanish Ministerio de Ciencia, Innovacion y Universidades (MCIU) and the Agencia Estatal de Investigacion (AEI)-PID2019- 111403GB-I00/AEI/10.13039/501100011033. G. K Patro acknowledges the support by TCS Research fellowship.; 5th ACM Conference on Fairness, Accountability, and Transparency, FAccT 2022 ; Conference date: 21-06-2022 Through 24-06-2022",
year = "2022",
month = jun,
day = "20",
doi = "10.48550/arXiv.2201.12662",
language = "English",
series = "ACM International Conference Proceeding Series",
publisher = "Association for Computing Machinery (ACM)",
pages = "1929--1942",
booktitle = "FAccT '22",
address = "United States",

}

Download

TY - GEN

T1 - Fair ranking

T2 - 5th ACM Conference on Fairness, Accountability, and Transparency, FAccT 2022

AU - Patro, Gourab K.

AU - Porcaro, Lorenzo

AU - Mitchell, Laura

AU - Zhang, Qiuyue

AU - Zehlike, Meike

AU - Garg, Nikhil

N1 - Funding Information: The authors would like to thank Francesco Fabbri, Jessie Finocchiaro, Faidra Monachou, Ignacio Rios, and Ana-Andreea Stoica for helpful comments. This project has been a part of the MD4SG working group on Bias, Discrimination, and Fairness. This research has received funding under European Research Council (ERC) Marie Sklodowska-Curie grant (agreement no. 860630) for the project No- BIAS, Spanish Ministerio de Ciencia, Innovacion y Universidades (MCIU) and the Agencia Estatal de Investigacion (AEI)-PID2019- 111403GB-I00/AEI/10.13039/501100011033. G. K Patro acknowledges the support by TCS Research fellowship.

PY - 2022/6/20

Y1 - 2022/6/20

N2 - Ranking, recommendation, and retrieval systems are widely used in online platforms and other societal systems, including e-commerce, media-streaming, admissions, gig platforms, and hiring. In the recent past, a large "fair ranking"research literature has been developed around making these systems fair to the individuals, providers, or content that are being ranked. Most of this literature defines fairness for a single instance of retrieval, or as a simple additive notion for multiple instances of retrievals over time. This work provides a critical overview of this literature, detailing the often context-specific concerns that such approaches miss: the gap between high ranking placements and true provider utility, spillovers and compounding effects over time, induced strategic incentives, and the effect of statistical uncertainty. We then provide a path forward for a more holistic and impact-oriented fair ranking research agenda, including methodological lessons from other fields and the role of the broader stakeholder community in overcoming data bottlenecks and designing effective regulatory environments.

AB - Ranking, recommendation, and retrieval systems are widely used in online platforms and other societal systems, including e-commerce, media-streaming, admissions, gig platforms, and hiring. In the recent past, a large "fair ranking"research literature has been developed around making these systems fair to the individuals, providers, or content that are being ranked. Most of this literature defines fairness for a single instance of retrieval, or as a simple additive notion for multiple instances of retrievals over time. This work provides a critical overview of this literature, detailing the often context-specific concerns that such approaches miss: the gap between high ranking placements and true provider utility, spillovers and compounding effects over time, induced strategic incentives, and the effect of statistical uncertainty. We then provide a path forward for a more holistic and impact-oriented fair ranking research agenda, including methodological lessons from other fields and the role of the broader stakeholder community in overcoming data bottlenecks and designing effective regulatory environments.

KW - Algorithmic Impact Assessment

KW - Exposure

KW - Fairness

KW - Ranking

KW - Recommendation

KW - Strategic Behaviour

UR - http://www.scopus.com/inward/record.url?scp=85133030574&partnerID=8YFLogxK

U2 - 10.48550/arXiv.2201.12662

DO - 10.48550/arXiv.2201.12662

M3 - Conference contribution

AN - SCOPUS:85133030574

T3 - ACM International Conference Proceeding Series

SP - 1929

EP - 1942

BT - FAccT '22

PB - Association for Computing Machinery (ACM)

Y2 - 21 June 2022 through 24 June 2022

ER -