Details
| Original language | English |
|---|---|
| Title of host publication | ICTIR 2022 |
| Subtitle of host publication | Proceedings of the 2022 ACM SIGIR International Conference on the Theory of Information Retrieval |
| Pages | 115-120 |
| Number of pages | 6 |
| ISBN (electronic) | 9781450394123 |
| Publication status | Published - 25 Aug 2022 |
| Event | 8th ACM SIGIR International Conference on the Theory of Information Retrieval, ICTIR 2022 - Virtual, Online, Spain Duration: 11 Jul 2022 → 12 Jul 2022 |
Abstract
Contextual ranking models based on BERT are now well established for a wide range of passage and document ranking tasks. However, the robustness of BERT-based ranking models under adversarial inputs is under-explored. In this paper, we argue that BERT-rankers are not immune to adversarial attacks targeting retrieved documents given a query. Firstly, we propose algorithms for adversarial perturbation of both highly relevant and non-relevant documents using gradient-based optimization methods. The aim of our algorithms is to add/replace a small number of tokens to a highly relevant or non-relevant document to cause a large rank demotion or promotion. Our experiments show that a small number of tokens can already result in a large change in the rank of a document. Moreover, we find that BERT-rankers heavily rely on the document start/head for relevance prediction, making the initial part of the document more susceptible to adversarial attacks. More interestingly, we find a small set of recurring adversarial words that when added to documents result in successful rank demotion/promotion of any relevant/non-relevant document respectively. Finally, our adversarial tokens also show particular topic preferences within and across datasets, exposing potential biases from BERT pre-training or downstream datasets.
Keywords
- adversarial attack, bert, biases, neural networks, ranking
ASJC Scopus subject areas
- Computer Science(all)
- Computer Science (miscellaneous)
- Computer Science(all)
- Information Systems
Cite this
- Standard
- Harvard
- Apa
- Vancouver
- BibTeX
- RIS
ICTIR 2022 : Proceedings of the 2022 ACM SIGIR International Conference on the Theory of Information Retrieval. 2022. p. 115-120.
Research output: Chapter in book/report/conference proceeding › Conference contribution › Research › peer review
}
TY - GEN
T1 - BERT Rankers are Brittle
T2 - 8th ACM SIGIR International Conference on the Theory of Information Retrieval, ICTIR 2022
AU - Wang, Yumeng
AU - Lyu, Lijun
AU - Anand, Avishek
N1 - Funding Information: This work is partially supported by DFG Project AN 996/1-1.
PY - 2022/8/25
Y1 - 2022/8/25
N2 - Contextual ranking models based on BERT are now well established for a wide range of passage and document ranking tasks. However, the robustness of BERT-based ranking models under adversarial inputs is under-explored. In this paper, we argue that BERT-rankers are not immune to adversarial attacks targeting retrieved documents given a query. Firstly, we propose algorithms for adversarial perturbation of both highly relevant and non-relevant documents using gradient-based optimization methods. The aim of our algorithms is to add/replace a small number of tokens to a highly relevant or non-relevant document to cause a large rank demotion or promotion. Our experiments show that a small number of tokens can already result in a large change in the rank of a document. Moreover, we find that BERT-rankers heavily rely on the document start/head for relevance prediction, making the initial part of the document more susceptible to adversarial attacks. More interestingly, we find a small set of recurring adversarial words that when added to documents result in successful rank demotion/promotion of any relevant/non-relevant document respectively. Finally, our adversarial tokens also show particular topic preferences within and across datasets, exposing potential biases from BERT pre-training or downstream datasets.
AB - Contextual ranking models based on BERT are now well established for a wide range of passage and document ranking tasks. However, the robustness of BERT-based ranking models under adversarial inputs is under-explored. In this paper, we argue that BERT-rankers are not immune to adversarial attacks targeting retrieved documents given a query. Firstly, we propose algorithms for adversarial perturbation of both highly relevant and non-relevant documents using gradient-based optimization methods. The aim of our algorithms is to add/replace a small number of tokens to a highly relevant or non-relevant document to cause a large rank demotion or promotion. Our experiments show that a small number of tokens can already result in a large change in the rank of a document. Moreover, we find that BERT-rankers heavily rely on the document start/head for relevance prediction, making the initial part of the document more susceptible to adversarial attacks. More interestingly, we find a small set of recurring adversarial words that when added to documents result in successful rank demotion/promotion of any relevant/non-relevant document respectively. Finally, our adversarial tokens also show particular topic preferences within and across datasets, exposing potential biases from BERT pre-training or downstream datasets.
KW - adversarial attack
KW - bert
KW - biases
KW - neural networks
KW - ranking
UR - http://www.scopus.com/inward/record.url?scp=85138329521&partnerID=8YFLogxK
U2 - 10.1145/3539813.3545122
DO - 10.1145/3539813.3545122
M3 - Conference contribution
AN - SCOPUS:85138329521
SP - 115
EP - 120
BT - ICTIR 2022
Y2 - 11 July 2022 through 12 July 2022
ER -