Details
| Originalsprache | Englisch |
|---|---|
| Titel des Sammelwerks | ICTIR 2022 |
| Untertitel | Proceedings of the 2022 ACM SIGIR International Conference on the Theory of Information Retrieval |
| Seiten | 115-120 |
| Seitenumfang | 6 |
| ISBN (elektronisch) | 9781450394123 |
| Publikationsstatus | Veröffentlicht - 25 Aug. 2022 |
| Veranstaltung | 8th ACM SIGIR International Conference on the Theory of Information Retrieval, ICTIR 2022 - Virtual, Online, Spanien Dauer: 11 Juli 2022 → 12 Juli 2022 |
Abstract
Contextual ranking models based on BERT are now well established for a wide range of passage and document ranking tasks. However, the robustness of BERT-based ranking models under adversarial inputs is under-explored. In this paper, we argue that BERT-rankers are not immune to adversarial attacks targeting retrieved documents given a query. Firstly, we propose algorithms for adversarial perturbation of both highly relevant and non-relevant documents using gradient-based optimization methods. The aim of our algorithms is to add/replace a small number of tokens to a highly relevant or non-relevant document to cause a large rank demotion or promotion. Our experiments show that a small number of tokens can already result in a large change in the rank of a document. Moreover, we find that BERT-rankers heavily rely on the document start/head for relevance prediction, making the initial part of the document more susceptible to adversarial attacks. More interestingly, we find a small set of recurring adversarial words that when added to documents result in successful rank demotion/promotion of any relevant/non-relevant document respectively. Finally, our adversarial tokens also show particular topic preferences within and across datasets, exposing potential biases from BERT pre-training or downstream datasets.
ASJC Scopus Sachgebiete
- Informatik (insg.)
- Informatik (sonstige)
- Informatik (insg.)
- Information systems
Zitieren
- Standard
- Harvard
- Apa
- Vancouver
- BibTex
- RIS
ICTIR 2022 : Proceedings of the 2022 ACM SIGIR International Conference on the Theory of Information Retrieval. 2022. S. 115-120.
Publikation: Beitrag in Buch/Bericht/Sammelwerk/Konferenzband › Aufsatz in Konferenzband › Forschung › Peer-Review
}
TY - GEN
T1 - BERT Rankers are Brittle
T2 - 8th ACM SIGIR International Conference on the Theory of Information Retrieval, ICTIR 2022
AU - Wang, Yumeng
AU - Lyu, Lijun
AU - Anand, Avishek
N1 - Funding Information: This work is partially supported by DFG Project AN 996/1-1.
PY - 2022/8/25
Y1 - 2022/8/25
N2 - Contextual ranking models based on BERT are now well established for a wide range of passage and document ranking tasks. However, the robustness of BERT-based ranking models under adversarial inputs is under-explored. In this paper, we argue that BERT-rankers are not immune to adversarial attacks targeting retrieved documents given a query. Firstly, we propose algorithms for adversarial perturbation of both highly relevant and non-relevant documents using gradient-based optimization methods. The aim of our algorithms is to add/replace a small number of tokens to a highly relevant or non-relevant document to cause a large rank demotion or promotion. Our experiments show that a small number of tokens can already result in a large change in the rank of a document. Moreover, we find that BERT-rankers heavily rely on the document start/head for relevance prediction, making the initial part of the document more susceptible to adversarial attacks. More interestingly, we find a small set of recurring adversarial words that when added to documents result in successful rank demotion/promotion of any relevant/non-relevant document respectively. Finally, our adversarial tokens also show particular topic preferences within and across datasets, exposing potential biases from BERT pre-training or downstream datasets.
AB - Contextual ranking models based on BERT are now well established for a wide range of passage and document ranking tasks. However, the robustness of BERT-based ranking models under adversarial inputs is under-explored. In this paper, we argue that BERT-rankers are not immune to adversarial attacks targeting retrieved documents given a query. Firstly, we propose algorithms for adversarial perturbation of both highly relevant and non-relevant documents using gradient-based optimization methods. The aim of our algorithms is to add/replace a small number of tokens to a highly relevant or non-relevant document to cause a large rank demotion or promotion. Our experiments show that a small number of tokens can already result in a large change in the rank of a document. Moreover, we find that BERT-rankers heavily rely on the document start/head for relevance prediction, making the initial part of the document more susceptible to adversarial attacks. More interestingly, we find a small set of recurring adversarial words that when added to documents result in successful rank demotion/promotion of any relevant/non-relevant document respectively. Finally, our adversarial tokens also show particular topic preferences within and across datasets, exposing potential biases from BERT pre-training or downstream datasets.
KW - adversarial attack
KW - bert
KW - biases
KW - neural networks
KW - ranking
UR - http://www.scopus.com/inward/record.url?scp=85138329521&partnerID=8YFLogxK
U2 - 10.1145/3539813.3545122
DO - 10.1145/3539813.3545122
M3 - Conference contribution
AN - SCOPUS:85138329521
SP - 115
EP - 120
BT - ICTIR 2022
Y2 - 11 July 2022 through 12 July 2022
ER -