TY - JOUR

T1 - AutoML in the Age of Large Language Models: Current Challenges, Future Opportunities and Risks

AU - Tornede, Alexander

AU - Deng, Difan

AU - Eimer, Theresa

AU - Giovanelli, Joseph

AU - Mohan, Aditya

AU - Ruhkopf, Tim

AU - Segel, Sarah

AU - Theodorakopoulos, Daphne

AU - Tornede, Tanja

AU - Wachsmuth, Henning

AU - Lindauer, Marius

PY - 2024/2/9

Y1 - 2024/2/9

N2 - The fields of both Natural Language Processing (NLP) and Automated Machine Learning (AutoML) have achieved remarkable results over the past years. In NLP, especially Large Language Models (LLMs) have experienced a rapid series of breakthroughs very recently. We envision that the two fields can radically push the boundaries of each other through tight integration. To showcase this vision, we explore the potential of a symbiotic relationship between AutoML and LLMs, shedding light on how they can benefit each other. In particular, we investigate both the opportunities to enhance AutoML approaches with LLMs from different perspectives and the challenges of leveraging AutoML to further improve LLMs. To this end, we survey existing work, and we critically assess risks. We strongly believe that the integration of the two fields has the potential to disrupt both fields, NLP and AutoML. By highlighting conceivable synergies, but also risks, we aim to foster further exploration at the intersection of AutoML and LLMs.

AB - The fields of both Natural Language Processing (NLP) and Automated Machine Learning (AutoML) have achieved remarkable results over the past years. In NLP, especially Large Language Models (LLMs) have experienced a rapid series of breakthroughs very recently. We envision that the two fields can radically push the boundaries of each other through tight integration. To showcase this vision, we explore the potential of a symbiotic relationship between AutoML and LLMs, shedding light on how they can benefit each other. In particular, we investigate both the opportunities to enhance AutoML approaches with LLMs from different perspectives and the challenges of leveraging AutoML to further improve LLMs. To this end, we survey existing work, and we critically assess risks. We strongly believe that the integration of the two fields has the potential to disrupt both fields, NLP and AutoML. By highlighting conceivable synergies, but also risks, we aim to foster further exploration at the intersection of AutoML and LLMs.

KW - cs.LG

KW - cs.CL

U2 - 10.48550/arXiv.2306.08107

DO - 10.48550/arXiv.2306.08107

M3 - Article

JO - Transactions on Machine Learning Research

JF - Transactions on Machine Learning Research

SN - 2835-8856

ER -