Details
| Originalsprache | Englisch |
|---|---|
| Titel des Sammelwerks | Natural Language Processing and Information Systems |
| Untertitel | 29th International Conference on Applications of Natural Language to Information Systems, NLDB 2024, Proceedings |
| Herausgeber/-innen | Amon Rapp, Luigi Di Caro, Farid Meziane, Vijayan Sugumaran |
| Herausgeber (Verlag) | Springer Science and Business Media Deutschland GmbH |
| Seiten | 183-194 |
| Seitenumfang | 12 |
| ISBN (elektronisch) | 978-3-031-70242-6 |
| ISBN (Print) | 9783031702419 |
| Publikationsstatus | Veröffentlicht - 20 Sept. 2024 |
| Veranstaltung | 29th International Conference on Natural Language and Information Systems, NLDB 2024 - Turin, Italien Dauer: 25 Juni 2024 → 27 Juni 2024 |
Publikationsreihe
| Name | Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) |
|---|---|
| Band | 14763 LNCS |
| ISSN (Print) | 0302-9743 |
| ISSN (elektronisch) | 1611-3349 |
Abstract
Questions are an integral part of test formats in education. Also online learning platforms like Coursera or Udemy use questions to check learners’ understanding. However, the manual creation of questions can be very time-intensive. This problem can be mitigated through automatic question generation. In this paper, we present a comparison of fine-tuned text-generating transformers for question generation. Our methods include (i) a comparison of multiple fine-tuned transformers to identify differences in the generated output, (ii) a comparison of multiple token search strategies evaluated on each model to find differences in generated questions across different strategies and (iii) a newly developed manual evaluation metric that evaluates generated questions regarding aspects of naturalness and suitability. Our experiments show a difference in question length, structure and quality depending on the used transformer architecture, which indicates a correlation between transformer architecture and question structure. Furthermore, different search strategies for the same model architecture do not greatly impact structure or quality.
ASJC Scopus Sachgebiete
- Mathematik (insg.)
- Theoretische Informatik
- Informatik (insg.)
- Allgemeine Computerwissenschaft
Zitieren
- Standard
- Harvard
- Apa
- Vancouver
- BibTex
- RIS
Natural Language Processing and Information Systems : 29th International Conference on Applications of Natural Language to Information Systems, NLDB 2024, Proceedings. Hrsg. / Amon Rapp; Luigi Di Caro; Farid Meziane; Vijayan Sugumaran. Springer Science and Business Media Deutschland GmbH, 2024. S. 183-194 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Band 14763 LNCS).
Publikation: Beitrag in Buch/Bericht/Sammelwerk/Konferenzband › Aufsatz in Konferenzband › Forschung › Peer-Review
}
TY - GEN
T1 - Question Generation Capabilities of “Small" Large Language Models
AU - Berger, Joshua
AU - Koß, Jonathan
AU - Stamatakis, Markos
AU - Hoppe, Anett
AU - Ewerth, Ralph
AU - Wartena, Christian
N1 - Publisher Copyright: © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.
PY - 2024/9/20
Y1 - 2024/9/20
N2 - Questions are an integral part of test formats in education. Also online learning platforms like Coursera or Udemy use questions to check learners’ understanding. However, the manual creation of questions can be very time-intensive. This problem can be mitigated through automatic question generation. In this paper, we present a comparison of fine-tuned text-generating transformers for question generation. Our methods include (i) a comparison of multiple fine-tuned transformers to identify differences in the generated output, (ii) a comparison of multiple token search strategies evaluated on each model to find differences in generated questions across different strategies and (iii) a newly developed manual evaluation metric that evaluates generated questions regarding aspects of naturalness and suitability. Our experiments show a difference in question length, structure and quality depending on the used transformer architecture, which indicates a correlation between transformer architecture and question structure. Furthermore, different search strategies for the same model architecture do not greatly impact structure or quality.
AB - Questions are an integral part of test formats in education. Also online learning platforms like Coursera or Udemy use questions to check learners’ understanding. However, the manual creation of questions can be very time-intensive. This problem can be mitigated through automatic question generation. In this paper, we present a comparison of fine-tuned text-generating transformers for question generation. Our methods include (i) a comparison of multiple fine-tuned transformers to identify differences in the generated output, (ii) a comparison of multiple token search strategies evaluated on each model to find differences in generated questions across different strategies and (iii) a newly developed manual evaluation metric that evaluates generated questions regarding aspects of naturalness and suitability. Our experiments show a difference in question length, structure and quality depending on the used transformer architecture, which indicates a correlation between transformer architecture and question structure. Furthermore, different search strategies for the same model architecture do not greatly impact structure or quality.
KW - Automatic Question Generation
KW - Pre-trained Transformer
KW - Transformer Architecture
UR - http://www.scopus.com/inward/record.url?scp=85205514065&partnerID=8YFLogxK
U2 - 10.1007/978-3-031-70242-6_18
DO - 10.1007/978-3-031-70242-6_18
M3 - Conference contribution
AN - SCOPUS:85205514065
SN - 9783031702419
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 183
EP - 194
BT - Natural Language Processing and Information Systems
A2 - Rapp, Amon
A2 - Di Caro, Luigi
A2 - Meziane, Farid
A2 - Sugumaran, Vijayan
PB - Springer Science and Business Media Deutschland GmbH
T2 - 29th International Conference on Natural Language and Information Systems, NLDB 2024
Y2 - 25 June 2024 through 27 June 2024
ER -