Details
| Original language | English |
|---|---|
| Title of host publication | Artificial Intelligence in Education |
| Subtitle of host publication | 22nd International Conference, AIED 2021, Utrecht, The Netherlands, June 14–18, 2021, Proceedings, Part I |
| Editors | Ido Roll, Danielle McNamara, Sergey Sosnovsky, Rose Luckin, Vania Dimitrova |
| Pages | 318-330 |
| Number of pages | 13 |
| Volume | 1 |
| ISBN (electronic) | 978-3-030-78292-4 |
| Publication status | Published - 11 Jun 2021 |
Publication series
| Name | Artificial Intelligence in Education |
|---|---|
| Volume | 12748 |
| ISSN (Print) | 0302-9743 |
| ISSN (electronic) | 1611-3349 |
Abstract
In informal learning scenarios the popularity of multimedia content, such as video tutorials or lectures, has significantly increased. Yet, the users’ interactions, navigation behavior, and consequently learning outcome, have not been researched extensively. Related work in this field, also called search as learning, has focused on behavioral or text resource features to predict learning outcome and knowledge gain. In this paper, we investigate whether we can exploit features representing multimedia resource consumption to predict the knowledge gain (KG) during Web search from in-session data, that is without prior knowledge about the learner. For this purpose, we suggest a set of multimedia features related to image and video consumption. Our feature extraction is evaluated in a lab study with 113 participants where we collected data for a given search as learning task on the formation of thunderstorms and lightning. We automatically analyze the monitored log data and utilize state-of-the-art computer vision methods to extract features about the seen multimedia resources. Experimental results demonstrate that multimedia features can improve KG prediction. Finally, we provide an analysis on feature importance (text and multimedia) for KG prediction.
Keywords
- Document layout analysis, Knowledge gain, Learning resources, Multimedia information extraction, Search as learning
ASJC Scopus subject areas
- Mathematics(all)
- Theoretical Computer Science
- Computer Science(all)
- General Computer Science
Cite this
- Standard
- Harvard
- Apa
- Vancouver
- BibTeX
- RIS
Artificial Intelligence in Education: 22nd International Conference, AIED 2021, Utrecht, The Netherlands, June 14–18, 2021, Proceedings, Part I. ed. / Ido Roll; Danielle McNamara; Sergey Sosnovsky; Rose Luckin; Vania Dimitrova. Vol. 1 2021. p. 318-330 Chapter 26 (Artificial Intelligence in Education; Vol. 12748).
Research output: Chapter in book/report/conference proceeding › Conference contribution › Research › peer review
}
TY - GEN
T1 - Predicting Knowledge Gain During Web Search Based on Multimedia Resource Consumption
AU - Otto, Christian
AU - Yu, Ran
AU - Pardi, Georg
AU - Von Hoyer, Johannes
AU - Rokicki, Markus
AU - Hoppe, Anett
AU - Holtz, Peter
AU - Kammerer, Yvonne
AU - Dietze, Stefan
AU - Ewerth, Ralph
N1 - Funding Information: Keywords: Knowledge gain · Multimedia information extraction · Document layout analysis · Search as learning · Learning resources C. Otto and R. Yu—Authors contributed equally to this work. Part of this work is financially supported by the Leibniz Association, Germany (Leibniz Competition 2018, funding line “Collaborative Excellence”, project SALIENT [K68/2017]).
PY - 2021/6/11
Y1 - 2021/6/11
N2 - In informal learning scenarios the popularity of multimedia content, such as video tutorials or lectures, has significantly increased. Yet, the users’ interactions, navigation behavior, and consequently learning outcome, have not been researched extensively. Related work in this field, also called search as learning, has focused on behavioral or text resource features to predict learning outcome and knowledge gain. In this paper, we investigate whether we can exploit features representing multimedia resource consumption to predict the knowledge gain (KG) during Web search from in-session data, that is without prior knowledge about the learner. For this purpose, we suggest a set of multimedia features related to image and video consumption. Our feature extraction is evaluated in a lab study with 113 participants where we collected data for a given search as learning task on the formation of thunderstorms and lightning. We automatically analyze the monitored log data and utilize state-of-the-art computer vision methods to extract features about the seen multimedia resources. Experimental results demonstrate that multimedia features can improve KG prediction. Finally, we provide an analysis on feature importance (text and multimedia) for KG prediction.
AB - In informal learning scenarios the popularity of multimedia content, such as video tutorials or lectures, has significantly increased. Yet, the users’ interactions, navigation behavior, and consequently learning outcome, have not been researched extensively. Related work in this field, also called search as learning, has focused on behavioral or text resource features to predict learning outcome and knowledge gain. In this paper, we investigate whether we can exploit features representing multimedia resource consumption to predict the knowledge gain (KG) during Web search from in-session data, that is without prior knowledge about the learner. For this purpose, we suggest a set of multimedia features related to image and video consumption. Our feature extraction is evaluated in a lab study with 113 participants where we collected data for a given search as learning task on the formation of thunderstorms and lightning. We automatically analyze the monitored log data and utilize state-of-the-art computer vision methods to extract features about the seen multimedia resources. Experimental results demonstrate that multimedia features can improve KG prediction. Finally, we provide an analysis on feature importance (text and multimedia) for KG prediction.
KW - Document layout analysis
KW - Knowledge gain
KW - Learning resources
KW - Multimedia information extraction
KW - Search as learning
UR - http://www.scopus.com/inward/record.url?scp=85122892374&partnerID=8YFLogxK
U2 - 10.1007/978-3-030-78292-4_26
DO - 10.1007/978-3-030-78292-4_26
M3 - Conference contribution
SN - 978-3-030-78291-7
VL - 1
T3 - Artificial Intelligence in Education
SP - 318
EP - 330
BT - Artificial Intelligence in Education
A2 - Roll, Ido
A2 - McNamara, Danielle
A2 - Sosnovsky, Sergey
A2 - Luckin, Rose
A2 - Dimitrova, Vania
ER -