Evaluation of Automated Image Descriptions for Visually Impaired Students

Research output: Chapter in book/report/conference proceedingConference contributionResearchpeer review

Authors

  • Anett Hoppe
  • David Morris
  • Ralph Ewerth

External Research Organisations

  • German National Library of Science and Technology (TIB)
View graph of relations

Details

Original languageEnglish
Title of host publicationArtificial Intelligence in Education
Subtitle of host publication22nd International Conference, AIED 2021, Utrecht, The Netherlands, June 14–18, 2021, Proceedings, Part II
EditorsIdo Roll, Danielle McNamara, Sergey Sosnovsky, Rose Luckin, Vania Dimitrova
Place of PublicationCham
Pages196-201
Number of pages6
ISBN (electronic)978-3-030-78270-2
Publication statusPublished - 12 Jun 2021

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume12749 LNAI
ISSN (Print)0302-9743
ISSN (electronic)1611-3349

Abstract

Illustrations are widely used in education, and sometimes, alternatives are not available for visually impaired students. Therefore, those students would benefit greatly from an automatic illustration description system, but only if those descriptions were complete, correct, and easily understandable using a screenreader. In this paper, we report on a study for the assessment of automated image descriptions. We interviewed experts to establish evaluation criteria, which we then used to create an evaluation questionnaire for sighted non-expert raters, and description templates. We used this questionnaire to evaluate the quality of descriptions which could be generated with a template-based automatic image describer. We present evidence that these templates have the potential to generate useful descriptions, and that the questionnaire identifies problems with description templates.

Keywords

    Accessibility, Automatic image description, Blind and visually impaired, Educational resources

ASJC Scopus subject areas

Cite this

Evaluation of Automated Image Descriptions for Visually Impaired Students. / Hoppe, Anett; Morris, David; Ewerth, Ralph.
Artificial Intelligence in Education: 22nd International Conference, AIED 2021, Utrecht, The Netherlands, June 14–18, 2021, Proceedings, Part II. ed. / Ido Roll; Danielle McNamara; Sergey Sosnovsky; Rose Luckin; Vania Dimitrova. Cham, 2021. p. 196-201 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12749 LNAI).

Research output: Chapter in book/report/conference proceedingConference contributionResearchpeer review

Hoppe, A, Morris, D & Ewerth, R 2021, Evaluation of Automated Image Descriptions for Visually Impaired Students. in I Roll, D McNamara, S Sosnovsky, R Luckin & V Dimitrova (eds), Artificial Intelligence in Education: 22nd International Conference, AIED 2021, Utrecht, The Netherlands, June 14–18, 2021, Proceedings, Part II. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 12749 LNAI, Cham, pp. 196-201. https://doi.org/10.1007/978-3-030-78270-2_35
Hoppe, A., Morris, D., & Ewerth, R. (2021). Evaluation of Automated Image Descriptions for Visually Impaired Students. In I. Roll, D. McNamara, S. Sosnovsky, R. Luckin, & V. Dimitrova (Eds.), Artificial Intelligence in Education: 22nd International Conference, AIED 2021, Utrecht, The Netherlands, June 14–18, 2021, Proceedings, Part II (pp. 196-201). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12749 LNAI).. https://doi.org/10.1007/978-3-030-78270-2_35
Hoppe A, Morris D, Ewerth R. Evaluation of Automated Image Descriptions for Visually Impaired Students. In Roll I, McNamara D, Sosnovsky S, Luckin R, Dimitrova V, editors, Artificial Intelligence in Education: 22nd International Conference, AIED 2021, Utrecht, The Netherlands, June 14–18, 2021, Proceedings, Part II. Cham. 2021. p. 196-201. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-78270-2_35
Hoppe, Anett ; Morris, David ; Ewerth, Ralph. / Evaluation of Automated Image Descriptions for Visually Impaired Students. Artificial Intelligence in Education: 22nd International Conference, AIED 2021, Utrecht, The Netherlands, June 14–18, 2021, Proceedings, Part II. editor / Ido Roll ; Danielle McNamara ; Sergey Sosnovsky ; Rose Luckin ; Vania Dimitrova. Cham, 2021. pp. 196-201 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
Download
@inproceedings{2998b35cff764567b7653c8bb6ef8eb4,
title = "Evaluation of Automated Image Descriptions for Visually Impaired Students",
abstract = "Illustrations are widely used in education, and sometimes, alternatives are not available for visually impaired students. Therefore, those students would benefit greatly from an automatic illustration description system, but only if those descriptions were complete, correct, and easily understandable using a screenreader. In this paper, we report on a study for the assessment of automated image descriptions. We interviewed experts to establish evaluation criteria, which we then used to create an evaluation questionnaire for sighted non-expert raters, and description templates. We used this questionnaire to evaluate the quality of descriptions which could be generated with a template-based automatic image describer. We present evidence that these templates have the potential to generate useful descriptions, and that the questionnaire identifies problems with description templates.",
keywords = "Accessibility, Automatic image description, Blind and visually impaired, Educational resources",
author = "Anett Hoppe and David Morris and Ralph Ewerth",
note = "Funding Information: This work is financially supported by the German Federal Ministry of Education and Research (BMBF) and the European Social Fund (ESF) (Project InclusiveOCW, no. 01PE17004).",
year = "2021",
month = jun,
day = "12",
doi = "10.1007/978-3-030-78270-2_35",
language = "English",
isbn = "978-3-030-78269-6",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
pages = "196--201",
editor = "Ido Roll and Danielle McNamara and Sergey Sosnovsky and Rose Luckin and Vania Dimitrova",
booktitle = "Artificial Intelligence in Education",

}

Download

TY - GEN

T1 - Evaluation of Automated Image Descriptions for Visually Impaired Students

AU - Hoppe, Anett

AU - Morris, David

AU - Ewerth, Ralph

N1 - Funding Information: This work is financially supported by the German Federal Ministry of Education and Research (BMBF) and the European Social Fund (ESF) (Project InclusiveOCW, no. 01PE17004).

PY - 2021/6/12

Y1 - 2021/6/12

N2 - Illustrations are widely used in education, and sometimes, alternatives are not available for visually impaired students. Therefore, those students would benefit greatly from an automatic illustration description system, but only if those descriptions were complete, correct, and easily understandable using a screenreader. In this paper, we report on a study for the assessment of automated image descriptions. We interviewed experts to establish evaluation criteria, which we then used to create an evaluation questionnaire for sighted non-expert raters, and description templates. We used this questionnaire to evaluate the quality of descriptions which could be generated with a template-based automatic image describer. We present evidence that these templates have the potential to generate useful descriptions, and that the questionnaire identifies problems with description templates.

AB - Illustrations are widely used in education, and sometimes, alternatives are not available for visually impaired students. Therefore, those students would benefit greatly from an automatic illustration description system, but only if those descriptions were complete, correct, and easily understandable using a screenreader. In this paper, we report on a study for the assessment of automated image descriptions. We interviewed experts to establish evaluation criteria, which we then used to create an evaluation questionnaire for sighted non-expert raters, and description templates. We used this questionnaire to evaluate the quality of descriptions which could be generated with a template-based automatic image describer. We present evidence that these templates have the potential to generate useful descriptions, and that the questionnaire identifies problems with description templates.

KW - Accessibility

KW - Automatic image description

KW - Blind and visually impaired

KW - Educational resources

UR - http://www.scopus.com/inward/record.url?scp=85126586388&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-78270-2_35

DO - 10.1007/978-3-030-78270-2_35

M3 - Conference contribution

SN - 978-3-030-78269-6

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 196

EP - 201

BT - Artificial Intelligence in Education

A2 - Roll, Ido

A2 - McNamara, Danielle

A2 - Sosnovsky, Sergey

A2 - Luckin, Rose

A2 - Dimitrova, Vania

CY - Cham

ER -