Details
| Originalsprache | Englisch |
|---|---|
| Titel des Sammelwerks | Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems, CHI EA 2021 |
| Erscheinungsort | New York, NY, USA |
| Herausgeber (Verlag) | Association for Computing Machinery (ACM) |
| Seiten | 1-7 |
| ISBN (elektronisch) | 9781450380959 |
| Publikationsstatus | Veröffentlicht - 8 Mai 2021 |
| Veranstaltung | CHI Conference on Human Factors in Computing Systems: Making Waves, Combining Strengths (CHI EA 2021) - Virtual, Online, Japan Dauer: 8 Mai 2021 → 13 Mai 2021 |
Publikationsreihe
| Name | Conference on Human Factors in Computing Systems - Proceedings |
|---|
Abstract
People with visual impairments face challenges in scene and object recognition, especially in unknown environments. We combined the mobile scene detection framework Apple ARKit with MobileNet-v2 and 3D spatial audio to provide an auditory scene description to people with visual impairments. The combination of ARKit and MobileNet allows keeping recognized objects in the scene even if the user turns away from the object. An object can thus serve as an auditory landmark. With a search function, the system can even guide the user to a particular item. The system also provides spatial audio warnings for nearby objects and walls to avoid collisions. We evaluated the implemented app in a preliminary user study. The results show that users can find items without visual feedback using the proposed application. The study also reveals that the range of local object detection through MobileNet-v2 was insufficient, which we aim to overcome using more accurate object detection frameworks in future work (YOLOv5x).
ASJC Scopus Sachgebiete
- Informatik (insg.)
- Mensch-Maschine-Interaktion
- Informatik (insg.)
- Computergrafik und computergestütztes Design
- Informatik (insg.)
- Software
Zitieren
- Standard
- Harvard
- Apa
- Vancouver
- BibTex
- RIS
Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems, CHI EA 2021. New York, NY, USA: Association for Computing Machinery (ACM), 2021. S. 1-7 394 (Conference on Human Factors in Computing Systems - Proceedings).
Publikation: Beitrag in Buch/Bericht/Sammelwerk/Konferenzband › Aufsatz in Konferenzband › Forschung › Peer-Review
}
TY - GEN
T1 - Mobile Recognition and Tracking of Objects in the Environment through Augmented Reality and 3D Audio Cues for People with Visual Impairments
AU - Kaul, Oliver Beren
AU - Behrens, Kersten
AU - Rohs, Michael
PY - 2021/5/8
Y1 - 2021/5/8
N2 - People with visual impairments face challenges in scene and object recognition, especially in unknown environments. We combined the mobile scene detection framework Apple ARKit with MobileNet-v2 and 3D spatial audio to provide an auditory scene description to people with visual impairments. The combination of ARKit and MobileNet allows keeping recognized objects in the scene even if the user turns away from the object. An object can thus serve as an auditory landmark. With a search function, the system can even guide the user to a particular item. The system also provides spatial audio warnings for nearby objects and walls to avoid collisions. We evaluated the implemented app in a preliminary user study. The results show that users can find items without visual feedback using the proposed application. The study also reveals that the range of local object detection through MobileNet-v2 was insufficient, which we aim to overcome using more accurate object detection frameworks in future work (YOLOv5x).
AB - People with visual impairments face challenges in scene and object recognition, especially in unknown environments. We combined the mobile scene detection framework Apple ARKit with MobileNet-v2 and 3D spatial audio to provide an auditory scene description to people with visual impairments. The combination of ARKit and MobileNet allows keeping recognized objects in the scene even if the user turns away from the object. An object can thus serve as an auditory landmark. With a search function, the system can even guide the user to a particular item. The system also provides spatial audio warnings for nearby objects and walls to avoid collisions. We evaluated the implemented app in a preliminary user study. The results show that users can find items without visual feedback using the proposed application. The study also reveals that the range of local object detection through MobileNet-v2 was insufficient, which we aim to overcome using more accurate object detection frameworks in future work (YOLOv5x).
KW - Accessibility
KW - Auditory Scene Description
KW - Collision Warnings
KW - Mobile Scene Recognition
KW - Object Detection
KW - Spatial Audio
KW - Text-to-Speech
KW - Visually Impaired
UR - http://www.scopus.com/inward/record.url?scp=85105779312&partnerID=8YFLogxK
U2 - 10.1145/3411763.3451611
DO - 10.1145/3411763.3451611
M3 - Conference contribution
AN - SCOPUS:85105779312
T3 - Conference on Human Factors in Computing Systems - Proceedings
SP - 1
EP - 7
BT - Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems, CHI EA 2021
PB - Association for Computing Machinery (ACM)
CY - New York, NY, USA
T2 - CHI Conference on Human Factors in Computing Systems: Making Waves, Combining Strengths (CHI EA 2021)
Y2 - 8 May 2021 through 13 May 2021
ER -