Details
| Original language | English |
|---|---|
| Pages (from-to) | 408-425 |
| Number of pages | 18 |
| Journal | Cognitive computation |
| Volume | 10 |
| Issue number | 3 |
| Publication status | Published - 1 Jun 2018 |
| Externally published | Yes |
Abstract
In computational systems for visuo-haptic object recognition, vision and haptics are often modeled as separate processes. But this is far from what really happens in the human brain, where cross- as well as multimodal interactions take place between the two sensory modalities. Generally, three main principles can be identified as underlying the processing of the visual and haptic object-related stimuli in the brain: (1) hierarchical processing, (2) the divergence of the processing onto substreams for object shape and material perception, and (3) the experience-driven self-organization of the integratory neural circuits. The question arises whether an object recognition system can benefit in terms of performance from adopting these brain-inspired processing principles for the integration of the visual and haptic inputs. To address this, we compare the integration strategy that incorporates all three principles to the two commonly used integration strategies in the literature. We collected data with a NAO robot enhanced with inexpensive contact microphones as tactile sensors. The results of our experiments involving every-day objects indicate that (1) the contact microphones are a good alternative to capturing tactile information and that (2) organizing the processing of the visual and haptic inputs hierarchically and in two pre-processing streams is helpful performance-wise. Nevertheless, further research is needed to effectively quantify the role of each identified principle by itself as well as in combination with others.
Keywords
- Brain-inspired architectures, Haptics, Multimodal integration, Object recognition, Tactile sensing, Vision
ASJC Scopus subject areas
- Computer Science(all)
- Computer Vision and Pattern Recognition
- Computer Science(all)
- Computer Science Applications
- Neuroscience(all)
- Cognitive Neuroscience
Cite this
- Standard
- Harvard
- Apa
- Vancouver
- BibTeX
- RIS
In: Cognitive computation, Vol. 10, No. 3, 01.06.2018, p. 408-425.
Research output: Contribution to journal › Article › Research › peer review
}
TY - JOUR
T1 - Evaluating Integration Strategies for Visuo-Haptic Object Recognition
AU - Toprak, Sibel
AU - Navarro-Guerrero, Nicolás
AU - Wermter, Stefan
N1 - Publisher Copyright: © 2017, The Author(s).
PY - 2018/6/1
Y1 - 2018/6/1
N2 - In computational systems for visuo-haptic object recognition, vision and haptics are often modeled as separate processes. But this is far from what really happens in the human brain, where cross- as well as multimodal interactions take place between the two sensory modalities. Generally, three main principles can be identified as underlying the processing of the visual and haptic object-related stimuli in the brain: (1) hierarchical processing, (2) the divergence of the processing onto substreams for object shape and material perception, and (3) the experience-driven self-organization of the integratory neural circuits. The question arises whether an object recognition system can benefit in terms of performance from adopting these brain-inspired processing principles for the integration of the visual and haptic inputs. To address this, we compare the integration strategy that incorporates all three principles to the two commonly used integration strategies in the literature. We collected data with a NAO robot enhanced with inexpensive contact microphones as tactile sensors. The results of our experiments involving every-day objects indicate that (1) the contact microphones are a good alternative to capturing tactile information and that (2) organizing the processing of the visual and haptic inputs hierarchically and in two pre-processing streams is helpful performance-wise. Nevertheless, further research is needed to effectively quantify the role of each identified principle by itself as well as in combination with others.
AB - In computational systems for visuo-haptic object recognition, vision and haptics are often modeled as separate processes. But this is far from what really happens in the human brain, where cross- as well as multimodal interactions take place between the two sensory modalities. Generally, three main principles can be identified as underlying the processing of the visual and haptic object-related stimuli in the brain: (1) hierarchical processing, (2) the divergence of the processing onto substreams for object shape and material perception, and (3) the experience-driven self-organization of the integratory neural circuits. The question arises whether an object recognition system can benefit in terms of performance from adopting these brain-inspired processing principles for the integration of the visual and haptic inputs. To address this, we compare the integration strategy that incorporates all three principles to the two commonly used integration strategies in the literature. We collected data with a NAO robot enhanced with inexpensive contact microphones as tactile sensors. The results of our experiments involving every-day objects indicate that (1) the contact microphones are a good alternative to capturing tactile information and that (2) organizing the processing of the visual and haptic inputs hierarchically and in two pre-processing streams is helpful performance-wise. Nevertheless, further research is needed to effectively quantify the role of each identified principle by itself as well as in combination with others.
KW - Brain-inspired architectures
KW - Haptics
KW - Multimodal integration
KW - Object recognition
KW - Tactile sensing
KW - Vision
UR - http://www.scopus.com/inward/record.url?scp=85039561066&partnerID=8YFLogxK
U2 - 10.1007/s12559-017-9536-7
DO - 10.1007/s12559-017-9536-7
M3 - Article
AN - SCOPUS:85039561066
VL - 10
SP - 408
EP - 425
JO - Cognitive computation
JF - Cognitive computation
SN - 1866-9956
IS - 3
ER -