Details
| Original language | English |
|---|---|
| Title of host publication | 2023 IEEE International Conference on Acoustics, Speech and Signal Processing |
| Subtitle of host publication | ICASSP |
| Publisher | Institute of Electrical and Electronics Engineers Inc. |
| Number of pages | 5 |
| ISBN (electronic) | 978-1-7281-6327-7 |
| ISBN (print) | 978-1-7281-6328-4 |
| Publication status | Published - 2023 |
| Event | 48th IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2023 - Rhodes Island, Greece Duration: 4 Jun 2023 → 10 Jun 2023 |
Abstract
Speech emotion recognition (SER) has been a popular research topic in human-computer interaction (HCI). As edge devices are rapidly springing up, applying SER to edge devices is promising for a huge number of HCI applications. Although deep learning has been investigated to improve the performance of SER by training complex models, the memory space and computational capability of edge devices represents a constraint for embedding deep learning models. We propose a neural structured learning (NSL) framework through building synthesized graphs. An SER model is trained on a source dataset and used to build graphs on a target dataset. A relatively lightweight model is then trained with the speech samples and graphs together as the input. Our experiments demonstrate that training a lightweight SER model on the target dataset with speech samples and graphs can not only produce small SER models, but also enhance the model performance compared to models with speech samples only and those using classic transfer learning strategies.
Keywords
- edge device, lightweight deep learning, neural structured learning, Speech emotion recognition
ASJC Scopus subject areas
- Computer Science(all)
- Software
- Computer Science(all)
- Signal Processing
- Engineering(all)
- Electrical and Electronic Engineering
Cite this
- Standard
- Harvard
- Apa
- Vancouver
- BibTeX
- RIS
2023 IEEE International Conference on Acoustics, Speech and Signal Processing : ICASSP. Institute of Electrical and Electronics Engineers Inc., 2023.
Research output: Chapter in book/report/conference proceeding › Conference contribution › Research › peer review
}
TY - GEN
T1 - Knowledge Transfer for on-Device Speech Emotion Recognition With Neural Structured Learning
AU - Chang, Yi
AU - Ren, Zhao
AU - Nguyen, Thanh Tam
AU - Qian, Kun
AU - Schuller, Bjorn W.
N1 - Funding Information: This research was partially funded by the Federal Ministry of Education and Research (BMBF), Germany under the project LeibnizKILabor with grant No. 01DD20003, and the research projects “IIP-Ecosphere”, granted by the German Federal Ministry for Economics and Climate Action (BMWK) via funding code No. 01MK20006A, the Ministry of Science and Technology of the People’s Republic of China (No. 2021ZD0201900), the National Natural Science Foundation of China (No. 62272044), the National High-Level Young Talent Project, and the BIT Teli Young Fellow Program from the Beijing Institute of Technology, China.
PY - 2023
Y1 - 2023
N2 - Speech emotion recognition (SER) has been a popular research topic in human-computer interaction (HCI). As edge devices are rapidly springing up, applying SER to edge devices is promising for a huge number of HCI applications. Although deep learning has been investigated to improve the performance of SER by training complex models, the memory space and computational capability of edge devices represents a constraint for embedding deep learning models. We propose a neural structured learning (NSL) framework through building synthesized graphs. An SER model is trained on a source dataset and used to build graphs on a target dataset. A relatively lightweight model is then trained with the speech samples and graphs together as the input. Our experiments demonstrate that training a lightweight SER model on the target dataset with speech samples and graphs can not only produce small SER models, but also enhance the model performance compared to models with speech samples only and those using classic transfer learning strategies.
AB - Speech emotion recognition (SER) has been a popular research topic in human-computer interaction (HCI). As edge devices are rapidly springing up, applying SER to edge devices is promising for a huge number of HCI applications. Although deep learning has been investigated to improve the performance of SER by training complex models, the memory space and computational capability of edge devices represents a constraint for embedding deep learning models. We propose a neural structured learning (NSL) framework through building synthesized graphs. An SER model is trained on a source dataset and used to build graphs on a target dataset. A relatively lightweight model is then trained with the speech samples and graphs together as the input. Our experiments demonstrate that training a lightweight SER model on the target dataset with speech samples and graphs can not only produce small SER models, but also enhance the model performance compared to models with speech samples only and those using classic transfer learning strategies.
KW - edge device
KW - lightweight deep learning
KW - neural structured learning
KW - Speech emotion recognition
UR - http://www.scopus.com/inward/record.url?scp=85171249470&partnerID=8YFLogxK
U2 - 10.48550/arXiv.2210.14977
DO - 10.48550/arXiv.2210.14977
M3 - Conference contribution
AN - SCOPUS:85171249470
SN - 978-1-7281-6328-4
BT - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 48th IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2023
Y2 - 4 June 2023 through 10 June 2023
ER -