Details
| Original language | English |
|---|---|
| Pages (from-to) | 3054 - 3061 |
| Number of pages | 8 |
| Journal | IEEE Robotics and Automation Letters |
| Volume | 7 |
| Issue number | 2 |
| Publication status | Published - 14 Jan 2022 |
Abstract
Keywords
- Data integration, Feature extraction, Location awareness, Object Detection, Point cloud compression, Proposals, Segmentation and Categorization, Sensor Fusion, Sensor Networks, Three-dimensional displays, Vehicle detection, Sensor fusion, object detection, sensor networks, segmentation and categorization
ASJC Scopus subject areas
- Engineering(all)
- Mechanical Engineering
- Mathematics(all)
- Control and Optimization
- Computer Science(all)
- Artificial Intelligence
- Computer Science(all)
- Human-Computer Interaction
- Engineering(all)
- Control and Systems Engineering
- Computer Science(all)
- Computer Vision and Pattern Recognition
- Engineering(all)
- Biomedical Engineering
- Computer Science(all)
- Computer Science Applications
Cite this
- Standard
- Harvard
- Apa
- Vancouver
- BibTeX
- RIS
In: IEEE Robotics and Automation Letters, Vol. 7, No. 2, 14.01.2022, p. 3054 - 3061.
Research output: Contribution to journal › Article › Research › peer review
}
TY - JOUR
T1 - Keypoints-Based Deep Feature Fusion for Cooperative Vehicle Detection of Autonomous Driving
AU - Yuan, Yunshuang
AU - Cheng, Hao
AU - Sester, Monika
N1 - Funding Information: This letter was recommended for publication by Associate Editor G. Costante and Editor E. Marchand upon evaluation of the reviewers' comments. This work was supported by the Projects DFG RTC1931 SocialCars and DFG GRK2159 i.c.sens.
PY - 2022/1/14
Y1 - 2022/1/14
N2 - Sharing collective perception messages (CPM) between vehicles is investigated to decrease occlusions, so as to improve perception accuracy and safety of autonomous driving. However, highly accurate data sharing and low communication overhead is a big challenge for collective perception, especially when real-time communication is required among connected and automated vehicles. In this paper, we propose an efficient and effective keypoints-based deep feature fusion framework, called FPV-RCNN, for collective perception, which is built on top of the 3D object detector PV-RCNN. We introduce a bounding box proposal matching module and a keypoints selection strategy to compress the CPM size and solve the multi-vehicle data fusion problem. Compared to a bird's-eye view (BEV) keypoints feature fusion, FPV-RCNN achieves improved detection accuracy by about 14% at a high evaluation criterion (IoU 0.7) on a synthetic dataset COMAP dedicated to collective perception. Also, its performance is comparable to two raw data fusion baselines that have no data loss in sharing. Moreover, our method also significantly decreases the CPM size to less than 0.3KB, which is about 50 times smaller than the BEV feature map sharing used in previous works. Even with a further decreased number of CPM feature channels, i.e., from 128 to 32, the detection performance only drops about 1%. The code of our method is available at https://github.com/YuanYunshuang/FPV_RCNN.
AB - Sharing collective perception messages (CPM) between vehicles is investigated to decrease occlusions, so as to improve perception accuracy and safety of autonomous driving. However, highly accurate data sharing and low communication overhead is a big challenge for collective perception, especially when real-time communication is required among connected and automated vehicles. In this paper, we propose an efficient and effective keypoints-based deep feature fusion framework, called FPV-RCNN, for collective perception, which is built on top of the 3D object detector PV-RCNN. We introduce a bounding box proposal matching module and a keypoints selection strategy to compress the CPM size and solve the multi-vehicle data fusion problem. Compared to a bird's-eye view (BEV) keypoints feature fusion, FPV-RCNN achieves improved detection accuracy by about 14% at a high evaluation criterion (IoU 0.7) on a synthetic dataset COMAP dedicated to collective perception. Also, its performance is comparable to two raw data fusion baselines that have no data loss in sharing. Moreover, our method also significantly decreases the CPM size to less than 0.3KB, which is about 50 times smaller than the BEV feature map sharing used in previous works. Even with a further decreased number of CPM feature channels, i.e., from 128 to 32, the detection performance only drops about 1%. The code of our method is available at https://github.com/YuanYunshuang/FPV_RCNN.
KW - Data integration
KW - Feature extraction
KW - Location awareness
KW - Object Detection
KW - Point cloud compression
KW - Proposals
KW - Segmentation and Categorization
KW - Sensor Fusion
KW - Sensor Networks
KW - Three-dimensional displays
KW - Vehicle detection
KW - Sensor fusion
KW - object detection
KW - sensor networks
KW - segmentation and categorization
UR - http://www.scopus.com/inward/record.url?scp=85123314530&partnerID=8YFLogxK
U2 - 10.1109/LRA.2022.3143299
DO - 10.1109/LRA.2022.3143299
M3 - Article
VL - 7
SP - 3054
EP - 3061
JO - IEEE Robotics and Automation Letters
JF - IEEE Robotics and Automation Letters
SN - 2377-3766
IS - 2
ER -