Details
| Original language | English |
|---|---|
| Pages (from-to) | 1685-1695 |
| Number of pages | 11 |
| Journal | International journal of computer assisted radiology and surgery |
| Volume | 17 |
| Issue number | 9 |
| Early online date | 28 Jul 2022 |
| Publication status | Published - Sept 2022 |
Abstract
Purpose:: Robotic scrub nurses have the potential to become an attractive solution for the operating room. Surgical instrument detection is a fundamental task for these systems, which is the focus of this work. We address the detection of the complete surgery set for wisdom teeth extraction, and propose a data augmentation technique tailored for this task. Methods:: Using a robotic scrub nurse system, we create a dataset of 369 unique multi-instrument images with manual annotations. We then propose the Mask-Based Object Insertion method, capable of automatically generating a large amount of synthetic images. By using both real and artificial data, different Mask R-CNN models are trained and evaluated. Results:: Our experiments reveal that models trained on the synthetic data created with our method achieve comparable performance to that of models trained on real images. Moreover, we demonstrate that the combination of real and our artificial data can lead to a superior level of generalization. Conclusion:: The proposed data augmentation technique is capable of dramatically reducing the labelling work required for training a deep-learning-based detection algorithm. A dataset for the complete instrument set for wisdom teeth extraction is made available for the scientific community, as well as the raw information required for the generation of the synthetic data (https://github.com/Jorebs/Deep-learning-based-instrument-detection-for-intra operative-robotic-assistance).
Keywords
- Data augmentation, Dataset, Mask R-CNN, Mask-based object insertion, Robot-assisted surgery, Robotic scrub nurse
ASJC Scopus subject areas
- Medicine(all)
- Surgery
- Engineering(all)
- Biomedical Engineering
- Medicine(all)
- Radiology Nuclear Medicine and imaging
- Computer Science(all)
- Computer Vision and Pattern Recognition
- Computer Science(all)
- Computer Science Applications
- Medicine(all)
- Health Informatics
- Computer Science(all)
- Computer Graphics and Computer-Aided Design
Cite this
- Standard
- Harvard
- Apa
- Vancouver
- BibTeX
- RIS
In: International journal of computer assisted radiology and surgery, Vol. 17, No. 9, 09.2022, p. 1685-1695.
Research output: Contribution to journal › Article › Research › peer review
}
TY - JOUR
T1 - Deep-learning-based instrument detection for intra-operative robotic assistance
AU - Badilla Solórzano, Jorge
AU - Spindeldreier, Svenja
AU - Ihler, Sontje
AU - Gellrich, Nils-Claudius
AU - Spalthoff, Simon
N1 - Funding Information: The main author wants to offer his gratitude to the University of Costa Rica and the German Academic Exchange Service (DAAD) for providing financial support, enabling the completion of the hereby presented research.
PY - 2022/9
Y1 - 2022/9
N2 - Purpose:: Robotic scrub nurses have the potential to become an attractive solution for the operating room. Surgical instrument detection is a fundamental task for these systems, which is the focus of this work. We address the detection of the complete surgery set for wisdom teeth extraction, and propose a data augmentation technique tailored for this task. Methods:: Using a robotic scrub nurse system, we create a dataset of 369 unique multi-instrument images with manual annotations. We then propose the Mask-Based Object Insertion method, capable of automatically generating a large amount of synthetic images. By using both real and artificial data, different Mask R-CNN models are trained and evaluated. Results:: Our experiments reveal that models trained on the synthetic data created with our method achieve comparable performance to that of models trained on real images. Moreover, we demonstrate that the combination of real and our artificial data can lead to a superior level of generalization. Conclusion:: The proposed data augmentation technique is capable of dramatically reducing the labelling work required for training a deep-learning-based detection algorithm. A dataset for the complete instrument set for wisdom teeth extraction is made available for the scientific community, as well as the raw information required for the generation of the synthetic data (https://github.com/Jorebs/Deep-learning-based-instrument-detection-for-intra operative-robotic-assistance).
AB - Purpose:: Robotic scrub nurses have the potential to become an attractive solution for the operating room. Surgical instrument detection is a fundamental task for these systems, which is the focus of this work. We address the detection of the complete surgery set for wisdom teeth extraction, and propose a data augmentation technique tailored for this task. Methods:: Using a robotic scrub nurse system, we create a dataset of 369 unique multi-instrument images with manual annotations. We then propose the Mask-Based Object Insertion method, capable of automatically generating a large amount of synthetic images. By using both real and artificial data, different Mask R-CNN models are trained and evaluated. Results:: Our experiments reveal that models trained on the synthetic data created with our method achieve comparable performance to that of models trained on real images. Moreover, we demonstrate that the combination of real and our artificial data can lead to a superior level of generalization. Conclusion:: The proposed data augmentation technique is capable of dramatically reducing the labelling work required for training a deep-learning-based detection algorithm. A dataset for the complete instrument set for wisdom teeth extraction is made available for the scientific community, as well as the raw information required for the generation of the synthetic data (https://github.com/Jorebs/Deep-learning-based-instrument-detection-for-intra operative-robotic-assistance).
KW - Data augmentation
KW - Dataset
KW - Mask R-CNN
KW - Mask-based object insertion
KW - Robot-assisted surgery
KW - Robotic scrub nurse
UR - http://www.scopus.com/inward/record.url?scp=85137956787&partnerID=8YFLogxK
U2 - 10.1007/s11548-022-02715-y
DO - 10.1007/s11548-022-02715-y
M3 - Article
C2 - 35896914
AN - SCOPUS:85137956787
VL - 17
SP - 1685
EP - 1695
JO - International journal of computer assisted radiology and surgery
JF - International journal of computer assisted radiology and surgery
SN - 1861-6410
IS - 9
ER -