TY - JOUR

T1 - Deep-learning-based instrument detection for intra-operative robotic assistance

AU - Badilla Solórzano, Jorge

AU - Spindeldreier, Svenja

AU - Ihler, Sontje

AU - Gellrich, Nils-Claudius

AU - Spalthoff, Simon

N1 - Funding Information: The main author wants to offer his gratitude to the University of Costa Rica and the German Academic Exchange Service (DAAD) for providing financial support, enabling the completion of the hereby presented research.

PY - 2022/9

Y1 - 2022/9

N2 - Purpose:: Robotic scrub nurses have the potential to become an attractive solution for the operating room. Surgical instrument detection is a fundamental task for these systems, which is the focus of this work. We address the detection of the complete surgery set for wisdom teeth extraction, and propose a data augmentation technique tailored for this task. Methods:: Using a robotic scrub nurse system, we create a dataset of 369 unique multi-instrument images with manual annotations. We then propose the Mask-Based Object Insertion method, capable of automatically generating a large amount of synthetic images. By using both real and artificial data, different Mask R-CNN models are trained and evaluated. Results:: Our experiments reveal that models trained on the synthetic data created with our method achieve comparable performance to that of models trained on real images. Moreover, we demonstrate that the combination of real and our artificial data can lead to a superior level of generalization. Conclusion:: The proposed data augmentation technique is capable of dramatically reducing the labelling work required for training a deep-learning-based detection algorithm. A dataset for the complete instrument set for wisdom teeth extraction is made available for the scientific community, as well as the raw information required for the generation of the synthetic data (https://github.com/Jorebs/Deep-learning-based-instrument-detection-for-intra operative-robotic-assistance).

AB - Purpose:: Robotic scrub nurses have the potential to become an attractive solution for the operating room. Surgical instrument detection is a fundamental task for these systems, which is the focus of this work. We address the detection of the complete surgery set for wisdom teeth extraction, and propose a data augmentation technique tailored for this task. Methods:: Using a robotic scrub nurse system, we create a dataset of 369 unique multi-instrument images with manual annotations. We then propose the Mask-Based Object Insertion method, capable of automatically generating a large amount of synthetic images. By using both real and artificial data, different Mask R-CNN models are trained and evaluated. Results:: Our experiments reveal that models trained on the synthetic data created with our method achieve comparable performance to that of models trained on real images. Moreover, we demonstrate that the combination of real and our artificial data can lead to a superior level of generalization. Conclusion:: The proposed data augmentation technique is capable of dramatically reducing the labelling work required for training a deep-learning-based detection algorithm. A dataset for the complete instrument set for wisdom teeth extraction is made available for the scientific community, as well as the raw information required for the generation of the synthetic data (https://github.com/Jorebs/Deep-learning-based-instrument-detection-for-intra operative-robotic-assistance).

KW - Data augmentation

KW - Dataset

KW - Mask R-CNN

KW - Mask-based object insertion

KW - Robot-assisted surgery

KW - Robotic scrub nurse

UR - http://www.scopus.com/inward/record.url?scp=85137956787&partnerID=8YFLogxK

U2 - 10.1007/s11548-022-02715-y

DO - 10.1007/s11548-022-02715-y

M3 - Article

C2 - 35896914

AN - SCOPUS:85137956787

VL - 17

SP - 1685

EP - 1695

JO - International journal of computer assisted radiology and surgery

JF - International journal of computer assisted radiology and surgery

SN - 1861-6410

IS - 9

ER -