An Autonomous Surgical Instrument Tracking Framework With a Binocular Camera for a Robotic Flexible Laparoscope
Publication in refereed journal
CUHK Authors
Full Text
Digital Object Identifier (DOI) DOI for CUHK Users |
Altmetrics Information
.
Other information
AbstractIn minimally invasive surgery (MIS), the field of view (FOV) plays a vital role. To enhance the stability of FOV and lighten the burden on surgeons, robot-assisted laparoscope systems have been developed and introduced into surgery. However, most of the existing automatic surgical tool tracking schemes with robotic laparoscopes either do not consider tracking of the depth direction or the remote center of motion (RCM) constraints. To tackle these problems, this article presents an autonomous three-dimensional surgical instrument tracking framework for a robotic laparoscope based on a binocular camera. A 3-degree-of-freedom visual tracking scheme is derived from the kinematics of the binocular robotic laparoscope. A programmable RCM constraint is introduced due to the MIS requirement. To precisely locate the surgical instruments in FOV, a surgical instrument tip segmentation dataset with 5471 images is built and a modified real-time instance segmentation network is trained based on the dataset. Then, a semi-global matching (SGM) algorithm is utilized with the binocular camera to estimate the depth in FOV. To meet the joint limit constraint and maintain the dexterity of the robotic system, a task-priority-based control method is adopted. The simulation and experiment results demonstrate the feasibility of the proposed framework for instrument tracking with the robotic laparoscope.
All Author(s) ListLi J, Huang YS, Zhang X, Xie K, Xian YT, Luo X, Chiu PWY, Li Z
Journal nameIEEE Robotics and Automation Letters
Year2023
Month7
Volume Number8
Issue Number7
PublisherIEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
Pages4291 - 4298
ISSN2377-3766
LanguagesEnglish-United Kingdom
KeywordsInstruments, Laparoscopes, Robots, Three-dimensional displays, Cameras, Surgery, Visualization, Robotic laparoscopy, binocular visual sensing, Index Terms, robot control, visual servoing, computer vision for surgical robotics
Web of Science Subject CategoriesRobotics;Robotics
