Speech-Vision Based Multi-Modal AI Control of a Magnetic Anchored and Actuated Endoscope
Refereed conference paper presented and published in conference proceedings

替代計量分析
.

其它資訊
摘要In minimally invasive surgery (MIS), controlling the endoscope view is crucial for the operation. Many robotic endoscope holders were developed aiming to address this prob-lem,. These systems rely on joystick, foot pedal, simple voice command, etc. to control the robot. These methods requires surgeons extra effort and are not intuitive enough. In this paper, we propose a speech-vision based multi-modal AI approach, which integrates deep learning based instrument detection, automatic speech recognition and robot visual servo control. Surgeons could communicate with the endoscope by speech to indicate their view preference, such as the instrument to be tracked. The instrument is detected by the deep learning neural network. Then the endoscope takes the detected instrument as the target and follows it with the visual servo controller. This method is applied to a magnetic anchored and guided endoscope and evaluated experimentally. Preliminary results demonstrated this approach is effective and requires little efforts for the surgeon to control the endoscope view intuitively.
著者Jixiu Li, Yisen Huang, Wing Yin Ng, Truman Cheng, Xixin Wu, Qi Dou, Helen Meng, Pheng Ann Heng, Yunhui Liu, Shannon Melissa Chan, David Navarro-Alarcon, Calvin Sze Hang Ng, Philip Wai Yan Chiu, Zheng Li
會議名稱2022 IEEE International Conference on Robotics and Biomimetics (ROBIO)
會議開始日05.12.2022
會議完結日09.12.2022
會議地點Xishuangbanna
會議國家/地區中國
出版年份2022
出版社IEEE
語言英式英語

上次更新時間 2023-25-10 於 02:55