Human-Machine Interaction Control of the Capsule Robot withMultiple Modes Based on Magnetic and Visual Harmonization
ZHANG Yongshun, YANG Huiyuan
Key Laboratory for Precision & Non-traditional Machining of Ministry of Education, School of Mechanical Engineering, Dalian University of Technology, Dalian 116024, China
张永顺, 杨慧远. 磁场与视觉共融的多模态胶囊机器人人机交互控制[J]. 机器人, 2018, 40(1): 72-80.DOI: 10.13973/j.cnki.robot.170166.
ZHANG Yongshun, YANG Huiyuan. Human-Machine Interaction Control of the Capsule Robot withMultiple Modes Based on Magnetic and Visual Harmonization. ROBOT, 2018, 40(1): 72-80. DOI: 10.13973/j.cnki.robot.170166.
Abstract:In order to achieve the panoramic observation based on the suspended posture adjustment of the capsule endoscopy in an appointed position by magnetic actuation, an underactuated dual hemisphere capsule robot (DHCR) is proposed, which achieves the key breakthrough in the modal conversion between the suspended posture adjustment and the rolling locomotion. To realize the rolling locomotion in the bending gastrointestinal (GI) tract, a human-machine interaction control strategy integrating the vision and the spatial universal rotating magnetic field (SURMF) is put forwarded. Theoretically, the SURMF superposition formula in form of three phase currents using the azimuth and altitude angles of the SURMF axis as independent variables, is derived in the latitude and longitude coordinate systems according to the principle of the orthogonal transformation, achieving the dimension reduction and the decoupling of control variables. Furthermore, the individual adjustment of the SURMF axis along azimuth direction or altitude direction can be implemented respectively. Practically, the camera of the DHCR can scan along azimuth or altitude directions respectively by the magnetic torque servo effect until the DHCR axis is aligned with the bending direction in each section of the GI tract, and steering along each bending direction can be realized by applying the relevant rolling magnetic moment respectively. Finally, the in-vitro pig large intestine is employed as the simulation environment to verify the man-machine interactive performance. The experiments show that the panoramic observation based on the arbitrary posture adjustment under the passive mode, along with the rolling locomotion in the curved environment under the active mode, can be achieved by the proposed DHCR with the human-machine interaction combining the image of the curved GI tract and the SURMF orientation.
[1] Moglia A, Menciassi A, Schurr M O, et al. Wireless capsule endoscopy:From diagnostic devices to multipurpose robotic systems[J]. Biomedical Microdevices, 2007, 9(2):235-243.
[2] Lander E S, Weinberg R A. Genomics:Journey to the center of biology[J]. Science, 2000, 287(5459):1777-1782.
[3] Meron G D. The development of the swallowable video capsule (M2A)[J]. Gastrointestinal Endoscopy, 2000, 52(6):817-819.
[4] 倪自强,王田苗,刘达.医疗机器人技术发展综述[J].机械工程学报,2015,51(13):45-52.Ni Z Q, Wang T M, Liu D. Survey on medical robotics[J]. Journal of Mechanical Engineering, 2015, 51(13):45-52.
[5] Zhu B Q, Yan G Z, Liu G, et al. A wireless video capsule endoscopy system:Design and realization[J]. Journal of Shanghai Jiaotong University:Science, 2015, 20(6):649-653.
[6] Li Z Y, Ren B B, Tan H Y, et al. Capsule design for blue light therapy against helicobacter pylori[J]. PLOS ONE, 2016, 11(1):No. e0147531.
[7] Wang J S, Wang Z, Leach M, et al. RF characteristics of wireless capsule endoscopy in human body[J]. Journal of Central South University, 2016, 23(5):1198-1207.
[8] 张永顺,迟明路,程存欣,等.一种高性能花瓣廓形胶囊机器人[J].机械工程学报,2017,53(3):9-16.Zhang Y S, Chi M L, Cheng C X, et al. Petal-shaped capsule robot with high performance[J]. Journal of Mechanical Engineering, 2017, 53(3):9-16.
[9] Carpi F, Kastelein N, Talcott M, et al. Magnetically controllable gastrointestinal steering of video capsules[J]. IEEE Transac-tions on Biomedical Engineering, 2011, 58(2):231-234.
[10] Yim S, Gultepe E, Gracias D H, et al. Biopsy using a magnetic capsule endoscope carrying, releasing, and retrieving untethered microgrippers[J]. IEEE Transactions on Biomedical Engineering, 2014, 61(2):513-521.
[11] 张永顺,孙颖,杜春雨,等.胶囊机器人弯曲环境通过性与磁矢量控制[J].机械工程学报,2014,50(5):26-32.Zhang Y S, Sun Y, Du C Y, et al. Trafficability characteristic and magnetic vector control of a capsule robot in bending environment[J]. Journal of Mechanical Engineering, 2014, 50(5):26-32.
[12] Park H, Park S, Yoon E, et al. Paddling based microrobot for capsule endoscopes[C]//IEEE International Conference on Robotics and Automation. Piscataway, USA:IEEE, 2007:3377-3382.
[13] Tortora G, Valdastri P, Susilo E, et al. Propeller-based wireless device for active capsular endoscopy in the gastric district[J]. Minimally Invasive Therapy & Allied Technologies, 2009, 18(5):280-290.
[14] Woods S P, Constandinou T G. Wireless capsule endoscope for targeted drug delivery:Mechanics and design considerations[J]. IEEE Transactions on Biomedical Engineering, 2013, 60(4):945-953.
[15] Temel F Z, Erman A G, Yesilyurt S. Characterization and modeling of biomimetic untethered robots swimming in viscous fluids inside circular channels[J]. IEEE/ASME Transactions on Mechatronics, 2013, 19(5):1562-1573.
[16] 张永顺,苏忠侃,杨振强,等.一种主被动双半球形胶囊机器人及其姿态调整与转弯驱动控制方法:中国,CN104983385A[P]. 2015-10-21.Zhang Y S, Su Z K, Yang Z Q, et al. An active/passive dual hemisphere capsule robot and its control strategy for posture adjustment and steering driving:China, CN104983385A[P]. 2015-10-21.
[17] 张永顺,周华涛,张林霞,等.一种新型双半球形胶囊机器人[J].机械工程学报,2017,53(15):110-118.Zhang Y S, Zhou H T, Zhang L X, et al. A new kind of dual hemisphere capsule robot[J]. Journal of Mechanical Engineering, 2017, 53(15):110-118.
[18] Zhang Y S, Yu Z C, Yang H Y, et al. Orthogonal transformation operation theorem of a spatial universal uniform rotating magnetic field and its application in capsule endoscopy[J]. Science China:Technological Sciences, 2017, 60(6):1-11.
[19] 赵荧.基于视觉的胶囊内窥镜导航研究[D].哈尔滨:哈尔滨工业大学,2013.Zhao Y. Vision based navigation for capsule endoscopies[D]. Harbin:Harbin Institute of Technology, 2013.
[20] Asari V K, Kumar S, Kassim I M. A fully autonomous microrobotic endoscopy system[J]. Journal of Intelligent & Robotic Systems, 2000, 28(4):325-341.
[21] 耿明志.无人机自动着陆过程中的视觉导航技术研究[D].南京:南京航空航天大学,2007.Geng Z M. The application of visual navigation in auto-landing of unmanned aerial vehicle[D]. Nanjing:Nanjing University of Aeronautics and Astronautics, 2007.