WANG Yinghua1, SONG Guangming1, LIU Shengsong2, LIAN Jie1, SUN Huiyu1, SONG Aiguo1
1. Jiangsu Key Lab of Remote Measurement and Control, State Key Laboratory of Bioelectronics, School of Instrument Science and Engineering, Southeast University, Nanjing 210096, China;
2. Power Dispatching Control Center of Jiangsu Electric Power Company, Nanjing 210024, China
Abstract：Aiming at the difficulty of relative positioning between the aerial robot and the grabbed target when the aerial manipulator system performs the tasks of grabbing, dropping, etc., a design method for vision-guided aerial manipulators is presented. Firstly, the overall mechanism design of the aerial manipulator system is introduced, and kinematics and dynamics models are developed for the aircraft and the aerial operation device. Secondly, the on-board monocular camera is utilized to detect the ArUco markers on the grabbed target according to the pinhole imaging model on the premise that the marker size is known, and the perspective-n-point (PnP) algorithm is used to calculate the camera pose. Then the flying robot and the operation device are controlled hierarchically based on the camera pose information. Finally, the effectiveness of the pose estimation algorithm is validated by the stationary and outdoor hover experiments, and the effectiveness and rationality of visual guidance are further verified by autonomously grabbing a tubular object with a diameter of 2 cm and mass of 100 g.
 杨斌,何玉庆,韩建达,等.作业型飞行机器人研究现状与展望[J].机器人, 2015, 37(5):628-640. Yang B, He Y Q, Han J D, et al. Survey on aerial manipulator systems[J]. Robot, 2015, 37(5):628-640.
 Thomas J, Loianno G, Daniilidis K, et al. The role of vision in perching and grasping for MAVs[C]//Micro-and Nanotechnology Sensors, Systems, and Applications VⅢ. Bellingham, USA:SPIE, 2016:No.98361S.
 Bisgaard M, la Cour-Harbo A, Bendtsen J D. Adaptive control system for autonomous helicopter slung load operations[J]. Control Engineering Practice, 2010, 18(7):800-811.
 Lindsey Q, Mellinger D, Kumar V. Construction of cubic structures with quadrotor teams[C]//Robotics:Science and Systems VⅡ. 2011. DOI:10.15607/RSS.2011.VⅡ.025.
 Mellinger D, Lindsey Q, Shomin M, et al. Design, modeling, estimation and control for aerial grasping and manipulation[C]//IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway, USA:IEEE, 2011:2668-2673.
 Pounds P E I, Bersak D R, Dollar A M. Grasping from the air:Hovering capture and load stability[C]//IEEE International Conference on Robotics and Automation. Piscataway, USA:IEEE, 2011:2491-2498.
 Thomas J, Loianno G, Sreenath K, et al. Toward image based visual servoing for aerial grasping and perching[C]//IEEE International Conference on Robotics and Automation. Piscataway, USA:IEEE, 2014:2113-2118.
 Kim S, Seo H, Choi S, et al. Vision-guided aerial manipulation using a multirotor with a robotic arm[J]. IEEE/ASME Transactions on Mechatronics, 2016, 21(4):1912-1923.
 Kim S, Seo H, Kim H J. Operating an unknown drawer using an aerial manipulator[C]//IEEE International Conference on Robotics and Automation. Piscataway, USA:IEEE, 2015:5503-5508.
 Seo H, Kim S, Kim H J. Aerial grasping of cylindrical object using visual servoing based on stochastic model predictive control[C]//IEEE International Conference on Robotics and Automation. Piscataway, USA:IEEE, 2017:6362-6368.
 钟杭,王耀南,李玲,等.旋翼飞行机械臂建模及动态重心补偿控制[J].控制理论与应用, 2016, 33(3):311-320. Zhong H, Wang Y N, Li L, et al. Rotor-flying manipulator modeling and control with dynamic compensation for gravity offset[J]. Control Theory & Applications, 2016, 33(3):311-320.
 宋大雷,孟祥冬,齐俊桐,等. 3自由度旋翼飞行机械臂系统动力学建模与预测控制方法[J].机器人, 2015, 37(2):152-160. Song D L, Meng X D, Qi J T, et al. Strategy of dynamic modeling and predictive control on 3-DoF rotorcraft aerial manipulator system[J]. Robot, 2015, 37(2):152-160.
 Garrido-Jurado S, Munoz-Salinas R, Madrid-Cuevas F J, et al. Automatic generation and detection of highly reliable fiducial markers under occlusion[J]. Pattern Recognition, 2014, 47(6):2280-2292.
 孙富春,朱纪洪,刘国栋,等.机器人学导论——分析、控制及应用[M]. 2版.北京:电子工业出版社, 2013:27-73. Sun F C, Zhu J H, Liu G D, et al. Introduction to robotics:Analysis, control, application[M]. 2nd ed. Beijing:Publishing House of Electronics Industry, 2013:27-73.
 Mahony R, Kumar V, Corke P. Multirotor aerial vehicles:Modeling, estimation, and control of quadrotor[J]. IEEE Robotics and Automation Magazine, 2012, 19(3):20-32.
 Alexis K, Nikolakopoulos G, Tzes A. Model predictive quadrotor control:Attitude, altitude and position experimental studies[J]. IET Control Theory and Applications, 2012, 6(12):1812-1827.
 Wang P, Sun C K, Zhang Z M. Linear pose estimation with a monocular vision system[J]. Chinese Journal of Scientific Instrument, 2011, 32(5):1126-1131.
 Baggio D L, Emami S, Escriva D M, et al. Mastering OpenCV with practical computer vision projects[M]. Birmingham, UK:Packt Publishing Ltd, 2012:47-82.
 Zhang Z Y. A flexible new technique for camera calibration[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 22(11):1330-1334.
 Otsu N. A threshold selection method from gray-level histograms[J]. IEEE Transactions on Systems, Man, and Cybernetics, 1979, 9(1):62-66.