Pose Estimation and Collision Detection in Human-robot Coexistence
HUANG Yanjiang1,2, WANG Ziqin1,2, ZHANG Xianmin1,2, WU Yanbin1,2
1. School of Mechanical and Automotive Engineering, South China University of Technology, Guangzhou 510640, China; 2. Guangdong Provincial Key Laboratory of Precision Equipment and Manufacturing Technology,South China University of Technology, Guangzhou 510640, China
Abstract：A method of pose estimation and collision detection in human-robot coexistence is proposed. Firstly, an optical three-dimensional motion capture system is used to obtain the pose information of mark points to establish a kinematic model of the human arm. Secondly, in order to solve the problem that the pose information of some mark points is lost due to obstacles in the workspace, the elbow joint angle obtained by the angle sensor is used as the input of the human arm kinematics model to obtain the pose information of the end of the human arm. Thirdly, the capsule model of the human arm and the collaborative robot is established, and the shortest distance among the capsules is calculated, so as to judge the relative pose relationship between the human and the machine and realize the collision detection. Finally, 10 persons are used to evaluate the human-machine pose estimation and collision detection methods in different human-machine coexistence scenarios. The experimental results show that the error of the end position of the human arm estimated by the proposed method is within 20mm, and the maximum error of the shortest distance between the human and the machine is 14.53mm, which is sufficient for man-machine collision detection.
[1] Sheikholeslami S, Moon A, Croft E A. Cooperative gestures for industry: Exploring the efficacy of robot hand configurations in expression of instructional gestures for human-robot interaction[J]. International Journal of Robotics Research, 2017, 36(5-7): 699-720. [2] Portugal D, Alvito P, Christodoulou E, et al. A study on the deployment of a service robot in an elderly care center[J]. International Journal of Social Robotics, 2019, 11: 317-341. [3] de Stefano M, Mishra H, Balachandran R, et al. Multi-rate tracking control for a space robot on a controlled satellite: A passivity-based strategy[J]. IEEE Robotics and Automation Letters, 2019, 4(2): 1319-1326. [4] 贾计东，张明路． 人机安全交互技术研究进展及发展趋势[J]．机械工程学报， 2020， 56(3)： 16-30.Jia J D, Zhang M L. Research progress and development trend of human-machine safe interaction technology[J]. Journal of Mechanical Engineering, 2020, 56(3): 16-30. [5] Ersen M, Oztop E, Sariel S. Cognition-enabled robot manipulation in human environments: Requirements, recent work, and open problems[J]. IEEE Robotics & Automation Magazine, 2017, 24(3): 108-122. [6] Amorim A, Guimares D, Mendona T, et al. Robust human position estimation in cooperative robotic cells[J]. Robotics and Computer-Integrated Manufacturing, 2021, 67. DOI: 10.1016/ j.rcim.2020.102035. [7] 王天然．机器人技术的发展[J]．机器人，2017，39(4)：385-386. Wang T R. Development of the robotics[J]. Robot, 2017, 39(4): 385-386. [8] Saenz J, Behrens R, Schulenburg E, et al. Methods for conside- ring safety in design of robotics applications featuring human-robot collaboration[J]. The International Journal of Advanced Manufacturing Technology, 2020, 107: 2313-2331. [9] Safeea M, Neto P. Minimum distance calculation using laser scanner and IMUs for safe human-robot interaction[J]. Robotics and Computer-Integrated Manufacturing, 2019, 58: 33-42. [10] Safeea M, Neto P, Bearee R. On-line collision avoidance for collaborative robot manipulators by adjusting off-line generated paths: An industrial use case[J]. Robotics and Autonomous Systems, 2019, 119: 278-288. [11] Ordóñez F J, Roggen D. Deep convolutional and LSTM recurrent neural networks for multimodal wearable activity recognition[J]. Sensors, 2016, 16(1). DOI: 10.3390/s16010115. [12] Wolf M T, Assad C, Vernacchia M T, et al. Gesture-based robot control with variable autonomy from the JPL BioSleeve[C]// IEEE International Conference on Robotics and Automation. Piscataway, USA: IEEE, 2013: 1160-1165. [13] 陈锴彬．基于表面肌电信号的人体手臂关节运动信息预估及人机顺应协作研究[D]．广州：华南理工大学，2020.Chen K B. Research on the prediction of human arm joint motion information based on surface EMG signal and human-machine compliance collaboration[D]. Guangzhou: South China University of Technology, 2020. [14] Huang Y J, Chen K B, Zhang X M, et al. Joint torque estimation for the human arm from sEMG using backpropagation neural networks and autoencoders[J]. Biomedical Signal Processing and Control, 2020, 62. DOI: 10.1016/j.bspc.2020.102051. [15] 丁其川，赵新刚，韩建达．基于肌电信号的上肢多关节连续运动估计[J]．机器人，2014，36(4)：469-476.Ding Q C, Zhao X G, Han J D. EMG-based estimation for multi-joint continuous movement of human upper limb[J]. Robot, 2014, 36(4): 469-476. [16] Kousi N, Stoubos C, Gkournelos C, et al. Enabling human robot interaction in flexible robotic assembly lines: An augmented reality based software suite[J]. Procedia CIRP, 2019, 81: 1429-1434. [17] Magrini E, Ferraguti F, Ronga A J, et al. Human-robot coexistence and interaction in open industrial cells[J]. Robotics and Computer-Integrated Manufacturing, 2020, 61. DOI: 10. 1016/j.rcim.2019.101846. [18] Casalino A, Guzman S, Zanchettin A M, et al. Human pose estimation in presence of occlusion using depth camera sensors, in human-robot coexistence scenarios[C]//IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway, USA: IEEE, 2018: 1-7. [19] 冯柯，陈临强．户外环境下抗遮挡的运动目标跟踪方法[J]．计算机工程与应用，2011，47(11)：187-189,200. Feng K, Chen L Q. Anti-occlusion moving object tracking method in outdoor environment[J]. Computer Engineering and Applications, 2011, 47(11): 187-189,200. [20] Makris A, Kyriazis N, Argyros A A. Hierarchical particle filtering for 3D hand tracking[C]//IEEE Conference on Computer Vision and Pattern Recognition Workshops. Piscataway, USA: IEEE, 2015: 8-17. [21] John V, Trucco E, Ivekovic S. Markerless human articulated tracking using hierarchical particle swarm optimization[J]. Image and Vision Computing, 2010, 28(11): 1530-1547. [22] Balan L, Bone G M. Real-time 3D collision avoidance method for safe human and robot coexistence[C]//IEEE/RSJ International Conference on Intelligent Rbots and Systems. Piscataway, USA: IEEE, 2006: 276-282. [23] Liu C L, Tomizuka M. Algorithmic safety measures for intelligent industrial co-robots[C]//IEEE International Conference on Robotics and Automation. Piscataway, USA: IEEE, 2016: 3095-3102. [24] Safeea M, Neto P, Bearee R. Efficient calculation of minimum distance between capsules and its use in robotics[J]. IEEE Access, 2018, 7: 5368-5373. [25] Fang C, Rocchi A, Hoffman E M, et al. Efficient self-collision avoidance based on focus of interest for humanoid robots[C]// IEEE-RAS 15th International Conference on Humanoid Ro-bots. Piscataway, USA: IEEE, 2015: 1060-1066. [26] Ouyang F, Zhang T. Virtual velocity vector-based offline collision-free path planning of industrial robotic manipulator[J]. International Journal of Advanced Robotic Systems, 2015, 12(9). DOI: 10.5772/60127. [27] Kuhn S, Henrich D. Fast vision-based minimum distance determination between known and unkown objects[C]//IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway, USA: IEEE, 2007: 2186-2191. [28] Afaghani A Y, Aiyama Y. On-line collision avoidance between two robot manipulators using collision map and simple escaping method[C]//IEEE/SICE International Symposium on System Integration. Piscataway, USA: IEEE, 2013: 105-110. [29] Fabrizio F, Luca A D. Real-time computation of distance to dynamic obstacles with multiple depth sensors[J]. IEEE Robotics and Automation Letters, 2017, 2(1): 56-63. [30] Kaldestad K B, Haddadin S, Belder R, et al. Collision avoidance with potential fields based on parallel processing of 3D-point cloud data on the GPU[C]//IEEE International Conference on Robotics and Automation. Piscataway, USA: IEEE, 2014: 3250-3257. [31] 杨明辉．基于循环神经网络的运动目标轨迹预测[D]．武汉：武汉大学，2019. Yang M H. Prediction of moving target trajectory based on recurrent neural network[D]. Wuhan: Wuhan University, 2019. [32] NOKOV．几种智能机器人室内定位方法对比[EB/OL]．(2013-02-28) [2021-06-30]．https://www.nokov.com/support/blogs_detail/Comparison-of-several-indoor-positioning-methods-for-robots.html.NOKOV. Comparison of several indoor positioning methods of intelligent-robots[EB/OL]. (2013-02-28) [2021-06-30].https://www.nokov.com/support/blogs_detail/Comparison-of-several-indoor-positioning-methods-for-robots.html.
