Biomechanical Interface System and Neural-like Cooperative Control for the Intelligent Prosthetic Arm
LI Jiwei1,2,3, ZHANG Bi1,2, YAO Jie1,2, ZHAO Ming1,2, XU Zhuang1,2, ZHAO Xingang1,2
1. State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang 110016, China; 2. Institute of Robotics and Intelligent Manufacturing Innovation, Chinese Academy of Sciences, Shenyang 110169, China; 3. University of Chinese Academy of Sciences, Beijing 100049, China
Abstract：To address the problem of prosthetic limb control for patients with physical disability, an sEMG (surface electromyography) based intelligent prosthetic arm system is developed to achieve coordinated hand-elbow control for patients with a higher degree of arm disability. Firstly, the non-negative matrix factorization (NMF) method is applied to extracting muscle synergy based on the muscle synergy theory, and hand movement recognition and continuous motion estimation of the elbow joint are implemented. Secondly, a ''feedforward-feedback'' control framework is constructed based on the intention recognition results, and feedforward supervision and feedback detection are performed on the subjects to improve the comfort and robustness of the prosthetic system by adjusting the desired control input based on the feedforward-feedback results. Then, an adaptive grip force adjustment framework is constructed for hand movements to achieve adaptive grip of objects of different stiffness and shapes through alternating force and position information control; for elbow movements, an impedance control algorithm based on recognition results is designed to achieve stable human-machine interaction control of the hand-elbow integrated prosthesis. Finally, the above control strategy is experimentally verified by 6 healthy subjects and an arm handicapped subject in order to achieve more accurate intention recognition for the overall arm motion, and the result indicates that the proposed approach can complete stable elbow flexion and extension as well as hand grasping function to achieve coordinated control of the integrated hand-elbow. The system was realized in the Beijing 2022 Winter Paralympic Games for application demonstration.
[1] 陈新．第二次全国残疾人抽样调查数据[J]．中国生育健康 杂志， 2008， 19(2)： 68. Chen X. Data of the second national sample survey of disabled persons[J]. Chinese Journal of Reproductive Health, 2008, 19(2): 68. [2] 赵芳杰．北京市肢体残疾人现状及假肢的需求分析[D]．长 春：吉林大学， 2016. Zhao F Z. The current status of physical disability and needs of prosthesis in Beijing[D]. Changchun: Jilin University, 2016. [3] Cordella F, Ciancio A L, Sacchetti R, et al. Literature review on needs of upper limb prosthesis users[J]. Frontiers in Neuroscience, 2016, 10. DOI: 10.3389/fnins.2016.00209. [4] Liu H, Meusel P, Hirzinger G, et al. The modular multisensory DLR-HIT-Hand: Hardware and software architecture[J]. IEEE/ASME Transactions on Mechatronics, 2008, 13(4): 461- 469. [5] van der Niet O O, Reinders-Messelink H A, Bongers R M, et al. The i-LIMB hand and the DMC plus hand compared: A case report[J]. Prosthetics and Orthotics International, 2010, 34(2): 216-220. [6] Krausz N E, Rorrer R A L, Weir R F. Design and fabrication of a six degree-of-freedom open source hand[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2016, 24(5): 562-572. [7] Smit G, Plettenburg D H, van der Helm F C T. The lightweight Delft Cylinder Hand: First multi-articulating hand that meets the basic user requirements[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2015, 23(3): 431-440. [8] Gu G Y, Zhang N B, Xu H P, et al. A soft neuroprosthetic hand providing simultaneous myoelectric control and tactile feedback[J]. Nature Biomedical Engineering, 2021. DOI: 10.1038/ s41551-021-00767-0. [9] Artemiadis P K, Kyriakopoulos K J. An EMG-based robot control scheme robust to time-varying EMG signal features[J]. IEEE Transactions on Information Technology in Biomedicine, 2010, 14(3): 582-588. [10] 丁其川，熊安斌，赵新刚，等．基于表面肌电的运动意图 识别方法研究及应用综述[J]．自动化学报， 2016， 42(1)： 13-25. Ding Q C, Xiong A B, Zhao X G, et al. A review on researches and applications of sEMG-based motion intent recognition methods[J]. Acta Automatica Sinica, 2016, 42(1): 13-25. [11] Englehart K, Hudgin B, Parker P A. A wavelet-based continuous classification scheme for multifunction myoelectric control [J]. IEEE Transactions on Biomedical Engineering, 2001, 48(3): 302-311. [12] Lloyd E, Jiang N. Convolution neural network for EMG-based finger gesture classification for novel and trained gestures[C]// IEEE International Conference on Systems, Man and Cybernetics. Piscataway, USA: IEEE, 2019: 3724-3728. [13] Zhang F, Li P F, Hou Z G, et al. sEMG-based continuous estimation of joint angles of human legs by using BP neural network[J]. Neurocomputing, 2012, 78(1): 139-148. [14] Han J D, Ding Q C, Xiong A B, et al. A state-space EMG model for the estimation of continuous joint movements[J]. IEEE Transactions on Industrial Electronics, 2015, 62(7): 4267-4275. [15] 李自由，赵新刚，张弼，等．基于表面肌电的意图识别 方法在非理想条件下的研究进展[J]．自动化学报， 2021， 47(5)： 955-969. Li Z Y, Zhao X G, Zhang B, et al. Review of sEMG-based motion intent recognition methods in non-ideal conditions[J]. Acta Automatica Sinica, 2021, 47(5): 955-969. [16] Ting L H, McKay J L. Neuromechanics of muscle synergies for posture and movement[J]. Current Opinion in Neurobiology, 2007, 17(6): 622-628. [17] d’Avella A, Saltiel P, Bizzi E. Combinations of muscle synergies in the construction of a natural motor behavior[J]. Nature Neuroscience, 2003, 6(3): 300-308. [18] Jiang N, Vujaklija I, Rehbaum H, et al. Is accurate mapping of EMG signals on kinematics needed for precise online myoelectric control?[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2014, 22(3): 549-558. [19] Luo X Y, Wu X Y, Chen L, et al. Synergistic myoelectrical activities of forearm muscles improving robust recognition of multi-fingered gestures[J]. Sensors, 2019, 19(3). DOI: 10.3390/ s19030610. [20] Antuvan C W, Bisio F, Marini F, et al. Role of muscle synergies in real-time classification of upper limb motions using extreme learning machines[J]. Journal of NeuroEngineering and Rehabilitation, 2016, 13. DOI: 10.1186/s12984-016-0183-0. [21] Liu Y X, Wang R L, Gutierrez-Farewik E M. A muscle synergyinspired method of detecting human movement intentions based on wearable sensor fusion[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2021, 29: 1089-1098. [22] D’Anna E, Valle G, Mazzoni A, et al. A closed-loop hand prosthesis with simultaneous intraneural tactile and position feedback[J]. Science Robotics, 2019, 4(27). DOI: 10.1126/sci robotics.aau8892. [23] Zollo L, di Pino G, Ciancio A L, et al. Restoring tactile sensations via neural interfaces for real-time force-and-slippage closed-loop control of bionic hands[J]. Science Robotics, 2019, 4(27). DOI: 10.1126/scirobotics.aau992. [24] Furui A, Eto S, Nakagaki K, et al. A myoelectric prosthetic hand with muscle synergy-based motion determination and impedance model-based biomimetic control[J]. Science Robotics, 2019, 4(31). DOI: 10.1126/scirobotics.aaw6339. [25] Lee D D, Seung H S. Learning the parts of objects by nonnegative matrix factorization[J]. Nature, 1999, 401(6755): 788- 791. [26] Clark D J, Ting L H, Zajac F E, et al. Merging of healthy motor modules predicts reduced locomotor performance and muscle coordination complexity post-stroke[J]. Journal of Neurophysiology, 2010, 103(2): 844-857. [27] Breiman L. Random forests[J]. Machine Learning, 2001, 45(1): 5-32. [28] Rumelhart D E, Hinton G E, Williams R J. Learning representations by back-propagating errors[J]. Nature, 1986, 323(6088): 533-536. [29] Ma L, Chablat D, Bennis F, et al. A new simple dynamic muscle fatigue model and its validation[J]. International Journal of Industrial Ergonomics, 2009, 39(1): 211-220. [30] Chandra S, Hayashibe M, Thondiyath A. Muscle fatigue induced hand tremor clustering in dynamic laparoscopic manipulation[J]. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2018, 50(12): 5420-5431. [31] de Leva P. Adjustments to Zatsiorsky-Seluyanov’s segment inertia parameters[J]. Journal of Biomechanics, 1996, 29(9): 1223- 1230. [32] Ding Q C, Han J D, Zhao X G. Continuous estimation of human multi-joint angles from sEMG using a state-space model [J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2017, 25(9): 1518-1528. [33] 姜力，杨斌，黄琦，等．智能假肢手的生机电集成[J]．机 器人， 2017， 39(4)： 387-394. Jiang L, Yang B, Huang Q, et al. Biomechatronic integration of intelligent prosthetic hand[J]. Robot, 2017, 39(4): 387-394. [34] 陈豫生，张琴，熊蔡华．截瘫助行外骨骼研究综述：从拟 人设计依据到外骨骼研究现状[J]．机器人， 2021， 43(5)： 585-605. Chen Y S, Zhang Q, Xiong C H. From anthropomorphic design basis to exoskeleton research status: A review on walking assist exoskeletons for paraplegics[J]. Robot, 2021, 43(05): 585-605.
