A Motion Control Method for Robotic Arm Based on a Wearable Hybrid Human-Machine Interface

LU Zilin; ZHOU Yajun; HUANG Qiyun; LI Yuanqing

doi:10.13973/j.cnki.robot.230254

Volume 46 Issue 1

Jan. 2024

Turn off MathJax

Article Contents

Abstract

References

Supplements

ROBOT > 2024 > 46(1): 68-80. > DOI: 10.13973/j.cnki.robot.230254 CSTR: 32165.14.robot.230254

LU Zilin, ZHOU Yajun, HUANG Qiyun, LI Yuanqing. A Motion Control Method for Robotic Arm Based on a Wearable Hybrid Human-Machine Interface[J]. ROBOT, 2024, 46(1): 68-80. DOI: 10.13973/j.cnki.robot.230254

Citation:

PDF (1142 KB)

A Motion Control Method for Robotic Arm Based on a Wearable Hybrid Human-Machine Interface

LU Zilin^{1, 2, 3},
ZHOU Yajun²,
HUANG Qiyun²,
LI Yuanqing^{1, 2, 3, ,}

1.
School of Automation Science and Engineering, South China University of Technology, Guangzhou 510640, China
2.
Research Center for Brain-Computer Interface, Pazhou Lab, Guangzhou 510330, China
3.
South China Brain-Computer Interface Technology Co., Ltd, Guangzhou 510320, China

More Information

Received Date: September 14, 2023
Revised Date: November 23, 2023
Accepted Date: November 21, 2023
Available Online: February 03, 2024

Graphical Abstract

Abstract

Abstract

Existing HMI (human-machine interface) systems suffer from issues such as limited commands, complex operation, and restricted task capabilities, preventing effective expansion into multi-dimensional motion control for robotic arms. This paper introduces a method for controlling robotic arm movements based on a wearable hybrid HMI. This method combines various signals, including electrooculography (EOG), head posture, and speech from the user, transforming them into control commands, thereby enabling continuous two-dimensional (2D) and three-dimensional (3D) motion control of the robotic arm at any angle. 10 participants complete tests involving command output, 2D target tracking, alphabetic writing, and 3D object grasping. The results indicate that the blink-generated commands of the proposed system have an average accuracy of 96.67%, an average response time of 1.51 s, an average information transfer rate (ITR) of 142.53 bit/min, and an average false positive rate (FPR) of 0.05 event/min. Additionally, the root mean square deviations of target tracking along 2 different routes on a 2D plane are 0.12 and 0.14 (normalized), while the average trajectory efficiency of 3D object grasping is 92.65%. The control performance of the system is comparable to manual control. The experimental results verify the feasibility of using a hybrid HMI for achieving efficient motion control of robotic arms and its potential application in assisting upper-limb mobility functions.
- hybrid human-machine interface,
- wearable device,
- robotic arm,
- motion control

FullText(HTML)

References (44)

References

[1]	KUMAR D, VERMA S, BHATTACHARYA S, et al. Audio-visual stimulation in conjunction with functional electrical stimulation to address upper limb and lower limb movement disorder[J]. European Journal of Translational Myology, 2016, 26(2). doi: 10.4081/ejtm.2016.6030
[2]	LEE B O, SARAGIH I D, BATUBARA S O. Robotic arm use for upper limb rehabilitation after stroke: A systematic review and meta-analysis[J]. Kaohsiung Journal of Medical Sciences, 2023, 39(5): 435-445. doi: 10.1002/kjm2.12679
[3]	FROLOV A A, KOZLOVSKAIA I B, BIRYUKOVA E V, et al. Robotic devices in poststroke rehabilitation[J]. Zhurnal Vysshei Nervnoi Deyatelnosti Imeni I P Pavlova, 2017, 67(4): 394-413. doi: 10.7868/s004446771704-0017
[4]	BOCKBRADER M A, FRANCISCO G, LEE R, et al. Brain computer interfaces in rehabilitation medicine[J]. PM & R, 2018, 10(9S2): S233-S243. doi: 10.1016/j.pmrj.2018.05.028
[5]	CHAUDHARY U, BIRBAUMER N, RAMOS-MURGUIAL-DAY A. Brain-computer interfaces for communication and rehabilitation[J]. Nature Reviews Neurology, 2016, 12(9): 513-525. doi: 10.1038/nrneurol.2016.113
[6]	REMSIK A, YOUNG B, VERMILYEA R, et al. A review of the progression and future implications of brain-computer interface therapies for restoration of distal upper extremity motor function after stroke[J]. Expert Review of Medical Devices, 2016, 13(5): 445-454. doi: 10.1080/17434440.2016.1174572
[7]	HORTAL E, IÁÑEZ E, ÚBEDA A, et al. Combining a brain-machine interface and an electrooculography interface to perform pick and place tasks with a robotic arm[J]. Robotics and Autonomous Systems, 2015, 72: 181-188. doi: 10.1016/j.robot.2015.05.010
[8]	HAYASHI H, TSUJI T. Human-machine interfaces based on bioelectric signals: A narrative review with a novel system proposal[J]. IEEJ Transactions on Electrical and Electronic Engineering, 2022, 17(11): 1536-1544. doi: 10.1002/tee.23646
[9]	DONOGHUE J P. Bridging the brain to the world: A perspective on neural interface systems[J]. Neuron, 2008, 60(3): 511-521. doi: 10.1016/j.neuron.2008.10.037
[10]	VILELA M, HOCHBERG L R. Applications of brain-computer interfaces to the control of robotic and prosthetic arms[J]. Handbook of Clinical Neurology, 2020, 168: 87-99. doi: 10.1016/b978-0-444-63934-9.00008-1
[11]	WANG H F, DONG X J, CHEN Z K, et al. Hybrid gaze/EEG brain computer interface for robot arm control on a pick and place task[C]//37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Piscataway, USA: IEEE, 2015: 1476-1479. doi: 10.1109/EMBC.2015.7318649
[12]	SHARMA K, JAIN N, PAL P K. Detection of eye closing/opening from EOG and its application in robotic arm control[J]. Biocybernetics and Biomedical Engineering, 2020, 40(1): 173-186. doi: 10.1016/j.bbe.2019.10.004
[13]	ARTEMIADIS P K, KYRIAKOPOULOS K J. A switching regime model for the EMG-based control of a robot arm[J]. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 2011, 41(1): 53-63. doi: 10.1109/tsmcb.2010.2045120
[14]	CHENG L W, LI D L, YU G J, et al. Robotic arm control system based on brain-muscle mixed signals[J]. Biomedical Signal Processing and Control, 2022, 77. doi: 10.1016/j.bspc.2022.103754
[15]	TADANO K, KAWASHIMA K. A pneumatic laparoscope holder controlled by head movement[J]. The International Journal of Medical Robotics and Computer Assisted Surgery, 2015, 11(3): 331-340. doi: 10.1002/rcs.1606
[16]	KILMARX J, ABIRI R, BORHANI S, et al. Sequence-based manipulation of robotic arm control in brain machine interface[J]. International Journal of Intelligent Robotics and Applications, 2018, 2(2): 149-160. doi: 10.1007/s41315-018-0049-7
[17]	GAO Q, DOU L X, BELKACEM A N, et al. Noninvasive electroencephalogram based control of a robotic arm for writing task using hybrid BCI system[J]. BioMed Research International, 2017. doi: 10.1155/2017/8316485
[18]	JOHNSON G D, WAYTOWICH N R, COX D J, et al. Extending the discrete selection capabilities of the P300 speller to goal-oriented robotic arm control[C]//3rd IEEE RAS and EMBS International Conference on Biomedical Robotics and Biomechatronics. Piscataway, USA: IEEE, 2010: 572-575. doi: 10.1109/BIOROB.2010.5628039
[19]	PATHIRAGE I, KHOKAR K, KLAY E, et al. A vision based P300 brain computer interface for grasping using a wheelchair-mounted robotic arm[C]//IEEE/ASME International Conference on Advanced Intelligent Mechatronics. Piscataway, USA: IEEE, 2013: 188-193. doi: 10.1109/AIM.2013.6584090
[20]	AI J K, MENG J J, MAI X M, et al. BCI control of a robotic arm based on SSVEP with moving stimuli for reach and grasp tasks[J]. IEEE Journal of Biomedical and Health Informatics, 2023, 27(8): 3818-3829. doi: 10.1109/jbhi.2023.3277612
[21]	CHEN X, ZHAO B, WANG Y, et al. Control of a 7-DOF robotic arm system with an SSVEP-based BCI[J]. International Journal of Neural Systems, 2018, 28(8). doi: 10.1142/S0129065718500181
[22]	ALFRED J, S H, ALEX J S R. BCI based robotic arm control using MI-EEG and spiking neural network[C]//13th International Conference on Computing Communication and Networking Technologies. Piscataway, USA: IEEE, 2022. doi: 10.1109ICCCNT54827.2022.9984240.
[23]	XU Y, DING C, SHU X K, et al. Shared control of a robotic arm using non-invasive brain-computer interface and computer vision guidance[J]. Robotics and Autonomous Systems, 2019, 115: 121-129. doi: 10.1016/j.robot.2019.02.014
[24]	HE S H, ZHOU Y J, YU T Y, et al. EEG- and EOG-based asynchronous hybrid BCI: A system integrating a speller, a web browser, an e-mail client, and a file explorer[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2020, 28(2): 519-530. doi: 10.1109/tnsre.2019.2961309
[25]	UBEDA A, IÁÑEZ E, AZORÍN J M. Wireless and portable EOG-based interface for assisting disabled people[J]. IEEE/ASME Transactions on Mechatronics, 2011, 16(5): 870-873. doi: 10.1109/tmech.2011.2160354
[26]	ASPELUND S, PATEL P, LEE M H, et al. Controlling a robotic arm for functional tasks using a wireless head-joystick: A case study of a child with congenital absence of upper and lower limbs[J]. PLOS ONE, 2020, 15(8). doi: 10.1371/journal.pone.0226052
[27]	JOHN R A, VARGHESE S, SHAJI S T, et al. Assistive device for physically challenged persons using voice controlled intelligent robotic arm[C]//6th International Conference on Advanced Computing and Communication Systems. Piscataway, USA: IEEE, 2020: 806-810. doi: 10.1109/icaccs48705.2020.9074236
[28]	CHOI I, RHIU I, LEE Y, et al. A systematic review of hybrid brain-computer interfaces: Taxonomy and usability perspectives[J]. PLOS ONE, 2017, 12(4). doi: 10.1371/journal.pone.0176674
[29]	ZHOU Y J, YU T Y, GAO W, et al. Shared three-dimensional robotic arm control based on asynchronous BCI and computer vision[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2023, 31: 3163-3175. doi: 10.1109/tnsre.2023.3299350
[30]	MINATI L, YOSHIMURA N, KOIKE Y. Hybrid control of a vision-guided robot arm by EOG, EMG, EEG biosignals and head movement acquired via a consumer-grade wearable device[J]. IEEE Access, 2016, 4: 9528-9541. doi: 10.1109/access.2017.2647851
[31]	XU B G, LI W L, LIU D P, et al. Continuous hybrid BCI control for robotic arm using noninvasive electroencephalogram, computer vision, and eye tracking[J]. Mathematics, 2022, 10(4). doi: 10.3390/math10040618
[32]	ZHU Y L, LI Y, LU J L, et al. A hybrid BCI based on SSVEP and EOG for robotic arm control[J]. Frontiers in Neurorobotics, 2020, 14. doi: 10.3389/fnbot.2020.583641
[33]	蒋鑫, 刘红星, 刘铁兵, 等. 生物电采集中右腿驱动电路参数的确定[J]. 北京生物医学工程, 2011, 30(5): 506-511. doi: 10.3969/j.issn.1002-3208.2011.05.14 JIANG X, LIU H X, LIU T B, et al. Parameters for the driven-right-leg in biopotential measurements[J]. Beijing Biomedical Engineering, 2011, 30(5): 506-511. doi: 10.3969/j.issn.1002-3208.2011.05.14
[34]	杨正, 王静敏, 朱樟明, 等. 利用右腿驱动技术的心电信号模拟前端设计[J]. 西安电子科技大学学报(自然科学版), 2016, 43(4): 166-171. doi: 10.3969/j.issn.1001-2400.2016.04.029 YANG Z, WANG J M, ZHU Z M, et al. ECG front-end subsystem with the driven-right-leg circuit[J]. Journal of Xidian University (Natural Science), 2016, 43(4): 166-171. doi: 10.3969/j.issn.1001-2400.2016.04.029
[35]	贺庆, 郝思聪, 司娟宁, 等. 面向脑机接口的脑电采集设备硬件系统综述[J]. 中国生物医学工程学报, 2020, 39(6): 747-758. doi: 10.3969/j.issn.0258-8021.2020.06.012 HE Q, HAO S C, SI J N, et al. Research development of electroencephalogram acquisition devices for brain computer interface[J]. Chinese Journal of Biomedical Engineering, 2020, 39(6): 747-758. doi: 10.3969/j.issn.0258-8021.2020.06.012
[36]	LI Y Q, HE S H, HUANG Q Y, et al. A EOG-based switch and its application for "start/stop" control of a wheelchair[J]. Neurocomputing, 2018, 275: 1350-1357. doi: 10.1016/j.neucom.2017.09.085
[37]	MA J X, ZHANG Y, CICHOCKI A, et al. A novel EOG/EEG hybrid human-machine interface adopting eye movements and ERPs: Application to robot control[J]. IEEE Transactions on Biomedical Engineering, 2015, 62(3): 876-889. doi: 10.1109/TBME.2014.2369483
[38]	WOLPAW J R, BIRBAUMER N, HEETDERKS W J, et al. Brain-computer interface technology: A review of the first international meeting[J]. IEEE Transactions on Rehabilitation Engineering, 2000, 8(2): 164-173. doi: 10.1109/TRE.2000.847807
[39]	PANICKER R C, PUTHUSSERYPADY S, SUN Y. An asynchronous P300 BCI with SSVEP-based control state detection[J]. IEEE Transactions on Biomedical Engineering, 2011, 58(6): 1781-1788. doi: 10.1109/TBME.2011.2116018
[40]	CHAI T, DRAXLER R R. Root mean square error (RMSE) or mean absolute error (MAE)? – Arguments against avoiding RMSE in the literature[J]. Geoscientific Model Development, 2014, 7(3): 1247-1250. doi: 10.5194/gmd-7-1247-2014
[41]	MILLAN J R, MOURIÑO J. Asynchronous BCI and local neural classifiers: An overview of the adaptive brain interface project[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2003, 11(2): 159-161. doi: 10.1109/TNSRE.2003.814435
[42]	ZHAO X F, LIU D, XIE S Q, et al. Multiple action movement control scheme for assistive robot based on binary motor imagery EEG[C]//9th International Conference on Communications, Signal Processing, and Systems. Singapore: Springer, 2021: 760-768. doi: 10.1007/978-981-15-8411-4_101
[43]	ALJALAL M, IBRAHIM S, DJEMAL R, et al. Comprehensive review on brain-controlled mobile robots and robotic arms based on electroencephalography signals[J]. Intelligent Service Robotics, 2020, 13(4): 539-563. doi: 10.1007/s11370-020-00328-5
[44]	PESHORI K R, SCHICATANO E J, GOPALASWAMY R, et al. Aging of the trigeminal blink system[J]. Experimental Brain Research, 2001, 136(3): 351-363. doi: 10.1007/s002210000585

Supplements (1)

Supplements
Other Related Supplements
- WeChat link
  https://mp.weixin.qq.com/s?__biz=Mzg3MzkwNjAwMw==&mid=2247490403&idx=1&sn=66e8edecc0b6f63d9f7e7263d958d6c5&chksm=ced9b15ff9ae3849e61bca34ad7e403e6c4f578c445dc02b28c247e10856ba1ae701db66d696&mpshare=1&scene=1&srcid=0312qHtNEb6qMzlsU5ynQD9E&sharer_shareinfo=d41545666e393daeb5cfbc0ba33442e9&sharer_shareinfo_first=d41545666e393daeb5cfbc0ba33442e9&from=industrynews&version=4.1.20.6024&platform=win#rd