基于混合视线-脑机接口与共享控制的人-机器人交互系统

doi:10.13973/j.cnki.robot.180134

摘要
图/表
参考文献
相关文章 (15)

全文: PDF (819 KB) HTML ( ) 输出: BibTeX \| EndNote (RIS)
引用本文:
王言鑫, 纪鹏, 曾洪, 宋爱国, 吴常铖, 徐宝国, 李会军. 基于混合视线-脑机接口与共享控制的人-机器人交互系统[J]. 机器人, 2018, 40(4): 431-439.DOI: 10.13973/j.cnki.robot.180134. WANG Yanxin, JI Peng, ZENG Hong, SONG Aiguo, WU Changcheng, XU Baoguo, LI Huijun. A Human-Robot Interaction System Based on Hybrid GazeBrain-Machine Interface and Shared Control. ROBOT, 2018, 40(4): 431-439. DOI: 10.13973/j.cnki.robot.180134.

摘要设计了一种基于混合视线-脑机接口与共享控制的人-机器人交互系统，以使得用户可通过视线和意念对机器人末端在2维空间进行连续的运动控制，并在避障和趋近目标的任务中获得机器智能的辅助.首先，按照用户运动意念的强度对机器人末端的运动速度大小进行等比例连续调节，以提高用户对机器人的控制感以及完成任务的参与性.然后，提出了机器人末端运动方向的一种共享控制策略，动态地融合基于视线追踪技术所得到的用户方向控制指令以及由机器人避障和趋近目标的行为设定所得到的机器人系统方向控制指令，自适应地调整机器人系统对用户的辅助力度，以减轻用户脑力负荷，提高任务完成成功率.最后，针对搭建的基于混合视线-脑机接口和共享控制的人-机器人交互平台，通过实验验证了所提系统的有效性.

	服务

	把本文推荐给朋友
	加入我的书架
	加入引用管理器
	E-mail Alert
	RSS
	作者相关文章
	王言鑫
	纪鹏
	曾洪
	宋爱国
	吴常铖
	徐宝国
	李会军

关键词 ：人机交互, 脑-机接口, 视线追踪, 混合视线-脑机接口, 共享控制

Abstract：A human-robot interaction system is designed with the hybrid gaze brain-machine interface and the shared control strategy, to make the user continuously control the robot end-effector in 2D space with his/her gaze and thought, meanwhile obtaining timely assistance from the machine intelligence in the task of avoiding obstacles and reaching target objects. Firstly, the movement speed of the robot end-effector is adjusted continuously and proportionally by the motor imagery strength of the user, in order to increase the user's sense of control and his/her engagement in the task. Next, a shared control strategy is proposed for controlling the movement direction of the robot end-effector, which dynamically fuses the direction commands from the user obtained by gaze tracking and from the robotic system for avoiding obstacles and reaching target objects. Such a shared control strategy adaptively adjusts the assist level for the user, so as to reduce the mental workload of the user and improve the success rate of completing the task. Finally, with respect to the constructed human-robot interaction system based on the hybrid gaze brain-machine interface and the shared control, experiments are conducted to verify its effectiveness.

Key words： human-robot interaction brain-machine interface gaze tracking hybrid gaze brain-machine interface shared control

收稿日期: 2018-02-12 录用日期: 2018-05-18

TP242

基金资助:国家重点研发计划（2016YFB1001301）；国家自然科学基金（61673105，91648206）

通讯作者: 曾洪,hzeng@seu.edu.cn E-mail: hzeng@seu.edu.cn

作者简介: 王言鑫(1993-),男,硕士生.研究领域:脑-机器人交互.
纪鹏(1987-),男,博士,讲师.研究领域:机器视觉,机器人控制,移动机器人.
曾洪(1981-),男,博士,副教授.研究领域:脑-机器人交互,力触觉感知反馈,康复机器人.

链接本文:

https://robot.sia.cn/CN/10.13973/j.cnki.robot.180134 或 https://robot.sia.cn/CN/Y2018/V40/I4/431

[1] Millán J D R, Rupp R, Müller-Putz G R, et al. Combining brain-computer interfaces and assistive technologies:State-of-the-art and challenges[J]. Frontiers in Neuroscience, 2010, 4:No.161.
[2] Hochberg L R, Bacher D, Jarosiewicz B, et al. Reach and grasp by people with tetraplegia using a neurally controlled robotic arm[J]. Nature, 2012, 485(7398):372.
[3] Morgante L, Morgante F, Moro E, et al. How many parkinsonian patients are suitable candidates for deep brain stimulation of subthalamic nucleus? Results of a questionnaire[J]. Parkinsonism and Related Disorders, 2007, 13(8):528-531.
[4] Gudayol-Ferré E, Peró-Cebollero M, González-Garrido A A, et al. Changes in brain connectivity related to the treatment of depression measured through fMRI:A systematic review[J]. Frontiers in Human Neuroscience, 2015, 9:No.582.
[5] Naseer N, Hong K S. Corrigendum:fNIRS-based brain-computer interfaces:A review[J]. Frontiers in Human Neuroscience, 2015, 9:No.3.
[6] Fukuma R, Yanagisawa T, Saitoh Y, et al. Corrigendum:Real-time control of a neuroprosthetic hand by magnetoencephalographic signals from paralysed patients[J]. Scientific Reports, 2016, 6:No.21781.
[7] Moghimi S, Kushki A, Guerguerian A M, et al. A review of EEG-based brain-computer interfaces as access pathways for individuals with severe disabilities[J]. Assistive Technology, 2013, 25(2):99-110.
[8] Carlson T, Millan J D R. Brain-controlled wheelchairs:A robotic architecture[J]. IEEE Robotics and Automation Magazine, 2013, 20(1):65-73.
[9] Li Z J, Zhao S N, Duan J D, et al. Human cooperative wheelchair with brain——machine interaction based on shared control strategy[J]. IEEE/ASME Transactions on Mechatronics, 2017, 22(1):185-195.
[10] He S H, Zhang R, Wang Q H, et al. A P300-based threshold-free brain switch and its application in wheelchair control[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2017, 25(6):715-725.
[11] Chen X G, Wang Y J, Nakanishi M, et al. High-speed spelling with a noninvasive brain-computer interface[J]. Proceedings of the National Academy of Sciences of the United States of America, 2015, 112(44):E6058-E6067.
[12] Chen Y Q, Ke Y F, Meng G F, et al. Enhancing performance of P300-Speller under mental workload by incorporating dual-task data during classifier training[J]. Computer Methods and Programs in Biomedicine, 2017, 152:35-43.
[13] Zhao X G, Chu Y Q, Han J D, et al. SSVEP-based brain-computer interface controlled functional electrical stimulation system for upper extremity rehabilitation[J]. IEEE Transactions on Systems, Man, and Cybernetics:Systems, 2016, 46(7):947-956.
[14] 徐宝国,彭思,宋爱国.基于运动想象脑电的上肢康复机器人[J].机器人,2011,33(3):307-313.Xu B G, Peng S, Song A G. Upper-limb rehabilitation robot based on motor imagery EEG[J]. Robot, 2011, 33(3):307-313.
[15] 俞建成,张进,李伟.基于事件相关电位的水下机械手脑电波控制[J].机器人,2017,39(4):395-404.Yu J C, Zhang J, Li W. Controlling an underwater manipulator via event-related potentials of brainwaves[J]. Robot, 2017, 39(4):395-404.
[16] 李敏,徐光华,谢俊,等.脑卒中意念控制的主被动运动康复技术[J].机器人,2017,39(5):759-768.Lin M, Xu G H, Xie J, et al. Motor rehabilitation with control based on human intent for stroke survivors[J]. Robot, 2017, 39(5):759-768.
[17] Wang Y X, Zeng H, and Liu J. Low-cost eye-tracking glasses with real-time head rotation compensation[C]//10th International Conference on Sensing Technology. Piscataway, USA:IEEE, 2016.
[18] Soekadar S R, Witkowski M, Gómez C, et al. Hybrid EEG/EOG-based brain/neural hand exoskeleton restores fully independent daily living activities after quadriplegia[J]. Science Robotics, 2016, 1(1):32-96.
[19] Yuan P, Chen X G, Wang Y J, et al. Enhancing performances of SSVEP-based brain-computer interfaces via exploiting inter-subject information[J]. Journal of Neural Engineering, 2015, 12(4):No.046006.
[20] Khan M J, Hong K S. Hybrid EEG-fNIRS-based eight-com-mand decoding for BCI:Application to quadcopter control[J]. Frontiers in Neurorobotics, 2017, 11:No.6.
[21] Hong K S, Khan M J. Hybrid brain-computer interface techniques for improved classification accuracy and increased number of commands:A review[J]. Frontiers in Neurorobotics, 2017, 11:No.35.
[22] Zeng H, Wang Y X, Wu C C, et al. Closed-loop hybrid gaze brain-machine interface based robotic arm control with augmented reality feedback[J]. Frontiers in Neurorobotics, 2017, 11:No.60.
[23] McMullen D P, Hotson G, Katyal K D, et al. Demonstration of a semi-autonomous hybrid brain-machine interface using human intracranial EEG, eye tracking, and computer vision to control a robotic upper limb prosthetic[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2014, 22(4):784-796.
[24] Frisoli A, Loconsole C, Leonardis D, et al. A new gaze-BCI-driven control of an upper limb exoskeleton for rehabilitation in real-world tasks[J]. IEEE Transactions on Systems, Man, and Cybernetics, Part C:Applications and Reviews, 2012, 42(6):1169-1179.
[25] Kim Y J, Park S W, Yeom H G, et al. A study on a robot arm driven by three-dimensional trajectories predicted from non-invasive neural signals[J]. Biomedical Engineering Online, 2015, 14:No.81.
[26] 吴朝晖,俞一鹏,潘纲,等.脑机融合系统综述[J].生命科学,2014(6):645-649.Wu C H, Yu Y P, Pan G, et al. Brain-machine integrated systems[J]. Chinese Bulletin of Life Sciences, 2014(6):645-649.