服务机器人的触觉手势识别

doi:10.13973/j.cnki.robot.210045

摘要
图/表
参考文献
相关文章 (15)

全文: PDF (651 KB) HTML ( ) 输出: BibTeX \| EndNote (RIS)
引用本文:
杨天昊, 李云开, 王雅昕, 孟庆浩. 服务机器人的触觉手势识别[J]. 机器人, 2022, 44(3): 310-320.DOI: 10.13973/j.cnki.robot.210045. YANG Tianhao, LI Yunkai, WANG Yaxin, MENG Qinghao. Touch Gesture Recognition for Service Robots. ROBOT, 2022, 44(3): 310-320. DOI: 10.13973/j.cnki.robot.210045.

摘要为了实现机器人在人机交互过程中的触觉感知，提出了一种用于服务机器人的触觉手势识别方法。首先，将电子皮肤安装在服务机器人上，通过采集15位被试者的10种手势动作信号，构建了情感手势数据集。然后，使用时空分离卷积神经网络，对被试者触摸服务机器人时做出的触摸手势进行分类。结果表明，被试内手势识别率为90.25%，跨被试手势识别率为83.44%。通过调节模型中的时空通道调节因子，在几乎不降低识别率的同时，可以大幅减少模型参数量。基于电子皮肤的触觉手势识别实验，初步认为使用时空分离卷积神经网络能够以较高的准确率和较低的计算代价实现对人的触觉手势识别，这为服务机器人通过电子皮肤与人实现情感交互提供了可能。

	服务

	把本文推荐给朋友
	加入我的书架
	加入引用管理器
	E-mail Alert
	RSS
	作者相关文章
	杨天昊
	李云开
	王雅昕
	孟庆浩

关键词 ：触觉, 手势识别, 人机交互, 卷积神经网络

Abstract：In order to realize tactile perception of robots in the process of human-robot interaction, a touch gesture recognition method for service robots is proposed. Firstly, the electronic skin is installed on the service robot, and an affective gesture dataset is built by collecting 10 kinds of touch gesture signals from 15 subjects. Then, a factorized spatio-temporal convolutional neural network ((2+1)D CNN) is used to classify the gestures applied to the service robot. The results show that the gesture recognition accuracies within-subject and across-subject are 90.25% and 83.44%, respectively. By changing the adjustment factors of spatio-temporal channels in the model, the model parameters can be greatly reduced while the recognition accuracy decreases slightly. Based on the touch gesture recognition experiment using electronic skin, it is preliminarily believed that the use of (2+1)D CNN can achieve human touch gesture recognition with higher accuracy and lower computational cost, which can realize the emotional interaction between service robots and humans through electronic skin.

Key words： haptics gesture recognition human-robot interaction convolutional neural network

收稿日期: 2021-01-25 录用日期: 2021-05-28

TP391.4

基金资助:国家重点研发计划(2017YFC0306200)

通讯作者: 孟庆浩,qh_meng@tju.edu.cn E-mail: qh_meng@tju.edu.cn

作者简介: 杨天昊（1997-），男，硕士生。研究领域：触觉手势识别，深度学习。
李云开（1995-），男，博士生。研究领域：人机交互，触觉感知，深度学习。
王雅昕（1998-），女，硕士生。研究领域：触觉手势识别，人机对话。

链接本文:

https://robot.sia.cn/CN/10.13973/j.cnki.robot.210045 或 https://robot.sia.cn/CN/Y2022/V44/I3/310

[1] 邓卫斌，于国龙．社交机器人发展现状及关键技术研究[J]．科学技术与工程，2016，16(12)：163-170. Deng W B, Yu G L. Development and key technology of social robot[J]. Science Technology and Engineering, 2016, 16(12): 163-170.
[2] Keshmiri S, Shiomi M, Sumioka H, et al. Gentle versus strong touch classification: Preliminary results, challenges, and potentials[J]. Sensors, 2020, 20(11). DOI: 10.3390/s20113033.
[3] Hertenstein M J, Holmes R, McCullough M, et al. The communication of emotion via touch[J]. Emotion, 2009, 9(4): 566-573.
[4] 彭玉青，赵晓松，陶慧芳，等．复杂背景下基于深度学习的手势识别[J]．机器人，2019，41(4)：534-542. Peng Y Q, Zhao X S, Tao H F, et al. Hand gesture recognition against complex background based on deep learning[J]. Robot, 2019, 41(4): 534-542.
[5] Li Y K, Wang B W, Li Y Y, et al. Design and output characteristics of magnetostrictive tactile sensor for detecting force and stiffness of manipulated objects[J]. IEEE Transactions on Industrial Informatics, 2019, 15(2): 1219-1225.
[6] Šabanović S, Bennett C C, Chang W L, et al. PARO robot affects diverse interaction modalities in group sensory therapy for older adults with dementia[C]//IEEE 13th International Conference on Rehabilitation Robotics. Piscataway, USA: IEEE, 2013. DOI: 10.1109/ICORR.2013.6650427.
[7] Saunderson S, Nejat G. How robots influence humans: A survey of nonverbal communication in social human-robot interaction[J]. International Journal of Social Robotics, 2019, 11(4): 575-608.
[8] Jung M M, Poel M, Poppe R, et al. Automatic recognition of touch gestures in the corpus of social touch[J]. Journal on Multimodal User Interfaces, 2017, 11(1): 81-96.
[9] Jung M M, Poppe R, Poel M, et al. Touching the void - Introducing CoST: corpus of social touch[C]//16th International Conference on Multimodal Interaction. New York, USA: ACM, 2014: 120-127.
[10] van Wingerden S, Uebbing T J, Jung M M, et al. A neural network based approach to social touch classification[C]// 2014 Workshop on Emotion Representation and Modelling in Human-Computer-Interaction-Systems. New York, USA: ACM, 2014: 7-12.
[11] Yohanan S, MacLean K E. The role of affective touch in human-robot interaction: Human intent and expectations in touching the haptic creature[J]. International Journal of Social Robotics, 2012, 4(2): 163-180.
[12] Hughes D, Farrow N, Profita H, et al. Detecting and identifying tactile gestures using deep autoencoders, geometric moments and gesture level features[C]//ACM International Conference on Multimodal Interaction. New York, USA: ACM, 2015: 415-422.
[13] Ta V C, Johal W, Portaz M, et al. The Grenoble system for the social touch challenge at ICMI 2015[C]//ACM International Conference on Multimodal Interaction. New York, USA: ACM, 2015: 391-398.
[14] Altuglu T B, Altun K. Recognizing touch gestures for social human-robot interaction[C]//ACM International Conference on Multimodal Interaction. New York, USA: ACM, 2015: 407-413.
[15] Hughes D, Krauthammer A, Correll N. Recognizing social touch gestures using recurrent and convolutional neural networks[C]//IEEE International Conference on Robotics and Automation. Piscataway, USA: IEEE, 2017: 2315-2321.
[16] Albawi S, Bayat O, Al-Azawi S, et al. Social touch gesture recognition using convolutional neural network[J]. Computational Intelligence and Neuroscience, 2018. DOI: 10.1155/ 2018/6973103.
[17] Altun K, MacLean K E. Recognizing affect in human touch of a robot[J]. Pattern Recognition Letters. 2015, 66(15): 31-40.
[18] Gamboa-Montero J J, Alonso-Martin F, Castillo J C, et al. Detecting, locating and recognising human touches in social robots with contact microphones[J]. Engineering Applications of Artificial Intelligence, 2020, 92. DOI: 10.1016/j.engappai.2020. 103670.
[19] 许会超，苗新刚，汪苏．基于 FBG 的机器人柔性触觉传感器[J]．机器人，2018，40(5)：634-639,722. Xu H C, Miao X G, Wang S. A flexible tactile sensor for robot based on FBG[J]. Robot, 2018, 40(5): 634-639,722.
[20] Silvera-Tawil D, Rye D, Velonaki M. Interpretation of social touch on an artificial arm covered with an EIT-based sensitive skin[J]. International Journal of Social Robotics, 2014, 6(4): 489-505.
[21] Someya T, Sekitani T, Iba S, et al. A large-area, flexible pressure sensor matrix with organic field-effect transistors for artificial skin applications[J]. PNAS, 2004, 101(27): 9966-9970.
[22] Wang X W, Gu Y, Xiong Z P, et al. Silk-molded flexible, ultrasensitive, and highly stable electronic skin for monitoring human physiological signals[J]. Advanced Materials, 2014, 26(9): 1336-1342.
[23] Zhao X, Zhang Z, Liao Q, et al. Self-powered user-interactive electronic skin for programmable touch operation platform[J]. Science Advances, 2020, 6(28). DOI: 10.1126/sciadv.aba4294.
[24] Fang B, Sun F C, Liu H P, et al. Wearable technology for robotic manipulation and learning[M]. Singapore: Springer, 2020.
[25] Sundaram S, Kellnhofer P, Li Y, et al. Learning the signatures of the human grasp using a scalable tactile glove[J]. Nature, 2019, 569(7758): 698-702.
[26] Kale S, Mane S, Patil P. Wearable biomedical parameter monitoring system: A review[C]//International Conference of Electronics, Communication and Aerospace Technology. Piscataway, USA: IEEE, 2017: 614-617.
[27] Flagg A, MacLean K. Affective touch gesture recognition for a furry zoomorphic machine[C]//7th International Conference on Tangible, Embedded and Embodied Interaction. New York, USA: ACM, 2013: 25-32.
[28] Russell J A, Weiss A, Mendelsohn G A. Affect grid: A single-item scale of pleasure and arousal[J]. Journal of Personality and Social Psychology, 1989, 57: 493-502.
[29] Ekman P. An argument for basic emotions[J]. Cognition and Emotion, 1992, 6(3-4): 169-200.
[30] Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6): 84-90.
[31] Tran D, Wang H, Torresani L, et al. A closer look at spatio-temporal convolutions for action recognition[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway, USA: IEEE, 2018: 6450-6459.
[32] Glorot X, Bordes A, Bengio Y. Deep sparse rectifier neural networks[C]//14th International Conference on Artificial Intelligence and Statistics. 2011: 315-323.
[33] Srivastava N, Hinton G E, Krizhevsky A, et al. Dropout: A simple way to prevent neural networks from overfitting[J]. Journal of Machine Learning Research, 2014, 15(1): 1929-1958.
[34] Ioffe S, Szegedy C. Batch normalization: Accelerating deep network training by reducing internal covariate shift[C]//32nd International Conference on Machine Learning. New York, USA: ACM, 2015: 448-456.
[35] Krauledat M, Tangermann M, Blankertz B, et al. Towards zero training for brain-computer interfacing[J]. PLoS One, 2008, 3(8). DOI: 10.1371/journal.pone.0002967.
[36] Li X, Song D, Zhang P, et al. Exploring EEG features in cross-subject emotion recognition[J]. Frontiers in Neuroscience, 2018, 12. DOI: 10.3389/fnins.2018.00162.
[37] Jones S E, Brown B C. Touch attitudes and behaviors, recollections of early childhood touch, and social self-confidence[J]. Journal of Nonverbal Behavior, 1996, 20(3): 147-163.
[38] Deethardt J F, Hines D G. Tactile communication and personality differences[J]. Journal of Nonverbal Behavior, 1983, 8(2): 143-156.