兼具多种理疗手法和穴位识别能力的多模态交互理疗机器人

A Multi-modal Interactive Physiotherapy Robot with Multiple Physiotherapy Operations and Acupoint Recognition

  • 摘要: 为改善理疗机器人在动态穴位识别及交互性能方面的不足,提出一种兼具多种理疗手法和动态穴位精准识别能力的多模态交互理疗机器人系统。首先,结合用户个体特征以及对理疗过程安全性与柔顺性的诉求,提出一种结合视觉、触觉、力、位置与语言的多模态人机交互框架并构建了理疗机器人系统,同时集成一款可模拟多种医师手法的气动末端执行器。其次,建立人体背部动态穴位识别范式,包括数据集建设、数据集标注及处理、穴位识别模型训练及优化。提出一种两阶段穴位处理方法并建立多尺度穴位识别模型RTOP-AcuDet(real-time object and pose-based acupoint detection),以实现对背部穴位的识别与追踪。最后,结合理疗手法特征,基于力/位混合控制方法与理疗任务规划方法控制末端执行器,进而高效复现理疗手法。通过实验验证,提出的背部穴位高精度动态识别方法的模型平均误差控制在5.52 mm,平均召回率为91.43%左右,模型检测效率为31.5帧/秒,并且该方法与多模态人机交互框架相结合后可形成多模态理疗机器人系统,适用于多种理疗服务场景。

     

    Abstract: To address the shortcomings of physiotherapy robots in dynamic acupoint recognition and interaction performance, a multimodal interactive physiotherapy robot system is proposed, which combines multiple physiotherapy operations with dynamic acupoint precise recognition. Firstly, a multimodal human-machine interaction framework integrating vision, touch, force, position, and language is proposed by considering the individual user characteristics and the requirements for safety and compliance in the physiotherapy process, and a physiotherapy robot system is constructed. Additionally, a pneumatic end-effector capable of simulating various physician operations is integrated into the system. Secondly, a dynamic acupoint recognition paradigm for the human back is developed, including dataset construction, annotation and processing of datasets, as well as training and optimization of acupoint recognition models. A two-stage acupoint processing method is introduced, and a multiscale acupoint recognition model is established, named RTOP-AcuDet (real-time object and pose-based acupoint detection), to achieve recognition and tracking of back acupoints. Finally, the end-effector is controlled based on the characteristics of physiotherapy operations by using a hybrid force-position control method combined with task planning methods for efficient reproduction of physiotherapy operations. In experimental verification, the proposed high-precision dynamic recognition method of back acupoints achieves the model average error of about 5.52 mm, the average recall rate of about 91.43%, and the model detection efficiency of 31.5 frames/s, and can be combined with the multi-modal humanmachine interaction framework to form a multi-modal physiotherapy robot system suitable for a variety of physiotherapy service scenarios.

     

/

返回文章
返回