Vision-servo-based Target Alignment for Prosthetic Hands in Constrained Conditions

The prosthetic hands with wrist joints have multiple active degrees of freedom (DOFs) and rely solely on electromyographic (EMG) control, which may cause wearer fatigue and affect the accuracy of EMG decoding and intention recognition. To address the problem, the visual servo technology is used to replace EMG signals for controlling the wrist joints of prosthetic hands. When users wear the prosthetic hand to perform manipulation tasks, the wrist joint is firstly controlled by the visual servo method to achieve feature alignment, followed by integrating EMG signals to enable precise finger manipulation. Given the limited computing power and space, a manipulation and alignment method for prosthetic hands integrating visual servo is designed. In the method, the image-processing module identifies the target contour via pre-processing and edge detection, and determines key feature positions through feature-point matching. The feature-alignment module adjusts the joint posture by a servo algorithm based on the processing results to achieve precise alignment, breaking through the technical bottleneck of cross-modal fusion of visual and EMG signals. Finally, the visual servo algorithm is zero-shot deployed on the prosthetic hand for comprehensive experiments. Results indicate that this method has an average adjustment time of no more than 1.81 s, a manipulation success rate of over 70%, and a recognition accuracy of no less than 90%. To some extent, it can replace EMG signals for wrist-joint control, relieve the wearer's burden, reduce fatigue effects, effectively solve the issues of relying solely on EMG control, and has promising application prospects.