Interference-resistant Multimodal Flexible Electronic Skin for Robotic Interactive Recognition

abstract Traditional array-type robotic haptic sensing systems are difficult to be widely applied due to the problems of insufficient interface flexibility as well as the contradiction between array density and refresh frequency. In order to improve the haptic sensing efficiency of robots, a multimodal flexible electronic skin is proposed inspired by the structure of human skin, and its applications to robotic object recognition and anti-magnetic field interference is studied. For the first time, the silicone rubber mixed with the magnetic powder and the distributed flexible tactile sensing array are integrated into a multimodal force-tactile sensing layer, which senses the tactile information and the force-tactile array information synchronously in time and spaced domain, and solves the problem of weak spatio-temporal pairing of the traditional tactile sensing system. A multimodal fusion algorithm based on CNN-SVM-MLP (convolution neural network, support vector machine, multilayer perceptron) is constructed to achieve efficient fusion of multimodal haptic information, and an object recognition framework for robot is constructed based on the proposed algorithm to achieve accurate recognition of interactive objects. Interaction experiments and analyses of non-magnetic and magnetic objects are carried out, and the results show that the proposed multimodal flexible e-skin can effectively improve the magnetic haptic sensor's ability to resist magnetic field interference, and accurately recognize 24 types of objects with an accuracy of 99.42%.