A Comprehensive Framework for Evaluating the System Performance of Brain-controlled Mobile Robots

For the brain-controlled mobile robots, there is a lack of a unified and comprehensive evaluation framework to systematically assess the performance of such systems in practical applications. This gap makes it challenging to effectively compare different research results, hindering the further optimization of relevant system performance. To address this issue, a multidimensional comprehensive evaluation framework is proposed by considering 3 components of the user, the brain-computer interface (BCI), and the robot, which covers user experience, BCI performance, and task execution capabilities. The key technologies of the BCI, control strategy, and robot intelligence involved in brain-controlled mobile robots are outlined firstly. Then, the core evaluation indexes covered by the system are sorted out. On this basis, the index data are normalized, and the entropy weighting method and the analytic hierarchy process are comprehensively applied to determining the component indexes and dimensional weights. A comprehensive scoring framework model for the system performance is constructed, with several typical research systems selected for the evaluation and analysis. The results show that the proposed evaluation framework has good comprehensive evaluation capability. Finally, the proposed evaluation framework is summarized, and its future development directions are envisioned.