王梦瑶, 宋薇. 动态场景下基于自适应语义分割的RGB-D SLAM算法[J]. 机器人, 2023, 45(1): 16-27.DOI: 10.13973/j.cnki.robot.210368.
WANG Mengyao, SONG Wei. An RGB-D SLAM Algorithm Based on Adaptive Semantic Segmentation in Dynamic Environment. ROBOT, 2023, 45(1): 16-27. DOI: 10.13973/j.cnki.robot.210368.
Abstract:When the existing visual SLAM (simultaneous localization and mapping) algorithms are applied to dynamic environments, the pose error estimated by the system often increases sharply or even the algorithm fails due to the interference of dynamic objects. In order to solve the above problems, a visual SLAM system is proposed in this paper for indoor dynamic environments. By adaptively judging whether the current frame needs semantic segmentation according to the motion level information of the feature points in the current frame, the cross-frame detection of semantic information is realized. According to the prior information provided by the semantic segmentation network and the motion state of the object in the previous scene, each feature point is assigned a motion level and is classified as static point, movable static point or dynamic point. Some appropriate feature points (static points) are selected for initial pose estimation, and then secondary optimization of the pose is performed according to the results of weighted static constraints. In order to verify the effectiveness of the proposed algorithm, experiments are carried out on the TUM RGB-D dynamic scene dataset, and compared with ORB-SLAM2 and other SLAM algorithms for dynamic environments. The results show that the proposed algorithm performs well on most datasets, and the positioning accuracy in indoor dynamic environments can be improved by 90.57% compared with the ORBSLAM algorithm without the improvement.
[1] Mur-Artal R, Montiel J M M, Tardos J D. ORB-SLAM:A versatile and accurate monocular SLAM system[J]. IEEE Transactions on Robotics, 2015, 31(5):1147-1163. [2] Mur-Artal R, Tardos J D. ORB-SLAM2:An open-source SLAM system for monocular, stereo, and RGB-D cameras[J]. IEEE Transactions on Robotics, 2017, 33(5):1255-1262. [3] Elvira R, Tardós J D, Montiel J M M. ORBSLAM-Atlas:A robust and accurate multi-map system[C]//IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway, USA:IEEE, 2019:6253-6259. [4] Qin T, Li P L, Shen S J. VINS-Mono:A robust and versatile monocular visual-inertial state estimator[J]. IEEE Transactions on Robotics, 2018, 34(4):1004-1020. [5] Forster C, Pizzoli M, Scaramuzza D. SVO:Fast semi-direct monocular visual odometry[C]//IEEE International Conference on Robotics and Automation. Piscataway, USA:IEEE, 2014:15-22. [6] Engel J, Koltun V, Cremers D. Direct sparse odometry[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(3):611-625. [7] Wang Y B, Huang S D. Towards dense moving object segmentation based robust dense RGB-D SLAM in dynamic scenarios[C]//13th International Conference on Control, Automation, Robotics & Vision. Piscataway, USA:IEEE, 2014:1841-1846. [8] Sun Y X, Liu M, Meng M Q H. Improving RGB-D SLAM in dynamic environments:A motion removal approach[J]. Robotics and Autonomous Systems, 2017, 89:110-122. [9] 林志林,张国良,姚二亮,等.动态场景下基于运动物体检测的立体视觉里程计[J].光学学报, 2017, 37(11):187-195. Lin Z L, Zhang G L, Yao E L, et al. Stereo visual odometry based on motion object detection in the dynamic scene[J]. Acta Optica Sinica, 2017, 37(11):187-195. [10] Chen L, Fan L, Xie G D, et al. Moving-object detection from consecutive stereo pairs using slanted plane smoothing[J]. IEEE Transactions on Intelligent Transportation Systems, 2017, 18(11):3093-3102. [11] Wang R Z, Wan W H, Wang Y K, et al. A new RGB-D SLAM method with moving object detection for dynamic indoor scenes[J]. Remote Sensing, 2019, 11(10). DOI:10.3390/rs11101143. [12] 张慧娟,方灶军,杨桂林.动态环境下基于线特征的RGBD视觉里程计[J].机器人, 2019, 41(1):75-82. Zhang H J, Fang Z J, Yang G L. RGB-D visual odometry in dynamic environments using line features[J]. Robot, 2019, 41(1):75-82. [13] 魏彤,李绪.动态环境下基于动态区域剔除的双目视觉SLAM算法[J].机器人, 2020, 42(3):82-91. Wei T, Li X. Binocular vision SLAM algorithm based on dynamic region elimination in dynamic environment[J]. Robot, 2020, 42(3):82-91. [14] Bojko A, Dupont R, Tamaazousti M, et al. Learning to segment dynamic objects using SLAM outliers[C]//25th International Conference on Pattern Recognition. Piscataway, USA:IEEE, 2021:9780-9787. [15] Bescos B, Cadena C, Neira J. Empty cities:A dynamic-objectinvariant space for visual SLAM[J]. IEEE Transactions on Robotics, 2021, 37(2):433-451. [16] Bao R Q, Komatsu R, Miyagusuku R, et al. Stereo camera visual SLAM with hierarchical masking and motion-state classification at outdoor construction sites containing large dynamic objects[J]. Advanced Robotics, 2021, 35(4):228-241. [17] Bescos B, Facil J M, Civera J, et al. DynaSLAM:Tracking, mapping, and inpainting in dynamic scenes[J]. IEEE Robotics and Automation Letters, 2018, 3(4):4076-4083. [18] He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition[C]//IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, USA:IEEE, 2016:770-778. [19] Yu C, Liu Z X, Liu X J, et al. DS-SLAM:A semantic visual SLAM towards dynamic environments[C]//IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway, USA:IEEE, 2018:1168-1174. [20] Badrinarayanan V, Kendall A, Cipolla R. SegNet:A deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12):2481-2495. [21] Xiao L H, Wang J G, Qiu X S, et al. Dynamic-SLAM:Semantic monocular visual localization and mapping based on deep learning in dynamic environment[J]. Robotics and Autonomous Systems, 2019, 117:1-16. [22] Liu W, Anguelov D, Erhan D, et al. SSD:Single shot multibox detector[C]//European Conference on Computer Vision. Cham, Switzerland:Springer, 2016:21-37. [23] Zhong F W, Wang S, Zhang Z Q, et al. Detect-SLAM:Making object detection and SLAM mutually beneficial[C]//IEEE Winter Conference on Applications of Computer Vision. Piscataway, USA:IEEE, 2018:1001-1010. [24] Yuan X, Chen S. SaD-SLAM:A visual SLAM based on semantic and depth information[C]//IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway, USA:IEEE, 2020:4930-4935. [25] Zhang J, Henein M, Mahony R, et al. VDO-SLAM:A visual dynamic object-aware SLAM system[DB/OL]. (2020-05-22)[2021-08-01]. https://arxiv.org/abs/2005.11052. [26] Nekrasov V, Shen C, Reid I. Light-weight RefineNet for realtime semantic segmentation[DB/OL]. (2018-10-08)[2021-08-01]. https://arxiv.org/abs/1810.03272. [27] Sturm J, Engelhard N, Endres F, et al. A benchmark for the evaluation of RGB-D SLAM systems[C]//IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway, USA:IEEE, 2012:573-580. [28] 艾青林,刘刚江,徐巧宁.动态环境下基于改进几何与运动约束的机器人RGB-D SLAM算法[J].机器人, 2021, 43(2):167-176. Ai Q L, Liu G J, Xu Q N. An RGB-D SLAM algorithm for robot based on the improved geometric and motion constraints in dynamic environment[J]. Robot, 2021, 43(2):167-176.