动态场景下基于自适应语义分割的RGB-D SLAM算法

doi:10.13973/j.cnki.robot.210368

摘要
图/表
参考文献
相关文章 (4)

全文: PDF (989 KB) HTML ( ) 输出: BibTeX \| EndNote (RIS)
引用本文:
王梦瑶, 宋薇. 动态场景下基于自适应语义分割的RGB-D SLAM算法[J]. 机器人, 2023, 45(1): 16-27.DOI: 10.13973/j.cnki.robot.210368. WANG Mengyao, SONG Wei. An RGB-D SLAM Algorithm Based on Adaptive Semantic Segmentation in Dynamic Environment. ROBOT, 2023, 45(1): 16-27. DOI: 10.13973/j.cnki.robot.210368.

摘要目前较为成熟的视觉SLAM算法在应用于动态场景时,往往会因动态对象干扰而导致系统所估计的位姿误差急剧增大甚至算法失效。为解决上述问题,本文提出一种适用于室内动态场景的视觉SLAM算法,根据当前帧中特征点的运动等级信息自适应判断当前帧是否需要进行语义分割,进而实现语义信息的跨帧检测;根据语义分割网络提供的先验信息以及该对象在先前场景中的运动状态,为每个特征点分配运动等级,将其归类为静态点、可移静态点或动态点。选取合适的特征点进行位姿的初估计,再根据加权静态约束的结果对位姿进行二次优化。最后为验证本文算法的有效性,在TUM RGB-D动态场景数据集上进行实验,并与ORB-SLAM2 算法及其他处理动态场景的SLAM算法进行对比,结果表明本文算法在大部分数据集上表现良好,相较改进前的ORB-SLAM算法,本文算法在室内动态场景中的定位精度可提升90.57%。

	服务

	把本文推荐给朋友
	加入我的书架
	加入引用管理器
	E-mail Alert
	RSS

关键词 ：动态场景, 跨帧检测, 语义信息, 运动等级, 静态约束

Abstract：When the existing visual SLAM (simultaneous localization and mapping) algorithms are applied to dynamic environments, the pose error estimated by the system often increases sharply or even the algorithm fails due to the interference of dynamic objects. In order to solve the above problems, a visual SLAM system is proposed in this paper for indoor dynamic environments. By adaptively judging whether the current frame needs semantic segmentation according to the motion level information of the feature points in the current frame, the cross-frame detection of semantic information is realized. According to the prior information provided by the semantic segmentation network and the motion state of the object in the previous scene, each feature point is assigned a motion level and is classified as static point, movable static point or dynamic point. Some appropriate feature points (static points) are selected for initial pose estimation, and then secondary optimization of the pose is performed according to the results of weighted static constraints. In order to verify the effectiveness of the proposed algorithm, experiments are carried out on the TUM RGB-D dynamic scene dataset, and compared with ORB-SLAM2 and other SLAM algorithms for dynamic environments. The results show that the proposed algorithm performs well on most datasets, and the positioning accuracy in indoor dynamic environments can be improved by 90.57% compared with the ORBSLAM algorithm without the improvement.

Key words： dynamic environment cross-frame detection semantic information motion level static constraint

收稿日期: 2021-08-18 录用日期: 2022-02-08

ZTFLH:

TP242.6

通讯作者: 宋薇, song wei@shu.edu.cn E-mail: song wei@shu.edu.cn

作者简介: 王梦瑶(1998-),女,硕士生。研究领域:移动机器人,视觉SLAM。
宋薇(1981-),女,博士,副研究员。研究领域:机器人,机器视觉,精密测量。

链接本文:

https://robot.sia.cn/CN/10.13973/j.cnki.robot.210368 或 https://robot.sia.cn/CN/Y2023/V45/I1/16

[1] Mur-Artal R, Montiel J M M, Tardos J D. ORB-SLAM:A versatile and accurate monocular SLAM system[J]. IEEE Transactions on Robotics, 2015, 31(5):1147-1163.
[2] Mur-Artal R, Tardos J D. ORB-SLAM2:An open-source SLAM system for monocular, stereo, and RGB-D cameras[J]. IEEE Transactions on Robotics, 2017, 33(5):1255-1262.
[3] Elvira R, Tardós J D, Montiel J M M. ORBSLAM-Atlas:A robust and accurate multi-map system[C]//IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway, USA:IEEE, 2019:6253-6259.
[4] Qin T, Li P L, Shen S J. VINS-Mono:A robust and versatile monocular visual-inertial state estimator[J]. IEEE Transactions on Robotics, 2018, 34(4):1004-1020.
[5] Forster C, Pizzoli M, Scaramuzza D. SVO:Fast semi-direct monocular visual odometry[C]//IEEE International Conference on Robotics and Automation. Piscataway, USA:IEEE, 2014:15-22.
[6] Engel J, Koltun V, Cremers D. Direct sparse odometry[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(3):611-625.
[7] Wang Y B, Huang S D. Towards dense moving object segmentation based robust dense RGB-D SLAM in dynamic scenarios[C]//13th International Conference on Control, Automation, Robotics & Vision. Piscataway, USA:IEEE, 2014:1841-1846.
[8] Sun Y X, Liu M, Meng M Q H. Improving RGB-D SLAM in dynamic environments:A motion removal approach[J]. Robotics and Autonomous Systems, 2017, 89:110-122.
[9] 林志林,张国良,姚二亮,等.动态场景下基于运动物体检测的立体视觉里程计[J].光学学报, 2017, 37(11):187-195. Lin Z L, Zhang G L, Yao E L, et al. Stereo visual odometry based on motion object detection in the dynamic scene[J]. Acta Optica Sinica, 2017, 37(11):187-195.
[10] Chen L, Fan L, Xie G D, et al. Moving-object detection from consecutive stereo pairs using slanted plane smoothing[J]. IEEE Transactions on Intelligent Transportation Systems, 2017, 18(11):3093-3102.
[11] Wang R Z, Wan W H, Wang Y K, et al. A new RGB-D SLAM method with moving object detection for dynamic indoor scenes[J]. Remote Sensing, 2019, 11(10). DOI:10.3390/rs11101143.
[12] 张慧娟,方灶军,杨桂林.动态环境下基于线特征的RGBD视觉里程计[J].机器人, 2019, 41(1):75-82. Zhang H J, Fang Z J, Yang G L. RGB-D visual odometry in dynamic environments using line features[J]. Robot, 2019, 41(1):75-82.
[13] 魏彤,李绪.动态环境下基于动态区域剔除的双目视觉SLAM算法[J].机器人, 2020, 42(3):82-91. Wei T, Li X. Binocular vision SLAM algorithm based on dynamic region elimination in dynamic environment[J]. Robot, 2020, 42(3):82-91.
[14] Bojko A, Dupont R, Tamaazousti M, et al. Learning to segment dynamic objects using SLAM outliers[C]//25th International Conference on Pattern Recognition. Piscataway, USA:IEEE, 2021:9780-9787.
[15] Bescos B, Cadena C, Neira J. Empty cities:A dynamic-objectinvariant space for visual SLAM[J]. IEEE Transactions on Robotics, 2021, 37(2):433-451.
[16] Bao R Q, Komatsu R, Miyagusuku R, et al. Stereo camera visual SLAM with hierarchical masking and motion-state classification at outdoor construction sites containing large dynamic objects[J]. Advanced Robotics, 2021, 35(4):228-241.
[17] Bescos B, Facil J M, Civera J, et al. DynaSLAM:Tracking, mapping, and inpainting in dynamic scenes[J]. IEEE Robotics and Automation Letters, 2018, 3(4):4076-4083.
[18] He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition[C]//IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, USA:IEEE, 2016:770-778.
[19] Yu C, Liu Z X, Liu X J, et al. DS-SLAM:A semantic visual SLAM towards dynamic environments[C]//IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway, USA:IEEE, 2018:1168-1174.
[20] Badrinarayanan V, Kendall A, Cipolla R. SegNet:A deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12):2481-2495.
[21] Xiao L H, Wang J G, Qiu X S, et al. Dynamic-SLAM:Semantic monocular visual localization and mapping based on deep learning in dynamic environment[J]. Robotics and Autonomous Systems, 2019, 117:1-16.
[22] Liu W, Anguelov D, Erhan D, et al. SSD:Single shot multibox detector[C]//European Conference on Computer Vision. Cham, Switzerland:Springer, 2016:21-37.
[23] Zhong F W, Wang S, Zhang Z Q, et al. Detect-SLAM:Making object detection and SLAM mutually beneficial[C]//IEEE Winter Conference on Applications of Computer Vision. Piscataway, USA:IEEE, 2018:1001-1010.
[24] Yuan X, Chen S. SaD-SLAM:A visual SLAM based on semantic and depth information[C]//IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway, USA:IEEE, 2020:4930-4935.
[25] Zhang J, Henein M, Mahony R, et al. VDO-SLAM:A visual dynamic object-aware SLAM system[DB/OL]. (2020-05-22)[2021-08-01]. https://arxiv.org/abs/2005.11052.
[26] Nekrasov V, Shen C, Reid I. Light-weight RefineNet for realtime semantic segmentation[DB/OL]. (2018-10-08)[2021-08-01]. https://arxiv.org/abs/1810.03272.
[27] Sturm J, Engelhard N, Endres F, et al. A benchmark for the evaluation of RGB-D SLAM systems[C]//IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway, USA:IEEE, 2012:573-580.
[28] 艾青林,刘刚江,徐巧宁.动态环境下基于改进几何与运动约束的机器人RGB-D SLAM算法[J].机器人, 2021, 43(2):167-176. Ai Q L, Liu G J, Xu Q N. An RGB-D SLAM algorithm for robot based on the improved geometric and motion constraints in dynamic environment[J]. Robot, 2021, 43(2):167-176.