Robust SLAM Algorithm Based on Semantic Information and Edge Consistency
YAO Erliang1, ZHANG Hexin1, SONG Haitao1, ZHANG Guoliang2
1. Department of Control Engineering, Rocket Force University of Engineering, Xi'an 710025, China;
2. School of Control Engineering, Chengdu University of Information Technology, Chengdu 610225, China
姚二亮, 张合新, 宋海涛, 张国良. 基于语义信息和边缘一致性的鲁棒SLAM算法[J]. 机器人, 2019, 41(6): 751-760.DOI: 10.13973/j.cnki.robot.180697.
YAO Erliang, ZHANG Hexin, SONG Haitao, ZHANG Guoliang. Robust SLAM Algorithm Based on Semantic Information and Edge Consistency. ROBOT, 2019, 41(6): 751-760. DOI: 10.13973/j.cnki.robot.180697.
Abstract:To handle the performance degradation and the insufficient robustness of visual localization in dynamic environments, and to improve the created environment map, a robust simultaneous localization and mapping (SLAM) algorithm based on the semantic information and the edge consistency is proposed. Firstly, the semantic information of the environment is acquired by YOLOv3 algorithm, and the semantically dynamic-static segmentations of the image are obtained preliminarily. The consistency evaluation is conducted based on the distance transform errors and photometric errors of edges in the images to refine the dynamic-static area. Moreover, the dynamic regions are corrected by the connected component analysis and the loophole mending algorithm. The feature points in the non-dynamic regions are matched and the camera poses are optimized by minimizing the reprojection errors of feature points by the nonlinear optimization algorithm. The mapping keyframes are selected based on the covisibility of feature points and the areas of the dynamic-static regions. And the static environment map is created without the information of the dynamic objects. The experiment on the highly dynamic scenes in the public datasets show that the proposed method can distinguish the dynamic-static information accurately, and perform the precise localization and mapping in dynamic environments. Besides, the degradation of the positioning accuracy does not exist in the proposed method in the static environment.
[1] Fuentes-Pacheco J, Ruiz-Ascencio J, Manuel Rendon-Mancha J. Visual simultaneous localization and mapping:A survey[J]. Artificial Intelligence Review, 2012, 43(1):55-81.
[2] 赵洋,刘国良,田国会,等.基于深度学习的视觉SLAM综述[J].机器人,2017,39(6):889-896.Zhao Y, Liu G L, Tian G H, et al. A survey of visual SLAM based on deep learning[J]. Robot, 2017, 39(6):889-896.
[3] 于金山,吴皓,田国会,等.基于云的语义库设计及机器人语义地图构建[J].机器人,2016,38(4):410-419.Yu J S, Wu H, Tian G H, et al. Semantic database design and semantic map construction of robots based on the cloud[J]. Robot, 2016, 38(4):410-419.
[4] Cadena C, Carlone L, Carrillo H, et al. Simultaneous localization and mapping:Present, future, and the robust-perception age[J]. IEEE Transactions on Robotics, 2016, 32(6):1309-1332.
[5] Lianos K N, Schönberger J L, Pollefeys M, et al. VSO:Visual semantic odometry[C]//European Conference on Computer Vision. Berlin, Germany:Springer, 2018:234-250.
[6] Geiger A, Lenz P, Stiller C, et al. Vision meets robotics:The KITTI dataset[J]. International Journal of Robotics Research, 2013, 32(11):1231-1237.
[7] Lenz P, Ziegler J, Geiger A, et al. Sparse scene flow segmentation for moving object detection in urban environments[C]//IEEE Intelligent Vehicles Symposium. Piscataway, USA:IEEE, 2011:926-932.
[8] Whelan T, Salas-Moreno R F, Glocker B, et al. ElasticFusion[J]. International Journal of Robotics Research, 2016, 35(14):1697-1716.
[9] He K, Gkioxari G, Dollar P, et al. Mask R-CNN[C]//IEEE International Conference on Computer Vision. Piscataway, USA:IEEE, 2017:2980-2988.
[10] Sturm J, Engelhard N, Endres F, et al. A benchmark for the evaluation of RGB-D SLAM systems[C]//IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway, USA:IEEE, 2012:573-580.
[11] Lenz P, Ziegler J, Geiger A, et al. Sparse scene flow segmentation for moving object detection in urban environments[C]//IEEE Intelligent Vehicles Symposium. Piscataway, USA:IEEE, 2011:926-932.
[12] Li P, Qin T, Shen S. Stereo vision-based semantic 3D object and ego-motion tracking for autonomous driving[C]//European Conference on Computer Vision. Berlin, Germany:Springer, 2018:664-679.
[13] Girshick R. Fast R-CNN[C]//IEEE International Conference on Computer Vision. Piscataway, USA:IEEE, 2015:1440-1448.
[14] An L, Zhang X, Gao H, et al. Semantic segmentation-aided visual odometry for urban autonomous driving[J]. International Journal of Advanced Robotic Systems, 2017, 14(5):11pp.
[15] Badrinarayanan V, Kendall A, Cipolla R. SegNet:A deep convolutional encoder-decoder architecture for image segmentation[EB/OL]. (2016-10-10)[2018-11-25]. http://arxiv.org/abs/1511.00561.
[16] Zhou D, Frémont V, Quost B, et al. Moving object detection and segmentation in urban environments from a moving platform[J]. Image and Vision Computing, 2017, 68:76-87.
[17] Reddy N D, Singhal P, Chari V, et al. Dynamic body VSLAM with semantic constraints[C]//IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway, USA:IEEE, 2015:1897-1904.
[18] Yao E L, Zhang H X, Xu H, et al. Robust RGB-D visual odometry based on edges and points[J]. Robotics and Autonomous Systems, 2018, 107:209-220.
[19] Redmon J, Farhadi A. YOLOv3:An incremental improvement[EB/OL]. (2018-04-08)[2018-11-25]. http://arxiv.org/abs/1804.02767.
[20] Rublee E, Rabaud V, Konolige K, et al. ORB:An efficient alternative to SIFT or SURF[C]//IEEE International Conference on Computer Vision. Piscataway, USA:IEEE, 2011:2564-2571.
[21] Felzenszwalb P F, Huttenlocher D P. Distance transforms of sampled functions[J]. Theory of Computing, 2004, 8(19):415-428.
[22] Canny J. A computational approach to edge detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1986, 8(6):679-698.
[23] Mur-Artal R, Tardós J D. ORB-SLAM2:An open-source SLAM system for monocular, stereo, and RGB-D cameras[J]. IEEE Transactions on Robotics, 2017, 33(5):1255-1262.
[24] Sun Y, Liu M, Meng Q H. Improving RGB-D SLAM in dynamicenvironments:A motion removal approach[J]. Robotics and Autonomous Systems, 2017, 89:110-122.
[25] Technical University of Munich. Tools for the RGB-D benchmark[EB/OL]. (2016-10-10)[2018-11-25]. https://svncvpr.in.tum.de/cvpr-ros-pkg/trunk/rgbd_benchmark/rgbd_benchmark_tools/src/rgbd_benchmark_tools/.