A Survey of Visual-Inertial SLAM for Mobile Robots
SHI Junyi1, ZHA Fusheng1,2, SUN Lining1, GUO Wei1, WANG Pengfei1, LI Mantian1
1. State Key Laboratory of Robotics and System, Harbin Institute of Technology, Harbin 150080, China; 2. Shenzhen Academy of Aerospace Technology, Shenzhen 518057, China
施俊屹, 查富生, 孙立宁, 郭伟, 王鹏飞, 李满天. 移动机器人视觉惯性SLAM研究进展[J]. 机器人, 2020, 42(6): 734-748.DOI: 10.13973/j.cnki.robot.190685.
SHI Junyi, ZHA Fusheng, SUN Lining, GUO Wei, WANG Pengfei, LI Mantian. A Survey of Visual-Inertial SLAM for Mobile Robots. ROBOT, 2020, 42(6): 734-748. DOI: 10.13973/j.cnki.robot.190685.
Abstract：Filtering-based and optimization-based methods are the two leading methods of VI-SLAM (visual-inertial simultaneous localization and mapping) used in the research field. Firstly, VI-SLAM based on these two methods are introduced, and its latest research progress and key issues are illustrated. Furthermore, the representative frameworks of VI-SLAM are compared. Finally, the future of visual-inertial SLAM is discussed.
[1] Lowry S, Sünderhauf N, Newman P, et al. Visual place recognition:A survey[J]. IEEE Transactions on Robotics, 2015, 32(1):1-19. [2] Gálvez-López D, Tardos J D. Bags of binary words for fast place recognition in image sequences[J]. IEEE Transactions on Robotics, 2012, 28(5):1188-1197. [3] 刘强,段富海,桑勇,等.复杂环境下视觉SLAM闭环检测方法综述[J].机器人,2019,41(1):112-123,136. Liu Q, Duan F H, Sang Y, et al. A survey of loop-closure detection method of visual SLAM in complex environments[J]. Robot, 2019, 41(1):112-123,136. [4] Cadena C, Carlone L, Carrillo H, et al. Past, present, and future of simultaneous localization and mapping:Toward the robust-perception age[J]. IEEE Transactions on Robotics, 2016, 32(6):1309-1332. [5] Durrant-Whyte H, Bailey T. Simultaneous localization and mapping:Part I[J]. IEEE Robotics & Automation Magazine, 2006, 13(2):99-110. [6] Bailey T, Durrant-Whyte H. Simultaneous localization and mapping (SLAM):Part II[J]. IEEE Robotics & Automation Magazine, 2006, 13(3):108-117. [7] Thrun S, Burgard W, Fox D. Probabilistic robotics[M]. Cambridge, USA:MIT press, 2005. [8] Strasdat H, Davison A J, Montiel J M M, et al. Double window optimisation for constant time visual SLAM[C]//IEEE International Conference on Computer Vision. Piscataway, USA:IEEE, 2011:2352-2359. [9] Dissanayake G, Huang S D, Wang Z, et al. A review of recent developments in simultaneous localization and mapping[C]//6th International Conference on Industrial and Information Sys-tems. Piscataway, USA:IEEE, 2011:477-482. [10] Kelly A. Mobile robotics:Mathematics, models, and methods[M]. Cambridge, UK:Cambridge University Press, 2013. [11] 孙立宁,周兆英,龚振邦.MEMS国内外发展状况及我国MEMS发展战略的思考[J].机器人技术与应用,2002(2):2-4. Sun L N, Zhou Z Y, Gong Z B. Thoughts on the development status of MEMS and MEMS development strategy[J]. Robot Technique and Application, 2002(2):2-4. [12] Forster C, Carlone L, Dellaert F, et al. On-manifold preintegration for real-time visual-inertial odometry[J]. IEEE Transac-tions on Robotics, 2017, 33(1):1-21. [13] Barfoot T D. State estimation for robotics[M]. Cambridge, UK:Cambridge University Press, 2017. [14] Klein G, Murray D. Parallel tracking and mapping for small AR workspaces[C]//6th IEEE and ACM International Symposium on Mixed and Augmented Reality. Piscataway, USA:IEEE, 2007. DOI:10.1109/ISMAR.2007.4538852. [15] Forster C, Zhang Z C, Gassner M, et al. SVO:Semidirect visual odometry for monocular and multicamera systems[J]. IEEE Transactions on Robotics, 2017, 33(2):249-265. [16] Engel J, Schöps T, Cremers D. LSD-SLAM:Large-scale direct monocular SLAM[C]//European Conference on Computer Vision. Cham, Switzerland:Springer, 2014:834-849. [17] Mur-Artal R, Montiel J M M, Tardós J D. ORB-SLAM:A versatile and accurate monocular SLAM system[J]. IEEE Transactions on Robotics, 2015, 31(5):1147-1163. [18] Engel J, Koltun V, Cremers D. Direct sparse odometry[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(3):611-625. [19] Wolf H. Odometry and insect navigation[J]. Journal of Experimental Biology, 2011, 214(10):1629-1641. [20] 陈诺.基于UWB与惯导融合的室内导航系统研究[D].哈尔滨:哈尔滨工业大学,2018. Chen N. Research on indoor navigation system based on UWB and INS fusion[D]. Harbin:Harbin Institute of Technology, 2018. [21] Wang P F, Chen N, Zha F S, et al. Research on adaptive Monte Carlo location algorithm aided by ultra-wideband array[C]//13th World Congress on Intelligent Control and Automation. Piscataway, USA:IEEE, 2018:566-571. [22] Gui J J, Gu D B, Wang S, et al. A review of visual inertial odometry from filtering and optimisation perspectives[J]. Advanced Robotics, 2015, 29(20):1289-1301. [23] Weiss S, Achtelik M W, Lynen S, et al. Real-time onboard visual-inertial state estimation and self-calibration of MAVs in unknown environments[C]//IEEE International Conference on Robotics and Automation. Piscataway, USA:IEEE, 2012:957-964. [24] Li M, Mourikis A I. Improving the accuracy of EKF-based visual-inertial odometry[C]//IEEE International Conference on Robotics and Automation. Piscataway, USA:IEEE, 2012:828-835. [25] Martinelli A. Closed-form solution of visual-inertial structure from motion[J]. International Journal of Computer Vision, 2014, 106(2):138-152. [26] Jones E S, Soatto S. Visual-inertial navigation, mapping and localization:A scalable real-time causal approach[J]. International Journal of Robotics Research, 2011, 30(4):407-430. [27] Kelly J, Sukhatme G S. Visual-inertial sensor fusion:Localization, mapping and sensor-to-sensor self-calibration[J]. International Journal of Robotics Research, 2011, 30(1):56-79. [28] Mourikis A I, Roumeliotis S I. A multi-state constraint Kalman filter for vision-aided inertial navigation[C]//IEEE International Conference on Robotics and Automation. Piscataway, USA:IEEE, 2007:3565-3572. [29] Kalman R E. A new approach to linear filtering and prediction problems[J]. Journal of Basic Engineering, 1960, 82(1):35-45. [30] Prince S J D. Computer vision:Models, learning, and inference[M]. Cambridge, UK:Cambridge University Press, 2012. [31] McGee L A, Schmidt S F. Discovery of the Kalman filter as a practical tool for aerospace and industry[R]. Washington, USA:NASA, 1985. [32] Julier S, Uhlmann J K. A general method for approximating nonlinear transformations of probability distributions[R]. Oxford, UK:University of Oxford, 1996. [33] Thrun S, Fox D, Burgard W, et al. Robust Monte Carlo localization for mobile robots[J]. Artificial Intelligence, 2001, 128(1-2):99-141. [34] Weiss S, Achtelik M W, Lynen S, et al. Monocular vision for long-term micro aerial vehicle state estimation:A compendium[J]. Journal of Field Robotics, 2013, 30(5):803-831. [35] Newcombe R A, Lovegrove S J, Davison A J. DTAM:Dense tracking and mapping in real-time[C]//IEEE International Conference on Computer Vision. Piscataway, USA:IEEE, 2011:2320-2327. [36] Henry P, Krainin M, Herbst E, et al. RGB-D mapping:Using Kinect-style depth cameras for dense 3D modeling of indoor environments[J]. International Journal of Robotics Research, 2012, 31(5):647-663. [37] Smith R, Self M, Cheeseman P. Estimating uncertain spatial relationships in robotics[M]//Autonomous Robot Vehicles. New York, USA:Springer, 1990:167-193. [38] Durrant-Whyte H F. Uncertain geometry in robotics[J]. IEEE Journal on Robotics and Automation, 1988, 4(1):23-31. [39] Smith R C, Cheeseman P. On the representation and estimation of spatial uncertainty[J]. International Journal of Robotics Research, 1986, 5(4):56-68. [40] Smith R, Self M, Cheeseman P. A stochastic map for uncertain spatial relationships[C]//4th International Symposium of Robotics Research. Cambridge, USA:MIT Press, 1988:467-474. [41] Li M Y, Mourikis A I. High-precision, consistent EKF-based visual-inertial odometry[J]. International Journal of Robotics Research, 2013, 32(6):690-711. [42] Zhu A Z, Atanasov N, Daniilidis K. Event-based visual inertial odometry[C]//30th IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway, USA:IEEE, 2017:5816-5824. [43] Zheng X, Moratto Z, Li M Y, et al. Photometric patch-based visual-inertial odometry[C]//IEEE International Conference on Robotics and Automation. Piscataway, USA:IEEE, 2017:3264-3271. [44] Sun K, Mohta K, Pfrommer B, et al. Robust stereo visual inertial odometry for fast autonomous flight[J]. IEEE Robotics and Automation Letters, 2018, 3(2):965-972. [45] Huai Z, Huang G Q. Robocentric visual-inertial odometry[C]//IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway, USA:IEEE, 2018:6319-6326. [46] Bloesch M, Omani S, Hutter M, et al. Robust visual inertial odometry using a direct EKF-based approach[C]//IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway, USA:IEEE, 2015:298-304. [47] Bloesch M, Burri M, Omari S, et al. Iterated extended Kalman filter based visual-inertial odometry using direct photometric feedback[J]. International Journal of Robotics Research, 2017, 36(10):1053-1072. [48] Schneider T, Dymczyk M, Fehr M, et al. Maplab:An open framework for research in visual-inertial mapping and localization[J]. IEEE Robotics and Automation Letters, 2018, 3(3):1418-1425. [49] Faessler M, Fontana F, Forster C, et al. Autonomous, vision-based flight and live dense 3D mapping with a quadrotor micro aerial vehicle[J]. Journal of Field Robotics, 2016, 33(4):431-450. [50] Rosten E, Drummond T. Machine learning for high-speed corner detection[C]//9th European Conference on Computer Vision. Berlin, Germany:Springer, 2006:430-443. [51] Lynen S, Achtelik M W, Weiss S, et al. A robust and modular multi-sensor fusion approach applied to MAV navigation[C]//IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway, USA:IEEE, 2013:3923-3929. [52] Triggs B, McLauchlan P F, Hartley R I, et al. Bundle adjustment-A modern synthesis[C]//International Workshop on Vision Algorithms. Berlin, Germany:Springer, 2000:298-372. [53] Kümmerle R, Grisetti G, Strasdat H, et al. g2o:A general framework for graph optimization[C]//IEEE International Conference on Robotics and Automation. Piscataway, USA:IEEE, 2011:3607-3613. [54] Agarwal S, Mierle K. Ceres solver[DB/OL]. (2012-05-01)[2019-06-15]. http://ceres-solver.org. [55] Dellaert F. Factor graphs and GTSAM:A hands-on introduction[R]. Atlanta, USA:Georgia Institute of Technology, 2012. [56] Leutenegger S, Lynen S, Bosse M, et al. Keyframe-based visual-inertial odometry using nonlinear optimization[J]. International Journal of Robotics Research, 2015, 34(3):314-334. [57] Leutenegger S, Furgale P, Rabaud V, et al. Keyframe-based visual-inertial SLAM using nonlinear optimization[C]//Robotics:Science and Systems. 2013. DOI:10.15607/RSS. 2013.IX.037. [58] Mourikis A I, Roumeliotis S I. A dual-layer estimator architecture for long-term localization[C]//IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, USA:IEEE, 2008:1310-1317. [59] Sibley G, Matthies L, Sukhatme G. Sliding window filter with application to planetary landing[J]. Journal of Field Robotics, 2010, 27(5):587-608. [60] Dong-Si T C, Mourikis A I. Motion tracking with fixed-lag smoothing:Algorithm and consistency analysis[C]//IEEE International Conference on Robotics and Automation. Piscataway, USA:IEEE, 2011:5655-5662. [61] Strasdat H, Montiel J M M, Davison A J. Real-time monocular SLAM:Why filter?[C]//IEEE International Conference on Robotics and Automation. Piscataway, USA:IEEE, 2010:2657-2664. [62] Nerurkar E D, Wu K J, Roumeliotis S I. C-KLAM:Constrained keyframe-based localization and mapping[C]//IEEE International Conference on Robotics and Automation. Piscataway, USA:IEEE, 2014:3638-3643. [63] Klein G, Murray D. Parallel tracking and mapping on a camera phone[C]//8th IEEE International Symposium on Mixed and Augmented Reality. Piscataway, USA:IEEE, 2009:83-86. [64] Maybeck P S. Stochastic models, estimation, and control[M]. New York, USA:Academic Press, 1982. [65] Huang G Q, Mourikis A I, Roumeliotis S I. An observability-constrained sliding window filter for SLAM[C]//IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway, USA:IEEE, 2011:65-72. [66] Hesch J A, Kottas D G, Bowman S L, et al. Camera-IMU-based localization:Observability analysis and consistency improvement[J]. International Journal of Robotics Research, 2014, 33(1):182-201. [67] Harris C, Stephens M. A combined corner and edge detector[C]//4th Alvey Vision Conference. 1988:147-151. [68] Leutenegger S, Chli M, Siegwart R. BRISK:Binary robust invariant scalable keypoints[C]//IEEE International Conference on Computer Vision. Piscataway, USA:IEEE, 2011:2548-2555. [69] Qin T, Li P L, Shen S J. VINS-mono:A robust and versatile monocular visual-inertial state estimator[J]. IEEE Transactions on Robotics, 2018, 34(4):1004-1020. [70] Lucas B D, Kanade T. An iterative image registration technique with an application to stereo vision[C]//7th International Joint Conference on Artificial Intelligence. San Francisco, USA:Morgan Kaufmann Publishers Inc, 1981:674-679. [71] Yang Z F, Shen S J. Monocular visual-inertial state estimation with online initialization and camera-IMU extrinsic calibration[J]. IEEE Transactions on Automation Science and Engineering, 2017, 14(1):39-51. [72] Qin T, Shen S J. Robust initialization of monocular visual-inertial estimation on aerial robots[C]//IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway, USA:IEEE, 2017:4225-4232. [73] Li P L, Qin T, Hu B T, et al. Monocular visual-inertial state estimation for mobile augmented reality[C]//IEEE International Symposium on Mixed and Augmented Reality. Piscataway, USA:IEEE, 2017:11-21. [74] Qin T, Shen S J. Online temporal calibration for monocular visual-inertial systems[C]//IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway, USA:IEEE, 2018:3662-3669. [75] Liu H M, Chen M Y, Zhang G, et al. ICE-BA:Incremental, consistent and efficient bundle adjustment for visual-inertial SLAM[C]//31 st IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway, USA:IEEE, 2018:1974-1982. [76] Mur-Artal R, Tardós J D. Visual-inertial monocular SLAM with map reuse[J]. IEEE Robotics and Automation Letters, 2017, 2(2):796-803. [77] von Stumberg L, Usenko V, Cremers D. Direct sparse visual-inertial odometry using dynamic marginalization[C]//IEEE International Conference on Robotics and Automation. Piscataway, USA:IEEE, 2018:2510-2517. [78] Usenko V, Engel J, Stückler J, et al. Direct visual-inertial odometry with stereo cameras[C]//IEEE International Conference on Robotics and Automation. Piscataway, USA:IEEE, 2016:1885-1892. [79] Rublee E, Rabaud V, Konolige K, et al. ORB:An efficient alternative to SIFT or SURF[C]//IEEE International Conference on Computer Vision. Piscataway, USA:IEEE, 2011:2564-2571. [80] Calonder M, Lepetit V, Strecha C, et al. BRIEF:Binary robust independent elementary features[C]//11th European Conference on Computer Vision. Berlin, Germany:Springer, 2010:778-792. [81] Mur-Artal R, Tardós J D. Fast relocalisation and loop closing in keyframe-based SLAM[C]//IEEE International Conference on Robotics and Automation. Piscataway, USA:IEEE, 2014:846-853. [82] Mur-Artal R, Tardós J D. ORB-SLAM:Tracking and mapping recognizable features[C]//Robotics:Science and Systems (RSS) Workshop on Multi View Geometry. 2014. [83] Mur-Artal R, Tardós J D. Probabilistic semi-dense mapping from highly accurate feature-based monocular SLAM[C]//Robotics:Science and Systems. 2015. DOI:10.15607/RSS. 2015.XI.041. [84] Mur-Artal R, Tardós J D. ORB-SLAM2:An open-source SLAM system for monocular, stereo, and RGB-D cameras[J]. IEEE Transactions on Robotics, 2017, 33(5):1255-1262. [85] 张玉龙.基于关键帧的视觉惯性SLAM算法[D].天津:天津大学,2017. Zhang Y L. Keyframe-based visual-inertial SLAM algorithm[D]. Tianjin:Tianjin University, 2017. [86] 张玉龙,张国山.基于关键帧的视觉惯性SLAM闭环检测算法[J].计算机科学与探索,2018,12(11):1777-1787. Zhang Y L, Zhang G S. Loop-closing detection algorithm of keyframe-based visual-inertial SLAM[J]. Journal of Frontiers of Computer Science and Technology, 2018, 12(11):1777-1787. [87] 代维.室内环境下基于视觉!/!惯性!/!里程计的自主定位技术研究[D].南京:东南大学,2018. Dai W. Research on autonomous positioning technology based on visual inertial mileage meter in indoor environment[D]. Nanjing:Southeast University, 2018. [88] 徐晓苏,代维,杨博,等.室内环境下基于图优化的视觉惯性SLAM方法[J].中国惯性技术学报,2017,25(3):313-319. Xu X S, Dai W, Yang B, et al. Visual-aid inertial SLAM method based on graph optimization in indoor[J]. Journal of Chinese Inertial Technology, 2017, 25(3):313-319. [89] Jung S H, Taylor C J. Camera trajectory estimation using inertial sensor measurements and structure from motion results[C]//IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Piscataway, USA:IEEE, 2001:732-737. [90] Strelow D, Singh S. Motion estimation from image and inertial measurements[J]. International Journal of Robotics Research, 2004, 23(12):1157-1195. [91] Bryson M, Johnson-Roberson M, Sukkarieh S. Airborne smoothing and mapping using vision and inertial sensors[C]//IEEE International Conference on Robotics and Automation. Piscataway, USA:IEEE, 2009:3143-3148. [92] Indelman V, Williams S, Kaess M, et al. Information fusion in navigation systems via factor graph based incremental smoothing[J]. Robotics and Autonomous Systems, 2013, 61(8):721-738. [93] Patron-Perez A, Lovegrove S, Sibley G. A spline-based trajectory representation for sensor fusion and rolling shutter cameras[J]. International Journal of Computer Vision, 2015, 113(3):208-219. [94] Kaess M, Ranganathan A, Dellaert F. iSAM:Incremental smoothing and mapping[J]. IEEE Transactions on Robotics, 2008, 24(6):1365-1378. [95] Kaess M, Johannsson H, Roberts R, et al. iSAM2:Incremental smoothing and mapping using the Bayes tree[J]. International Journal of Robotics Research, 2012, 31(2):216-235. [96] Dellaert F, Kaess M. Factor graphs for robot perception[J]. Foundations and Trends in Robotics, 2017, 6(1-2):1-139. [97] Koller D, Friedman N. Probabilistic graphical models:Principles and techniques[M]. Cambridge, USA:MIT Press, 2009. [98] Kaess M, Dellaert F, Roberts R, et al. The Bayes tree:Enabling incremental reordering and fluid relinearization for online mapping[R]. Cambridge, USA:MIT Press, 2010. [99] Huang G Q. Improving the consistency of nonlinear estimators:Analysis, algorithms, and applications[D]. Minneapolis, USA:University of Minnesota, 2012. [100] Martinelli A. State estimation based on the concept of continuous symmetry and observability analysis:The case of calibration[J]. IEEE Transactions on Robotics, 2011, 27(2):239-255. [101] Weiss S M. Vision based navigation for micro helicopters[D]. Zurich, Switzerland:ETH, 2012. [102] Martinelli A. Visual-inertial structure from motion:Observability vs minimum number of sensors[C]//IEEE International Conference on Robotics and Automation. Piscataway, USA:IEEE, 2014:1020-1027. [103] Hesch J A, Kottas D G, Bowman S L, et al. Towards consistent vision-aided inertial navigation[C]//10th Workshop on the Algorithmic Foundations of Robotics. Berlin, Germany:Springer, 2013:559-574. [104] Huang G Q, Kaess M, Leonard J J. Towards consistent visual-inertial navigation[C]//IEEE International Conference on Robotics and Automation. Piscataway, USA:IEEE, 2014:4926-4933. [105] Kottas D G, Hesch J A, Bowman S L, et al. On the consistency of vision-aided inertial navigation[C]//13th International Symposium on Experimental Robotics. Berlin, Germany:Springer, 2013:303-317. [106] Martinelli A. Observability properties and deterministic algorithms in visual-inertial structure from motion[J]. Foundations and Trends in Robotics, 2013, 3(3):139-209. [107] Huang G Q, Mourikis A I, Roumeliotis S I. A first-estimates Jacobian EKF for improving SLAM consistency[C]//11th International Symposium on Experimental Robotics. Berlin, Germany:Springer, 2009:373-382. [108] Guo C X, Roumeliotis S I. IMU-RGBD camera 3D pose estimation and extrinsic calibration:Observability analysis and consistency improvement[C]//IEEE International Conference on Robotics and Automation. Piscataway, USA:IEEE, 2013:2935-2942. [109] Martinelli A. Visual-inertial structure from motion:Observability and resolvability[C]//IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway, USA:IEEE, 2013:4235-4242. [110] Hesch J A, Kottas D G, Bowman S L, et al. Observability-constrained vision-aided inertial navigation[R]. Minneapolis, USA:University of Minnesota, 2012. [111] Civera J, Bueno D R, Davison A J, et al. Camera self-calibration for sequential Bayesian structure from motion[C]//IEEE International Conference on Robotics and Automation. Piscataway, USA:IEEE, 2009:403-408. [112] Mirzaei F M, Roumeliotis S I. A Kalman filter-based algorithm for IMU-camera calibration:Observability analysis and performance evaluation[J]. IEEE Transactions on Robotics, 2008, 24(5):1143-1156. [113] Li M Y, Mourikis A I. Online temporal calibration for camera-IMU systems:Theory and algorithms[J]. International Journal of Robotics Research, 2014, 33(7):947-964. [114] Kelly J, Sukhatme G S. A general framework for temporal calibration of multiple proprioceptive and exteroceptive sensors[C]//12th International Symposium on Experimental Robotics. Berlin, Germany:Springer, 2014:195-209. [115] Indelman V, Williams S, Kaess M, et al. Factor graph based incremental smoothing in inertial navigation systems[C]//15th International Conference on Information Fusion. Piscataway, USA:IEEE, 2012:2154-2161. [116] Indelman V, Williams S, Kaess M, et al. Information fusion in navigation systems via factor graph based incremental smoothing[J]. Robotics and Autonomous Systems, 2013, 61(8):721-738. [117] Indelman V, Melim A, Dellaert F. Incremental light bundle adjustment for robotics navigation[C]//IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway, USA:IEEE, 2013:1952-1959. [118] Shen S J. Autonomous navigation in complex indoor and outdoor environments with micro aerial vehicles[D]. Philadelphia, USA:University of Pennsylvania, 2014. [119] Keivan N, Patron-Perez A, Sibley G. Asynchronous adaptive conditioning for visual-inertial SLAM[C]//14th International Symposium on Experimental Robotics. Cham, Switzerland:Springer, 2016:309-321. [120] Lupton T, Sukkarieh S. Visual-inertial-aided navigation for high-dynamic motion in built environments without initial conditions[J]. IEEE Transactions on Robotics, 2012, 28(1):61-76. [121] Forster C, Carlone L, Dellaert F, et al. IMU preintegration on manifold for efficient visual-inertial maximum-a-posteriori estimation[C]//Robotics Science and Systems. 2015. DOI:10.15607/rss.2015.xi.006. [122] Shen S J, Michael N, Kumar V. Tightly-coupled monocular visual-inertial fusion for autonomous flight of rotorcraft MAVs[C]//IEEE International Conference on Robotics and Automation. Piscataway, USA:IEEE, 2015:5303-5310. [123] Huang W B, Liu H. Online initialization and automatic camera-IMU extrinsic calibration for monocular visual-inertial SLAM[C]//IEEE International Conference on Robotics and Automation. Piscataway, USA:IEEE, 2018:5182-5189. [124] Delmerico J, Scaramuzza D. A benchmark comparison of monocular visual-inertial odometry algorithms for flying robots[C]//IEEE International Conference on Robotics and Automation. Piscataway, USA:IEEE, 2018:2502-2509. [125] Burri M, Nikolic J, Gohl P, et al. The EuRoC micro aerial vehicle datasets[J]. International Journal of Robotics Research, 2016, 35(10):1157-1163. [126] Geiger A, Lenz P, Stiller C, et al. Vision meets robotics:The KITTI dataset[J]. International Journal of Robotics Research, 2013, 32(11):1231-1237. [127] Schubert D, Goll T, Demmel N, et al. The TUM VI benchmark for evaluating visual-inertial odometry[C]//IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway, USA:IEEE, 2018:1680-1687. [128] Majdik A L, Till C, Scaramuzza D. The Zurich urban micro aerial vehicle dataset[J]. International Journal of Robotics Research, 2017, 36(3):269-273. [129] Blanco-Claraco J L, Moreno-Dueñas F Á, González-Jiménez J. The Málaga urban dataset:High-rate stereo and LiDAR in a realistic urban scenario[J]. International Journal of Robotics Research, 2014, 33(2):207-214. [130] Carlevaris-Bianco N, Ushani A K, Eustice R M. University of Michigan North Campus long-term vision and lidar dataset[J]. International Journal of Robotics Research, 2016, 35(9):1023-1035. [131] Pfrommer B, Sanket N, Daniilidis K, et al. PennCOSYVIO:A challenging visual inertial odometry benchmark[C]//IEEE International Conference on Robotics and Automation. Piscataway, USA:IEEE, 2017:3847-3854. [132] 赵洋,刘国良,田国会,等.基于深度学习的视觉SLAM综述[J].机器人,2017,39(6):889-896. Zhao Y, Liu G L, Tian G H, et al. A survey of visual SLAM based on deep learning[J]. Robot, 2017, 39(6):889-896. [133] 伍锡如,黄国明,孙立宁.基于深度学习的工业分拣机器人快速视觉识别与定位算法[J].机器人,2016,38(6):711-719.Wu X R, Huang G M, Sun L N. Fast visual identification and location algorithm for industrial sorting robots based on deep learning[J]. Robot, 2016, 38(6):711-719. [134] Wang S, Clark R, Wen H, et al. End-to-end, sequence-to-sequence probabilistic visual odometry through deep neural networks[J]. International Journal of Robotics Research, 2018, 37(4-5):513-542. [135] Wang S, Clark R, Wen H, et al. DeepVO:Towards end-to-end visual odometry with deep recurrent convolutional neural networks[C]//IEEE International Conference on Robotics and Automation. Piscataway, USA:IEEE, 2017:2043-2050. [136] Abolfazli Esfahani, M, Wang H, Wu K, et al. AbolDeepIO:A novel deep inertial odometry network for autonomous vehicles[J]. IEEE Transactions on Intelligent Transportation Systems, 2020, 21(5):1941-1950. [137] Clark R, Wang S, Wen H, et al. VINet:Visual-inertial odometry as a sequence-to-sequence learning problem[C]//31st AAAI Conference on Artificial Intelligence. Palo Alto, USA:AAAI, 2017:3995-4001. [138] Ilg E, Mayer N, Saikia T, et al. FlowNet 2.0:Evolution of optical flow estimation with deep networks[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway, USA:IEEE, 2017. DOI:10.1109/CVPR.2017.179. [139] Hochreiter S, Schmidhuber J. Long short-term memory[J]. Neural Computation, 1997, 9(8):1735-1780. [140] Shamwell E J, Lindgren K, Leung S, et al. Unsupervised deep visual-inertial odometry with online error correction for RGB-D imagery[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019. DOI:10.1109/TPAMI.2019.2909895. [141] Chen C, Rosa S, Miao Y, et al. Selective sensor fusion for neural visual-inertial odometry[C]//IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, USA:IEEE, 2019:10534-10543. [142] Lee H, McCrink M H, Gregory J W. Visual-inertial odometry for unmanned aerial vehicle using deep learning[C]//AIAA Scitech 2019 Forum. Reston, USA:AIAA, 2019. DOI:10.2514/6.2019-1410. [143] Wan G W, Yang X L, Cai R L, et al. Robust and precise vehicle localization based on multi-sensor fusion in diverse city scenes[C]//IEEE International Conference on Robotics and Automation. Piscataway, USA:IEEE, 2018:4670-4677. [144] Li P L, Qin T, Hu B T, et al. Monocular visual-inertial state estimation for mobile augmented reality[C]//IEEE International Symposium on Mixed and Augmented Reality. Piscataway, USA:IEEE, 2017:11-21. [145] 林辉灿,吕强,张洋,等.稀疏和稠密的VSLAM的研究进展[J].机器人,2016,38(5):621-631. Lin H C, Lü Q, Zhang Y, et al. The sparse and dense VSLAM:A survey[J]. Robot, 2016, 38(5):621-631. [146] Nikolskiy V P, Stegailov V V, Vecher V S. Efficiency of the Tegra K1 and X1 systems-on-chip for classical molecular dynamics[C]//International Conference on High Performance Computing & Simulation. Piscataway, USA:IEEE, 2016:682-689.
