基于结构化深度学习的单目图像深度估计

doi:10.13973/j.cnki.robot.2017.0812

摘要
图/表
参考文献
相关文章 (15)

全文: PDF (633 KB) HTML ( ) 输出: BibTeX \| EndNote (RIS)
引用本文:
李耀宇, 王宏民, 张一帆, 卢汉清. 基于结构化深度学习的单目图像深度估计[J]. 机器人, 2017, 39(6): 812-819.DOI: 10.13973/j.cnki.robot.2017.0812. LI Yaoyu, WANG Hongmin, ZHANG Yifan, LU Hanqing. Structured Deep Learning Based Depth Estimation from a Monocular Image. ROBOT, 2017, 39(6): 812-819. DOI: 10.13973/j.cnki.robot.2017.0812.

摘要为从单目图像中提取到丰富的3D结构特征，并用以推测场景的深度信息，针对单目图像深度估计任务提出了一种结构化深度学习模型，该模型将一种新的多尺度卷积神经网络与连续条件随机场统一于一个深度学习框架中.卷积神经网络可以从图像中学习到相关特征表达，而连续条件随机场可以根据图像像素的位置、颜色信息对卷积神经网络输出进行优化，将二者参数以联合优化的方式进行学习可以提升模型的泛化性能.通过在NYU Depth数据集上的实验验证了模型的有效性与优越性，该模型预测结果的平均相对误差为0.187，均方根误差为0.074，对数空间平均误差为0.671.

	服务

	把本文推荐给朋友
	加入我的书架
	加入引用管理器
	E-mail Alert
	RSS
	作者相关文章
	李耀宇
	王宏民
	张一帆
	卢汉清

关键词 ：深度估计, 卷积神经网络, 单目图像, 结构化深度学习, 条件随机场

Abstract：For the purposes of extracting rich 3D structural features from a monocular image and inferring depth information for the scene, a structured deep learning model is proposed for the task of depth estimation from a monocular image. The model combines a novel multi-scale convolutional neural network (CNN) and continuous conditional random field (CCRF) in a unified deep learning framework. CNN can learn related feature representations from an image, and CCRF can optimize the output of CNN according to the position and color information of the image pixels. By jointly learning the parameters of CCRF and CNN, the generalization ability of the model can be improved. Experiments on NYU Depth dataset demonstrate the effectiveness and superiority of the model. The average relative error of the predictions of the model is 0.187, and the root mean squared error is 0.074, the average log₁₀ error is 0.671.

Key words： depth estimation convolutional neural network monocular image structured deep learning conditional random field

收稿日期: 2017-01-10 录用日期: 2017-06-05

TP391.41

基金资助:国家自然科学基金（61572500）

通讯作者: 王宏民,13904810908@163.com E-mail: 13904810908@163.com

作者简介: 李耀宇(1991-),男,硕士生.研究领域:图像与视频分析,网络多媒体搜索
王宏民(1962-),男,博士,教授.研究领域:电子信息与光电技术
张一帆(1982-),男,博士,副研究员.研究领域:图像与视频分析,网络多媒体搜索.

链接本文:

https://robot.sia.cn/CN/10.13973/j.cnki.robot.2017.0812 或 https://robot.sia.cn/CN/Y2017/V39/I6/812

[1] Ladicky L, Shi J B, Pollefeys M. Pulling things out of perspective[C]//IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, USA:IEEE, 2014:89-96.
[2] Shotton J, Girshick R, Fitzgibbon A, et al. Efficient human pose estimation from single depth images[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(12):2821-2840.
[3] Song S R, Xiao J X. Sliding shapes for 3D object detection in depth images[C]//13th European Conference on Computer Vision. Berlin, Germany:Springer, 2014:634-651.
[4] Scharstein D, Szeliski R, Zabih R. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms[C]//IEEE Workshop on Stereo and Multi-Baseline Vision. Piscataway, USA:IEEE, 2001:131-140.
[5] Hedau V, Hoiem D, Forsyth D. Thinking inside the box:Using appearance models and context based on room geometry[C]//11th European Conference on Computer Vision. Berlin, Germany:Springer, 2010:224-237.
[6] Gupta A, Efros A A, Hebert M. Blocks world revisited:Image understanding using qualitative geometry and mechanics[C]//11th European Conference on Computer Vision. Berlin, Germany:Springer, 2010:482-496.
[7] Lee D C, Gupta A, Hebert M, et al. Estimating spatial layout of rooms using volumetric reasoning about objects and surfaces[C]//Advances in Neural Information Processing Systems 23. 2010:1288-1296.
[8] Saxena A, Sun M, Ng A Y. Make3D:Learning 3D scene structure from a single still image[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2009, 31(5):824-840.
[9] Saxena A, Chung S H, Ng A Y. Learning depth from single monocular images[C]//Advances in Neural Information Processing Systems. 2005:1161-1168.
[10] Liu M M, Salzmann M, He X M. Discrete-continuous depth estimation from a single image[C]//IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, USA:IEEE, 2014:716-723.
[11] Eigen D, Puhrsch C, Fergus R. Depth map prediction from a single image using a multi-scale deep network[C]//Advances in Neural Information Processing Systems. 2014:2366-2374.
[12] Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation[C]//IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, USA:IEEE, 2015:3431-3440.
[13] Liu F Y, Shen C H, Lin G S, et al. Learning depth from single monocular images using deep convolutional neural fields[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(10):2024-2039.
[14] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition[C/OL]. (2015-04-10)[2017-01-01]. https://arxiv.org/pdf/1409.1556.pdf.
[15] Radosavljevic V, Vucetic S, Obradovic Z. Continuous conditional random fields for regression in remote sensing[C]//19th European Conference on Artificial Intelligence/6th Conference on Prestigious Applications of Intelligent Systems. Amsterdam, Netherlands:IOS Press, 2010:809-814.
[16] Bishop C M. Pattern recognition and machine learning (Information science and statistics)[M]. New York, USA:Springer-Verlag, 2006.
[17] Adams A, Baek J, Davis M A. Fast high-dimensional filtering using the permutohedral lattice[J]. Computer Graphics Forum, 2010, 29(2):753-762.
[18] Silberman N, Hoiem D, Kohli P, et al. Indoor segmentation and support inference from RGBD images[C]//12th European Conference on Computer Vision. Berlin, Germany:Springer, 2012:746-760.
[19] Jia Y Q, Shelhamer E, Donahue J, et al. Caffe:Convolutional architecture for fast feature embedding[C]//ACM Conference on Multimedia. New York, USA:ACM, 2014:675-678.