An Enhancement Method for Underwater Images under Natural Illumination Based on Scene Depth Estimation
WANG Dan1, ZHANG Ziyu1, ZHAO Jinbao1, LIANG Wenfeng1, YANG Xieliu1, FAN Huijie2, TANG Yandong2
1. School of Mechanical Engineering, Shenyang Jianzhu University, Shenyang 110168, China; 2. State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang 110016, China
王丹, 张子玉, 赵金宝, 梁文峰, 杨谢柳, 范慧杰, 唐延东. 基于场景深度估计的自然光照水下图像增强方法[J]. 机器人, 2021, 43(3): 364-372.DOI: 10.13973/j.cnki.robot.200275.
WANG Dan, ZHANG Ziyu, ZHAO Jinbao, LIANG Wenfeng, YANG Xieliu, FAN Huijie, TANG Yandong. An Enhancement Method for Underwater Images under Natural Illumination Based on Scene Depth Estimation. ROBOT, 2021, 43(3): 364-372. DOI: 10.13973/j.cnki.robot.200275.
Abstract:To address the problems of blur, low contrast and color cast in underwater images, an enhancement method for underwater images under natural illumination based on scene depth estimation is proposed by combining the refined underwater imaging model. Firstly, the brightness information of an image is used to estimate the scene depth by minimum filtering and soft matting method, according to the prior theory that the brightness of the underwater scene under natural illumination conditions is generally proportional to its scene depth. Secondly, the backscatter of some discrete pixels is estimated by using the dark channel prior and the scene depth information, and then the backscatter in the whole image is fitted and removed by combining the refined underwater imaging model. On this basis, the chromatic-adaptation-based color correction method is applied to the direct component of the image to remove the color cast. In addition, the linear stretching method is adopted to improve the brightness and contrast of the restored image. The experiment results in various underwater scenes show that the proposed method can effectively remove the foggy blur caused by backscatter, and improve the contrast and correct the color cast of underwater images.
[1] Schettini R, Corchs S. Underwater image processing: State of the art of restoration and image enhancement methods[J]. EURASIP Journal on Advances in Signal Processing, 2010. DOI: 10.1155/2010/746052. [2] Mangeruga M, Bruno F, Cozza M, et al. Guidelines for underwater image enhancement based on benchmarking of different methods[J]. Remote Sensing, 2018, 10(10). DOI: 10.3390/ rs10101652. [3] Han M, Lyu Z, Qiu T, et al. A review on intelligence dehazing and color restoration for underwater images[J]. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2020, 50(5): 1820-1832. [4] Wang N, Zheng H Y, Zheng B. Underwater image restoration via maximum attenuation identification[J]. IEEE Access, 2017, 5: 18941-18952. [5] He K M, Sun J, Tang X O. Single image haze removal using dark channel prior[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(12): 2341-2353. [6] Peng Y-T, Cao K M, Cosman P C. Generalization of the dark channel prior for single image restoration[J]. IEEE Transactions on Image Processing, 2018, 27(6): 2856-2868. [7] 汤忠强,周波,戴先中,等. 基于改进DCP算法的水下机器人视觉增强[J].机器人, 2018, 40(2): 222-230. Tang Z Q, Zhou B, Dai X H, et al. Underwater robot visual enhancements based on the improved DCP algorithm[J]. Robot, 2018, 40(2): 222-230. [8] Cheng C-Y, Sung C-C, Chang H-H. Underwater image restoration by red-dark channel prior and point spread function deconvolution[C]//IEEE International Conference on Signal and Image Processing Applications. Piscataway, USA: IEEE, 2015: 110-115. [9] Drews P L J, Nascimento E R, Botelho S S C, et al. Underwater depth estimation and image restoration based on single images[J]. IEEE Computer Graphics and Applications, 2016, 36(2): 24-35. [10] Singhai J, Rawat P. Image enhancement method for underwater, ground and satellite images using brightness preserving histogram equalization with maximum entropy[C]//International Conference on Computational Intelligence and Multimedia Applications. Piscataway, USA: IEEE, 2007: 507-512. [11] Liu Y-C, Chan W-H, Chen Y-Q. Automatic white balance for digital still camera[J]. IEEE Transactions on Consumer Electronics, 1995, 41(3): 460-466. [12] Ao J, Ma C B. Adaptive stretching method for underwater image color correction[J]. International Journal of Pattern Recognition and Artificial Intelligence, 2018, 32(2). DOI: 10.1142/ S0218001418540010. [13] Fu X Y, Zhuang P X, Huang Y, et al. A retinex-based enhancing approach for single underwater image[C]//IEEE International Conference on Image Processing. Piscataway, USA: IEEE, 2014: 4572-4576. [14] Ancuti C, Ancuti C O, Haber T, et al. Enhancing underwater images and videos by fusion[C]//IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, USA: IEEE, 2012: 81-88. [15] Yang H-Y, Chen P-Y, Huang C-C, et al. Low complexity underwater image enhancement based on dark channel prior[C]//2nd International Conference on Innovations in Bio-inspired Computing and Applications. Piscataway, USA: IEEE, 2011: 17-20. [16] Codevilla F M, da Costa Botelho S S, Drews P, et al. Underwater single image restoration using dark channel prior[C]//Symposium on Automation and Computation for Naval, Offshore and Subsea. Piscataway, USA: IEEE, 2014: 18-21. [17] Galdran A, Pardo D, Picón A, et al. Automatic red-channel underwater image restoration[J]. Journal of Visual Communication and Image Representation, 2015, 26: 132-145. [18] Peng Y-T, Cosman P C. Underwater image restoration based on image blurriness and light absorption[J]. IEEE Transactions on Image Processing, 2017, 26(4): 1579-1594. [19] Akkaynak D, Treibitz T. Sea-thru: A method for removing water from underwater images[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway, USA: IEEE, 2019: 1682-1691. [20] Reza A M. Realization of the contrast limited adaptive histogram equalization (CLAHE) for real-time image enhancement[J]. Journal of VLSI Signal Processing Systems for Signal, Image and Video Technology, 2004, 38(1): 35-44. [21] Iqbal K, Odetayo M, James A, et al. Enhancing the low quality images using unsupervised colour correction method[C]//IEEE International Conference on Systems, Man and Cybernetics. Piscataway, USA: IEEE, 2010: 1703-1709. [22] van de Weijer J, Gevers T, Gijsenij A. Edge-based color constancy[J]. IEEE Transactions on Image Processing, 2007, 16(9): 2207-2214. [23] Provenzi E, Gatta C, Fierro M, et al. A spatially variant whitepatch and gray-world method for color image enhancement driven by local contrast[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008, 30(10): 1757-1770. [24] Buchsbaum G. A spatial processor model for object color perception[J]. Journal of the Franklin Institute, 1980, 310(1): 1-26. [25] Land E H. The retinex theory of color vision[J]. Scientific American, 1977, 237(6): 108-129. [26] Bianco G, Muzzupappa M, Bruno F, et al. A new color correction method for underwater imaging[J]. The International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, 2015, 40(W5): 25-32. [27] 杨燕,刘珑珑,张得欣,等. 结合自适应雾气估计的快速单幅图像去雾[J].光学精密工程, 2019, 27(10): 2263- 2271. Yang Y, Liu L L, Zhang D X, et al. Fast single image dehazing combined with adaptive haze estimation[J]. Optical Precision Engineering, 2019, 27(10): 2263-2271. [28] Johannes V K. Chromatic adaptation[M]//Sources of Color Science. Cambridge, USA: MIT Press, 1970: 120-127. [29] Yang X L, Yin C Y, Zhang Z Y, et al. Robust chromatic adaptation based color correction technology for underwater images[J]. Applied Sciences, 2020, 10(18). DOI: 10.3390/app 10186392. [30] Li C Y, Guo C L, Ren W Q, et al. An underwater image enhancement benchmark dataset and beyond[J]. IEEE Transactions on Image Processing, 2020, 29: 4376-4389. [31] Song W, Wang Y, Huang D M, et al. A rapid scene depth estimation model based on underwater light attenuation prior for underwater image restoration[C]//Pacific Rim Conference on Multimedia. Cham, Switzerland: Springer, 2018: 678-688. [32] Li C Y, Quo Y, Pang S, et al. Single underwater image restoration by blue-green channels dehazing and red channel correction[C]//IEEE International Conference on Acoustics, Speech and Signal Processing. Piscataway, USA: IEEE, 2016: 1731- 1735. [33] Wang Y, Song W, Fortino G, et al. An experimental-based review of image enhancement and image restoration methods for underwater imaging[J]. IEEE Access, 2019, 7: 140233-140251. [34] Wang Z, Bovik A C, Sheikh H R, et al. Image quality assessment: From error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 13(4): 600-612.