STUDY OF IMAGE ENHANCEMENT AND RESTORATION TECHNIQUES FOR UNDERWATER IMAGES

Abstract

Underwater image processing serves several purposes across various domains such as marine biology for ecology, oceanography in environmental monitoring, underwater archaeology, search for rescue operations, underwater surveillance for security, underwater photography in filmmaking, commercial diving, and many more. In this thesis, we have investigated the problem of degraded underwater images. Over the decade, many efforts made to develop algorithms for enhancing degraded underwater images, but still it is a challenging task due to limited visibility, non-uniform illumination, and diminished contrast. To start with, The most popular and prominent existing state-of-the-art methods have been reviewed. Based on the literature, it finds that the major challenges are imposed by underwater medium properties. At deeper depths, the one drawback of degraded underwater images is diminishing light attenuation visibility. Improvement in the clarity of degraded underwater images artificial lighting sources are employed to extend visibility, but they often create non-illumination patterns. Meanwhile, marine snow with small floating particles comes within the camera’s range which causes forward and backward scattering of light energy in the image plane. This leads to a low-contrast and degraded appearance in the captured images. Furthermore, light undergoes attenuation as it propagates laterally, diminishing the amount of light energy reaching the camera and causing a loss of natural color. In this thesis, our motivation lies in tackling challenges associated with underwater imaging, such as restricted visibility, non-uniform illumination, reduced contrast, noise removal, blurring of images, artifacts removal, structural preservation, and color restoration. The aim is to develop algorithms for enhancing and restoring underwater images, to overcome the limitations of existing state-of-the-art methods. Consequently, the main objective of underwater image enhancement is to eliminate the hazy veil and adjust color using a single underwater image. Image restoration involves several challenges such as noise removal, blurring of images, artifacts removal, structural preservation, and color restoration. This thesis comprises of total six proposed methods in three chapters to address the above mentioned problems. First, a novel underwater image enhancement method based on multiscale decomposition and brightness adjustment is presented in the third chapter. The v proposed method is constructed by decomposing the degraded underwater image into illumination and reflectance using the weighted least square filter. Further, gamma correction is applied to the base layer. The brightness adjustment is performed on the illumination component using the sigmoid function. The effectiveness of the proposed method is validated by comparison of it with state-of-the-art methods on multiple standard datasets such as underwater image enhancement benchmark (UIEB), real-world underwater image enhancement (RUIE), a dataset of real-world underwater video of artifacts (DRUVA) and underwater-45 (U45). A novel method for image restoration is proposed in the third chapter of this thesis. It is based on illumination, reflectance, and white balance technique. In the first stage, we applied the white balance technique to the degraded underwater input image, and the obtained image is converted into YCbCr color space. In the second stage, the Y-component of the YCbCr color space has decomposed into illumination and reflectance. The reflectance is improved by contrast-limited adaptive histogram equalization (CLAHE) and the histogram definition of the illumination image is modified by creating a weighted histogram based on Bi-log transformation. The improved reflectance and illumination are combined to get an enhanced Y-component of YCbCr color space. Finally, the Cr component, Cb-component, and restored Y-component are concatenated to get the restored image. The effectiveness of the proposed method is tested by comparing it with the existing state-of-the-art method on the UIEB dataset. A new method is proposed for image enhancement using the blending technique in the fourth chapter. In the first stage of this method, color correction of the underwater degraded image has been done. In the second stage of this method, the white balance technique and contrast-limited adaptive histogram equalization are applied in parallel to the color corrected image of the first stage. Then RGB output image corresponding to the white balance technique (Known as input1) and contrast-limited adaptive histogram equalization (Known as input2) are converted into YCbCr color space and finally decomposed into Y component, Cb-component, and Cr-component. Next, the Laplacian, saliency, and saturation filtering are applied to the Y-component, and normalization is performed on the obtained image as the third stage of this method. Finally, the blending process is used to get an enhanced image. The proposed method is evaluated on the UIEB, RUIE, and U45 datasets in terms of parameters such as patch-based contrast quality index (PCQI), underwater image quality measure (UIQM), underwater color image quality evaluation vi (UCIQE), peak signal to noise ratio (PSNR), absolute mean brightness enhancement (AMBE), contrast per pixel (CPP), discrete entropy (DE), modified measure of enhancement (MEME), and structure similarity index measure (SSIM). Next, a novel method has been proposed for underwater image restoration based on color correction, and empirical mode decomposition in fourth chapter. In the first stage of this method, color correction is applied to degraded underwater images. Then color color corrected image has been converted into HSV color space. The empirical mode decomposition is applied to all components (i.e., H-component, S-component, and V component) of HSV color space. The weighted sum of the first four IMFs of all three components is used for the restoration of HSV color space. Finally, this HSV color space is converted into RGB color space. The proposed method is compared with existing state-of the-art methods on publicly available datasets such as UIEB and U45. The effectiveness of the proposed method is evaluated in terms of the visual quality of the image and various parameters such as SSIM, CPP, measure of enhancement (EME), UCIQE, UIQM, PCQI, AMBE, PSNR, and DE. Two different novel methods are used in the fifth chapter to address the problem of enhancement and restoration of underwater images and videos. The first proposed method deals with the problem of image enhancement, whereas the second proposed method is used for image enhancement and restoration. The first method proposed in the fifth chapter for underwater image enhancement using principal component analysis based on the fusion of background light and transmission optimization. In the first stage, the background light (Known as 𝐼1) is evaluated from input underwater images. Parallalley, the RGB color space has been converted into the international commission on illumination-luminance chrominance (CIE-Lab) color space. In the second stage, the transmission estimation has been done on the L-component of the CIE-Lab color space. Further, transmission optimization is applied to estimated transmission. Now, the A-component, B-component, and optimised transmission of L component of CIE-Lab color space are multiplied to obtain an enhanced image (Known as 𝐼2) in CIE-Lab color space. In the third stage, we evaluated the principal components of image 𝐼1 (Known as 𝑃𝐶1 ) and image 𝐼2 (Known as 𝑃𝐶2 ). Then, principal component 𝑃𝐶1 fused with the background light of the input image and principal component 𝑃𝐶2 fused with the enhanced image in CIE-Lab color space. The obtained images are added together, and vii it is known as a fused image. In the final stage, the color correction is applied to the fused image and it is converted from CIE-Lab color space into RGB color space. The effectiveness of the proposed framework is tested on two standard datasets such as UIEB and RUIE by evaluating various parameters such as PSNR, SSIM, EME, DE, UIQM, and UCIQE. Finally, an ablation study has been conducted to validate the effectiveness of the proposed method. In the initial stage of the second proposed method of the fifth chapter, applied the color correction on the input image. Then the color-corrected image is converted into YCbCr color space. In the second stage, the weighted list square (WLS) filter is applied on the Y component of YCbCr color space for the multiscale decomposition (i.e., base layer, detail layer1, and detail layer2). Then, gamma correction is applied on the base layer and gradient-domain enhancement is carried out on detail layers. Parallelly, the color saturation and restoration are applied on the Cb-component and Cr-component of YCbCr color space. The enhanced Y-component is obtained by adding the gamma-corrected base layer and two details layers after applying the gradient domain enhancement. The image restoration is achieved by adding the modified Cb-component and Cr-component (Cb-component and Cr-component after applying color saturation and restoration) of YCbCr color space. Finally, image enhancement and restoration in YCbCr color space is achieved by multiplying the enhanced Y-component with the modified Cb-component and Cr component. Then YCbCr is converted into RGB color space. The proposed method is evaluated on multiple datasets such as UIEB, UCCS, UIQS, and U45. The effectiveness of the method is tested by comparing it with existing state-of-the-art methods in terms of the visual quality of images and various parameters such as PSNR, SSIM, EME, DE, UIQM, and UCIQE. Additionally, the proposed method tested for other applications such as low light images with the exclusively dark dataset and image database TID2013. Finally, in this thesis, we summarize the conclusions inferred from our research work and highlight the potential future scope in the field of underwater imaging.