Design of estimators for restoration of images degraded by haze using genetic programming

https://doi.org/10.1016/j.swevo.2018.11.008Get rights and content

Abstract

Restoring hazy images is challenging since it must account for several physical factors that are related to the image formation process. Existing analytical methods can only provide partial solutions because they rely on assumptions that may not be valid in practice. This research presents an effective method for restoring hazy images based on genetic programming. Using basic mathematical operators several computer programs that estimate the medium transmission function of hazy scenes are automatically evolved. Afterwards, image restoration is performed using the estimated transmission function in a physics-based restoration model. The proposed estimators are optimized with respect to the mean-absolute-error. Thus, the effects of haze are effectively removed while minimizing overprocessing artifacts. The performance of the evolved GP estimators given in terms of objective metrics and a subjective visual criterion, is evaluated on synthetic and real-life hazy images. Comparisons are carried out with state-of-the-art methods, showing that the evolved estimators can outperform these methods without incurring a loss in efficiency, and in most scenarios achieving improved performance that is statistically significant.

Introduction

In outdoor scenes, captured images can suffer from a visibility loss in the presence of a turbid medium in the atmosphere, for instance, rain, haze or fog. In such a scenario, the light reflected by scene objects (reflectance function of the scene) is scattered by suspended particles in the medium causing a twofold effect. First, the reflectance function of the scene is attenuated by the transmission function of the medium. This function dissipates the light reflected by scene objects as the distance between the objects and the camera increases. Second, the attenuated light is substituted by scattered light coming from environmental illumination; called airlight. Light scattering in outdoor scenes causes a reduction of color fidelity and a loss of contrast in captured images. These issues diminish the performance of computer vision systems and, moreover, can be a critical source of risk in many human activities such as vehicle navigation. Thus, it is necessary to develop accurate and efficient algorithms that can increase the visibility range in a scattering medium, for instance due to bad weather.

The problem of image dehazing consists in recovering an undegraded image by processing degraded images that were captured in a scattering medium. It should be noted that this problem is challenging because several physical factors need to be estimated, such as the depth distribution function of the scene, the concentration density of suspended particles in the medium, and the magnitude of environmental light, among others. Over the years, several methods for the restoration of hazy images have been proposed [[1], [2], [3], [4]]. For instance, an existing approach consists in estimating the depth distribution function of the scene by processing several hazy images, each captured under different atmospheric conditions [2]. This method can be used either for monochrome or color images. Another approach is based on processing multiple images captured with different polarization filters. In this approach it is assumed that the polarization of the light reflected by scene objects is different than that of the airlight. However, this approach can only yield good results in low haze conditions [3]. Note that both of these methods require the processing of several images that were captured under different optical conditions. Hence, they are not appropriate for real-time applications or scenarios where a single image is available.

Nowadays, single image dehazing is a widely used approach for the restoration of images captured in bad weather conditions. First, it is necessary to estimate the medium transmission function of the captured scene and the parameters of an image formation model based on atmospheric optics. Next, an estimate of the haze-free image is obtained using a restoration function obtained from the image model. In recent years, several successful methods for single image dehazing have been proposed [[5], [6], [7], [8], [9], [10], [11], [12], [13]]. A method based on independent component analysis (ICA) to estimate the medium transmission function is proposed in Ref. [6]. This method relies on the assumption that the reflectance and medium transmission functions are independent. Although this method is able to produce good results, it possesses a high computational complexity which make it unfeasible for real-time applications. A fast algorithm based on median filtering has also been proposed [7]. This method is computationally efficient but produces very noticeable overprocessing artifacts that distort the scene colors in the restored image. The Dark Channel Prior (DCP) [8] is a well known approach for single image dehazing. This method is based on the assumption that within a small fragment of an input image at least one channel of the RGB color model contains pixels with intensity values close to zero for a haze-free image and intensity values proportional to the haze density for images captured in a scattering medium. However, this assumption fails for bright objects in a scene. Recently, a color attenuation prior (CAP) approach was also proposed [10]. This method assumes that the depth function of a hazy scene can be reconstructed as a linear combination of the brightness and saturation components of the captured image in the hue-saturation-value (HSV) color space. However, this method cannot be applied to monochrome images.

A main drawback of existing single image dehazing methods is that they introduce undesirable edge artifacts and overprocessing effects to the restored images [14,15]. Therefore, for real-time applications the single image approach is appropriate, but still remains a challenging open problem. Existing methods that implement this approach are only able to provide partial solutions. Thus, the exploration of new alternatives for designing effective image dehazing algorithms is desirable.

In this work, we present a novel methodology based on Genetic Programming (GP) for the automatic design of estimators of the medium transmission of captured scenes in the presence of haze. Given a set of synthetic images and their depth functions, we pose two supervised learning problems where the goal is to find operators that can approximate the known ground truth by minimizing the mean-absolute-error (MAE). GP explores the space of symbolic representations for such functions, and evolves the operators to the desired goal. The evolved estimators can be used to effectively remove the effects of haze in distant objects of the scene, while minimizing the introduction of overprocessing effects in objects that are located near the camera. The methods compare favorably with state-of-the-art methods, achieving statistically significant improvements in performance and low computational cost.

This paper is organized as follows. Section 2 presents the formulation of the problem of image restoration in the presence of haze, as well as a brief review of existing techniques for single image dehazing. Section 3 presents a short overview of GP and related works from the field. Section 4 describes the proposed GP-based methodology for the design of estimators of the medium transmission function of hazy scenes. Section 5 presents the results obtained with the proposed approach and a performance comparison with state-of-the-art methods. Finally, Section 6 summarizes our conclusions.

Section snippets

Problem definition

In an homogeneous scattering medium, a captured image I(x) in the presence of haze can be represented according to atmospheric optics, as follows [1,16]:I(x)=J(x)eβd(x)+A(1eβd(x)),where x = [x, y] denotes Cartesian coordinates, J(x) represents the reflectance function of the scene (undegraded image), d(x) is the depth distribution of the scene with respect to the camera, β is an attenuation coefficient determined by the weather condition and A is a scalar constant representing the airlight.

Genetic programming and related works

GP is a well-known evolutionary algorithm, where the goal is to evolve syntactic expressions that represent a function, operator, model or in general anything that performs some form of computation [17,18]. To perform this task, GP uses a special representation for the solution candidates. The most common representation is syntax trees but other approaches are possible, such as graphs, lists or grammars [18]. The most common application of GP is to solve supervised machine learning tasks, such

Design of estimators of the medium transmission using genetic programming

Consider a hazy image I(x) and its ground truth transmission function t(x). The problem of finding an optimal operator K to estimate the function t(x) from I(x), can be stated as follows:KargminKGErr[K(I(x)),t(x)],where K(I(x)) denotes an operator which is applied to I(x), Err[⋅] represents an error measure, and G is the search (function) space containing all feasible estimators K.

In the present work, several synthetic hazy images Ij(x) and their ground truth transmissions tj(x) are given

Results

The performance of the proposed GP-based approach is presented and discussed in this section. The experiments consider four different configurations of the proposed GP approach; namely, standard GP using local image statistics (GP-SD), standard GP using local pixel information (GP-PD), neat-GP using local image statistics (GP-SN) and neat-GP using local pixel information (GP-PN). These versions of the proposed GP approach were executed using the parameters described in Table 1, performing

Conclusions

A methodology based on GP for the design of estimators of the medium transmission function for the restoration of images degraded by haze was presented. The proposed GP approach was implemented using four different configurations, considering two GP-variants and two different approaches for the extraction of local image information that is given as input to the estimators. The performance of the evolved estimators was evaluated in terms of objective metrics regarding the restoration of

Acknowledgments

This research was supported by Secretaría de Investigación y Posgrado - Instituto Politécnico Nacional, project SIP20181489. This work was also partially supported by FP7-Marie Curie-IRSES 2013 European Commission program with project ACoBSEC with contract No. 612689 and by CONACYT (Mexico) Fronteras de la Ciencia 2015-2 Project No. FC-2015-2:944.

References (32)

  • R. Fattal

    Single image dehazing

    ACM Trans. Graph.

    (2008)
  • J.-P. Tarel, N. Hautiere, Fast visibility restoration from a single color or gray level image, in: Computer Vision,...
  • K. He et al.

    Single image haze removal using dark channel prior

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2011)
  • R. Gao et al.

    Fast algorithm for dark channel prior

    Electron. Lett.

    (2014)
  • Q. Zhu et al.

    A fast single image haze removal algorithm using color attenuation prior

    IEEE Trans. Image Process.

    (2015)
  • V.H. Diaz-Ramirez et al.

    Real-time haze removal in monocular images using locally adaptive processing

    J. Real-Time Image Process.

    (2017)
  • Cited by (12)

    • Attentive U-recurrent encoder-decoder network for image dehazing

      2021, Neurocomputing
      Citation Excerpt :

      Yu et al. present a Pixel-wise Alpha Blending method (PWAB) for dehazing nighttime images which obtains the transmission map by blending the results estimated by the dark channel prior and the bright channel prior [19]. Hernandez-Beltran et al. propose a dehazing method based on the Genetic Programming Estimator (GPE) for predicting the transmission function automatically [20]. Li et al. propose a Multi-scale Pyramid Fusion method (MPFM) which fuses the local detail enhancement and the global contrast enhancement in a multi-scale pyramid strategy [5].

    • Structural Health Fuzzy Classification of Bridge based on Subjective and Objective Inspections

      2020, 2020 IEEE 12th International Conference on Humanoid, Nanotechnology, Information Technology, Communication and Control, Environment, and Management, HNICEM 2020
    View all citing articles on Scopus
    View full text