Design of estimators for restoration of images degraded by haze using genetic programming
Introduction
In outdoor scenes, captured images can suffer from a visibility loss in the presence of a turbid medium in the atmosphere, for instance, rain, haze or fog. In such a scenario, the light reflected by scene objects (reflectance function of the scene) is scattered by suspended particles in the medium causing a twofold effect. First, the reflectance function of the scene is attenuated by the transmission function of the medium. This function dissipates the light reflected by scene objects as the distance between the objects and the camera increases. Second, the attenuated light is substituted by scattered light coming from environmental illumination; called airlight. Light scattering in outdoor scenes causes a reduction of color fidelity and a loss of contrast in captured images. These issues diminish the performance of computer vision systems and, moreover, can be a critical source of risk in many human activities such as vehicle navigation. Thus, it is necessary to develop accurate and efficient algorithms that can increase the visibility range in a scattering medium, for instance due to bad weather.
The problem of image dehazing consists in recovering an undegraded image by processing degraded images that were captured in a scattering medium. It should be noted that this problem is challenging because several physical factors need to be estimated, such as the depth distribution function of the scene, the concentration density of suspended particles in the medium, and the magnitude of environmental light, among others. Over the years, several methods for the restoration of hazy images have been proposed [[1], [2], [3], [4]]. For instance, an existing approach consists in estimating the depth distribution function of the scene by processing several hazy images, each captured under different atmospheric conditions [2]. This method can be used either for monochrome or color images. Another approach is based on processing multiple images captured with different polarization filters. In this approach it is assumed that the polarization of the light reflected by scene objects is different than that of the airlight. However, this approach can only yield good results in low haze conditions [3]. Note that both of these methods require the processing of several images that were captured under different optical conditions. Hence, they are not appropriate for real-time applications or scenarios where a single image is available.
Nowadays, single image dehazing is a widely used approach for the restoration of images captured in bad weather conditions. First, it is necessary to estimate the medium transmission function of the captured scene and the parameters of an image formation model based on atmospheric optics. Next, an estimate of the haze-free image is obtained using a restoration function obtained from the image model. In recent years, several successful methods for single image dehazing have been proposed [[5], [6], [7], [8], [9], [10], [11], [12], [13]]. A method based on independent component analysis (ICA) to estimate the medium transmission function is proposed in Ref. [6]. This method relies on the assumption that the reflectance and medium transmission functions are independent. Although this method is able to produce good results, it possesses a high computational complexity which make it unfeasible for real-time applications. A fast algorithm based on median filtering has also been proposed [7]. This method is computationally efficient but produces very noticeable overprocessing artifacts that distort the scene colors in the restored image. The Dark Channel Prior (DCP) [8] is a well known approach for single image dehazing. This method is based on the assumption that within a small fragment of an input image at least one channel of the RGB color model contains pixels with intensity values close to zero for a haze-free image and intensity values proportional to the haze density for images captured in a scattering medium. However, this assumption fails for bright objects in a scene. Recently, a color attenuation prior (CAP) approach was also proposed [10]. This method assumes that the depth function of a hazy scene can be reconstructed as a linear combination of the brightness and saturation components of the captured image in the hue-saturation-value (HSV) color space. However, this method cannot be applied to monochrome images.
A main drawback of existing single image dehazing methods is that they introduce undesirable edge artifacts and overprocessing effects to the restored images [14,15]. Therefore, for real-time applications the single image approach is appropriate, but still remains a challenging open problem. Existing methods that implement this approach are only able to provide partial solutions. Thus, the exploration of new alternatives for designing effective image dehazing algorithms is desirable.
In this work, we present a novel methodology based on Genetic Programming (GP) for the automatic design of estimators of the medium transmission of captured scenes in the presence of haze. Given a set of synthetic images and their depth functions, we pose two supervised learning problems where the goal is to find operators that can approximate the known ground truth by minimizing the mean-absolute-error (MAE). GP explores the space of symbolic representations for such functions, and evolves the operators to the desired goal. The evolved estimators can be used to effectively remove the effects of haze in distant objects of the scene, while minimizing the introduction of overprocessing effects in objects that are located near the camera. The methods compare favorably with state-of-the-art methods, achieving statistically significant improvements in performance and low computational cost.
This paper is organized as follows. Section 2 presents the formulation of the problem of image restoration in the presence of haze, as well as a brief review of existing techniques for single image dehazing. Section 3 presents a short overview of GP and related works from the field. Section 4 describes the proposed GP-based methodology for the design of estimators of the medium transmission function of hazy scenes. Section 5 presents the results obtained with the proposed approach and a performance comparison with state-of-the-art methods. Finally, Section 6 summarizes our conclusions.
Section snippets
Problem definition
In an homogeneous scattering medium, a captured image I(x) in the presence of haze can be represented according to atmospheric optics, as follows [1,16]:where x = [x, y] denotes Cartesian coordinates, J(x) represents the reflectance function of the scene (undegraded image), d(x) is the depth distribution of the scene with respect to the camera, β is an attenuation coefficient determined by the weather condition and A is a scalar constant representing the airlight.
Genetic programming and related works
GP is a well-known evolutionary algorithm, where the goal is to evolve syntactic expressions that represent a function, operator, model or in general anything that performs some form of computation [17,18]. To perform this task, GP uses a special representation for the solution candidates. The most common representation is syntax trees but other approaches are possible, such as graphs, lists or grammars [18]. The most common application of GP is to solve supervised machine learning tasks, such
Design of estimators of the medium transmission using genetic programming
Consider a hazy image I(x) and its ground truth transmission function t(x). The problem of finding an optimal operator K† to estimate the function t(x) from I(x), can be stated as follows:where K(I(x)) denotes an operator which is applied to I(x), Err[⋅] represents an error measure, and G is the search (function) space containing all feasible estimators K.
In the present work, several synthetic hazy images and their ground truth transmissions are given
Results
The performance of the proposed GP-based approach is presented and discussed in this section. The experiments consider four different configurations of the proposed GP approach; namely, standard GP using local image statistics (GP-SD), standard GP using local pixel information (GP-PD), neat-GP using local image statistics (GP-SN) and neat-GP using local pixel information (GP-PN). These versions of the proposed GP approach were executed using the parameters described in Table 1, performing
Conclusions
A methodology based on GP for the design of estimators of the medium transmission function for the restoration of images degraded by haze was presented. The proposed GP approach was implemented using four different configurations, considering two GP-variants and two different approaches for the extraction of local image information that is given as input to the estimators. The performance of the evolved estimators was evaluated in terms of objective metrics regarding the restoration of
Acknowledgments
This research was supported by Secretaría de Investigación y Posgrado - Instituto Politécnico Nacional, project SIP20181489. This work was also partially supported by FP7-Marie Curie-IRSES 2013 European Commission program with project ACoBSEC with contract No. 612689 and by CONACYT (Mexico) Fronteras de la Ciencia 2015-2 Project No. FC-2015-2:944.
References (32)
- et al.
Evolutionary-computer-assisted design of image operators that detect interest points using genetic programming
Image Vis Comput.
(2011) - et al.
Evolving estimators of the pointwise hölder exponent with genetic programming
Inf. Sci.
(2012) - et al.
Genetic programming based blind image deconvolution for surveillancesystems
Eng. Appl. Artif. Intell.
(2013) - et al.
Image feature selection using genetic programming for figure-ground segmentation
Eng. Appl. Artif. Intell.
(2017) - et al.
Neat genetic programming: controlling bloat naturally
Inf. Sci.
(2016) - et al.
Vision and the atmosphere
Int. J. Comput. Vis.
(2002) - et al.
Contrast restoration of weather degraded images
IEEE Trans. Pattern Anal. Mach. Intell.
(2003) - et al.
Polarization-based vision through haze
Appl. Opt.
(2003) - et al.
Comprehensive survey on haze removal techniques
Multimed. Tool. Appl.
(2018) - R. T. Tan, Visibility in bad weather from a single image, in: Computer Vision and Pattern Recognition, 2008. CVPR 2008....
Single image dehazing
ACM Trans. Graph.
Single image haze removal using dark channel prior
IEEE Trans. Pattern Anal. Mach. Intell.
Fast algorithm for dark channel prior
Electron. Lett.
A fast single image haze removal algorithm using color attenuation prior
IEEE Trans. Image Process.
Real-time haze removal in monocular images using locally adaptive processing
J. Real-Time Image Process.
Cited by (12)
Evolutionary modeling approach based on multiobjective genetic programming for strip quality prediction
2024, Swarm and Evolutionary ComputationA comprehensive review of automatic programming methods
2023, Applied Soft ComputingAttentive U-recurrent encoder-decoder network for image dehazing
2021, NeurocomputingCitation Excerpt :Yu et al. present a Pixel-wise Alpha Blending method (PWAB) for dehazing nighttime images which obtains the transmission map by blending the results estimated by the dark channel prior and the bright channel prior [19]. Hernandez-Beltran et al. propose a dehazing method based on the Genetic Programming Estimator (GPE) for predicting the transmission function automatically [20]. Li et al. propose a Multi-scale Pyramid Fusion method (MPFM) which fuses the local detail enhancement and the global contrast enhancement in a multi-scale pyramid strategy [5].
A recent survey on the applications of genetic programming in image processing
2021, Computational IntelligenceA multi-scale attentive recurrent network for image dehazing
2021, Multimedia Tools and ApplicationsStructural Health Fuzzy Classification of Bridge based on Subjective and Objective Inspections
2020, 2020 IEEE 12th International Conference on Humanoid, Nanotechnology, Information Technology, Communication and Control, Environment, and Management, HNICEM 2020