A CONCERT EVALUATION OF EXEMPLAR BASED IMAGE INPAINTING ALGORITHMS FOR NATURAL SCENE IMAGE COMPLETION

K.Sangeetha1, Dr.P.Sengottuvelan2 and E.Balamurugan3

1

AP/CA, BIT, Sathy, India

kavigeeth@gmail.com

2 3

AP/IT, BIT, Sathy, India

psengottuvelan@rediffmail.com

AP/CA, BIT, Sathy, India

sanbala@rediffmail.com

ABSTRACT
Image inpainting derives from restoration of art works, and has been applied to repair ancient art works. Inpainting is a technique of restoring a partially damaged or occluded image in an undetectable way. It fills the damaged part of an image by employing information of the undamaged part according to some rules to make it look reasonable to human eyes. Digital image inpainting is relatively new area of research, but numerous and different approaches to tackle the inpainting problem have been proposed since the concept was first introduced. This paper analyzes and compares the recent exemplar based inpainting algorithms by Minqin Wang and Hao Guo et al. A number of examples on real images are demonstrated to evaluate the results of algorithms using Peak Signal to Noise Ratio (PSNR).

KEYWORDS
Exemplar based, Image inpainting, PSNR.

1. INTRODUCTION
The Image Inpainting is the art of modifying an image in a form that is not easily detectable by an ordinary observer. Image Inpainting is to try and fill a hole in an image with some meaningful data based on information in rest of the image. The aim of algorithms evaluated in this paper is not exactly reconstructing what used to be in that hole but instead to create a visually pleasing continuation of data around the hole in such a way that it is not detectable. In order to effectively retain image data, various researchers have continually proposed various methods of image inpainting and these works are classified into 2 major categories. One is non exemplar based method and the other is exemplar based method. The non exemplar based methods are based on pixel interpolation. Bertalmio et al [1] first presented the notion of digital image inpainting and used third order Partial Differential
Sundarapandian et al. (Eds): CoNeCo,WiMo, NLP, CRYPSIS, ICAIT, ICDIP, ITCSE, CS & IT 07, pp. 335340, 2012. CS & IT-CSCP 2012 DOI : 10.5121/csit.2012.2432

336

Computer Science & Information Technology ( CS & IT )

Equations (PDE) to propagate the known image information into the missing regions along the direction of isophote. Some other algorithms of this category are presented in [2-3]. These are effective for small missing regions like small scratches in a photograph and not suitable for large missing regions. The second category of approaches is the exemplar based inpainting algorithm. Criminisis [4] exploit a patch based algorithm, in which the filling order is decided by a predefined priority function to ensure that linear structures will propagate before texture filling to preserve the connectivity of object boundaries. Wu [5] proposed a cross Isophotes exemplar based Inpainting technique, in which a cross isotope patch priority term was designed based on analysis of anisotropic diffusion. Also Wong [6] proposed nonlocal means approach for exemplar based Inpainting algorithm. The image patch is inferred by nonlocal means of a single best match patch. Wohlberg [7] proposed Inpainting with sparse linear combinations of exemplars. Compared with the diffusion-based inpainting algorithms, the exemplar-based inpainting algorithms have performed plausible results for inpainting the large missing image region.

2. COMPARISON OF THE EXISTING EXEMPLAR BASED INPAINTING ALGORITHMS

The following is a brief introduction of Minqin Wang et als [9]and Hao Guo et als[8] exemplar based inpainting algorithms, which are chosen to do some experiments on real scene images in section 4.

2.1 Minqin Wang et als Exemplar Based Image Inpainting Algorithm

This algorithm consists of three main steps. In the first step, we segment image into several parts based on texture features. The crux in this step is texture feature extraction. The texture features of image are extracted based on Gabor wavelet here. Given its characteristics of grouping by similarity, synchronous pulse bursting and capturing, we employ PCNN model to perform the data clustering, which make us group the image based on those texture features extracted by Gabor filters. Thats means we can do segmenting through PCNN model and then get the information about regions and boundaries. In the second step the edges are grouped. In the third step, the algorithm performs the final inpainting based on exemplar guided by regions. This algorithm performs inpainting based on the segmented regions after having reconstructed the image structure, which shorten the searching time. The following notation is commonly used in exemplar based image inpainting algorithms.

Figure. 1 Exemplar-based Image Completion

Computer Science & Information Technology ( CS & IT )

337

(a) Original image, with the target region , its contour , and the source region .(b) Synthesizing the area delimited by the patch P centered on the point p .(c) The most similar candidate patch for P lie along the isophote.(d) Fill the P with the most similar patch q. Experiment results show that the algorithm succeeds in filling the target region and shortens the searching time by searching the exemplar in a selected region.

2.2 Hao Guo et als Exemplar Based Image Inpainting Algorithm

A structure/texture features synthesis algorithm based on exemplar-based texture synthesis is introduced in this paper [8]. This algorithm can restore some structure features (some ordinary structures such as linear contours or curves with small curvature) both for large and thick or long and thin damaged regions without blurring. It includes two steps as follows: First, inside a search scope, a patch of each unknown pixels, which is covered by structuring element currently, is compared with that of known pixels left to right and top to bottom, to find the one with maximal similarity based on the Sum of Squared Distance (SSD). Then, when the structure features patch around a known pixel with maximal similarity (minimal SSD) is found in the search scope, the value of the known pixels in the scope of the structuring element are put into the unknown pixels covered by the structuring element, thus completing one step of restoration. In this way, the structure/texture features of the known areas are propagated into the damaged areas, and the damaged patch at every erosion step is restored. The restoration of structure/texture features for every erosion step is improved: the center of the search scope is adjusted to the original boundary all along, and only the part of the search scope inside the original image is used to reflect the fact that only the information from the original image is reliable. This process includes the following steps: first, a center pixel for the search scope is searched on the original boundary closest to the unknown pixel; then, the search scope is defined based on the center pixel, excluding the parts of the original damaged area between the original boundary and the advanced boundary, which have already been restored. It performs well as previous techniques designed for the restoration of small scratches, and, in instances in which larger objects are removed; it dramatically outperforms earlier work in terms of both perceptual quality and computational efficiency.

3. EVALUATION METHOD
Formulating an accurate evaluation method for determining the success of the two algorithms was a very important yet difficult task. This was because no common method for evaluating inpainting algorithms has been presented in the literature. To try and provide a good and accurate evaluation of the algorithms, it was decided to use both a qualitative and a quantitative approach. The assessment of the results for the qualitative tests was done mainly by visual analysis. The quantitative evaluation was performed by calculating the peak signal-to-noise ratio (PSNR) between the two images. PSNR is an engineering term for the ratio between the maximum possible power of a signal and the power of corrupting noise that affects the delity of its representation. PSNR values are represented in decibels (dB). The equation to calculate a PSNR value is given below:

338

Computer Science & Information Technology ( CS & IT )

PSNR=20.log10 (MAXI / MSE ) where MSE= (1/ mn) and MAX I = 255.

i =0 j =0

m 1

n 1

I (i, j ) K (i, j )

4. RESULTS OF EMPIRICAL EVALUATION

We have experimented with the two methods on some images comparing with PSNR. These algorithms are programmed by matlab2008Ra and all experiments are run on a 2.93GHz PC. The algorithm proposed by Zhaolin Lu et al succeeds in filling the target region without implicit or explicit segmentation. Hao Guo et als algorithm performs well as previous techniques designed for the restoration of small scratches, and, in instances in which larger objects are removed . Fig.2 fig.4 show the results produced by the aforementioned exemplar based Inpainting algorithms. The images in each figure are arranged as original image, an image with occluded region, the final result of methods in [9] and [8] respectively. Figure 2. Lady occluded region of image. (a) Original Image. (b) Mask image. (c) The output image by Minqin Wang et als algorithm. (d) The output image by Hao Guo et als algorithm.

(a)

(b)

(c)

(d)

Figure 3.Man occluded region of image. (a) Original Image. (b) Mask image. (c) The output image by Minqin Wang et als algorithm. (d) The output image by Hao Guo et als algorithm.

(a)

(b)

(c)

(d)

Computer Science & Information Technology ( CS & IT )

339

Figure 4.Bird occluded region of image. (a) Original Image. (b) Mask image. (c) The output image by Minqin Wang et als algorithm. (d) The output image by Hao Guo et als algorithm.

(a)

(b)

(c)

(d)

During the testing it was found that while some images could look visually pleasing, they may have extremely low PSNR values. Table 1 PSNR value for the two exemplar based image inpainting Results Image Method Minqin Wang et als Hao Guo et als Lady occluded image 35.65 36.29 Man occluded image 35.79 35.89 Bird occluded image 35.62 36.53

5. LIMITATIONS
Each of the algorithms presented here have a number of different problems and limitations. Hao Guo et als algorithm works surprisingly well, yet it still has a problem of reconstructing the curved structure in the occlusion. Minqin Wang et als algorithm works well only if the missing region consists of simple structure and texture. If there are not enough samples in the image, it will be impossible to synthesize the desired image. In addition to these shortcomings, there are certain cases where the inpainting algorithms described here and in the literature would fail to successfully reconstruct the image.

6. CONCLUSION AND FURTHER WORK

In this paper, we have looked at two different exemplar based inpainting methods. For each of the algorithms, we have provided a detailed explanation of the process used for lling an occlusion making use of images. In addition, we have performed both a qualitative and quantitative analysis of the algorithms. From this analysis, a number of shortcomings and limitations were highlighted in relation to the type of information each algorithm can restore.

340

Computer Science & Information Technology ( CS & IT )

The digital inpainting problem is still far from being completely solved. Although a large number of algorithms exist that are capable of producing amazing results, they are usually limited to images that portray certain features. Overall, the implemented inpainting methods also have some limitations for which more research needs to be done. It is hoped that the results obtained can provide a good framework for additional research that might be undertaken to improve upon the methods presented here.

REFERENCES
[1] [2] [3] [4] [5] [6] [7] [8] [9] M. Bertalmio, A. Bertozzi, G. Sapiro, Navier-Stokes, fluid dynamics, and image and video inpainting, Proceedings of the IEEE Computer Vision and Pattern Recognition (CVPR), Hawaii, 2001. T.F. Chan, S.-H. Kang, J. Shen, Eulers elastica and curvature based inpainting, SIAM J. Appl. Math. 63 (2002) 564592. T.F. Chan, J. Shen, Mathematical models for local nontexture inpainting, SIAM J. Appl. Math. 62 (2002) 10191043. A. Criminisi, P. Perez, and K. Toyama, Region filling and object removal by exemplar based image inpainting, IEEE Trans. Image Process., vol. 13, pp. 12001212, 2004. J.Wu and Q. Ruan, Object removal by cross isophotes examplar based image inpainting, in Proc. Int. Conf. Pattern Recognition, 2006, pp.810813. A. Wong and J. Orchard, A nonlocal-means approach to examplarbased inpainting, presented at the IEEE Int. Conf. Image Processing, 2008. B. Wohlberg, Inpainting with sparse linear combinations of exemplars, in Proc. IEEE ICASSP, 2009. Hao Guo, Jubai, An Image restoration with morphological erosion and exemplar-based texture synthesis presented in IEEE, 2010. Minqin Wang, Guoqiang Han, Yongqiu Tu, Edge-based Image Completing Guided by Region Segmentation, presented in International Colloquium on Computing, Communication, Control, and Management,2008.