A COMPARITIVE STUDY OF DIFFERENT BASIS FUNCTIONS IN THE CONTROL GRID INTERPOLATION APPROACH FOR MOTION ESTIMATION

Ramsri G. Golla, Deepta Rajan

Ira A. Fulton School of Engineering, Arizona State University, Tempe, AZ ABSTRACT We know that interpolation is a key process in motion estimation and compensation. Accuracy in computation of motion vectors can be achieved through optimum selection of basis functions during interpolation. This paper presents a comparative study of employing three different basis functions to estimate motion using Control Grid Interpolation (CGI). The three basis are: Radial Basis Functions, Inverse Distance Weighted and Bilinear basis. CGI is a technique characterized by features drawn from both optical flow and block-matching [4]. We compare the performance of the algorithm developed in estimating motion using Radial Basis Functions with those of Bilinear and Inverse Distance Weighted methods, for different pairs of image sequences containing varied spatial components. A comprehensive summary of their performances measured by the prediction error is provided. Index TermsRadial Basis Function (RBF), Bilinear Interpolation, Inverse Distance Weighted (IDW), Control Grid Interpolation (CGI), Motion Estimation 1. INTRODUCTION Data interpolation with knowledge of only a few sample points is a critical engineering problem. To solve this, Block-matching algorithms are used in most cases owing to its computational simplicity. However, they are subjected to blocking artifacts in image prediction. This occurs when adjacent blocks use different motion vectors thus hampering the continuous representation of the motion vector field. They also rank poorly in terms of flexibility. On the other hand, Control Grid Interpolation (CGI) is a spatial transformation technique used to create a smooth interpolated motion field [1]. In this method, the image region is divided into sub regions of rectangular grids whose corners comprise the control points, each of which is associated with a motion vector. Optical Flow equations are employed to calculate these vectors thus enabling detection of complex non-translational motion [2]. Data points can be interpolated between these control points using different neighborhood operations. To tackle underlying function complexities in estimating these data points, a Radial Basis Function (RBF) is used to provide a viable solution. Motion Estimation is an ill-posed problem which involves determining motion vectors. These pixel displacements find a wide range of applications, a few of which are: reconstruction, filtering, object tracking and pattern recognition. Prediction of the next image frame using these motion vectors is called Motion Compensation. It allows for significant reduction of the amount of data required for transmission, thereby facilitating conservation of available bandwidth. Together, motion estimation and compensation are very important in the process of video compression. Therefore, determining a subjective best method to estimate motion vectors is extremely crucial. Toward this end, we have developed a summary that gives a comparative analysis of the accuracy of motion estimated and image predicted between a pair of image sequences using CGI with three different interpolation methods. The three methods employed are: RBF, Bilinear Interpolation and Inverse Distance Weighted. Our focus is on exploring the viability of using RBF despite its computational cost, which has been proved to be reduced by methods such as Center Reduction and Fast Multipole Method [3]. In this paper section 2 describes the three interpolation methods used and provides a detailed explanation of the algorithm developed for interpolation using RBF. Section 3 shows the performance metric obtained on testing different pairs of images using each of the three interpolation methods. Finally, section 4 discusses the results obtained from section 3. 2. INTERPOLATION METHODS AND IMPLEMENTATION Interpolation methods are necessary to assign a certain weight to interest points using the known weights of the points in the specified neighborhood region. We use three interpolation techniques primarily for our comparative analysis, namely: Radial Basis Function (RBF), Bilinear Interpolation and Inverse Distance Weighted (IDW) method. Implementation of these methods to interpolate between a pair of images is described below.

2.1. Inverse Distance Weighted Method It is a weighted average interpolator wherein the value of the weighting factor is controlled by a power N. Our algorithm takes N = 2. The weights, being a fraction, decreases with increase in N and the sum of all the fractions is 1.0. The weighting influence on the interest point declines with increase in distance from the known grid point. When the interest point coincides with the grid point i.e., the distance between the two being 0.0, a weight of 1.0 is assigned to that point, and a value of 0.0 is assigned to all other points [5]. Consider an 8x8 pixel region with four grid points at the corners as shown in Figure 1. The weighting factor assigned to interest point A due to the point at the topleft corner is given by:

From Figure 1, the interpolated data at point A can be calculated by:

= [1 2 3 4]

(3)

2.3 Radial Basis Function (RBF) Method Radial Basis Function (RBF) is a function whose values are symmetric along the line from the center point or origin. This function consists of an input layer, a hidden layer and an output layer [7]. A non-linear mapping from input to hidden space is followed by a linear mapping from hidden to the output space in this method. A function : Rs R is called radial, provided there exists a univariate function

W1 =

(1)

: [0, R such that: x = r

IDW is an exact or smoothing interpolator and can be summarized by the following formula to estimate the function value at point A by:

(4)

(2)

where, r = is the Euclidean norm chosen to map an ndimensional space to Rs enabling the use of RBFs in interpolation for multi-dimensional data [3]. The value of the function is thus radially symmetric. Popular Radial Basis Functions used are: Thin-Plate Spline: Multiquadric: Gaussian:

r = r2logr r = ; b0 r = ; b0

(5)

In our algorithm, we interpolate at every point in the 8x8 pixel region shown in Figure 1 using the values of the four corner points. The RBFs chosen for interpolation are: Figure 1. An 8x8 pixel block region with four control grid points and an interest point A. 2.2. Bilinear Interpolation Method In this method a linear interpolation is performed in the horizontal-direction followed by that in the verticaldirection, and this is implied by the very term bi-linear. The two linear operations are independent. The neighborhood region in this method is specified by the nearest 2x2 pixel block. The weights are assigned due to the influence of a fixed set of only four points. The basis functions are calculated according to the following equations:

(r) = r r = ;
=
being a constant.

(6)

The function value X) at any given point is the product of the known RBF ( )) and their corresponding weights (i) of each of the four grid points. This is shown by the following equation:

X) =

(7)

1 = 3 =

; 2 = ; 4 =

N is the number of points for which the function value is known. Here, N=4 since the function value is known only at four corner points as seen in Figure 1. Using matrix representation of (7), the values of can be computed by:

(8)

Once the values of s are obtained we can calculate the value of the function at all 64 points in the specified region of Figure1 using the equation:

64(X) = 64(X) =

64

64

(9)

The following images show the frames used and also an estimate of image1 obtained using Bilinear interpolation basis functions. We hereby depict the estimated image of only the Bilinear case for reference. However, the quality of rest of the interpolation basis functions can be understood by comparing the MSE of bilinear versus other basis functions from the Table 1. Image1 Image2 Image1_estimated
(Using Bilinear basis)

Here, 64 is the matrix of new RBF values at all points. From (9), 64 is the basis function matrix that is used to calculate the values of the function 64(X) at new interest points using the known values of the function F. In (9), F is a 4x1 matrix of function values of four corners of Figure1 and the dimensionality of the matrices 64 and are shown below:

(10)

3. RESULTS In this paper we have conducted a comparative analysis on the performance of different interpolation basis functions namely: Bilinear, Inverse Distance & RBF in Control Grid Interpolation of varied image sequence pairs. Two images (image1 & image2) are taken from a motion sequence and motion vectors between them are calculated. Image1 is then estimated from Image2 using these motion vectors. A mean square error is calculated between the original Image1 and estimated Image1 which serves as a comparative measure for the quality of interpolation using the different basis functions. Table1. The Mean Square Error (MSE) values for image sequences using various basis functions. Image Sequence Bilinear Inverse Distance Weighted (IDW) 70.619 21.646 1374.0 698.53 304.38 52.614 RBF ((r)=r) RBF ( (r) =
=

Ping Pong Face Football Garden Toy Office

68.064 21.501 1267.6 639.20 292.25 51.247

64.465 21.791 1282.6 630.02 307.10 49.220

64.941 21.640 1323.2 659.47 325.07 49.222

Figure 2. Test image sequences, row-wise (top to bottom) titled Ping Pong, Face, Football, Garden, Toy, and Office. 4. CONCLUSIONS We observe that Bilinear Interpolation basis functions give good results in high motion sequences and comparable results in other non-high motion sequences. If the adjacent

motion vectors are smoothly varying then the RBF linear ( r =r ) produces good interpolated results and performs better or comparable to Bilinear. In high motion sequences where the adjacent motion vectors differ significantly RBF linear fails to outperform Bilinear. RBF Multiquadrics
(

[7] Xian Luo, Youpeng Xu, Jintao Xu, Application of Radial Basis Function Network for spatial precipitation interpolation, Geoinformatics, 2010 18th International Conference, pp. 1-5, 2010. [8] Test image sequences world wide web resources:
http://www.cipr.rpi.edu/resource/sequences/

r =

does not produce better results than RBF

linear in most cases. Inverse distance weighted basis functions have this property that the interpolated value is always between the maximum and minimum motion vectors in all cases. This inherent property of IDW makes the interpolated values more prone to error in this case when compared to Bilinear and Radial Basis functions. Hence IDW has failed to outperform the other two methods in our experiment. Future work can be carried out to adapt the algorithm to choose a different basis function to interpolate at different regions of the image depending on the motion vectors obtained at that particular region. 5. REFERENCES [1] G. J. Sullivan and R. L. Baker, Motion compensation for video compression using control grid interpolation, in Proceedings ICASP, pp. 2713-2716, 1991. [2] D.H. Frakes, C.P. Conrad, T.M. Healy, J.W. Monaco, M. J. T. Smith, Yoganathan, P. Ajit, Application of an adaptive control grid interpolation technique to morphological vascular reconstruction: a component of a comprehensive surgical planning and evaluation tool, Proceedings of the 23rd Annual International Conference of the IEEE, Vol. 3, pp. 2602-2605, 2001. [3] F. Alghabi, P. Kabiri, Novel Approach for Speeding up RBF-based Interpolation Methods, Second International Symposium on Intelligent Information Technology Application, Vol. 3, pp. 8-13, 2008. [4] D. H. Frakes, L. P. Dasi, K. Pekkan, H. D. Kitajima, K. Sundareswaran, A. P. Yoganathan, M. J. T. Smith, A New Method for Registration-Based Medical Image Interpolation, Medical Imaging, IEEE Transactions, Vol. 27, pp. 370-377, 2008. [5] Chin-Shung Yang, Szu-Pyng Kao, Fen-Bin Lee, PenShan Hung, Twelve different interpolation methods: a case study of surfer 8.0, Feng Chia University, Taichung, Taiwan, ROC. [6] D. H. Frakes, J. W. Monaco, M. J. T. Smith, Suppression of atmospheric turbulence in video using an adaptive control grid interpolation approach, Acoustics, Speech, and Signal Processing (ICASSP '01), Vol. 3, pp. 1881-1884, 2001.