Sunteți pe pagina 1din 7

1108

IEEE Transactions on Consumer Electronics, Vol. 56, No. 2, May 2010

Enhanced Frame Rate Up-Conversion Method for UHD Video

Tae-Shick Wang, Kang-Sun Choi, Member, IEEE, Hyung-Seok Jang, Aldo W. Morales, Senior Member, IEEE, and Sung-Jea Ko, Senior Member, IEEE

Abstract In this paper, a segment-based motion estimation (ME) method for frame rate up-conversion is proposed that can achieve better visual quality for next generation video formats. The proposed ME method can reduce the computational complexity of the ME drastically by adjusting the search range (SR) for each segment and adopting a new efficient distortion criterion, called the sub- sampled sum of absolute differences (SSAD). The proposed method does not degrade the motion accuracy by using the temporal consistency of the segment with large size. Experimental results show that the proposed algorithm outperforms conventional methods both objectively and subjectively with the significantly reduced computational complexity 1 .

Index Terms — Frame rate up-conversion, true motion estimation, block-based segmentation, low complexity, ultra high definition.

segmentation, low complexity, ultra high definition. I. INTRODUCTION With the dramatic grow th for large-size
segmentation, low complexity, ultra high definition. I. INTRODUCTION With the dramatic grow th for large-size
segmentation, low complexity, ultra high definition. I. INTRODUCTION With the dramatic grow th for large-size
segmentation, low complexity, ultra high definition. I. INTRODUCTION With the dramatic grow th for large-size
segmentation, low complexity, ultra high definition. I. INTRODUCTION With the dramatic grow th for large-size
segmentation, low complexity, ultra high definition. I. INTRODUCTION With the dramatic grow th for large-size
segmentation, low complexity, ultra high definition. I. INTRODUCTION With the dramatic grow th for large-size

I.

INTRODUCTION

low complexity, ultra high definition. I. INTRODUCTION With the dramatic grow th for large-size liquid crystal

With the dramatic growth for large-size liquid crystal display (LCD) devices among commercial and end users, delivering a higher quality of viewing experience (QoE) to the users becomes more significantly competitive advantage in the marketplace. Recently, ultra high definition (UHD) which provides the four to sixteen times higher resolution than high definition (HD) is being investigated as a next generation video format. However, since the motion blur caused by inherent characteristic of LCD still remains [1], reduction of the blur can provide viewers with a strong sense of satisfaction as shown in Fig. 1. In order to alleviate the motion blur, various frame rate up-conversion (FRUC) methods have been developed and successfully employed [2]-[11]. In most FRUC methods, motion compensated frame interpolation (MCFI) is employed to accurately generate an interpolated frame by considering the object movement. Generally, MCFI methods consist of motion estimation (ME) and mo tion-compensated interpolation (MCI).

1 This research was supported by Seoul Future Contents Convergence (SFCC) Cluster established by Seoul R&BD Program (No. 10570). This work was supported by the Korea Science and Engineering Foundation(KOSEF) grant funded by the Korea government(MEST) (No. 2009-0080547) T.-S. Wang, K.-S. Choi, H.-S. Jang, and S.-J. Ko are with the School of Electrical Engineering, Korea University, Seoul, Korea, 136-701 (e-mail:

kschoi@dali.korea.ac.kr, sjko@korea.ac.kr). A. W. Morales is with the Department of Electrical Engineering, Penn State University, Harrisburg, PA 17057, USA (e-mail: awm2@psu.edu).

Fig. 1. Comparison of QoEs both with and without FRUC.

The motion vector field (MVF) is estimated between two successive frames, and then the interpolated frame is obtained by merging the frames based on the MVF. In the MCFI, the MVF should represent the actual object motion. More precisely, the MVF should be smooth within each object and discontinuous between objects of different motion. These requirements are different from those for applications of video compression, where the estimation minimizes the energy of the displaced frame difference. Block matching algorithm (BMA) widely used for video compression cannot produce true motion vectors (MVs) especially when the block is very small with respect to the frame size. In this case, the discontinuity among the MVs belonging to one object causes blocking artifacts. On the contrary, if the block is large and spans objects of different motion, ghost artifacts are produced due to inaccurate MVs estimated by BMA. Several ME methods for MCFI have been proposed to find the true motion trajectories based on BMA [2]-[4]. In [2], they assumed that that the block with large size gives a more correct motion trajectory. Based on the assumption, overlapped matching block is used to get more accurate MVF. However, when the overlapped block spans multiple objects of different motion, an unreliable MV is obtained. A multi- size BMA method was presented in [3], where the block size is adaptively determined depending on the similarity of the

Manuscript received April 15, 2010 Current version published 06 29 2010; Electronic version published 07 06 2010.

0098 3063/10/$20.00 © 2010 IEEE

T.-S. Wang et al.: Enhanced Frame Rate Up-Conversion Method for UHD Video

MVs obtained in the previous frame. However, these ME methods cannot be directly employed for real-time FRUC applications for UHD video due to a drastic increase in computational complexity. For MCI, bi-directional MCI using overlapped block motion compensation (OBMC) has been widely utilized since OBMC has been proved to be an effective method in reducing blocking artifact [5]-[12]. To improve the performance of OBMC, several modified versions have been presented [8],[9]. In FRUC applications, smoothing effect caused by OBMC produces pleasing results along object boundaries. Since the performance of the MCFI highly depends on the accuracy of the MVF, in this paper, we focus on the true ME and our previous MCI method in [6] is utilized for the MCI. To cope with the problems in both computational complexity and visual quality, we propose a novel FRUC method which achieves better visual quality with an efficient MCFI algorithm for next generation video formats. In order to obtain the true motion with low complexity, a segment-based ME method is proposed, where each segment consists of the blocks with identical image pattern. Before the MV of each segment is actually estimated, we predict the reliability of the MV to be obtained for each segment by exploiting the characteristics of the segment such as size and temporal consistency. Then, the MVs of the segments with high reliability are first determined and propagated to the neighboring segments. In the proposed algorithm, both search range (SR) adaptation and subsampled sum of absolute differences (SSAD) are utilized for reducing the computational complexity of the ME significantly. However, since the SR adaptation and SSAD are controlled appropriately based on the characteristics of the segment, the true motion can be obtained without the accuracy degradation. The rest of the paper is organized as follows: In Section II, we analyze the characteristics of motion information obtained by conventional full search based (FS) ME method and discuss the ME scheme for obtaining the true motion. In Section III, based on the motion analysis results, we propose a segment-based ME method. Experimental results for the proposed algorithm and conclusions are given in Section IV and V, respectively.

II. MOTION ANALYSIS FOR UHD VIDEO

Fig. 2 illustrates the statistics including mean, variance, and maximum values of the actual object movement between successive frames in various video sequences of different resolutions. It is apparent from Fig. 2 that a larger SR, in proportion to the increase of the image size, is required to accurately estimate the objects’ movements. However, simple SR enlargement cannot effectively improve the accuracy of the motion obtained by the FS BMA, since the enlarged SR can span regions which belong to another object but are more similar to the current block.

1109

object but are more similar to the current block. 1109 Fig. 2. The statisti cal characteristic

Fig. 2. The statistical characteristic of the object movement between successive frames.

TABLE I

THE AVERAGE ORDER OF THE TRUE MOTION FOR DIFFERENT SRS

Block

 

SRs

Size

32

64

96

128

256

8

109.5

540.4

562.7

766.8

2076.2

16

6.7

9.3

13.7

22.9

61.9

32

4.6

6.6

7.0

10.3

35.2

In order to evaluate the accuracy of the motion information obtained by the FS BMA for different SR sizes, we sorted all the MV candidates within the SR for each block in a UHD video frame in an increasing order of sum of absolute differences (SAD) and obtained the average order of the MV representing the true motion which is identified manually. In order to compensate for the accuracy of the manually obtained true motion, not only the manually obtained MV but also its eight neighboring MVs are considered for the true motion. As a result, among the nine MVs, the one with the lowest order is selected to get the order of the true motion. As the average order of the true motion increases, the FS BMA tends to produce an inaccurate MV which is very different from the true motion. As summarized in Table I, the accuracy of the obtained MV is degraded drastically by enlarging the SR. In order to exploit both contradictory experimental results consistently, the SR size should be kept as small as possible if a reliable MV is initially given. Note that enlarging the block size leads to improve the accuracy of the obtained MVs. However, if the block is large enough to include several objects of different motion, the MV is determined inaccurately as mentioned above. Consequently, in order to obtain the true motion accurately, it is required that only the blocks with similar motion are merged to an entity as many as possible and that the motion is estimated for the entity. In the next section, we introduce a segment-based ME algorithm that divides the frame into several homogeneous segments and estimates the motion for each segment.

1110

f n f n 1
f
n
f
n
1

Fig. 3. Architecture of the proposed ME algorithm.

III. EFFICIENT MOTION ESTIMATION BASED ON BLOCK- BASED OBJECT SEGMENTATION

Different homogenous regions always exist in images, which are usually separated by boundaries consisting of relatively distinct edge lines. In the video sequences, large homogenous regions tend to move rigidly between consecutive frames, whereas boundaries typically undergo non-rigid deformation. Therefore, we can assume that the motion of a large homogenous region can be obtained more reliably. Note that in general the motion ambiguity caused by the aperture problem is more severe in the small region than the large one [13]. In this section, we propose a new ME algorithm using block-based object segmentation to estimate the true motion accurately. The architecture of the proposed ME scheme is

shown in Fig. 3. The current frame,

f

n

, is partitioned into

several segments each of which contains the blocks with similar features. Prior to ME for the segments, in the segment selection, we choose some segments whose motion can be obtained more reliably based on selection conditions satisfying the assumption mentioned above. Then, the MVs for the segments are estimated by three-stage ME in order of their expected reliability. Specifically, the highly reliable MVs of large segments are firstly determined by temporal prediction followed by refinement, and propagated to adjacent segments. Finally, the MVs of the remaining segments are determined by using the FS-based ME. In the following sub- sections, the proposed ME method is described in detail.

A. Object segmentation for true motion estimation

Fig. 4 illustrates the proposed block-based segmentation

algorithm that classifies each block in

edge, plane, and texture. For each block,

calculated by using the Sobel operator [14]. Let denote the

number of the significant pixels whose gradient magnitude is

is classified as "plane".

larger than a threshold

Otherwise, the gradient direction histogram is generated by using the four directional symmetric partitions as shown in Fig. 5 with the gradient angles of the significant pixels.

f

n

into three classes:

i

b

n

, the gradient is

T

0

. If

<

T

1

,

i

b

n

If the ratio of the largest bin size in the histogram to the

, the block

is

i has the "texture" pattern.

,

number of the significant pixels, r , is larger than

T

2

can have dominant directional edges. In that case,

classified as "edge", otherwise

b n

b

n

The "plane", "edge", and "texture" patterns are denoted by

P

P

i

P

E

, and

P

T

, respectively.

IEEE Transactions on Consumer Electronics, Vol. 56, No. 2, May 2010

b i , j

T P 1 P r T P 2 T P E Fig. 4. Flowchart of
T
P
1
P
r T
P
2
T
P
E
Fig. 4. Flowchart of the block pattern classification.
3 4 2 1 1 2 4 3
3
4
2
1
1
2
4
3

Fig. 5. Four directional partitions for the gradient direction histogram.

Then, homogeneous segments are obtained by merging adjacent blocks with an identical pattern if and only if the

difference of the average intensities of the blocks is less than

T

3

.

B. Segment selection

In the segment selection, we find segments whose motion can be obtained accurately. As mentioned above, the motion

of a large homogenous region can be obtained more reliably. In order to improve both the accuracy and efficiency of the following ME, the temporal consistency between corresponding segments is additionally taken into account.

. First, the

MVF of the previous frame is extrapolated to the current

frame by negating the MVF. Assuming that the MVF of the

previous frame represents true motion trajectories and

based on the linear motion model, the

extrapolated homogeneous MVs of

can be temporally consistent if a dominant MV exists

among the extrapolated MVs passing through

is

translated

Let

S

k

n

denote the k-th segment obtained from

f

n

k

S

n

1

l

to

S

k

n

S

n

1

pass through

S

k

n

S

n

l

. Thus,

S

k

n

. As a result,

the segment with the temporal consistency can be found if the following three conditions are all satisfied,

1. The pattern of segment

2. The number of blocks inside

S

k

n

should be

P

P

or

P

T

S

k

n

denoted by

.

N

B

(

k

n

S

)

3.

S

k

n

should be greater than a threshold

should have a dominant MV in a set of the extrapolated MVs passing through

T

4

S

.

k

n

.

T.-S. Wang et al.: Enhanced Frame Rate Up-Conversion Method for UHD Video

In this paper, an MV is considered to be dominant when

over 70% of the extrapolated MVs are identical to the MV.

Note that the temporal consistency verification is computationally efficient since simple operations such as negation and counting are performed.

C. Efficient true motion estimation using segmentation

information In the proposed method, the computational complexity of the ME can be dramatically reduced by reconfiguring the ME scheme adaptively based on several characteristics of the segment. First, instead of the SAD, the SSAD is proposed for the block distortion measurement criterion, the computational complexity of which can be varied as follows:

SSAD( ,

;

x y MV

x

,

MV

y

;

)

B

/

B

/

12

f ( x , y ) f x ( MV , y MV ), n
f
(
x
,
y
)
f x
(
MV
,
y
MV
),
n
2
2
n
xy
2

x

1

0

y

1

0

where

xx

2

yy

2

x

1

y

1 .

(1)

(2)

In (1), (x, y) and (MV x , MV y ) represent the top left position and the displacement of a block, respectively. B and denote the block size and the sub-sampling rate which can control the computational complexity of (1), respectively. Generally, the computational complexity of (1) is much reduced for large , while the accuracy of the distortion measurement is also degraded. In the proposed algorithm, varies depending on the segment size. Since many sample differences are available as for a large segment even with large the distortion measurement accuracy

is

can be maintained. In this work, for the segment set to:

S

k

n

16,

4,

1,

N

B

30

N

B

300,

)

(

(

(

S

n

N

B

S

k

n

)

k

S

n

30.

k

)

300,

(3)

Secondly, the size of the SR for the refinement is adaptively determined to reduce computational loads. As shown in Table I, enlarging the SR can degrade the accuracy of the obtained MV. If an MV can be predicted with high reliability, an accurate MV can be obtained by setting a small SR centered at the predicted MV. To this end, all the SSADs of the blocks in

are initially obtained for the predicted MV. Let

denote the number of the reliable block whose SSAD

is less than

can indicate the accuracy of the predicted MV. If the predicted

. Then, the ratio of

S

k

n

N

RB

(

k

S

n

)

5 B 2
5 B
2

N

RB

(

k )

S

n

and

N

B

(

k )

S

n

MV

is very similar to the actual motion of the segment, the

ratio

approaches 1. In the proposed algorithm, the SR size for

refining the initially predicted MV,

L

SR

10

B 1

N

RB

(

S

k

n

)

N

B

(

S

k

n

)

L SR

1111

, is calculated by

,

(4)

where is the low limit of the SR size which is set to 8. By employing the SSAD and the variable SR size, the computational complexity of the proposed ME is significantly decreased without any loss of the accuracy of the ME. At the first stage of the proposed ME, the MVs for the segments chosen in the segment selection are estimated firstly, which are expected to be highly reliable. Since the selected segments are temporally consistent, the dominant MV obtained in the segment selection can be utilized for an initial predicted MV for the segment. The MV for the segment is determined by refining the initial MV within an SR whose size is calculated by (4). The MV of a segment is defined as a valid one if the MV has been determined before. In the second stage of the proposed ME, we propagate valid MVs obtained in the first

stage to neighboring segments iteratively, assuming that the

valid MVs are highly reliable. For the segment whose MV has not been obtained, the average SSADs are computed with the valid MVs of the neighboring segments, respectively. If the minimum value

, the

among the average SSADs is less than

corresponding MV is utilized for the initial predicted MV of the segment. The MV of the segment is refined with the SR determined by (4). At the third stage of the proposed ME, the MVs of the remaining segments are determined by using the SR of the maximum size without any predicted MVs. Note that computational loads can be reduced in SSAD calculation if the segment is large enough.

5 B 2
5 B
2

IV. EXPERIMENTAL RESULTS

In this section, various simulation results are demonstrated

to evaluate the performance of the proposed algorithm. The

experimental configuration used in this work is described in

the Table II. Three video sequences of UHD frame resolution,

Toy and Calendar, Table Setting, and Tractor, were used. To compare the results of the proposed algorithm objectively and subjectively, Ha’s [2] and Huang’s [10] methods were also performed.

TABLE II

EXPERIMENTAL CONFIGURATION

Block Size, B Maximum Search Range

 

8

128

Test Video Resolution Input Frame Rate Output Frame Rate

3840

1920

15

fps

30

fps

1112

1112 (a) (b) (d) Fig. 6. Performance of the proposed ME algorithm. (a) Original frame of
1112 (a) (b) (d) Fig. 6. Performance of the proposed ME algorithm. (a) Original frame of

(a)

1112 (a) (b) (d) Fig. 6. Performance of the proposed ME algorithm. (a) Original frame of
1112 (a) (b) (d) Fig. 6. Performance of the proposed ME algorithm. (a) Original frame of

(b)

1112 (a) (b) (d) Fig. 6. Performance of the proposed ME algorithm. (a) Original frame of

(d)

Fig. 6. Performance of the proposed ME algorithm. (a) Original frame of the Table Setting sequence. (b) Result of the proposed segmentation method. (c) MVF obtained by using the Ha's method. (d) MVF obtained with the Huang's method. (e) MVF obtained with the proposed method.

(c)

(e)

method. (e) MVF obtained with the proposed method. (c) (e) (a) (c) (b) (d) Fig. 7.

(a)

(e) MVF obtained with the proposed method. (c) (e) (a) (c) (b) (d) Fig. 7. Estimation

(c)

(e) MVF obtained with the proposed method. (c) (e) (a) (c) (b) (d) Fig. 7. Estimation

(b)

MVF obtained with the proposed method. (c) (e) (a) (c) (b) (d) Fig. 7. Estimation of

(d)

Fig. 7. Estimation of the segment MVs in the proposed three-stage ME. (a) Original frame of the Toy and Calendar sequence. (b) Segments whose motion is determined at the first stage of the ME. (c) Segments utilized at the second stage of the ME. (d) Segments utilized at the third stage of the ME.

For each sequence, the threshold values associated with object segmentation, T 0 , T 1 , T 2 , and T 3 , are chosen empirically as 135, 12, 0.5, and 15, respectively. In the segment selection, T 4 can be adaptively adjusted based on the tradeoff between computational complexity and motion accuracy. While a low value for T 4 achieves the computational complexity reduction, temporal consistency can be used for incorrectly matched segments. In this work, we set T 4 to 200 by exhaustive experiments. Fig. 6 demonstrates that the segmentation information obtained by using the proposed block-based segmentation algorithm can improve the result of the ME effectively. Figs. 6(a) and (b) show an original frame of the Table Setting sequence and the segmentation result, respectively. As shown in Fig. 6(b), the proposed segmentation algorithm can divide the frame into meaningful segments and provide information about the objects in the scene. Figs. 6(c), (d), and (e) show the respective MVF results of the conventional FRUC methods and the proposed method for the square-marked region of the center plate. The MVF obtained with Ha's method seems very inconsistent, while the MVFs obtained with Huang's method and the proposed algorithm represent consistent motions.

IEEE Transactions on Consumer Electronics, Vol. 56, No. 2, May 2010

on Consumer Electronics, Vol. 56, No. 2, May 2010 (a) (b) (c) (d) Fig. 8. Subjective
on Consumer Electronics, Vol. 56, No. 2, May 2010 (a) (b) (c) (d) Fig. 8. Subjective

(a)

on Consumer Electronics, Vol. 56, No. 2, May 2010 (a) (b) (c) (d) Fig. 8. Subjective

(b) (c)

Consumer Electronics, Vol. 56, No. 2, May 2010 (a) (b) (c) (d) Fig. 8. Subjective comparison

(d)

Fig. 8. Subjective comparison of the FRUC methods with the Toy and Calendar sequence. (a) Original frame. (b) Part of the interpolated frame obtained by using the Ha's method. (c) Part of the interpolated frame by the Huang's method. (d) Part of the interpolated frame by the proposed method.

(d) Part of the interpolated frame by the proposed method. (a) (b) (c) (d) Fig. 9.
(d) Part of the interpolated frame by the proposed method. (a) (b) (c) (d) Fig. 9.

(a)

Part of the interpolated frame by the proposed method. (a) (b) (c) (d) Fig. 9. Subjective

(b) (c)

the interpolated frame by the proposed method. (a) (b) (c) (d) Fig. 9. Subjective comparison of

(d)

Fig. 9. Subjective comparison of the FRUC methods with the Tractor sequence. (a) Original frame. (b) Part of the interpolated frame obtained by using the Ha's method. (c) Part of the interpolated frame by the Huang's method. (d) Part of the interpolated frame by the proposed method.

However, in the MVF of Huang's method, several prominent MVs which are different from the neighboring MVs exist near the border of the plate. Fig. 7 illustrates how the proposed ME determines the MVs of the segments in the frame. At the first-stage of the ME, the MVs of the segments chosen in the segment selection are determined. As shown in Fig. 7(b), the selected segments are characterized as large plane or texture regions. Then, the MVs

T.-S. Wang et al.: Enhanced Frame Rate Up-Conversion Method for UHD Video

for smaller neighboring segments are determined. Finally, the

MVs

for remaining segments are determined. There are two

main

moving objects in the scene: the calendar and the toy

truck. Note that the calendar is selected and processed at the first stage due to its temporal consistency, while the truck against the linear motion assumption is not selected since it

turns to the right direction.

The performance of the proposed FRUC method is subjectively compared with that of the conventional FRUC

methods in Figs. 8 and 9. In Fig. 8, the conventional methods introduce blocking artifacts on the calendar, while the proposed method produces the clear and pleasing result. If the

MV of the block including only the white pixels is

inaccurately estimated across the line or the number on the

calendar, the blocking artifact can occur on the interpolated frame. However, since the white blocks surrounded with lines are merged into one segment in the proposed method, the MV of the segment can be obtained exactly. In Figs. 9(b) and (c), the conventional methods produce the

ghost artifact in front of the windshield since inaccurate MVs

are obtained due to the occlusion covered by the tractor. In the proposed method, the ground surrounded by the side mirror, the windshield, and the engine hood is merged into one segment and one MV is assigned to this large segment. Therefore, the proposed method produces visually satisfactory

result as shown in Fig. 9(d).

The visual quality of the proposed method is objectively compared with that of the Ha's and the Huang's algorithms for

the first 50 frames of each test sequence in terms of the peak signal to noise ratio (PSNR). The test sequences were temporally skipped by a factor of two to generate the sequences of 15 fps. The skipped frames were interpolated by

using the conventional and the proposed methods. In addition,

to compare the computational complexity, the relative complexity defined below is calculated,

Relative Complexity ( RC

)

Processing Time

Compared

Processing Time

Proposed

.

(5)

The comparison results of the PSNR and the RC are summarized in Table III. The proposed method does not only achieve the improved visual quality for the interpolated frame by about 2~3dB, but also reduce the computational load significantly up to 81% of that of a conventional method.

TABLE III

OBJECTIVE COMPARISON OF THE FRUC METHODS

 

HA'S

HUANG'S

PROPOSED

PSNR

RC

PSNR

RC

PSNR

RC

Toy and

           

Calendar

30.55

7.5

32.98

3.5

35.02

1

Table Setting

28.85

5.0

30.44

2.4

32.77

1

Tractor

28.52

3.4

27.43

1.6

29.13

1

V.

CONCLUSIONS

1113

In this paper, we proposed the computationally efficient segment-based ME method for FRUC which can provide the improved visual quality for UHD video. The proposed block-based segmentation method was confirmed to produce meaningful segment information with low complexity. By estimating the MV for each segment, the true motion can be obtained accurately. In order to reduce the computational complexity of the ME, the SR adaptation and the SSAD are utilized in the proposed method. Since they were designed based on the motion analysis of UHD video, significant computation reduction can be achieved without the accuracy degradation in the estimated MVF. The experimental results show that the proposed FRUC method outperforms the conventional methods subjectively and objectively with reduced computational loads. The proposed MCFI method can not only be suitable for the FRUC application for UHD video but also be successfully employed for various applications including de-interlacing and view interpolation for multi- view video.

REFERENCES

[1] H. Pan, X.-F. Feng, and S. Daly, "LCD motion blur modeling and

analysis," in Proc. ICIP, Sep. 2005, vol. 2, pp. 21-24. T. Ha, S. Lee, and J. Kim, "Motion compensated frame interpolation by new block-based motion estimation algorithm," IEEE Trans. Consumer

Electron., vol. 50, no. 2, pp. 752-759, May 2004.

[2]

[3] S. Fujiwara and A. Taguchi, “Motion-compensated frame rate up- conversion based on block matching algorithm with multi-size blocks,” in Proc. Int. Symp. Intelligent Signal Processing and Communication Systems, Dec. 2005, pp. 353–356.

[4]

S.-J. Kang, K.-R. Cho, “Motion compensated frame rate up-conversion

[5]

using extended bilateral motion estimation,” IEEE Trans. Consumer Electron., Vol. 53, No. 4, Nov. 2007. B.-D. Choi, J.-W. Han, C.-S. Kim, and S.-J. Ko, "Frame rate up- conversion using perspective transform," IEEE Trans. Consumer Electron., vol. 52, no. 8, pp. 975-982, Aug. 2006.

[6] B.-T. Choi, S.-H. Lee, and S.-J. Ko, “New frame rate up-conversion using bi-directional motion estimation,” IEEE Trans. Consumer Electron., vol. 46, no. 3, pp. 603–609, Aug. 2000. [7] S.-H Lee, O.-J. Kwon and R.-H. Park, “Weighted-adaptive motion- compensated frame rate up-conversion,” IEEE Trans. Consumer Electron., Vol. 49, no.3, pp. 485-492, Aug. 2003.

[8]

B.-D. Choi, J.-W. Han, C.-S Kim, and S.-J. Ko, “Motion-compensated

[9]

frame interpolation using bilateral motion estimation and adaptive overlapped block motion compensation,” IEEE Trans. Circuits Syst. Video Technol., vol. 17, no. 4, pp. 407-416, Apr. 2007. B.-D. Choi, J.-W. Han, and S.-J. Ko, “Irregular-grid-overlapped block

motion compensation and its practical application,” IEEE Trans. Circuits Syst. Video Technol., vol. 19, no. 8, pp. 1221-1226, Aug. 2009. [10] A.-M. Huang and T. Nguyen, "A multistage motion vector processing method for motion-compensated frame interpolation," IEEE Trans. Image Process., vol. 17, no. 5, pp. 694-708, May 2008. [11] T. Thaipanich, P.-H Wu and C.-C. J. Kuo, “Low complexity algorithm for robjust video frame rate up-conversion (FRUC) technique,” IEEE Trans. Consumer Electron., Vol. 55, no.1, pp. 220-228, Feb. 2009.

[12] M. T. Orchard and G. J. Sullivan, “Overlapped block motion compensation: an estimation-theoretic approach,” IEEE Trans. Image

Process., vol. 3, no. 5, pp. 693-699, Sep. 1994. [13] M. Yamamoto, "A general aperture problem for direct estimation of 3-D motion parameters," IEEE Trans. on Pattern Anal. Mach. Intell., vol. 11,

no. 5, pp. 528-536, 1989 [14] R. C. Gonzalez and R.E. Woods, Digital Image Processing, 2nd Edition, Prentice Hall, New Jersey, 2002.

1114

1114 BIOGRAPHIES Tae-Shick Wang received the B.S. and M.S. degree in Electronic Engineering from Korea University,

BIOGRAPHIES

Tae-Shick Wang received the B.S. and M.S. degree in Electronic Engineering from Korea University, in 2006 and 2008, respectively. He is now a Ph.D. candidate in Department of Electronic Engineering at Korea University. His research interests include video format conversion, image and video signal processing.

Kang-Sun Choi (M’03) received the Ph.D. degree in 2003, the M.S. degree in 1999, and the B.S. degree in 1997 in Electronic Engineering from Korea University. From 2003 to 2005, he was a visiting scholar at University of Southern California. From 2005 to 2008, he worked in Kang-Sun Choi (M’03) received the Ph.D. degree in Samsung Electronics, Korea, as a Senior Software Engineer. Samsung Electronics, Korea, as a Senior Software Engineer. In 2008, he joined the Department of Electronic Engineering at Korea University, where he is currently a Research Professor. He is a Member of IEEE. His research interests are in the areas of signal processing, multimedia compression, and communications.

processing, multimedia compression, and communications. Hyung-Seok Jang received the B.S. degree from Korea

Hyung-Seok Jang received the B.S. degree from Korea University, in Electronic Engineering, 2008. He is now pursuing M.S. degree in Department of Electronic Engineering at Korea University. His current research interests are in image and video signal processing.

IEEE Transactions on Consumer Electronics, Vol. 56, No. 2, May 2010

Aldo W. Morales (SM’99) received the electronics engineering degree with distinction from the University of Aldo W. Morales (SM’99) received the electronics Tarapaca, Arica, Chile (formerly Northern University) and the M.S. Tarapaca, Arica, Chile (formerly Northern University) and the M.S. and Ph.D. degrees in electrical and computer engineering from the State University of New York at Buffalo in 1978, 1986 and 1990, respectively. From September 1990 to July 2001 he was with the College of Engineering, Penn State at DuBois, PA. He is now a Professor of Electrical Engineering, Penn State University at Harrisburg, PA. His research interests are in mathematical morphology, digital image processing, computer vision, signal integrity, high-definition television and wavelets.

Sung-Jea Ko (M’88-SM’97) received the Ph.D. degree in 1988 and the M.S. degree in 1986, both in Electrical and Computer Engineering, from State University of New York at Buffalo, and the B.S. degree in Electronic Engineering at Korea University in 1980. In 1992, he joined the Department of Electronic Engineering at Korea University where he is currently a Professor. From 1988 to 1992, he was an Assistant Professor of the Department of Electrical and Computer Engineering at the University of Michigan-Dearborn. He has published over 130 international journal articles. He also holds over 40 patents on video signal processing and multimedia communications. He is currently a Fellow in the IET and a Korean representative of IEEE Consumer Electronics society. He is the 1999 Recipient of the LG Research Award given to the Outstanding Information and Communication Researcher. He received the Hae-Dong best paper award from the IEEK (1997) and the best paper award from the IEEE Asia Pacific Conference on Circuits and Systems (1996), and the research excellence award from Korea University (2004).Sung-Jea Ko