IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 7, NO.

3, JUNE 1997

539

Transactions Briefs
Fuzzy Detection of Edge-Direction for Video Line Doubling
Fran ois Michaud, Chon Tam Le Dinh, and G rard Lachiver c e
AbstractVideo line doubling can be realized using time-space interpolation lters. To improve their performances for moving diagonal lines, we have developed a fuzzy edge-direction detector. This fuzzy detector works by identifying small pixel variations in ve orientations (0 , 45 , and 60 ) and by using rules to infer the prevailing direction. This direction is then used to spatially rotate the interpolation lter. For the fuzzy detector, three fuzzy sets are used to characterize the inputs, and two rule bases have been validated. This article presents the characteristics of the fuzzy edge-direction detector along with the methodology used for fuzzy sets positioning. Detection and interpolation results are also presented.

Index TermsEdge-direction detection, fuzzy logic, video line doubling. Fig. 1. Interpolation lter.

I. INTRODUCTION The standard television video system is based on an interlaced video signal in which an images odd-numbered lines make up one eld, and its even-numbered lines make up another eld. Video line doubling consists of converting an interlaced video signal into a progressively scanned video image where the total number of lines are doubled. The original lines and the newly interpolated lines are alternatively pieced together within one eld. Thus, the line doubling of video image is usually a crucial stage in the up conversion of a standard video signal into a progressive signal, which in turn can be converted into a high-denition video signal by appropriate processing [9]. Presently, two families of interpolation methods are worth considering. The rst family is composed of linear methods which employ vertical and/or temporal-vertical lters [2]. This family of methods generally yields acceptable results except in the case of moving diagonal lines. An undesirable staircase effect is produced on the interpolated moving diagonal lines. This deciency can be resolved if diagonal lines within the original image can be detected in order to rotate the spatial lter impulse response [3]. The second family is composed of adaptive methods in which the interpolation is adjusted according to movement estimation [6], thereby reducing the staircase effect. However, movement estimation can run into difculties if there is a multitude of movements in the original image or if the image does not follow the employed model of movement. In our work, we use a three-eld temporal-vertical lter with linear phase [10] for line interpolation. An impulse response example of this kind of temporal-vertical lter is illustrated in Fig. 1, in which the pixel X to be interpolated is determined by the sum of the pixels denoted by a dot multiplied by the given coefcients. In order to improve its performances for moving diagonal lines, we have developed a fuzzy edge-direction detector indicating the proper spatial orientation of the interpolation lter. This detector uses fuzzy linguistic rules to independently evaluate small pixel variations in ve different directions and to infer the resulting edge-direction. The purpose of this paper is to present the fuzzy detector developed.
Manuscript received November 3, 1995; revised January 4, 1996. This paper was recommended by Associate Editor J. Juhola. The authors are with the Department of Electrical and Computer Engineering, Universit de Sherbrooke, Sherbrooke, Qu bec, J1K 2R1, Canada. e e Publisher Item Identier S 1051-8215(97)02561-5.

Fig. 2. Pixel windows and examples of pixel variations considered by the fuzzy edge-direction detector for p(x; y ).

II. FUZZY EDGE-DIRECTION DETECTOR Fuzzy logic has already demonstrated its usefulness in image processing [5], [7], [8]. It allows reasoning with levels of truth instead of using binary membership evaluation commonly used with ordinary sets. Our fuzzy detector follows the same processing steps as a fuzzy controller [4], i.e., fuzzication, rule inference, and defuzzication. A. Fuzzication Fuzzication converts inputs into their fuzzy set representations. For the fuzzy detector, input data are pixel variations in ve directions. Fig. 2 shows the pixel window of the Present Field considered by the fuzzy edge-direction detector, where x refers to the column number and y to the line number. The ve directions considered in reference to the y -axis are also illustrated. With the interpolation lter used, preliminary results have indicated that these ve directions are sufcient for good interpolation quality. Let us consider the interpolation of the pixel p x; y , in bold. Pixel variations, used as inputs to the fuzzy detector, are dened as

( )

1i; (x; y) = abs[p(x + i +  ; y 0 1) 0 p(x + i 0  ; y +1)] (1) where 1i; (x; y ) is the pixel variation in direction  at the pixel p(x + i; y ) (i is the horizontal offset), and  is the orientation offset equal to 02, 01, 0, 1, 2, respectively for  equal to 60 , 45 , 0 , 045 , 060 . For example, 10; ( ) are depicted in Fig. 2.
x; y

10518215/97$10.00 1997 IEEE

540

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 7, NO. 3, JUNE 1997

ORIGINAL RULE BASE

TABLE I WITH ONE RULE PER DIRECTION

Fig. 3. Membership function SMALL6 .

These inputs are converted into fuzzy variables represented by the name of the associated fuzzy sets and a membership value. A fuzzy set is dened by a membership function. For the fuzzy detector, three fuzzy sets are used to characterize small pixel variations in 0 , 6 45 and 6 60 . The same membership function is used for directions  and 0 for a symmetrical evaluation. These membership functions, named SMALL6 , have all the same shape, as illustrated in Fig. 3. However, each one is dened differently by two parameters: a6 referring to the threshold for unit membership, and d6 dening the upper bound of the function. So, to be small, i; x; y must be lower than d6 . Its grade of membership increases if it is smaller than d6 and lower than a6 . If it is lower than a6 ; i; x; y is considered to have a truth level  equal to one.

ADDITIONAL RULES

FOR THE

TABLE II ENLARGED RULE BASE

1 ( ) 1 ( )

B. Fuzzy Control Rules The fuzzy rule base characterizes the control policy needed to infer fuzzy control decisions, which are directions for our fuzzy detector. A rule is described as a conditional statement in which antecedents are the conditions, and the consequence is a control decision. The conjunction of antecedent membership values gives the truth level of the consequence of the rule. The minimum and the algebraic product are the fuzzy conjunction operators used with the fuzzy detector. All the rules that have any truth in their premises will re and contribute to the output. Afterwards, the truth levels of the same consequences are unied using the fuzzy disjunction maximum. For the fuzzy detector, the formulation of the fuzzy control rules has been made based on the general assumption that the direction of an edge can be detected when pixel variations in that direction are small. Each rule examines small luminance variations in a particular direction to detect the possible orientation of an edge. The evaluation of all the rules before making a decision (which happens during defuzzication) and the use of truth levels help avoid the possible inconsistencies that can arise from this simple assumption for detecting an edge. Two different rule bases have been tested for the fuzzy detector. Table I shows the rst rule base used. For example, the third rule can be read as

The second rule base uses three rules for each direction by considering past and future detections (dir x 0 ; y and dir x ; y , respectively) to reinforce the current detection dir x; y . The objective of this rule base is to see whether or not additional rules, constraining more the conditions for edge detection, can help improve detection performance. Table II presents the rules added to Table I to constitute the enlarged rule base set.

1 )

1 )

( )

( +

C. Defuzzication Defuzzication converts the output of the fuzzy process logic into crisp (i.e., numerically precise) solution variables [1]. To make the nal decision about the edge-direction at the pixel p x; y , defuzzication for our fuzzy detector takes the direction with the maximum membership value, as described by (2). If some membership values are equal, priority is given in decreasing order to directions 0 , 045 , 45 , 060 and 60 . The overall result of defuzzication is the orientation of the interpolation lter. Interpolation can then take place

( )

10 ; (x; y) is SMALL6 1 ; (x; y) is SMALL6 AND 1 ; (x; y ) is SMALL6

IF
1 45

45

AND

Direction x; y THEN dir x; y

0 45

45 45

1 45

( ) = 45

( ) = arg max(dir x;y  ): 

( )=

(2)

D. Implementation The simplicity of the fuzzy detector ensures real-time implementation and low hardware cost. When the position of the membership functions is xed, they can be implemented using lookup tables whose sizes are determined by the d6 parameters. Using the original rule base, 68 mathematical operations are necessary for each pixel to interpolate. The number of operations for the enlarged rule base is 134. If product conjunction is used, 13 products are needed with the original rule base compared to 39 for the enlarged rule base. Continuous membership functions can also be used. The number of operations then depends on the input value position in relation to parameters a6 and d6 of the membership

which is equivalent to IF

10 ; (x; y) AND 1 ;
1 45

0 45

are SMALL645 THEN dir x; y

AND 1 ; (x; y ) ( ) = 45:

1 45

In this rule base, only one rule is used to detect each of the ve directions. In addition, the number of antecedents in the rules increases for directions farther from 0 . This method reinforces rule decision by considering adjacent pixel variations in the same direction. Fuzzy singleton consequences are used to dene the direction dir x; y detected by these rules.

( )

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 7, NO. 3, JUNE 1997

541

functions. More operations are needed during fuzzication if an input value lies in between a and d for the given membership function (one subtraction and one division with a constant). A maximum of 129 and 215 operations for, respectively, the original and enlarged rule base are possible if all inputs must be fuzzied this way. E. Special Remarks Before presenting detection results with this fuzzy detector, some remarks about its decision process must be presented. When minimum conjunction is used, the direction detected corresponds to the direction of the pixel variation with the highest membership value (which reects the smallest pixel variation) according to its membership function SMALL6 . If a6 equals d6 for all membership functions SMALL, then binary logic detection is realized according to the decision threshold d6 and direction priority during defuzzication. The use of a different membership function for each direction allows the lter to characterize independently the pixel variation inuences according to its direction. By adjusting these membership functions for each direction, the decision becomes more fuzzy in nature, instead of following binary logic principles. Using the product conjunction operator also increases the fuzziness of the interpretation. The unit membership range value dened by a6 for the membership functions SMALL allows the lter to neglect small differences in pixel variations for different directions (which are then all considered to be small) and to use direction priorities to make the nal decision. The membership range given by d6 for these membership functions denes what pixel variation can be considered to be small. Finally, the membership range between a6 and d6 makes use of pixel variation values to select the appropriate direction, because the truth level is determined between zero and one according to them. III. DETERMINATION OF MEMBERSHIP FUNCTION PARAMETERS FOR THE FUZZY EDGE-DIRECTION DETECTOR One of the most critical stages of designing a fuzzy system is to nd the appropriate membership parameters. A heuristic and iterative procedure is usually needed to nd the appropriate fuzzy partition [4]. With the fuzzy detector described in this article, six membership parameters for the three membership functions must be determined: d0 ; a0 ; d645 ; a645 ; d660 ; and a660 . To do so, numerous experiences following an iterative methodology have been made to understand membership parameter inuences on edgedirection detection. Detection quality has been evaluated visually by examining edge-direction continuity and appropriate detection for different images. First, all a6 parameters have been kept at zero in order to study detection inuences of the d6 parameters. When all d6 parameters are identical, detection quality is mediocre but is improved when these parameters are all increased until they reach the value 50. This range denes what can be considered to be small pixel variations to be used for detection. When different d6 parameters are used between directions, the case d0 < d645 < d660 gives coarse detections and also incorrect detections in some places. The case d0  d645  d660 results in better detections. Detection quality is better then when all d6 parameters are identical, but still needs to be improved. In all of these cases, using product conjunction instead of the minimum causes a small increase in detection quality. Second, experiences with non zero a6 parameters have been done. When all d6 parameters and all a6 parameters are respectively identical to each other, detection quality greatly increases. Small

Fig. 4. Detection of

045 using the original rule base on image Flower.

Fig. 5. Interpolation result using the fuzzy detector with the original rule base on image Flower.

detection imprecision visible with null a6 parameters are eliminated. Using product conjunction instead of the minimum gives similar results, but the a6 parameters must not be too close to the d6 parameters in order to maintain good detection quality. When different a6 parameters are used, and by keeping all d6 parameters identical, the condition a0  a645  a660 gives the best results. The case a0 < a645 < a660 causes clusters of incorrect detections. Finally, based on the observations made during these experiences, it has been then possible to adjust the a6 and the d6 parameters to further increase detection quality. Results are presented in Section IV. By following this methodology for determining the membership parameters, we have been able to understand that for each direction, the d6 parameter must rst be set to identify appropriate small pixel variations i; x; y , and then the a6 parameter must be adjusted so that the importance and the interpretation of small pixel variations be preserved. By using different d6 and a6 values for each direction, distinct linguistic interpretations of SMALL are dened in the context of the direction evaluated.

1 ( )

IV. DETECTION AND INTERPOLATION RESULTS USING THE FUZZY EDGE-DIRECTION DETECTOR Even without rotation, the temporal-vertical interpolation lter used in our research is quite good by itself. To see its dif-

542

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 7, NO. 3, JUNE 1997

Fig. 6. Detection of

045 using the enlarged rule set on image Flower.

Fig. 7. Detection of

045 using the linear detector on image Flower.

culties on interpolated moving diagonal lines, we must look at dynamic image sequences. Therefore, it is difcult to present here the interpolation differences when the fuzzy edge-direction detection lter is used. Consequently, the results presented in this section are oriented to demonstrate the effectiveness of the lter to detect adequately the appropriate directions in the well-known image Flower. Membership function parameters used for the following gures are

do two eld iterations and with only a small increase in processing operations for the rst eld iteration. V. CONCLUSION This article has presented the characteristics of a fuzzy edgedirection detector. It has proved to be an efcient method for detecting edge directions oriented in 0 ,6 45 and 6 60 , which can then be used to orient a temporal-vertical interpolation lter. The proposed detection policy implemented in the rule bases evaluates small pixel variations in these directions to extract edge-direction. One advantage of the fuzzy detector presented is the possibility of adjusting membership function parameters to easily adapt or optimize detection characteristics. Other types of processing, like elimination of isolated detection, can be used after the fuzzy edge-direction detections to improve interpolation quality. ACKNOWLEDGMENT The authors would like to thank Miranda for its support in this research and N. Plaziac for taking the photographs. The authors also thank the anonymous reviewers for their valuable comments and suggestions. REFERENCES
[1] E. Cox, Fuzzy fundamentals, IEEE Spectrum, pp. 5861, Oct. 1992. [2] V. G. Devereux, Standards conversion between 1250/50 and 625/50 TV systems, in Proc. IBC 1992, Amsterdam, pp. 5155. [3] T. Doyle and M. Looymans, Progressive scan conversion using edge information, in Signal Processing of HDTV II. Amsterdam: Elsevier Science, 1990. [4] C. C. Lee, Fuzzy logic in control systems, IEEE Trans. Syst., Man, Cybern., vol. 20, pp. 404435, Mar./Apr. 1990. [5] E. Levrat, V. Bombardier, M. Lamotte, and J. Bremont, Multi-level image segmentation using fuzzy clustering and local membership variations detection, in Proc. IEEE Int. Conf. Fuzzy Systems, 1992, pp. 221228. [6] D. M. Martinez and J. S. Lim, Spatial interpolation of interlaced television pictures, in Proc. ICASSP, 1989, pp. 18861889. [7] S. K. Pal, Fuzzy set theoretic tools for image analysis, Advances Electron., Electron Phys., vol. 88, pp. 247297, 1994. [8] F. Russo and G. Ramponi, Nonlinear fuzzy operators for image processing, Signal Processing, vol. 38, pp. 429440, 1994. [9] G. J. Tonge, Image processing for higher denition television, IEEE Trans. Circuits Syst., vol. CAS-34, pp. 13851398, Nov. 1987. [10] Miranda Recherche Inc., Montr al, Qu bec, Canada, private communie e cation.

and a660 = 10, using the product conjunction operator. Fig. 4 shows 045 detection using the rule base with one rule per direction. Continuous detections are made at the branch and on the top of the houses. The interpolated image using the same fuzzy detector conguration is presented in Fig. 5. Fig. 6 presents 045 detection using the enlarged rule base. In general, detections made with the enlarged rule base are sharper than the ones made with the original rule set, but at a higher processing and hardware cost. We have compared our fuzzy detector with a linear detector using the same pixel window. A small difference in a particular direction between the two adjacent lines of the pixel to be interpolated serves to identify the edge-direction. This linear detector is described by

d0

= 80;

a0

= 20;

d645

= 60;

a645

= 10;

d660

= 40;

J (x; y )
= abs

2
j= 2

p(x + j +  ; y 0 1) 0

2
j= 2

p(x + j 0  ; y + 1)
(3)

Direction(x; y ) = arg max(J (x; y )):



(4)

This detector requires 54 operations but cannot by itself correctly identify edge-directions because it is sensible to small differences in pixel variations, and because pixel permutations can cause false detection. In a second eld iteration, this detector keeps only the detections which can be correlated with their adjacent horizontal detections, according to the following conditional statement (written in pseudocode): IF (Direction(x; y ) == Direction(x + 1; y )) AND (Direction(x; y ) == Direction(x 0 1; y ))

THEN NewDirection(x; y ) = Direction(x; y ) ELSE NewDirection(x; y ) = 0

Fig. 7 presents 045 detection using the linear detector. In comparison, our fuzzy detector gives better results without having to

(the default orientation for the interpolation lter):