EE Times - Polyphase Video Scaling in FPGAs

1 / 6

Design How-To

Polyphase Video Scaling in FPGAs

Suhel Dhanani, Altera
9/6/2007 07:00 AM EDT Post a comment Tweet
0

Video scaling is an increasingly common function used to convert images of one resolution and aspect ratio to another "target" resolution and/or aspect ratio. The most familiar example of video scaling is scaling a VGA signal (640 x 480) output from a standard laptop to an SXGA signal (1280 x 1024) for display on LCD monitors. For high-volume systems dealing with standardized image sizes (i.e. high definition (HD) television), video scaling is most efficiently done using application specific standard products (ASSPs). However, many video applications such as video surveillance, broadcast display and monitoring, video conferencing, and specialty displays, need solutions that can handle custom image sizes and differing levels of quality. This often requires custom multi-tap polyphase scaling algorithms. FPGAs with an array of high-performance DSP structures are ideally suited for such algorithms, and FPGA vendors are beginning to offer usercustomizable video scaling IP blocks that can be quickly configured for any application. This article gives a basic overview of polyphase video scaling followed by a detailed description of how to implement polyphase scaling in an FPGA. Understanding Video Scaling Video scaling, whether upscaling or downscaling, is the process of generating pixels that did not exist in the original image. To illustrate this, let's look at a simple example; scaling a 2 x 2 pixel image to a 4 x 4 pixel image, as shown in Figure 1. In the 'New Image', the white pixels are pixels from the 'Existing Image' and the black pixels are those that need to be generated from the existing pixels. There are many methods for generating new pixels; the simplest is called the nearest neighbor method, or 1x1 interpolation. In this method a new pixel value is simply equal to the value of the preceding pixel. In a slightly more sophisticated approach, called bilinear scaling, a new pixel is equal to the average of the two neighboring pixels in both the vertical and horizontal dimensions. Both of these techniques are illustrated in Figure 1.

http://www.eetimes.com/document.asp?doc_id=1275450&print=yes

2013-11-28

EE Times - Polyphase Video Scaling in FPGAs

2 / 6

(Click to enlarge) Figure 1. A 2 x 2 pixel image is enlarged (upscale) to a 4 x 4 pixel image. Bilinear scaling implicitly assumes equal weighting of the four neighboring pixels, i.e. each pixel is multiplied by 0.25. In video terminology this scaler function is scalar (all coefficients are equal); it has two taps in the horizontal dimension, two taps in the vertical dimension, and has one phase (i.e. one set of coefficients). The term "taps" is in reference to filter taps, as scaling is mathematically identical to generalized filtering, i.e. multiplying coefficients by inputs (taps) and summing. To further illustrate this, consider the 4x scaler shown in Figure 2. Here four new (black) pixels are generated for every one existing (white) pixel. Like the previous example, it has two taps in the horizontal dimension. Unlike the previous example, each new pixel is generated using a different weighting of the two existing pixels, making this a four phase scaler. The weighting coefficients for each pixel are given below.
l l l l

New pixel 1 will just be a copy of the original pixel 1 New pixel 2 will weigh 75% of original pixel 1 and 25% of original pixel 2 New pixel 3 will weigh 50% of original pixel 1 and 50% of original pixel 2 New pixel 4 will weigh 25% of original pixel 1 and 75% of original pixel 2

Figure 2. Two pixels are used to generate the value of the new pixels. More complex algorithms Bilinear and nearest neighbor are the easiest scaling algorithms. Now let's consider some more complex, more commonly used algorithms. Figure 3 shows an example of what is known as bicubic scaling, or 4 x 4 scaling. In this example, a sample 4 x 4 matrix of image pixels is downscaled by a factor of 4 in both the horizontal and the vertical dimension (i.e. a 4 x 4 matrix of pixels is reduced to a single pixel).

http://www.eetimes.com/document.asp?doc_id=1275450&print=yes

2013-11-28

EE Times - Polyphase Video Scaling in FPGAs

3 / 6

A resource-efficient way to implement this 2-D scaler (i.e. 2-D filter) is by cascading two 1-D filters, as shown in Figure 3. First, each vertical line is filtered to create four pixels. Then, the horizontal pixels are filtered in the horizontal dimension to create a single output pixel. Figure 4 shows an efficient FPGA implementation of this algorithm.

(Click to enlarge) Figure 3. Scaling first in the vertical dimension and then in the horizontal dimension.

(Click to enlarge) Figure 4. FPGA architecture used to implement a 2-D scaling by cascading two 1-D scalers. The implementation in Figure 4 consists of two stages, one for each 1-D filter. In the first stage, the vertical lines of pixels are fed into line delay buffers and then fed to an array of parallel multipliers. The outputs of the multipliers are then summed and sent to a 'Bit Narrower' which adjusts the size of the output to fit into the number of bits allowed. The second stage has the same basic structure as the first, and filters the 'horizontal pixels' output from the first stage to produce the final output pixel. This structure can be extended to perform 5 x 5, 6 x 6, or 9 x 9 multi-tap scaling. The principles remain the same. However, larger kernels will require more FPGA resources. As mentioned previously, each new pixel may use a different set of coefficients depending on its location; thereby requiring different coefficient sets. This type of efficient implementation is feasible only in a programmable logic device such as an FPGA with abundant hardware resources. In the implementation presented, four multipliers are needed for scaling in the vertical dimension (one for each column of the scaling kernel), four multipliers are need for scaling in the horizontal dimension (one for each

http://www.eetimes.com/document.asp?doc_id=1275450&print=yes

2013-11-28

EE Times - Polyphase Video Scaling in FPGAs

4 / 6

row of the kernel), and a significant amount of on-chip memory is needed for delay-line buffers. DSP architectures, which typically have 1-2 multiply-and-accumulate (MAC) units and significantly lower memory bandwidth, do not have the parallelism for such an implementation. This makes FPGAs the only real choice for quickly implementing custom multi-tap scaling of HD quality images in real time. FPGA implementation Implementing Video Scaling in FPGAs To speed up FPGA implementations of video scalers, FPGA vendors provide IP megacores that abstract away all the mathematical details, enabling designers to implement highly complex scaling algorithms in a matter of minutes. These megacores can generate a set of scaling coefficients using a standard polynomial interpolation algorithm or allow the customer to use their own custom coefficients. Figure 5 shows the GUI of an Altera-based polyphase scaler. The scaler allows implementation of both simple nearest neighbor/bilinear scaling as well as polyphase scaling. For polyphase scaling, the number of vertical and horizontal taps can be selected independently. The number of phases can also be set independently in each dimension. Figure 6 shows an example coefficient set generated by the IP function for a four tap, 16phase filter.

(Click to enlarge) Figure 5. A scaling function allows easy implementation of multi-tap, polyphase scaling algorithms.

Figure 6. A preview of the vertical coefficient set created by a 4-tap, sixteen phase filter. Coefficients can be selected by the user or derived by choosing one of the built-in filtering

http://www.eetimes.com/document.asp?doc_id=1275450&print=yes

2013-11-28

EE Times - Polyphase Video Scaling in FPGAs

5 / 6

functions shown in Figure 7. The bicubic scaling mode uses cubic curve-fitting. Scaling modes higher than bicubic (i.e. 5 x 5, 6 x 6, etc.) use polynomial fitting. Scaling coefficients are generated based on the Lanczos function. The Lanczos algorithm is a multivariate interpolation method, which uses a windowed product of sinc functions to calculate the coefficients. A more exhaustive description of this function is beyond the scope of this article.

Figure 7. A scaling megacore can implement custom OR built-in coefficients based on the Lanzcos interpolation technique. In polyphase mode, the scaling parameters must be chosen carefully. The parameters which have the largest effect on image quality are the number of taps and the filter function used to generate the coefficients. The number of phases and number of bits of precision used are less important to the image quality. Altera provides a set of guidelines on selecting scaling parameters in their Video Image Processing User Guide. Conclusion Video is rapidly going HD, not only in television sets, but in video conferencing, surveillance, industrial, medical, military, and broadcasting systems. This is necessitating an exponential increase in the amount of processing power required to encode, decode, resize, sharpen, or enhance HD quality video streams. FPGAs are unique in providing the required digital signal processing power in a single device. Mapping a video design onto an FPGA is no trivial design task. However, FPGA vendors are beginning to provide a range of increasingly sophisticated IP cores that can dramatically simplify the mapping of complex algorithms onto an FPGA platform. System designers wishing to realize the performance, price, and power advantages of FPGAs, while shortening their design cycles, would be well advised to familiarize themselves with the ever expanding range video-oriented IP offered by FPGA vendors. About the author

http://www.eetimes.com/document.asp?doc_id=1275450&print=yes

2013-11-28

EE Times - Polyphase Video Scaling in FPGAs

6 / 6

Suhel Dhanani is Senior Product Marketing Manager in the IP and Technology Marketing group. He is responsible for all product marketing aspects of Altera's video and image processing solutions. Suhel has over 15 years of industry experience in semiconductors with both large companies such as Xilinx and VLSI Technology as well as with Silicon Valley startups such as Anadigm and Tabula. Suhel has completed a graduate certificate in Management Science from Stanford and also holds M.S.E.E. and M.B.A. degrees from Arizona State University.
EMAIL THIS PRINT COMMENT Copyright 2013 UBM Electronics, A UBM company, All rights reserved. Privacy Policy | Terms of Service

http://www.eetimes.com/document.asp?doc_id=1275450&print=yes

2013-11-28