A Critique of JPEG Decoding INTRODUCTION

In todays world JPEG is the most common format for storing and transmitting images on the World Wide Web, which is why any device supporting internet usage needs to be capable of JPEG Decoding. The name "JPEG" stands for Joint Photographic Experts Group, the name of the committee that created the JPEG standard and also other still picture coding standards. The JPEG is a DCT-based standard where the encoding process comprises of Forward DCT of image data followed by quantization and then by Entropy encoding. The Decoding of JPEG being the reverse process.

The Reconstructed image data obtained after decoding is in YCbCr format which still needs to be color converted to RGB format before being used by the display.

BITSTREAM SYNTAX AND DECODING PROCESS

There are four distinct modes of operation under which the various coding processes are defined: sequential DCT-based, progressive DCT-based, lossless, and hierarchical, where sequential DCTbased mode is the most common one and is the subject of this papers discussion. A jpeg file typically consists of Markers followed by the encoded data, where the encoded data may be present in one scan (as in sequential mode) or in several scans (as in progressive mode). Markers serve to identify the various structural parts of the compressed data formats such as Huffman tables, quantization tables and frame data. The initial task of any decoder would be to parse the markers for getting a summary of the encoded data that follows such as height, width, subsampling information for allocating the various resources needed for the actual decoding. The JPEG decoding process is an inverse transformation where the encoded data is first decoded and restored to 8 pixels by 8 pixel

data blocks using a preliminary Decoding stage. This stage includes both Huffman Decoding and Run-Length decoding two distinct coding schemes. Next the Quantization table specification is used to approximately regain the spectral components of the image block, while low frequency components may be fully restored the high frequency components may be severely distorted however this distortion is barely perceptible. Finally the Inverse Discrete Cosine Transform approximately recovers the original 8x8 data block. Now all the previous computations are constant time operations i.e. for a particular implementation (hardware and software configuration) they take the same machine instructions for any 8x8 encoded input. The process of color conversion is also similar in nature, where a particular implementation takes constant time for any kind of input. The total time T1 for decoding a jpeg file is hence Where, b is the time taken for decoding a macroblock , N is the total number of Minimum coded Units [MCU][1], is the total time taken for parsing the various markers and frame data. The Color space conversion is a time consuming operation. It has been reported that it consumes up to 40% of the processing time of a highly optimized decoder [3, 2]. The total time T2 taken for color space conversion by any decoder is Where, c is the total time taken for converting one MCU.

COMMERCIALLY AVAILABLE MULTIMEDIA HARDWARES

The commercially available multimedia hardwares now a days have hardware blocks for IDCT, Entropy Decoding and other video computations. These blocks are clubbed together in a subsystem where they can be pipelined together. For Color conversion a separate hardware is provided, which can either be pipelined with the video computation blocks or is present in a separate subsystem where it cannot be directly pipelined with the video computational blocks. In such cases where the direct pipelining is not possible, the host processor needs to configure the Color conversion hardware to access the output of the decoder from external memory.

DECODING MODES
The interaction between the decoder and color convertor can be in two modes namely, Image mode and Segment/Slice mode. In Image mode the complete image is first decoded to YCbCr format, which is then color converted, hence the total time taken for the whole use-case is T1 + T2 and

memory ( proportional to the dimensions of the image) needs to be reserved for holding YCbCr and RGB images. Alternatively the decoder can decode a segment of the image and while it is decoding the next segment, the previous segment already decoded can be fed for color conversion. This mode clearly has its advantages in terms of memory and execution time, since fixed size segments need to be accommodated in memory and time can be saved due to the parallel execution of image decoding and color conversion. Assuming the exchanged segments between the decoder and color convertor, have a size of k MCUs (hence total segments) and is the time taken for configuring and starting the color convertor for operating on a segment then, the time taken for decoding a segment is (excluding the time taken for header parsing) and the time taken for color conversion of the segment is . Now, if if if ( ( ) ( ) ) , , .

EXTENT OF PIPELINING
Assuming in equations (3), (4) and (5) values and are optimized (to the maximum possible level), then Total time depends on the choice of and on the value of . The value heavily depends on the placement of the color conversion hardware since if pipelining (at the hardware level) is possible, would be minimum whereas if the host processor intervention is required, then shall include the latency of the host processor and external memory read time which is usually greater than the hardware pipelining. The value depends on the minimum size of the decoder output and color convertor input, if both hardwares can operate at the MCU level then the value of can be brought to 1. Hence the total time is least, when color conversion block and decoder are hardware pipelined and interact with individual MCUs.

REFERENCES
[1] jpegstandard [2] M. Bartkowiak. Optimisations of Color Transformation for Real Time Video Decoding. In Digital Signal Processing for Multimedia Communications and Services, September 2001. [3] F. Bensaali and A. Amira. Accelerating Colour Space Conversion on Reconfigurable Hardware. Image and Vision Computing, 23:935942, 2005.