158

IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 43, NO. 2/3/4, FEBRUARY/MARCH/APRIL

1995

Reversible Variable Length Codes

Y. Takishima, M. Wada and H. Murakami

Abstract - In t h i s paper, we propose s o m e reversible variable length codes (RVLC's) which can be decoded instantaneously both in the forward and backward directions a n d have high t r a n s m i s s i o n efficiency. These codes can be used. for example, in the backward reconstruction of video signals from the data last received when some signal is l o s t midway i n the t r a n s m i s s i o n . Schemes f o r a symmetrical RVLC requiring only a single code table and for an asymmetrical RVLC having short a v e r a g e code length are introduced. They compare favorably with other reversible codes such a s B 2 codes in several aspects.

1. IhTKODI.:CTION

Variable length ccdes (VLC's) are of prime importance i n the efficient transmission o f digital signals with non-uniform occurrence probability distributions. Compact codes such as the Huffman codes, [l], which have the highest efficiency and shortest average code length, are most prevalent in practical operation. However, there arc other VLC criteria than the transmission efficiency that may be important in the application environment, for esample, channel bit-error resilience, 121, 131, masimum code word length limitation due to hardware capacity, 141, etc. Reversibility o f variable length codes that makes instantaneous decoding possible both in the forward and backward directions is one such criterion. For example, in the Dif-ferential Pulse Code Modulation (DPCM) coding used for data compression in video coding, a VLC is assigned to the differential signals. This scheme has a problem, however, i n that a single bit error on one differential signal in the channel causes continuous errors in the reconstructed signals even if n o other transmission bit error oxcurs. A backtracking scheme is a promising solution for avoiding continuous errors. I t can detect the erroneous signal, and precisely reconstruct the following signals by sending the PCM value o f the last signal and tracking backward from it provided that no more than one signal is damaged by bit errors between the signals whose PCM values are transmitted, (51, 161. Fig. 1 shows an example of 1-D DPCM (transmission of more PCM values makes backtracking possible also for 2-D DPCM). This scheme can provide error resilient transmission with little overhead. Another potential use of this code is in the random access of o r in the searching of an entropy coded stream of data. The ability to search in two directions should halve the amount o f indexing overhead or the number o f index points necessary to achieve the same average search time as compared to a onedirectional VLC. A reversible variable length ccxle (RVLC), however, must satisfy the suffix condition for instantaneous backward
Paper approved by Stephen G. Wilson, the Editor for Coding Theory and Applications of the IEEE Communications Society blanuscript received February 1 I. l(992;revised August 3 I. 1'992 and June 2, lCB3. Y Takishima, ,GI Wada and H. Murakami are with KDD R&D laboratories, 2-1-15 Ohara. Kamifukuoka-shi, Saitama, 3-56. J a r " IEEE Log Number 9410040. 0090-6778/95$4.00

decoding as well as the prefix condition for instantaneous foward decoding. The suffix condition is that each code word does not coincide with the suffixes of longer code words, while the prefis condition expresses that there is no coincidence with the prefixes of longer code words. For example, in decoding a conventional VLC, C i l , in Table 1 which satisfies only the 0100" (left t o right in the prefix condition, a bit stream forward direction) cannot be decoded instantaneously in the backward direction because the last two bits "00" might be either symbol "A" or the suffix of "D". The suffix condition is sufficient for instantaneous decoding i n the backward direction just a the prefix condition is in thc forward dircction. In this paper, the basic steps for constructing a RVLC are proposed. Section 2 describes a RVLC composed entirely of symmetrical code words, and Section 3 describes a RVLC composed of asymmetrical code words. Section 4 compares the characteristics and average ctde lengths of these codes with the Huffman ccxle, the fixed length code and the B2 code.
'I------

correct decoding

-_-_-_-_-_-_(a) fonvard dccding only wrong decoding

X
error p i n t

0
pixel whose PCM value is transmitted
(b) forward and backward decoding

Fig 1 Reconstruction of signals i n 1 3 R : ! v v I coding

Table 1 lluffman code and reversible vanable length codes (RVLC's)

Sym- h o b bility
A B C

(31

ClZ

c 1 3

c I J

cl5

D E

0.33 0.30 0.18 0.10 0.W

00 01 11 100 101 2.19

00
I1 010 io1 0110

00 01 II 1010 10010

00 01 10
111 110

00 01 10 Ill 11011

Average code length

7.46

2.37

1.37

(111 IIuffman code ( 3 2 Symmetncal RVLC ( 3 3 Asymmetrical KVLCl

0 1995 IEEE

Ci-c non-RVLC
C 15 .\symmetncd KVLC1

___

IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 43, NO. 2/3/4, FEBRUARY/MARCH/APRIL 1995

159

11. SYMMETRICAL RVLC

ClexIy, a VLC cc~inposedentirely of sy"tr-ical code words is reversible. Furthermore the samc code table can be used for decoding both in the fclrward and backward directions. This is an advantage from the viewpoint of memory usage and simplicity. A symmetrical RVLC with high transmission efficiency a n be designed by staning from a non-reversible VLC with an optimum bit assignment of code words for the txcurrencc probabilities, such ; 1 s a Huffman code, and converting the ccxle by a top-down scheme into a symmetrical RVLC. This process is justified when we consider that the bit lcngth vector of a RVLC, (nrev( 11, nreV(2),...... , nrev(12rev-mar))whcre nrev(i) is the number of code words having a length i and Lrev-mar is the minimum length of code words in this code, is restricted in the top-down direction in the convcrsion process U follows. On a binary tree, symmetrical code words are located as shown in Fig. 2. The number of' symmetrical code words of' length I,, m&!,), is given as follows. If L = 2.k where k 2 1 and k is an integer. mciId)= 2

( i i r ) u(i, I , ) = 0 in all other ca..cs. Considcnng the decrease o f available symmctrical ctxic words under the pref'ix condition:

m(L)=:mo(L)-

( Eu(
1-1

[ L S ~ J

i, L ) *n(i) - x(L)

where n(i) are components o l the bit length vector of the

(4)

original VLC, and x(L) is total number of symmetrical code

words unavailable due to the violation of thc prefix condition in ciiqe (ii) above, while:
L- 1

x(L)=
I

-1

,F(i.L)

(5)

i . I. 21 + I

-, (1) otherwise (I, = 2*k+1 where k 2 0 and k is an integer), mdl,)= 2k.2

7Li?

- 9 (L+1) / 2 -

(2)

tngxlJ)= 2['L+')!21 (3) where 1x1 is the largest integer less than o r equal to x. On the other hand, the number o f symmetrical code words of length 1, i n an instantaneously decodable VLC, m(L), satisfying the prefix condition which means the suffix condition at the same time in this case is given as follows. A node at the i-th level from the root (P and S in Fig. 3) is included in as many as u(i, L ) symmetrical code words of length L ( L > i), where: L (i) uti, 15)= m&-2) when i 5 7 ,and the part of length L 2 i

..

from P to Q in Fig. 3 (a) is symmetrical; I, (ii) u(i, L ) = 1 when i I,, and the part of length 2i-L 2 from R to S in Fig. 3 (b) is symmetrical;

where .i(i , L ) is the number of code words o f length i whose ( Z - L ) bit suffixcs (Fig. 3 (b)) are symmetrical. In (4) the first and second terms are decided only by the code word length, L , and the bit length vector, n(i), while the last term, x ( L ) , depends on the bit alignment pattern of each code word. This implies that a RVLC with a diflerent bit length vector can bc gcncritcd staling from a VLC with a different bit alignment pattern but an idcntical bit length vector. Based on the above restriction, a symmetrical RVLC is obtained as follows: (i) The bit lcngth vector o f thc target symmetrical RVLC is initialized by that of the starting VLC: nrev(i)= n(i) (v i 2 1) . (6) (ii) The number of code words o f length i, nrev(i),is restricted by the bit alignment of code words of length less than i. If nrev(i)5 m ( i ) ,nrpv(i) is not changed. Otherwise one bit is added to some ccde words, 1.c.: nrev(i+l) = nrCv(i+l) + nrev(i)- m(i) , (7) tirev(i)= m ( i ) . t 8) (iii) Step (ii) is rcpeatcd until thc bit allocation is finished for all ctxie words. For example, Table 1 shows a Huffman ccde (C11)and a symmetrical RVLC ( C p ) produced from Cl 1. Note that when the original VLC is a bit-by-bit code whose bit length vector is (1, 1, ......, 1, 2 ) , these steps generate the RVLC given in 151. Decreasing x ( L ) in (5) to provide a more efficient RVLC is a topic for further study.

(a) when i s

0
0

symmetrical ccwle word asymmetrical code word

L c k L 2 rvhcrc i is thc !cngth o fi l codc word and Lis the level of symmetry found 0 . red n d c 0 : virtual node 0 : real leaf :virtu l a i f
(h)when-

L 2

Fig. 2 Dstnbution d symmetrical code words

Fig. 3 Bit length and symmetrical code words

160

IEEE TRANSACTIONS O N COMMUNICATIONS, VOL. 43, NO. 2/3/4, FEBRUARY/MARCH/APRIL 1995

111. ASYbYC\.ErRI(:.\I. KVLC

(111)
~I

I n most cases an asymmetrical RVLC offers better efficiency than a symmetrical RVLC because a more flexible bit assignment is allowed, although two types of coding tables are necessary. An asymmetrical RVLC can also be obtained from a base VLC by adding bits. A s was mentioned previously, the suffis condition is sufficient for instantaneous backward decoding' An instantaneously vLc in the forward direction satisfies the prefix condition. On the other hand, the suffix of each code word can be extended by adding bits, which does not prevent instantaneous forward decoding. This property suggests an effective method of generating an asvmmetrical RVLC, or add,ne . . Some bits to the bottom of specific code words to satisfy the suffis condition. The steps for producing a n asymmetrical RVLC are as follows (See Fie. 4 ) : (i) A reverse binary tree is set up in such a way that the end of the code words are placed on the root from shorter code words to longer code words (Fig. 4 (b), (c)). (ii) When a suffix which is also a prefix in the reverse tree coincides with a shorter code word, another code word with the same bit length is assigned in the reverse tree instead. In Fig. q d ) "D", is assigned in the reverse tree instead of "C" since the suffis of "C" coincides with " A " .
Y I

When there is n o other code word with the same bit length whose sufl'is does n o t coincide with onc o f the shorter code words, the minimum number of bits necdcd t o satisfy the suffix conditionare added to the of the code word, I n Fig. (e) one bit ,, 1,, is attached to the end of ,,r,,
L .

(iv) After bit length assignment is completed, new code words are sorted by bit length and they are reassigned to the symbols according to their probabilities (Fig. RVLC For esamplc, c,3 i n Table is an produced from 'l1' Incrmse of average code length duc to these Steps, AI,av7 is expressed as:

4 ( 1 ) ) .

AIAuv =
j

s,,

2
=I

hL(i)*P(c,,v(i)+Ac.~v(i)),

(9)

11

(a) original code

S
L)

(b) reverse t r w - 1
101

9pI
(c) reverse tree - 2
(d) reverse tree - 3

where S , is the number of code words requiring additional bits, A L ( i ) is the length of the additional bits, P o ) is the occurrcncc probability of the j-th code word, cw(i) is the sequential number of the code \vord rcquiring additional bits and cw(i)+Acw(i) is the sequential number after sorting of- the code rvord with additional bits. We find i n (9) that a VLC with fewer code words whose suffixes coincide with shorter code nords can be converted to a RVLC with shorter avenge codc length even if the averagc code length o f the original VLC is equivalent to that of other VLC's. Making use of this property, we can improve the transmission efficiency of a RVLC. Equivalent transforms, [3], that convert a VLC to another with the same average code length by exchanging a node or a leaf at the same level in a binary tree can provide a VLC with fewer code words coinciding with the suffises of longer code words (Fig. 5). The steps are as follows: (i) The shortest and most probable code word is selected as the seed word. (ii) I f necessary, an equivalent transform is carried out at an appropriate level in ccxie words longer than the seed word so that the seed word does not coincide with their suffises. (iii) When the suffixes have been checked on all the longer code words, the next probable code word is selected as the seed word, and step (ii) is repeated. In these steps, a node or a leaf that has already been checked l'or the suffix condition is not exchanged by an equivalent transform. The VLC C11 is converted to C14 by this method as shown in Table 1. A RVLC (CIS)produced from C 14 has a shorter average code length than C13.

101

(e) reverse tree - 4

(f) asymmetrical RVLC (after sorting)

(a) more coinciding suffixes A: suffix of D B: suffix of E

(b) fewer coinciding suffixes C: suffix of D

Fig. 4 Conversion into asymmetncal RVLC

Fig. 5 Minimiung coincident surfixes by equivalent transform

IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 43, NO. 2/3/4, FEBRUARY/MARCH/APRIL 1995

161

Iv.

CObIPAKISON A,L1ONCi I<VI,C'S

v.

CONCI ,LJSIOX

Some properties of the symmetrical and asymmetrical RVLC's described in Sections 2 and 3, respectiveky, m compared to other types of reversible codes in this section. A fixed length code (FXC) and a 8 2 code, [7], are considered for reference. A R 2 code (Fig. 6) is composed of ctde words with evennumbered bit lengths; tdd bits of each ctxie word are continuity flag bits ('W' for the end of the code word and "1" for continuation), while even hit$ are information bits. Since the cont.in1iit.yfbg hit shnws t.he end of [ : d e words7iwtantawous tlcrtxline !Strictly .;pt:ikiny, tierociiny with 7 hits d ~ l : i y )i s p q i h l e in the hackward direction. The numtwr of c:twie wonlq of length I . is given hy:
"I>-\-, ll,.Jl)

Schemes for generating reversible variable length codes that can be decoded instantaneously both in the forward and backward directions were proposed. These codes are useful for preventing error propgiticm, and will he applicable t o playback of stored media o r fast searching nf an entropy c:c>ded stream of data. There is still a broad range of selection available in combining ones and zeros in both symmetrical and asymmetrical reversible variable length codes, and further improvement of average ctde length may be ,mssihle. Stiitiy of the mwimiim m d e word length is As(> an ripen prohlem
APi%hzi:i

2 /

p'

Table 2 shnws !he c*hawctcristirs of rach reversihlc emir anri

(!e)
;I

Huffman code for reference. I t is easy to design the FLC and the B2 ctxfe because their code word lengths can be decided independently of the occurrence probabilities 01 the symbols, although In man): cases they have a ionger average c d e iengih compared io oiiler P ~ ~ C <>f SRVLC'S ( t h r;cicgc ~ C ~ !ci13th C of EX'S Y/GU!C! bc short when the probability distribu!ior! is a n i ! ~ r m ,v&i!e th:1! of B3 codes would bc short when specific symbols have quitc high Occurrence probabilities). In order to compare the average code length of the reversible codes quantitatively, code words and average code length of each of these reversible codes and the Huffman code for the English alphabct, [ 3 ] ,arc providcd in Tablc 3 as an cxamplc. In the tilhle G I is a non-reversible Huffman ccxie, C22 is a FLC with cnde words of 5 bits, and C33 i s a 33,codc. A code C2.l is a symmetrical RVLC generated from C?!. A code C?j is XIasymmetrical RVLC directly produced from C ~ I while , C27 is another asymmetrical RVLC based on a non-reversible VLC (226 converted from by the equivalent transform and consideration of suffix conditions. In this example, the FLC ( C y ) and the B2 code C ( ), have 20.3 % and 13.1 % longer while the average length over the Huffman code (GI), symmetrical RVLC ((224) and the asymmetrical RVLC ( C y ) are 13.0 % and 4.9 % longer respectively. In transmission efficiency an asymmetrical RVLC is very advantageous and the equivalent transform is effective because i t provides 1.6 % improvement to get C27 from C25.

The maximum length of a B3 code is given by [log?_( N + 3)- 11 2 (A. 1) where N is the total number of code'words (in Table 3) and [ x ] is the smallest integer greater than or equal to x. Proof When the level of the tree is i (Fig. 6), - - . L = 2-1 ( A . '2) (A. 3; ~~mux= - ~22 imax - U?

Num (i ) = 3' (A. 4) where L is the length ot code words and Num(i) is the number 0 1 the code words respeclively at this level i , whiie Lm, ~3 is is thelevel the masimum length of the code and imcLr_~z corrcsponding to Lmax-~z. Thcn thc maximum numbcr of code words fully assigned in the tree, S,,,,,, is given as:
'_U

S,,,

2
I

-02

Niun( i)

I-

- 2 ' m J2+l - 2 . (A. 5) Note that for backward decoding only half of the nodes are assigned to the code words at the bottom level as well as at other levels, while all the nodes are assigned in usual forcvardonly cases for better transmission efficiency. This leads to an inequality:
z'nm-82

-2 < N

?imJ3Z+i-3

(A. 6)

Then:
imu.r-62 < log 2 ( N + 2 ) 5 imax-R2+ 1

imm-E = [log.( N

+ 2) - 11

(A. 7 (A. 8)
(A. 9)

where

[ X I is the smallest integer greater than or equal to x .

Lmax-B?, = [lc>ggz(N+2) - 11 2 .

..
level
i=l

ACKNOWLED(;~IE~UT The authors wish to express their deep appreciation to Dr. Y. Umno, &rector of Laboratones, for his encouragement and supporl. REFERE"
D. Huffman. "A method for the construction of minimum redundancy codes." in Proc. Inst. Radio Eng., vol. 40,pp. 1098-1101, Sept. 1952. Y . Talushima M. Wada and H. Murakami, "VariableLength Code with High Resynchronization Capability," the Journal of the Institute of Television Engineers of Japan, vol. 43, NO. 8 (1989) pp. 844-846. J. Maxted, J. P. Robinson, "Error Recovery for Variable Length Codes," I E E Trans. IT-3 1, 6, pp. 7'3.1801 (Xov.. 19885).

i=2

Fig. 6 Structure of B2 code

162

IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 43, NO. 2/3/4, FEBRUARY/MARCH/APRIL 1995

14) 11. blurakami, S. Matsumoto and 11. Yamamoto, "Algorithm lor

16) K. Matsui and 14. ,\chiha, Proceedings ol' the I980 National

(hnstruction of Variable Lcngth Code with Limited Maximum Word Length." IEX Trans., (:Obl-33, 10, pp. 1157-1 159 (Oct., 19%). 151 K. bl. Rose. "Enhancement of One-Ilimensional Variable-Length DPCM Images Corrupted by Transmission Errors," IEEE Trans.. COICI37.4, pp. 373-379 (flpr., 1989).

(_'onvention, IEICIi 1 ( M pp 5-73. 171 IWI' CLKI':3, Revised version ol Dralt Report ADICMTT: Digital l'ransmission of Component-Coded Television Signal at 30-34 blbitis and 45 Mbitis, June 1989.

Table 3- Comparison of 1ZVIl:'s -1 (Characteristics)

independent of pdf.

# I : Derivation is shown in the appendis & -83 iz-l 2l #7: !,AVE-B2 = 1 3- I 1 P( Nyrm ( I - 1) + j )
L =I

1'1

where L -

B2 =

[logz ( N + 3) -11

.v- .v,

~(Lmr m R7,z-i)

{3 , ,z

-82

( ' f N.u,,n( ,r

/jz '

1) +

0)

3,

Num(i)=

1 n(?k-l

k).

N: total number of code words, n(i): number of code words ol length i bits, P ( i ] : txcurrence probability oi r-th cvde word,
and [.TI : the smallest integer greater than or equal to .T. #3: L-AVE-SRVLC, L-AVE-ARVLC > L-AVE. and they are analyzed by examplcs.

Table 3 Comparison of RVLC's -3 (English alphabet)