158 IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 43, NO.

2/3/4, FEBRUARY/MARCH/APRIL 1995

Reversible Variable Length Codes

Y. Takishima, M. Wada and H. Murakami

Abstract - In t h i s paper, we propose s o m e
reversible variable length codes (RVLC's) which can
be decoded instantaneously both in the forward and
backward directions a n d have high t r a n s m i s s i o n
efficiency. These codes can be used. for example, in
the backward reconstruction of video signals from
the data last received when some signal is l o s t
midway i n the t r a n s m i s s i o n . Schemes f o r a
decoding as well as the prefix condition for instantaneous
foward decoding. The suffix condition is that each code word
does not coincide with the suffixes of longer code words, while
the prefis condition expresses that there is no coincidence with
the prefixes of longer code words. For example, in decoding a
conventional VLC, C i l , in Table 1 which satisfies only the
prefix condition, a bit stream 0100" (left t o right in the
'I------

symmetrical RVLC requiring only a single code table forward direction) cannot be decoded instantaneously in the
and for an asymmetrical RVLC having short a v e r a g e backward direction because the last two bits "00" might be
code length are introduced. They compare favorably either symbol "A" or the suffix of "D". The suffix condition
with other reversible codes such a s B 2 codes in is sufficient for instantaneous decoding i n the backward
several aspects. direction just a the prefix condition is in thc forward dircction.
In this paper, the basic steps for constructing a RVLC are
1. IhTKODI.:CTION proposed. Section 2 describes a RVLC composed entirely of
symmetrical code words, and Section 3 describes a RVLC
Variable length ccdes (VLC's) are of prime importance i n composed of asymmetrical code words. Section 4 compares
the efficient transmission o f digital signals with non-uniform the characteristics and average ctde lengths of these codes with
occurrence probability distributions. Compact codes such as the Huffman ccxle, the fixed length code and the B2 code.
the Huffman codes, [l], which have the highest efficiency and
shortest average code length, are most prevalent in practical
operation. However, there arc other VLC criteria than the
transmission efficiency that may be important in the
application environment, for esample, channel bit-error
resilience, 121, 131, masimum code word length limitation due
to hardware capacity, 141, etc.
correct decoding
Reversibility o f variable length codes that makes
instantaneous decoding possible both in the forward and -_-_-_-_-_-_-
backward directions is one such criterion. For example, in the wrong decoding
Dif-ferential Pulse Code Modulation (DPCM) coding used for (a) fonvard dccding only
data compression in video coding, a VLC is assigned to the X
differential signals. This scheme has a problem, however, i n error p i n t
that a single bit error on one differential signal in the channel
causes continuous errors in the reconstructed signals even if n o 0
other transmission bit error oxcurs. A backtracking scheme is pixel whose PCM
a promising solution for avoiding continuous errors. I t can value is transmitted
detect the erroneous signal, and precisely reconstruct the
(b) forward and backward decoding
following signals by sending the PCM value o f the last signal
and tracking backward from it provided that no more than one
Fig 1 Reconstruction of signals i n 13R:!vvI coding
signal is damaged by bit errors between the signals whose
PCM values are transmitted, (51, 161. Fig. 1 shows an
example of 1-D DPCM (transmission of more PCM values
makes backtracking possible also for 2-D DPCM). This Table 1 lluffman code and reversible vanable length codes (RVLC's)
scheme can provide error resilient transmission with little
overhead. Sym- h o b -
Another potential use of this code is in the random access bility (31 ClZ c13 cIJ cl5

of o r in the searching of an entropy coded stream of data. The A 0.33 00 00 00 00 00

ability to search in two directions should halve the amount o f B 0.30 01 I1 01 01 01
indexing overhead or the number o f index points necessary to C 0.18 11 010 II 10 10
achieve the same average search time as compared to a one- D 0.10 100 io1 1010 111 Ill
directional VLC. E 0.W 101 0110 10010 110 11011
A reversible variable length ccxle (RVLC), however, must Average
satisfy the suffix condition for instantaneous backward code 2.19 7.46 2.37 - 1.37
length
Paper approved by Stephen G. Wilson, the Editor for Coding Theory and
Applications of the IEEE Communications Society blanuscript received (111 IIuffman code Ci-c non-RVLC
February 1 I. l(992;revised August 3 I. 1'992 and June 2, lCB3. ( 3 2 Symmetncal RVLC C 15 .\symmetncd KVLC1
Y Takishima, ,GI Wada and H. Murakami are with KDD R&D
laboratories, 2-1-15 Ohara. Kamifukuoka-shi, Saitama, 3-56. J a r " ( 3 3 Asymmetrical KVLCl

IEEE Log Number 9410040. 0090-6778/95$4.00 0 1995 IEEE

Authorized licensed use limited to: Jaypee Institute of Technology. Downloaded on September 25, 2009 at 11:02 from IEEE Xplore. Restrictions apply.
 ___

IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 43, NO. 2/3/4, FEBRUARY/MARCH/APRIL 1995 159

11. SYMMETRICAL RVLC ( i i r ) u(i, I , ) = 0 in all other ca..cs.

Considcnng the decrease o f available symmctrical ctxic words
ClexIy, a VLC cc~inposedentirely of sy"tr-ical code under the pref'ix condition:
words is reversible. Furthermore the samc code table can be
used for decoding both in the fclrward and backward directions.
This is an advantage from the viewpoint of memory usage and
simplicity.
A symmetrical RVLC with high transmission efficiency
m(L)=:mo(L)-
[ L S ~ J

( Eu(1-1

where n(i) are components o l the bit length vector of the

1
i, L ) *n(i) - x(L) (4)

original VLC, and x(L) is total number of symmetrical code

a n be designed by staning from a non-reversible VLC with an
optimum bit assignment of code words for the txcurrencc
probabilities, such ;1s a Huffman code, and converting the ccxle
by a top-down scheme into a symmetrical RVLC. This
process is justified when we consider that the bit lcngth vector
of a RVLC, (nrev( 11, nreV(2),...... , nrev(12rev-mar))whcre
nrev(i) is the number of code words having a length i and
Lrev-mar is the minimum length of code words in this code, is
restricted in the top-down direction in the convcrsion process U
follows.
On a binary tree, symmetrical code words are located as
shown in Fig. 2. The number of' symmetrical code words of'
length I,, m&!,), is given as follows.
If L = 2.k where k 2 1 and k is an integer.
mciId)= 2
-- 7Li?
, (1)
otherwise (I, = 2*k+1 where k 2 0 and k is an integer),
mdl,)= 2k.2
- 9 (L+1) / 2
- (2)
.. tngxlJ)= 2['L+')!21 (3)
where 1x1 is the largest integer less than o r equal to x.
On the other hand, the number o f symmetrical code words of
length 1, i n an instantaneously decodable VLC, m(L),
satisfying the prefix condition which means the suffix
condition at the same time in this case is given as follows.
A node at the i-th level from the root (P and S in Fig. 3) is
included in as many as u(i, L ) symmetrical code words of
length L ( L > i), where:
L is (1, 1, ......, 1, 2 ) , these steps generate the RVLC given in
(i) uti, 15)= m&-2) when i 5 7 ,and the part of length L 2 i
- 151. Decreasing x ( L ) in (5) to provide a more efficient RVLC
from P to Q in Fig. 3 (a) is symmetrical;
I,
(ii) u(i, L ) = 1 when
2
-
i I,, and the part of length 2i-L
from R to S in Fig. 3 (b) is symmetrical;
words unavailable due to the violation of thc prefix condition
in ciiqe (ii) above, while:
L- 1
x(L)=
I -1
i. I. 21 + I
,F(i.L) (5)
where .i(i , L ) is the number of code words o f length i whose
( Z - L ) bit suffixcs (Fig. 3 (b)) are symmetrical.
In (4) the first and second terms are decided only by the code
word length, L , and the bit length vector, n(i), while the last
term, x ( L ) , depends on the bit alignment pattern of each code
word. This implies that a RVLC with a diflerent bit length
vector can bc gcncritcd staling from a VLC with a different bit
alignment pattern but an idcntical bit length vector.
Based on the above restriction, a symmetrical RVLC is
obtained as follows:
(i) The bit lcngth vector o f thc target symmetrical RVLC is
initialized by that of the starting VLC:
nrev(i)= n(i) (v i 2 1) . (6)
(ii) The number of code words o f length i, nrev(i),is restricted
by the bit alignment of code words of length less than i.
If nrev(i)5 m ( i ) ,nrpv(i) is not changed.
Otherwise one bit is added to some ccde words, 1.c.:
nrev(i+l) = nrCv(i+l)+ nrev(i)- m(i) , (7)
tirev(i)= m ( i ) . t 8)
(iii) Step (ii) is rcpeatcd until thc bit allocation is finished for
all ctxie words.
For example, Table 1 shows a Huffman ccde (C11)and a
symmetrical RVLC ( C p ) produced from Cl 1. Note that when
the original VLC is a bit-by-bit code whose bit length vector
is a topic for further study.

(a) when i s -
L
2
L
(h)when-
2
c k L

rvhcrc i is thc !cngth of il codc word

and Lis the level of symmetry found
0 symmetrical ccwle word 0 . red n d c 0 : virtual node
0 asymmetrical code word 0 : real leaf : v i r t u laif
Fig. 2 Dstnbution d symmetrical code words Fig. 3 Bit length and symmetrical code words

Authorized licensed use limited to: Jaypee Institute of Technology. Downloaded on September 25, 2009 at 11:02 from IEEE Xplore. Restrictions apply.
 160 IEEE TRANSACTIONS O N COMMUNICATIONS, VOL. 43, NO. 2/3/4, FEBRUARY/MARCH/APRIL 1995
111. ASYbYC\.ErRI(:.\I. KVLC
I n most cases an asymmetrical RVLC offers better
efficiency than a symmetrical RVLC because a more flexible
bit assignment is allowed, although two types of coding tables
are necessary. An asymmetrical RVLC can also be obtained
from a base VLC by adding bits. A s was mentioned
previously, the suffis condition is sufficient for instantaneous
backward decoding' An instantaneously vLc in the
forward direction satisfies the prefix condition. On the other
hand, the suffix of each code word can be extended by adding
bits, which does not prevent instantaneous forward decoding.
This property suggests an effective method of generating an
asvmmetrical RVLC, or add,ne.. Some bits to the bottom of
specific code words to satisfy the suffis condition.
The steps for producing a n asymmetrical RVLC are as
follows (See Fie. 4):
Y I
(i) A reverse binary tree is set up in such a way that the end of
the code words are placed on the root from shorter code
words to longer code words (Fig. 4 (b), (c)).
(ii) When a suffix which is also a prefix in the reverse tree
coincides with a shorter code word, another code word with
the same bit length is assigned in the reverse tree instead.
In Fig. q d ) "D", is assigned in the reverse tree instead of
"C" since the suffis of "C" coincides with " A " .
11
(a) original code
S(b) reverse t r w - 1
L) 101
9pI
0 the suffix condition is not exchanged by an equivalent
(c) reverse tree - 2 (d) reverse tree - 3
101
(e) reverse tree - 4 (f) asymmetrical RVLC
(after sorting)
Fig. 4 Conversion into asymmetncal RVLC
Authorized licensed use limited to: Jaypee Institute of Technology. Downloaded on September 25, 2009 at 11:02 from IEEE Xplore. Restrictions apply.
(111)
~I
When there is n o other code word with the same bit
length whose sufl'is does n o t coincide with onc o f the
shorter code words, the minimum number of bits necdcd t o
satisfy the suffix conditionare added to the of the code
word, I n Fig. (e) one bit ,, 1,, is attached to the end of
,,r,,
L .
(iv) After bit length assignment is completed, new code words
are sorted by bit length and they are reassigned to the
symbols according to their probabilities (Fig.
4(1)).
For esamplc, c,3 i n Table is an RVLC
produced from 'l1'
Incrmse of average code length duc to these Steps, AI,av7 is
expressed as:
s,,
AIAuv= 2 hL(i)*P(c,,v(i)+Ac.~v(i)), (9)
j =I
where S, is the number of code words requiring additional bits,
A L ( i ) is the length of the additional bits, P o ) is the occurrcncc
probability of the j-th code word, cw(i) is the sequential
number of the code \vord rcquiring additional bits and
cw(i)+Acw(i)is the sequential number after sorting of- the code
rvord with additional bits. We find i n (9) that a VLC with
fewer code words whose suffixes coincide with shorter code
nords can be converted to a RVLC with shorter avenge codc
length even if the averagc code length of the original VLC is
equivalent to that of other VLC's.
Making use of this property, we can improve the
transmission efficiency of a RVLC. Equivalent transforms,
[ 3 ] ,that convert a VLC to another with the same average code
length by exchanging a node or a leaf at the same level in a
binary tree can provide a VLC with fewer code words
coinciding with the suffises of longer code words (Fig. 5).
The steps are as follows:
(i) The shortest and most probable code word is selected as the
seed word.
(ii) I f necessary, an equivalent transform is carried out at an
appropriate level in ccxie words longer than the seed word so
that the seed word does not coincide with their suffises.
(iii) When the suffixes have been checked on all the longer
code words, the next probable code word is selected as the
seed word, and step (ii) is repeated.
In these steps, a node or a leaf that has already been checked l'or
transform. The VLC C11 is converted to C14 by this method
as shown in Table 1. A RVLC (CIS)produced from C 14 has a
shorter average code length than C13.
(a) more coinciding suffixes (b) fewer coinciding suffixes
A: suffix of D C: suffix of D
B: suffix of E
Fig. 5 Minimiung coincident surfixes by equivalent transform
 IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 43, NO. 2/3/4, FEBRUARY/MARCH/APRIL 1995 161

Iv. CObIPAKISON A,L1ONCi I<VI,C'S v. CONCI ,LJSIOX

Some properties of the symmetrical and asymmetrical
RVLC's described in Sections 2 and 3, respectiveky, m
compared to other types of reversible codes in this section. A
fixed length code (FXC) and a 8 2 code, [7], are considered for
reference.
A R 2 code (Fig. 6) is composed of ctde words with even-
numbered bit lengths; tdd bits of each ctxie word are continuity
flag bits ('W' for the end of the code word and "1" for
continuation), while even hit$ are information bits. Since the
cont.in1iit.yfbg hit shnws t.he end of [ : d e words7iwtantawous
tlcrtxline !Strictly .;pt:ikiny, tierociiny with 7 hits d ~ l : i y )i s
p q i h l e in the hackward direction. The numtwr of c:twie wonlq
of length I . is given hy:
"I>-\-,
ll,.Jl) /
2 p' , (!e)
Table 2 shnws !he c*hawctcristirs of rach reversihlc emir anri
Schemes for generating reversible variable length codes that
can be decoded instantaneously both in the forward and
backward directions were proposed. These codes are useful for
preventing error propgiticm, and will he applicable t o play-
back of stored media o r fast searching nf an entropy c:c>ded
stream of data.
There is still a broad range of selection available in
combining ones and zeros in both symmetrical and
asymmetrical reversible variable length codes, and further
improvement of average ctde length may be ,mssihle. Stiitiy
of the mwimiim m d e word length is As(> an ripen prohlem
APi%hzi:i
The maximum length of a B3 code is given by
;I

Huffman code for reference. [log?_(N + 3)- 11 2 (A. 1)

I t is easy to design the FLC and the B2 ctxfe because their where N is the total number of code'words (in Table 3) and
code word lengths can be decided independently of the [ x ] is the smallest integer greater than or equal to x.
occurrence probabilities 01 the symbols, although In man): Proof
cases they have a ionger average c d e iengih compared io oiiler When the level of the tree is i (Fig. 6),
P ~ ~ C <>fSRVLC'S ( t h r;cicgc
~ C ~ !ci13th
C of EX'S Y/GU!C! bc - - .
L = 2-1 ( A . '2)
short when the probability distribu!ior! is a n i ! ~ r m ,v&i!e th:1!
of B3 codes would bc short when specific symbols have quitc
~~mux= - ~22 imax - U? . (A. 3;
high Occurrence probabilities). Num (i ) = 3' (A. 4)
In order to compare the average code length of the reversible where L is the length ot code words and Num(i) is the number
codes quantitatively, code words and average code length of 0 1 the code words respeclively at this level i , whiie Lm, ~3 is
each of these reversible codes and the Huffman code for the the masimum length of the code and imcLr_~z is thelevel
English alphabct, [ 3 ] ,arc providcd in Tablc 3 as an cxamplc. corrcsponding to Lmax-~z.Thcn thc maximum numbcr of
In the tilhle G Iis a non-reversible Huffman ccxie, C22 is a code words fully assigned in the tree, S,,,,,, is given as:
FLC with cnde words of 5 bits, and C33 i s a €33,codc. A code '_U -02

C2.l is a symmetrical RVLC generated from C?!. A code C?j S,,, = 2 Niun( i)
I- I
is XIasymmetrical RVLC directly produced from C ~ I while ,
C27 is another asymmetrical RVLC based on a non-reversible
- 2 ' m J2+l - 2 . (A. 5)
VLC (226 converted from by the equivalent transform and Note that for backward decoding only half of the nodes are
assigned to the code words at the bottom level as well as at
consideration of suffix conditions. In this example, the FLC
other levels, while all the nodes are assigned in usual forcvard-
( C y ) and the B2 code C (), have 20.3 % and 13.1 % longer
only cases for better transmission efficiency.
average length over the Huffman code (GI), while the
This leads to an inequality:
symmetrical RVLC ((224) and the asymmetrical RVLC ( C y )
are 13.0 % and 4.9 % longer respectively. In transmission z'nm-82 -2 < N ?imJ3Z+i-3 . (A. 6)
efficiency an asymmetrical RVLC is very advantageous and the Then:
equivalent transform is effective because i t provides 1.6 % imu.r-62 < log 2 ( N + 2 ) 5 imax-R2+ 1 (A. 7
improvement to get C27 from C25. = [log.( N
imm-E + 2) - 11 (A. 8)
where [ X I is the smallest integer greater than or equal to x .
.. = [lc>ggz(N+2) - 11 2 .
Lmax-B?, (A. 9)

level ACKNOWLED(;~IE~UT
The authors wish to express their deep appreciation to Dr.
i=l
Y. Umno, &rector of Laboratones, for his encouragement and
supporl.

REFERE"
i=2
D. Huffman. "A method for the construction of minimum redundancy
codes." in Proc. Inst. Radio Eng., vol. 40,pp. 1098-1101, Sept. 1952.
Y . Talushima M. Wada and H. Murakami, "VariableLength Code with
High Resynchronization Capability," the Journal of the Institute of
Television Engineers of Japan, vol. 43, NO. 8 (1989) pp. 844-846.
J. Maxted, J. P. Robinson, "Error Recovery for Variable Length Codes,"
I E E Trans. IT-3 1, 6, pp. 7'3.1801 (Xov.. 19885).
Fig. 6 Structure of B2 code

Authorized licensed use limited to: Jaypee Institute of Technology. Downloaded on September 25, 2009 at 11:02 from IEEE Xplore. Restrictions apply.
 162 IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 43, NO. 2/3/4, FEBRUARY/MARCH/APRIL 1995

14) 11. blurakami, S. Matsumoto and 11. Yamamoto, "Algorithm lor
(hnstruction of Variable Lcngth Code with Limited Maximum Word
Length." IEX Trans., (:Obl-33, 10, pp. 1157-1 159 (Oct., 19%).
151 K. bl. Rose. "Enhancement of One-Ilimensional Variable-Length
DPCM Images Corrupted by Transmission Errors," IEEE Trans.. COICI-
37.4, pp. 373-379 (flpr., 1989).
16) K. Matsui and 14. ,\chiha, Proceedings ol' the I980 National
(_'onvention, IEICIi 1 ( M pp 5-73.
171 IWI' CLKI':3, Revised version ol Dralt Report ADICMTT: Digital
l'ransmission of Component-Coded Television Signal at 30-34 blbitis
and 45 Mbitis, June 1989.

Table 3- Comparison of 1ZVIl:'s -1 (Characteristics)

independent of pdf.

# I : Derivation is shown in the appendis

& -83 iz-l 2l .v- .v, ~(Lmr
m R7,z-i)
#7: !,AVE-B2 = 1
L =I {
3- I 1 P( Nyrm( I - 1) + j )
1'1 } +
2 { 3, , z -82 ('f N.u,,n( ,r /jz ' - 1) + 0)

where L- B2 = [logz ( N + 3) -11 - 3,

,
Num(i)= 1 n(?- k).
k-l
N: total number of code words, n(i): number of code words ol length i bits, P ( i ] : txcurrence probability oi r-th cvde word,
and [.TI
: the smallest integer greater than or equal to .T.
#3: L-AVE-SRVLC, L-AVE-ARVLC > L-AVE. and they are analyzed by examplcs.

Table 3 Comparison of RVLC's -3 (English alphabet)

Authorized licensed use limited to: Jaypee Institute of Technology. Downloaded on September 25, 2009 at 11:02 from IEEE Xplore. Restrictions apply.