CONVOLUTIONAL CODES IN VSS

............................

.....

By: Dr. Kurt R. Matis Director of Systems Research

SIMULATION OF CONVOLUTIONAL CODES IN VSS

This document describes the modeling and simulation of short constraintlength convolutional codes used in conjunction with Viterbi decoding in the Visual System Simulator (VSS). After a brief review of the history and application of convolutional codes, a detailed description of VSS models for encoding/decoding of these codes is presented. Step-by-step examples illustrate how to construct simulations and analyze results.

Convolutional Code Basics

This note begins with some background information on the use of convolutional codes. The development of convolutional codes is discussed along with a history of important applications. This information is meant to provide a perspective on the selection of the convolutional code models that are provided in VSS. Transmission efficiency and reliability can be improved by encoding information digits in a way that creates an interdependence between symbols which are transmitted over a channel. At the receiving end, the interdependence can be exploited to detect or even correct transmission errors, provided erroneous symbols are not received too frequently. Such coding is called error-control coding and is shown in the configuration of Figure 1.
Source Symbols Encoded Symbols Channel Encoder Received Symbols Transmitter Decoded Symbols Channel Decoder

{ ai }

{ b k}

s(t)

Channel

r(t)

Receiver

} {b k

{ i }

{ r k}

Figure 1. System Employing Error-Control Coding

Visual System Simulator

CONVOLUTIONAL CODES IN VSS Simulation of Convolutional Codes in VSS

Encoders for error control are usually called channel encoders to differentiate them from various encoders used for other purposes within digital communication systems. The process of error-control coding is also referred to variously as Error Detection And Correction (EDAC) and Cyclic Redundancy Check (CRC), although CRC actually refers to a specialized type of code. Coding used for error detection is often called, as one might expect, error detection coding. It is often used within Automatic Request for Query (ARQ) systems to detect a block or packet error and request the transmitter to retransmit a message. Again, intuitively, when error-control coding is used for the purpose of actually correcting certain received symbols at the decoder it is usually called error correcting coding or error correction coding. In some cases it is called Forward Error Correction (FEC) coding to distinguish it from the process used in conjunction with the aforementioned ARQ error-correction scheme. Finally, error control codes are sometimes used for both correction of a limited number of errors and detection of a certain number of errors when there are too many errors to correct. As shown in Figure 1, the encoder inserts redundant symbols into the original symbol sequence, ! a i " , to create the output sequence, ! b k " , at a rate 1 # R times the rate of the original transmitted sequence ! a i " . The quantity R is referred to as the code rate. Since the symbols, ! b k " , are no longer independent, the decoder deals with vectors of received symbols, r , which may potentially be very long, perhaps encompassing the entire transmission. From a practical point of view, error control codes impart a dependence to the symbols within the transmitted data stream that is localized from within a few symbols to a few hundred symbols. This means that the decoder need not examine the entire transmission before making decisions about the transmitted symbols, but can decode information symbols by examining a subsequence of symbols, b, which is "near" the original information symbol sequence a. We will quantify these notions when discussing the specific classes of codes. Whatever the length of the sequence, a, the decoder must produce an estimate, , based on the received sequence, r. Equivalently, since there is a one-to-one correspondence between the information sequence a and the code word, b, the decoder can produce an estimate of the code word, b, and then map this sequence back into the original information sequence, a. = b. Clearly, = a if and only if b

Visual System Simulator

CONVOLUTIONAL CODES IN VSS Simulation of Convolutional Codes in VSS

Convolutional Codes Compared with Block Codes

The two classes of error control codes are block codes and convolutional codes. An (N, K) block encoder maps blocks of K source bits into blocks of N encoded bits. There is complete independence between blocks. N is called the block length of the code and R = K / N is the code rate.
In contrast to block codes, which impose a block structure on the input information sequence, convolutional codes impose a sliding dependency over a span of input symbols. The encoder can be implemented as a shift register into which information bits are shifted 'b' at a time. After each shift, n modulo-2 sums are computed from the contents of selected shift register stages. These n modulo-2 sums then constitute the encoder output subsequence associated with a particular subsequence of b-input information bits. The normalized code rate in this case is R = b / n. This indicates the number of bits, on average, that are carried by each symbol output from the encoder.1 The quantity k is called the constraint length of the code and is equal to the length of the encoding shift register. A shift register of length k has k-1 delay or storage elements, each storing one bit. A typical k = 3, R = 1/2 encoder is shown in Figure 2.

1.

Unfortunately, the classical literature describing convolutional codes has used some of the same nomenclature historically used to represent quite distinct attributes of block codes. For example, the capital letter N represents the length of a block code word, whereas the small letter n represents the number of adders in a convolutional encoder.

Application Notes

CONVOLUTIONAL CODES IN VSS Simulation of Convolutional Codes in VSS

Figure 2. Convolutional Encoder for k = 7, R = 1/2 Convolutional Code

Note that the terminology concerning convolutional codes is not uniform in the field. We generally try to follow the usage in Viterbi and Omura [1], which is consistent with Figure 2. In this note, we restrict attention to linear, feedforward convolutional codes which encode a binary input stream into a binary output stream. This is the encoding model depicted in Figure 2. Other VSS models address the functionality of feedback codes, nonlinear codes and codes designed for use with more general alphabets. The number of storage elements in this case is two - always one less than the constraint length, k. Depending on the connections to the mod-2 adders (described by the so-called code connection vectors), the code will have different properties as discussed below. The convolutional encoder can be thought of as a finite-state machine with a number of states corresponding to all possibilities of bit values contained within the storage elements of the shift register. With any given state and any specified input (a single 0 or 1 in this case), both the next state and the output of the encoder can be predicted. This model of the encoder as a finite-state machine is embodied in the FSM (finite-state machine) diagram at the right of the figure. The FSM provides a succinct description of all possible evolutions of the sequence of states as successive encodings are performed. Tabulations of good convolutional code constructions are available for various values of k and R [2], [3], [4]. For any given value of k, n and b, the "code construction" procedure consists of finding connections of the modulo-2 adders shown in Figure 2 to the appropriate shift-register stages

Visual System Simulator

CONVOLUTIONAL CODES IN VSS Simulation of Convolutional Codes in VSS

to generate a sequence with the maximum error-correcting capability. VSS contains extensive tabulations of optimum convolutional codes. The evolution in time of the possible sequence of encoder states can be described in terms of a trellis diagram.Figure 3 illustrates a trellis diagram for the code of Figure 2. Generated bits are determined from a combination of the current input bit sequence of b bits plus the "state" of the encoder, which consists of the most recent k - b bits in the shift register. b For the k = 3, R = 1/2 code, there are four possible states with 2 = 2 branches into and out of each state. Each branch has n output symbols associated with it, in this case, 2.

00 11 11

00 11

00

a 11

00

a 11

00

a=

00

10 01 01

10

11 00

b 10 01 c

11 00

b 10 01 c

b= 11 00 c=

01

10

01 10

01 10

01 10

d=

11

Figure 3. Trellis-Code Representation for Encoder of Figure 2

Convolutional codes were first introduced by Elias in 1955 [5]. In 1961, Wozencraft proposed a probabilistic method for decoding convolutional codes called sequential decoding [6]. In 1963, Massey proposed a less efficient but simple decoding algorithm called "threshold decoding" [7]. Finally, in 1967, Viterbi [8] proposed a maximum-likelihood decoding algorithm that still bears his name. The Viterbi decoding algorithm is relatively easy to implement for codes with short constraint lengths and allows effective use of soft-decision

Application Notes

CONVOLUTIONAL CODES IN VSS Simulation of Convolutional Codes in VSS

information in the form of real-valued or quantized receiver outputs. Although other forms of decoding are still used widely in certain applications, Viterbi decoding has become by far the most widely-used decoding method for convolutional codes. The Viterbi algorithm is an iterative algorithm for correlating received sequences with all possible transmitted sequences in the trellis to arrive at the one with the best fit. The Viterbi algorithm, commonly used with short constraint-length convolutional codes, performs an efficient search of the trellis associated with the code. Specifically, the Viterbi algorithm is an iterative algorithm which compares all possible paths through the trellis with the received realvalued sequence. An important feature of the Viterbi algorithm is that it can efficiently perform real-valued correlations against the incoming sequence in the process of measuring the fit between the received sequence and the possible code sequences. For binary convolutional codes, a linear correlation is optimum for measuring the fit between the received sequence and all possible transmitted sequences. This is sometimes called linear combining. Since the correlation is employed as a measure of distance, it is sometimes called a metric. There are many other types of metrics that are appropriate for different signalling schemes and various channels. The only metric considered here is the correlation metric. Referring to Figure 3, the Viterbi algorithm operates by visiting each state in the trellis (states are arranged vertically) and calculating branch metrics corresponding to possible n-tuples emitted as bits enter the encoder. For the rate 1/2 code presented here, two branches emanate forward from each state. These branch metrics are used in the iterative calculation of distance as the decoder moves forward through each decoding step, or each node level in the trellis.

Code Performance
In this section, code performance is analyzed in terms of decoded bit error rate (BER) as a function of received signal-to-noise ratio. To provide a basis for comparison with uncoded system performance, BER is normally tabulated as a function of E b # N 0 . Here, E b represents the average energy transmitted per information bit and N 0 represents the single-sided power spectral density of an assumed Additive White Gaussian Noise (AWGN) component. Energy per bit is, of course, related to energy per encoded symbol by the rate of the code: E s = E b $ b # n .

Visual System Simulator

CONVOLUTIONAL CODES IN VSS Simulation of Convolutional Codes in VSS

The decoder measures the distance between all of the possible transmitted sequences in the trellis and the noisy received sequence. An error is generated if the received sequence is closer to some other sequence in the trellis than the actual transmitted sequence. Let d represent the number of bit positions in which any two code sequences in the trellis differ. This is known as the Hamming distance between the two sequences. We denote the probability of confusing two code sequences differing in d positions as P e % d & . Error probability may be calculated by considering the distance between the actual transmitted sequence and all possible contenders in the ensemble of code sequences described by the trellis. Intuitively, good codes have large distances between all possible pairs of code sequences. When calculating error probability, it is adequate to assume that the all 0s sequence has been transmitted. For the special class of linear codes considered here, the set of distances between the all 0s code sequence and other sequences in the ensemble is the same as the set of distances from any arbitrary code sequence and other code sequences. This allows the bit error probability for a given convolutional code to be bounded by:
(

1 P b ' -b

) wd $ Pe % d &
d=0

(1)

Here, P e % d & is as described above and ! w d " represents the so-called weight spectrum of the code. Each term w d represents the average number of bit errors associated with sequences of weight d. w d is zero for d less than or equal to d free , known as the free distance of the code. The weight spectrum is tabulated for many convolutional codes. It can be calculated using a procedure described by McLiece in [9], given the structure of the encoder in Figure 2. In practice, only a few terms in (1) are needed, since P e % d & tends to decrease quite rapidly with d for practical channels. The weight spectrum for the industry-standard rate 1/2 constraint length 7 convolutional code is given in Table 1 on page 8. Note that the free distance of this code is 10 and that 4 terms in the weight spectrum have been tabulated. This code is the default code for the CNVENC and VITDEC models. Specific tap connections for this code are found in the on-line documentation for these models.

Application Notes

CONVOLUTIONAL CODES IN VSS Simulation of Convolutional Codes in VSS

Weight, d
10 12 14 16

No. of bit errors in adversaries of weight d, w d

36 211 1404 11633

Table 1. Weight Spectrum of Rate r=1/2, Constraint Length k=7 Convolutional Code

For linear combining of antipodal signals in AWGN, the sequence error probability takes on a particularly simple form:
P e % d & = Q % 2 dE s # N 0 & ,

(2)

where Q % x & represents the normalized Gaussian tail function.

Bit Error Results

Figure 4 shows predicted BER performance for the rate 1/2 constraint length 7 convolutional code with the weight spectrum shown in Table 1.

Visual System Simulator

CONVOLUTIONAL CODES IN VSS Simulation of Convolutional Codes in VSS

Figure 4. Upper Union Bound on Bit Error Probability for Rate r=1/2, Constraint Length k=7 Convolutional Code, Plotted in Conjunction with Theoretical Uncoded Bit Error Probability

This curve is plotted with the theoretical curve of uncoded BER for antipodal signalling. This would represent the error probability of 2-level

Application Notes

CONVOLUTIONAL CODES IN VSS Simulation of Convolutional Codes in VSS

PAM baseband modulation or BPSK carrier modulation systems with coherent detection. The same error probability would be attained for coherent QPSK on a per-channel basis. Note that the uncoded error probability is asymptotic to 1/2 at low signal-to-noise ratios (SNRs). The bound on decoded BER performance, on the other hand, becomes greater than 1.0 for low SNRs. This is because the various paths in the trellis characterized by the weight spectrum are not distinct. Thus, the union of the various error events in the trellis is actually less than the sum described in the union bound of (1). Union bounds tend to be loose at low SNRs, but become tight as SNR increases. For the classes of codes considered here, the error probability tends to be dominated by the probability of the lowest order term. This is the term for which for d = d free . The reduction in required SNR to achieve given BER objectives is called 6 coding gain. At interesting error rates (say at 10 ) the rate 1/2 constraint length 7 code provides a coding gain of 5 dB over uncoded transmission. For many years, this was considered the state-of-the-art improvement that could be practically offered by error-control coding on Gaussian channels. This improvement was achieved only at the cost of a two-fold increase in transmission bandwidth. This was not objectionable for many non-bandlimited channels, such as the deep space channel. In the 1980s and 1990s additional improvements were offered by coding technologies such as trellis-coded modulation and turbo codes. Other VSS application notes address these techniques.

Punctured Codes
Puncturing is a method of transforming a convolutional code with a given rate into a code with a higher rate. For example, a rate 1/2 code may expand the transmitted bandwidth too much, while providing more error protection than is necessary for a given application. In these cases, not all of the encoded bits need to be transmitted. If the omitted bits are chosen carefully, the decoder can still provide good performance without needing the full redundancy of the rate 1/2 code. The output bits to delete are specified by a puncturing matrix or array, which indicates a periodic pattern of bits to be discarded from the mother encoder output. This capability is provided in the CNVENC model and is diagrammed in Figure 5.

10

Visual System Simulator

CONVOLUTIONAL CODES IN VSS Simulation of Convolutional Codes in VSS

Binary Signal

Convolutional Encoding

Puncturing (optional)

Encoded Signal

Figure 5. Structure of Overall Encoder Showing Mother Encoder and Puncturer

Note that puncturing is generally a sub-optimum means of creating highrate codes from lower rate codes. It can be shown that for most rates and constraint lengths, constructive codes exist that have greater d free than the best punctured codes. This is true of most of the tabulated high-rate constructive codes within this model. Puncturing is often employed to create higher rate codes from lower rate codes to allow use of existing coding hardware. Puncturing is sometimes implemented as a simple external applique to existing off-the-shelf rate 1/2 coders/decoders, thus avoiding the need for a custom chip or program. This is exemplified by the DVB punctured coding scheme described in the CNV_ENC on-line Help documentation [10]. Punctured codes also allow scalable rates in RateCompatible coding applications. The amount of redundancy in the transmitted data stream is controllable by adjusting the number of bits punctured. With punctured codes, comparable bits between data segments with different rates will agree. This is generally not the case with constructive codes. At the decoder, punctured bit positions are stuffed with floating point values of 0.0. This allows punctured bit positions to be treated essentially as erasures by the decoder, giving no weight to these positions in the process of calculating branch metrics. This operation is illustrated in Figure 6. The decoder can accept both real-valued inputs and binary digital inputs. Normally, a standard decoder expects to receive an antipodal real-valued decision statistic from the received output. This assumes that the transmitter (for example, BPSK or QPSK) maps the 0 and 1 bits emitted from the convolutional encoder into values of plus A (+A) and minus A (-A). Various scalings and normalization may be employed in any particular receiver. Within real hardware, in fact, outputs are usually quantized to 8 bits with a maximum range of +-1.0. The output levels are symmetric about zero, to provide the decoder with proper input data for the correlation operation implicit in the Viterbi algorithm.

Application Notes

11

CONVOLUTIONAL CODES IN VSS Simulation of Convolutional Codes in VSS

Figure 6. The Internal Structure of the VIT_DEC Model

Note in Figure 6 that the VIT_DEC model also supports input of harddecisions. These are the decisions that the receiver would make if no encoding/decoding were present. Sometimes it is desirable to first make hard decisions at the receiver before decoding or it may be desirable to simulate the decoder operating in conjunction with a receiver that has no real-valued output. In this case, the hard-decisions (represented as 0 and 1 in VSS) must be transformed to -1.0 and 1.0 to map a binary metric into a linear metric. Typically, a loss of about 2 dB is incurred at interesting BERs when hard decisions versus soft decisions are employed. This operation is performed automatically when the decoder input is connected to a binary digital input. Digital data ports are recognizable by their unique shape and green color, as indicated in the diagram. Finally, it is sometimes desirable to operate a binary Viterbi decoder with non-binary modulation schemes. In this case, a typical scheme is to dissemble M-ary (4-ary, 8-ary, etc.) outputs into bits and then pass them into the VITDEC model. This operation must be performed using an external SYM2B model.

Simulation Examples
This section presents two simple examples that illustrate the procedure for generating BER and SER curves and plotting these curves in conjunction with theoretical curves.

12

Visual System Simulator

CONVOLUTIONAL CODES IN VSS Simulation of Convolutional Codes in VSS

Rate 1/2 Example

Figure 7 illustrates a simulation of a coded QPSK link. In addition to the specific model names, by choosing Schematic > Add Text the block diagram is annotated with generic names that correspond to the function of each individual block. Additionally, by choosing Options > Project Options, clicking the Schematic/Diagrams tab, and selecting the Hide parameters check box, on-screen display of the model parameters is disabled for clarity on the diagram. Finally, by choosing Diagram > Show Grid Snap the grid was toggled off.

Figure 7. Illustration of Coded QPSK Link - Block Diagram

In this example, a CNVENC model is fed by a RND_D model. The RND_D model is a general-purpose source of pseudo-random digits. The binary encoded sequence is fed into a QPSK_TX transmitter model. The input bits are interleaved onto the I/Q rails. Each bit thus maps into an antipodal value of +-A onto either rail in an alternating fashion. The transmitted signal is corrupted by the AWGNCH model, which adds white Gaussian noise of the proper intensity. The receiver employs an integrateand-dump matched filter which produces a real decision statistic for each of the I/Q rails. This decision statistic is fed out of the complex output port, which is the lower port on the receiver output. The QPSK receiver also has a hard decision output at the top port. The QPSK signal is detected inside the receiver and deinterleaved through this output port. The output data is binary digital data. In this example, the decoder is fed with the real-valued outputs from the I/Q channels by converting the complex data to I/Q data

Application Notes

13

CONVOLUTIONAL CODES IN VSS Simulation of Convolutional Codes in VSS

(or Real/Imaginary components of the complex-envelope) and then multiplexing them into the decoder input port with a C2RI and a MUX2 model preceding the decoder. A bit error counter model tallies errors and calculates average bit error probability for the various stepped SNR values. Figure 8 shows the results of a BER simulation for the rate 1/2 constraint length 7 convolutional code. As a basis for comparison, the BER results for hard-decision operation as overlaid on the plot. This curve was generated in a simulation in which the decoder input was connected to the DIGITAL output of the QPSK receiver. This is a binary output, with the I/Q bits deinterleaved from the corresponding demodulators/detectors. This is the inverse of the operation performed in the transmitter. Note that the hard decision results exhibit a degradation of about 2 dB at interesting SNRs.

14

Visual System Simulator

CONVOLUTIONAL CODES IN VSS Simulation of Convolutional Codes in VSS

Figure 8. Error Rates for QPSK System with Rate r = 1/2, constraint length k=7 Convolutional Code Used in Conjunction with Soft-Decision Viterbi Decoding

The uncoded bit error probability is also plotted on the graph to demonstrate correspondence with theory. Note that whereas the uncoded error results follow the theoretical result closely at all SNR values simulated, the coded error rate results diverge from the upper bound at low SNRs.

Application Notes

15

CONVOLUTIONAL CODES IN VSS Simulation of Convolutional Codes in VSS

Figure 9 displays the BER results from two rate 3/4 codes. Each has a constraint length of 7. This experiment was performed to demonstrate the superiority of constructive codes versus punctured codes for certain rates and constraint lengths. The punctured code is the code specified within the DVB standard[10]. The constructive code was found in [4] by exhaustive search. As a basis of comparison, the free distance of the punctured code is 5, whereas the free distance of the constructive code is 6.

Figure 9. Simulation Results for Decoding of Rate r=3/4, Constraint Length k=7 Convolutional Codes with QPSK Signalling.

16

Visual System Simulator

CONVOLUTIONAL CODES IN VSS Simulation of Convolutional Codes in VSS

Conclusion
We have illustrated the use of standard and punctured convolutional codes in conjunction with Viterbi decoding. Results demonstrate correspondence with theoretical bounds. Efficiency of punctured codes and constructive codes was compared. We hope that this information is helpful to those employing VSS simulations for end-to-end link performance analysis.

References
[1] [2] [3] [4] [5] [6] [7] [8] [9] A.J.Viterbi and J.K. Omura; Principles of Digital Communication and Coding, McGraw-Hill, 1979. J.P. Oldenwalder, "Optimum Decoding of Convolutional Codes", Ph.D. dissertation, Dept. Syst. Sci., Univ. California, Los Angeles, CA, 1970. K.J. Larsen, "Short Convolutional Codes with Maximal Free Distance for Rates 1/2, 1/3 and 1/4", IEEE Trans. Info. Theory, vol. IT-19, pp.371-372, May 1973. D.G. Daut, J.W. Modestino and L.D. Wismer, "New Short Constraint-Length Convolutional Code Construction for Selected Rational Rates", IEEE Trans. Info. Theory, vol. IT-28, pp. 774-800, September 1982. P. Elias, "Coding for Noisy Channels", IRE Conv. Rec. Part 4, pp. 37-47, 1955. J.M. Wozencraft and B. Reiffen, "Sequential Decoding", MIT Press, Cambridge, MA, 1961. J.L. Massey, "Threshold Decoding", MIT Press, Cambridge, MA, 1963. A.J. Viterbi, "Error Bounds for Convolutional Codes and an Asymptotically Optimum Decoding Algorithm", IEEE Trans. Info. Theory, IT-13, pp.260-269, April 1967. R. McLiece, S. Dolinar, F. Pollara, H. VanTilborg; "Some Easily Analyzable Codes, presented at Proceedings of the Third Workshop on ECC, IBM Almaden Research Center; September, 1989.

[10] EN 300 744 V1.1.2 Digital Video Broadcasting (DVB), ETSI, pg. 13

Application Notes

17