39

Hand out # 7


Duality

The optimal solution of an LP problem represents a snapshot of the conditions that
prevail at the time the model is formulated. In real world, decision environments
rarely remain static and it is essential to equip LP with the capability to determine
changes in the optimal solution that results from making changes in the parameters
of the model. This is what sensitivity analysis does. It provides efficient computation
techniques that allow us to study the dynamic behavior of the optimal solution.

Before discussing sensitivity analysis we need to discuss first the concept of duality.

Definition of the Dual


We recall the following diet problem :

A team from the nutrition aid program is considering two food
supplements S
1
and S
2
for distribution among a population deficient in
nutrients N
1
, N
2
and N
3
. The nutritional content and prices of the food
supplements are given in the table below :


S
1
(per kg) S
2
(per kg) Min. daily
reqt/person
N
1
4 12 96
N
2
8 4 112
N
3
5 5 100
Prices

P6.00 P4.00


The problem is to determine the daily quantities of food supplement for
each person that meet the minimum daily requirement and entail the least
cost.


The complete mathematical model for the Diet Problem is as follows :

minimize z = 6x
1
+ 4x
2


subject to
4x
1
+ 12x
2
96
8x
1
+ 4x
2
112
5x
1
+ 5x
2
100
x
1
0
x
2
0

40

Now we consider a problem from the standpoint of a producer who would like to
produce 3 capsules that would provide each of the nutrients.

Let y
1
= selling price of N
1
capsule (peso/kg)
y
2
= selling price of N
2
capsule (peso/kg)
y
3
= selling price of N
3
capsule (peso/kg)


The producer would naturally want to maximize his income while still providing the
daily minimum requirement of the nutrients and making sure that producing the
capsules would be cheaper than producing the food supplement.


Hence we have the following mathematical model :

maximize z = 96y
1
+ 112y
2
+ 100y
3


subject to
4y
1
+ 8y
2
+ 5y
3
6
12y
1
+ 4y
2
+ 5y
3
4
5y
1
+ 5y
2
+ 100
y
1
, y
2
, y
3
0


This is called the dual of the problem (primal).


The LP model we develop for a situation is referred to as the primal problem. The
dual problem is a closely related mathematical definition that can be derived directly
from the primal problem. In our discussion we assume that the primal problem is
expressed in the standard form which is defined as
n .... , 1 j 0 x
m .... , 1 i b x a
to subject
x c z Minimize or Maximize
j
i j
n
1 j
ij
n
1 j
j j
=
= =
=

=
=




The variables x
j
j = 1,,n include the surplus and slacks if any. For the purpose of
constructing the dual, we arrange the coefficients of the primal schematically as
shown in the following table .




41

Right side of dual constraint

x
1
x
2
x
j
.x
n
Primal Variables

c
1
c
2
c
j
c
n

a
11
a
12

a
1j
a
1n
a
21
a
22

a
2j
a
2n
.
.
.
a
m1
a
m2

a
mj
a
mn

b
1
b
2

.
.
.
b
m
y
1

y
2
.
.
.
y
m


Left side coefficients Dual objective Dual variable
of dual constraints jth dual constraint

The diagram shows that the dual is obtained symmetrically from the
primal according to the following rules :

1) For every primal constraint there is a dual variable.
2) For every primal variable there is a dual constraint.
3) The constraint coefficients of a primal variable form the left side coefficients of
the corresponding dual constraint; and the objective coefficient of the same
variable becomes the right side of the dual constraint.

The following table gives us the sense of optimization, the type of constraints, and
the sign of the dual variables :

Dual
Standard Primal
Objective
Objective Constraints Variables
Maximization Minimization Unrestricted
Minimization Maximization Unrestricted

Reminder : All primal constraints are equations and all variables
are non-negative.

Examples :

1) Recall again the red gadget/ blue gadget problem :

max z = 300x
1
+ 500x
2

subject to
x
1
500


2x
2
1200
3x
1
+ 2x
2
1800
x
1
, x
2
0

42
Standard Form :
max z = 300x
1
+ 500x
2

subject to
x
1
+ s
1
500 =
2x
2
+ s
2
1200 =
3x
1
+ 2x
2
+ s
3
= 1800
x
1
, x
2,
s
1
, s
2
, s
3
0

Dual :
min w = 500y
1
+ 1200y
2
+ 1800y
3

subject to
y
1
+ 0y
2
+ 3y
3
300


2y
2
+ 2y
3
500
y
1
, y
2
, y
3
0
y
1
, y
2
, y
3
unrestricted (redundant)

Exercises

1) Write the dual of the following primal problems

a) max z = -5x
1
+ 2x
2

subject to


-x
1
+ x
2
2
2x
1
+ 3x
2
5
0 x , x
2 1


b) max z = x
1
+ 2x
2

subject to


2x
1
+ x
2
= 5
-x
1
+ 5x
2
5
x
1
unrestricted
0 x
2


Relationship between the optimal primal and dual solutions

The primal and dual problems are so closely related that the optimal solution of one
problem can be secured directly without further computations from the optimal
simplex tableau of the other.

We have the following properties

Property I. At any simplex iteration of the primal or the dual


|
|
|

\
|

=
|
|

\
|
problem other the in j
int constra of side hand right
us min side hand Left
problem one in i iable var of
t coefficien objective


43
Property I can be used to determine the optimal solution of one problem
(directly) for the optimal simplex tableau of the other.

This result could be advantageous computationally if the computations associated
with the solved problem is considerably less than those associated with the other
problem. For example, if a model has 100 variables and 500 constraints it is
advantageous computationally to solve the dual since it has only 100 constraints.

Consider again the red gadget/ blue gadget problem :

max z = 300x
1
+ 500x
2

subject to
x
1
500


2x
2
1200
3x
1
+ 2x
2
1800
x
1
, x
2
0

Standard Form :

max z = 300x
1
+ 500x
2

subject to
x
1
+ s
1
500 =
2x
2
+ s
2
1200 =
3x
1
+ 2x
2
+ s
3
= 1800
x
1
, x
2,
s
1
, s
2
, s
3
0

Dual :

min w = 500y
1
+ 1200y
2
+ 1800y
3

subject to
y
1
+ 0y
2
+ 3y
3
300


2y
2
+ 2y
3
500
y
1
, y
2
, y
3
0

The following tableaux provide the simplex iterations for the primal problem

Tableau # 1

Basic
z x
1
x
2
s
1
s
2
s
3
Solution
Row 0
z 1 -300 -500 0 0 0 0
Row 1
s
1
0 1 0 1 0 0 500
Row 2
s
2
0 0 2 0 1 0 1200
Row 3
s
3
0 3 2 0 0 1 1800





44
Tableau # 2

Basic z x
1
x
2
s
1
s
2
s
3
Solution
z 1 -300 0 0 250 0 300,000
s
1
0 1 0 1 0 0 500
x
2
0 0 1 0 1/2 0 600
s
3
0 3 0 0 -1 1 600

Tableau # 3 (optimal)

Basic Z x
1
x
2
s
1
s
2
s
3
Solution
z 1 0 0 0 150 100 360,000
s
1
0 0 0 1 1/3 -1/3 300
x
2
0 0 1 0 1/2 0 600
x
1
0 1 0 0 -1/3 1/3 200


Applying property I to the starting solution variables s
1
, x
2
, s
3
in the optimal tableau,
we have the following :

Starting primal variables s
1
x
2
s
3
z- equation coefficient
(optimal tableau)
0 0 100
Associated dual
constraint
y
1
0 2y
2
+ 2y
3
500 y
3
0
Equation resulting from
Property I
y
1
= 0
2y2 + 2y3 500 = 0
y
3
= 100


The solution of the given equation yields y
1
= 0, y
2
= 150, y
3
= 100. If you solve the
dual problem independently you will obtain the same solution.

Also because of the symmetry property with respect to the primal and dual problem,
a similar application to the starting variables in the optimal dual tableau will
automatically yield the optimal primal solution x
1
= 200, x
2
= 600, x
3
= 0.

Property II. For any pair of feasible primal and dual solutions.


|
|

\
|

|
|

\
|
problem imization min the
in value objective
problem imization max the
in value objective


At the optimum the relation holds a strict equality.

This property reveals that for all feasible solutions of the primal and the dual, the
objective value in the minimization problem always provides an upper bound on the
objective value of the maximization problem. The successive iterations of the
maximization problem will result in increasing the value of the objective function, and
those in the minimization problem will result in decreasing the value of the objective
function. Eventually in the course of the successive iterations, an equilibrium point
45
will be reached where the maximization and minimization objective values must be
equal.

Exercises

1) Consider the following LP :

Maximize z = x
1
+ 5x
2
+ 3x
3

subject to
x
1
+ 2x
2
+ x
3
= 3
2x
1
x
2
= 4
x
1
, x
2
, x
3
0

a) Write the associated dual problem.
b) Given the information that the optimal basic variables are x
1
and x
2
,
determine the associated dual solution.

2) Consider the following LP :

Maximize z = 2x
1
+ 4x
2
+ 4x
3
3x
4

subject to
x
1
+ x
2
+ x
3
= 4
x
1
+ 4x
2
+ x
4
= 8
x
1
, x
2
, x
3
, x
4
0
Use the dual problem to verify that the basic solution (x
1
, x
2
) is not optimal.

3) In the last problem, the objective equation in the optimal tableau is
z + 2x
1
+ 0x
2
+ 0x
3
+ 3x
4
= 16, determine the associated optimal dual solution.

c) Write the associated dual problem.
d) Given the information that the optimal basic variables are x
1
and x
2
,
determine the associated dual solution.


Economic Interpretations of Duality

Let us look again at the representations of the general primal and dual problems

Primal Dual
n ..... , 2 , 1 j , 0 x
m ....., , 2 , 1 i , b x a
to subject
x c z Maximize
j
i
n
1 j
j ij
n
1 j
j j

= == =
= == =


= == =
= == =

m ..... , 2 , 1 j , 0 y
n ....., , 2 , 1 j , c y a
to subject
y b w Minimize
i
j
m
1 j
i ij
m
1 j
i i

= == =
= == =


= == =
= == =




46
The linear programming primal problem can be viewed as a resource allocation
model in which the objective is to maximize revenue of profit subject to the
availability of limited resources.

Looking at the problem from this standpoint, the associated dual problem offers
interesting economic interpretations of the LP resource allocation model.

From the standpoint of the resource allocation model the primal problem has
n economic activities
m resources
c
j
profit per unit of activity j
b
j
maximum availability of resource j
a
ij
rate at which resource i is consumed by activity j.



Economic Interpretations of Dual Variables

Recall that Property II states that, the values of the objective functions when finite
must satisfy


= =
= =
n
1 j
n
1 i
i i j j
w y b x c z .
Strict inequality holds when both the primal and dual solutions are optimal.

We consider when z = w.

Given that the primal problem represents a resource allocation model, we can
therefore think of z as representing profit. Since b
i
represents the number of units
available of resource i, it follows the z = w can be expressed dimensionally as

=
i
) i resource of unit per ($ ) i resource of units ( $ .

Hence the dual variables y
i
, must represent the unit worth of resource i.
These dual variables are called dual prices or shadow prices or

simplex multipliers.

The inequality z < w associated with any two feasible primal and dual solutions is
interpreted as (Profit) < (Worth of Resources).

This relationship says that so long as the total return from all the activities is less
than the worth of the resources, the corresponding primal and dual solutions cannot
be optimal. Optimality (maximum return) is reached only when the resources have
been fully exploited completely, which can only happen when the input (worth of
resources) equals the output (profit dollars).





47
Example

Recall that the red gadget/blue gadget problem model deals with production of red
gadgets and blue gadgets using steel for red gadgets and wood for blue gadgets,
both require hand processing. The model wishes to determine the number of blue
gadgets and red gadgets to be produced to maximize revenue.

The optimal solution shows that the worth per unit of steel y
1
= 0, this is so since
steel is an abundant resource. y
2
= 150 is the worth unit of wood which means that
for every one unit increase of steel, there is a corresponding increase of 150 in
revenue provided the increase is in the feasibility range. (The feasibility range of the
changes in resources levels will be discussed later.) Similarly, y
3
= 100 means for
every one hour increase in the availability of man hours there is a corresponding
increase of 100 in revenue, provided also that the increase falls within the feasibility
range.

Similar interpretations can be given if the resource levels are below the current levels
with in the ranges of feasibility.


Economic Interpretations of Dual Constraints

Using property one as discussed earlier , at any primal iteration
(objective coefficient of x
j
) =

m
1 i
j i ij
c y a .
Using dimensional analysis to interpret this equation, note that c
j
, the profit per unit
of activity j is in dollars or pesos per unit. Since c
j
represents profit

=
m
1 i
i ij
y a which
appears in the equation with an opposite sign, must stand for cost. Note also that a
ij

is the amount of resource i used by activity j hence the dual variable y
I
must
represent the imputed cost per unit of resource I and we can think of the quantity

=
m
1 i
i ij
y a as the imputed cost of all the resources to produce 1 unit of activity j.
The maximization optimality condition of the simplex method says that an increase in
the level of an unused (nonbasic) activity j can improve only if its objective coefficient
(

m
1 i
j i ij
) c y a is negative.

In terms of the preceding interpretations, this condition states that

|
|

\
|
<
|
|

\
|
j activity of
unit per ofit Pr
resources used
of t cos imputed


The maximization optimality condition says that is is economically advantageous to
increase an activity to a positive level if its unit profit exceeds its unit imputed cost.
48
We let z
j =

=
n
1 i
i ij
y a
. The notation z
j
c
j
is the objective coefficient of x
j
in the simplex
tableau and is often referred to as the reduced cost of activity j.

You can now see that an ununsed economic activity (i.e. a non basic variable) can
become economically viable in one of two ways : by decreasing its per unit use of
resources, or by increasing its per unit revenue (through price increase) A
combination of two may also be permisible. In economic sense, it is logical to think of
the first option as being more viable an option because it implies that any
improvement is being realized through a more efficient use of resources. Raising
prices, on the other hand may not be as viable because the market may be
controlled by other factors including competition.


Dual Simplex Method

We introduce a different simplex algorithm that is motivated by the relationship
between the primal and dual problems. It is the dual simplex method. This method is
essential in carrying out sensitivity analysis.

In the dual simplex algorithm :

1) LP is in canonical form except that the RHS contains some negative
components.
2) For maximization problems, the starting tableau has non positive entries in the
objective row .
For minimization problems, the starting tableau has non negative entries in the
objective row .

Hence, the starting tableau will satisfy the optimality test but would not have an
infeasible starting basic solution.

Comparisons with the original simplex algorithm.

1) Both us the same tableau format and pivot rules.
2) The difference is in the order with which we choose the pivot row/column :

In dual simplex algorithm, we choose the leaving variable first to be
one with the most negative right hand side. Ties are broken arbitrarily

In the primal simplex method we were careful to choose the pivot row
to maintain the non negativity of the right hand side and work toward
the optimality criterion. In the dual simplex algorithm we maintain
optimality criterion then work toward feasibility.


To maintain optimality and, simultaneously move toward feasibility at each new
iteration, the following two conditions are employed :


49
Dual feasibility condition : As mentioned earlier, the leaving variable x
r
, is the basic
having the most negative value, with ties broken
arbitrarily.

Dual optimality condition : The entering variable is determined from among the non
basic variables as the one corresponding to

<

0 ,
c z
min
rj
rj
j j
j
x basic Non


Example

Minimize z = 3x
1
+ 2x
2

Subject to
3x
1
+ x
2
3
4x
1
+ 3x
2
6
x
1
+ x
2
3
x
1
, x
2
0

Standard form

Minimize z = 3x
1
+ 2x
2

Subject to
3x
1
+ x
2
- x
3
= 3 - 3x
1
- x
2
+ x
3
= - 3
4x
1
+ 3x
2
- x
4
= 6 -4x
1
- 3x
2
+ x
4
= - 6
x
1
+ x
2
+ x
5
= 3 x
1
+ x
2
+ x
5
= 3
x
1
, x
2
, x
3
, x
4
, x
5
0

The starting tableau for the problem is given by

x
1
x
2
x
3
x
4
x
5
Solution
z -3 -2 0 0 0 0
x
3
-3 -1 1 0 0 -3
x
4
-4 -3 0 1 0 -6
x
5
1 1 0 0 1 3

The starting basis is optimal but infeasible since the right hand side have negative
values.
The dual feasibility condition indicates that x
4
= -6, is the leaving variable.
Applying the dual optimal condition to determine the entering variable

Variable x
1
x
2
x
3
x
4
x
5
z-row -3 -2 0 0 0
x
4
-row,
4j
-4 -3 0 1 0
Ratio
j 4
j j
c z

4
3

3
2

- - -

50
The ratio shows that x
2
is the leaving variable.

The next tableau was obtained using the same pivot rule in the primal simplex
method.

x
1
x
2
x
3
x
4
x
5
Solution
z -1/3 0 0 -2/3 0 0
x
3
-5/3 0 1 -1/3 0 -1
x
2
4/3 1 0 -1/3 0 2
x
5
-1/3 0 0 1/3 1 1
Ratio 1/5 - - 2 -

The preceding tableau shows that x
3
leaves and x
1
enters, thus yielding the following
tableau.

x
1
x
2
x
3
x
4
x
5
Solution
z 0 0 -1/5 -3/5 0 21/5
x
1
1 0 -3/5 1/5 0 3/5
x
2
0 1 4/5 -3/5 0 6/5
x
5
0 0 -1/5 2/5 1 6/5

The tableau is feasible (and optimal), thus ending the algorithm. The corresponding
solution is x
1
= 3/5, x
2
= 6/5 and z = 21/5.

Exercises

1) Solve the following problems by the dual simplex algorithm :


a) Minimize z = 2x
1
+ 3x
2

subject to
2x
1
+ 2x
2
30
x
1
+ 2x
2
10
x
1
, x
2
0

b) Minimize z = 5x
1
+ 6x
2

subject to
x
1
+ x
2
2
4x
1
+ x
2
4
x
1
, x
2
0









51
Primal- Dual Computations

In carrying out sensitivity analysis, we are interested in one principal result :

Will the changes in the models coefficients change (the optimality or feasibility of)
the current solution. If so how can we determine the new optimum?

One way is to solve the problem anew, but what if we have a big problem with
hundreds of variables and constraints? This may be inefficient.

To acquire the desired results efficiently, we need first to understand how the
simplex computations are affected when changes are made in the original
coefficients of the model. In particular, how are the optimality (objective-row
coefficients) and feasibility (right hand side of the tableau) of the simplex tableau
affected when such changes are made?

A compact way of tracking the simplex computations is by using matrices (You need
to recall what you have learned in Math 114 or Math 110.2) Particularly, you need to
recall (row vector) x (matrix) , (matrix) x (column vector) , (scalar ) x (matrix),
getting the inverse of matrix. You may consult Appendix A of Taha for a more
complete review of matrices.

We can express the general LP problem as

Maximize z =

=
n
1 j
j j
x c
subject to

b x P
n
1 j
j j
=

=

all x
j
0

where Pj and b are column vectors of size m each

In sensitivity analysis, we are interested in studying the effect of making changes in
the coefficients c
j
and right hand side vector b on the optimality and feasibility of the
problem.

We use the following general matrix representation of the simplex tableau

Basic
z
..x
j

z
j
- c
j
Starting X
B
z
j
- c
j

Solution

X
B


B
-1
P
j


B
-1

B
-1
b



52
where x
B
- current basic vector
B - matrix whose columns are the coefficients of the current basic
variables in the constraints of the model
B
-1
- inverse of B
The elements of B and hence B
-1
change from one tableau to the next as a function
of the current x
B
.
P
j
column vector whose entries are coefficients of x
j
in the
constraints of the model.

We have shown in section 4.5 how the dual variables can be computed by solving a
set of simultaneous linear equations (Property I)

A more compact way, yet equivalent way for calculating the dual vectors or dual
prices is Y = (y
1
, y
2
, .y
m
) = c
B
B
-1
where c
B
= m row vector comprised of the original objective
function coefficient c
j
associated with
basis vector x
B
.
Then z
j
c
j
=
j
m
1 i
i ij
c y a

=
= YP
j
- c
j
. Notice that z
j
c
j
is the difference between the
left and right hand sides of the dual constraints.


All the elements of the current tableau are computed from the current inverse B
-1
and
the original data of the problem.


To illustrate what we have discussed. Let us take a look at the optimal tableau of the
red gadget-blue gadget problem. z
j
- c
j



Basic z x
1
x
2
x
3
x
4
x
5
Solution
z 1 0 0 0 150 100 360,000
x
3
0 0 0 1 1/3 -1/3 300
x
2
0 0 1 0 1/2 0 600
x
1
0 1 0 0 -1/3 1/3 200



X
B
B
-1
P
1
B
-1
P
2
B
-1
B
-1
b

Even without going through the simplex iterations we can obtain the above table
using the general matrix representation of the simplex tableau. All we need to know
is the basis vectors of the current simplex iteration.

53
Knowing that X
B
=
(
(
(

1
2
3
x
x
x
so B =
(
(
(

3 2 0
0 2 0
1 0 1
hence B
-1
=
(
(
(
(
(
(

3
1
3
1
0
0
2
1
0
3
1
3
1
1
.
B
-1
P
1
=
(
(
(
(
(
(

3
1
3
1
0
0
2
1
0
3
1
3
1
1

(
(
(

=
(
(
(

1
0
0
3
0
1


B
-1
P
2
=
(
(
(
(
(
(

3
1
3
1
0
0
2
1
0
3
1
3
1
1

(
(
(

=
(
(
(

0
1
0
2
2
0



X
B
= B
-1
b =
(
(
(
(
(
(

3
1
3
1
0
0
2
1
0
3
1
3
1
1

(
(
(

=
(
(
(

200
600
300
1800
1200
500


Thus the associated dual vector is computed as

( ) [ ] 300 500 0 B C Y y y y
1
B 3 2 1
= = =

(
(
(
(
(
(

3
1
3
1
0
0
2
1
0
3
1
3
1
1
= [ ] 100 150 0

The associated z- equation coefficient is

z
1
c
1
= YP
1
c
1
= [ ] 100 150 0 0 300
3
0
1
=
(
(
(

z
2
c
2
= YP
2
c
2
= [ ] 100 150 0 0 500
2
2
0
=
(
(
(

54

z
3
c
3
= YP
3
c
3
= [ ] 100 150 0 0 0
0
0
1
=
(
(
(

z
4
c
4
= YP
4
c
4
= [ ] 100 150 0 150 0
0
1
0
=
(
(
(

Exercises

1) Using the general matrix representation of the simplex tableau, obtain the
tableau #2 of the red gadget/blue gadget problem.

2) Consider the following LP model :
Maximize z = 3x
1
+ 2x
2
+ 5x
3

subject to
x
1
+ 2x
2
+ x
3
+ x
4
= 30
3x
1
+ + 2x
3
+ x
5
= 60
x
1
+ 4x
2
+ x
6
= 20
x
1
, x
2
, x
3
, x
4
, x
5
, x
6
0

Check the optimality and feasibility of the following basic solutions :

a)
(
(
(

=
6
3
4
B
x
x
x
X , B
-1
=
(
(
(

1 0 0
0 2 / 1 0
0 2 / 1 1

b)
(
(
(

=
1
3
2
B
x
x
x
X , B
-1
=
(
(
(

2 / 1 2 / 1 1
4 / 3 4 / 1 2 / 3
8 / 1 8 / 1 4 / 1

c)
(
(
(

=
6
3
2
B
x
x
x
X , B
-1
=
(
(
(

1 1 2
0 2 / 1 0
0 4 / 1 2 / 1



The entire simplex tableau at any iteration can be generated from the associated B
-1

and the original data of the model.

Thus, in terms of carrying out sensitivity analysis on an optimal LP solution whose
inverse basis is B
-1
, we can study the effect of any changes in the objective
coefficients and the right hand side of the constraints simply by re-computing all
z
j
- c
j
(z-equation) and B
-1
b (right hand side of the tableau).