Optimization - Vikas Srivastava

Linear programming (LP) is all about inequalities. It's an extremely useful mathematical technique
for business decisions. As mentioned, there are three ways to approach the underlying mathematics:

1. Intuitively through the geometry
2. Computationally through the Simplex method
3. Algebraically through duality

One key to understand the mathematics is to see the geometric meaning of the inequalities. So,
exactly what an inequality does? The thing to remember is that an inequality divides n-dimensional
space into a half-space in which the inequality is satisfied, and a half-space in which it is not.

Another constraint fundamental to LP is that the variables x and y are required to be non-negative.
The important step is to impose all the constraints at once. Generally they combine to give an area
which is common in all of them. This common area is known as feasible set.
A feasible set is composed of the solutions to a family of linear inequalities like Ax>=b.
We also require that every component of x is non-negative.
The more constraints we impose, the smaller the feasible set.
The feasible set can be bounded or even empty.
What LP actually does?
It looks for the feasible point that maximizes or minimizes a certain cost function. The purpose of
the LP is to find the point that lies in the feasible set and minimizes the cost.

Remember The optimal vector occurs at a corner of a feasible set. This is guaranteed by the
geometry.

Every LP problem falls into the one of three possible categories -

1. The feasible set is empty.
2. The objective function is unbounded on the feasible set.
3. The value of the objective function reaches it minimum (or maximum) on the feasible set
Best case.

Primal and Dual Problems -

Every linear program has a dual.
If the original problem is a minimization, its dual is a maximization.
The minimum in the given primal problem equals the maximum in its dual.
Duality theorem maximum equals minimum.

Objective function A mathematical expression in LP that maximizes or minimizes some
quantity.
Constraints Restrictions that limit the degree to which a manager can pursue an objective.
Decision variables Choices available to a decision maker.
Sensitivity analysis An analysis that projects how much a solution may change if there
are changes in the variables or input data.
Shadow price for a constraint Improvement in the objective function that results from a
one-unit increase in the RHS of the constraint.
Optimization - Vikas Srivastava


Simplex method
1. The essence of the method is that it goes from corner to corner along the edges of the
feasible set.
2. In general there are (n+m)!/n!m! Possible intersections. It is actually the count of number of
ways to choose n plane equations out of n+m.
3. The corner of the feasible set are the basic feasible solutions of Ax=b.
4. Remember A solution is basic when n of its n+m components are zero, and it is feasible
when it satisfies x>=0
5. Phase I of the simplex method finds one BFS and Phase II moves step by step to obtain
optimal x*.
6. A corner is said to be degenerate if more than the usual n components of x are zero.

Applications of LP

Out of numerous applications of LP, two of them feature vividly in our curriculum.

1. Transportation Problem

Purpose : Given supplies at n points and demands at n markets, choose shipments x
ij
from suppliers
to market that minimizes the total cost C
ij
x
ij.
In simpler words, it is the means of finding least cost
means of shipping supplies from several origins to several destinations.

Min (Total Transportation Cost)


Destination Roorkee Bareilly Dehradun Haridwar Availability Origin
C
11
C
12
C
ij
a
1
Lucknow
C
23
a
2
Kanpur
C
31
C
34
a
3
or a
i
New Delhi
Demand b1 b
4
or b
j


Types:

1. Balanced Transportation Problem Total demand/requirement = Available Supplies.

Solved using Simplex method. Algebraic and geometrical method cumbersome.
The approach is two step. First obtain IBFS using Heuristic techniques and then solve for
Optimality using other techniques.
Step I Obtain IBFS. Heuristic techniques incorporate Phase I of the Simplex Method to
save on computation time. Recall initializing a simplex tableau.
North-West Corner Rule (Fastest, bad because doesn't consider objective function)
Minimum Cost Rule (Better than NWCR, utilizes shipping cost)
Penalty-Cost method or Vogel's Approximation Method (VAM) [Best]
Step II Solve for Optimality.
Optimization - Vikas Srivastava
Stepping-Stone Method (Lengthy)
MODI (Modified Distribution Method) (Application of Duality, preferred method)
VAM -
Provides IBFS which is often the optimal solution.
Finds a good IBFS by taking into account the costs associated with each alternative
route.
Penalty The additional cost incurred if we ship over the second-best route instead of
least-cost route.

Performance of IBFS from VAM>from Min Cost>North-West Corner Rule
Remember we can obtain solution only if the solution is
feasible should satisfy the inequalities/ lie in the shaded region
basic at the corners of the feasible region
non-degenerate
total no. of allocations should exactly be equal to m+n-1 (m = suppliers, n=number
of destinations)
all allocations must be independent (checked through looping)

Check for Optimality MODI/u-v Method

Algorithm if IBFS is Non-degenerate (No. of allocated cells=m+n-1)

Write u
i
along rows and v
j
along columns of IBFS obtained from Step 1.
Assign u
1
=0. There are only two varieties of cells in our matrix.
Occupied/Allocated cells Use C
ij
=u
i
+v
j
. To obtain values of u & v.
Unoccupied cells Use C
ij
-(u
i
+v
j
) to obtain values for the cell.
If all the resulting values in unoccupied cells turn out to be +ve, Optimal Solution reached.
If there is a negative value in some cell, introduce at the location.
Form a loop, get values. [Refer notebook]
Reiterate till you get all non-negative values.
If you get a 0 at some location and all others as positive, it implies that there is an alternative
optimum. Solve for that.

If IBFS is Degenerate (No. of allocated cells < m+n-1)

Assignment Problem (Transportation problem with square matrix and 1-1 mapping)

n
2
variables, 2n constraints and n allocations optimum.
Transportation algorithm is not used as it is a degenerate transportation problem. 2n-1 will
be optimum in a transportation problem. Here, optimum is 'n'.
If you have a feasible solution with z=0, it is also an optimal solution (as C
ij
>=0).
Remember Every job goes to exactly one person, every person gets exactly one job.
Job 'i' goes to person 'j'. X
ij
is the decision variable.
STEP 1 IBFS - Due to the above rule, you may end up having lesser assignments than
possible. For e.g. 3 assignments in 4x4 matrix. Use Maximal Rule to get the maximum
assignments possible.
Optimization - Vikas Srivastava
Maximal Rule -
Make the assignment if there is exactly a single 0 in the row.
If number of 0 is greater than 1 in a row, don't assign, proceed. Return later to see if
you were able to solve this deadlock.
If you are not able to get number of assignments = number of persons. Follow
the procedure stated below to create a new Assignment Matrix-
Tick all unassigned rows.
If a ticked row has a 0, then tick the corresponding column.
If a ticked column has an assignment, then tick the corresponding row.
Repeat Step 2 & 3 till no more ticking is possible.
Draw a line through unticked rows & ticked columns.
The number of lines represent the number of assignments possible.
Find the smallest(say ) number which has no line passing through it.
Update the C
ij
matrix using the following changes
If the number has no lines passing through it : C
ij
= C
ij
-
If the number has one line passing through it : C
ij
= C
ij
- NO CHANGE
If the number has two lines passing through it : C
ij
= C
ij
+
Repeat till you get all the assignments.


STEP 2 Check for Optimality
Use Hungarian Algorithm. Dual of Transportation problem.
It is a dual algorithm, unlike Transportation algorithm which is primal.
Original Cost Matrix C
ij
Job Opportunity Matrix C'
ij
= C
ij
- u
i
Total Opportunity
Matrix C''
ij
= C
ij
(u
i
+ v
j
)