UUM 526 Optimization Techniques in Engineering

Lecture 1: Introduction
Asst. Prof. N. Kemal Ure
Istanbul Technical University
ure@itu.edu.tr

September 17, 2015

Asst. Prof. N. Kemal Ure (ITU)

Lecture 1

September 17, 2015

1 / 16

Overview
1

Problem formulation
What is Optimization?
Constrained and Unconstrained Optimization

Examples
Example: Data Fitting
Example: Spacecraft Control
Example: Structural Design
More Examples

Types of Optimization Problems

Least-Squares Problems
Linear Programming
Convex Optimization
Nonlinear Programming

Asst. Prof. N. Kemal Ure (ITU)

Lecture 1

September 17, 2015

2 / 16

Problem formulation

What is Optimization?

What is Optimization?

Wikipedia: selection of a best element (with regard to some criteria)

from some set of available alternatives.

Nice, but this is not useful. We need a formal (mathematical) description

Mathematical Optimization (Programming)

Set of elements (will be also called decision variables): x A

Evaluate the cost of the element: f A R

Optimization Problem:
minimize

Asst. Prof. N. Kemal Ure (ITU)

Lecture 1

f (x)

September 17, 2015

3 / 16

Problem formulation

Constrained and Unconstrained Optimization

Constrained and Unconstrained Optimization

Generally A = Rn and we specify the constraints via functional
equalities and inequalities
Unconstrained Problem: minimize f (x), f Rn R
Constrained Problem:

minimize

f0 (x)

subject to

fi (x) 0, i = 1, . . . , m

Functions fi Rn R represents constraints of the problem.

We can embed the equality constraints by imposing fi (x) 0 and
fi (x) 0 at the same time.

Cool. But how is this connected to engineering?

Asst. Prof. N. Kemal Ure (ITU)

Lecture 1

September 17, 2015

4 / 16

Examples

Example: Data Fitting

Data Fitting

xi

Experiment

yi

An experiment is conducted with inputs xi Rn . Outputs yi R are

observed, i = 1, . . . , ndata
Your job is to find a linear function that explains the relationship
between x and y as good as possible
Can we formulate this as an optimization problem? What is the set
of decisions? What is the cost function? What are the constraints?

Asst. Prof. N. Kemal Ure (ITU)

Lecture 1

September 17, 2015

5 / 16

Examples

Example: Data Fitting

Data Fitting
xi

yi

Experiment

Let w Rn , so that y = wT x is a linear function. w is the decision

variable.
The unconstrained formulation:
ndata

minimize

T
[yi w xi ]

i=1

If we do not want the weights too large, we can add:

n 2
w R
subject to w2 =
i

i=1

R is a positive real number that constraints weights from growing too big

Asst. Prof. N. Kemal Ure (ITU)

Lecture 1

September 17, 2015

6 / 16

Examples

Example: Spacecraft Control

Spaceraft Control
A satellite is orbiting the earth with
ability to control its orientation with
thrusters. The pitch angle of the
spacecraft is represented by [, ].
Spacecraft dynamics can be modeled as
a linear dynamical system in discrete
time
k+1 = Ak + Buk , k = 0, 1, . . . , tf
Your job is to find a sequence of thruster commands uk such that
spacecraft is steered from init to f inal in t = tf
The fuel usage should be as low as possible. Thrusters can also exert
a limited amount of moment at per step.
Asst. Prof. N. Kemal Ure (ITU)

Lecture 1

September 17, 2015

7 / 16

Examples

Example: Spacecraft Control

Spaceraft Control
Main idea: Objective function represents fuel usage, constraint
functions represent spacecraft dynamics and thruster limits

minimize
subject to

k=t 1

f
k=0 uk
uk umax , k = 0, . . . , tf 1

0 = init ,
tf = f inal ,
k+1 Ak Buk = 0, k = 0, . . . , tf 1
Note that problem gets interesting when the dynamics are
time-variant A = Ak , B = Bk . We can handle that case too.
We can formulate any optimal open loop linear control problem
similarly.
Asst. Prof. N. Kemal Ure (ITU)

Lecture 1

September 17, 2015

8 / 16

Examples

Example: Structural Design

Design of a Wall Bracket

Your job is to design a wall bracket that

supports W amount of load. The
material for the bars are fixed with
failure stress a .
You need to determine the cross
sectional areas A1 and A2 , such that the
total weight is minimized.
Length of bars l1 and l2 can be
computed from geometry. Forces acting
bars F1 and F2 can be computed by
solving the statics equations

Asst. Prof. N. Kemal Ure (ITU)

Lecture 1

September 17, 2015

9 / 16

Examples

Example: Structural Design

Design of a Wall Bracket

Main idea: Cost function represents the weight, constraint functions

represent stress failure conditions and positivity of decision variables

minimize

l1 A1 + l2 A2

subject to

A1 0
A2 0,
F1
A1 a 0,
F2
A2 a 0

Many other structural design problems can be formulated similarly

Asst. Prof. N. Kemal Ure (ITU)

Lecture 1

September 17, 2015

10 / 16

Examples

More Examples

More Examples
We will be looking at many other examples throughout the class

Airline Operations: Origin-Destination Pairs, Crew Planning,

Space Robotics: Shortest Path Problems, Descent Trajectory Generation

Propulsion: Fuel Burning Rate, Nozzle Design

Aerodynamics: Airfoil Design, CFD

Avionics : Circuit Design, Sensor Design

Good News: Almost any engineering problem can be cast as an

optimization problem!
Bad News: We do not know how to efficiently solve the general
optimization problem. We need to understand the special cases we
can efficiently solve.
Asst. Prof. N. Kemal Ure (ITU)

Lecture 1

September 17, 2015

11 / 16

Types of Optimization Problems

Taxonomy of Optimization Problems

The general problem is difficult for arbitrary f and fi
minimize

f0 (x)

subject to

fi (x) 0, i = 1, . . . , m

There are certain exceptions which can be solved efficiently

Least-squares problems
Linear Programming
Convex Optimization

If you can formulate your problem as one of the above, you have
basically solved the problem
Otherwise, you might need to focus on development of new
algorithms
Asst. Prof. N. Kemal Ure (ITU)

Lecture 1

September 17, 2015

12 / 16

Types of Optimization Problems

Least-Squares Problems

Least-Squares Problems

minimize

Ax b22

Analytical solution exists! x = (AT A)

AT b

There are efficient algorithms for computing the solution

If A Rnk , runtime complexity is around n2 k
It is easy to recognize least-squares problems

Data Fitting is a least-squares problem!

Just set A = [xT1 , . . . , xTndata ], b = [y1 , . . . , yndata ]T

Asst. Prof. N. Kemal Ure (ITU)

Lecture 1

September 17, 2015

13 / 16

Types of Optimization Problems

Linear Programming

Linear Programming

minimize

cT x

subject to

aTi x bi , i = 1, . . . , m

Cost and constraints are linear functions of the decision variable x

No analytical solution
There are efficient algorithms for computing the solution
If x Rn , runtime complexity is around n2 m
It is not as easy to recognize linear programming problems but there
are a few standard tricks
Spacecraft Control is a linear programming problem! Cost function and
constraints are linear functions of uk
Many other practical examples, such as airline scheduling.

Asst. Prof. N. Kemal Ure (ITU)

Lecture 1

September 17, 2015

14 / 16

Types of Optimization Problems

Convex Optimization

Convex Optimization
The general problem where f and fi are convex functions with
convex domains.
minimize

f0 (x)

subject to

fi (x) 0, i = 1, . . . , m

Let 1 , 2 0, 1 + 2 = 1

Convex Function :f (1 x + 2 y) 1 f (x) + 2 f (y)

Convex Set: x, y A 1 x + 2 y A

Includes least-squares and linear programming as special cases

Efficient polynomial time algorithms exist for finding global optimum,
even for nonlinear convex problems
Can be difficult to recognize
Wall Bracket design is a convex optimization problem! (just check the
convexity definitions)
Many other important applications exist. Convex Optimization is at the
heart of modern applications.

Asst. Prof. N. Kemal Ure (ITU)

Lecture 1

September 17, 2015

15 / 16

Types of Optimization Problems

Nonlinear Programming

Nonlinear Programming
Back to the general form. f and fi are arbitrary (i.e. not convex)
minimize

f0 (x)

subject to

fi (x) 0, i = 1, . . . , m

No general polynomial time algorithm exists for global optimum.

Existing algorithms are either slow or get stuck in local optima

Many challenging engineering problems are nonlinear optimization

problems

Nozzle design, almost anything that involve PDEs

Convex approximations are available! (Major research trust in the

recent years)
Focus is more on developing efficient algorithms rather than
formulating problems
Asst. Prof. N. Kemal Ure (ITU)

Lecture 1

September 17, 2015

16 / 16