Introduction to Engineering Design Optimization

Prof. James T. Allison

Engineering System Design Laboratory
University of Illinois at Urbana-Champaign
What is the difference between:

1) Learning theory and numerical

methods for mathematical optimization,
and

2) Learning engineering design

optimization?
What is Design?
 Design:
Latin: "designare"
• To "mark out, devise, choose, designate, appoint," 1540s.

Noun: a plan or drawing produced to show the look and function or

workings of a building, garment, or other object before it is built or
made.

Verb: decide upon the look and functioning of (a building, garment, or

other object), typically by making a detailed drawing of it.

But how can we make good decisions/designations/plans for our

engineering designs?

Why is engineering design important?

Analysis

Design
 What is the relationship between analysis
and design?

• Analysis intuition vs design intuition?

• How can we obtain design intuition?
• Open-ended vs. structured nature?
• How can we support practicing design engineers
create more successful designs?
(especially for increasingly complex, integrated systems)
Engineering Design is the Inverse of Engineering Analysis

Engineering Analysis

System Description System Analysis

Engineering Design
 The design process is iterative.

Desired System
Performance Design Specification
 The design process is iterative.

Physical
Computer
Analysis
Testing
Simulation
Desired System
Performance Candidate Check Specification
Design Performance
Specification

Manual
Designed
Update
Optimization
Experiments
Design
Tuning
Algorithm
What is engineering design
optimization?
 Engineering Design Optimization
Framed engineering Mathematical optimization problem
design problem: formulation:
• Scope • Well-posed, solvable
• Assumptions • Models (approximates) original
• Goals target problem well

• Core SE 413 course objective: learn how to formulate

practical engineering design problems as optimization
problems
• This requires not only knowledge of optimization theory and
algorithms, but expertise in design thinking and application
knowledge
 Approaches to learning design
optimization:

1. Primary purpose: advance knowledge in design optimization

(e.g., PhD students doing research in design optimization
method creation and analysis) – Take SE 413, IE 513, and
other optimization courses
2. Primary purpose: Use design optimization as a tool in other
research domains (e.g., performing numerical experiments
for electro-thermal systems or components) – Take SE 413,
CS 450, other applied relevant courses
3. Primary purpose: Use design optimization in engineering
practice (e.g., engineers working in industry) – Take SE 413
(online sometime in the future?)
 How does engineering design
optimization relate to other design
strategies?
 Selected Design Strategies
Design by Tinkering:
• Make design decisions as you build
• Trial and error. Art and craftsmanship.
• Very helpful for building intuition, learning
the media of engineering
• Complexity of systems that can be designed
this way is limited
• Resulting performance/efficiency is limited
• Design fixation limitations (physical 
emotional)
Model/Simulation-Based Design
• Simulate results rapidly (often faster/less
expensive than physical experiments)
• Predict effects of design decisions
• Gain intuition through model development
• Discover what physical mechanisms are
influential with respect to design problem
• Increased design flexibility and complexity
Rigorous Experimentation
• Support design decisions using systematic
physical experiments
• Evidence-based decisions
• More objective and efficient design
exploration
• DOE: IE 400
Design Optimization
• Explore design space efficiently (find best
design without having to test them all)
• Go beyond meeting requirements (RDD)
• Make new things possible, create and
understand unprecedented systems
• Automate parts of the design process
 Selected Design Strategies
Design by Tinkering: Rigorous Experimentation
• Make design decisions as you build • Support design decisions using systematic
• Trial and error. Art and craftsmanship. physical experiments
• Very helpful for building intuition, learning • Evidence-based decisions
• Every design strategy involves significant creativity
the media of engineering • More objective and efficient design
• Complexity of systems that can be designed exploration
• What decisions/innovations
this way is limited • are made by
DOE: IE 400
• Resulting performance/efficiency is limited
• human designers?
Design fixation limitations (physical 
• Increased design strategy sophistication:
emotional)

• Enhances design
Model/Simulation-Based Design capabilities (speed,
Design Optimization
performance,
• Simulate results system
rapidly (often faster/less
expensive than physical experiments)
complexity)
• Explore design space efficiently (find best
design without having to test them all)
• effects
• Predict Requires additional investment
of design decisions (time,
• Go beyond meeting expertise,
requirements (RDD)
• Gain intuition through model development • Make new things possible, create and
resources)
• Discover what physical mechanisms are understand unprecedented systems
influential with respect to design problem • Automate parts of the design process
• Increased design flexibility and complexity
 Analytical Design (Design With Analysis)

We can only analyze abstractions:

A model is an abstract description of the real

world giving and approximate representation
of more complex functions of physical systems.
(POD p. 4)
 Analytical Design (Design With Analysis)

Engineering model types:

• Physical models: prototypes
• Symbolic models: language, drawings, mental
models, mathematical models
– Mathematical models include: first principles
models, empirical models, computer models.
Design Optimization:

• Translate a real engineering design problem into a solvable

optimization formulation.
• Application of optimization theory and algorithms.
• Development of system models that work with optimization
• Analysis of results, extraction of insights (beyond delivering an
‘optimal’ design)
• Support design of systems where good solutions are non-
obvious (good design not apparent via intuition)
 Challenges in Modern Engineering Design

Design optimization tools can help shift these curves:

System Knowledge

Design Flexibility

Original Design Process Improved Design Process

AIAA Technical Committee on Multidisciplinary Design Optimization, 1991

Types of Design Optimization:

- System architecture design

- Topology design
- Design tuning
Design Optimization:
Selecting the best design within the best available means

Three questions:
• How can we describe design alternatives?
– Design representation
• How can we quantitatively compare design alternatives?
– Comparative Metrics (objectives and constraints)
• Given a design, how can we predict its performance?
– Predictive mathematical models
– Important for exploring what is physically realizable

Design Optimization formalizes what engineering designers have

always done.

Design optimization paradigm: NLP formulation is a model of a design

problem.
Design Optimization Conceptual Example

• How can we describe design alternatives?

• How can we quantitatively compare design alternatives?
• How can we predict performance?
 Concept 1
Attainable Set

f 1 (x )

Concept 2
Attainable Set
Concept 3
Attainable Set

Target Point

Utopia Point

f 2 (x )
Optimization: Beyond Requirements-driven design:

• Exploit design DOF to deliver superior designs (# decisions > # reqs)

• What is mathematical optimization?
• Not just improvement, as the term is often used.
• Design Optimization: Translate an engineering design problem into a
formal mathematical optimization problem, and then solve it using
numerical optimization algorithms.

Example: Minimize deflection of a structure, with respect to geometric design,

subject to stress, fatigue, and cost constraints.
Design Variables:

• Design variables are independent quantities (designer must have

direct control over them)
• Often interchangeable with parameters
• System design must be defined completely by assigning values to
design variables:

• Must learn to think of x as the design.

A design is represented abstractly by the vector of design variables.

Inequality Design Constraints:

Example: axial stress requirement

Standard (negative null) form:

Engineering design constraints are most often inequality

constraints.
Other similar examples of inequality constraints:
• Temperature, voltage, current, velocity (bounds on state)
Optimization: Modeling a Design Problem
(not just an engineering system)

Solutions are optimal with respect to the design model (i.e.,

the optimization formulation).
Optimization: Beyond Requirements-driven design:

• Design Optimization: Translate an engineering design problem into a

formal mathematical optimization problem, and then solve it using
numerical optimization algorithms.

Let’s have a look at a few simple mathematical

optimization problems.

(Note: difference between mathematical/analytical

optimization example, and a design optimization problem.)
Analytical Optimization Examples
1D Optimization Example
2D Optimization Example
Engineering Design Optimization
Example
Design Optimization Example
Helical Compression Spring
Design Optimization Example
Helical Compression Spring

• g(x) is a vector-valued function with seven output components.

• MATLAB implementation of multiple inequality constraints: define a
constraint function with multiple outputs.
Problem Visualization and Abstraction
Optimization Problem Visualization

• Possible in lower dimensions

• Useful abstraction of the original engineering design problem
– Insightful to visualize optimization problem space as a constrained
surface
– Provides useful language for reasoning through design problems
– Design Optimization Paradigm (overview article)
– DO abstraction: breakthrough concept in engineering design research
(1970s)
• Concepts extend to higher-dimensional problems
Optimization Problem Visualization: Minimization

• How can we find our way to the lowest spot?

• How can we find what direction is downhill?
– Vision: expensive, what if we can’t see?
– Ankle rotation
– Take some small steps nearby (samples)
What can we learn from optimization problem visualization?

• Intuition for optimization

• Conceptual abstraction and language for design optimization
– (design space, response surface, descent, etc.)
• Nature of design problem
– Smooth vs. nonsmooth
– Unimodal vs. multimodal
– Interior vs. boundary optima
– Problem conditioning
– Monotonicity and well-boundedness (are we missing any constraints?)
Optimization Problem Visualization vs. Numerical Optimization

• Visualization requires exhaustive enumeration of design

alternatives
– Every point on the objective function surface represents a unique
design alternative
– DOE/sampling is common in industry
• Numerical Optimization: find the optimal solution without
testing all design alternatives
– How might we find the bottom of a valley without sampling/sensing all
points?
When is Design Optimization Useful?
When is design optimization warranted?

• Can require significant investment of effort (tradeoff between design

effort/design performance)
• Can be important when designing new types of systems without design
heritage
• Large-dimension, nonlinear, non-intuitive problems
– Spring design: small dimension, but nonlinear, coupled formulas
– Electro-thermal system design: Very difficult to get intuition for this coupling
• May be justified when enhanced performance or mass reduction is critical
(aerospace)
• Can enable more flexible design space exploration (e.g., more freeform
design, more comprehensive design problem formulations)
• Perhaps part of a problem can be solved using optimization, and other
parts solved using conventional methods (procedure-based design)
Non-trivial engineering design optimization problems:

• Solution is non-obvious:
– With some trivial problems, we can use intuition to reason through
what the optimal result will be.
– Monotonicity is often present in engineering design. Often one or
more constraints are active. (min mass, st stress)
– Active failure mode constraints are usually fine, but active arbitrary
bounds usually indicate something was overlooked in the formulation
(oversimplified model, improper design representation).
• Competing phenomena or other tradeoffs create design conditions where
a ‘sweet spot’ exists, and optimization helps us find this sweet spot
• Optimization formulation, modeling, and solution provide new design
insights
– Results provide new intuition for how certain types of systems should
be designed
Example: Simple, Non-Trivial Problem
Sweet Spot Example: Simple Structural Design

• Choose rod attachment position x, rod radius r

• Minimize rod mass, avoid buckling and yield failure
• Tradeoff from conflicting phenomena:
• How does increasing x influence axial rod force?
• How would it influence Pcr?
Example: Simple Structural Design
Example: Simple Structural Design
Design optimization formulation:
Example: Simple Structural Design
Analysis required for optimization (find Ns, Nb):

Rod axial force in terms

of known quantities

Yield stress safety factor

Critical load for buckling

Buckling safety factor

Example: Simple Structural Design
Analysis required for optimization (find Ns, Nb):

Rod axial force in terms

of known quantities
Design-Appropriate Model Development:
• Predict performance metrics as a function
Yield stress safety factor
of independent design variables
• Balance accuracy and Critical load for buckling
development/computational effort

Buckling safety factor

Example: Simple Structural Design
For a given arbitrary design (x,r), it is difficult to predict
which failure mode (Ns, Nb) will dominate.

How can we quantify which failure mode dominates?

• Whichever mode has a lower safety factor for a design.
• Due to nonlinear relationships, failure mode dominance
will likely change with design.

What does this dominance relationship look like for this

structural design problem?
 Nb > Ns
 Yield stress dominant
failure mode Nb < Ns
 Buckling dominant
failure mode

Large r  non-optimal
designs

For the lightweight

designs we care about,
buckling is dominant.
Analyzing the Design Problem:
• We can safely assume that buckling is the dominant
failure mode, and focus on satisfying the buckling
constraint.
• Let’s assume a value for r, and see how Nb and Fr change
with x.
• Is there a design ‘sweet spot’?
Nb initially increases with x Eventually Nb decreases with x
due to reduced axial force due to increased length

Synergy between axial

force/buckling stability Then this is the
produces this ‘sweet feasible domain
spot’ with maximum Nb. for x

What values of x and r

will give us the minimum
mass, feasible rod If we specify Nallow = 1
design? here
Design Optimization Inspired Design Strategy:
For each value of x:
• Assume that buckling is the dominant failure mode,
• Assume that the buckling constraint is active:

(right at the limit of failing due to buckling, if we are not

at the limit, then mass could be reduced further)
• Solve for r to obtain minimum mass rod for a given x:

• Plot mass as a function of x (assuming optimal r), find

the design ‘sweet spot’ where mass is minimized.
Problem Formulation Framework
 Design Methodology =
Problem Formulation
+ Solution/Search Method
Design Optimization Problem Formulation Framework

Three important dimensions of a design optimization

problem are:

1. Design Representation (parameterization)

2. Comparison Metrics (quantitative ranking,
objectives/constraints)
3. Predictive Model (maps design representation to
metrics)
 Design Optimization Problem Formulation Framework

Substantive
rationality
solution
Modeling for Design
 Modeling for Design
Requires a deeper understanding of the relationship
between engineering analysis and design.
• Design is the ‘inverse problem’
• Design optimization helps to focus modeling activities,
reveals model shortcomings
• Development of adequate models: perhaps the most
significant barrier to adoption of design optimization in
industry.
• Why might existing models be inadequate?
• Engineers who develop models often are not the ones
using them for design.
 Product Concept Systems-Level Detail Testing and Production
Planning Development Design Design Refinement Ramp-up

Steady-state

Lumped-parameter simulations
Error

Distributed-parameter simulations

Computational Expense
Two fundamental types of system interaction/coupling:

Analysis Coupling: Example: Aeroelasticity

• Influence of component or discipline behavior/properties
on another. Physics/energy/information transfer. Structural
Analysis
• Identifiable via analysis of physics models or sensitivity
studies.
• Used often in systems engineering: integration models, Aerodynamic
multiphysics simulation Analysis
• Overlook analysis coupling  inaccurate simulation

Design Coupling: Example: Wind Plant

• The effect that changes in one design domain has on Design
design decisions that should be made in another domain. Turbine
Design
• Identified via model-based optimization studies.
• Design coupling is only starting to be addressed formally in
systems engineering practice. Layout Design
• Overlook design coupling  suboptimal system design
 Modeling for Design
• A model must be constructed with its end
purpose in mind
• Different from conventional model development
• Start simple, add sophistication as needed
• Requirements for use with optimization:
– Flexibility and accuracy in design domain
– Access to independent design variables
– Reasonable computational expense
– Awareness of assumptions and validity domain
– C1 smooth (if ∇-based optimization used)
 Modeling for Design
How to select / create design-appropriate models?
• Consider design process stage • Commercial vs in-house options
• Tradeoff between expense/ accuracy • Data driven vs first principles
• Tradeoff: solution effort vs solution • Unique requirements for design
quality

Resistor 1D ODE
1. 2. Network 3. 2D PDE 4. 3D PDE
Network

Levels of model fidelity

Model trade-off
(electro-thermal system example)
 Modeling for Design
• Potential issues to consider when choosing whether to use
commercial simulation tools with design optimization:
– May have built-in optimization algorithms. (limited)
– Interface with optimization (PIDO tools: iSIGHT, Optimus, Model Center)
– Limited access to sensitivities
– Less flexible in what can be modeled and what can be changed
– General purpose modeling tools may not be as efficient as a targeted
custom model
• What are the pros/cons to developing your own model for design
optimization? (e.g., MATLAB FEA)
– Highly flexible, easy integration with optimization.
– Potential for very efficient, tailored model implementation.
– May involve significant development effort.
– Often difficult to create accurate models from scratch.
 Modeling for Design
Model Types:
• Data-Driven Models:
– Accurate within bounds of available data
– Often easier to construct (e.g., response surfaces)
– May not maintain predictive accuracy when changing
design (requires more data if design is to be changed).
• First Principles Models: (e.g., physics-based)
– Often better at predicting the effects of design
changes
– Can require more involved development effort
 Modeling for Design
Characteristics of design
appropriate models. Impact on design solution

Low Fidelity
• Inaccuracy has low impact on design Model
solution (robustness)
High Fidelity
• Allows for design flexibility
Model
• Support adjustment of independent
design variables
• Often must include important
physics couplings How far apart are the optima?
 How do we measure model suitability for a
design problem?
• Model validation: how closely do predicted
results match actual system behavior?
versus
• Assessing model suitability for design: Will using
the model get us to the right design solution?

We need models that are accurate with respect to design

solution, which may not be the same as accurate with respect to
predicted behavior.
Integrated Physical and Control System
Design: Co-Design
Integrated Design of Active Dynamic Systems
Mechatronics:
• Engineering discipline that aims for integrated design of actively controlled
electro-mechanical-hydraulic (etc.) systems
• Largely heuristic or procedure-based strategies for addressing design
coupling
• Deals with the design of highly multidisciplinary engineering systems
System-Optimal Design of Mechatronic Systems
• More formal strategies needed to better manage design coupling and
improve system performance
• MDA/MDO methods can be applied with extensions for dynamic systems
Co-Design: Integrated Physical and Control System Design
• Account for synergistic coupling between physical and control design
• Much more than just ‘design for control’
• Can be viewed as a specific subclass of MDO: coupled physical and control
system design subproblems
Design Process Options Co-Design
Integrated physical (plant) and control
system design
Conventional Sequential Design Simultaneous Design

Iterated Sequential Design Nested Design

Distributed optimization is also

an option
Example: Robotic System Design
• Exhibits coupling between physical and control system design.
• Passive dynamics essential for performance improvement.

Conventional sequential design:

Mechanical Design Control Design

• Strength and stiffness • Fixed plant design
• Kinematics • Objective based on
• Mass distribution and trajectories
balance • No plant constraints

In Reality: Control system design depends on mechanical design, AND

mechanical design performance depends on system dynamics and control.

But what if we considered physical and control system design simultaneously?

• Single system design objective that applies to both design domains
• Dynamic performance considered during mechanical design
• No longer constrained by rigid link assumption (downsize links, actuators)
Example: design of an energy-efficient
counterbalanced robotic manipulator (rigid links).
Task: move 20 kg payload from p0 to pf in 2 seconds.
Single system objective applied
to both design domains
Sequential Manipulator Design:
plant design followed by trajectory optimization.
(link lengths) (offsets) (masses)
xp = [1.0, 1.0, 0.3, 0.3, 10, 10 ]T

xc = [0.334, 0.192, 0.377, -0.875]T

E(xp, xc) = 27.6 Joules

Note: task is not a complete cycle.

Allison (2012), Allison (2013)

Optimal Co-Design:
simultaneous plant and control optimization.
(link lengths) (offsets) (masses)
xp = [0.838, 0.711, 0.216, 0.885, 3.89, 20.7]T

xc = [0.359, 0.163, 0.00829, -0.0249]T

E(xp, xc) = 5.86 × 10−5 Joules

Co-design can exploit synergy

between passive dynamics and
control system design.
Bose® approach for rethinking system design: assume
nothing about what connects wheel to vehicle, identify the
best possible force trajectory.
F(t)?

This approach allows engineers to rethink whole system

design when introducing active control.

Removing system architecture assumptions allows us to

deliver fundamentally better performance. Comfort
Example: shifting the suspension/handling tradeoff curve.
Handling
performance
Design under fewer constraining assumptions makes new
things possible.

But it brings new challenges …

Utilizing Co-Design in Systems Engineering
Significant potential for
• Enhancing design integration
• Improving system performance, capitalize on passive dynamics in an active system
• Tailor structural/mechanical/control system designs  system optimality
• Identifying problems due to interactions early
Howmight co-design be used within systems engineering?
• Especially appropriate for early-stage design (predesign)
• Identify qualitative synergy mechanisms that can guide later design efforts
• Tool for mechanical/structural designers to develop a design they are confident
has accounted for coupling with control system design

Adjust co-design formulation if

we discover something that
was overlooked

Deshmukh, Herber, and Allison (2015)

Why Learn Design Optimization?
Mastery of both engineering design and mathematical
optimization is a rare and valuable skill.
Value of Engineering Design Optimization
System Knowledge

• Reduce product development time Design Flexibility

Original Design Process Improved Design Process

• Improve product quality and performance

• Enable bigger design steps and faster innovation
– Go beyond small perturbations
• Reduce energy and resource consumption
• Help engineers focus on creative tasks
– Automate mundane tasks
How can I learn more?
SE 413: Engineering Design Optimization

• Spring 2019: Tu/Th, 3:30 – 4:50 pm

• 3 or 4 credits (4th credit: intensive project)
• Prerequisites: Calculus III, linear algebra, a course in programming
• Applied optimization course: optimization theory, numerical
methods, practical application to design
SE 413 Course Objectives:
The primary objective of this course is for students to gain the knowledge and creative skill
required to translate practical engineering design problems into mathematical
optimization problems that can be solved using numerical methods for optimization. In
supporting this primary objective, the following objectives should be met by students:

• Demonstrate an understanding of how design optimization fits into the overall

engineering design process.
• Learn how to formulate practical engineering design problems as well-posed
optimization problems.
• Understand continuous optimization theory and its implications for algorithm
development, problem formulation, and system modeling.
• Develop a detailed understanding of numerical methods for optimization through
implementation in MATLAB.
Specific Skills
Formulation
• Translate engineering design problems into mathematical optimization problems
• Know how to formulate well-posed, solvable optimization problems
• Be creative through problem definition

Optimization Algorithms:
• Understand them well enough to choose the right algorithm for a problem
• Know what to do when things go wrong
• Algorithm, formulation, model
• Write your own basic optimization algorithms
• Explore design alternatives efficiently

Modeling:
• Learn how to develop engineering analysis models that work well for design
optimization
• Understand how to use the right model fidelity and complexity for a problem
Other Relevant Courses:
• IE 513: Optimal System Design
• ECE 490: Introduction to Optimization
• AE 498: Topology Optimization
• GE 598: Dynamic System Design
• AE 502: Optimal Control

See: tinyurl.com/esdl-info
Appendix
Integrated System Design
System Design Paradigm:
The whole is greater than the sum of the parts
• Base design decisions on what is best for the overall system, not individual parts

Designing subsystems in isolation can result in several issues:

• Problems may arise due to neglected system interactions that are not realized until
physical testing (failure, unexpected behavior, etc.)
• Missed opportunities for performance improvement or cost reduction

Integrated design methods can capitalize on synergistic

relationships between system components.
• Strive for system-optimal designs (optimal with respect to overall system
performance, not individual component optimality)
Sample of ESDL design applications

Renewable Robotic System Hybrid Powertrain Rheologically

Energy System Design and Suspension Complex Material
Design Design Design
- +

system-level
3: Inverter 4: Battery

design
2: Generator

1: Engine 5: Motor

material-level
design
6: Gears

7: Vehicle

Spacecraft Structural and Electro-Thermal Static Flow Mixer

Design Materials Design System Design Design
“An astutely chosen technology platform can shorten
and improve product development cycles. ...

Tools that help companies conduct thorough design

analyses earlier ... foster improved product quality and
faster development.

Simulating and analyzing a design earlier will improve

product performance, reliability, and overall
development time.”

-Erich Buergel, Mechanical Engineering, August 2010

Optimization: Beyond Requirements-driven design:

• Exploit design DOF to deliver superior designs

• What is optimization?
• Design Optimization: Translate an engineering design problem into a
formal mathematical optimization problem, and then solve it using
numerical optimization algorithms.

Example: Minimize deflection of a structure, with respect to geometric design,

subject to stress, fatigue, and cost constraints.
State of Engineering Design Optimization in Industry

• Early stages of industry acceptance (think FEA in 1990)

• Pioneered in aerospace, now somewhat common in
automotive (at least with structural design)
• Opportunity for impact on how engineering design is done