APPLIED OPTIMIZATION

WITH MATLAB
PROGRAMMING

P. Venkataraman
Rochester Institute of Technology

A WILEY-INTERSCIENCE PUBLICATION
JOHN WiLEY & SONS, INC.

CONTENTS

PREFACE
1

xiii

Introduction
1.1

Optimization Fundamentals / 2
1.1.1 Elements of Problem Formulation / 4
1.1.2 Mathematical Modeling / 10
1.1.3 Nature of Solution / 16
1.1.4 Characteristics of the Search Procedure / 20

1.2 Introduction to MATLAB / 25

1.2.1

Why MATLAB? / 25

1.2.2
1.2.3
1.2.4
1.2.5

MATLAB

1.2.6

Creating a Program / 40

Installation Issues / 26
Using MATLAB the First Time / 27
Using the Editor / 33
Creating a Code Snippet / 37

Problems / 44
2

Graphical Optimization

45

2.1 Problem Definition / 45

2.1.1 Example2.1 / 46
2.1.2 Format for the Graphical Display / 47
2.2 Graphical Solution / 48
2.2.1 MATLAB High-Level Graphics Functions / 48
2.2.2 Example2.1Graphical Solution / 50
2.2.3 Displaying the Graphics / 53
2.2.4 Customizing the Figure / 54
2.3 Additional Examples / 56
vii

VIII

CONTENTS

2.3.1 Example 2.2 / 56

2.3.2 Example 2.3 / 64
2.3.3 Example 2.4 / 73
2.4 Additional MATLAB Graphics / 79
2.4.1 Handle Graphics / 80
2.4.2 Graphical User Interface / 81
2.4.3 GUI Code / 84
References / 91
Problems / 92
3 Linear Programming

93

3.1 Problem Definition / 94

3.1.1 Standard Format / 94
3.1.2 Modeling Issues / 98
3.2 Graphical Solution / 107
3.2.1 Example 3.1 / H O
3.2.2 Characteristics of the Solution / 111
3.2.3 Different Solution Types / 114
3.3 Numerical Solutionthe Simplex Method / 115
3.3.1 Features of the Simplex Method / 115
3.3.2 Application of Simplex Method / 117
3.3.3 Solution Using MATLAB / 120
3.3.4 Solution Using MATLAB'S Optimization Toolbox / 123
3.4 Additional Examples / 124
3.4.1 Example 3.2Transportation Problem / 124
3.4.2 Example 3.3Equality Constraints and Unrestricted
Variables / 130
3.4.3 Example 3.4A Four-Variable Problem / 134
3.5 Additional Topics in Linear Programming / 138
3.5.1 Primal and Dual Problem / 138
3.5.2

Sensitivity Analysis / 148

References / 151
Problems / 152
4

Nonlinear Programming
4.1 Problem Definition / 155

154

CONTENTS

IX

4.1.1 Problem FormulationExample 4.1 / 155

4.1.2 Discussion of Constraints / 157
4.2 Mathematical Concepts / 159
4.2.1 Symbolic Computation Using MATLAB / 159
4.2.2 Basic Mathematical Concepts / 162
4.2.3 Taylor' s Theorem/Series / 169
4.3 Graphical Solutions / 171
4.3.1 Unconstrained Problem / 171
4.3.2 Equality Constrained Problem / 172
4.3.3 Inequality Constrained Problem / 173
4.3.4 Equality and Inequality Constraints / 174
4.4 Analytical Conditions / 175
4.4.1 Unconstrained Problem / 176
4.4.2 Equality Constrained Problem / 179
4.4.3 Inequality Constrained Optimization / 186
4.4.4 General Optimization Problem / 191
4.5 Examples / 194
4.5.1 Example 4.2 / 194
4.5.2 Example 4.3 / 196
References / 200
Problems / 201
Numerical TechniquesThe One-Dimensional Problem
5.1

203

Problem Definition / 204

5.1.1 Constrained One-Dimensional Problem / 204

5.2 Solution to the Problem / 205

5.2.1 Graphical Solution / 205
5.2.2 Newton-Raphson Technique / 206
5.2.3 Bisection Technique / 209
5.2.4 Polynomial Approximation / 211
5.2.5 Golden Section Method / 214
5.3 Importance of the One-Dimensional Problem / 217
5.4 Additional Examples / 219
5.4.1 Example 5.2Illustration of General Golden Section Method / 219
5.4.2 Example 5.3Two-Point Boundary Value Problem / 220
5.4.3 Example 5.4Root Finding with Golden Section / 223

CONTENTS

References / 225
Problems / 225
6

Numerical Techniques for Unconstrained Optimization

6.1

227

Problem Definition / 227

6.1.1 Example6.1 / 228
6.1.2 Necessary and Sufficient Conditions / 228
6.1.3 Elements of a Numerical Technique / 229

6.2 Numerical TechniquesNongradient Methods / 230

6.2.1 Random Walk / 230
6.2.2 Pattern Search / 234
6.2.3 Powell's Method / 238
6.3 Numerical TechniquesGradient-Based Methods / 241
6.3.1 Steepest Descent Method / 241
6.3.2 Conjugate Gradient (Fletcher-Reeves) Method / 244
6.3.3 Davidon-Fletcher-Powell Method / 246
6.3.4 Broydon-Fletcher-Goldfarb-Shanno Method / 249
6.4 Numerical TechniquesSecond Order / 251
6.5 Additional Examples / 253
6.5.1 Example6.2Rosenbrock Problem / 253
6.5.2 Example6.3Three-Dimensional Flow near a Rotating Disk / 255
6.5.3 Example 6.4Fitting Bezier Parametric Curves / 258
References / 262
Problems / 263
7

Numerical Techniques for Constrained Optimization

7.1 Problem Definition / 266
7.1.1 Problem FormulationExample 7.1 / 266
7.1.2 Necessary Conditions / 267
7.1.3 Elements of a Numerical Technique / 269
7.2 Indirect Methods for Constrained Optimization / 270
7.2.1 Exterior Penalty Function (EPF) Method / 271
7.2.2 Augmented Lagrange Multiplier (ALM) Method / 276
7.3 Direct Methods for Constrained Optimization / 281
7.3.1 Sequential Linear Programming (SLP) / 284
7.3.2 Sequential Quadratic Programming (SQP) / 289

265

CONTENTS

7.3.3

Generalized Reduced Gradient (GRG) Method / 297

7.3.4

Sequential Gradient Restoration Algorithm (SGRA) / 302

XI

7.4 Additional Examples / 307

7.4.1 Example 7.2Flagpole Problem / 307
7.4.2 Example 7.3Beam Design / 310
7.4.3 Example 7.4Optimal Control / 313
References / 316
Problems / 316
8 Discrete Optimization

318

8.1 Concepts in Discrete Programming / 320

8.1.1 Problem Relaxation / 321
8.1.2 Discrete Optimal Solution / 322
8.2 Discrete Optimization Techniques / 324
8.2.1 Exhaustive Enumeration / 326
8.2.2 Branch and Bound / 329
8.2.3 Dynamic Programming / 336
8.3 Additional Examples / 341
8.3.1 Example 8.4I Beam Design / 341
8.3.2 Zero-One Integer Programming / 343
References / 348
Problems / 348
9 Global Optimization

350

9.1 Problem Definition / 351

9.1.1 Global Minimum / 351
9.1.2 Natureofthe Solution / 354
9.1.3 Elements of a Numerical Technique / 356
9.2 Numerical Techniques and Additional Examples / 357
9.2.1 Simulated Annealing (SA) / 358
9.2.2

Genetic Algorithm (GA) / 366

References / 377
Problems / 378
10 Optimization Toolbox from

MATLAB

10.1 The Optimization Toolbox / 380

379

XM

CONTENTS

10.1.1 Programs / 380

10.1.2 Using Programs / 382
10.1.3 Setting Optimization Parameters / 384
10.2 Examples / 385
10.2.1 Linear Programming / 385
10.2.2 Quadratic Programming / 386
10.2.3 Unconstrained Optimization / 388
10.2.4 Constrained Optimization / 389
Reference / 391
Index