15.2.

2 Linear-Quadratic Problems

Next: 15.2.3 Pontryagin's Minimum Principle

Up: 15.2 Continuous-Time Dynamic Programming
Previous: 15.2.1.3
Variants of the

15.2.2 Linear-Quadratic Problems

This section briefly describes a problem for which the HJB equation
can be directly solved to yield a closed-form
expression, as opposed
to an algorithm that computes numerical approximations. Suppose that
a linear system is given
by (13.37), which requires
specifying the matrices
and . The task is to design a feedback
plan that asymptotically
stabilizes the system from any initial state.
This is an infinite-horizon problem, and no termination action is
applied.

An optimal solution is requested with respect to a cost functional

based on matrix quadratic forms. Let

nonnegative definite15.4

matrix, and let

be a
positive definite

be a

matrix. The quadratic cost

functional

is defined as

(15.20)

To guarantee that a solution exists that yields finite cost, several

assumptions must be made on the matrices. The pair
must be
stabilizable, and
must be detectable; see
[28] for specific conditions and a full derivation of
the
solution presented here.

Although it is not done here, the HJB equation can be used to derive
the algebraic Riccati equation,

(15.21)

in which all matrices except

space of
nonnegative definite

were already given. Methods exist

that solve for
matrices.

The linear vector field

http://planning.cs.uiuc.edu/node817.html[5/3/2016 8:28:47 PM]

, which is a unique solution in the

15.2.2 Linear-Quadratic Problems

(15.22)

is asymptotically stable (the real parts of all eigenvalues of the

matrix are negative). This vector field is obtained if

selected using a feedback plan defined as

is

(15.23)

The feedback plan

is in fact optimal, and the optimal

cost-to-go is simply

(15.24)

Thus, for linear systems with quadratic cost, an elegant solution

exists without resorting to numerical approximations.
Unfortunately,
the solution techniques do not generalize to nonlinear systems or
linear systems among obstacles.
Hence, the planning methods of
Chapter 14 are justified.

However, many variations and extensions of the solutions given here do

exist, but only for other problems that are
expressed as linear
systems with quadratic cost. In every case, some variant of Riccati
equations is obtained by
application of the HJB equation. Solutions
to time-varying systems are derived in [28]. If there is
Gaussian uncertainty
in predictability, then the linear-quadratic
Gaussian (LQG) problem
is obtained [564]. Linear-quadratic problems and
solutions
even exist for differential games of the form (13.204)
[59].

Next: 15.2.3 Pontryagin's Minimum Principle

Up: 15.2 Continuous-Time Dynamic Programming
Previous: 15.2.1.3
Variants of the

Steven M LaValle
2012-04-20

http://planning.cs.uiuc.edu/node817.html[5/3/2016 8:28:47 PM]