A Computational Algorithm for the Greens Function Method of Sensitivity Analysis in Chemical Kinetics

EUGENE P. DOUGHERTY* and HERSCHEL RABITZI

Chemistry Department, Princeton Uniuersity, Princeton, New Jersey 08540

Abstract
The recent interest in numerical modeling of chemical kinetics has generated the need for proper analysis of the system sensitivities in such models. This paper describes the logic for a program developed by the authors to implement the Greens function method of sensitivity analysis in complex kinetic schemes. The relevant equations and numerical details of the algorithm are outlined, two flow charts are provided, and some special programming considerations are discussed in some detail. Computer storage and computational time considerations are also treated. Finally, applications of sensitivity information to understanding complex kinetic system behavior and analyzing experimental results are suggested.

Introduction
The recent interest in the numerical solution of the differential equations of chemical kinetics [1,2] has brought about considerable research in the area of sensitivity analysis of reaction mechanisms [3-lo]. Sensitivity analysis of the coupled differential equations of reaction kinetics involves determining the sensitivity of output concentrations at various times with respect to the input parameters of the model (e.g., the rate constant parameters and initial species concentrations). The Greens function method (GFM) has already been shown [9,10] to be a very promising approach to the sensitivity analysis of large kinetic models. The present paper outlines the methods and logic for an algorithm which implements the GFM to study complex kinetic schemes. It is expected that extensions of this technology should be applicable in areas other than chemical kinetics such as quantum dynamics [Ill,chemical engineering [12,13],and world models for resources and population [14,15]. This paper is organized as follows: The second section outlines the theoretical and numerical considerations relevant to the GFM algorithm. Two flow charts are provided to allow the potential
*Present address: Research Laboratories, Rohm and Haas Company, Spring House, PA 19477. t Alfred P. Sloan Fellow and Camille and Henry Dreyfus Teacher-Scholar. International Journal of Chemical Kinetics, Vol. XI, 1237-1248 (1979) 0538-8066/79/00 11- 1237$01.00 8 1979 John Wiley & Sons, Inc.

1238

DOUGHERTY AND RABITZ

user to construct his own sensitivity code. The third section treats some special programming considerations and procedures necessary for a proper analysis of sensitivities. The fourth section considers potential applications of sensitivity analysis for understanding and predicting system behavior and for treating experimental results. The concluding section of the paper presents a few suggestions for possible improvements to the algorithm proposed in this paper.

Theory and Numerical Considerations

Here we summarize the basic equations and discuss some relevant numerical considerations for implementation of the GFM to complex kinetic schemes. Although some of these details have already been treated in two earlier papers [9,10],a brief discussion is given here so that the program logic may be more easily understood by the reader. The coupled first-order ordinary differential equations (ODES) describing chemical kinetic systems are of the following general form:
(1)
Y = f(Y,cy,t),

Y(0) = YO)

Here y is an n vector of species concentrations, cy is an m vector of input parameters, and the dot indicates differentiation with respect to time. For each species, f i is determined by the law of mass action in chemical kinetics [16],which defines the rate of reaction in terms of the concentrations of its reactants; f i is a polynomial up to third order in the species concentrations y. Equation (1)is generally numerically stiff [17,18],and should be dealt with by special software designed to treat stiff ODES [19]. The first-order sensitivity coefficient is defined as the partial derivative of the ith species concentration with respect to the j t h input parameter byilbaj. By differentiating eq. (1)with respect to aj, we obtain

where the matrix elements of the n X n Jacobian matrix J are given by

(3)
and the notation

(4)

Z,(t) = - Z(t)
dX

is used. The form of the Jacobian is known analytically once the reaction

model is specified. The initial condition y$) is the zero vector, unless aj
is the initial concentration of the ith species concentration, in which case
y$) = 6i. The symbol 6i is a vector whose components are all zero except

SENSITIVITY AND ANALYSIS

1239

the ith component, which has the value 1. An n X n Greens function matrix K(t,7) can be constructed for eq. (2) which satisfies (5a) (5b)
- K(t,7) - J ( t ) K ( t , 7 )=

d dt

0, t

>T

K(t,7) = 1

It can be shown [9] that y,(t) is given by

In practice K(t,7) is computed by first solving eq. (1)to obtain the concentration at an adequate number of grid points so that simple interpolation formulas can be employed to calculate a sufficiently accurate J(t) for use in eq. (5). Suppose that time t = tl is the first instance where sensitivity information is sought. It is more convenient to calculate the adjoint Greens function matrix Kt(7,t) [9], and the differential equations for each row of this matrix are solved by proceeding backward in time from t 1 to zero. The relevant initial value problem is

d d7
(7b)

- kf ( 7 , t l )

+ k{

( 7 , t l )J(T)=

0, 7

< tl

k (tlJ1) = 6il f

where kf, row vector, refers to the lth row of the adjoint Greens function a matrix. This row corresponds to the lth species concentration. Equation (7), like eq. (l), generally stiff; a special ODE solver is then required for is efficient solution. Linear interpolation of logy is used to construct J(7). Since 191

(8)

Kf(7,t) = K(t,7)

we may construct the integrand of eq. (6) for the Zth species for all m input parameters by a simple row-column vector multiplication. For the j t h input parameter the integrand is
(9)

ILj(7,tl) = kf (7,tl) * faj(7)

The integrands may be constructed a t the same time as kfis being solved on [0, t l ] by evaluating faj(7). To obtain the sensitivity integrals, we use the step sizes (or some portion of them) taken automatically by the ODE solver in treating eq. (7). Since the behavior of the integrands is mainly dictated by the behavior of kf( 7 , t l ) ,these step sizes taken by the solver are used as a guide in constructing the integration grid. Every IJUMPth step is taken to ensure accuracy. Generally, IJUMP = 5 or 6 was found to be sufficient. By using the solvers own mesh points to construct this grid, we are able to deal with even rapidly varying integrands which result

1240

DOUGHERTY AND RABITZ

from the stiff nature of eq. (7). To carry out the actual numerical integration, either one of two possible quadrature schemes was used. For rapidly varying integrals, locally exponential behavior is assumed to hold between two adjacent grid points 7 1 and 7 2 ,

where the constants Alj and Rlj are determined by Ilj(7-1)and Ilj(72). For slowly varying integrals and for the case in which the integrand changes sign or goes to zero, a simple trapezoid rule was used. The value of the entire sensitivity integral is obtained by summing up the individual integral contributions from each pair of adjacent grid points. By the above procedure the sensitivity of the Ith concentration to all m rate constant parameters can be obtained simultaneously with the solution of kfon [ti,O]. The same procedure is repeated for the other n-1 rows. To obtain sensitivity information a t times longer than t l , the recursion formula [9] is used

Thus it is necessary to compute Kt(7,t) only on short 7 intervals [ t k , O ] , [ t 2 ,t I], etc., rather than repeatedly on the longer intervals [tk,O]. Altogether n 1 sets of n-vector stiff ODES are solved and mN integrals must be evaluated. Here N is the number of times sensitivies are to be computed and m is the number of rate constant parameters. For the m initial condition parameters, no integrals need to be evaluated in eq. (6) since in this case fa, = 0. As one can see from the preceding section, evaluation of sensitivities involves reasonably straightforward programming. Simple interpolation, numerical integration, indexing, matrix arithmetic, and numerical solution of stiff ordinary differential equations are required. All are standard procedures in numerical analysis. To give the contents of this section in a more concise form, we have included two somewhat detailed flow charts (Figs. 1and 2). The first gives a general outline of the sensitivity code. The second shows how the sensitivity integrals are actually evaluated in the course of solving eq. (7). The notation corresponds to that used in this section.

Some Special Programming Considerations for Sensitivity Analysis of Kinetic Models

In developing the equations and considering the relevant numerical aspects in the previous section, we gave a brief overview of what is involved in determining sensitivities for reaction mechanisms. In this section we

SENSITIVITY AND ANALYSIS

1241

Read d a t a : n(No. o f s p e c i e s ) , m'(No. o f r a t e c o n s t a n t s p a r a m e t e r s ) , rate c o n s t a n t s , m"(No. o f i n i t i a l c o n c e n t r a t i o n p a r a m e t e r s ) , m=m'+m", i n i t i a l c o n c e n t r a t i o n s , e r r o r t o l e r a n c e , i n t e g r a t i o n o p t i o n , IJUMP S o l v e d i f f e r e n t i a l e q u a t i o n s f o r c o n c e n t r a t i o n s by c a l l i n g ODE s o l v e r . Solver e v a l u a t e s and 2 a s n e e d e d . Save y a t a g r i d o f t i m e s f o r f u t u r interpolation. Check mass c o n s e r v a t i o n p e r i o d i c a l l y .

Loop o v e r No. of t i m e s s e n s i t i v i t i e s are t o b e computed, i . e tl, tr, tN. (to = 0)

..., ...,
9=, .1.

. ., n
~ ~~~~

Interpolate to find y ( t r ) . Compute a j / y t ( t r ) . j = l m normalized s e n s i t i v i t i e s l a t e r . Evaluate f ( t r ) , j=l,

-cL:

,...,
~~~

l c h e c k s e n s i t i v i t y mass c o n s e r v a t i o n r e l a t i o n s

Figure 1. Flow chart for the Green's function method of sensitivity analysis of chemical reaction mechanisms.

amplify some of these latter points to show how to carry out a thorough and efficient determination of sensitivities for problems of chemical interest. We consider a few preliminaries necessary to set up the study, certain programming hints that should be useful, and estimates of programming effort in terms of execution time and storage requirements. The first step in kinetic modeling is to build a mechanism suitable to the system of interest. Naturally a thorough literature search is required to become familiar with the expected system behavior and to obtain rate constant parameters for the elementary steps involved. Information from thermodynamics as well as kinetics is useful in building the reaction set and determining the best rate constants. Estimates are often necessary [20], and appropriate caution should be used here. A conservative approach is desirable to ensure that no important step is omitted, since sensitivity analysis can be useful in refining the model, as discussed below. The second step involves translating the reaction mechanism into an

1242

DOUGHERTY A N D RABITZ

ODE s o l v e r t a k e s f i r s t s t e p .

I n i t i a l i z e integrals to 0.

S e t T 2 = t r (time t h a t s e n s i t i v i t i e s a r e d e s i r e d ) .

I n i t i a l i z e integrand I
= c u r r e n t time.

(T2)=

f,(tT),

j=l,

...,m'

I n t e r p o l a t e ( i f n e c e s s a r y ) t o g e t Y(T ) Compute J a 2 ( ~ l j=1,. . ,m' )

L .
Loop o v e r r a t e c o n s t a n t

+---

integration

Add r e s u l t of t h i s p i e c e t o o l d v a l u e o f i n t e g r a l . S t o r e new i n t e g r a l .

.~

ODE s o l v e r t a k e s next s t e p S t e p i s chosen s o t h a t

and normalized

'tr-l

) t i m e . r e a c h e d does n o t g e t
s m a l l e r than tr-l. Increment a c o u n t e r a t each s t e p taken.

sensitivity coefficients a t t r f o r m ' r a t e cons t a n t s and m" i n i t i a l species concentrations. Use r e c u r s i o n formula

Figure 2. Flow chart to illustrate computation of sensitivity coefficients in the ) course of solving k t ( T , t r ) , T c(t,-l, h .

appropriate code to compute concentrations and sensitivities. For small problems the algebra involved is trivial and can be performed by the modeler. Otherwise an automated precompiler [21] should be utilized. An efficient method must be used to construct the algebraic formulas involving y necessary for evaluating the species derivatives on the right-hand side of eq. (I). For treating both eqs. (1)and (7), an analytical expression for the Jacobian matrix [see eq. (3)] is desirable. Finally, it is desirable to construct the fa, array analytically as well. This involves simple differ-

SENSITIVITY AND ANALYSIS

1243

entiation, yet can become tedious for larger systems, so an automated procedure is clearly useful. The third step is to carry out test calculations of y(t) and a few sensitivities y , ( t ) to become familiar with the system of interest. Test calculations are used to assign proper stepsizes, tolerances, the interpolation grid of y, etc. The times where concentrations are changing rapidly, the times where sensitivity information is most desirable, and the behavior of a few sensitivity integrands should particularly be checked. From the results of these preliminary calculations, the quadrature scheme, the interpolation procedure, and tolerances can be chosen for an efficient and accurate study of the kinetics. A few additional programming hints are also useful. It was found to be most convenient to divide the program into several modular units. First there is a solver package for treating both stiff and nonstiff ODES. For sensitivity analysis the authors recommend a package written by Hindmarsch [22]which is a revision of an earlier code by Gear [23]. Besides the solver package, it is convenient to have additional program modules, each of which carries out a special task for sensitivity analysis. We have found it useful to have separate modules to read in the rate constants and other parameters, set up the mechanism and rate constant arrays, perform the numerical integration, carry out interpolation, and evaluate the arrays f, J, and fB,. Computational times for both linear and nonlinear kinetic systems have been determined in previous studies [9,10] and comparisons with other methods of sensitivity analysis [3,8] indicate the GFM to be a practical approach. In ref. 10 concentrations and sensitivities at a single time were determined both by the GFM and the direct method [3-51 for a system with 13 species and 24 parameters. The direct method, which solves eqs. (1) and (2) together for each parameter, required 2.8 min of execution time. The GFM required only 24 sec. While the direct method and the GFM calculate exactly the same time of sensitivity information (i.e., the sensitivity coefficients dy,ldcr,), the Fourier amplitude sensitivity test (FAST) [6-81 computes a slightly different measure of sensitivity. Nevertheless, this earlier study showed that all three approaches predict essentially the same set of important rate constants. The computational time required to obtain the same sensitivities by the FAST approach was estimated to be 1hr. These estimates are for double-precision calculations on an IBM 360/91 computer. In general the computation time required for the GFM should be competitive with that of the DM provided that m , the number of input parameters, is equal to or greater than n , the number of species. (For most kinetic models this is indeed the case.) The reason for the difference in computational effort is that the direct method requires solution of a set of 2tz [see eqs. (1) and (2)] coupled equations for each of the m parameters;

1244

DOUGHERTY AND RABIrZ

whereas the GFM requires only solutions of n 1sets of n coupled equations plus some extra time for numerical integration. Actually, practically all the effort for the GFM is due to solving the relevant ODES. Numerical quadrature is not very time-consuming, as evidenced by the fact that the required execution time is generally independent of the number of parameters present. A useful rule of thumb is that the Greens-function approach for determining all relevant sensitivities at a single time requires about 1.1n times the labor for a single kinetic calculation. Each additional time at which sensitivities are computed requires a small fractional increase in computational effort mainly because of additional operations by the ODE solver to start-up the solution of a new initial-value problem. Although the amount of start-up time is problem-dependent, a typical increase is about 0.1 n times additional effort for each new time considered. Generally, computation of sensitivities at three or four times is sufficient. The GFM can be very inexpensive in cases where sensitivities for only a single species at a single time are of interest. Then only two sets of n-coupled equations-one from the concentrations and one for the proper row of the adjoint Greens function matrix-must be solved; the effort may even be less than that required for two kinetic calculations! The following empirical formula is useful for estimating the amount of storage required to carry out a sensitivity calculation for a particular system: (12) STORAGE(ki1obytes) = 95

+ 2.4n + 0.016n2 + 0.024mn + 0.08m

The formula above refers to the FORTRAN program we have developed, which includes the ODE solver package. Special overlaying procedures or out-of-core matrix handling techniques could be used to reduce this storage.

Applications of Sensitivity Information

Sensitivity information can be utilized in a variety of different ways to understand system behavior. The sensitivity results should be useful not only in kinetic modeling but in analyzing results of kinetic experiments. Sensitivity information can be applied to study and improve the kinetic model, to elucidate mechanistic details, to estimate system behavior at different values of the parameters, to optimize species profiles, and to determine relevant sensitivities for the design of kinetics experiments. We now discuss each of these applications briefly. The reader is referred to another paper [lo] for more extensive discussions. First, by computing the normalized sensitivity coefficients (i.e., b 1ny;lb In aj) the relative importance of the parameters in the model can be assessed. Rate constants are often not accurately known and identification of the rate constant exhibiting the largest sensitivities aids in determining

SENSITIVITY AND ANALYSIS

1245

which ones require additional experimental study. If there is a large sensitivity to a rate constant whose value is in error, the results of the model itself may be suspect. On the other hand, if very small normalized sensitivities are observed for a particular rate constant parameter, its uncertainty does not really matter. Thus sensitivity analysis may be useful in determining the important reactions in the model, and the unimportant ones may be eliminated. In identifying the important parameters, sensitivity analysis is also useful in elucidating mechanistic behavior. Often, the mechanism for a particular process is difficult to obtain. Tedious experimentation must be carried out to confirm or deny certain mechanistic details. Rate constants exhibiting the largest normalized sensitivities are inevitably the rate constants for the most important elementary reaction steps in the mechanism. Sensitivity analysis is a useful tool for studying complicated reaction mechanisms. In a recent study of a 57-reaction model for the H2-02 system [24],the thirteen elementary reactions exhibiting the largest normalized sensitivities for most species of a particularly well-studied pressure temperature regime of the system are virtually identical to the elementary reactions of a mechanism [25] deduced by much more tedious analysis. Knowledge of sensitivities to initial species concentrations is useful in identifying which reactants are most critical in determining species concentrations at later times. In the design of a particular chemical process involving complicated reactions, such knowledge is useful for optimization and control of important species profiles. Sensitivity information can be used to estimate system behavior when a parameter value is changed. In a previous paper [lo] we have developed an exponential formula which estimates such behavior quite well. For one species y and one parameter a, the formula for the estimated concentration at the new value a A is

The success of eq. (13) (and its generalization to m parameters) in predicting the concentrations at new parameter values likely reflects the remnant of exponential behavior which is ubiquitous in chemical kinetics. Equation (13) is useful for obtaining a quick estimate of the effect on a given species concentration when a new value of a particular parameter has been obtained. The elementary sensitivity coefficients described up to this point, d y J d a j , can also be employed to obtain so-called derived sensitivity coefficients, which are useful in various experimentallmodeling situations. For example, derived sensitivities are applicable when it is more appropriate to consider a rate constant hi as an output quantity and a species concentration y~ as an input parameter in the model. This is the case when a rate

1246

DOUGHERTY AND RABITZ

constant is determined from an experiment and a particular concentration profile is monitored. This situation can be generalized to several of the rate constants and concentrations [lo], although the total number of inputs and outputs (i.e., independent and dependent variables) must remain the same. The relevant derived sensitivity coefficients are of the form (bk,/b k J ) t l , + or (dhl/dYl)k,,k, in contrast to the elementary sensitivity ( b y [ / d h J ) k , , k , Techniques from multivariable calculus may be used to evaluate these derived sensitivities from the usual set of elementary sensitivity coefficients. No additional differential equations need to be treated; only a set of simple linear algebraic equations has to be solved [lo]. Derived sensitivities of the type (dk,ldk, ) j L , k rare useful in investigating the correlation of h, with k, These correlations can aid in assigning realistic error bounds on the experimental result for k,. If such correlations are very high, the results are suspect. Finally, derived sensitivities of the type (bkJ dy[)k,,+ may be useful in planning experimental studies. Their magnitudes serve as a guide in deciding which species profiles to monitor in order to determine an unknown rate constant. The reader is referred elsewhere [10,24,26] for concrete applications of these sensitivities to particular experimentallmodeling situations.

Concluding Remarks
In this paper we have described an algorithm designed to implement the Greens function method for sensitivity analysis of reaction mechanisms. The theory of the Greens function approach, relevant computational procedures, and various applications of sensitivity information were discussed. Although the computational algorithm described here is efficient even for complicated kinetic analyses, it is likely that improvements can be made [27]. In this section four possible future areas for improvements are suggested. One suggestion concerns the methods used for numerical integration. In the current version of the code, numerical integration is done by means of the trapezoid rule or the exponential formula [eq. (lo)]. By improved quadrature procedures or fuller utilization of the potentially valuable information easily available from the differential equation solver, it may be possible to speed up the integration. A second possible area of improvement concerns optional programming to reduce the storage required. Special overlaying procedures, out-of-core matrix handling techniques. and/or proper use of special buffers or disks may be implemented to reduce storage. This is clearly important for large and more realistic kinetic models. A third suggestion for possible improvement concerns reduction in start-up time for the differential equation solver. The choice of a particular ODE solver is an important question for the kinetics alone. For the Greens function approach, special consideration should be given to the amount of start-up time required by a particular

SENSITIVITY A N D ANALYSIS

1247

solver, especially if sensitivities are desired at a large number of times. A final suggestion for efficient computation of sensitivity information deals with the manner of performing the operations described in the second section. For example, since the same Jacobian occurs for each row of the adjoint Greens function matrix, it may be more efficient to save its matrix elements at various times instead of saving y for interpolation purposes. Careful timing studies will probably be useful to determine the optimal arrangement of computational procedures. These suggestions have not been implemented at this time, but they arose as natural possible areas of improvement from evaluating the present code. Although coding or certain numerical improvements can likely still be made, the present procedure outlined in this paper is now practical and reasonably efficient. We hope this paper will stimulate further applications as well as basic development of the GFM in chemical kinetics.

Acknowledgments
The authors are grateful to the Department of Energy and the Exxon Corporation for support of this work, and to Dr. Dave Edelson and Dr. Charles Westbrook for helpful discussions. One of us (E.P.D.) would like to thank Doren Indritz for help with the computer codes and Jenn-Tai Hwang for several helpful suggestions.

Bibliography
[l] D. Edelson, J . Chem. Ed., 52,642 (1975). [2] J. Phys. Chem., 81(25) (1977) (this issue contains many articles on numerical modeling in chemical kinetics). [3] Y. Bard, Nonlinear Parameter Estimation, Academic Press, New York, 1974. 141 R. W. Atherton, R. B. Schainker, and E. R. Ducot, AZChE J., 21,441 (1975). [5] R. P. Dickinson and R. J. Gelinas, J . Cornp. Phys., 21,123 (1976). [S] R. I. Cukier, C. M. Fortuin, K. E. Shuler, A. G . Petschek, and J. H. Schaibly, J . Chrrn. Phys., 59,3873 (1973). [7] M. Koda, G. J. McCrae, and J. H. Seinfeld, Znt. J . Chem Kinet., 9,427 (1979). [8] A. A. Boni and R. C. Penner, Combust. Sci. Technol., 15,99 (1976). [9] J.-T. Hwang, E. P. Dougherty, S. Rabitz, and H. Rabitz, J . Chem. Phys., 69, 5180 (1978). [lo] E. P. Dougherty, J.-T. Hwang, and H. Rabitz, J . Chem. Phys., 7, 1794 (1979). [ll] J.-T. Hwang and H. Rabitz, J . Chem. Phys., 70, 4609, (1979). [In] R. Tomovic and M. Vukobratovic, General Stability Theory, Academic Press, New York, 1978. [13] P. M. Frank, Introduction to System Sensitivity Theory, Academic Press, New York, 1978. [14] D. H. Meadows et al., The Limits to Growth, Potomac AssociatesKJniverse Books, New York, 1972. 1151 P. J. Vermeulen and D. C. H. de Jongh, Appl. Math. Modeling, 1,29 (1976). 1161 A. A. Frost and R. G. Pearson, Kinetics and Mechanism, Wiley, New York, 1961. [17] C. F. Curtiss and J. 0. Hirschfelder, Proc. Natl. Acad. Sci., U.S.A., 38,235 (1952).

1248

DOUGHERTY AND RABITZ

[18] C. W. Gear, Numerical Initial Value Problems in Ordinary Differential Equations, Prentice-Hall, Englewood Cliffs, N.J., 1971, Chap. 11. [19] D. D. Wagner, J . Phys. Chem., 81,2329 (1977); see also references therein. [20] S. W. Benson, Proceedings of the Symposium on Chemical Kinetics Data for the Upper and Lower Atmosphere,Znt. J. Chem. Kinet., Suppl. 1,7,359 (1975). [21] D. Edelson, Cornput. Chem., 1,29 (1976). [22] A. C. Hindmarsch, Gear: Ordinary Differential Equation System Solver, UCID-30001, Rev. 3, Lawrence Livermore Laboratory, Livermore, CA, December 1974. [23] C. W. Gear, ACM Commun., 14, 176 (1971). [24] E. P. Dougherty and H. Rabitz, J . Chem. Phys., to appear. [25] R. R. Baldwin, M. E. Fuller, J. S. Hillman, D. Jackson, and R. W. Walker, J. Chem. SOC. Faraday Trans. 1, 4,635 (1974). [26] M. W. Slack, Combust. Flame, 28,241 (1977). [27] D. Edelson, Private communication.

Received June 27,1979 Accepted August 6,1979