30 PRACTICAL NUMERICAL METHODS

Chapter 9 Regression Practice Problems

Use Excel and VBA to solve the following problems. Document your solutions using the Expert Problem
Solving steps outlined in Table 1.2.

1. The following data for the vapor-liquid equilibrium mole fractions of methanol in a binary methanol-
water mixture were obtained at atmospheric pressure. Use Excel’s Data Analysis/Regression to fit the
data to 1st, 2nd, 3rd, and 4th order polynomials and generate residual plots. Set the intercept to 0 (why?)
Which polynomial do you recommend for a function to relate the mass fraction of methanol in the
vapor (y) to the mass fraction of methanol in the liquid (x)? Why?

X Y
1 1
0.882 0.929
0.765 0.849
0.653 0.764
0.545 0.673
0.443 0.575
0.344 0.471
0.25 0.359
0.159 0.241
0.072 0.114
0 0

Answer: Y  1.64 X  0.871X 2  0.231X 3

2. The following data are expected to follow a functional relationship y  ax b . Obtain the values of a
and b from a log-log plot.

x 1.21 1.35 2.40 2.75 4.50 5.1 7.1 8.1

y 1.2 1.82 5.0 8.80 19.5 32.5 55.0 80.0

3. Experimental data were obtained from a heat transfer experiment for the Nusselt (Nu), Prandtl (Pr),
and Reynolds (Re) numbers in the form y  Nu Pr 0.4 and x = Re. Plot y versus x using log-log
scales and determine the relationship between Nu, Pr, and Re numbers.

x y
675.1 171,900
368.7 100,600
549.6 142,500
424.7 79,430
187.8 34,910
175.0 35,000
120.7 19,990
593.9 172,100
361.8 97,600

4. The vapor pressure of water over the temperature rage 60 to 150oC are given in the following table.
Use the Solver to fit the data to the Antoine equation:

B
log  P   A 
C T
PRACTICE PROBLEMS 31

Compare your results to published values: A = 7.96681, B = 1668.21, C = 228.000.

Water vapor pressure data.

T/oC 60 65 70 80 90 92 94 96 98 100 102 104 106 108 110 150
P/mm Hg 149.4 187.5 233.7 355.1 525.8 567 610.9 657.6 707.3 760 815.9 875.1 937.9 1004.4 1074.6 3570.4

Answer: A=7.929, B=1643, C = 225.6

5. Use the set of pressure-volume data below to find the values for the coefficients, A1 and A2 for the
best fit of the data to the equation shown below. T = 298 K, Rg = 0.0821 L-atm/gmol-K.
PV A A
 1  1  22
Rg T V V
P(atm) 0.969 1.09 1.341 1.606
V(L) 25 22.2 18 15

Answer: A1 = -0.2736, A2 = 0.6364

6. Determine the best values of the constants a, b, and c that fit the following data to the function
Q  aS b R c

Q R S
1.91 0.5 1.5 x 10-3
3.48 0.5 5.0 x 10-3
4.67 0.5 9.0 x 10-3
3.1 1 1.5 x 10-3
6.66 1 5.0 x 10-3
7.59 1 9.0 x 10-3
4.11 1.5 1.5 x 10-3
7.51 1.5 5.0 x 10-3
10.08 1.5 9.0 x 10-3
5.03 2 1.5 x 10-3
9.19 2 5.0 x 10-3
12.33 2 9.0 x 10-3

Answer: a = 78.66, b = 0.4927, c = 0.6740

7. The thermal conductivity of iron (k) is found to vary with temperature (T) as follows. Find the best
constants, a and b, for a relationship of the form Tka = b.

T/K 200 600 1000 1400

k/(W/cm-K) 1.0 0.4 0.3 0.25

Answer: a = 1.374, b = 192.0

8. Fit the data to the following function using Excel and least-squares to determine the values for a and b.

y  exp ax 2  b 
x 1 1.1 1.2 1.3 1.4
y 0.14 0.21 0.33 0.54 0.92

Answer: a =1.96896, b = -3.94283

32 PRACTICAL NUMERICAL METHODS

9. The following adsorption data may be correlated by the following equation. Find the constants that
best fit the data. Does this equation do a good job of representing the behavior? Explain.
bp
q find K and b
1  Kp

p q
0 0
5.00x10-4 14
1.00x10-3 22
2.00x10-3 34
5.00x10-3 68
1.00x10-2 88

10. Fit the following data for heat capacity of Carbon to the function to determine the best values of a, b,
and c. Plot your function with the data for comparison. How good is the fit?
c
c p  a  bT 
T2

T/K  273  373  473  573  673  773  873  973  1073  1173  1273  1373 
Cp/(J/mol∙K)  7.87  11.13  14.54  16.9  17.3  19.23  19.56  21.76  23.5  23.96  24.88  26.24 

11. Fit the data in the table to the following two equations to get values for a, b, c, and d for each equa-
tion. Which equation is a better fit? Why? Is either equation a good model of the data? Provide evi-
dence to support your answers.

y  a  bx  cx 2  dx 3
c
y  a  bx   dx 2
x

x  0.1  0.2  0.3  0.4 0.5 0.6 0.7 0.8 0.9 1 

y  11.13  6.31  4.82  3.96 3.77 3.53 3.65 3.78 3.87 4.08 

12. Fit the following function to the  T data to determine the values of the parameters a, b, c, and d.
Validate the regression. Show your work.

a  bT

1  cT  dT 2

T/K 273 293 313 333 353 373 393 413 433 453 473
3
/(kmol/m ) 17.5 17.1 16.7 16.3 15.9 15.5 15.0 14.3 13.7 12.9 12.0

13. Use the method of cubic splines to interpolate the following reaction t C data. Use your interpolating
function to determine the time when the concentration reaches a maximum value. Report the concen-
tration at the maximum time. Show your work.

t/min  0  1 2 3 4  5  6 7 8 9 10
3
C/(kmol/m )  1  2 4 8 10 10 8 5 3 2 1
PRACTICE PROBLEMS 33

14. For the following data:

x 1 2 3 4 6 8 10
y 2 2.5 7 10.5 12.75 13 13

a) Recommend a polynomial to fit the data in Table 2. Plot the recommended polynomial with the
data (y on vertical axis, x on horizontal axis). Explain your choice. Label your axis on the plot.
b) Fit the following equation to the data in Table 2. Show the values for a, b, c, p, and q. Plot the
equation with the data with axis labels.
a  bx  cx 2
y
1  px  qx 2
c) Which model from parts (a) and (b) do you recommend and why? Provide evidence to support
your recommendation.
15. Create a macro that generates a normal residual versus z-score plot using Filiben’s criteria.
16. An orifice meter generates the following data. Is the velocity proportional to the square root of the
pressure drop through the orifice?

u/(ft/s) 1 2 3 4 5 6 7 8 9 10
P/(in Hg) 5 8 10 12 14 16 18 19 23 24

17. The relationship between the pressure (P) and volume (V) of the air in a cylinder during the upstroke
of a piston in an air compressor can be expressed as PV k  C , where k and C are constants. Using
the following data determine the best values for k and C:

P/(mm Hg) 760 1140 1520 2280 3040 3800

V (mL) 48.3 37.4 31.3 24.1 20.0 17.4

18. A model for the effect of impurity on crystal growth, G, is

G  GL 1

G0  G KC m
where C (ppm) is the impurity concentration, GL= 1.8x10-3 mm/min is the limiting growth rate, G0 =
3.0x10-3 mm/min is the growth rate when no impurity is present, and K and m are model parameters.
Growth rates were measured for several impurity concentrations with the following results. Deter-
mine K and m using linearized and nonlinear regression methods. Which one gives more accurate re-
sults and why?

C/ppm 50 75 100 125 150

3
G x 10 /(mm/min) 2.50 2.20 2.04 1.95 1.90

19. A quantity p varies with another quantity q according to the following formula:
1
p2 
aq  b
Using the following data, determine the best values of a and b.

q 1 2 3 5
p 0.95 0.59 0.43 0.35
34 PRACTICAL NUMERICAL METHODS

20. The vapor pressure of water absorbed on silica gel, Pgel, can be expressed as a function of the vapor
pressure of pure water, P0, for various gel loadings:
Pgel  aP0b

Determine the values of a and b using the following data and compare them with the values obtained
from the linearized form of the equation.

Pgel/(in Hg) 0.038 0.08 0.174 0.448 1.43 5.13 9.47

P0/(in Hg) 0.2 0.4 0.8 2.0 6.0 20 35

21. A tank contains 100 gallons of brine in which 50 lb of salt are dissolved. Brine containing 1 lb/gal of
salt enters the tank at a rate of 2 gal/min and the mixture leaves the tank at the same volumetric rate.
What is the theoretical equation for the mass of salt S lb in the tank at time t min? Measurements
yield the following data:

t/min 10 20 30 40 50 60
S/lb 59 66 73 78 82 85

Fit the data to the theoretical equation and compare the results.
22. The rate of diffusion through a semipermeable membrane is determined by the diffusivity D (cm2/s)
of a gas. D varies with temperature according to the Arrhenius equation:
 E 
D  D0 exp   
 RT 
where R is the gas constant (1.987 cal/mol K), E is the activation energy, and D0 is the preexponential
factor. Plot the following data for diffusivities of SO2 in a fluorosilicone rubber membrane in a semi-
log plot and determine D0 and E. Compare the results using nonlinear regression.

T/K 347.0 374.2 396.2 420.7 447.7 471.2

D/(cm2/s) x 106 1.34 2.59 4.55 8.52 14.07 19.99

23. A chemical reaction A→B is carried out in a batch reactor. A proposed mechanism predicts that the
concentration of A, CA (mol/L) at any time t (min), should be related by the expression
 C  C Ae 
ln  A    kt
 C A0  C Ae 
where CA0 is the initial concentration and CAe is the equilibrium concentration, and k is the reaction
rate constant. Plot the following data in a semi-log plot and determine if the data support the predict-
ed mechanism. If so, determine k. Compare with the value of k determined from nonlinear regres-
sion.

t/min 0 36 65 100 160 Equilibrium

CA/(mol/L) 0.1823 0.1453 0.1216 0.1025 0.0795 0.0495

24. Magnetic taconite is separated mechanically from a crushed ore slurry and rolled into marble sized
balls before charging into a pelletizing furnace. After burning in the furnace, the pellets must have a
crushing strength of 8 to 10 lb. A binder is added ruing the balling process to increase the strength of
the pellets. From the following data, propose an equation for the effect of binder concentration on
strength.

Pellet Strength/lb 3.6 9.8 14.7 16.2 16.0 15.5

Binder Content/(lb/ton of taconite) 0.00 4.0 8.0 12.0 16.0 20.0
PRACTICE PROBLEMS 35

25. The irritant factor Y of polluted air can be modeled as a linear function of the concentrations of SO2
and NO2 in the atmosphere. The following data are available where X1 is the concentration of NO2 in
parts per 107 parts of air, and X2 is the concentration of SO2 in parts per 108 parts of air. Determine
the irritant factor as a function of X1 and X2.

Y 65 72 82 95 110 122 125 130

X1 10 12 15 16 19 21 25 28
X2 12.5 15 18 21 26 30 35 40

26. The following data were collected in a batch reactor. Use 2nd order finite difference derivative ap-
proximations to plot –dC/dt versus C for each value of C and determine the value of k and n.
dC
 kC n 
dt
t/min 1 2 3 4 5 6 7 8 9 10
C/M 6.84 5.05 3.91 3.14 2.58 2.17 1.85 1.60 1.40 1.23
Answer: k=0.11, n=1.57
27. The flow rate (Q) in a pipe is measured as a function of the pressure drop (ΔP) and diameter (D). The
data from experiments for Q are given in the following table.

ΔP D Q ΔP D Q ΔP D Q ΔP D Q
0.5 0.3 0.13 0.5 0.5 0.43 0.5 1 2.1 0.5 1.4 4.55
0.9 0.3 0.25 0.9 0.5 0.81 0.9 1 4.0 0.9 1.4 8.69
1.2 0.3 0.34 1.2 0.5 1.12 1.2 1 5.5 1.2 1.4 11.92
1.8 0.3 0.54 1.8 0.5 1.74 1.8 1 8.59 1.8 1.4 18.63
Use the method of nonlinear least-squares in Excel to obtain a best fit for the flow rate as a function of
the two independent variables D and ΔP by the following equation. Report the values of a, b, and c.
Q  aD b P c
Answer: a = 1.1, b = 2.3, c = 4.5
28. The temperature of a furnace wall varies with time due to changes in the outside temperature
throughout the day. Measurements for temperature (T) at various times (t) after midnight are listed in
the following table.

t/(hr) 2 3 5 8 10 15 18 22 24
T/C 86.5 97.7 104.0 101.7 92.5 62.3 55.0 67.5 80.0

Use the method of linear least-squares in Excel to obtain the best fit of the following model to the data.
Report the values of A, B, and C.
 t    t 
T  A  B cos    C sin  
 12   12 
Answers: A = 79.56, B = -0.9495, C = 24.52

29. Use multiple linear regression in Excel to fit the following data with the equation: y=a0+a1x1+a2x2.
a. Report the values of a0, a1 and a2 and their expanded uncertainties a the 95% confidence
level.
36 PRACTICAL NUMERICAL METHODS

b. Show the goodness of the fit with residual plots and comment on the quality of the mod-
el.
x1  x2  y 
0  0  0.98 
0.1  0.2  1.76 
0.2  0.4  2.59 
0.3  0.6  3.35 
0.4  0.8  4.15 
0.5  1  4.99 
0.6  0.2  2.78 
0.7  0.4  3.63 
0.8  0.6  4.38 
0.9  0.8  5.21 
1  1  6.03 
Answers: a0 = 0.964±0.033, a1 = 2.069±0.064, a2 = 2.978±0.064
30. The following equation and data represent the velocity of a particle settling in a fluid over time. Fit
the data to the equation, where v0 is the terminal velocity at large time and t0 is the dimensionless time
constant.
a. Report the values and expanded uncertainty for v0 and t0.
b. Show the goodness of the fit by plotting the model equation with the data and a plot of the re-
siduals. Comment on the quality of the model.

  t 
v  v0 1  exp    
  t0  

t/s  0  0.4 0.8 1.2 1.6 2

v/(mm/s)  0  1.72 2.44 2.76 2.87 2.96
Answers: v0 = 2.987±0.029 mm/s, t0 = 0.469±0.017 s