CS计算机代考程序代写 AI flex Java scheme chain Excel database algorithm information retrieval Fortran matlab data structure finance compiler assembly NUMERICAL MATHEMATICS AND COMPUTING

Pentadiagonal n × n System
=
 
d1 c1 f1
a1 d2 c2 f2
e1 a2 d3 c3 f3
 e2 a3 d4 c4 f4
 … … … … …
 ei−2 ai−1 di ci fi
… … … … …
en−4 an−3 dn−2 cn−2 en−3 an−2 dn−1 en−2 an−1
x1  x2 
x3  x4    . 
xi  . 
fn−2     x n − 2  cn−1 xn−1 dn xn
  b2 
 
.
  b n − 2 
2.3 Tridiagonal and Banded Systems 107
Pentadiagonal Systems
The principles illustrated by procedure Tri can be applied to matrices that have wider bands of nonzero elements. A procedure called Penta is given here to solve the five-diagonal system:
In the pseudocode, the solution vector is placed in an n × 1 array (xi ). Also, one should not use this routine if n ≦ 4. (Why?)
b1
 b3 
b4
 .   bi 
bn−1 bn
procedure Penta(n, (ei ), (ai ), (di ), (ci ), ( fi ), (bi ), (xi )) integer i, n; real r, s, xmult
real array (ei )1:n , (ai )1:n , (di )1:n , (ci )1:n , ( fi )1:n , (bi )1:n , (xi )1:n r ← a1
s ← a2
t ← e1
fori =2ton−1
xmult ← r/di−1
di ←di −(xmult)ci−1
ci ←ci −(xmult)fi−1
bi ←bi −(xmult)bi−1 xmult ← t/di−1
r ← s − (xmult)ci−1
di+1 ← di+1 − (xmult) fi−1 bi+1 ←bi+1 −(xmult)bi−1
s ← ai+1
t ← ei end for
xmult ← r/dn−1
dn ← dn − (xmult)cn−1
xn ← (bn − (xmult)bn−1)/dn xn−1 ←(bn−1 −cn−1xn)/dn−1 fori =n−2to1
xi ←(bi − fixi+2 −cixi+1)/di end for
end procedure Penta
Penta Procedure Pseudocode
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

108 Chapter 2 Linear Systems
Symmetric Penta Key Call
To be able to solve symmetric pentadiagonal systems with the same code and with a mini- mum of storage, we have used variables r, s, and t to store temporarily some information rather than overwriting into arrays. This allows us to solve a symmetric pentadiagonal system with a procedure call of the form
call Penta(n, ( fi ), (ci ), (di ), (ci ), ( fi ), (bi ), (bi ))
This reduces the number of linear arrays from seven to four! Of course, the original data in some of these arrays may be corrupted. The computed solution may be stored in the (bi ) array. Here, we assume that all linear arrays are padded with zeros to length n in order not to exceed the array dimensions in the pseudocode.
Block Pentadiagonal Systems
Many mathematical problems involve matrices with block structures. In many cases, there are advantages in exploiting the block structure in the numerical solution. This is particularly true in solving partial differential equations numerically as in Section 12.3.
We can consider a pentadiagonal system as a block tridiagonal system
Block Pentadiagonal n × n System
…
… …
  .   . 
where
􏰆d2i−1 c2i−1 􏰇 􏰆e2i−1 c2i−1 􏰇 􏰆 f2i−1 0 Di=a d , Ai=0 e , Ci=c f
2i−1 2i 2i 2i−1
􏰇
 D1 C1
 A1 D2 C2
X1  B1 
 
  

X3  B3
A2 D3 C3
 ii
ADC
  X  =  B 
i−1 i i
… … …
X2  B2

.  .  An−2 Dn−1 Cn−1   X n − 1   B n − 1 
An−1Dn Xn Bn
􏰈 D ← D − A D−1 C
i i i−1i−1i−1
B ←B −A D−1 B (2≦i≦m) i i i−1 i−1 i−1
Forward Elimination
Back Substitution
Here, we assume that n is even, say n = 2m. If n is not even, then the system can be padded with an extra equation xn+1 = 1 so that the number of rows is even.
The algorithm for this block tridiagonal system is similar to the one for tridiagonal systems. Hence, we have the forward elimination phase
and the back substitution phase
2i
􏰈X ←D−1B nnn
X ←D−1(B−CX ) (m−1≦i≦1) i i i ii+1
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

Here, we let
D−1=1􏰆 d2i
i 􏰱 −a2i−1
−c2i−1􏰇 d2i−1
2.3
Tridiagonal and Banded Systems 109
FIGURE 2.5
Mesh points in natural order
Sample Sparse 9 × 9 System
where 􏰱 = d2i d2i−1 − a2i−1c2i−1.
Code for solving a pentadiagonal system using this block procedure is left as Computer
Exercise 2.3.21. The results from the block pentadiagonal code are the same as those from the procedure Penta, except for roundoff error. Also, this procedure can be used for symmetric pentadiagonal systems (in which the subdiagonals are the same as the superdiagonals).
In Section 12.3, we discuss two-dimensional elliptic partial differential equations. For example, the Laplace equation is defined on the unit square with a 3×3 mesh of grid points placed over the unit square region which are ordered in the natural ordering (left-to-right and up) as shown in Figure 2.5.
789
456
123
In the Laplace equation, the second-order partial derivatives are approximated by second- order centered finite difference formulas. This results in a 9 × 9 system of linear equations having a sparse coefficient matrix with this nonzero pattern:
×××  × × × ×  ×× ×  × ×
A= × ×  × ×  × ×× ××××
×××
Here, the nonzero entries in the matrix are indicated by the × symbol, and the zero entries are a blank. This matrix is block tridiagonal, and each nonzero block is either tridiagonal or diagonal. Other orderings of the mesh points result in sparse matrices with different patterns.
Summary 2.3
• In many applications, tridiagonal, pentadiagonal, and other banded systems are solved using special algorithms use Gaussian elimination without pivoting. The forward elim- ination procedure for a tridiagonal linear system A = Tridiagonal[(ai ), (di ), (ci )] is
×× ×××
××
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

110
Chapter 2
Linear Systems
di ←di −􏰋ai−1􏰌ci−1 di −1
bi ←bi −􏰋ai−1􏰌bi−1 (2≦i≦n) di −1
1.
2.
WhathappenstothetridiagonalSystem(1)ifGaussian elimination with partial pivoting is used to solve it? In general, what happens to a banded system?
Count the long arithmetic operations involved in proce- dures:
aa. Tri b. Penta
5. WhatistheappearanceofamatrixAifitselementssat- isfyaij=0when:
a. ji+1
a6. ConsiderastrictlydiagonallydominantmatrixAwhose elements satisfy ai j = 0 when i > j + 1. Does Gaussian elimination without pivoting preserve the strictly diago- nal dominance? Why or why not?
a7. Let A be a matrix of form (1) such that aici > 0 for 1≦i≦n−1.Findthegeneralformofthediagonalma- trix D = Diag(αi) with αi ≠ 0 such that D−1 AD is symmetric. What is the general form of D−1 A D?
2. Repeat the previous computer problem for procedure Penta with six arrays (ei), (ai), (di), (ci), ( fi), and (bi). Use the example that begins this chapter as one of the test cases.
The back substitution procedure is
• A strictly diagonally dominant matrix A = (ai j )n×n is one in which the magnitude of the diagonal entry is larger than the sum of the magnitudes of the off-diagonal entries in the same row, and this is true for all rows, namely,
j ≠ i
necessary because zero divisors will not be encountered.
• The forward elimination and back substitution procedures for a pentadiagonal linear system A = Pentadiagonal [(ei ), (ai ), (di ), (ci ), ( fi )] is similar to that for a tridiagonal system.
a3. Howmanystoragelocationsareneededforasystemof n linear equations if the coefficient matrix has banded structureinwhichaij =0for|i− j|≧k+1?
4. Giveanexampleofasystemoflinearequationsintridi- agonal form that cannot be solved without pivoting.
1. Rewrite procedure Tri using only four arrays, (ai ), (di ), (ci ), and (bi ), and storing the solution in the (bi ) array. Test the code with both a nonsymmetric and a symmetric tridiagonal system.
xi ← 1􏰀bi −cixi+1􏰁 (i=n−1,n−2,…,1) di
􏰄n j=1
|ai j | (1 ≦ i ≦ n)
For strictly diagonally dominant tridiagonal coefficient matrices, partial pivoting is not
|aii | >
Exercises 2.3
Computer Exercises 2.3
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

a3. Write and test a special procedure to solve the tridiagonal systeminwhichai =ci =1foralli.
a4. Use procedure Tri to solve the following system of 100 equations. Compare the numerical solution to the obvious exact solution.
2.3 Tridiagonal and Banded Systems 111 For n odd, write and test
procedureX Gauss(n,(ai),(di),(bi))
that does the forward elimination phase of Gaussian elim-
ination (without scaled partial pivoting) and
procedureX Solve(n,(ai),(di),(bi),(xi))
that does the back substitution for cross-systems of this
form.
11. Consider the n × n lower-triangular system Ax = b,
where A=(aij)andaij =0fori < j. aa. Write an algorithm (in mathematical terms) for solv- ing for x by forward substitution. b. Write procedureForward Sub(n,(ai),(bi),(xi)) which uses this algorithm. c. Determine the number of divisions, multiplications, and additions (or subtractions) in using this algorithm to solve for x. d. ShouldGaussianeliminationwithpartialpivotingbe used to solve such a system? a 12. (Normalized Tridiagonal Algorithm) Construct an al- gorithm for handling tridiagonal systems in which the normalized Gaussian elimination procedure without piv- oting is used. In this process, each pivot row is divided by the diagonal element before a multiple of the row is subtracted from the successive rows. Write the equations involved in the forward elimination phase and store the upper diagonal entries back in array (ci ) and the right- hand side entries back in array (bi ). Write the equations for the back substitution phase, storing the solution in array (bi). Code and test this procedure. What are its advantages and disadvantages? 13. For a (2n) × (2n) tridiagonal system, write and test a procedure that proceeds as follows: In the forward elim- ination phase, the routine simultaneously eliminates the elements in the subdiagonal from the top to the middle and in the superdiagonal from the bottom to the middle. In the back substitution phase, the unknowns are deter- mined two at a time from the middle outward. 14. (Continuation) Rewrite and test the procedure in the pre- ceding computer problem for a general n × n tridiagonal matrix. x1 + 0.5x2 = 1.5 􏰈0.5xi−1 + xi + 0.5xi+1 = 2.0 0.5x99 + x100 = 1.5 5. Solvethesystem (2≦i≦99) 􏰈4x1 −x2 =−20 xj−1−4xj + xj+1= 40 (2≦j≦n−1) − xn−1 + 4xn = −20 using procedure Tri with n = 100. 6. Let A be the 50 × 50 tridiagonal matrix  5 −1  −1 5−1  −15−1   ... ... ...   −1 5−1 −1 5 Consider the problem Ax = b for 50 different vectors b of the form [1,2,...,49,50]T , [2,3,...,50,1]T , [3,4,...,50,1,2]T , ... Write and test an efficient code for solving this problem. Hint: Rewrite procedure Tri. 7. RewriteandtestprocedureTrisothatitperformsGaus- sian elimination with scaled partial pivoting. Hint: Additional temporary storage arrays may be needed. 8. RewriteandtestPentasothatitdoesGaussianelimina- tion with scaled partial pivoting. Is this worthwhile? 9. UsingtheideasillustratedinPenta,writeaprocedurefor solving seven-diagonal systems. Test it on several such systems. 10. Considerthesystemofequations(n=7) da x b 17 11  d2 a6 x2  b2   d3 a5 x3  b3   d4 x4=b4  a3 d5   x 5   b 5  a2 d6 x6 b6 a1 d7 x7 b7 Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 112 Chapter 2 Linear Systems 15. Suppose procedure Tri Normal(n, (ai ), (di ), (ci ), (bi ), (xi )) performs the normalized Gaussian elimination algorithm of Computer Exercise 2.3.12 and procedure Tri 2n(n, (ai ), (di ), (ci ), (bi ), (xi )) performs the algorithm outlined in Computer Exer- cise 2.3.13. Using a timing routine on your computer, compare Tri, Tri Normal, and Tri 2n to determine which of them is fastest for the tridiagonal system 􏰂ai =i(n−i+1), ci =(i+1)(n−i−1), di =(2i+1)n−i−2i, bi =i with a large even value of n. Note: Mathematical algorithms may behave differently on parallel and vector computers. Generally speaking, parallel computations completely alter our conventional notions about what’s best or most efficient. 16. Consideraspecialbidiagonallinearsystemofthefollow- ing form (illustrated with n = 7) with nonzero diagonal elements: forsolvingabackwardtridiagonalsystemoflinearequa- tions of the form a1 d1 a2 d2 c1 a3 d3 c2  18. x1  b1 x3   b3  ... ...  an−1 dn−1 cn−1 dn cn−1 ...    . x n − 1   . b n − 1   x2   b2       .  = . xn bn using Gaussian elimination without pivoting. An upper Hessenberg matrix is of the form  a11  a21    a12 a13 ··· a1n x1 b1 a22 a23 ··· a2n x2  b2  a32 a33 ··· a3nx3=b3 ... ... .  .   .  an,n−1 ann xn bn  d1  a1 d2  a2 d3  a3  Write and test d4 a4 d 5 x1 b1  x2 b2  x3 b3    x4  =  b4  a 5   x 5   b 5  d6a6x6b6 d7 x7 b7 19. Write a procedure for solving such a system, and test it on a system having 10 or more equations. An n × n banded coefficient matrix with l subdiagonals and m superdiagonals can be stored in banded storage mode in an n × (l + m + 1) array. The matrix is stored with the row and diagonal structure preserved with almost all 0 elements unstored. If the original n×n banded matrix hadtheformshowninthefigure,thenthen×(l+m+1) array in banded storage mode would be as shown. The main diagonal would be the l + 1st column of the new array. Write and test a procedure for solving a linear sys- tem with the coefficient matrix stored in banded storage mode. m m procedure Bi Diagional(n, (ai ), (di ), (bi )) to solve the general system of order n (odd). Store the so- lution in array b, and assume that all arrays are of length n. Do not use forward elimination because the system can be solved quite easily without it. 17. Write and test procedureBackward Tri(n,(ai),(di),(ci),(bi),(xi)) Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole deemed that any suppressed content does not materially affect the overall learning experience. m m mm Lower-Band-Packed Storage mode Symmetric banded array 20. An n × n symmetric banded coefficient matrix with m subdiagonals and m superdiagonals can be stored in sym- metric banded storage mode in an n × (m + 1) array. Only the main diagonal and subdiagonals are stored so that the main diagonal is the last column in the new array. Write and test a procedure for solving a linear system with the coefficient matrix stored in symmetric banded storage mode. or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. m ømm profile values for the long wave components, and z − w are those for the short wave components. Use this system to test the Penta code using various values of α. Hint: For test systems, select a simple solution vector w = [1,−1,1,−1,...,1]T with a modest value for n, and then compute the right-hand side using matrix-vector multiplication z = (I + α4 Q)w. (Continuation, Periodic Spline Filter) The filter equa- tion for the periodic spline filter is given by the n × n system 􏰀 I + α 4 􏰛Q 􏰁 w􏰛 = 􏰛z where the matrix is 1 −4 6 ... 1 Banded array 21. Writecodeforsolvingblockpentadiagonalsystemsand test it on the systems with block submatrices. Compare the code to Penta using symmetric and nonsymmetric systems. 22. (Nonperiodic Spline Filter) The filter equation for the nonperiodic spline filter is given by the n × n system 6−4 −4 6 1−4 1 −4 1 1 −41  􏰀I + α4 Q􏰁w = z where the matrix is    ... ... ... ...    −4 6−4 1 1−21 −25−4 1 1−4 6 −4 1     −4 1 5 −2 −2 1  1 1 −4 6 −4 1 −4 6 Q=  ... ... ... ... 1 −4 6  1 −4 1 ... are used in cases of filtering closed Here the parameter α = 1/[2sin(π􏰱x/λc)] involves measurement values of the profile, dimensions, and wave- length over a sampling interval. The solution w gives the 23. 24. pseudocode to handle this system and then code and test it. Use mathematical software such as MATLAB, Maple, or Mathematica to generate a tridiagonal system and solve it. For example, use the 5 × 5 tridiagonal sys- tem A = Band Matrix(−1,2,1) with right-hand side b = [1,4,9,16,25]T . 2.3 Tridiagonal and Banded Systems 113 ø m øø BLAS-General-Band Storage mode 􏰛Q = Periodic spline filters −4 1 profiles. Making use of the symmetry, modify the Penta Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 3 Nonlinear Equations An electric power cable is suspended (at points of equal height) from two towers that are 100 meters apart. The cable is allowed to dip 10 meters in the middle. How long is the cable? x It is known that the curve assumed by a suspended cable is a catenary. When the y-axis passes through the lowest point, we can assume an equa- tion of the form y = λcosh(x/λ). Here λ is a parameter to be determined. The conditions of the problem are that y ( 50) = y ( 0) + 10. Hence, we obtain 􏰋50􏰌 λ cosh = λ + 10 λ By the methods of this chapter, the parameter is found to be λ = 126.632. After this value is substituted into the arc length formula of the catenary, the length is determined to be 102.619 meters. (See Computer Exercise 5.1.4.) 3.1 Bisection Method Introduction Cable y y (50) 10 m y (0) 250 0 50 Sample Functions 114 Let f be a real- or complex-valued function of a real or complex variable. A number r, real or complex, for which f (r) = 0 is called a root of that equation or a zero of f . For example, the function f (x) = 6x2 − 7x + 2 has 1 and 2 as zeros, as can be verified by direct substitution or by writing f in its factored 23 form: f (x) = (2x − 1)(3x − 2) For another example, the function g(x) = cos 3x − cos 7x Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. Engineering Problem 3.1 Bisection Method 115 has not only the obvious zero x = 0, but every integer multiple of π/5 and of π/2 as well, which we discover by applying the trigonometric identity cos A − cos B = 2 sin􏰜 1 (a + b)􏰝 sin􏰜 1 (b − a)􏰝 22 Consequently, we find g(x) = 2 sin(5x) sin(2x) Why is locating roots important? Frequently, the solution to a scientific problem is a number about which we have little information other than that it satisfies some equation. Since every equation can be written so that a function stands on one side and zero on the other, the desired number must be a zero of the function. Thus, if we possess an arsenal of methods for locating zeros of functions, we shall be able to solve such problems. We illustrate this claim by use of a specific engineering problem whose solution is the root of an equation. In a certain electrical circuit, the voltage V and current I are related by two equations of the form 􏰂 I = a(ebV − 1) c=dI+V in which a, b, c, and d are constants. For our purpose, these four numbers are assumed to be known. When these equations are combined by eliminating I between them, the result is a single equation: c=ad(ebV −1)+V In a concrete case, this might reduce to 12=14.3(e2V −1)+V and its solution is required. (It turns out that V ≈ 0.299 in this case.) In some problems in which a root of an equation is sought, we can perform the required calculation with a hand calculator. But how can we locate zeros of complicated functions such as these? f (x) = 3.24x8 − 2.42x7 + 10.34x6 + 11.01x2 + 47.98 g(x)=2x2 −􏰓10x+1 h(x) = cosh􏰀 x2 +1−ex􏰁+log|sinx| What is needed is a general numerical method that does not depend on special properties of our functions. Of course, continuity and differentiability are special properties, but they are common attributes of functions that are usually encountered. The sort of special property that we probably cannot easily exploit in general-purpose codes is typified by the trigonometric identity mentioned previously. Hundreds of methods are available for locating zeros of functions, and three of the most useful have been selected for study here: the bisection method, Newton’s method, and the secant method. Let f be a function that has values of opposite sign at the two ends of an interval. Suppose also that f is continuous on that interval. To fix the notation, let a < b and f (a) f (b) < 0. It then follows that f has a root in the interval (a, b). In other words, there More Samples Functions Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 116 Chapter 3 Nonlinear Equations Method Description must exist a number r that satisfies the two conditions a < r < b and f (r) = 0. How is this conclusion reached? One must recall the Intermediate-Value Theorem.∗ If x traverses an interval [a, b], then the values of f (x) completely fill out the interval between f (a) and f (b). No intermediate values can be skipped. Hence, a specific function f must take on the value zero somewhere in the interval (a, b) because f (a) and f (b) are of opposite signs. Bisection Algorithm The bisection method exploits this property of continuous functions. At each step in this algorithm, we have an interval [a, b] and the values u = f (a) and v = f (b). The numbers u and v satisfy uv < 0. Next, we construct the midpoint of the interval, c = 1 (a + b), 2 and compute w = f (c). It can happen fortuitously that f (c) = 0. If so, the objective of the algorithm has been fulfilled. In the usual case, w ≠ 0, and either wu < 0 or wv < 0. (Why?) If wu < 0, we can be sure that a root of f exists in the interval [a, c]. Consequently, westorethevalueofcinbandwinv.Ifwu > 0,thenwecannotbesurethat f hasarootin
[a,c],butsincewv < 0, f musthavearootin[c,b].Inthiscase,westorethevalueofcina and w in u. In either case, the situation at the end of this step is just like that at the beginning except that the final interval is half as large as the initial interval. This step can now be repeated until the interval is satisfactorily small, say |b − a| < 1 × 10−6. At the end, the 2 bestestimateoftherootwouldbe(a+b)/2,where[a,b]isthelastintervalintheprocedure. Pseudocode Now let’s construct pseudocode to carry out this procedure. We shall not try to create a piece of high-quality software with many “bells and whistles,” but we write the pseudocode in the form of a procedure for general use. This allows the reader an opportunity to review how a main program and one or more procedures can be connected. As a general rule, in programming routines to locate the roots of arbitrary functions, unnecessary evaluations of the function should be avoided because a given function may be costly to evaluate in terms of computer time. Thus, any value of the function that may be needed later should be stored rather than recomputed. A careless programming of the bisection method might violate this principle. The procedure to be constructed operates on an arbitrary function f . An interval [a, b] is also specified, and the number of steps to be taken, nmax, is given. Pseudocode to perform nmax steps of the bisection algorithm follows: procedure Bisection( f, a, b, nmax, ε) integer n, nmax; real a, b, c, fa, fb, fc, error fa← f(a) fb← f(b) if sign(fa) = sign(fb) then output a, b, fa, fb output “function has same signs at a and b” return (Continued) ∗Intermediate-Value Theorem: If the function f is continuous on the closed interval [a, b], and if f(a)≦y≦ f(b)or f(b)≦y≦ f(a),thenthereexistsapointcsuchthata≦c≦band f(c)=y. Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. Bisection Pseudocode Find a Root for Each Function f and g Here many modifications are incorporated to enhance the pseudocode. For example, we use fa, fb, fc as mnemonics for u, v, w, respectively. Also, we illustrate some techniques of structured programming and some other alternatives, such as a test for convergence. For example, if u, v, or w is close to zero, then uv or wu may underflow. Similarly, an overflow situation may arise. A test involving the intrinsic function sign could be used to avoid these difficulties, such as a test that determines whether sign(u) ≠ sign(v). Here, the iterations terminate if they exceed nmax or if the error bound (discussed later in this section) is less than ε. The reader should trace the steps in the routine to see that it does what is claimed! Numerical Examples Now we want to illustrate how the bisection pseudocode can be used. Suppose that we have two functions, and for each, we seek a zero in a specified interval: f(x)=x3−3x+1, on[0,1] g(x)=x3−2sinx, on[0.5,2] First, we write two procedure functions to compute f (x) and g(x). Then we input the initial intervals and the number of steps to be performed in a main program. Since this is a rather simple example, this information can be assigned directly in the main program or by way of statements in the subprograms rather than being read into the program. Also, depending on the computer language being used, an external or interface statement is needed to tell the compiler that the parameter f in the bisection procedure is not an ordinary variable with numerical values, but the name of a function procedure defined externally to the main program. In this example, there are two function procedures and two calls to the bisection procedure. Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 3.1 Bisection Method 117 end if error ← b − a for n = 0 to nmax error ← error/2 c ← a + error fc← f(c) output n, c, fc, error if |error| < ε then output “convergence” return end if if sign(fa) ≠ sign(fc) then b←c fb ← fc else a←c fa ← fc end if end for end procedure Bisection 118 Chapter 3 Nonlinear Equations f (x) Output g(x) Output 2 0.375 3 0.3125 4 0.34375 . 19 0.34729 67 20 0.34729 62 Also, the results for g(x) are as follows: n cn 0 1.25 1 0.875 2 1.0625 3 1.15625 4 1.20312 5 . 19 1.23618 27 20 1.23618 34 −7.23 × 10−2 9.30 × 10−2 9.37 × 10−3 −9.54 × 10−7 3.58 × 10−7 g(cn) 5.52 × 10−2 −0.865 −0.548 −0.285 −0.125 −4.88 × 10−6 −2.15 × 10−6 A main program follows which calls the bisection routine for each of these functions: program Test Bisection integer n, nmax ← 20 real a, b, ε ← 1 10−6 2 external function f, g a ← 0.0 b ← 1.0 call Bisection ( f, a, b, nmax, ε) a ← 0.5 b ← 2.0 call Bisection (g, a, b, nmax, ε) end program Test Bisection real function f (x) real x f ← x3 − 3x + 1 end function f real function g(x) real x g ← x 3 − 2 sin x end function g Test Bisection Pseudocode Here are the computer results with the iterative steps of the bisection method for f (x): n cn f(cn) 0 0.5 −0.375 1 0.25 0.266 error 0.5 0.25 0.125 6.25 × 10−2 3.125 × 10−2 9.54 × 10−7 4.77 × 10−7 error 0.75 0.375 0.188 9.38 × 10−2 4.69 × 10−2 1.43 × 10−6 7.15 × 10−7 Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. g(x) Output Mathematical Software To verify these results, we use sophisticated procedures in mathematical software such as MATLAB, Mathematica, or Maple to find the desired roots of f and g to be 0.34729 63553 and 1.23618 3928, respectively. Since f is a polynomial, we can use a routine for finding numerical approximations to all the zeros of a polynomial function. However, when more complicated nonpolynomial functions are involved, there is generally no systematic pro- cedure for finding all zeros. In this case, a routine can be used to search for zeros (one at a time), but we have to specify a point at which to start the search, and different starting points may result in the same or different zeros. It may be particularly troublesome to find all the zeros of a function whose behavior is unknown. Convergence Analysis Now let us investigate the accuracy with which the bisection method determines a root of a function. Suppose that f is a continuous function that takes values of opposite sign at the ends of an interval [a0,b0]. Then there is a root r in [a0,b0], and if we use the midpoint c0 = (a0 + b0)/2 as our estimate of r, we have FIGURE 3.1 Bisection method: Illustrating error upper bound Error Bound ■ Theorem1 a0 r c0 If the bisection algorithm is now applied and if the computed quantities are denoted by a0, b0, c0, a1, b1, c1, and so on, then by the same reasoning, |r − cn | ≦ bn − an (n ≧ 0) 2 Since the widths of the intervals are divided by 2 in each step, we conclude that |r − cn | ≦ b0 − a0 (1) 2n+1 To summarize, a theorem can be written as follows: If an error tolerance has been prescribed in advance, it is possible to determine the number of steps required in the bisection method. Suppose that we want |r − cn | < ε as illustrated in Figure 3.1. |r −c0|≦ b0 −a0 2 (b0 – a0)y2 |r 2 c0| 3.1 Bisection Method 119 b0 Bisection Method Theorem If the bisection algorithm is applied to a continuous function f on an interval [a, b], where f (a) f (b) < 0, then, after n steps, an approximate root will have been computed with error at most (b − a)/2n+1. Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 120 Chapter 3 Nonlinear Equations EXAMPLE 1 Solution Then it is necessary to solve the following inequality for n: b−a 2n+1 <ε By taking logarithms (with any convenient base), we obtain n > log(b − a) − log(2ε) (2) log 2
How many steps of the bisection algorithm are needed to compute a root of f to full machine single precision on a 32-bit word-length computer if a = 16 and b = 17?
The root is between the two binary numbers a = (10 000.0)2 and b = (10 001.0)2 . Thus, we already know five of the binary digits in the answer. Since we can use only 24 bits altogether, that leaves 19 bits to determine. We want the last one to be correct, so we want the error to be less than 2−19 or 2−20 (being conservative). Since a 32-bit word-length computer has a 24-bit mantissa, we can expect the answer to have an accuracy of only 2−20. From the equation above, we want
(b−a)/2n+1 <ε Since b−a = 1 and ε = 2−20, we have 1/2n+1 < 2−20. Taking reciprocals gives 2n+1 > 220, or n ≧ 20.
Alternatively, we can use Inequality (2), which in this case is
log1−log2−19
n>
log 2
Using a basic property of logarithms (log x y = y log x ), we find that n ≧ 20. In this ex- ample, each step of the algorithm determines the root with one additional binary digit of precision. ■
A sequence {xn } exhibits linear convergence to a limit x if there is a constant C in the interval [0, 1) such that
|xn+1 −x|≦C|xn −x| (n≧1) (3) If this inequality is true for all n, then
|xn+1 −x| ≦ C|xn −x|≦C2|xn−1 −x|≦ ··· ≦Cn|x1 −x| Thus, it is a consequence of linear convergence that
|xn+1 −x|≦ ACn (0≦C <1) (4) The sequence produced by the bisection method obeys Inequality (4), as we see from Inequality (1). However, the sequence need not obey Inequality (3). The bisection method is the simplest way to solve a nonlinear equation f (x) = 0. It arrives at the root by constraining the interval in which a root lies, and it eventually makes the interval quite small. Because the bisection method halves the width of the interval at each step, one can predict exactly how long it takes to find the root within any desired degree of accuracy. In the bisection method, not every guess is closer to the root than the previous guess because the bisection method does not use the nature of the function itself. Often the bisection method is used to get close to the root before switching to a faster method. (Root finding by the bisection method uses the same idea as in the binary search method taught in data structures.) Linear Convergence Error Bound (Linear Convergence) Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. False Position (Regula Falsi) Method The false position method retains the main feature of the bisection method: that a root is trapped in a sequence of intervals of decreasing size. Rather than selecting the midpoint of each interval, this method uses the point where the secant lines intersect the x-axis. a (a, f (a)) y 5 f (x) rcb Secant line x 3.1 Bisection Method 121 (b, f(b)) FIGURE 3.2 False position method In Figure 3.2, the secant line over the interval [a, b] is the chord between (a, f (a)) and (b, f (b)). The two right triangles in the figure are similar, which means that It is easy to show that c=b− f(b)􏰜 a−b f(a)− f(b) b−c c−a = f(b) −f(a) 􏰝=a− f(a)􏰜 b−a f(b)− f(a) 􏰝= af(b)−bf(a) f(b)− f(a) FP Method Description We then compute f (c) and proceed to the next step with the interval [a, c] if f (a) f (c) < 0 ortotheinterval[c,b]if f(c)f(b)<0. In the general case, the false position method starts with the interval [a0 , b0 ] contain- ing a root: f (a0) and f (b0) are of opposite signs. The false position method uses intervals [ak,bk] that contain roots in almost the same way that the bisection method does. How- ever, instead of finding the midpoint of the interval, it finds where the secant line joining (ak , f (ak )) and (bk , f (bk )) crosses the x -axis and then selects it to be the new endpoint. At the kth step, it computes ck = ak f(bk)−bk f(ak) f(bk)− f(ak) If f (ak) and f (ck) have the same sign, then set ak+1 = ck and bk+1 = bk; otherwise, set ak+1 = ak and bk+1 = ck . The process is repeated until the root is approximated sufficiently well. Modified False Position Method For some functions, the false position method may repeatedly select the same endpoint, and the process may degrade to linear convergence. There are various approaches to rectify this. Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 122 Chapter 3 Nonlinear Equations MFP Method Description For example, when the same endpoint is to be retained twice, the modified false position method uses ak f(bk)−2bk f(ak)  , if f(ak)f(bk)<0 (m)  f (bk ) − 2 f (ak ) ck =2ak f(bk)−bk f(ak), if f(ak)f(bk)>0 2f(bk)− f(ak)
So rather than selecting points on the same side of the root as the regular false position method does, the modified false position method changes the slope of the straight line so that it is closer to the root. See Figure 3.3.
(bk21, f(bk21))
ak21 5 ak r (ak, 12 f(ak))
(ak21, f(ak21))
ck(m) ck
ck21 5 bk bk21
x
(bk, f(bk)) y 5 f (x)
FIGURE 3.3
Modified false position method
The bisection method uses only the fact that f (a) f (b) < 0 for each new interval [a, b], but the false position method uses the values of f (a) and f (b). This is an example showing how to include additional information in an algorithm to build a better one. In the next section, Newton’s method uses not only the function, but also its first derivative. Some variants of the modified false position procedure have superlinear convergence, which we discuss in Section 3.3. (See, for example, Ford [1995].) Another modified false position method replaces the secant lines by straight lines with ever smaller slope until the iterate falls to the opposite side of the root. (See Conte and de Boor [1980].) Early versions of the false position method date back to a Chinese mathematical text (200 B.C.E. to 100 C.E.) and an Indian mathematical text (3 B.C.E.). Summary 3.1 • For finding a zero r of a given continuous function f in an interval [a,b], n steps of the bisection method produces a sequence of intervals [a, b] = [a0, b0], [a1, b1], [a2,b2],...,[an,bn] with each containing the desired root of the function. The mid- points of these intervals c0,c1,c2,...,cn form a sequence of approximations to the root, namely, ci = 1(ai +bi). On each interval [ai,bi], the error ei = r −ci obeys the 2 inequality and after n steps we have 1 |ei|≦ 2(bi −ai) 1 |en|≦ 2n+1(b0 −a0) Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 3.1 Bisection Method 123 • For an error tolerance ε such that |en| < ε, n steps are needed, where n satisfies the inequality log(b − a) − log 2ε n>
log 2
• Forthekthstepofthefalsepositionmethodovertheinterval[ak,bk],let ck = ak f(bk)−bk f(ak)
f(bk)− f(ak)
If f(ak)f(ck) > 0, set ak+1 = ck and bk+1 = bk; otherwise, set ak+1 = ak and
a1. 2.
3. a4. 5. 6.
7. 8.
a9. a 10. 11.
a 12.
bk+1 =ck.
Find where the graphs of y = 3x and y = ex intersect by finding roots of ex − 3x = 0 correct to four decimal digits.
Give a graphical demonstration that the equation tan x = x has infinitely many roots. Determine one root precisely and another approximately by using a graph. Hint: Use the approach of the preceding exercise.
Demonstrate graphically that the equation 50π + sin x = 100 arctan x has infinitely many solutions.
Bygraphicalmethods,locateapproximationstoallroots of the nonlinear equation ln(x + 1) + tan(2x) = 0.
Give an example of a function for which the bisection method does not converge linearly.
Drawagraphofafunctionthatisdiscontinuousyetthe bisection method converges. Repeat, getting a function for which it diverges.
ProveInequality(1).
Ifa=0.1andb=1.0,howmanystepsofthebisection
method are needed to determine the root with an error of
atmost1×10−8? 2
Find all the roots of f(x) = cosx − cos3x. Use two different methods.
(Continuation) Find the root or roots of ln[(1 + x)/ (1−x2)]=0.
If f has an inverse, then the equation f(x) = 0 can be solved by simply writing x = f −1(0). Does this re- mark eliminate the problem of finding roots of equations? Illustrate with sin x = 1/π .
How many binary digits of precision are gained in each step of the bisection method? How many steps are re- quired for each decimal digit of precision?
13. Try to devise a stopping criterion for the bisection method to guarantee that the root is determined with relative error at most ε.
14. Denote the successive intervals that arise in the bisection method by [a0, b0], [a1, b1], [a2, b2], and so on. Show that
a. a0≦a1≦a2≦ ···andb0≧b1≧b2≧ ···.
b. bn−an=2−n(b0−a0).
c. anbn + an−1bn−1 = an−1bn + anbn−1, for all n.
15. (Continuation)Cana0=a1=a2=···happen? 16. (Continuation) Let cn = (an + bn )/2. Show that
lim cn = lim an = lim bn n→∞ n→∞ n→∞
a17. (Continuation) Consider the bisection method with the initial interval [a0 , b0 ]. Show that after 10 steps with this method,
􏰣􏰣􏰣1(a10 + b10) − 1(a9 + b9)􏰣􏰣􏰣 = 2−11(b0 − a0) 22
Also, determine how many steps are required to guar- antee an approximation of a root to six decimal places (rounded).
18. (True-False)Ifthebisectionmethodgeneratesintervals [a0 , b0 ], [a1 , b1 ], and so on, which of these inequali- ties are true for the root r that is being calculated? Give proofs or counterexamples in each case.
a. |r −an|≦2|r −bn|
ab. |r−an|≦2−n−1(b0−a0)
c. |r−1(an+bn)|≦2−n−2(b0−a0) 2
ad. 0≦r−an≦2−n(b0−a0) e. |r − bn|≦ 2−n−1(b0 − a0)
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
Exercises 3.1

124 Chapter 3 Nonlinear Equations
correct number of steps to compute the root with full machine precision on a 32-bit word-length computer?
22. If the bisection method is applied with starting interval [2m , 2m +1 ], where m is a positive or negative integer, how many steps should be taken to compute the root to full machine precision on a 32-bit word-length computer?
a23. Everypolynomialofdegreenhasnzeros(countingmul- tiplicities) in the complex plane. Does every real polyno-
a􏰃
21. Ifthebisectionmethodisappliedwithstartinginterval mialhavenrealzeros?Doeseverypolynomialofinfinite
19. (True-False) Using the notation of the text, determine which of these assertions are true and which are gener- ally false:
aa. |r−cn|<|r−cn−1| c. cn≦r≦bn b. an≦r≦cn d. |r−an|≦2−n ae. |r−bn|≦2−n(b0−a0) 20. Provethat|cn−cn+1|=2−n−2(b0−a0). [a,a+1]anda=2m,wheren≧24−m0,whatisthe degree f(x)= ∞n=0anxn haveinfinitelymanyzeros? 1. Using the bisection method, determine the point of in- tersection of the curves given by y = x3 − 2x + 1 and y = x2. 2. Findarootofthefollowingequationintheinterval[0,1] by using the bisection method: 9x 4 + 18x 3 + 38x 2 − 57x+14=0. 3. Find a root of the equation tan x = x on the interval [4, 5] by using the bisection method. What happens on the interval [1, 2]? 4. Findarootoftheequation6(ex −x)=6+3x2+2x3 between −1 and +1 using the bisection method. 5. Use the bisection method to find a zero of the equation λ cosh(50/λ) = λ + 10 that begins this chapter. 6. Program the bisection method as a recursive procedure and test it on one or two of the examples in the text. 7. Usethebisectionmethodtodeterminerootsofthesefunc- tions on the intervals indicated. Process all three functions in one computer run. a 10. f (a) f (b) < 0. Then c is computed as the root of the lin- ear function that agrees with f at a and b. We retain either [a, c] or [c, b], depending on whether f (a) f (c) < 0 or f (c) f (b) < 0. Test your program on several functions. Select a routine from your program library to solve polynomial equations and use it to find the roots of the equation x8 − 36x7 + 546x6 − 4536x5 + 22449x4 − 67284x3 + 118124x2 − 109584x + 40320 = 0 The correct roots are the integers 1, 2, . . . , 8. Next, solve the same equation when the coefficient of x7 is changed to −37. Observe how a minor perturbation in the coeffi- cients can cause massive changes in the roots. Thus, the roots are unstable functions of the coefficients. (Be sure to program the exercise to allow for complex roots.) Cul- tural Note: This is a simplified version of Wilkinson’s polynomial, which is found in Computer Exercise 3.3.9. A circular metal shaft is being used to transmit power. It is known that at a certain critical angular velocity ω, any jarring of the shaft during rotation will cause the shaft to deform or buckle. This is a dangerous situation because the shaft might shatter under the increased cen- trifugal force. To find this critical velocity ω, we must first compute a number x that satisfies the equation tan x + tanh x = 0 This number is then used in a formula to obtain ω. Solve forx(x>0).
Usingbuilt-inroutinesinmathematicalsoftwaresystems such as MATLAB, Mathematica, or Maple, find the roots for f(x)=x3−3x+1on[0,1]andg(x)=x3−sinx on [0.5, 2] to more digits of accuracy than shown in the text.
􏰈 f (x) = x3 + 3x − 1
g(x) = x3 − 2 sin x h(x)=x+10−xcosh(50/x) on[120,130]
Find each root to full machine precision. Use the correct number of steps, at least approximately. Repeat using the false position method.
8. Test the three bisection routines on f (x ) = x 3 + 2×2+10x−20,witha=1andb=2.Thezerois 1.36880 8108. In programming this polynomial function, use nested multiplication. Repeat using the modified false position method.
9. Write a program to find a zero of a function f in the fol- lowing way: In each step, an interval [a, b] is given and
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole deemed that any suppressed content does not materially affect the overall learning experience.
on [0, 1] on [0.5, 2]
a11.
12.
or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
Computer Exercises 3.1

13. (Engineering Problem) Nonlinear equations occur in almost all fields of engineering. For example, suppose a given task is expressed in the form f (x) = 0 and the objective is to find values of x that satisfy this condition. It is often difficult to find an explicit solution, and an ap- proximate solution is sought with the aid of mathematical software. Find a solution of
1
17. (Board Across Hallways) In a building, two intersecting halls with widths w1 = 9 feet and w2 = 7 feet meet at an angle α = 125◦, as shown:
3.2 Newton’s Method 125
􏰵
􏰶2
1 10
−(1/2)x2
Plot the curve in the range [−3.5, 3.5] for x values and
e
[−0.5, 0.5] for y = f (x) values.
14. (CircuitProblem)AsimplecircuitwithresistanceR,ca- pacitance C in series with a battery of voltage V is given by Q = CV[1−e−T/(RC)],where Q isthechargeofthe capacitor and T is the time needed to obtain the charge. We wish to solve for the unknown C. For example, solve this exercise
f(x)= √ 2π
+
sin(πx)
􏰴􏰳
􏰗 􏰀 −0.004/(2000x)􏰁 f(x)= 10x 1−e
􏰘
Assuming a two-dimensional situation, what is the longest board that can negotiate the turn? Ignore the thickness of the board. The relationship between the an- glesθandthelengthoftheboardl=l1+l2is l1 = w1 csc(β), l2 = w2 csc(γ), β = π − α − γ and l = w1 csc(π − α − γ) + w2 csc(γ). The maximum length of the board that can make the turn is found by minimizing l as a function of γ . Taking the derivative and setting d l/d γ = 0, we obtain
w1 cot(π −α−γ ) csc(π −α−γ )−w2 cot(γ ) csc(γ ) = 0 Substitute in the known values and numerically solve the
nonlinear equation.
18. Find the rectangle of maximum area if its vertices are at (0, 0), (x, 0), (x, cos x), (0, cos x). Assume that 0 ≦ x ≦ π/2.
19. Programthefalsepositionalgorithmandtestitonsome examples such as some of the nonlinear exercises in the text or in the computer exercises. Compare your results with those given for the bisection method.
20. Program the modified false position method, test it, and compare it to the false position method when using some sample functions.
−0.00001 Hint: You may wish to magnify the vertical scale by using
Plot the curve.
y = 105 f (x).
15. (Engineering Polynomials) Equations such as
A+Bx2eCx =0 A+Bx+Cx2+Dx3+Ex4 = 0
occur in engineering problems. Using mathematical soft- ware, find one or more solutions to the following equa- tions and plot their curves:
a. 2 − x2e−0.385x = 0
b. 1−32x+160×2−256×3+128×4=0
16. (ReinforcedConcrete)Inthedesignofreinforcedcon- crete with regard to stress, one needs to solve numerically a quadratic equation such as
24147 07.2x [450 − 0.822x (225)] − 265,000,000 = 0 Find approximate values of the roots.
3.2 Newton’s Method
􏰶1
The procedure known as Newton’s method is also called the Newton-Raphson iteration. It has a more general form than the one seen here, and the more general form can be used to find roots of systems of equations. Indeed, it is one of the more important procedures in numerical analysis, and its applicability extends to differential equations and integral equations. Here it is being applied to a single equation of the form f (x) = 0. As before, we seek one or more points at which the value of the function f is zero.
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
,1 ,2

126 Chapter 3
Nonlinear Equations
Tangent Line Approach
Interpretations of Newton’s Method
In Newton’s method, it is assumed at once that the function f is differentiable. This implies that the graph of f has a definite slope at each point and hence a unique tangent line. Now let’s pursue the following simple idea. At a certain point (x0, f (x0)) on the graph of f , there is a tangent, which is a rather good approximation to the curve in the vicinity of that point. Analytically, it means that the linear function
l(x)= f′(x0)(x−x0)+ f(x0)
is close to the given function f near x0. At x0, the two functions l and f agree. We take the
zero of l as an approximation to the zero of f . The zero of l is easily found: x1=x0− f(x0)
Geometric Approach
FIGURE 3.4
Newton’s method
Taylor Series Approach
we pass to a new point x1 obtained from the preceding formula. Naturally, this process can be repeated (iterated) to produce a sequence of points:
x2 =x1 − f(x1), x3 =x2 − f(x2), andsoon. f ′(x1) f ′(x2)
Under favorable conditions, the sequence of points approaches a zero of f .
The geometry of Newton’s method is shown in Figure 3.4. The line y = l(x) is tangent
to the curve y = f (x). It intersects the x-axis at a point x1. The slope of l(x) is f ′(x0). y
f ′(x0)
Thus, starting with point x0 (which we may interpret as an approximation to the root sought),
y 5 f(x)
Tangent line y5 (x)
x
r x1 x0
There are other ways of interpreting Newton’s method. Suppose again that x0 is an
initial approximation to a root of f . We ask:
What correction h should be added to x0 to obtain the root precisely?
Obviously, we want
f (x0 + h) = 0
If f is a sufficiently well-behaved function, it has a Taylor series at x0. (See Equation (11)
in Section 1.2.) Thus, we could write
f(x0)+hf′(x0)+ 2 f′′(x0)+···=0
Determining h from this equation is, of course, not easy. Therefore, we give up the expec- tation of arriving at the true root in one step and seek only an approximation to h. This can be obtained by ignoring all but the first two terms in the series:
f (x0) + h f ′(x0) = 0
h2
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

3.2 Newton’s Method 127 The h that solves this is not the h that solves f (x0 + h) = 0, but it is the easily computed
number
Our new approximation is then
h = − f (x0) f ′(x0)
x1=x0+h=x0− f(x0) f ′(x0)
Newton’s Method Formula
EXAMPLE1
Solution
and the process can be repeated. In retrospect, we see that the Taylor series was not needed after all because we used only the first two terms. In the analysis to be given later, it is assumed that f ′′ is continuous in a neighborhood of the root. This assumption enables us to estimate the errors in the process.
If Newton’s method is described in terms of a sequence x0, x1, . . . , then the following recursive or inductive definition applies:
xn+1=xn− f(xn) f′(xn)
Naturally, the interesting question is whether
lim xn = r
n→∞
where r is the desired root.
If f(x)=x3 −x+1andx0 =1,whatarex1 andx2 intheNewtoniteration?
Fromthebasicformula,x =x −f(x)/f′(x).Nowf′(x)=3×2−1,andsof′(1)=2.
􏰀􏰁 􏰀􏰁12228
1000
Also,wefind f(1)=1.Hence,wehavex =1−1 = 1.Similarly,weobtain f 1 = 5,
f′ 1 =−1,andx2=3. ■ 24
Pseudocode
A pseudocode procedure for Newton’s method can be written as follows:
procedure Newton( f, f ′, x, nmax, ε, δ) integer n, nmax; real x, fx, fp, ε, δ external function f, f ′
fx← f(x)
output 0, x , fx
for n = 1 to nmax
fp← f′(x)
if | f p| < δ then output “small derivative” return end if d ← fx/fp x←x−d fx← f(x) (Continued) Newton Pseudocode Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 128 Chapter 3 Nonlinear Equations output n, x, fx if |d| < ε then output “convergence” return end if end for end procedure Newton Sample Problem Using the initial value of x as the starting point, we carry out a maximum of nmax iterations of Newton’s method. Procedures must be supplied for the two external functions f (x) and f ′(x). The parameters ε and δ are used to control the convergence and are related to the accuracy desired or to the machine precision available. Illustration Now we illustrate Newton’s method by locating a root of x3 + x = 2x2 + 3. We apply the methodtothefunction f(x)=x3−2x2+x−3,startingwithx0 =3.Ofcourse, f′(x)= 3x2 − 4x + 1, and these two functions should be arranged in nested form for efficiency: f(x) = ((x −2)x +1)x −3 f ′(x) = (3x − 4)x + 1 To see in greater detail the rapid convergence of Newton’s method, we use arithmetic with double the normal precision in the program and obtain the following results: n xn f(xn) 0 3.0 9.0 1 2.4375 2.04 2 2.21303 27224 73144 5 3 2.17555 49386 14368 4 4 2.17456 01006 55071 4 5 2.17455 94102 93284 1 2.56 × 10−1 6.46 × 10−3 4.48 × 10−6 1.97 × 10−12 Notice the doubling of the accuracy in f (x) (and also in x) until the maximum precision of the computer is encountered. Figure 3.5 shows a computer plot of three iterations of Newton’s method for this sample problem. y 10 8 6 4 2 y 5 f(x) FIGURE 3.5 Three steps of r Newton’s method f(x) = x3−2x2+x−3 0x 2 2.2 2.4 2.6 2.8 3 3.2 x2 x1 x0 Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. Mathematical Software Using mathematical software that allows for complex roots such as in MATLAB, Maple, or Mathematica, we find that this polynomial has a single real root, 2.17456, and a pair of complex conjugate roots, −0.0872797 ± 1.17131i . Convergence Analysis Anyone who has experimented with Newton’s method—for instance, by working some of the exercises in this section—has observed the remarkable rapidity in the convergence of the sequence to the root. This phenomenon is also noticeable in the example just given. Usually, the number of correct figures in the answer nearly doubles at each successive step! Indeed, in the preceding example, we have first 0 and then 1, 2, 3, 6, 12, 24, . . . accurate digits from each Newton iteration. Five or six steps of Newton’s method often suffice to yield full machine precision in the determination of a root. There is a theoretical basis for this dramatic performance, as we shall now see. Let the function f , whose zero we seek, possess two continuous derivatives f ′ and f′′, and let r be a zero of f. Assume further that r is a simple zero; that is, f′(r) ≠ 0. Then Newton’s method, if started sufficiently close to r , converges quadratically to r . This means that the errors in successive steps obey an inequality of the form |r − xn+1|≦ c|r − xn|2 We shall establish this fact presently, but first, an informal interpretation of the inequality may be helpful. Suppose, for simplicity, that c = 1. Suppose also that xn is an estimate of the root r that differs from it by at most one unit in the kth decimal place. This means that |r − xn | ≦ 10−k The two inequalities above imply that |r − xn+1| ≦ 10−2k In other words, xn+1 differs from r by at most one unit in the (2k)th decimal place. So xn+1 has approximately twice as many correct digits as xn! This is the doubling of significant digits alluded to previously. Error Bound Quadratic Convergence Informal Interpretation ■ Theorem1 Proof To establish the quadratic convergence of Newton’s method, let en = r − xn . The formula that defines the sequence {xn} then gives en+1 =r−xn+1 =r−xn + f(xn) =en + f(xn) = en f′(xn)+ f(xn) f ′(xn) f ′(xn) f ′(xn) 3.2 Newton’s Method 129 Newton’s Method Theorem If f, f′,and f′′ arecontinuousinaneighborhoodofarootr of f andif f′(r)≠ 0, then there is a positive δ with the following property: If the initial point in Newton’s method satisfies |r − x0| ≦ δ, then all subsequent points xn satisfy the same inequality, converge to r , and do so quadratically; that is, |r − xn+1|≦ c(δ)|r − xn|2 where c(δ) is given by Equation (2). Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 130 Chapter 3 Nonlinear Equations Error Vector Recursive Relation By Taylor’s Theorem (see Section 1.2), there exists a point ξn situated between xn and r for which 0= f(r)= f(xn +en)= f(xn)+en f′(xn)+1en2 f′′(ξn) 2 (Thesubscriptonξn emphasizesthedependenceonxn.)Thislastequationcanberearranged to read en f′(xn)+ f(xn)=−1en2 f′′(ξn) 2 and if this is used in the previous equation for en+1, the result is en+1 =−1􏰋f′′(ξn)􏰌en2 (1) define a function max | f ′′(x)| c(δ)= 1|x−r|≦δ (δ>0) (2) By virtue of this definition, we can assert that, for any two points x and ξ within distance
2 f′(xn)
This is, at least qualitatively, the sort of equation we want. Continuing the analysis, we
2 min |f′(x)| |x−r|≦δ
δ of the root r, the inequality 1|f′′(ξ)/f′(x)|≦c(δ) is true. Now select δ so small that 2
Error Bound (Quadratic Convergence)
|r − xn+1| = |en+1|≦ ρ|en|≦ |en|≦ δ If the initial point x0 is chosen within distance δ of r, then
|en|≦ ρ|en−1|≦ ρ2|en−1|≦ ··· ≦ ρn|e0| Since0<ρ<1,limn→∞ρn =0andlimn→∞en =0.Inotherwords,weobtain lim xn = r n→∞ In this process, we have |en+1| ≦ c(δ)en2 ■ Discussion of Newton’s Method In the use of Newton’s method, consideration must be given to the proper choice of a starting point. Usually, one must have some insight into the shape of the graph of the function. Sometimes a coarse graph is adequate, but in other cases, a step-by-step evaluation of the δc(δ) < 1. This is possible because as δ approaches 0, c(δ) converges to 1 | f ′′(r)/f ′(r)|, 2 and so δc(δ) converges to 0. Recall that we assumed that f ′(r) ̸= 0. Let ρ = δc(δ). In the remainder of this argument, we hold δ, c(δ), and ρ fixed with ρ < 1. Suppose now that some iterate xn lies within distance δ from the root r. We have |en|=|r−xn|≦δ and |ξn −r|≦δ Bythedefinitionofc(δ),itfollowsthat 1|f′′(ξn)|/|f′(xn)|≦c(δ).FromEquation(1),we 2 now have |en+1| = 1􏰣􏰣􏰣 f ′′(ξn)􏰣􏰣􏰣en2 ≦ c(δ)en2 ≦ δc(δ)|en| = ρ|en| 2 f′(xn) Consequently, xn+1 is also within distance δ of r because Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. y 5 f (x) r x0 x1x2 (a) Runaway x1 y 5 f (x) r x0 x 3.2 Newton’s Method 131 x FIGURE 3.6 Failure of Newton’s method due to bad starting points Poor Choices of x0 (c) Cycle Special Cases Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. r x0 5 x2 function at various points may be necessary to find a point near the root. Often several steps of the bisection method are used to obtain a suitable starting point, so that Newton’s method converges more rapidly. Although Newton’s method is truly a marvelous invention, its convergence depends upon hypotheses that are difficult to verify a priori. Some graphical examples show what can happen. In Figure 3.6a, the tangent to the graph of the function f at x0 intersects the x-axis at a point remote from the root r, and successive points in Newton’s iteration recede from r instead of converging to r. The difficulty can be ascribed to a poor choice of the initial point x0; it is not sufficiently close to r. In Figure 3.6b, the tangent to the curve is parallel to the x-axis and x1 = ±∞, or it is assigned the value of machine infinity in a computer. In Figure 3.6c, the iteration values cycle because x2 = x0. In a computer, roundoff errors or limited precision may eventually cause this situation to become unbalanced such that the iterates either spiral inward and converge or spiral outward and diverge. The analysis that establishes the quadratic convergence discloses another troublesome hypothesis; namely, f′(r) ≠ 0. If f′(r) = 0, then r is a zero of f and f′. Such a zero is termed a multiple zero of f —in this case, at least a double zero. Newton’s iteration for a multiple zero converges only linearly! Ordinarily, one would not know in advance that the zero sought was a multiple zero. If one knew that the multiplicity was m, however, Newton’s method could be accelerated by modifying the equation to read in which m is the multiplicity of the zero in question. The multiplicity of the zero r is the least m such that f (k)(r) = 0 for 0 ≦ k < m, but f (m)(r) ≠ 0. (See Exercise 3.2.35.) As is shown in Figure 3.7 (p. 132), the equation p2(x) = x2 −2x +1 = 0 has a root at 1 of multiplicity 2, and the equation p3(x) = x3 −3x2 +3x −1 = 0 has a root at 1 of multiplicity 3. It is instructive to plot these curves. Both curves are rather flat at the roots, which slows down the convergence of the regular Newton’s method. Also, the figures illustrate the curves of two nonlinear functions with multiplicities as well as their regions of uncertainty about the curves. So the computed solutions could be anywhere within the indicated intervals on the x-axis. This is an indication of the difficulty in obtaining precise solutions of nonlinear functions with multiplicities. Multiple Zero Modified Newton’s Method f′(xn) xn+1 =xn −m f(xn) y 5 f (x) (b) Flat spot x 132 Chapter 3 Nonlinear Equations FIGURE 3.7 Curves p2 and p3 with multiplicity 2and3 y 5 p2(x) y 5 p3(x) []x[]x 0202 r51 r51 (a)p2(x)5x2 22x 11 (b)p3(x)5x3 23x2 13x 21 Systems of Nonlinear Equations Some physical problems involve the solution of systems of N nonlinear equations in N unknowns. One approach is to linearize and solve, repeatedly. This is the same strategy used by Newton’s method in solving a single nonlinear equation. Not surprisingly, a natural extension of Newton’s method for nonlinear systems can be found. The topic of systems of nonlinear equations requires some familiarity with matrices and their inverses. In the general case, a system of N nonlinear equations in N unknowns xi can be displayed in the form   f ( x , x , . . . , x ) = 0 112 N f2(x1,x2,...,xN) = 0  . fN(x1,x2,...,xN) = 0 Using vector notation, we can write this system in a more elegant form: Nonlinear Systems N×N Nonlinear System 3 × 3 not computed, but rather a related system of equations is solved. We illustrate the development of this procedure using three nonlinear equations f1(x1,x2,x3) = 0 f2(x1,x2,x3) = 0 (3) f3(x1,x2,x3) = 0 by defining column vectors as F(X) = 0 F = [ f1, f2, . . . , fN ]T X = [x1,x2,...,xN]T The extension of Newton’s method for nonlinear systems is X(k+1) = X(k) − 􏰗F ′􏰀X(k)􏰁􏰘−1F􏰀X(k)􏰁 where F′􏰀X(k)􏰁 is the Jacobian matrix, which will be defined presently. It comprises partial derivatives of F evaluated at X(k) = 􏰗x(k), x(k), . . . , x(k)􏰘T . This formula is similar to 12N the previously seen version of Newton’s method, except that the derivative expression is not in the denominator but in the numerator as the inverse of a matrix. In the computational form of the formula, X(0) = 􏰗x(0), x(0), . . . , x(0)􏰘T is an initial approximation vector, taken 12N to be close to the solution of the nonlinear system, and the inverse of the Jacobian matrix is Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. Jacobian Matrix ∂f1 ∂f1 ∂x1 ∂x2 􏰀 􏰁 ∂f2 ∂f2 F′ X(0) =∂x1 ∂x2 ∂f3 ∂f3 ∂f1  ∂x3  ∂f2  ∂x3  ∂f3  3.2 Newton’s Method 133 Recall the Taylor expansion in three variables for i = 1, 2, 3: f(x +h,x +h,x +h)=f(x,x,x)+h ∂fi +h ∂fi +h ∂fi +··· (4) i 1 1 2 2 3 3 i 1 2 3 1∂x 2∂x 3∂x 123 where the partial derivatives are evaluated at the point (x1, x2, x3). Here only the linear terms in step sizes h are shown. Suppose that the vector X(0) = 􏰀x(0), x(0), x(0)􏰁T is an i 􏰗􏰘T 123 approximate solution to (3). Let H = h1, h2, h3 be a computed correction to the initial guesssothatX(0)+H=􏰗x(0)+h ,x(0)+h ,x(0)+h 􏰘T isabetterapproximatesolution. 112233 Discarding the higher-order terms in the Taylor expansion (4), we have in vector notation 0 ≈ F􏰀X(0) + H􏰁 ≈ F􏰀X(0)􏰁 + F ′􏰀X(0)􏰁H (5) where the Jacobian matrix is defined by Jacobian Linear Systems Newton’s Method for Nonlinear Systems ∂x1 ∂x2 ∂x3 Here all of the partial derivatives are evaluated at X(0); namely, ∂fi = ∂fi􏰀X(0)􏰁 ∂xj ∂xj Also, we assume that the Jacobian matrix F ′􏰀X(0)􏰁 is nonsingular, so its inverse exists. Solving for H in (5), we have H ≈ −􏰗F ′􏰀X(0)􏰁􏰘−1F􏰀X(0)􏰁 Let X(1) = X(0) + H be the better approximation after the correction; we then arrive at the first iteration of Newton’s method for nonlinear systems X(1) = X(0) − 􏰗F ′􏰀X(0)􏰁􏰘−1F􏰀X(0)􏰁 In general, Newton’s method uses this iteration: X(k+1) = X(k) − 􏰗F ′􏰀X(k)􏰁􏰘−1F􏰀X(k)􏰁 In practice, the computational form of Newton’s method does not involve inverting the Jacobian matrix but rather solves the Jacobian linear systems 􏰗F ′􏰀X(k)􏰁􏰘H(k) = −F􏰀X(k)􏰁 (6) The next iteration of Newton’s method is then X(k+1) = X(k) + H(k) (7) This is Newton’s method for nonlinear systems. The linear system (6) can be solved by procedures Gauss and Solve as discussed in Chapter 2. Small systems of order 2 can be solved easily. (See Exercise 2.2.39.) Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 134 Chapter 3 Nonlinear Equations EXAMPLE 2 Solution As an illustration, we can write a pseudocode to solve the following nonlinear system of equations using a variant of Newton’s method given by (6) and (7):  x+y+z = 3 x2+y2+z2 =5 (8) ex +xy−xz = 1 With a sharp eye, the reader immediately sees that the solution of this system is x = 0, y = 1, z = 2. But in most realistic problems, the solution is not so obvious. We wish to develop a numerical procedure for finding such a solution. Here is the pseudocode: X = 􏰗0.1, 1.2, 2.5􏰘T fork =1to10  x1+x2+x3−3  F= x12+x2+x32−5  ex1 +x1x2 −x1x3 −1 111 J =  2x1 2x2 2x3  ex1 +x2−x3 x1 −x1 solve JH = F X=X−H end for Basin of Attraction of Fractals When programmed and executed on a computer, we found that it converges to x = (0, 1, 2), but when we change to a different starting vector, (1, 0, 1), it converges to another root, (1.2244, −0.0931, 1.8687). (Why?) ■ We can use mathematical software such as in MATLAB, Maple, or Mathematica and their procedures for solving the system of nonlinear equations (8). An important application area of solving systems of nonlinear equations is used in Chapter 13 on minimization of functions. Fractal Basins of Attraction The applicability of Newton’s method for finding complex roots is one of its outstanding strengths. One need only program Newton’s method using complex arithmetic. The frontiers of numerical analysis and nonlinear dynamics overlap in some intriguing ways. Computer-generated displays with fractal patterns, such as in Figure 3.8, can easily be created with the help of the Newton iteration. The resulting pictures show intricately interwoven sets in the plane that are quite beautiful and colorful when displayed on a computer. One begins with a polynomial in the complex variable z. For example, p(z) = z4 − 1 is suitable. This polynomial has four zeros, which are the fourth roots of unity. Each of these zeros has a basin of attraction, that is, the set of all points z0 such that Newton’s iteration, started at z0, will converge to that zero. These four basins of attraction are disjoint from each other, because if the Newton iteration starting at z0 converges to one zero, then it cannot also converge to another zero. One would naturally expect each basin to be a simple set surrounding the zero in the complex plane. But they turn out to be far from simple. To Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. FIGURE 3.8 Basins of attraction see what they are, we can systematically determine, for a large number of points, which zero of p the Newton iteration converges to if started at z0. Points in each basin can be assigned different colors. The (rare) points for which the Newton iteration does not converge can be left uncolored. Computer Exercise 3.2.27 suggests how to do this. Summary 3.2 • For finding a zero of a continuous and differentiable function f , Newton’s method is given by xn+1=xn− f(xn) (n≧0) f′(xn) It requires a given initial value x0 and two function evaluations (for f and f ′) per step. • The errors are related by en+1 =−1􏰋f′′(ξn)􏰌en2 2 f′(xn) |en+1|≦ c|en|2 This means that Newton’s method has quadratic convergence behavior for x0 suffi- ciently close to the root r. • For an N × N system of nonlinear equations F(X) = 0, Newton’s method is written as X(k+1) = X(k) − 􏰗F ′􏰀X(k)􏰁􏰘−1F􏰀X(k)􏰁 (k ≧ 0) which involves the Jacobian matrix F′􏰀X(k)􏰁 = J = 􏰗􏰀∂fi􏰀X(k)􏰁/∂xj􏰁􏰘N×N. In practice, one solves the Jacobian linear system 􏰗F ′(X(k)􏰁􏰘H(k) = −F􏰀X(k)􏰁 using Gaussian elimination and then finds the next iterate from the equation X(k+1) = X(k) + H(k) 3.2 Newton’s Method 135 which leads to the inequality Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 136 Chapter 3 Nonlinear Equations 1. Verify that when Newton’s method is used to compute √R (by solving the equation x2 = R), the sequence of iterates is defined by xn+1 = 1􏰋xn + R 􏰌 2 xn 2. (Continuation) Show that if the sequence {xn} is defined 13. as in the preceding exercise, then x 2 − R = 􏰜 x n2 − R 􏰝 2 n+1 2xn Interpret this equation in terms of quadratic convergence. a3. WriteNewton’smethodinsimplifiedformfordetermin- ing the reciprocal of the square root of a positive number. n+1 3 n xn2 n+1 2n xn Do they converge for any nonzero initial point? If so, to what values? Each of the following functions has √3 R as a zero for any positive real number R. Determine the formulas for Newton’s method for each and any necessary restrictions on the choice for x0. 12. Considerthefollowingprocedures: aa. x = 1􏰋2x − r 􏰌 b. x = 1x + 1 Perform two iterations to approximate 1/ ± withx0 =1andx0 =−1. √ 5, starting aa. a(x)=x3−R ac. c(x)=x2−R/x ae. e(x)=1−R/x3 ag. g(x)=1/x2 −x/R b. b(x)=1/x3−1/R d. d(x)=x−R/x2 f. f(x)=1/x−x2/R h. h(x)=1−x3/R a4. Two of the four zeros of x4 + 2x3 − 7x2 + 3 are positive. Find them by Newton’s method, correct to two significant figures. 14. Determine the formulas for Newton’s method for find- ing a root of the function f(x) = x −e/x. What is the behavior of the iterates? a15. IfNewton’smethodisusedon f(x)=x3−x+1starting with x0 = 1, what will x1 be? 5. Theequationx−Rx−1 =0hasx =±R1/2 foritssolu- tion. Establish Newton’s iterative scheme, in simplified form, for this situation. Carry out five steps for R = 25 16. andx0=1. Locate the root of f (x) = e−x − cos x that is nearest π/2. IfNewton’smethodisusedon f(x)=x5−x3+3and ifxn=1,whatisxn+1? 6. Using a calculator, observe the sluggishness with which a 17. Newton’s method converges in the case of f (x ) = (x − 1)m with m = 8 or 12. Reconcile this with the theory. Use x0 = 1.1. a7. What linear function y = ax + b approximates f (x) = sin x best in the vicinity of x = π/4? How does this exercise relate to Newton’s method? 8. In Exercises 1.2.10–1.2.12, several methods are sug- gested for computing ln 2. Compare them with the use of Newton’s method applied to the equation ex = 2. a9. Define a sequence xn+1 = xn −tan xn with x0 = 3. What is limn→∞ xn? 10. Theiterationformula xn+1 = xn − (cos xn)(sin xn) + R cos2 xn where R is a positive constant, was obtained by apply- ing Newton’s method to some function f (x). What was f (x)? What can this formula be used for? a11. Establish Newton’s iterative scheme in simplified form, not involving the reciprocal of x, for the function f (x) = x R − x −1 . Carry out three steps of this procedure using R=4andx0 =−1. Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole deemed that any suppressed content does not materially affect the overall learning experience. 18. DetermineNewton’siterationformulaforcomputingthe cube root of N/M for nonzero integers N and M. a19. ForwhatstartingvalueswillNewton’smethodconverge if the function f is f (x) = x2/(1 + x2)? 20. Startingatx =3,x <3,orx >3,analyzewhathap- pens when Newton’s method is applied to the function
f (x) = 2×3 − 9×2 + 12x + 15.
a21. (Continuation) Repeat for f(x) = √|x|, starting with
x <0orx >0.
a22. Todeterminex =√3 R,wecansolvetheequationx3 = R by Newton’s method. Write the loop that carries out this process, starting from the initial approximation x0 = R.
23. ThereciprocalofanumberRcanbecomputedwithout division by the iterative formula
xn+1 =xn(2−xnR)
Establish this relation by applying Newton’s method to some f (x). Beginning with x0 = 0.2, compute the recip- rocal of 4 correct to six decimal digits or more by this rule.
or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
Exercises 3.2

24.
Tabulate the error at each step and observe the quadratic convergence.
On a certain modern computer, floating-point numbers have a 48-bit mantissa. Moreover, floating-point hard- ware can perform addition, subtraction, multiplication, and reciprocation, but not division. Unfortunately, the re- ciprocation hardware produces a result accurate to less than full precision, whereas the other operations produce results accurate to full floating-point precision.
a. Show that Newton’s method can be used to find a zero of the function f (x) = 1 − 1/(ax). This will provide an approximation to 1/a that is accurate to full floating-point precision. How many iterations are required?
b. Show how to obtain an approximation to b/a that is accurate to full floating-point precision.
√
simple zeros that is three times continuously differen- tiable. Show that the convergence of the resulting method to any zero r of f (x) is at least quadratic.
Hint: Apply the result in the text to F, making sure that F has the required properties.
The Taylor series for a function f looks like this:
a 32.
3.2 Newton’s Method 137
h2 h3 f(x+h)= f(x)+hf′(x)+ f′′(x)+
26
f′′′(x)+···
Supposethat f (x), f ′(x),and f ′′(x)areeasilycomputed. Derive an algorithm like Newton’s method that uses three terms in the Taylor series. The algorithm should take as input an approximation to the root and produce as output a better approximation to the root. Show that the method is cubically convergent.
Hint: Use en = en+1 − h and ignore e2 n+1
negligible.
terms as being
25.
26.
a 27.
28.
a29. 30. a 31.
R is 􏰋􏰌
Newton’s method for finding xn+1=1 xn+R
33.
34.
a 35.
a36.
To avoid computing the derivative at each step in New- ton’s method, it has been proposed to replace f ′ (xn ) by f ′(x0). Derive the rate of convergence for this method.
Refer to the discussion of Newton’s method and establish
2 xn
Perform three iterations of this scheme for computing
that
􏰀 􏰁 1􏰜 f′′(r)􏰝
How can this be used in a practical case to test whether the convergence is quadratic? Devise an example in which r , f ′(r), and f ′′(r) are all known, and test numerically the
convergence of e e−2. n+1 n
Show that in the case of a zero of multiplicity m, the modified Newton’s method
xn+1=xn−m f(xn) f′(xn)
is quadratically convergent.
√2, starting with x0 = 1, and of the bisection method for
√
2, starting with interval [1, 2]. How many iterations are needed for each method in order to obtain 10−6 accuracy?
lim e e−2 =−
n→∞ n+1n 2 f′(r)
(Continuation) Newton’s method for finding R = AB, gives this approximation:
√
R , where
√
A+B AB 4 + A + B
AB ≈
Show that if x0 = A or B, then two iterations of Newton’s
method are needed to obtain this approximation, whereas
if x0 = 1 (A + B), then only one iteration is needed. 2
Show that Newton’s method applied to xm − R and to 1−(R/xm)fordetermining √m Rresultsintwosimilaryet different iterative formulas. Here R > 0, m ≧ 2. Which formula is better and why?
Using a handheld calculator, carry out three iterations of Newton’smethodusingx0 =1and f(x)=3×3+x2− 15x + 3.
WhathappensiftheNewtoniterationisappliedtof(x)= arctan x with x0 = 2? For what starting values will New- ton’s method converge? (See Computer Exercise 3.2.7.)
Newton’smethodcanbeinterpretedasfollows:Suppose that f(x+h)=0.Then f′(x)≈[f(x+h)−f(x)]/h= − f (x)/h. Continue this argument.
Derive a formula for Newton’s method for the function F(x) = f (x)/f ′(x), where f (x) is a function with
Hint: Use f′(r +en).
Taylor series for
each of
+ en ) and
a 37.
f (r TheSteffensenmethodforsolvingtheequation f(x)=
0 uses the formula
xn+1=xn− f(xn)
g(xn )
in which g(x) = {f[x + f(x)] − f(x)}/f(x). It is
quadratically convergent, like Newton’s method. How many function evaluations are necessary per step? Us- ing Taylor series, show that g(x) ≈ f ′(x) if f (x) is small and thus relate Steffensen’s iteration to Newton’s. What advantage does Steffensen’s have? Establish the quadratic convergence.
A proposed Generalization of Newton’s method is xn+1 =xn −ω f(xn)
f′(xn)
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

138 Chapter 3 Nonlinear Equations
where the constant ω is an acceleration factor chosen to increase the rate of convergence. For what range of val- ues of ω is a simple root r of f (x) a point of attraction; that is, |g′(r)| < 1, where g(x) = x − ωf (x)/f ′(x)? This method is quadratically convergent only if ω = 1 because g′(r) ≠ 0 when ω ≠ 1. 38. Suppose that r is a double root of f(x) = 0; that is, f(r) = f′(r) = 0but f′′(r) ≠ 0,andsupposethat f and all derivatives up to and including the second are contin- uous in some neighborhood of r . Show that en+1 ≈ 1 en 2 for Newton’s method and thereby conclude that the rate of convergence is linear near a double root. (If the root has multiplicity m, then en+1 ≈ [(m − 1)/m]en.) 39. (SimultaneousNonlinearEquations)UsingtheTaylor series in two variables (x, y) of the form f(x+h,y+k)= f(x,y)+hfx(x,y)+kfy(x,y)+··· where fx = ∂f/∂x and fy = ∂f/∂y, establish that New- ton’s method for solving the two simultaneous nonlinear equations 􏰂 f (x, y) = 0 g(x, y) = 0 can be described with the formulas x =x−fgy−gfy  n+1 n fxgy −gx fy xy xy Here the functions f , f x , and so on are Newton’s method zn+1=zn− f(zn) f′(zn) xn+1=xn− ghy−hgy can be written in the form   hgx −ghx gxhy −gyhx yn+1 = yn − gxhy −gyhx Here all functions are evaluated at zn = xn + i yn . (xn, yn).  a41. 42. 43. 44. 45. 46. Considerthealgorithmofwhichonestepconsistsoftwo steps of Newton’s method. What is its order of conver- gence? (Continuation)Usingtheideaoftheprecedingexercise, show how we can easily create methods of arbitrarily high order for solving f (x) = 0. Why is the order of a method not the only criterion that should be considered in assessing its merits? If we want to solve the equation 2 − x = ex using New- ton’s iteration, what are the equations and functions that must be coded? Give a pseudocode for doing this exercise. Include a suitable starting point and a suitable stopping criterion. √ Suppose that we want to compute 2 by using Newton’s Method on the equation x2 = 2 (in the obvious, straight- forward way). If the starting point is x = 7 , what is the 05 numerical value of the correction that must be added to x0togetx1? Hint: The arithmetic is quite easy if you do it using ratios of integers. Apply Newton’s method to the equation f (x) = 0 with f (x) as given below. Find out what happens and why. a. f(x)=ex b. f(x)=ex+x2 Consider Newton’s method xn+1 = xn − f (xn )/ f ′(xn ). If the sequence converges then the limit point is a solu- tion. Explain why or why not. using nested multiplication. Stop the computation when two successive points differ by 1 × 10−5 or some other fxg−gx f  yn+1 = yn − f g − g f at evaluated 40. Newton’s method can be defined for the equation f (z) = g(x, y) + ih(x, y), where f (z) is an analytic function of the complex variable z = x + iy (x and y real) and g(x, y) and h(x, y) are real functions for all x and y. The derivative f ′(z) is given by f ′(z) = gx +ihx = hy −igy because the Cauchy-Riemann equations gx = hy and hx = −gy hold. Here the partial derivatives are defined as gx = ∂g/∂x, gy = ∂g/∂y, and so on. Show that 1. Using the procedure Newton and a single computer run, test your code on these examples: f (t ) = tan t − t with t√2 x0 = 7 and g(t) = e − t + 9 with x0 = 2. Print each iterate and its accompanying function value. 2. Writeasimple,self-containedprogramtoapplyNewton’s method to the equation x3 + 2x2 + 10x = 20, starting 3. with x0 = 2. Evaluate the appropriate f (x) and f ′(x), convenient tolerance close to your machine’s capability. Print all intermediate points and function values. Put an upper limit of ten on the number of steps. (Continuation) Repeat using double precision and more steps. Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. Computer Exercises 3.2 a4. Find the root of the equation 2x(1 − x2 + x) ln x = x2 − 1 in the interval [0, 1] by Newton’s method using double precision. Make a table that shows the number of correct digits in each step. a5. In 1685, John Wallis published a book called Algebra, in which he described a method devised by Newton for solving equations. In slightly modified form, this method was also published by Joseph Raphson in 1690. This form is the one now commonly called Newton’s method or the Newton-Raphson method. Newton himself discussed the method in 1669 and illustrated it with the equation x3 − 2x − 5 = 0. Wallis used the same example. Find a root of this equation in double precision, thus continuing the tradition that every numerical analysis student should solve this venerable equation. 6. In celestial mechanics, Kepler’s equation is important. It reads x = y −εsiny, in which x is a planet’s mean anomaly, y its eccentric anomaly, and ε the eccentricity of its orbit. Taking ε = 0.9, construct a table of y for 30 equally spaced values of x in the interval 0 ≦ x ≦ π . Use Newton’s method to obtain each value of y. The y corresponding to an x can be used as the starting point for the iteration when x is changed slightly. 7. In Newton’s method, we progress in each step from a given point x to a new point x − h, where h = f (x)/f ′(x). A refinement that is easily programmed is this: If | f (x − h)| is not smaller than | f (x)|, then reject this value of h and use h/2 instead. Test this refinement. a8. Write a brief program to compute a root of the equation x3 = x2 + x + 1, using Newton’s method. Be careful to select a suitable starting value. a9. Find the root of the equation 5(3x4 − 6x2 + 1) = 2(3x5 − 5x3) that lies in the interval [0, 1] by using Newton’s method and a short program. 10. Foreachequation,writeabriefprogramtocomputeand print eight steps of Newton’s method for finding a positive root. aa. x=2sinx ab. x3=sinx+7 ac. sinx=1−x ad. x5+x2=1+7x3forx≧2 11. Write and test a recursive procedure for Newton’s method. 12. Rewrite and test the Newton procedure so that it is a char- acter function and returns key words such as iterating, 3.2 Newton’s Method 139 success, near-zero, max-iteration. Then a case statement can be used to print the results. 13. Wouldyouliketoseethenumber0.55887766comeout of a calculation? Take three steps in Newton’s method on 10+x3 −12cosx =0startingwithx0 =1. a 14. Write a short program to solve for a root of the equation e−x 2 = cos x + 1 on [0, 4]. What happens in Newton’s methodifwestartwithx0 =0orwithx0 =1? 15. Findtherootoftheequation1x2+x+1−ex =0by 2 Newton’s method, starting with x0 = 1, and account for the slow convergence. 16. Using f(x)=x5−9x4−x3+17x2−8x−8andx0 =0, study and explain the behavior of Newton’s method. Hint: The iterates are initially cyclic. 17. Find the zero of the function f(x) = x − tanx that is closest to 99 (radians) by both the bisection method and Newton’s method. Hint: Extremely accurate starting values are needed for this function. Use the computer to construct a table of values of f (x) around 99 to determine the nature of this function. 18. Using the bisection method, find the positive root of 2x(1 + x2)−1 = arctan x. Using the root as x0, apply Newton’s method to the function arctanx. Interpret the results. 19. If the root of f (x) = 0 is a double root, then Newton’s method can be accelerated by using f(xn) xn+1 = xn − 2 f ′(xn) Numerically compare the convergence of this scheme with Newton’s method on a function with a known double root. 20. Program and test Steffensen’s method, as described in Exercise 3.2.36. 21. Considerthenonlinearsystem 􏰂 f (x, y) = x2 + y2 − 25 = 0 g(x, y) = x2 − y − 2 = 0 Using a software package that has 2D plotting capabili- ties, illustrate what is going on in solving such a system by plotting f (x, y), g(x, y), and show their intersection with the (x, y)-plane. Determine approximate roots of these equations from the graphical results. 22. Solve this pair of simultaneous nonlinear equations by first eliminating y and then solving the resulting Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 140 Chapter 3 Nonlinear Equations equation in x by Newton’s method. Start with the ini- tial value x0 = 1.0. 􏰂x3 − 2xy + y7 − 4x3y = 5 ysinx+3x2y+tanx =4 23. Using Equations (6) and (7), code Newton’s methods for nonlinear systems. Test your program by solving one or more of the following systems: a. SysteminComputerExercise3.2.21. b. SysteminComputerExercise3.2.22. d. Using starting values 􏰀3, 1,−1􏰁, solve 422 be computed only every other step. This method is given by x =x−f(x2n)  2 n + 1 2 n  f′(x2n) x2n+2 = x2n+1 − f (x2n+1) f′(x2n) c. System(3)usingstartingvalues(0,0,0). Numerically compare both proposed methods to New- ton’s method for several simple functions that have known roots. Print the error of each method on every iteration to monitor the convergence. How well do the proposed methods work? 27. (BasinofAttraction)Considerthecomplexpolynomial z3 − 1, whose zeros are the three cube roots of unity. Generate a picture showing three basins of attraction in the complex plane in the square region defined by −1 ≦ Real(z) ≦ 1 and −1 ≦ Imaginary(z) ≦ 1. To do this, use a mesh of 1000 × 1000 pixels inside the square. The center point of each pixel is used to start the iteration of Newton’s method. Assign a particular basin color to each pixel if convergence to a root is obtained with nmax = 10 iterations. The large number of iterations suggested can be avoided by doing some analysis with the aid of Theo- rem 1, since the iterates get within a certain neighborhood of the root and the iteration can be stopped. The criterion for convergence is to check both |zn+1 − zn| < ε and |z3 −1|<εwithasmallvaluesuchasε=10−4 as n+1 well as a maximum number of iterations. Hint: It is best to debug your program and get a crude picture with only a small number of pixels such as 10×10. 28. (Continuation)Repeatforthepolynomialz4−1=0. 29. Write real function Sqrt(x) to compute the square root of a real argument x by the following algorithm: First, reduce the range of x by finding a real number r and an integer m such that x = 22mr with 1 ≦r < 1. Next, 4 compute x2 by using three iterations of Newton’s method given by  x + y + z = 0 x2 + y2 + z2 = 2 x(y + z) = −1 e. Usingstartingvalues(−0.01,−0.01),solve 􏰂4y2 + 4y + 52x − 19 = 0 169x2 + 3y2 + 111x − 10y − 10 = 0 f. Selectstartingvalues,andsolve 􏰂sin(x + y) = ex−y cos(x + 6) = x2 y2 24. InvestigatethebehaviorofNewton’smethodforfinding complex roots of polynomials with real coefficients. For example, the polynomial p(x) = x2 + 1 has the com- plex conjugate pair of roots ±i and Newton’s method is xn+1 = 1 (xn − 1/xn ). First, program this method using 2 real arithmetic and real numbers as starting values. Sec- ond, modify the program using complex arithmetic but still using only real starting values. Finally, use complex numbers as starting values. Observe the behavior of the iterates in each case. 25. Using Exercise 3.2.40, find a complex root of each of the following: a. z3−z−1=0 b. z4−2z3−2iz2+4iz=0 c. 2z3−6(1+i)z2−6(1−i)=0 d. z=ez Hint: For the last part, use Euler’s relation eiy = cos y + i sin y. 26. IntheNewtonmethodforfindingarootrof f(x)=0,we start with x0 and compute the sequence x1, x2, . . . using the formula xn+1 = xn − f (xn)/f ′(xn). To avoid com- puting the derivative at each step, it has been proposed to replace f ′(xn) with f ′(x0) in all steps. It has also been suggested that the derivative in Newton’s formula Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole deemed that any suppressed content does not materially affect the overall learning experience. xn+1 = 1􏰋xn + r 􏰌 2 xn with the special initial approximation x0 = 1.27235 367 + 0.24269 3281r − √ Then set values of x. Obtain a listing of the code for the square- root function on your computer system. By reading the comments, try to determine what algorithm it uses. or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 1.02966 039 1+r x ≈ 2mx2. Test this algorithm on various 30. The following method has third-order convergence for √ computing R: 􏰀2 􏰁 xn+1=xn xn+3R 3 x n2 + R Carry out some numerical experiments using this method and the method of the preceding exercise to see whether you observe a difference in the rate of convergence. Use the same starting procedures of range reduction and initial approximation. 31. Write real function CubeRoot(x) to compute the cube root of a real argument x by the following procedure: First, determine a real number r and an integer m such that x = r23m with 1 ≦r < 1. Compute x4 using four 8 iterations of Newton’s method: 2􏰋r􏰌 xn+1=3 xn+2xn2 with the special starting value x0 =2.502926− 8.04512 5(r + 0.38775 52) (r + 4.61224 4)(r + 0.38775 52) − 0.35984 96 Then set √3 x ≈ 2m x4. Test this algorithm on a variety of x values. 32. Use mathematical software in MATLAB, Maple, or Mathematica to compute ten iterates of Newton’s method startingwithx0 =0for f(x)=x3−2x2+x−3.With 100 decimal places of accuracy and after nine iterations, show that the value of x is 2.17455 94102 92980 07420 23189 88695 65392 56759 48725 33708 24983 36733 92030 23647 64792 75760 66115 28969 38832 0640 Show that the values of the function at each iteration are 9.0, 2.0, 0.26, 0.0065, 0.45 × 10−5 , 0.22 × 10−11 , 0.50×10−24, 0.27×10−49, 0.1×10−98, and 0.1×10−98. Again notice that the number of digits of accuracy in Newton’s method doubles (approximately) with each it- eration once they are sufficiently close to the root. (Also, see Bornemann, Wagon, and Waldvogel [2004] for a 100- Digit Challenge, which is a study in high-accuracy nu- merical computing.) 33. (Continuation)UseMATLAB,MapleorMathematicato discover that this root is exactly 􏰕 3791√ 1 2 54+6 77+ 􏰕3 79 1√ +3 9 + 77 54 6 Clearly, the decimal results are of more interest to us in our study of numerical methods. 34. 35. 3.2 Newton’s Method 141 (Continuation) Find all the roots including complex roots. Numerically, find all the roots of the following systems of nonlinear equations. Then plot the curves to verify your results: a. y=2x2+3x−4,y=x2+2x+3 b. y+x+3=0,x2+y2=17 c. y=1x−5,y=x2+2x−15 2 d. xy=1,x+y=2 e. y=x2,x2+(y−2)2=4 f. 3x2+2y2=35,4x2−3y2=24 g. x2−xy+y2=21,x2+2xy−8y2=0 Apply Newton’s method on these test problems: 36. a. b. c. f (x) = x2. Hint: The first derivative is zero at the root, and con- vergence may not be quadratic. f (x) = x + x4/3. Hint: There is no second derivative at the root, and convergence may fail to be quadratic. f(x)=x+x2sin(2/x)forx ≠ 0and f(0)=0and f′(x)=1+2xsin(2/x)−2cos(2/x)forx ≠ 0and f ′(0) = 1. Hint: The derivative of this function is not continuous at the root, and convergence may fail. 37. Let F(X) = 􏰆2 􏰇􏰆􏰇 x1 −x2+c=0. x2 −x1+c 0 Each component equation f1(x) = 0 and f2(x) = 0 de- scribes a parabola. Any point (x∗, y∗) where these two parabolas intersect is a solution to the nonlinear system of equations. Using Newton’s method for systems of nonlin- ear equations, find the solutions for each of these values of the parameter c = 1, 1,−1,−1. Give the Jacobian 242 matrix for each. Also for each of these values, plot the re- sulting curves showing the points of intersection (Heath [2000], p. 218). 38. 39. 40. LetF(X)= 􏰆2 􏰇􏰆 􏰇 x1 +2x2 −2 = 0 . x1 + 4x2 − 4 0 Solve this nonlinear system starting with X(0) = (1, 2). Give the Jacobian matrix. Also plot the resulting curves showing the point(s) of intersection. Using Newton’s method, find the zeros of f (z) = z3 − z with these starting values z(0) = 1 + 1.5i, 1 + 1.1i, 1 + 1.2i, 1 + 1.3i. Use Halley’s method to produce a plot of the basins of attraction for p(z) = z6 − 1. Compare to Figure 3.8. Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 142 Chapter 3 Nonlinear Equations 41. (Global Positioning System Project) Each time a GPS is used, a system of nonlinear equations of the form (x−a)2+(y−b)2+(z−c)2=[(C(t −D)]2 (x−a2)2+(y−b2)2+(z−ci)2 =[(C(t2−D)]2 (x−a3)2+(y−b3)2+(z−ci)2 =[(C(t3−D)]2 (x−a4)2+(y−b4)2+(z−ci)2 =[(C(t4−D)]2 nonlinear system. (See Hofmann-Wellenhof, Lichteneg- ger, and Collins [2001], Sauer [2012], and Strang and Borre [1997].) 1 1 i 1 is solved for the (x, y, z) coordinates of the receiver. For eachsatellitei,thelocationsare(ai,bi,ci),andti isthe synchronized transmission time from the satellite. Fur- ther, C is the speed of light, and D is the difference between the synchronized time of the satellite clocks and the earth-bound receiver clock. Although there are only two points on the intersection of three spheres (one of which can be determined to be the desired location), a fourth sphere (satellite) must be used to resolve the inaccuracy in the clock contained in the low-cost re- ceiver on earth. Explore various ways for solving such a 42. UsemathematicalsoftwaresuchasinMATLAB,Maple, or Mathematica and their built-in procedures to solve the system of nonlinear equations (8) in Example 2. Also, plot the given surfaces and the solution obtained. Hint: You may need to use a slightly perturbed starting point (0.5, 1.5, 0.5) to avoid a singularity in the Jacobian matrix. 3.3 Secant Method Method Description We now consider a general-purpose procedure that converges almost as fast as Newton’s method. This method mimics Newton’s method, but avoids the calculation of derivatives. Recall that Newton’s iteration defines xn+1 in terms of xn via the formula xn+1=xn− f(xn) (1) f′(xn) In the secant method, we replace f ′(xn) in Formula (1) by an approximation that is easily computed. Since the derivative is defined by Secant Method Formula (In Section 4.3, we revisit this subject and learn that this is a finite difference approximation to the first derivative.) In particular, if x = xn and h = xn−1 − xn, we have f′(xn)≈ f(xn−1)− f(xn) (2) xn−1 − xn When this is used in Equation (1), the result defines the secant method: xn+1 =xn −􏰋 xn −xn−1 􏰌f(xn) (3) f(xn)− f(xn−1) The secant method (like Newton’s) can be used to solve systems of equations as well. The name of the method is taken from the fact that the right member of Equation (2) is the slope of a secant line to the graph of f (see Figure 3.9). Of course, the left member is the slope of a tangent line to the graph of f . (Similarly, Newton’s method could be called the “tangent method.”) we can say that for small h, f′(x)=lim f(x+h)−f(x) h→0 h f′(x)≈ f(x+h)− f(x) h Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. FIGURE 3.9 Secant method y 5 f(x) r xn11 xn xn21 3.3 Secant Method 143 Secant line x A few remarks about Equation (3) are in order. Clearly, xn+1 depends on two previous elements of the sequence. So to start, two points (x0 and x1) must be provided. Equation (3) can then generate x2, x3, . . . . In programming the secant method, we could calculate and test the quantity f (xn)− f (xn−1). If it is nearly zero, an overflow can occur in Equation (3). Of course, if the method is succeeding, the points xn will be approaching a zero of f , so f (xn ) will be converging to zero. (We are assuming that f is continuous.) Also, f (xn−1) willbeconvergingtozero,and,afortiori, f(xn)− f(xn−1)willapproachzero.Iftheterms f(xn) and f(xn−1) have the same sign, additional significant digits are canceled in the subtraction.Sowecouldperhapshalttheiterationwhen|f(xn)− f(xn−1)|≦δ|f(xn)|for some specified tolerance δ, such as 1 × 10−6. (See Computer Exercise 3.3.18.) 2 Secant Algorithm A procedure pseudocode for nmax steps of the secant method applied to the function f starting with the interval [a, b] = [x0, x1] can be written as follows: procedure Secant( f, a, b, nmax, ε) integer n, nmax; real a, b, fa, fb, ε, d external function f fa← f(a) fb← f(b) if |fa| > |fb| then
a ←→ b
fa ←→ fb end if
output 0, a, fa output 1, b,fb
for n = 2 to nmax
if |fa| > |fb| then a ←→ b
f a ←→ f b end if
d ← (b − a)/(fb − fa) b←a
fb←fa
d ← d · fa
if |d| < ε then output “convergence” return end if (Continued) Secant Pseudocode Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 144 Chapter 3 Nonlinear Equations a←a−d fa← f(a) outputn,a, fa end for end procedure Secant Error Vector Approximation We can use mathematical software to find the single real root, −1.1053, and the two pairs of complex roots, −0.319201 ± 1.35008i and 0.871851 ± 0.806311i. ■ Convergence Analysis The advantages of the secant method are that (after the first step) only one function evaluation is required per step (in contrast to Newton’s iteration, which requires two) and that it is almost as rapidly convergent. It can be shown that the basic secant method defined by Equation (3) obeys an equation of the form en+1 = −1􏰋 f ′′(ξn)􏰌enen−1 ≈ −1􏰋 f ′′(r)􏰌enen−1 (4) 2 f′(ζn) 2 f′(r) EXAMPLE 1 Solution Here ←→ means interchange values. The endpoints [a, b] are interchanged, if necessary, to keep | f (a)| ≦ | f (b)|. Consequently, the absolute values of the function are nonincreasing; thus, we have | f (xn)|≧ | f (xn+1)| for n ≧ 1. Ifthesecantmethodisusedon p(x)=x5 +x3 +3withx0 =−1andx1 =1,whatisx8? The output from the computer program corresponding to the pseudocode for the secant method is as follows. (We used a 32-bit word-length computer.) n xn p(xn ) 0 −1.0 1.0 1 1.0 5.0 2 −1.5 −7.97 3 −1.05575 4 −1.11416 5 −1.10462 6 −1.10529 7 −1.10530 8 −1.10530 0.512 −9.991 × 10−2 7.593 × 10−3 1.011 × 10−4 2.990 × 10−7 2.990 × 10−7 where ξn and ζn are in the smallest interval that contains r, xn, and xn−1. Thus, the ratio en+1(enen−1)−1 converges to −1 f ′′(r)/f ′(r). The rapidity of convergence of this method 2 is, in general, between those for bisection and for Newton’s method. To prove the second part of Equation (4), we begin with the definition of the secant method in Equation (3) and the error en+1 = r − xn+1 =r− f(xn)xn−1−f(xn−1)xn f(xn)− f(xn−1) Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. = f (xn)en−1 − f (xn−1)en f(xn)− f(xn−1) f(xn) − f(xn−1) =􏰜 xn −xn−1 􏰝 en en−1 e e (5) 3.3 Secant Method 145 xn −xn−1  n n−1 f(xn)= f(r −en)= f(r)−en f′(r)+ 1en2 f′′(r)+O􏰀en3􏰁 f(xn)− f(xn−1)  By Taylor’s Theorem, we establish Since f (r) = 0, this gives us en 2 2 f(xn)=−f′(r)+1en f′′(r)+O􏰀en2􏰁 Changing the index to n − 1 yields f (xn−1) = − f ′(r) + 1e f ′′(r) + O􏰀e2 􏰁 en−1 2 n−1 n−1 By subtraction between these equations, we arrive at Error Bound (Superlinear Convergence) f(xn) − f(xn−1) = 1(e −e )f′′(r)+O􏰀e2 􏰁 en en−1 2n n−1 n−1 Since xn − xn−1 = en−1 − en , we reach the equation f(xn)− f(xn−1) en en−1 ≈−1 f′′(r) xn − xn−1 2 The first bracketed expression in Equation (5) can be written as xn−xn−1 ≈ 1 f (xn) − f (xn−1) f ′(r) Hence, we have shown the second part of Equation (4). We leave the establishment of the first part of Equation (4) as an exercise because it depends on some material to be covered in Chapter 4. (See Exercise 3.3.18.) From Equation (4), the order of convergence for the secant method can be expressed in terms of the inequality |en+1|≦ C|en|α (6) where α = 1 􏰀1+√5 􏰁 ≈ 1.62 is the golden ratio. Since α > 1, we say that the convergence
2
is superlinear. Assuming that Inequality (6) is true, we can show that the secant method
converges under certain conditions.
Letc = c(δ)bedefinedasinEquation(2)ofSection3.2.If|r−xn|≦δand|r−xn−1|≦δ,
for some root r , then Equation (4) yields
|en+1|≦ c|en||en−1| (7)
Suppose that the initial points x0 and x1 are sufficiently close to r that c|e0| ≦ D and c|e1| ≦ D for some D < 1. Then c|e1|≦D, c|e0|≦D c|e2| ≦ c|e1| c|e0| ≦ D2 Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 146 Chapter 3 Nonlinear Equations Fibonacci Sequence (n≧3) This is the recurrence relation for generating the famous Fibonacci sequence, 1, 1, 2, 3, 5, In general, we have where inductively, c|e3| ≦ c|e2| c|e1| ≦ D3 c|e4| ≦ c|e3| c|e2| ≦ D5 c|e5| ≦ c|e4| c|e3| ≦ D8 etc. |en | ≦ c−1 Dλn+1 􏰂λ1=1, λ2=1 (8) (9) Error Vector Approximation Alternative Convergence Analysis λn =λn−1 +λn−2 8, . . . . It can be shown to have the surprising explicit form 1􏰀n n􏰁 λn=√ α−β (10) 5 whereα= 1􏰀1+√5􏰁andβ= 1􏰀1−√5􏰁.SinceD<1andλn →∞,weconcludefrom 22 Inequality (8) that en → 0. Hence, xn → r as n → ∞, and the secant method converges to the root r if x0 and x1 are sufficiently close to it. Next, we show that Inequality (6) is in fact reasonable—not a proof. From Inequal- ity (7), we now have |en+1| ≦ c|en||en−1| = c|en|α|en|1−α|en−1| ≈ c|en|α􏰀c−1Dλn+1􏰁1−α􏰀c−1Dλn􏰁 = |en|αcα−1Dλn+1(1−α)+λn = |en|αcα−1Dλn+2−αλn+1 by using an approximation to Inequality (8). In the last line, we used the recurrence relation (9). Now λn+2 − αλn+1 converges to zero. (See Exercise 3.3.6.). Hence, cα−1 Dλn+2−αλn+1 is bounded, say by C, as a function of n. Thus, we have |en+1| ≈ C|en|α which is a reasonable approximation to Inequality (6). Another derivation (with a bit of hand waving) for the order of convergence of the secant method can be given by using a general recurrence relation. Equation (4) gives us en+1 ≈ Kenen−1 where K = −1 f ′′(r)/ f ′(r). We can write this as 2 |Ken+1| ≈ |Ken||Ken−1| Letzi =log|Kei|.Thenwewanttosolvetherecurrenceequation zn+1 = zn + zn−1 where z0 and z1 are arbitrary. This is a linear recurrence relation with constant coefficients similar to the one for the Fibonacci numbers (9) except that the first two values z0 and z1 Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. are unknown. The solution is of the form zn = Aαn + Bβn whereα= 1􏰀1+√5􏰁andβ= 1􏰀1−√5􏰁.Thesearetherootsofthequadraticequation zn ≈ Aαn for large n and for some constant A. Consequently, we have 3.3 Secant Method 147 (11) Error Vectors Approximation |en+1| ≈ C|en|α (12) for large n and for some constant C. Again, Inequality (6) is essentially established! A rigorous proof of Inequality (6) is tedious and quite long. Comparison of Methods In this chapter, three primary methods for solving an equation f (x ) = 0 have been presented. The bisection method is reliable but slow. Newton’s method is fast but often only near the root and requires f ′. The secant method is nearly as fast as Newton’s method and does not require knowledge of the derivative f ′, which may not be available or may be too expensive to compute. The user of the bisection method must provide two points at which the signs of f (x) differ, and the function f need only be continuous. In using Newton’s method, one must specify a starting point near the root, and f must be differentiable. The secant method requires two good starting points. Newton’s procedure can be interpreted as the repetition of a two-step procedure summarized by the prescription linearize and solve. This strategy is applicable in many other numerical problems, and its importance cannot be overemphasized. Both Newton’s method and the secant method fail to bracket a root. The modified false position method can retain the advantages of both methods. The secant method is often faster at approximating roots of nonlinear functions in comparison to bisection and false position. Unlike these two methods, the intervals [ak , bk ] do not have to be on opposite sides of the root and have a change of sign. Moreover, the slope of the secant line can become quite small, and a step can move far from the current point. The secant method can fail to find a root of a nonlinear function that has a small slope near the root because the secant line can jump a large amount. For nice functions and guesses relatively close to the root, most of these methods require relatively few iterations before coming close to the root. However, there are pathological functions that can cause troubles for any of those methods. When selecting a method for solving a given nonlinear problem, one must consider many issues such as what you know about the behavior of the function, an interval [a, b] satisfying f (a) f (b) < 0, the first derivative of the function, a good initial guess to the desired root, and so on. Pros and Cons of Bisection, Newton’s, and Secant Methods 22 λ2 −λ−1=0.Since|α|>|β|,theterm Aαn dominates,andwecansaythat
Then it follows that
Hence, we have
|Ken| ≈ 10Aαn
|Ken+1| ≈ 10Aαn+1 = 􏰀10Aαn 􏰁α = |Ken|α
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

148 Chapter 3
Nonlinear Equations
Combining Schemes
Yet Another Approach
Hybrid Schemes
In an effort to find the best algorithm for finding a zero of a given function, various hybrid methods have been developed. Some of these procedures combine the bisection method (used during the early iterations) with either the secant method or the Newton method. Also, adaptive schemes are used for monitoring the iterations and for carrying out stopping rules. More information on some hybrid secant-bisection methods and hybrid Newton-bisection methods with adaptive stopping rules can be found in Bus and Dekker [1975], Dekker [1969], Kahaner, Moler, and Nash [1989], and Novak, Ritter, and Woz ́niakowski [1995].
Fixed-Point Iteration
For a nonlinear equation f (x) = 0, we seek a point where the curve f intersects the x-axis (y = 0). An alternative approach is to recast the problem as a fixed-point problem x = g(x) for a related nonlinear function g. For the fixed point problem, we seek a point where the curve g intersects the diagonal line y = x. A value of x such that x = g(x) is a fixed point of g because x is unchanged when g is applied to it. Many iterative algorithms for solving a nonlinear equation f (x) = 0 are based on a fixed-point iterative method
x(n+1) = g􏰀x(n)􏰁
where g has fixed points that are solutions of f (x) = 0. An initial starting value x(0) is
selected, and the iterative method is applied repeatedly until it converges sufficiently well.
Apply the fixed-point procedure, where g(x ) = 1 + 2/x , starting with x (0) = 1, to compute a zero of the nonlinear function f (x) = x2 − x − 2. Graphically, trace the convergence process.
The fixed-point method is
x(n+1)=1+ 2 x(n)
Eight steps of the iterative algorithm are x (0) = 1, x (1) = 3, x (2) = 5/3, x (3) = 11/5, x(4) = 21/11, x(5) = 43/21, x(6) = 85/43, x(7) = 171/85, and x(8) = 341/171 ≈ 1.99415. In Figure 3.10, we see that these steps spiral into the fixed point 2.
EXAMPLE 2
Solution
y
3
2
1
y 5 1 1 2x
y 5 x
FIGURE 3.10
Fixed-point iterations forf(x)=x2−x−2
r
0123x ■
For a given nonlinear equation f (x) = 0, there may be many equivalent fixed-point problems x = g(x) with different functions g, some better than others. A simple way to
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

Locally Convergence
characterize the behavior of an iterative method x(n+1) = g􏰀x(n)􏰁 is locally convergent for x∗ if x∗ = g(x∗) and |g′(x∗)| < 1. By locally convergent, we mean that there is an interval containing x(0) such that the fixed-point method converges for any starting value x(0) within that interval. If |g′(x∗)| > 1, then the fixed-point method diverges for any starting point x(0) other than x∗. Fixed-point iterative methods are used in standard practice for solving many science and engineering problems. In fact, the fixed-point theory can simplify the proof of the convergence of Newton’s method.
For additional details and sample plots, see Kincaid and Cheney [2002] or Epureanu and Greenside [1998]. Also, look at other items in the Bibliography. For example, an expository paper by Ypma [1995] traces the historical development of Newton’s method through notes, letters, and publications by Isaac Newton, Joseph Raphson, and Thomas Simpson.
Summary 3.3
• The secant method for finding a zero r of a function f (x) is written as xn+1 =xn −􏰋 xn −xn−1 􏰌f(xn)
f(xn)− f(xn−1)
for n ≧ 1, which requires two initial values x0 and x1. After the first step, only one new
function evaluation per step is needed.
• Aftern+1stepsofthesecantmethod,theerroriteratesei =r−xi obeytheequation
en+1 =−1􏰋f′′(ξn)􏰌enen−1 2 f′(ζn)
which leads to the approximation
|en+1| ≈ C|en|(1+ 5)/2 ≈ C|en|1.62
Therefore, the secant method has superlinear convergence behavior.
a1. Calculate an approximate value for 43/4 using one step of the secant method with x0 = 3 and x1 = 2.
2. If we use the secant method on f(x) = x3 −2x +2 starting with x0 = 0 and x1 = 1, what is x2?
a3. Ifthesecantmethodisusedon f(x)=x5+x3+3and ifxn−2 =0andxn−1 =1,whatisxn?
a4. Ifxn+1=xn+(2−exn)(xn−xn−1)/(exn −exn−1)with x0 =0andx1 =1,whatislimn→∞xn?
5. Using the bisection method, Newton’s method, and the secant method, find the largest positive root correct to three decimal places of x3 − 5x + 3 = 0. (All roots are in [−3, +3].)
6. Prove that in the first analysis of the secant method, λn+1 − αλn converges to zero as n → ∞.
7. EstablishEquation(10).
8. Write out the derivation of the order of convergence of the secant method that uses recurrence relations; that is, find the constants A and B in Equation (11), and fill in the details in arriving at Equation (12).
a9. Whatistheappropriateformulaforfindingsquareroots using the secant method? (Refer to Exercise 3.2.1.)
10. Establish that the formula for the secant method can be written as
xn+1 = xn−1 f(xn)−xn f(xn−1) f(xn)− f(xn−1)
Explore whether it or Equation (3) is more numerically stable with better achievable accuracy in a computer program.
√
3.3 Secant Method 149
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
Exercises 3.3

150 Chapter 3 Nonlinear Equations
11.
a 12.
13.
Show that if the iterates in Newton’s method converge to a point r for which f ′(r) ≠ 0, then f (r) = 0. Establish the same assertion for the secant method.
Hint: In the latter, the Mean-Value Theorem of Differen- tial Calculus is useful. This is the case n = 0 in Taylor’s Theorem.
A method of finding a zero of a given function f proceeds as follows. Two initial approximations x0 and x1 to the zero are chosen, the value of x0 is fixed, and successive iterations are given by􏰋 xn − x0 􏰌
xn+1 = xn − f(xn)− f(x0) f(xn)
This process will converge to a zero of f under certain conditions. Show that the rate of convergence to a simple zero is linear under some conditions.
Test the following sequences for different types of con- vergence (i.e., linear, superlinear, or quadratic), where n = 1,2,3….
a −2 −na −2n a. xn = n b. xn = 2 c. xn = 2
d. xn =2−an witha0 =a1 =1andan+1 =an +an−1 for n ≧ 2
eration is as follows: Starting with any x0, we define xn+1 = f(xn),wheren=0,1,2,….Showthatif f is continuous and if the sequence {xn} converges, then its limitisafixedpointoff.
a15. (Continuation) Show that if f is a function defined on the whole real line whose derivative satisfies | f ′(x)| ≦ c with a constant c less than 1, then the method of functional iteration produces a fixed point of f .
Hint: In establishing this, the Mean-Value Theorem from Section 1.2 is helpful.
a 16. (Continuation) With a calculator, try the method of func- tional iteration with f (x) = x/2 + 1/x, taking x0 = 1. What is the limit of the resulting sequence?
a17. (Continuation) Using functional iteration, show that the equation 10−2x +sinx = 0 has a root. Locate the root approximately by drawing a graph. Starting with your ap- proximate root, use functional iteration to obtain the root accurately by using a calculator.
Hint: Write the equation in the form x = 5 + 1 sin x. 2
18. Establish the first part of Equation (4) using Equation (5). Hint: Use the relationship between divided differences and derivatives from Section 4.3.
cise 3.3.1. Print the final estimate of the solution and the value of the function at this point.
3. Findazeroofoneofthefunctionsgivenintheintroduc- tion of this chapter using one of the methods introduced in this chapter.
4. Write and test a recursive procedure for the secant method.
5. Reruntheexampleinthissectionwithx0 =0andx1 =1. Explain any unusual results.
6. Write a simple program to compare the secant method with Newton’s method for finding a root of each func- tion.
aa. x3−3x+1withx0=2 ab. x3−2sinxwithx0=1
2
Use the x1 value from Newton’s method as the second starting point for the secant method. Print out each itera- tion for both methods.
or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
14. This exercise and the next three deal with the method of functional iteration. The method of functional it-
a1. Use the secant method to find the zero near −0.5 of f (x) = ex − 3×2. This function also has a zero near 4.
Find this positive zero by Newton’s method. 2. Write
procedure Secant( f, x1, x2, epsi, delta, maxf, x, ierr)
which uses the secant method to solve f (x) = 0. The in- put parameters are as follows: f is the name of the given function; x1 and x2 are the initial estimates of the so- lution; epsi is a positive tolerance such that the iteration stops if the difference between two consecutive iterates is smaller than this value; delta is a positive tolerance such that the iteration stops if a function value is smaller in magnitude than this value; and maxf is a positive inte- ger bounding the number of evaluations of the function allowed. The output parameters are as follows: x is the final estimate of the solution, and ierr is an integer error flag that indicates whether a tolerance test was violated. Test this routine using the function of Computer Exer-
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole deemed that any suppressed content does not materially affect the overall learning experience.
Computer Exercises 3.3

a7. Write a simple program to find the root of f(x) = x3 +2×2 +10x −20 using the secant method with starting valuesx0 =2andx1 =1.Letitrunatmost20steps, and include a stopping test as well. Compare the number of steps needed here to the number needed in Newton’s method. Is the convergence quadratic?
8. Test the secant method on the set of functions fk(x) = 2e−kx+1−3e−kx fork = 1,2,3,…,10.Usethestart- ing points 0 and 1 in each case.
a9. An example by Wilkinson [1963] shows that minute al- terations in the coefficients of a polynomial may have massive effects on the roots. Let
f (x) = (x − 1)(x − 2) · · · (x − 20)
which has become known as the Wilkinson polynomial.
The zeros of f are, of course, the integers 1, 2, . . . , 20. Try to determine what happens to the zero r = 20 when the function is altered to f (x) − 10−8×19.
Hint: The secant method in double precision will locate a zero in the interval [20, 21].
10. Test the secant method on an example in which r,
f ′ (r ), and f ′′ (r ) are known in advance. Monitor the
ratios en+1/(enen−1) to see whether they converge to
−1 f ′′(r)/f ′(r). The function f (x) = arctan x is suit- 2
able for this experiment.
11. Usingafunctionofyourchoice,verifynumericallythat the iterative method
Test the algorithm on some simple cases such as (x, y) = (3, 4), (−5, 12), and (7, −24). Then write a routine that uses the function f (x, y) for approximating the
xn+1 = xn − 􏰓
is cubically convergent at a simple root but only linearly
16.
17.
15.
Study the following functions by starting with any ini- tial value of x0 in the domain [0, 2] and iterating xn+1 = F (xn ). First use a calculator and then a computer. Explain the results.
a. Usethetentfunction
􏰂2x if 2x < 1 F(x) = 2x−1 if2x≧1 b. Repeatusingthefunction F(x) = 10x (modulo 1) Hint: Don’t be surprised by chaotic behavior. The inter- ested reader can learn more about the dynamics of one- dimensional maps by reading papers such as the one by Bassien [1998]. Showhowthesecantmethodcanbeusedtosolvesystems of equations such as those in Computer Exercises 3.2.21– 3.2.23. (StudentResearchProject)Muller’smethodisanal- gorithm for computing solutions of an equation f (x) = 0. It is similar to the secant method in that it replaces f locally by a simple function and finds a root of it. Nat- urally, this step is repeated. The simple function chosen in Muller’s method is a quadratic polynomial, p, that interpolates f at the three most recent points. After p has been determined, its roots are computed, and one of them is chosen as the next point in the sequence. Since this quadratic function may have complex roots, the al- gorithm should be programmed with this in mind. Sup- pose that points xn−2, xn−1, and xn have been computed. Set p(x) = a(x − xn)(x − xn−1) + b(x − xn) + c 3.3 Secant Method 151 real function f (x, y) integern; reala,b,c,x,y f ←max{|x|,|y|} a ← min{|x|,|y|} for n = 1 to 3 b ← (a/f )2 c ← b/(4 + b) f ← f +2cf a ← ca end for end function f Euclideannormofavectorx = (x1,x2,...,xn);thatis, 􏰀2 2 2􏰁1/2 thenonnegativenumber∥x∥= x1+x2+···+xn . f(xn) [ f ′(xn)]2 − f (xn) f ′′(xn) convergent at a multiple root. 12. Test numerically whether Olver’s method, given by xn+1=xn− f(xn)−1 f′′(xn)􏰜f(xn)􏰝2 f ′(xn) 2 f ′(xn) f ′(xn) is cubically convergent to a root of f . Try to establish that it is. 13. (Continuation)RepeatforHalley’smethod x =x − 1 with a = f′(xn)−1􏰜f′′(xn)􏰝 n+1 n a n f(x) 2 f′(x) nnn 14. (Moler-Mo􏰓rrison Algorithm) Computing an approxi- mation for x2 + y2 does not require square roots. It can be done as follows: Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 152 Chapter 3 Nonlinear Equations 18. 19. p(x) = x3 + x2 − 10x − 10 22. If the first three points are 1, 2, 3, then you should find that the polynomial is p(x) = 7(x −3)(x −2)+14(x −3)−4 and x4 = 3.17971 086. Next, test your code on a polyno- 23. mial having real coefficients but some complex roots. Programandtestthecodeforthesecantalgorithmafter incorporating the stopping criterion described in the text. Using mathematical software such as MATLAB, Math- ematica, and Maple, find the real zero of the polynomial p(x) = x5 + x3 + 3. Attain more digits of accuracy than shown in the solution to Example 1 in the text. Find the fixed points for each of the following functions: a. ex+1 b. e−x−x c. x2−4sinx d. x3+6x2+11x−6 e. sinx Forthenonlinearequation f(x)=x2−x−2=0with roots 1 and 2, write four fixed-point problems x = g(x) that are equivalent. Plot all of these, and show that they all intersect the line x = y. Also, plot the convergence steps of each of these fixed-point iterations for differ- ent starting values x (0) . Show that the behavior of these fixed-point schemes can vary wildly: slow convergence, fast convergence, and divergence. where a, b, and c are determined so that p interpolates 20. f at the three points mentioned previously. Then find the (Continuation) Using mathematical software that allows for complex roots, find all zeros of the polynomial. roots of p and take xn+1 to be the root of p closest to xn. At the beginning, three points must be furnished by the user. Program the method, allowing for complex numbers throughout. Test your program on the example 21. Programahybridmethodforsolvingseveralofthenon- linear problems given as examples in the text, and com- pare your results with those given. Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 4 Interpolation and Numerical Differentiation The viscosity of water has been experimentally determined at different temperatures, as indicated in the following table: Temperature 0◦ 5◦ 10◦ 15◦ Viscosity 1.792 1.519 1.308 1.140 From this table, how can we estimate a reasonable value for the viscosity at temperature 8◦? The method of polynomial interpolation, described in Section 4.1, can be used to create a polynomial of degree 3 that assumes the values in the table. This polynomial should provide acceptable intermediate values for temperatures not tabulated. The value of that polynomial at the point 8◦ turns out to be 1.386. 4.1 Polynomial Interpolation Preliminary Remarks Problem 1 Problem 2 Problem 3 A Simple Function We pose three problems concerning the representation of functions to give an indication of the subject matter in this chapter, in Chapter 6 (on splines), and in Chapter 9 (on least squares). First, suppose that we have a table of numerical values of a function: x x0 x1 ··· xn y y0 y1 ··· yn The second problem is similar, but it is assumed that the given table of numerical values is contaminated by errors, as might occur if the values came from a physical experiment. Now we ask for a formula that represents the data (approximately) and, if possible, filters out the errors. As a third problem, a function f is given, perhaps in the form of a computer procedure, but it is an expensive function to evaluate. In this case, we ask for another function g that is simpler to evaluate and produces a reasonable approximation to f . Sometimes in this problem, we want g to approximate f with full machine precision. In all of these problems, a simple function p can be obtained that represents or ap- proximates the given table or function f . The representation p can always be taken to be a polynomial, although many other types of simple functions can also be used. Once a simple 153 Is it possible to find a simple and convenient formula that reproduces the given points exactly? Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 154 Chapter 4 Interpolation and Numerical Differentiation High-Degree Polynomial function p has been obtained, it can be used in place of f in many situations. For example, the integral of f could be estimated by the integral of p, and the latter should generally be easier to evaluate. In many situations, a polynomial solution to the problems outlined above are unsatisfac- tory from a practical point of view, and other classes of functions must be considered. In this book, one other class of versatile functions is discussed: the spline functions (see Chapter 6). The present chapter concerns polynomials exclusively, and Chapter 9 discusses general lin- ear families of functions, of which splines and polynomials are important examples. The obvious way in which a polynomial can fail as a practical solution to one of the preceding problems is that its degree may be unreasonably high. For instance, if the table considered contains 1,000 entries, a polynomial of degree 999 may be required to represent it. Polynomials also may have the surprising defect of being highly oscillatory. If the table is precisely represented by a polynomial p, then p(xi ) = yi for 0 ≦ i ≦ n. For points other than the given xi, however, p(x) may be a very poor representation of the function from which the table arose. The example in Section 4.2 involving the Runge function illustrates this phenomenon. Polynomial Interpolation We begin again with a table of values: x x0 x1 ··· xn y y0 y1 ··· yn and assume that the xi ’s form a set of n + 1 distinct points. The table represents n + 1 points in the Cartesian plane, and we want to find a polynomial curve that passes through all points. Thus, we seek to determine a polynomial that is defined for all x and takes on thecorrespondingvaluesofyi foreachofthen+1distinctxi’sinthistable.Apolynomial p for which p(xi ) = yi when 0 ≦ i ≦ n is said to interpolate the table. The points xi are called nodes. Consider the first and simplest case, n = 0. Here, a constant function solves the problem. In other words, the polynomial p of degree 0 defined by the equation p(x ) = y0 reproduces the one-node table. The next simplest case occurs when n = 1. Since a straight line can be passed through two points, a linear function is capable of solving the problem. Explicitly, the polynomial Polynomial Interpolations Table Constant Case Linear Case EXAMPLE 1 p defined by p(x) = 􏰋 x − x1 􏰌y0 + 􏰋 x − x0 􏰌y1 x0 − x1 x1 − x0 = y 0 + 􏰋 y 1 − y 0 􏰌( x − x 0 ) x1 − x0 is of first degree (at most) and reproduces the table. That means (in this case) that p(x0) = y0 and p(x1) = y1, as is easily verified. This p is used for linear interpolation. Find the polynomial of least degree that interpolates this table: x 1.4 1.25 y 3.7 3.9 Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. Cardinal Polynomial Expanded Form li(x)=􏰅n 􏰋x−xj 􏰌 (0≦i≦n) (2) xi −xj j ≠ i j=0 This formula indicates that li (x ) is the product of n linear factors: li(x)=􏰋x−x0 􏰌􏰋x−x1 􏰌···􏰋x−xi−1 􏰌􏰋x−xi+1 􏰌···􏰋x−xn 􏰌 xi −x0 xi −x1 xi −xi−1 xi −xi+1 xi −xn (The denominators are just numbers; the variable x occurs only in the numerators.) Thus, li is a polynomial of degree n. Notice that when li (x) is evaluated at x = xi , each factor in the preceding equation becomes 1. Hence, we obtain li (xi ) = 1. But when li (x ) is evaluated at any other node, say, x j , one of the factors in the preceding equation is 0, and li (x j ) = 0, fori≠ j. Solution 4.1 Polynomial Interpolation 155 By the linear case equation, the polynomial that is sought is p(x) = 􏰋 x −1.25 􏰌3.7+􏰋 x −1.4 􏰌3.9 1.4 − 1.25 1.25 − 1.4 = 3.7+􏰋 3.9−3.7 􏰌(x −1.4) 1.25 − 1.4 = 3.7− 4(x −1.4) 3 ■ Kronecker Delta Property Lagrange Form As we can see, an interpolating polynomial can be written in a variety of forms, in- cluding the Newton form and the Lagrange form. The Newton form is probably the most convenient and efficient; however, conceptually, the Lagrange form has several advantages. We begin with the Lagrange form, since it may be easier to understand. Interpolating Polynomial: Lagrange Form Supposethatwewishtointerpolatearbitraryfunctionsatasetoffixednodesx0,x1,...,xn. We first define a system of n + 1 special polynomials of degree n known as cardinal polynomials in interpolation theory. These are denoted by l0 , l1 , . . . , ln and have the property li(xj)=δij=􏰤0 ifi≠ j 1 ifi=j Once these are available, we can interpolate any function f by the Lagrange form of the interpolation polynomial: 􏰄n i=0 􏰄n i=0 li (x ) f (xi ) (1) of degree at most n. Furthermore, when we evaluate pn at x j , we get f (x j ): pn (x ) = This function pn , being a linear combination of the polynomials li , is itself a polynomial pn(xj)= Thus, pn istheinterpolatingpolynomialforthefunction f atnodesx0,x1,...,xn.Itremains li(xj)f(xi)=lj(xj)f(xj)= f(xj) now only to write the formula for the cardinal polynomial li , which is Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 156 Chapter 4 Interpolation and Numerical Differentiation y 1.2 1 ,0 0.8 0.6 0.4 0.2 0 20.2 20.4 20.6 21 l3(x), and l4(x). Write out the cardinal polynomials appropriate to the problem of interpolating the following ,1 ,2 ,3 ,4 FIGURE 4.1 First few Lagrange cardinal polynomials EXAMPLE 2 Solution x 20.8 20.6 20.4 20.2 Figure 4.1 shows the first few Lagrange cardinal polynomials: l0(x), l1(x), l2(x), table, and give the Lagrange form of the interpolating polynomial: 0 0.2 0.4 0.6 0.8 1 x111 34 f(x) 2 −1 7 Using Equation (2), we have l0(x)=􏰀 4􏰁􏰀 􏰁=−18 x− 􏰀x − 1 􏰁(x − 1) 􏰋 1−1 1−1 1 􏰌 (x−1) 4 􏰀3 4􏰁3 x−1(x−1) 􏰋 1􏰌 l1(x)=􏰀 3􏰁􏰀 􏰁=16 x− (x−1) 1−1 1−1 3 􏰀4 3􏰁􏰀4 􏰁 x−1x−1 􏰋1􏰌􏰋1􏰌 l2(x)=􏰀 3􏰁􏰀 4􏰁=2 x− x− 1−11−134 34 Therefore, the interpolating polynomial in Lagrange’s form is p2(x) = −36􏰋x − 1􏰌(x − 1) − 16􏰋x − 1􏰌(x − 1) + 14􏰋x − 1􏰌􏰋x − 1􏰌 4334 Existence of Interpolating Polynomial ■ Existence of p(x) The Lagrange interpolation formula proves the existence of an interpolating polynomial for any table of values. There is another constructive way of proving this fact, and it leads to a different formula. Suppose that we have succeeded in finding a polynomial p that reproduces part of the table. Assume, say, that p(xi ) = yi for 0 ≦ i ≦ k. We shall attempt to add to p another term that enables the new polynomial to reproduce one more entry in the table. We consider p(x)+c(x −x0)(x −x1)···(x −xk) where c is a constant to be determined. This is surely a polynomial. It also reproduces the first k points in the table because p itself does so, and the added portion takes the value 0 Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. Inductive Reasoning ■ Theorem1 Degree of p(x) Uniqueness of p(x) at each of the points x0, x1, . . . , xk . (Its form is chosen for precisely this reason.) Now we adjust the parameter c so that the new polynomial takes the value yk+1 at xk+1. Imposing this condition, we obtain p(xk+1)+c(xk+1 −x0)(xk+1 −x1)···(xk+1 −xk)= yk+1 The proper value of c can be obtained from this equation because none of the factors xk+1 − xi , for 0 ≦ i ≦ k, can be zero. Remember our original assumption that the xi ’s are all distinct. This analysis is an example of inductive reasoning. We have shown that the process can be started and that it can be continued. Hence, the following formal statement has been partially justified: Two parts of this formal statement must still be established. First, the degree of the poly- nomial increases by at most 1 in each step of the inductive argument. At the beginning, the degree was at most 0, so at the end, the degree is at most n. Second, we establish the uniqueness of the polynomial p. Suppose that another poly- nomial q claims to accomplish what p does; that is, q is also of degree at most n and satisfies q(xi)=yi for0≦i≦n.Thenthepolynomialp−qisofdegreeatmostnandtakesthe value 0 at x0,x1,...,xn. Recall, however, that a nonzero polynomial of degree n can have at most n roots. We conclude that p = q, which establishes the uniqueness of p. Interpolating Polynomial: Newton Form In Example 2, we found the Lagrange form of the interpolating polynomial: p2(x) = −36􏰋x − 1􏰌(x − 1) − 16􏰋x − 1􏰌(x − 1) + 14􏰋x − 1􏰌􏰋x − 1􏰌 4334 It can be simplified to p2(x) = −79 + 349x − 38x2 66 Now, we learn that this polynomial can be written in another form called the nested Newton form: p2(x) = 2 + 􏰋x − 1􏰌􏰜36 + 􏰋x − 1􏰌(−38)􏰝 34 It involves the fewest arithmetic operations and is recommended for evaluating p2(x). It cannot be overemphasized that the Newton and Lagrange forms are just two different derivations for precisely the same polynomial. The Newton form has the advantage of easy extensibility to accommodate additional data points. The preceding discussion provides a method for constructing an interpolating polyno- mial. The method is known as the Newton algorithm, and the resulting polynomial is the Newton form of the interpolating polynomial. 4.1 Polynomial Interpolation 157 Theorem on Existence of Polynomial Interpolation Ifpointsx0,x1,...,xn aredistinct,thenforarbitraryrealvaluesy0,y1,...,yn,there is a unique polynomial p of degree at most n such that p(xi) = yi for 0≦ i ≦ n. Nested Newton’s Form for p2(x) Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 158 Chapter 4 Interpolation and Numerical Differentiation EXAMPLE 3 Solution Polynomials p0, p1, p2, p3, p4 Using the Newton algorithm, find the interpolating polynomial of least degree for this table: x 0 1 −1 2 −2 y −5 −3 −15 39 −9 In the construction, five successive polynomials appear; these are labeled p0 , p1 , p2 , p3 , and p4. The polynomial p0 is defined to be p0(x) = −5 The polynomial p1 has the form p1(x)= p0(x)+c(x−x0)=−5+c(x−0) The interpolation condition placed on p1 is that p1(1) = −3. Therefore, we have −5 + c(1−0)=−3.Hence,c=2,and p1 is p1(x) = −5 + 2x The polynomial p2 has the form p2(x)= p1(x)+c(x−x0)(x−x1)=−5+2x+cx(x−1) The interpolation condition placed on p2 is that p2(−1) = −15. Hence, we have −5 + 2(−1) + c(−1)(−1 − 1) = −15. This yields c = −4, so p2(x) = −5 + 2x − 4x(x − 1) The remaining steps for p3(x) are similar. The final result is the Newton form of the interpolating polynomial: p4(x)=−5+2x−4x(x−1)+8x(x−1)(x+1)+3x(x−1)(x+1)(x−2) ■ Later, we develop a better algorithm for constructing the Newton interpolating poly- nomial. Nevertheless, the method just explained is a systematic one and involves very little computation. An important feature to notice is that each new polynomial in the algorithm is obtained from its predecessor by adding a new term. Thus, at the end, the final polynomial exhibits all the previous polynomials as constituents. Nested Form Before continuing, let’s rewrite the Newton form of the interpolating polynomial for efficient evaluation. Write the polynomial p4 of Example 3 in nested form and use it to evaluate p4(3). We write p4 as p4(x) = −5 + x􏰀2 + (x − 1)􏰀 − 4 + (x + 1)􏰀8 + (x − 2)3􏰁􏰁􏰁 Therefore, we obtain p4(3) = −5+3􏰀2+2􏰀−4+4(8+3)􏰁􏰁 = 241 Adding a New Term EXAMPLE 4 Solution Nested Form p4(x) Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. Nested Multiplication p4(x) Another solution, also in nested form, is p4(x) = −5 + x􏰀4 + x􏰀 − 7 + x(2 + 3x)􏰁􏰁 from which we obtain p4(3) = −5 + 3􏰀4 + 3􏰀 − 7 + 3(2 + 3 · 3)􏰁􏰁 = 241 This form is obtained by expanding and systematic factoring of the original polynomial. It is also known as a nested form, and its evaluation is by nested multiplication. ■ To describe nested multiplication in a formal way (so that it can be translated into a pseudocode), consider a general polynomial in the Newton form. It might be pn(x)=a0 +a1(x−x0)+a2(x−x0)(x−x1)+··· + an(x − x0)(x − x1)···(x − xn−1) The nested form of pn(x) is pn(x) = a0 +(x −x0)􏰀a1 +(x −x1)􏰀a2 +···+(x −xn−2)􏰀an−1 +(x −xn−1)(an)􏰁···􏰁􏰁 The Newton interpolation polynomial can be written succinctly as n i−1 pn(x)=􏰄ai 􏰅(x−xj) (3) Newton Interpolation Polynomial Newton Polynomials i=0 j=0 Here 􏰢−1 (x − xj) is interpreted to be 1. Also, we can write it as j=0 where pn(x)= 􏰄n i=0 􏰅i − 1 (x − xj) (4) πi(x) = Figure 4.2 shows the first few Newton polynomials: π0(x), π1(x), π2(x), π3(x), π4(x), and π5(x). y 3 2.5 2 1.5 p0 1 j=0 ai πi(x) 4.1 Polynomial Interpolation 159 FIGURE 4.2 First few Newton polynomials 0.5 0 20.5 21 p1 p2 p3 p4 20.8 20.6 20.4 20.2 0 0.2 0.4 0.6 0.8 1 x Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 160 Chapter 4 Interpolation and Numerical Differentiation integer i,n; real t,v; real array (ai)0:n,(xi)0:n v ← an fori =n−1to0step−1 v ← v(t − xi ) + ai end for Evaluation of Interpolation Polynomial Pseudocode In evaluating p(t ) for a given numerical value of t , we naturally start with the innermost parentheses, forming successively the following quantities: v0 = an v1 =v0(t−xn−1)+an−1 v2 =v1(t−xn−2)+an−2 . vn =vn−1(t−x0)+a0 The quantity vn is now p(t). In the following pseudocode, a subscripted variable is not needed for vi . Instead, we can write Here,thearray(ai)0:n containsthen+1coefficientsoftheNewtonformoftheinterpolating polynomial (3) of degree at most n, and the array (xi )0:n contains the n + 1 nodes xi . Calculating Coefficients ai Using Divided Differences Weturnnowtotheproblemofdeterminingthecoefficientsa0,a1,...,an efficiently.Again we start with a table of values of a function f : x x0 x1 x2 ··· xn f(x) f(x0) f(x1) f(x2) ··· f(xn) Thepointsx0,x1,...,xn areassumedtobedistinct,butnoassumptionismadeabouttheir positions on the real line. Previously, we established that for each n = 0, 1, . . . , there exists a unique polynomial pn such that It was shown that pn can be expressed in the Newton form pn(x)=a0 +a1(x−x0)+a2(x−x0)(x−x1)+··· + an(x − x0)···(x − xn−1) A crucial observation about pn is that the coefficients a0, a1, . . . do not depend on n. In other words, pn is obtained from pn−1 by adding one more term, without altering the coefficients already present in pn−1 itself. This is because we began with the hope that pn could be expressed in the form pn(x)= pn−1(x)+an(x−x0)···(x−xn−1) and discovered that it was indeed possible. Unique Polynomial pn Adding One Term at a Time • The degree of pn is at most n. • pn(xi)= f(xi)fori =0,1,...,n. Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. ak Divided Difference of Order k a0, a1, a2 Divided Differences EXAMPLE 5 Solution Equations (5) can be solved for the ai ’s in turn, starting with a0. Then we see that a0 depends on f (x0), that a1 depends on f (x0) and f (x1), and so on. In general, ak depends on f (x0), f(x1),..., f(xk).Inotherwords,ak dependsonthevaluesof f atthenodesx0,x1,...,xk. The traditional notation is ak = f[x0,x1,...,xk] (7) Thisequationdefines f[x0,x1,...,xk].Thequantity f[x0,x1,...,xk]iscalledthedivided difference of order k for f. Notice also that the coefficients a0,a1,...,ak are uniquely determined by System (6). Indeed, there is no possible choice for a0 other than a0 = f (x0). Similarly, there is now no choice for a1 other than [ f (x1) − a0]/(x1 − x0) and so on. Using Equations (5), we see that the first few divided differences can be written as 4.1 Polynomial Interpolation 161 Awayofsystematicallydeterminingtheunknowncoefficientsa0,a1,...,an istoset x equal in turn to x0, x1, . . . , xn in the Newton form (3) and to write down the resulting equations:  f (x0) = a0 f(x1)=a0 +a1(x1 −x0)  f(x2)=a0 +a1(x2 −x0)+a2(x2 −x0)(x2 −x1) etc. The compact form of Equations (5) is k i−1 (5) f(xk)=􏰄ai 􏰅(xk −xj) (0≦k≦n) (6) i=0 j=0 a0 = f(x0) a1 = f(x1)−a0 = f(x1)− f(x0) x1 − x0 x1 − x0 a2 = f(x2)−a0 −a1(x2 −x0) = f(x2)− f(x1) − f(x1)− f(x0) x2 −x1 x1 −x0 x2 −x0 For the table: x 1 −4 0 f(x) 3 13 −23 (x2 −x0)(x2 −x1) Newton Form of Interpolating Polynomial determine the quantities f [x0], f [x0, x1], and f [x0, x1, x2]. We write out the system ofEquations (5) for this concrete case:  3=a0 13 = a0 + a1(−5) −23 = a0 + a1(−1) + a2(−1)(4) The solution is a0 = 3, a1 = −2, and a2 = 7. Hence, for this function, f[1] = 3, f [1,−4] = −2, and f [1,−4,0] = 7. ■ With this new notation, the Newton form of the interpolating polynomial takes the form n􏰤 i−1 􏰥 pn(x)=􏰄 f[x0,x1,...,xi]􏰅(x−xj) (8) i=0 j=0 Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 162 Chapter 4 Interpolation and Numerical Differentiation with the usual convention that 􏰢−1 (x − x j ) = 1. No􏰢tice that the coefficient of x n in pn is j=0 f[x0,x1,...,xn]becausethetermxn occursonlyin n−1(x−xj).Itfollowsthatif f is j=0 apolynomialofdegree≦n−1,then f[x0,x1,...,xn]=0. We return to the question of how to compute the required divided differences f [x0, x1, . . . , xk ]. From System (5) or System (6), it is evident that this computation can be performed recursively. We simply solve Equation (6) for ak as follows: k−1 k−1 i−1 f(xk)=ak 􏰅(xk −xj)+􏰄ai 􏰅(xk −xj) j=0 i=0 j=0 k−1 􏰅􏰄 􏰅 k−1 􏰄􏰅􏰅 f[x0,x1,...,xk] = i=0 j=0 k−1 (xk −xj) j=0 Using Algorithm (10), write out the divided differences formulas for f[x0,x1,x2],and f[x0,x1,x2,x3]. f[x0]= f(x0) f[x0,x1] = f(x1)− f[x0] x1 − x0 f[x0,x1,x2]= f(x2)− f[x0]− f[x0,x1](x2 −x0) (x2 − x0)(x2 − x1) and ak = j=0 f(xk)− i=0 k−1 j=0 ai i−1 (xk −xj) (xk −xj) Using Equation (7), we have i−1 f(xk)− f[x0,x1,...,xi] (xk −xj) (9) Computing the Divided Differences of f • Set f[x0]= f(x0). • Fork =1,2,...,n,compute f[x0,x1,...,xk]byEquation(9). (10) ■ Algorithm EXAMPLE 6 Solution First Four Divided Differences Operation Count f [x0 ], f [x0 , x1 ], f[x0,x1,x2,x3]= f(x3)− f[x0]− f[x0,x1](x3 −x0)− f[x0,x1,x2](x3 −x0)(x3 −x1) (x3 − x0)(x3 − x1)(x3 − x2) ■ Algorithm (10) is easily programmed and is capable of computing the divided dif- ferences f[x0], f[x0,x1],..., f[x0,x1,...,xn] at the cost of 1n(3n + 1) additions, 2 (n − 1)(n − 2) multiplications, and n divisions excluding arithmetic operations on the indices. Now a more refined method is presented for which the pseudocode requires only three statements (!) and costs only 1 n(n + 1) divisions and n(n + 1) additions. 2 Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 4.1 Polynomial Interpolation 163 At the heart of the new method is the following remarkable theorem: Recursive Property of Divided Differences The divided differences obey the formula f[x0,x1,...,xk] = f[x1,x2,...,xk]− f[x0,x1,...,xk−1] (11) xk − x0 ■ Theorem2 Recursive Property Proof Since f [x0, x1, . . . , xk ] was defined to be equal to the coefficient ak in the Newton form of the interpolating polynomial pk of Equation (3), we can say that f [x0, x1, . . . , xk ] is the coefficientofxk inthepolynomial pk ofdegree≦k,whichinterpolates f atx0,x1,...,xk. Similarly, f[x1,x2,...,xk]isthecoefficientofxk−1 inthepolynomialqofdegree≦k−1, whichinterpolates f atx1,x2,...,xk.Likewise, f[x0,x1,...,xk−1]isthecoefficientof xk−1 inthepolynomial pk−1 ofdegree≦k−1,whichinterpolates f atx0,x1,...,xk−1.The three polynomials pk, q, and pk−1 are intimately related. In fact, pk(x)=q(x)+ x−xk [q(x)−pk−1(x)] (12) xk − x0 To establish Equation (12), observe that the right side is a polynomial of degree at most k. Evaluatingitatxi,for1≦i≦k−1,resultsin f(xi): q(xi ) + xi − xk [q(xi ) − pk−1(xi )] = f (xi ) + xi − xk [ f (xi ) − f (xi )] xk − x0 xk − x0 Similarly, evaluating it at x0 and xk gives f (x0) and f (xk ), respectively. By the uniqueness of interpolating polynomials, the right side of Equation (12) must be pk (x ), and Equation (12) is established. Completing the argument to justify Equation (11), we take the coefficient of xk on both sides of Equation (12). The result is Equation (11). Indeed, we see that f [x1, x2, . . . , xk ] is the coefficient of xk−1 in q, and f [x0, x1, . . . , xk−1] is the coefficient of xk−1 in pk−1. ■ Notice that f[x0,x1,...,xk] is not changed if the nodes x0,x1,...,xk are permuted. Thus, for example, we have f[x0,x1,x2] = f[x1,x2,x0] The reason is that f [x0, x1, x2] is the coefficient of x2 in the quadratic polynomial interpolat- ing f at x0, x1, x2, whereas f [x1, x2, x0] is the coefficient of x2 in the quadratic polynomial interpolating f at x1,x2,x0. These two polynomials are, of course, the same! A formal statement in mathematical language is as follows: = f(xi) Invariance Theorem The divided difference f [x0, x1, . . . , xk ] is invariant under all permutations of the arguments x0,x1,...,xk. ■ Theorem3 Invariance Property Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 164 Chapter 4 Interpolation and Numerical Differentiation First Three Divided Differences Divided-Difference Table EXAMPLE 7 Solution Sincethevariablesx0,x1,...,xk andkarearbitrary,therecursiveFormula(11)can also be written as f[xi,xi+1,...,xj−1,xj]= f[xi+1,xi+2,...,xj]− f[xi,xi+1,...,xj−1] (13) xj −xi The first three divided differences are thus f[xi]= f(xi) f[xi,xi+1] = f[xi+1]− f[xi] xi+1 −xi f[xi,xi+1,xi+2]= f[xi+1,xi+2]−f[xi,xi+1] xi+2 −xi Using Formula (13), we can construct a divided-difference table for a function f . It is customary to arrange it as follows (here n = 3): x f[] f[,] f[,,] f[,,,] x0 x1 f [x1] form of the interpolating polynomial (3). Construct a divided-difference diagram for the function f given in the following table, and write out the Newton form of the interpolating polynomial. f[x0] x1302 2 f(x) 3 13 3 5 43 􏰀13 −3􏰁 1 f[x0,x1]= 􏰀4 􏰁 = After completion of column 3, the first entry in column 4 is f[x0,x1] f[x1,x2] f[x2,x3] f [x1, x2, x3] In the table, the coefficients along the top diagonal are the ones needed to form the Newton x2 f [x2] x3 f [x3] The first entry is 3−12 2 f[x0,x1,x2]= The complete diagram is f[x1,x2]− f[x0,x1] x2 − x0 1 − 1 1 = 6 2 = 0 − 1 3 x f[] f[,] f[,,] f[,,,] 1 2 1 3 Sample Divided- Difference Table 1 3 13 24 6 03 −5 1 −2 3 3 2 5 3 Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. f [x0, x1, x2] f [x0, x1, x2, x3] 3 −2 Thus, we obtain p3(x) = 3 + 1 (x − 1) + 1 (x − 1)􏰀x − 3 􏰁 − 2(x − 1)􏰀x − 3 􏰁x ■ 2322 Algorithms and Pseudocode Turningnexttoalgorithms,wesupposethatatablefor f isgivenatpointsx0,x1,...,xn and thatallthedivideddifferencesaij ≡ f[xi,xi+1,...,xj]aretobecomputed.Thefollowing pseudocode accomplishes this: 4.1 Polynomial Interpolation 165 integer i, j, n; real array (ai j )0:n×0:n , (xi )0:n fori =0ton ai0 ← f(xi) end for for j = 1 to n fori =0ton− j aij ←(ai+1,j−1−ai,j−1)/(xi+j −xi) end for end for Divided Differences Pseudocode Observe that the coefficients of the interpolating polynomial (3) are stored in the first row of the array (ai j )0:n×0:n . If the divided differences are being computed for use only in constructing the Newton form of the interpolation polynomial n i−1 pn(x)=􏰄ai 􏰅(x−xj) i=0 j=0 whereai = f[x0,x1,...,xi],thereisnoneedtostoreallofthem.Only f[x0], f[x0,x1],..., f [x0, x1, . . . , xn ] need to be stored. Whenaone-dimensionalarray(ai)0:n isused,thedivideddifferencescanbeoverwritten each time from the last storage location backward so that, finally, only the desired coefficients remain. In this case, the amount of computing is the same as in the preceding case, but the storage requirements are less. (Why?) Here is a pseudocode to do this: integer i, j, n; real array (ai )0:n , (xi )0:n fori =0ton ai ← f(xi) end for for j = 1 to n for i = n to j step −1 ai ←(ai −ai−1)/(xi −xi−j) end for end for Improved Pseudocode This algorithm is more intricate, and the reader is invited to verify it—say, in the case n = 3. For the numerical experiments suggested in the computer problems, the following two procedures should be satisfactory. The first is called Coef. It requires as input the number n and tabular values in the arrays (xi) and (yi). Remember that the number of points in Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 166 Chapter 4 Interpolation and Numerical Differentiation the table is n + 1. The procedure then computes the coefficients required in the Newton interpolating polynomial, storing them in the array (ai ). procedure Coef (n, (xi ), (yi ), (ai )) integer i, j, n; real array (xi )0:n , (yi )0:n , (ai )0:n fori =0ton ai ← yi end for for j = 1 to n for i = n to j step −1 ai ←(ai −ai−1)/(xi −xi−j) end for end for end procedure Coef Coef Pseudocode The second is function Eval. It requires as input the array (xi ) from the original table and the array (ai ), which is output from Coef. The array (ai ) contains the coefficients for the Newton form of the interpolation polynomial. Finally, as input, a single real value for t is given. The function then returns the value of the interpolating polynomial at t. real function Eval(n, (xi ), (ai ), t) integer i,n; real t,temp; real array (xi)0:n,(ai)0:n temp ← an fori =n−1to0step−1 temp ← (temp)(t − xi ) + ai end for Eval ← temp end function Eval Eval Pseudocode EXAMPLE 8 Solution Since the coefficients of the interpolating polynomial need be computed only once, we call Coef first, and then all subsequent calls for evaluating this polynomial are accomplished with Eval. Notice that only the t argument should be changed between successive calls to function Eval. Write pseudocode for the Newton form of the interpolating polynomial p for sin x at ten equidistant points in the interval [0, 1.6875]. The code finds the maximum value of | sin x − p(x )| over a finer set of equally spaced points in the same interval. If we take 10 points, including the ends of the interval, then we create 9 subintervals, each of length h = 0.1875. The points are then xi = ih for i = 0,1,...,9. After obtaining the polynomial, we divide each subinterval into four panels, and we evaluate | sin x − p(x )| at 37 points (called t in the pseudocode). These are tj = jh/4 for j = 0,1,...,36. Here is a suitable main program in pseudocode that calls routines Coef and Eval previously given: program Test Coef Eval integer j, k, n, jmax; real e, h, p, emax, pmax, tmax, real array (xi )0:n , (yi )0:n , (ai )0:n (Continued) Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 4.1 Polynomial Interpolation 167 n←9 h ← 1.6875/n for k = 0 to n xk ←kh yk ←sin(xk) end for call Coef (n, (xi ), (yi ), (ai )) output (ai ); emax ← 0 for j = 0 to 4n t ← jh/4 p ← Eval(n,(xi)n,(ai)n,t) e ← |sin(t) − p| output j, t, p, e if e > emax then
jmax ← j;tmax ←t; pmax ← p;emax ←e end if
end for
output jmax, tmax, pmax, emax end program Test Coef Eval
Test Coef Eval
Pseudocode
f (x) Approximation with Basis Functions
Linear System
Monomials
The first coefficient in the Newton form of the interpolating polynomial is 0 (why?), and the others range in magnitude from approximately 0.99 to 0.18 × 10−5 . The deviation between sin x and p(x) is practically zero at each interpolation node. (Because of roundoff errors, they are not precisely zero.) From the computer output, the largest error is at jmax = 35, where sin(1.64062 5) ≈ 0.99756 31 with an error of 1.19 × 10−7 . ■
Vandermonde Matrix
Another view of interpolation is that for a given set of n + 1 data points (x0, y0), (x1, y1), …,(xn,yn),wewanttoexpressaninterpolatingfunction f(x)asalinearcombinationof a set of basis functions φ0,φ1,φ2,…,φn so that
f (x) ≈ c0φ0(x) + c1φ1(x) + c2φ2(x) + ··· + cnφn(x)
Here the coefficients c0 , c1 , c2 , . . . , cn are to be determined. We want the function f to
interpolate the data (xi , yi ). This means that we have linear equations of the form f(xi)=c0φ0(xi)+c1φ1(xi)+c2φ2(xi)+···+cnφn(xi)= yi
for each i = 0,1,2,…,n. This is a system of linear equations Ac = y
Here,theentriesinthecoefficientmatrixAaregivenbyaij =φj(xi),whichisthevalueof the jth basis function evaluated at the ith data point. The right-hand side vector y contains the known data values yi , and the components of the vector c are the unknown coefficients ci . Systems of linear equations are discussed in Chapters 2 and 8.
Polynomials are the simplest and most common basis functions. The natural basis for Pn consists of the monomials
φ0(x) = 1, φ1(x) = x, φ2(x) = x2, …, φn(x) = xn Figure 4.3 (p. 168) shows the first few monomials: 1, x, x2, x3, x4, and x5.
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

168 Chapter 4
Interpolation and Numerical Differentiation
1 0.8 0.6 0.4 0.2 0 20.2 20.4 20.6 20.8
1
y
x
x3
x4
x2
FIGURE 4.3
First few monomials
Vandermonde Matrix Linear System
21 x 21 20.8 20.6 20.4 20.2 0 0.2 0.4 0.6 0.8 1
Consequently, a given polynomial pn has the form pn(x)=c0 +c1x+c2x2 +···+cnxn
The corresponding linear system Ac = y has the form
Chebyshev Recursive Relation
First Six Chebyshev Polynomials
The coefficient matrix is called a Vandermonde matrix. It can be shown that this matrix is invertibleprovidedthatthepointsx0,x1,x2,…,xn aredistinct.Sowecan,intheory,solve the system for the polynomial interpolant. Although the Vandermonde matrix is invertible, it is ill-conditioned as n increases. For large n, the monomials are less distinguishable from one another, as shown in Figure 4.3. Moreover, the columns of the Vandermonde become nearly linearly dependent in this case. High-degree polynomials often oscillate wildly and are highly sensitive to small changes in the data.
As Figures 4.1–4.3 show, we have discussed three choices for the basis functions: the Lagrange cardinal polynomials li (x ), the Newton polynomials πi (x ), and the monomials. It turns out that there are better choices for the basis functions; namely, the Chebyshev polynomials have more desirable features.
The Chebyshev polynomials play an important role in mathematics because they have several special properties such as the recursive relation
􏰈T0(x)=1, T1(x)=x Ti(x) =2xTi−1(x)−Ti−2(x)
for i = 2, 3, 4, and so on. Thus, the first six Chebyshev polynomials are T0(x)=1, T1(x)=x, T2(x)=2×2 −1, T3(x)=4×3 −3x T4(x)=8×4 −8×2 +1, T5(x)=16×5 −20×3 +5x
These curves for these polynomials, as is shown in Figure 4.4, are quite different from one another. The Chebyshev polynomials are usually employed on the interval [−1, 1]. With changes of variable, they can be used on any interval, but the results will be more complicated.
 1 1
x 0 x1 x 2
x 02 x12 x 2 2
· · · ··· · · ·
x 0n   c 0   y 0  x1n c1  y1 
  1
 . . . … .  .   . 
x 2n     c 2   =   y 2   1xnxn2···xncn yn
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

FIGURE 4.4
First few Chebyshev polynomials
T4
21 x
y
4.1
Polynomial Interpolation 169
1 0.5 0 20.5
T0 T1
T2 T3
Inverse Interpolation
T5 21 20.5 0 0.5 1
One of the important properties of the Chebyshev polynomials is the equal oscillation property. Notice in Figure 4.4 that successive extreme points of the Chebyshev polynomi- als are equal in magnitude and alternate in sign. This property tends to distribute the error uniformly when the Chebyshev polynomials are used as the basis functions. In polynomial interpolation for continuous functions, it is particularly advantageous to select as the inter- polation points the roots or the extreme points of a Chebyshev polynomial. This causes the maximum error over the interval of interpolation to be minimized. An example of this is given in Section 4.2. In Section 9.2, we discuss Chebyshev polynomials in more detail.
Inverse Interpolation
A process called inverse interpolation is often used to approximate an inverse function. Supposethatvaluesyi = f(xi)havebeencomputedatx0,x1,…,xn.Usingthetable
y y0 y1 ··· yn
x x0 x1 ··· xn we form the interpolation polynomial
n i−1 p(y)=􏰄ci 􏰅(y−yj) i=0 j=0
The original relationship, y = f (x), has an inverse, under certain conditions. This inverse is being approximated by x = p(y). Coef and Eval can be used to carry out the inverse interpolation by reversing the arguments x and y in the calling sequence for Coef.
Inverse interpolation can be used to find where a given function f has a root or zero. This means inverting the equation f (x) = 0. We propose to do this by creating a table of values ( f (xi ), xi ) and interpolating with a polynomial, p. Thus, we obtain p(yi ) = xi . The points xi should be chosen near the unknown root, r. The approximate root is then given by r ≈ p(0). See Figure 4.5 for an example of function y = f (x) and its inverse function x = g(y) with the root r = g(0).
For a concrete case, let the table of known values be
y −0.57892 00 −0.36263 70 −0.18491 60 −0.03406 42 0.09698 58
Finding a Root
EXAMPLE 9
x 1.0 2.0
Find the inverse interpolation polynomial.
3.0 4.0 5.0
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

170
Chapter 4 Interpolation and Numerical Differentiation
yx y 5 f(x)
x 5 g(y)
FIGURE 4.5
Function y = f (x) and inverse function x = g(y)
Solution
Inverse Interpolation Polynomial
0
r
f(r) 5 0
r 5 g(0) xy
0
Neville: 1st Recursive Relation
The nodes in this problem are the points in the row of the table headed y, and the function values being interpolated are in the x row. The resulting polynomial is
p(y) = 0.25y4 + 1.2y3 + 3.69y2 + 7.39y + 4.24747 0086
and p(0) = 4.24747 0086. Only the last coefficient is shown with all the digits carried in
the calculation, as it is the only one needed for the problem at hand. ■
Polynomial Interpolation by Neville’s Algorithm
Another method of obtaining a polynomial interpolant from a given table of values x x0 x1 ··· xn
y y0 y1 ··· yn
was given by Neville. It builds the polynomial in steps, just as the Newton algorithm does. The constituent polynomials have interpolating properties of their own.
Let Pa,b,…,s(x) be the polynomial interpolating the given data at a sequence of nodes xa,xb,…,xs. We start with constant polynomials Pi(x) = f(xi). Selecting two nodes xi and x j with i > j , we define recursively
Pu,…,v(x)=􏰋x−xj 􏰌Pu,…,j−1,j+1,…,v(x)+􏰋xi −x􏰌Pu,…,i−1,i+1,…,v(x) xi −xj xi −xj
Using this formula repeatedly, we can create an array of polynomials:
Neville: 2nd Recursive Relation
Here, each successive polynomial can be determined from two adjacent polynomials in the previous column.
We can simplify the notation by letting
Si j (x) = Pi− j,i− j+1,…,i−1,i (x)
where Si j (x ) for i ≧ j denotes the interpolating polynomial of degree j on the j + 1 nodes xi− j , xi− j+1, . . . , xi−1, xi . Next we can rewrite the recurrence relation above as
Si j (x) = 􏰋 x − xi− j 􏰌Si, j−1(x) + 􏰋 xi − x 􏰌Si−1, j−1(x) xi −xi−j xi −xi−j
x0 P0(x)
x1 P1(x)
x2 P2(x)
x3 P3(x)
x4 P4(x)
P0,1(x)
P1,2(x) P0,1,2(x)
P2,3(x) P1,2,3(x) P0,1,2,3(x)
P3,4(x) P2,3,4(x) P1,2,3,4(x) P0,1,2,3,4(x)
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

So the displayed array becomes
x0 S00(x)
x1 S10(x)
x2 S20(x)
x3 S30(x)
x4 S40(x)
S11(x)
S21(x) S22(x)
S31(x) S32(x) S33(x)
S41(x) S42(x) S43(x) S44(x)
4.1 Polynomial Interpolation 171
Neville: 3rd Recursive Relation
■ Theorem4
Proof
Pj(x)=􏰋x−xi−j 􏰌Pj−1(x)+􏰋 xi −x 􏰌Pj−1(x) (14) i x −x i x −x i−1
In this equation, the superscripts are simply indices, not exponents. The range of j is 1 ≦ j ≦ n, while that of i is j ≦ i ≦ n. Formula (14) is seen again, in slightly different form, in the theory of B splines in Section 6.3.
The interpolation properties of these polynomials are given in the next result.
We use induction on j. When j = 0, the assertion in Equation (15) reads P0(x )= y (0≦i≦k≦i≦n)
To prove some theoretical results, we change the notation by making the superscript
the degree of the polynomial. At the beginning, we define constant polynomials (i.e., poly-
nomials of degree 0) as P0(x) = y for 0≦ i ≦ n. Then we define ii
i i−j i i−j
Interpolation Properties
The polynomials P j defined above interpolate as follows: i
P j (x ) = y (0 ≦ i − j ≦ k ≦ i ≦ n) (15) ikk
ikk
In other words, P0(x ) = y , which is true by the definition of P0. iii i
Now assume, as an induction hypothesis, that for some j ≧ 1,
P j−1(x ) = y (0 ≦ i − j + 1 ≦ k ≦ i ≦ n)
ikk
To prove the next case in Equation (15), we begin by verifying the two extreme cases for k,
namely, k = i − j and k = i. We have, by Equation (14),
Pj(x )=􏰋xi−xi−j􏰌Pj−1(x ) i i−j x −x i−1 i−j
0 ≦ i − 1 − j + 1 ≦ i − j ≦ i − 1 ≦ n. In the same way, we compute
Pj(x)=􏰋xi −xi−j􏰌Pj−1(x) iix−xii
Pj(x)=􏰋xk−xi−j􏰌Pj−1(x)+􏰋xi−xk 􏰌Pj−1(x) i k x−x i k x−x i−1 k
i i−j i i−j
i i−j =Pj−1(x )=y
i−1 i−j i−j
The last equality is justified by the induction hypothesis. It is necessary to observe that
i i−j
= P j−1(x ) = y
iii
Here, in using the induction hypothesis, observe that 0 ≦ i − j + 1 ≦ i ≦ i ≦ n.
Now let i − j < k < i. Then Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 172 Chapter 4 Interpolation and Numerical Differentiation Inthisequation,Pj−1(x )= y bytheinductionhypothesis,because0≦i−j+1≦k≦i≦n. ikk Likewise,Pj−1(x)=y because0≦i−1−j+1≦k≦i−1≦n.Thus,wehave i−1 k k Pj(x)=􏰋xk−xi−j􏰌y+􏰋xi−xk 􏰌y=y ikx−xkx−xkk ■ An algorithm follows in pseudocode to evaluate Pn(t) when a table of values is given: ii−j ii−j integer i, j, n; real array (xi )0:n , (yi )0:n , (Si j )0:n×0:n fori =0ton Si0 ← yi end for end for return Snn for j = 1 to n for i = j to n Sij ←􏰗(t−xi−j)Si,j−1 +(xi −t)Si−1,j−1􏰘􏰹(xi −xi−j) end for Evaluate Pn(t) Pseudocode We begin the algorithm by finding the node nearest the point t at which the evaluation is to be made. In general, interpolation is more accurate when this is done. Interpolation of Bivariate Functions The methods we have discussed for interpolating functions of one variable by polynomials extend to some cases of functions of two or more variables. An important case occurs when a function (x, y) 􏰷→ f (x, y) is to be approximated on a rectangle. This leads to what is known as tensor-product interpolation. Suppose the rectangle is the Cartesian product of two intervals: [a, b] × [α, β]. That is, the variables x and y run over the intervals [a, b], and [α, β], respectively. Select n nodes xi in[a,b],anddefinetheLagrangianpolynomials l i ( x ) = 􏰅n x − x j xi −xj j ≠ i j=1 Similarly,weselectmnodesyi in[α,β]anddefine ( 1 ≦ i ≦ n ) ( 1 ≦ i ≦ m ) P(x, y) = is a polynomial in two variables that interpolates f at the grid points (xi , y j ). There are nm such points of interpolation. l i ( y ) = 􏰅m j ≠ i j=1 y − y j yi −yj Tensor-Product Interpolation 􏰄n 􏰄m i=1 j=1 f (xi , yj )li (x)lj (y) Then the function Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. The proof of the interpolation property is quite simple because li (xq ) = lj(yp) = δjp. Consequently, we have 􏰄n 􏰄m i=1 j=1 􏰄n 􏰄m i=1 j=1 δi q and P(xq,yp) = = f(xi,yj)li(xq)lj(yp) f(xi,yj)δiqδjp = f(xq,yp) The same procedure can be used with spline interpolants (or indeed any other type of function). Summary 4.1 • The Lagrange form of the interpolation polynomial is 􏰄n i=0 l i ( x ) = 􏰅n 􏰋 x − x j 􏰌 ( 0 ≦ i ≦ n ) xi −xj j ≠ i j=0 that obey the Kronecker delta equation li(xj)=δij=􏰤0 ifi≠ j 1 ifi=j • The Newton form of the interpolation polynomial is n i−1 pn(x)=􏰄ai 􏰅(x−xj) i=0 j=0 with divided differences ai = f[x0,x1,...,xi] = f[x1,x2,...,xi]− f[x0,x1,...,xi−1] xi −x0 These are two different forms of the unique polynomial p of degree n that interpolates atableofn+1pairsofpoints(xi, f(xi))for0≦i≦n. • We can illustrate this with a small table for n = 2: x x0 x1 x2 f(x) f(x0) f(x1) f(x2) The Lagrange interpolating polynomial is p2(x)= (x−x1)(x−x2) f(x0)+ (x−x0)(x−x2) f(x1) (x0 − x1)(x0 − x2) (x1 − x0)(x1 − x2) + (x−x0)(x−x1) f(x2) (x2 − x0)(x2 − x1) Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 4.1 Polynomial Interpolation 173 with cardinal polynomials pn (x ) = li (x ) f (xi ) 174 Chapter 4 Interpolation and Numerical Differentiation a1. Use the Lagrange interpolation process to obtain a poly- nomial of least degree that assumes these values: x0234 y 7 11 28 63 2. (Continuation) Rearrange the points in the table of the preceding problem and find the Newton form of the interpolating polynomial. Show that the polynomi- als obtained are identical, although their forms may differ. a3. For the four interpolation nodes −1, 1, 3, 4, what are the li Functions (2) required in the Lagrange interpolation procedure? Draw the graphs of these four functions to show their essential properties. 4. Verifythatthepolynomials p(x) = 5x3 −27x2 +45x −21 q(x) = x4 −5x3 +8x2 −5x +3 interpolate the data x1234 y 2 1 6 47 5. and explain why this does not violate the uniqueness part of the theorem on existence of polynomial interpo- lation. Verifythatthepolynomials p(x) = 3+2(x −1)+4(x −1)(x +2) q(x) = 4x2 + 6x − 7 are both interpolating polynomials for the following ta- ble, and explain why this does not violate the unique- ness part of the existence theorem for polynomial inter- polation. x 1 −2 0 y 3 −3 −7 Find the polynomial p of least degree that takes these values: p(0) = 2, p(2) = 4, p(3) = −4, p(5) = 82. Use divided differences to get the correct polynomial. It is not necessary to write the polynomial in the standard form a0 + a1 x + a2 x 2 + · · ·. Complete the following divided-difference tables, and use them to obtain polynomials of degree 3 that inter- polate the function values indicated: Clearly, p2(x0) = f (x0), p2(x1) = f (x1), and p2(x2) = f (x2). Next, we form the divided-difference table: x0 x1 f (x1) f(x0) f[x1,x2] x2 f (x2) Using the divided-difference entries from the top diagonal, we have pn(x)= f(x0)+ f[x0,x1](x−x0)+ f[x0,x1,x2](x−x0)(x−x1) Again,itcanbeeasilyshownthatp2(x0)= f(x0),p2(x1)= f(x1),andp2(x)= f(x2). • We can use inverse polynomial interpolation to find an approximate value of a root roftheequation f(x)=0fromatableofvalues(xi,yi)for1≦i≦n.Hereweare assuming that the table values are in the vicinity of this zero of the function f . Flipping the table values, we use the reversed table values (yi , xi ) to determine the interpolating polynomial called pn(y). Now evaluating it at 0, we find a value that approximates the desired zero, namely, r ≈ pn(0) and f (pn(0)) ≈ f (r) = 0. • Other advanced polynomial interpolation methods discussed are Neville’s algorithm and bivariate function interpolation. Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 6. 7. f[x0,x1] f [x0, x1, x2] Exercises 4.1 aa. x f[] f[, ] f[, , ] −1 2 1−4 2 36 f[, , , ] a 12. 4.1 Polynomial Interpolation 175 Thepolynomialp(x)=x4−x3+x2−x+1hasthe following values: x −2 −1 0 1 2 3 p(x) 31 5 1 1 11 61 Find a polynomial q that takes these values: x −2 −1 0 1 2 3 q(x) 31 5 1 1 11 30 Hint: This can be done with little work. Usethedivided-differencemethodtoobtainapolynomial of least degree that fits the values shown. aa.x 0 1 2−13 y −1 −1 −1 −7 5 b.x 1 3 −2 4 5 y 2 6 −1 −4 2 Findtheinterpolatingpolynomialforthesedata: 2 b. x f[] f[,] f[,,] f[,,,] −1 2 1 −4 3 46 53.5 4 99.5 Write the final polynomials in a form most efficient for computing. a8. Findaninterpolatingpolynomialforthistable: x 1 2 2.5 3 4 y −1 −1 3 4 25 9. Giventhedata x01246 f(x) 1 9 23 93 259 do the following. 5 10 13. a14. 15. a16. 17. a18. a 19. x f (x) 1.0 2.0 −1.5 −0.5 2.5 3.0 4.0 0.0 0.5 1.5 3 32 3 Itissuspectedthatthetable x −2 −1 0 1 2 3 y 1 4 11 16 13 −4 comes from a cubic polynomial. How can this be tested? Explain. Thereexistsauniquepolynomialp(x)ofdegree2orless such that p(0) = 0, p(1) = 1, and p′(α) = 2 for any value of α between 0 and 1 (inclusive) except one value of α, say, α0. Determine α0, and give this polynomial for α ≠ α 0 . Determine by two methods the polynomial of degree 2 or less whose graph passes through the points (0, 1.1), (1, 2), and (2, 4.2). Verify that they are the same. Developthedivided-differencetablefromthegivendata. Write down the interpolating polynomial, and rearrange it for fast computation without simplifying. x01325 f(x) 2 1 5 6 −183 Checkpoint: f[1,3,2,5]=−7. Letf(x)=x3+2x2+x+1.Findthepolyno- mial of degree 4 that interpolates the values of f at aa. ab. 10. a. b. Constructthedivided-differencetable. UsingNewton’sinterpolationpolynomial,findanap- proximation to f (4.2). Hint: Use polynomials starting with 93 and involving factors (x − 4). Construct Newton’s interpolation polynomial for the data shown. x0234 y 7 11 28 63 Withoutsimplifyingit,writethepolynomialobtained in nested form for easy evaluation. 11. From census data, the approximate population of the United States was 150.7 million in 1950, 179.3 million in 1960, 203.3 million in 1970, 226.5 million in 1980, and 249.6 million in 1990. Using Newton’s interpolation polynomial for these data, find an approximate value for the population in 2000. Then use the polynomial to esti- mate the population in 1920 based on these data. What conclusion should be drawn? Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 176 Chapter 4 Interpolation and Numerical Differentiation 20. 21. a22. 23. 24. a 25. 26. a 27. a 28. 0.30103 0.47712 0.54407 0.60206 x = −2,−1,0,1,2. Find the polynomial of degree 2 that interpolates the values of f at x = −1, 0, 1. Withoutusingadivided-differencetable,deriveandsim- plify the polynomial of least degree that assumes these values: x −2 −1 0 1 2 y 2 14 4 2 2 (Continuation) Find a polynomial that takes the values shown in the preceding problem and has at x = 3 the value 10. Hint: Add a suitable polynomial to the p(x) of the previ- ous problem. Findapolynomialofleastdegreethattakesthesevalues: x 1.73 1.82 2.61 5.22 8.26 y 0 0 7.8 0 0 Hint: Rearrange the table so that the nonzero value of y is the last entry, or think of some better way. Formadivided-differencetableforthefollowingandex- plain what happened. x1231 y3557 Simple polynomial interpolation in two dimensions is not always possible. For example, suppose that the following data are to be represented by a polynomial of first degree inxandy,p(t)=a+bx+cy,wheret=(x,y): t (1,1) (3,2) (5,3) f(t) 3 2 6 Show that it is not possible. Consider a function f (x) such that f (2) = 1.5713, f (3) = 1.5719, f (5) = 1.5738, and f (6) = 1.5751. Estimate f (4) using a second-degree interpolating poly- nomial and a third-degree polynomial. Round the final results off to four decimal places. Is there any advantage here in using a third-degree polynomial? Use inverse interpolation to find an approximate value of x such that f (x) = 0 given the following table of values for f . Look into what happens and draw a conclusion. x −2 −1 1 2 3 f(x) −31 5 1 11 61 Find a polynomial p(x) of degree at most 3 such that p(0)=1, p(1)=0, p′(0)=0,and p′(−1)=−1. From a table of logarithms, we obtain the following val- ues of log x at the indicated tabular points: x 1 1.5 2 3 log x Form a divided-difference table based on these values. Interpolate for log 2.4 and log 1.2 using third-degree in- terpolation polynomials in Newton form. Showthatthedivideddifferencesarelinearmaps;thatis, (αf +βg)[x0,x1,...,xn] = αf[x0,x1,...,xn] + βg[x0, x1, . . . , xn ] Hint: Use induction. Show that another form for the polynomial pn of degree at most n that takes values y0,y1,...,yn at abscissas x0,x1,...,xn is n i−1 􏰄 f[xn,xn−1,...,xn−i]􏰅(x −xn−j) i=0 j=0 Use the uniqueness of the interpolating polynomial to verify that n n i−1 􏰄 f (xi )li (x) = 􏰄 f [x0, x1, . . . , xi ] 􏰅(x − x j ) i=0 i=0 j=0 (Continuation)Showthatthefollowingexplicitformula is valid for divided differences: 􏰄n f[x0,x1,...,xn]= 0 0.17609 3.5 4 29. 30. 31. 32. 33. 34. 35. 36. 􏰅n j ≠ i (xi −xj)−1 Hint: If two polynomials are equal, the coefficients of xn f(xi) li(x) = 1 for the case n = 1. Then establish the result for arbitrary values of n. WritetheLagrangeform(1)oftheinterpolatingpolyno- mial of degree at most 2 that interpolates f (x) at x0, x1, and x2, where x0 < x1 < x2. (Continuation)WritetheNewtonformoftheinterpolat- ing polynomial p2(x), and show that it is equivalent to the Lagrange form. (Continuation) Show directly that p′′(x)=2f[x ,x ,x ] 2 012 in each are equal. Verifydirectlythat 􏰄n i=0 i = 0 j=0 Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole deemed that any suppressed content does not materially affect the overall learning experience. or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 37. 38. (Continuation) Show directly for uniform spacing h = x1−x0=x2−x1that 41. a 42. 43. 4.1 Polynomial Interpolation 177 Verify directly that for any three distinct points x0,x1, and x2, f[x0,x1,x2] = f[x2,x0,x1] = f[x1,x2,x0] Compare this argument to the one in the text. Let p be a polynomial of degree n. What is p[x0,x1,...,xn+1]? Show that if f is continuously differentiable on the in- terval [x0, x1], then f [x0, x1] = f ′(c) for some c in (x0, x1). If f is a polynomial of degree n, show that in a divided-difference table for f, the nth column has a single constant value—a column containing entries f[xi,xi+1,...,xi+n]. Determine whether the following assertion is true or false. If x0, x1, . . . , xn are distinct, then for arbitrary real val- ues y0, y1, ..., yn, there is a unique polynomial pn+1 of degree≦n+1suchthatpn+1(xi)=yi foralli=0, 1,...,n. Show that if a function g interpolates the function f at x0,x1,...,xn−1 and h interpolates f at x1,x2,...,xn, a 39. cient [s!]/[(s − m)! m!], and s!/(s − m)! = s(s − 1)(s − 2)···(s−m+1)becauses canbeanyrealnumberand m! has the usual definition because m is an integer. (Continuation) From the following table of values of lnx, interpolate to obtain ln2.352 and ln2.387 using the Newton forward-difference form of the interpolating polynomial: f[x0,x1]= 􏰱f0, h, f[x0,x1,x2]= 􏰱2 f0 2h2 2 where􏰱fi=fi+1−fi,􏰱fi=􏰱fi+1−􏰱fi,and fi = f(xi). (Continuation) Establish Newton’s forward-difference form of the interpolating polynomial with uniform spacing 􏰋s􏰌 􏰋s􏰌 p2(x) = f0 + 􏰱f0 + 44. 45. 46. 47. 􏰱2 f0 where x = x0 + sh. Here, m is the binomial coeffi- 12 􏰀s 􏰁 a 􏰱2 f −0.00001 −0.00002 −0.00002 [g(x) − h(x)] then 􏰋 x0 − x 􏰌 xn − x0 x f(x) 2.35 0.85442 2.36 0.85866 2.37 0.86289 2.38 0.86710 2.39 0.87129 􏰱f 0.00424 0.00423 0.00421 0.00419 g(x) + interpolates f at x0,x1,...,xn. a 40. a1. 􏰣􏰣1x0 x2􏰣􏰣 􏰣 0􏰣 Using the correctly rounded values ln 2.352 ≈ 0.85527 and ln 2.387 ≈ 0.87004, show that the forward-difference formula is more accurate near the top of the table than it is near the bottom. Count the number of multiplications, divisions, and additions/subtractions in the generation of the divided- difference table that has n + 1 points. Test the procedure given in the text for determining the Newton form of the interpolating polynomial. For exam- ple, consider this table: 􏰣􏰣1x0 f 0 􏰣􏰣 􏰣􏰣1x1 f 1 􏰣􏰣 􏰣1x2 f2 􏰣 (Vandermonde Determinant) Using fi = f (xi ), show the following: a. f[x0,x1]= f1 􏰣 􏰣􏰣 1 x0􏰣􏰣 􏰣􏰣 1 􏰣􏰣 􏰣1 f 􏰣􏰣 0 􏰣􏰣 􏰣1 x1􏰣 b. f[x0,x1,x2]= x 1 2 3 −4 5 y 2 48 272 1182 2262 Find the interpolating polynomial and verify that p(−1) = 12. 􏰣􏰣1x1 x 12 􏰣􏰣 􏰣1x2 x2 􏰣 Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. Computer Exercises 4.1 178 Chapter 4 Interpolation and Numerical Differentiation 2. Find the polynomial of degree 10 that interpolates the function arctan x at 11 equally spaced points in the in- terval [1,6]. Print the coefficients in the Newton form of the polynomial. Compute and print the difference be- tween the polynomial and the function at 33 equally spaced points in the interval [0, 8]. What conclusion can be drawn? 3. Write a simple program using procedure Coef that inter- polates ex by a polynomial of degree 10 on [0,2] and then compares the polynomial to exp at 100 points. 4. Use as input data to procedure Coef the annual rainfall in your town for each of the last five years. Using func- tion Eval, predict the rainfall for this year. Is the answer reasonable? 5. A table of values of a function f is given at the points xi =i/10for0≦i≦100.Inordertoobtainagraphof f with the aid of an automatic plotter, the values of f are required at the points zi = i/20 for 0 ≦ i ≦ 200. Write a procedure to do this, using a cubic interpolating polyno- mial with nodes xi , xi+1, xi+2, and xi+3 to compute f at 1(xi+1 + xi+2). For z1 and z199, use the cubic polyno- 2 mial associated with z3 and z197, respectively. Compare this routine to Coef for a given function. 6. WriteroutinesanalogoustoCoefandEvalusingtheLa- grange form of the interpolation polynomial. Test on the example given in this section at 20 points with h/2. Does the Lagrange form have any advantage over the Newton form? 7. (Continuation) Design and carry out a numerical exper- iment to compare the accuracy of the Newton and La- grange forms of the interpolation polynomials at values throughout the interval [x0, xn]. 8. RewriteandtestroutinesCoefandEvalsothatthearray (ai ) is not used. Hint: When the elements in the array (yi ) are no longer needed, store the divided differences in their places. 9. Write a procedure for carrying out inverse interpolation to solve equations of the form f (x) = 0. Test it on the introductory example at the beginning of this chapter. 10. ForExample8,comparetheresultsfromyourcodewith that in the text. Redo using linear interpolation based on the 10 equidistant points. How do the errors compare at intermediate points? Plot curves to visualize the dif- ference between linear interpolation and a higher-degree polynomial interpolation. 11. Use mathematical software such as MATLAB, Maple, or Mathematica to find an interpolation polynomial for the points (0, 0), (1, 1), (2, 2.001), (3, 3), (4, 4), (5, 5). Evaluate the polynomial at the point x = 14 or x = 20 to show that slight roundoff errors in the data can lead to suspicious results in extrapolation. 12. Use symbolic mathematical software such as MATLAB, Maple, or Mathematica to generate the interpolation poly- nomial for the data points in Example 3. Plot the polyno- mial and the data points. 13. (Continuation.) Repeat these instructions using Exam- ple 7. 14. Carry out the details in Example 8 by writing a computer program that plots the data points and the curve for the interpolation polynomial. 15. (Continuation.) Repeat the instructions using Example 9. 16. Using mathematical software, carry out the details and verify the results in the introductory example to this chapter. 17. (Pade ́Interpolation)Findarationalfunctionoftheform g(x)= a+bx 1+cx that interpolates the function f(x) = arctan(x) at the pointsx0 =1,x1 =2,andx2 =3.Onthesameaxes, plot the graphs of f and g, using dashed and dotted lines, respectively. 4.2 Errors in Polynomial Interpolation Introduction When a function f is approximated on an interval [a,b] by means of an interpolating polynomial p, the discrepancy between f and p will (theoretically) be 0 at each node of interpolation. A natural expectation is that the function f is well approximated at all intermediate points and that as the number of nodes increases, this agreement will become better and better. Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. EXAMPLE 1 In the history of numerical mathematics, it was a severe shock to realize that this expectation was ill-founded! Of course, if the function being approximated is not required to be continuous, then there may be no agreement at all between p(x) and f (x) except at the nodes. Consider these five data points: (0, 8), (1, 12), (3, 2), (4, 6), (8, 0). Construct and plot the interpolation polynomial using the two outermost points. Repeat this process by adding one additional point at a time until all the points are included. What conclusions can you draw? y 35 30 4.2 Errors in Polynomial Interpolation 179 25 p4 20 15 10 5 p1 p3 FIGURE 4.6 Interpolant polynomials over datapoints Solution Poorly Fitting Polynomials Dirichlet Function Numerical Experiment p2 25 x 012345678 The first interpolation polynomial is the line between the outermost points (0, 8) and (8, 0). Then we added the points (3, 2), (4, 6), and (1, 12) in that order and plotted a curve for each additional point. All of these polynomials are shown in Figure 4.6. We were hoping for a smooth curve going through these points without wide fluctuations, but this did not happen! (Why?) It may seem counterintuitive, but as we added more points, the situation became worse instead of better! The reason for this comes from the nature of high-degree polynomials. A polynomial of degree n has n zeros. If all of these zero points are real, then the curve crosses the x-axis n times. The resulting curve must make many turns for this to happen, resulting in wild oscillations. In Chapter 6, we discuss fitting the data points with spline curves. ■ Dirichlet Function As a pathological example, consider the so-called Dirichlet function f , defined to be 1 at each irrational point and 0 at each rational point. If we choose nodes that are rational numbers, then p(x) ≡ 0 and f (x)− p(x) = 0 for all rational values of x, but f (x)− p(x) = 1 for all irrational values of x. However, if the function f is well behaved, can we not assume that the differences | f (x) − p(x)| are small when the number of interpolating nodes is large? The answer is still no, even for functions that possess continuous derivatives of all orders on the interval! Runge Function A specific example of this remarkable phenomenon is provided by the Runge function: f (x) = (1 + x2)−1 (1) 0 Runge Function Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 180 Chapter 4 Interpolation and Numerical Differentiation on the interval [−5, 5]. Let pn be the polynomial that interpolates this function at n + 1 Runge Function Numerical Experiment equally spaced points on the interval [−5, 5], including the endpoints. Then lim max |f(x)−pn(x)|=∞ n→∞ −5 ≦ x ≦ 5 Thus, the effect of requiring the agreement of f and pn at more and more points is to increase the error at nonnodal points, and the error actually increases beyond all bounds! The moral of this example, then, is that polynomial interpolation of high degree with many nodes is a risky operation; the resulting polynomials may be very unsatisfactory as representations of functions unless the set of nodes is chosen with great care. The reader can easily observe the phenomenon just described by using the pseudocodes already developed in this chapter. See Computer Exercise 4.2.1 for a suggested numerical experiment. In a more advanced study of this topic, it would be shown that the divergence of the polynomials can often be ascribed to the fact that the nodes are equally spaced. Again, contrary to intuition, equally distributed nodes are usually a very poor choice in interpolation. A much better choice for n + 1 nodes in [−1, 1] is the set of Chebyshev nodes: xi =cos􏰜􏰋2i+1􏰌π􏰝 (0≦i≦n) 2n + 2 The corresponding set of nodes on an arbitrary interval [a, b] would be derived from a linear mapping to obtain 1 1 􏰜􏰋2i+1􏰌 􏰝 xi =2(a+b)+2(b−a)cos 2n+2 π (0≦i≦n) Notice that these nodes are numbered from right to left. Since the theory does not depend on any particular ordering of the nodes, this is not troublesome. A simple graph illustrates this phenomenon best. Again, consider Equation (1) on the interval [−5, 5]. First, we select nine equally spaced nodes and use routines Coef and Eval with an automatic plotter to graph p8. As shown in Figure 4.7, the resulting curve assumes negativevalues,which,ofcourse, f(x)doesnothave!Addingmoreequallyspacednodes— and thereby obtaining a higher-degree polynomial—only makes matters worse with wilder oscillations. In Figure 4.8, nine Chebyshev nodes are used, and the resulting polynomial Chebyshev Nodes Better Fitting with Chebyshev Polynomials FIGURE 4.7 Polynomial interpolant with nine equally spaced nodes FIGURE 4.8 Polynomial interpolant with nine Chebyshev nodes 25 24 y 1 23 22 21 0 1 2 3 21 4 5 y 1 x x 25 24 23 22 21 0 21 1 2 3 4 5 Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. FIGURE 4.9 Chebyshev nodes ofT9 4.2 Errors in Polynomial Interpolation 181 5 25 0 5 curve is smoother. However, cubic splines (discussed in Section 6.2) produce an even better curve fit. The Chebyshev nodes of T9 are obtained by taking equally spaced points on the unit circle and projecting them onto the horizontal axis, as in Figure 4.9. Theorems on Interpolation Errors It is possible to assess the errors of interpolation by means of a formula that involves the (n + 1)st derivative of the function being interpolated. Here is the formal statement: First Interpolation Error Theorem If p is the polynomial of degree at most n that interpolates f at the n + 1 distinct nodes x0, x1, . . . , xn belonging to an interval [a, b] and if f (n+1) is continuous, then for each x in [a,b], there is a ξ in (a,b) for which 1 (n+1) 􏰅n f (x) − p(x) = (n + 1)! f (ξ) (x − xi ) (2) i=0 ■ Theorem1 Proof Observe first that Equation (2) is obviously valid if x is one of the nodes xi because then both sides of the equation reduce to zero. If x is not a node, let it be fixed in the remainder of the discussion, and define w(t ) = c = 􏰅n (t − xi ) (polynomial in the variable t ) i=0 f(x)− p(x) (3) (constant) φ(t) = f (t) − p(t) − cw(t) (function in the variable t) w(x ) Observe that c is well defined because w(x) ≠ 0 (x is not a node). Note also that φ takes thevalue0atthen+2pointsx0,x1,...,xn,andx.NowinvokeRolle’sTheorem,∗ which states that between any two roots of φ, there must occur a root of φ′. Thus, φ′ has at least n + 1 roots. By similar reasoning, φ′′ has at least n roots, φ′′′ has at least n − 1 roots, and so on. Finally, it can be inferred that φ(n+1) must have at least one root. Let ξ be a root of φ(n+1). All the roots being counted in this argument are in (a, b). Thus, we obtain 0 = φ(n+1)(ξ) = f (n+1)(ξ) − p(n+1)(ξ) − cw(n+1)(ξ) ∗ Rolle’s Theorem: Let f be a function that is continuous on [a, b] and differentiable on (a, b). If f(a)= f(b)=0,then f′(c)=0forsomepointcin(a,b). Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 182 Chapter 4 Interpolation and Numerical Differentiation ■ Lemma1 Proof To establish this inequality, fix x and select j so that xj ≦ x ≦ xj+1. It is an exercise in calculus (Exercise 5.2.2) to show that h2 |x−xj||x−xj+1|≦ 4 (5) FIGURE 4.10 Typical location of x in equally spaced nodes a 5 x0 . . . x xj11 xj12 . . . In this equation, p(n+1)(ξ) = 0 because p is a polynomial of degree ≦ n. Also, w(n+1)(ξ) = (n + 1)! because w(t) = tn+1+ (lower-order terms in t). Thus, we have 0= f(n+1)(ξ)−c(n+1)!= f(n+1)(ξ)−(n+1)![f(x)−p(x)] w(x ) This equation is a rearrangement of Equation (2). ■ A special case that often arises is the one in which the interpolation nodes are equally spaced. Upper Bound Lemma Supposethatxi =a+ihfori=0,1,...,nandthath=(b−a)/n.Thenforany x ∈ [a, b] 􏰅n 1 n + 1 |x−xi|≦ 4h n! (4) i=0 Using Equation (5), we have n j−1 n 􏰅|x−xi|≦ h2 􏰅(x−xi) 􏰅(xi −x) i=0 4 i=0 i=j+2 The sketch in Figure 4.10, showing a typical case of equally spaced nodes, may be helpful. xn21 xn 5 b Nowusethefactthatxi =a+ih.Thenwehavexj+1 −xi =(j−i+1)handxi −xj = (i − j)h. Therefore, we obtain x1 x2 x3 xj21 xj Since xj ≦ x ≦ xj+1, we have further n j−1 n 􏰅|x−xi|≦h2 􏰅(xj+1−xi) 􏰅(xi −xj) i=0 4 i=0 i=j+2 n j−1n 􏰅 h2 􏰅 􏰅 |x − xi | ≦ 4 h j hn−( j+2)+1 ( j − i + 1) i=0 i=0 i=j+2 ≦ 1hn+1(j+1)!(n−j)!≦1hn+1n! 44 (i − j) Inthelaststep,weusethefactthatif0≦ j≦n−1,then(j+1)!(n−j)!≦n!.This,too, is left as an exercise (Exercise 5.2.3). Hence, Inequality (4) is established. ■ Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. Proof EXAMPLE 2 Solution Use Theorem 1 on interpolation errors and Inequality (4) in Lemma 1. ■ This theorem gives loose upper bounds on the interpolation error for different values of n. By other means, one can find tighter upper bounds for small values of n (Cf. Exercise 4.2.5). If the nodes are not uniformly spaced, then a better bound can be found by use of the Chebyshev nodes. Assess the error if sin x is replaced by an interpolation polynomial that has 10 equally spaced nodes in [0, 1.6875]. (See the related Example 8 in Section 4.1.) We use Theorem 2 on interpolation errors, taking f (x) = sin x, n = 9, a = 0, and b = 1.6875. Since f (10)(x) = − sin x, | f (10)(x)| ≦ 1. Hence, in Equation (6), we can let M = 1. The result is |sinx − p(x)|≦1.34×10−9 Thus, p(x) represents sin x on this interval with an error of at most two units in the ninth decimal place. Therefore, the interpolation polynomial that has 10 equally spaced nodes on the interval [0, 1.6875] approximates sin x to at least eight decimal digits of accuracy. In fact, a careful check on a computer would reveal that the polynomial is accurate to even more decimal places. (Why?) ■ The error expression in polynomial interpolation can also be given in terms of divided differences: 4.2 Errors in Polynomial Interpolation 183 We can now find a bound on the interpolation error. Second Interpolation Error Theorem Let f beafunctionsuchthat f(n+1) iscontinuouson[a,b]andsatisfies|f(n+1)(x)|≦ M. Let p be the polynomial of degree ≦ n that interpolates f at n + 1 equally spaced nodes in [a, b], including the endpoints. Then on [a, b], |f(x)− p(x)|≦ 1 Mhn+1 (6) 4(n + 1) where h = (b − a)/n is the spacing between nodes. ■ Theorem2 Third Interpolation Error Theorem If pisthepolynomialofdegreenthatinterpolatesthefunction f atnodesx0,x1,...,xn, then for any x that is not a node, 􏰅n f(x)−p(x)= f[x0,x1,...,xn,x] (x−xi) i=0 ■ Theorem3 Proof Let t be any point, other than a node, where f (t) is defined. Let q be the polynomial of degree≦n+1thatinterpolates f atx0,x1,...,xn,t.BytheNewtonformoftheinterpolation Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 184 Chapter 4 Interpolation and Numerical Differentiation formula (Equation (8) in Section 4.1), we have q(x)= p(x)+ f[x0,x1,...,xn,t] Since q(t) = f (t), this yields at once 􏰅n (x−xi) i=0 􏰅n (t−xi) ■ i=0 f(t)= p(t)+ f[x0,x1,...,xn,t] The following theorem shows that there is a relationship between divided differences and derivatives. Divided Differences and Derivatives If f(n) iscontinuouson[a,b]andifx0,x1,...,xn areanyn+1distinctpointsin [a, b], then for some ξ in (a, b), f[x0,x1,...,xn]= 1 f(n)(ξ) n! ■ Theorem4 Proof Let p be the polynomial of degree ≦ n − 1 that interpolates f at x0, x1, . . . , xn−1. By Theorem 1 on interpolation errors, there is a point ξ such that 1 ( n ) 􏰅n − 1 f(xn)−p(xn)=n!f (ξ) (xn−xi) i=0 By Theorem 3 on interpolation errors, we obtain differences are zero for a polynomial. Is there a cubic polynomial that takes these values? x 1 −2 0 3 −1 7 y −2 −56 −2 4 −16 376 If such a polynomial exists, its fourth-order divided differences f [ , , , , ] would all be zero. We form a divided-difference table to check this possibility: f(xn)− p(xn)= f[x0,x1,...,xn−1,xn] As an immediate consequence of this theorem, we observe that all high-order divided 􏰅n−1 (xn −xi) ■ i=0 Divided Differences Corollary If f is a polynomial of degree n, then all of the divided differences f [x0, x1, . . . , xi ] are zero for i ≧ n + 1. ■ Corollary1 EXAMPLE 3 Solution Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. Sample Divided- Difference Table 4.2 Errors in Polynomial Interpolation 185 x f[] f[,] f[,,] f[,,,] f[,,,,] 1 −2 −56 27 0−2−5 0 22 34−3 0 52 Newton Interpolation Polynomials The data can be represented by a cubic polynomial because the fourth-order divided dif- ferences f [ , , , , ] are zero. From the Newton form of the interpolation formula, this polynomial is p3(x) = −2 + 18(x − 1) − 9(x − 1)(x + 2) + 2(x − 1)(x + 2)x ■ Summary 4.2 • The Runge function f (x) = 1/(1 + x2) on the interval [−5, 5] shows that high-degree polynomial interpolation and uniform spacing of nodes may not be satisfactory. The Chebyshev nodes for the interval [a, b] are given by xi =1(a+b)+1(b−a)cos􏰜􏰋2i+1􏰌π􏰝 2 2 2n+2 • There is a relationship between differences and derivatives: f[x0,x1,...,xn]= 1 f(n)(ξ) −1 −16 7 376 11 18 49 n! • Expressions for errors in polynomial interpolation are 1 (n+1) f (x) − p(x) = (n + 1)! f (ξ) 􏰅n f(x)−p(x)= f[x0,x1,...,xn,x] (x−xi) i=0 • For n + 1 equally spaced nodes, an upper bound on the error is given by M 􏰋 b − a 􏰌n+1 |f(x)− p(x)|≦ Here M isanupperboundon􏰣􏰣f(n+1)(x)􏰣􏰣whena≦x≦b. −9 2 􏰅n (x − xi ) i=0 4(n + 1) n • If f is a polynomial of degree n, then all of the divided differences f [x0, x1, . . . , xi ] are zero for i ≧ n + 1. −2 Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 186 Chapter 4 Interpolation and Numerical Differentiation a 1. Use a divided-difference table to show that the following data can be represented by a polynomial of degree 3: x −2 −1 0 1 2 3 y 1 4 11 16 13 −4 a9. An interpolating polynomial of degree 20 is to be used to approximate e−x on the interval [0, 2]. How accurate will it be? (Use 21 uniform nodes, including the endpoints of the interval. Compare results, using Theorems 1 and 2.) a10. Let the function f(x) = lnx be approximated by an interpolation polynomial of degree 9 with 10 nodes uni- formly distributed in the interval [1, 2]. What bound can be placed on the error? 11. In the first theorem on interpolation errors, show that if x0 < x1 < ··· < xn and x0 < x < xn, then x0 < ξ < xn. 12. (Continuation) In the same theorem, considering ξ as a function of x, show that f (n)[ξ(x)] is a continuous func- tion of x. Note: ξ(x) need not be a continuous function of x. a13. Supposecosxistobeapproximatedbyaninterpolating polynomial of degree n, using n+1 equally spaced nodes in the interval [0, 1]. How accurate is the approximation? (Express your answer in terms of n.) How accurate is the 2. Fill in a detail in the proof of Inequality (4) by proving Inequality (5). 3. (Continuation) Fill in another detail in the proof of In- equality (4) by showing that (j + 1)!(n − j)!≦n! if 0 ≦ j ≦ n − 1. Induction and a symmetry argument can be used. 4. For nonuniformly distributed nodes a = x0 < x1 < ···1/ 2.
7. Let f (x) = max{0, 1 − x}. Sketch the function f .
Then find interpolating polynomials p of degrees 2, 4, 8, 16, and 32 to f on the interval [−4, 4], using equally spaced nodes. Print out the discrepancy f (x) − p(x) at 128 equally spaced points. Then redo the problem using Chebyshev nodes.
8. UsingCoefandEvalandanautomaticplotter,fitapoly- nomial through the following data:
9.
a 10.
2.1 2.30 2.60 2.8 3.00 0.1 0.28 1.03 1.5 1.44
Does the resulting curve look like a good fit? Explain.
Find the polynomial p of degree ≦ 10 that interpolates |x | on [−1, 1] at 11 equally spaced points. Print the dif- ference |x| − p(x) at 41 equally spaced points. Then do the same with Chebyshev nodes. Compare.
Why are the Chebyshev nodes generally better than equally spaced nodes in polynomial interpolation? The
answer lies in the term n (x − xi ) that occurs in the i=0
error formula. If xi = cos[(2i + 1)π/(2n + 2)], then
4.3
Estimating Derivatives and Richardson Extrapolation 187
􏰓
11.
12. 13.
14.
􏰣􏰅n 􏰣
􏰣􏰣 ( x − x i ) 􏰣􏰣 ≦ 2 − n
i=0
for all x in [−1, 1]. Carry out a numerical experiment to
test the given inequality for n = 3, 7, 15.
(Student Research Project) Explore the topic of inter- polation of multivariate scattered data, such as those in geophysics and other areas.
Use mathematical software such as found in MATLAB, Maple, or Mathematica to reproduce Figures 4.7–4.8.
UsesymbolicmathematicalsoftwaresuchasMATLAB, Maple or Mathematica to generate the interpolation poly- nomial for the data points in Example 2. Plot the polyno- mial and the data points.
Usegraphicalsoftwaretoplotfourorfivepointsthathap- pen to generate an interpolating polynomial that exhibits a great deal of oscillations. This piece of software should let you use your computer mouse to click on three or four points that visually appear to be part of a smooth curve. Next it uses Newton’s interpolating polynomial to sketch the curve through these points. Then add another point that is somewhat remote from the curve and refit all the points. Repeat, adding other points. After a few points have been added in this way, you should have evidence that polynomials can oscillate wildly.
􏰢
4.3
Estimating Derivatives and Richardson Extrapolation
A numerical experiment outlined in Section 1.1 (p. 12) showed that determining the deriva- tive of a function f at a point x is not a trivial numerical problem. Specifically, if f (x) can be computed with only n digits of precision, it is difficult to calculate f ′(x) numerically
x 0.0 0.60 1.50 1.70 1.90
y −0.8 −0.34 0.59 0.59
0.23
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
Computer Exercises 4.2

188 Chapter 4
Interpolation and Numerical Differentiation
f ′(x) Approximation
f ′(x) Forward Difference Rule with Error Term
Error Analysis
EXAMPLE 1
Solution
with n digits of precision. This difficulty can be traced to the subtraction between quanti- ties that are nearly equal. In this section, several alternatives are offered for the numerical computation of f ′(x) and f ′′(x).
First-Derivative Formulas via Taylor Series
First, consider again the obvious method based on the definition of f ′(x). It consists of selecting one or more small values of h and writing
f′(x)≈ 1[f(x+h)− f(x)] (1) h
What truncation error is involved in this formula?
To find out, use Taylor’s Theorem:
f(x+h)= f(x)+hf′(x)+1h2 f′′(ξ) 2
Rearranging this equation gives
f′(x)= 1[f(x+h)− f(x)]−1hf′′(ξ) (2)
h2 Hence,weseethatapproximation(1)haserrorterm−1hf′′(ξ)=O(h),whereξ isinthe
2
interval having endpoints x and x + h.
Equation (2) shows that in general, as h → 0, the difference between f ′(x) and the
estimate h−1[ f (x + h) − f (x)] approaches zero at the same rate that h does—that is, O(h). Of course, if f ′′(x) = 0, then the error term is 1 h2 f ′′′(γ ), which converges to zero
6 somewhat faster at O(h2). But usually, f ′′(x) is not zero.
Equation (2) gives the truncation error for this numerical procedure, namely, − 1 h f ′′ (ξ ). This error is present even if the calculations are performed with infinite preci-
2
sion; it is due to our imitating the mathematical limit process by means of an approximation
formula. Additional (and worse) errors must be expected when calculations are performed on a computer with finite word length.
In Section 1.1 (p. 12), the program named First used the forward difference rule (1) to approximate the first derivative of the function f (x) = sin x at x = 0.5. Explain what happens when a large number of iterations are performed, say n = 50.
There is a total loss of all significant digits! When we examine the computer output closely, we find that, in fact, a good approximation f ′(0.5) ≈ 0.87758 was found, but it deteriorated as the process continued. This was caused by the subtraction of two nearly equal quantities
f (x + h) and f (x), resulting in a loss of significant digits as well as a magnification of this effect from dividing by a small value of h. We need to stop the iterations sooner! When to stop an iterative process is a common question in numerical algorithms. In this case, monitor the iterations to determine when they settle down, namely, when two successive ones are within a prescribed tolerance. Alternatively, we can use the truncation error term. If we want six significant digits of accuracy in the results, we set
􏰣􏰣􏰣− 1hf′′(ξ)􏰣􏰣􏰣≦ 14−n < 110−6 222 since |f′′(x)| < 1 and h = 1/4n. We find n > 6/log4 ≈ 9.97. So we should stop after about 10 steps in the process. (The least error of 3.1 × 10−9 was found at iteration 14.) ■
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

4.3 Estimating Derivatives and Richardson Extrapolation 189
As we saw in Newton’s method (Section 3.2) and will see in the Romberg method (Section 5.2), it is advantageous to have the convergence of numerical processes occur with higher powers of some quantity approaching zero. In the present situation, we want an approximation to f ′(x) in which the error behaves like O(h2). One such method is easily obtained with the aid of the following two Taylor series:
111
 ′ 2 ′′ 3 ′′′ 4 (4)
2! 3! 4!
By subtraction, we obtain
f(x+h)− f(x−h)=2hf′(x)+ 2h3 f′′′(x)+ 2h5 f(5)(x)+··· 3! 5!
This leads to a very important formula for approximating f ′(x):
1 h2 h4 f′(x)= [f(x+h)− f(x−h)]− f′′′(x)−
2h 3!5! Expressed otherwise,
f′(x)≈ 1 [f(x+h)− f(x−h)] 2h
with an error whose leading term is −1 h2 f ′′′(x), which makes it O(h2). 6
f(x+h)=f(x)+hf(x)+2!hf(x)+3!hf(x)+4!hf (x)+··· f(x−h)= f(x)−hf′(x)+ 1h2f′′(x)− 1h3f′′′(x)+ 1h4f(4)(x)−···
(3)
f(5)(x)−··· (4)
f ′(x) Central Difference Rule
(5)
By using Taylor’s Theorem with its error term, we could have obtained the following two expressions:
f(x+h)= f(x)+hf′(x)+1h2 f′′(x)+1h3 f′′′(ξ1) 26
f(x−h)= f(x)−hf′(x)+1h2 f′′(x)−1h3 f′′′(ξ2) 26
Then the subtraction would lead to
f′(x)= 1 [f(x+h)− f(x−h)]−1h2􏰜f′′′(ξ1)+ f′′′(ξ2)􏰝
f ′(x) Central Difference Rule with Error Term
EXAMPLE 2
between the least and greatest values of f ′′′ on this interval. If f ′′′ is continuous on this interval, then this average value is assumed at some point ξ. Hence, the formula with its error term can be written as
f′(x)= 1[f(x+h)−f(x−h)]−1h2f′′′(ξ) 2h 6
This is based on the sole assumption that f ′′′ is continuous on [x − h, x + h]. This formula for numerical differentiation turns out to be very useful in the numerical solution of certain differential equations, as we shall see in Chapter 11 (on boundary value problems) and Chapter 12 (on partial differential equations).
Modify program First in Section 1.1 so that it uses the central difference formula (5) to
approximate the first derivative of the function f (x) = sin x at x = 0.5. Determine how
many iterations are needed for error less than 1 10−6 . 2
2h 62
The error term here can be simplified by the following reasoning: The expression 1 [ f ′′′(ξ1)+
2
f ′′′(ξ2)] is the average of two values of f ′′′ on the interval [x − h, x + h]. It therefore lies
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

190 Chapter 4 Interpolation and Numerical Differentiation
Solution
f ′(x) Central Difference Rule with Error Series
φ(h) Formula
Using the truncation error term for the central difference formula (5), we set
􏰣􏰣􏰣− 1h2 f′′′(ξ)􏰣􏰣􏰣≦ 14−2n < 110−6 662 or n > (6 − log 3)/ log 16 ≈ 4.59. We obtain a good approximation after about five iterations
with this higher-order formula. (The least error of 3.6 × 10−12 was at step 9.) ■
Richardson Extrapolation
Returning now to Equation (4), we write it in a simpler form:
′1 246
f (x)= 2h[f(x+h)− f(x−h)]+a2h +a4h +a6h +··· (6)
φ(h) with Error Series
Linear Combination of φ
􏰸(h) Formula
inwhichtheconstantsa2,a4,…dependon f andx.Whensuchinformationisavailable about a numerical process, it is possible to use a powerful technique known as Richardson extrapolation to wring more accuracy out of the method. This procedure is explained here, using Equation (6) as our model.
Holding f and x fixed, we define a function of h by the formula
φ(h)= 1 [f(x+h)− f(x−h)] (7)
2h
From Equation (6), we see that φ(h) is an approximation to f ′(x) with error of order O(h2).
Our objective is to compute limh→0 φ(h) because this is the quantity f ′(x) that we wanted
in the first place. If we select a function f and plot φ(h) for h = 1, 1, 1, 1,…, then we 248
get a graph (Computer Exercise 4.3.5). Near zero, where we cannot actually calculate the value of φ from Equation (7), φ is approximately a quadratic function of h, since the higher- order terms from Equation (6) are negligible. Richardson extrapolation seeks to estimate the limiting value at 0 from some computed values of φ(h) near 0. Obviously, we can take any convenient sequence hn that converges to zero, calculate φ(hn) from Equation (7), and use these as approximations to f ′(x).
But something much more clever can be done. Suppose we compute φ(h) for some h and then compute φ(h/2). By Equation (6), we have
φ(h) = f′(x)−a2h2 −a4h4 −a6h6 −··· 􏰋h􏰌 ′ 􏰋h􏰌2 􏰋h􏰌4 􏰋h􏰌6
φ2 =f(x)−a2 2 −a4 2 −a6 2 −···
We can eliminate the dominant term in the error series by simple algebra. To do so, multiply
the second equation by 4 and subtract it from the first equation. The result is
φ(h)−4φ􏰋h􏰌=−3f′(x)− 3a4h4 − 15a6h6 −··· 2 416
We divide by −3 and rearrange this to get
φ􏰋h􏰌+ 1􏰜φ􏰋h􏰌−φ(h)􏰝= f′(x)+ 1a4h4 + 5 a6h6 +···
232 416
This is a marvelous discovery. Simply by adding 1 [φ(h/2) − φ(h)] to φ(h/2), we have 3
apparently improved the precision to O(h4) because the error series that accompanies this new combination begins with 1 a4h4. When h is small, this is a dramatic improvement!
4
We can repeat this process by letting
􏰸(h)= 4φ􏰋h􏰌− 1φ(h) 323
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

4.3 Estimating Derivatives and Richardson Extrapolation 191 Then we have from the previous derivation that
􏰸(h)= f′(x)+b4h4 +b6h6 +··· 􏰋h􏰌 ′ 􏰋h􏰌4 􏰋h􏰌6
􏰸2 =f(x)+b4 2 +b6 2 +···
We can combine these equations to eliminate the first term in the error series
Linear Combination of 􏰸
φ(h) Property
This is yet another apparent improvement in the precision to O(h6). And now, to top it off, note that the same procedure can be repeated over and over again to kill higher and higher terms in the error. This is Richardson extrapolation.
Essentially the same situation arises in the derivation of Romberg’s algorithm in Sec- tion 5.2. We begin a general discussion of the procedure here. We start with an equation that includes both situations. Let φ be a function such that
􏰄∞ k=1
where the coefficients a2k are not known. Equation (8) is not interpreted as the definition of φ, but rather as a property that φ possesses. It is assumed that φ(h) can be computed for any h > 0 and that our objective is to approximate L accurately using φ.
Hence, we have
􏰸(h)−16􏰸􏰋h􏰌=−15f′(x)+ 3b6h6 +··· 24
􏰸􏰋h􏰌+ 1 􏰜􏰸􏰋h􏰌−􏰸(h)􏰝= f′(x)− 1 b6h6 +··· 2 15 2 20
D(n, 0) Formula
Extrapolation Formula
D(n,m)
■ Theorem1
Because of Equation (8), we have
D(n,0) = L + A(k,0) 2n
φ(h) = L −
a2kh2k (8)
Select a convenient h, and compute the numbers
D(n, 0) = φ􏰋 h 􏰌 (n ≧ 0) (9)
2n
􏰄∞ 􏰋 h 􏰌 2 k
k=1
where A(k, 0) = −a2k . These quantities D(n, 0) give a crude estimate of the unknown num- ber L = limx→0 φ(x). More accurate estimates are obtained via Richardson extrapolation. The extrapolation formula is
4m 1
D(n,m)= 4m −1D(n,m−1)− 4m −1D(n−1,m−1) (1≦m≦n) (10)
Richardson Extrapolation Theorem
The quantities D(n, m) defined in the Richardson extrapolation process (10) obey the equation
􏰄∞ 􏰋 h 􏰌 2 k
D(n,m)=L+ A(k,m) 2n (0≦m≦n) (11)
k=m+1
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

192 Chapter 4 Interpolation and Numerical Differentiation
Proof
Equation (11) is true by hypothesis if m = 0. For the purpose of an inductive proof, we assume that Equation (11) is valid for an arbitrary value of m − 1, and we prove that Equation (11) is then valid for m. Now from Equations (10) and (11) for a fixed value m, we have
D(n,m)=L+ Thus, we are led to define
k=m
4m 􏰜 􏰄∞
􏰋h􏰌2k􏰝 A(k,m−1) 2n
D(n,m)=4m−1 L+
1 􏰜 􏰄∞
−4m−1 L+ After simplification, this becomes
k=m
􏰋 h 􏰌 2 k 􏰝 A(k,m−1) 2n−1
k=m
􏰄∞ 􏰋4m−4k􏰌􏰋h􏰌2k
A(k,m−1) 4m −1 2n (12) A(k,m)=A(k,m−1)􏰋4m −4k􏰌
D(n, m) Formula
4m −1
At the same time, we notice that A(m, m) = 0. Hence, Equation (12) can be written as
􏰄∞ 􏰋 h 􏰌 2 k D(n,m) = L + A(k,m) 2n
k=m+1
Equation (11) is true for m, and the induction is complete. ■
ThesignificanceofEquation(11)isthatthesummationbeginswiththeterm(h/2n)2m+2. Since h/2n is small, this indicates that the numbers D(n, m) are approaching L very rapidly, namely,
D(n,m)=L+O􏰋h2(m+1) 􏰌 22n(m+1)
In practice, one can arrange the quantities in a two-dimensional triangular array as follows:
Rule of Converging of
D(n, m)
2D Triangular Array
D(0, 0) D(1, 0) D(2, 0)
D(1, 1) D(2, 1)
D(2, 2)
. …
D(N,2) ··· The main tasks to generate such an array are as follows:
(13)
. D(N,0)
. D(N,1)
D(N, N)
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

Test Derivative
Pseudocode
■ Algorithm
EXAMPLE 3
Solution
Derivative Pseudocode
Notice that in this algorithm, the computation of D(i, j) follows Equation (10) but has been rearranged slightly to improve its numerical properties.
Write a procedure to compute the derivative of a function at a point by using Equation (5) and Richardson extrapolation.
The input to the procedure are a function f , a specific point x, a value of h, and a number n signifying how many rows in the array (13) are to be computed. The output is the array (13). Here is a suitable pseudocode:
4.3 Estimating Derivatives and Richardson Extrapolation 193
Richardson Extrapolation
1. Writeafunctionforφ.
2. Decide on suitable values for N and h.
3. For i = 0,1,…, N, compute D(i,0) = φ(h/2i). 4. For1≦i≦ j≦N,compute
D(i,j)=D(i,j−1)+(4j −1)−1[D(i,j−1)−D(i−1,j−1)]
procedure Derivative( f, x, n, h, (di j ))
integer i, j, n; real h, x; real array (di j )0:n×0:n external function f
fori =0ton
di0 ←[f(x+h)− f(x−h)]/(2h) for j = 1 to i
di,j ←di,j−1+(di,j−1−di−1,j−1)/(4j −1) end for
h ← h/2 end for
end procedure Derivative
To test the procedure, choose f (x ) = sin x , where x0 = 1.23095 94154 and h = 1. Then f ′(x) = cos x and f ′(x0) = 1 . A pseudocode is written as follows:
3
program Test Derivative
real array (di j )0:n×0:n ; external function f
integer n ← 10; real h ← 1; x ← 1.23095 94154 call Derivative( f, x, n, h, (di j ))
output (di j )
end program Test Derivative
real function f (x) real x
f ← sin(x) end function f
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

194 Chapter 4 Interpolation and Numerical Differentiation
Derivatives Symbolically
We invite the reader to program the pseudocode and execute it on a computer. The computer output is the triangular array (di j ) with indices 0 ≦ j ≦ i ≦ 10. The most accurate value is (d4,1 ) = 0.33333 33433. The values di 0 , which are obtained solely by Equations (7) and (9) without any extrapolation, are not as accurate, having no more than four correct digits. ■
Mathematical software is now available with algebraic manipulation capabilities. Using
them, we could write a computer program to find derivatives symbolically for a rather
large class of functions—probably all those you would encounter in a calculus course.
For example, we could verify the numerical results above by first finding the derivative
for approximating derivatives that cannot be determined exactly.
First-Derivative Formulas via Interpolation Polynomials
An important general stratagem can be used to approximate derivatives (as well as integrals and other quantities). The function f is first approximated by a polynomial p so that f ≈ p. Then we simply proceed to the approximation f ′(x) ≈ p′(x) as a consequence. Of course, this strategy should be used very cautiously because the behavior of the interpolating
polynomial can be oscillatory.
In practice, the approximating polynomial p is often determined by interpolation at
a few points. For example, suppose that p is the polynomial of degree at most 1 that interpolates f at two nodes, x0 and x1. Then from Equation (8) in Section 4.1 with n = 1, we have
p1(x)= f(x0)+ f[x0,x1](x−x0)
f′(x)≈ p1′(x)= f[x0,x1]= f(x1)− f(x0) (14)
x1 − x0
If x0 = x and x1 = x + h (see Figure 4.11), this formula is one previously considered,
f ′(x) Finite Difference Approximation
FIGURE 4.11
Forward difference: two nodes
FIGURE 4.12
Central difference: two nodes
namely, Equation (1):
exactly and then evaluating the numerical answer cos(1.23095 94154) ≈ 0.33333 33355
since arccos􏰀 1 􏰁 ≈ 1.23095 941543. Of course, the procedures discussed in this section are 3
Consequently, we have
f′(x)≈ 1[f(x+h)− f(x)] (15) h
x0 x1
x x1h
If x0 = x − h and x1 = x + h (see Figure 4.12), the resulting formula is Equation (5): f′(x)≈ 1 [f(x+h)− f(x−h)] (16)
2h
x0
x2h x x1h
Now consider interpolation with three nodes, x0, x1, and x2. The interpolating polyno- mial is obtained from Equation (8) in Section 4.1:
p2(x)= f(x0)+ f[x0,x1](x−x0)+ f[x0,x1,x2](x−x0)(x−x1)
x1
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

􏰀􏰁
Here the right-hand side consists of two terms. The first is the previous estimate in Equa- tion (14), and the second is a refinement or correction term.
If Equation (17) is used to evaluate f ′(x) when x = 1 (x0 + x1), as in Equation (16), 2
then the correction term in Equation (17) is zero. Thus, the first term in this case must be more accurate than those in other cases because the correction term adds nothing. This is why Equation (16) is more accurate than (15).
An analysis of the errors in this general procedure goes as follows: Suppose that pn is thepolynomialofleastdegreethatinterpolates f atthenodesx0,x1,…,xn.Thenaccording to Theorem 1 on interpolating errors in Section 4.2 (p. 181),
f (x) − pn(x) = 1 f (n+1)(ξ)w(x) (n + 1)!
whereξ isdependentonx,andw(x)=(x−x0)(x−x1)···(x−xn).Differentiatinggives f ′(x) − pn′ (x) = 1 w(x) d f (n+1)(ξ) + 1 f (n+1)(ξ)w′(x) (18)
(n + 1)! dx (n + 1)!
Here, we had to assume that f (n+1)(ξ) is differentiable as a function of x, a fact that is known if f (n+2) exists and is continuous.
The first observation to make about the error formula in Equation (18) is that w(x) vanishes at each node, so if the evaluation is at a node xi , the resulting equation is simpler:
f′(xi)= pn′(xi)+ 1 f(n+1)(ξ)w′(xi) (n + 1)!
For example, taking just two points x0 and x1, we obtain with n = 1 and i = 0, f′(x0)= f[x0,x1]+1f′′(ξ)d [(x−x0)(x−x1)]􏰣􏰣􏰣
2 dx x=x0 = f[x0,x1]+1f′′(ξ)(x0−x1)
2
This is Equation (2) in disguise when x0 = x and x1 = x + h. Similar results follow with
n = 1 and i = 1.
The second observation to make about Equation (18) is that it becomes simpler if x is
chosen as a point where w′(x) = 0. For instance, if n = 1, then w is a quadratic function that vanishes at the two nodes x0 and x1. Because a parabola is symmetric about its axis, w′[(x0 + x1)/2] = 0. The resulting formula is
′􏰋x0+x1􏰌 1 2d ′′
f = f[x0,x1]− (x1 −x0) f (ξ)
2 8 dx
As a final example, consider four interpolation points: x0, x1, x2, and x3. The interpo-
lating polynomial from Equation (8) in Section 4.1 with n = 3 is p3(x)= f(x0)+ f[x0,x1](x−x0)+ f[x0,x1,x2](x−x0)(x−x1)
and its derivative is
4.3 Estimating Derivatives and Richardson Extrapolation 195
p2′(x)= f[x0,x1]+ f[x0,x1,x2](2x−x0 −x1) (17)
f′ x0+x1 with Finite 2
Difference Formula
Its derivative is
+ f[x0,x1,x2,x3](x−x0)(x−x1)(x−x2)
p3′(x)= f[x0,x1]+ f[x0,x1,x2](2x−x0 −x1) + f [x0, x1, x2, x3]((x − x1)(x − x2)
+(x −x0)(x −x2)+(x −x0)(x −x1))
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

196
Chapter 4 Interpolation and Numerical Differentiation
FIGURE 4.13
Central difference: four nodes
x2x0 x1x3 x22h x2h x x1h x12h
Ausefulspecialcaseoccursifx0 =x−h,x1 =x+h,x2 =x−2h,andx3 =x+2h(see Figure 4.13). The resulting formula is
f′(x)≈− 2 [f(x+h)− f(x−h)]− 1 [f(x+2h)− f(x−2h)] 3h 12h
This can be arranged in a form in which it is computed with a principal term plus a correction or refining term:
f′(x)≈ 1 [f(x+h)− f(x−h)] 2h
f ′(x) Finite Difference Formula at Four Nodes
− 1 􏰟f(x +2h)−2[f(x +h)− f(x −h)]− f(x −2h)􏰠
12h
The error term is − 1 h4 f (5)(ξ) = O(h5).
30
Second-Derivative Formulas via Taylor Series
In the numerical solution of differential equations, it is often necessary to approximate second derivatives. We shall derive the most important formula for accomplishing this. Simply add the two Taylor series (3) for f (x + h) and f (x − h). The result is
f(x+h)+ f(x−h)=2f(x)+h2 f′′(x)+2􏰜1h4 f(4)(x)+···􏰝 4!
f ′′(x) Central Difference Formula and Error Term
EXAMPLE 4
Solution
E=−2􏰜1h2f(4)(x)+ 1h4f(6)(x)+···􏰝 4! 6!
By carrying out the same process using Taylor’s formula with a remainder, one can show that E is also given by
E = − 1 h2 f (4)(ξ) = O(h2) 12
for some ξ in the interval (x − h, x + h). Hence, we have the approximation
f′′(x)≈ 1[f(x+h)−2f(x)+ f(x−h)] (20)
h2
with error O(h2).
Repeat Example 2, using the central difference formula (20) to approximate the second
derivative of the function f (x) = sin x at the given point x = 0.5. Using the truncation error term, we set
􏰣􏰣􏰣− 1 h2 f(4)(ξ)􏰣􏰣􏰣≦ 1 4−2n < 110−6 12 12 2 When this is rearranged, we get f′′(x)= 1[f(x+h)−2f(x)+ f(x−h)]+E (19) h2 where the error series is Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. Noise and Truncation Error f ′(x) Central Difference Formula with Error Term and we obtain n > (6 − log 6)/ log 16 ≈ 4.34. Hence, the modified program First finds a good approximation of f ′′(0.5) ≈ −0.47942 after about four iterations. (The least error of 3.1 × 10−9 was obtained at iteration 6.) ■
Approximate derivative formulas of high order can be obtained by using unequally spaced points such as at Chebyshev nodes. Recently, software packages have been developed for automatic differentiation of functions that are expressible by a computer program. They produce true derivatives with only rounding errors and no discretization errors.
Noise in Computation
An interesting question is how noise in the evaluation of f (x) affects the computation of derivatives when using the standard formulas.
The formulas for derivatives are derived with the expectation that evaluation of the function at any point is possible, with complete precision. Then the approximate derivative produced by the formula differs from the actual derivative by a quantity called the error term, which involves the spacing of the sample points and some higher derivative of the function.
If there are errors in the values of the function (noise), they can vitiate the whole process! Those errors could overwhelm the error inherent in the formulas. The inherent error arises from the fact that in deriving the formulas, a Taylor series was truncated after only a few terms. It is called the truncation error. It is present even if the evaluation of the function at the required sample points is absolutely correct.
4.3 Estimating Derivatives and Richardson Extrapolation 197
For example, consider the formula
f(x+h)− f(x−h) h2 f ′(x) = −
2h 6
f ′′′(ξ)
The term with h2 is the error term. The point ξ is a nearby point (unknown). If f (x + h) and f (x − h) are in error by at most d, then one can see that the formula produces a value for f ′(x) that is in error by d/h, which is large when h is small. Noise completely spoils the process if d is large.
For a specific numerical case, suppose that h = 10−2 and | f ′′′(s)| ≦ 6. Then the trunca- tion error, E, satisfies |E| ≦ 10−4. The derivative computed from the formula with complete precision is within 10−4 of the actual derivative. Suppose, however, that there is noise in the evaluation of f (x ±h) of magnitude d = h. The correct value of [ f (x +h)− f (x −h)]/(2h) may differ from the noisy value by (2d)/(2h) = 1.
Summary 4.3
• A forward difference formula is
f′(x)≈ 1[f(x+h)− f(x)]
h
with error term − 1 h f ′′ (ξ ). 2
• A central difference formula is
f′(x)≈ 1 [f(x+h)− f(x−h)]
2h with error −1 h2 f ′′′(ξ) = O(h2).
6
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

198 Chapter 4
Interpolation and Numerical Differentiation
• A central difference formula with a correction term is f′(x)≈ 1 [f(x+h)− f(x−h)]
2h
− 1 [f(x +2h)−2f(x +h)+2f(x −h)− f(x −2h)]
12h
with error term − 1 h4 f (5)(ξ) = O(h4).
30
• For f ′′(x), a central difference formula is
f′′(x)≈ 1[f(x+h)−2f(x)+ f(x−h)] h2
with error term − 1 h2 f (4)(ξ). 12
• Ifφ(h)isoneoftheseformulaswitherrorseriesa2h2+a4h4+a6h6+···,thenwecan apply􏰈Richardson extrapolation as follows
D(n,0) =φ(h/2n)
D(n,m)= D(n,m−1)+[D(n,m−1)−D(n−1,m−1)]/(4m −1)
with error terms
a1. Determinetheerrortermfortheformula f′(x)≈ 1 [f(x+3h)− f(x−h)]
4h
a2. Using Taylor series, establish the error term for the for-
mula
f′(0)≈ 1 [f(2h)− f(0)] 2h
3. Derivetheapproximationformula
f′(x)≈ 1 [4f(x+h)−3f(x)− f(x+2h)]
2h
Show that its error term is of the form 1 h2 f ′′′(ξ).
3
a4. Can you find an approximation formula for f′(x) that has error term O(h3) and involves only two evaluations of the function f ? Prove or disprove.
5. Averaging the forward-difference formula f ′(x) ≈ [ f (x + h) − f (x)]/h and the backward-difference for- mula f′(x) ≈ [f(x) − f(x − h)]/h, each with er- ror term O(h), results in the central-difference formula
f′(x)≈[f(x+h)− f(x−h)]/(2h)witherrorO(h2). Show why.
Hint: Determine at least the first term in the error series for each formula.
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole deemed that any suppressed content does not materially affect the overall learning experience.
D(n,m)=L+O􏰍h2(m+1) 􏰎 22n(m+1)
a6.
Criticizethefollowinganalysis.ByTaylor’sformula,we have
f(x +h)− f(x) = hf′(x)+ ′
h2 h3 f′′(x)+ f′′′(ξ)
26
h2 ′′ h3 ′′′
f(x−h)−f(x)=−hf(x)+ f (x)− f (ξ) 26
7.
So by adding, we obtain an exact expression for f ′′(x): f (x + h) + f (x − h) − 2 f (x) = h2 f ′′(x)
Criticize the following analysis. By Taylor’s formula, we have
′ h2 ′′ h3 ′′′ f(x+h)−f(x)=hf(x)+2f(x)+6f (ξ1)
′ h2 ′′ h3 ′′′ f(x−h)−f(x)=−hf(x)+2f(x)−6f (ξ2)
Therefore, we have 1
h2[f(x+h)−2f(x)+ f(x−h)] h
= f′′(x)+ 6[f′′′(ξ1)− f′′′(ξ2)]
The error in the approximation formula for f ′′ is thus
O(h).
or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
Exercises 4.3

8.
9.
Derivethetwoformulas
aa. f′(x)≈ 1 [f(x+2h)− f(x−2h)]
a
a
′′′ 1
a. f (x)≈ 2h3[f(x+2h)−2f(x+h) 17.
+2f(x −h)− f(x −2h)]
(x)≈ h4[f(x+2h)−4f(x+h)+6f(x) 18.
(Continuation) State and prove a theorem analogous to the theorem on Richardson extrapolation for the situa- tion of the preceding problem.
Ifφ(h)= L−c1h1/2−c2h2/2−c3h3/2−···,thenwhat combination of φ(h) and φ(h/2) should give an accurate estimate of L?
4h2 15. Establish formulas for the errors in using them.
4.3
Estimating Derivatives and Richardson Extrapolation 199 14. ConsiderEquation(19).
a. Fillinthedetailsinitsderivation.
b. UsingTaylorseries,deriveitserrorterm.
ShowhowRichardsonextrapolationwouldworkonFor- mula (20).
a16. If φ(h) = L − c1h − c2h2 − c3h3 − ···, then what combination of φ(h) and φ(h/2) should give an accurate estimate of L ?
4h
b. f′′(x)≈ 1 [f(x+2h)−2f(x)+ f(x−2h)]
Derivethefollowingrulesfortheseestimatingderivatives and establish their error terms. Which is more accurate?
10.
19. Show that Richardson extrapolation can be carried out foranytwovaluesofh.Thus,ifφ(h)= L−O(hp),then from φ(h1) and φ(h2), a more accurate estimate of L is given by
h 2p
φ(h2) + h1p − h2p [φ(h2) − φ(h1)]
a20. Consider a function φ such that limh→0 φ(h) = L and L − φ(h) ≈ ce−1/h for some constant c. By combin- ing φ(h), φ(h/2), and φ(h/3), find an accurate estimate ofL.
21. Considertheapproximateformula
(4) 1
Hint: Consider the Taylor series for D(h) ≡ f (x + h) −
b. f
−4f(x −h)+ f(x −2h)] f(x−h)andS(h)≡ f(x+h)+ f(x−h).
Establish the formula
f′′(x)≈ 2 􏰆 f(x0) − f(x1)+ f(x2) 􏰇
h2 (1 + α) α α(α + 1)
in the following two ways, using the unevenly spaced pointsx0 < x1 < x2,wherex1−x0 = hand x2 − x1 = αh. Notice that this formula reduces to the standard central-difference formula (20) when α = 1. a. Approximate f (x) by the Newton form of the inter- polating polynomial of degree 2. b. Calculate the undetermined coefficients A, B, and C in the expression f ′′(x) ≈ A f (x0)+B f (x1)+C f (x2) by making it exact for the three polynomials 1, x −x1, and (x − x1)2 and thus exact for all polynomials of degree ≦ 2. (Continuation) Using Taylor series, show that f′(x1)= f(x2)− f(x0) +(α−1)h f′′(x1)+O(h2) x2 − x0 2 Establish that the error for approximating f′(x1) by [ f (x2) − f (x0)]/(x2 − x0) is O(h2), when x1 is mid- way between x0 and x2, but only O(h) otherwise. A certain calculation requires an approximation formula for f ′(x) + f ′′(x). How well does the expression 22. 􏰋2+h􏰌 􏰋2􏰌 􏰋2−h􏰌 D(3,3). 3􏰒h f′(x)≈ 2h3 Determine its error term. Does the function f have to be differentiable for the formula to be meaningful? Hint: This is a novel method of doing numerical differen- tiation. Read more about Lanczos’ generalized deriva- tive in Groetsch [1998]. a11. a 12. a 13. tf(x+t)dt −h 2h2 f(x +h)− h2 f(x)+ 2h2 f(x −h) 23. Differentiation and integration are mutual inverse pro- Derive the error terms for D(3, 0), D(3, 1), D(3, 2), and serve? Derive this approximation and its error term. The values of a function f are given at three points x0, x1, and x2. If a quadratic interpolating polynomial is used to estimate f ′(x) at x = 1 (x0 + x1), what formula will 2 result? cesses. Differentiation is an inherently sensitive prob- lem in which small changes in the data can cause large changes in the results. Integration is a smoothing process and is inherently stable. Display two functions that have very different derivatives but equal definite integrals and vice versa. Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 200 Chapter 4 Interpolation and Numerical Differentiation 24. Establishtheerrortermsfortheserules: a. f′′′(x) ≈ 1 [3f(x +h)−10f(x) 2h3 + 12f(x −h)−6f(x −2h)+ f(x −3h)] b. f′(x)+hf′′≈1[f(x+h)−f(x)] 2h 1. Test procedure Derivative on the following functions at the points indicated in a single computer run. Interpret the results. a. f(x)=cosxatx=0 b. f(x)=arctanxatx=1 c. f(x)=|x|atx=0 2. (Continuation) Write and test a procedure similar to Derivative that computes f ′′(x) with repeated Richard- son extrapolation. a3. Find f′(0.25) as accurately as possible, using only the function corresponding to the following pseudocode and a method for numerical differentiation: c. f(iv)(x)≈ 1􏰜4f(x+3h)−6f(x+2h) h4 3 􏰝 + 12 f (x + h) 4. Carry out a numerical experiment to compare the accu- racy of Formulas (5) and (19) on a function f whose derivative can be computed precisely. Take a sequence of valuesforh,suchas4−n with0≦n≦12. 5. Using the discussion of the geometric interpretation of Richardson extrapolation, produce a graph to show that φ(h) looks like a quadratic curve in h. 6. UsesymbolicmathematicalsoftwaresuchasMATLAB, Maple or Mathematica to establish the first term in the error series for Equation (19). 7. Use mathematical software such as found in MATLAB, Maple, or Mathematica to redo Example 1. if f(x)= f′(x)=0. real function f (x) integer i ; real a, b, c, x a ← 1; b ← cos(x) fori =1to5 c←b √ b← ab a ← (a + c)/2 end for f ← 2 arctan(1)/a end function f Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. Computer Exercises 4.3 5 Numerical Integration In electrical field theory, it is proved that the magnetic field induced by a current flowing in a circular loop of wire has intensity 4Ir 􏰒 π/2􏰜 􏰋x􏰌2 2 􏰝1/2 H(x)= 2 2 1− sinθ dθ where I is the current, r is the radius of the loop, and x is the distance from the center to the point where the magnetic intensity is being computed ( 0 ≦ x ≦ r ) . If I , r , and x are given, we have a formidable integral to evaluate. It is an elliptic integral and not expressible in terms of familiar functions. But H can be computed precisely by the methods of this chapter. For example, if I = 15.3, r = 120, and x = 84, we find H = 1.355661135 accurate to nine decimals. 5.1 Trapezoid Method r−x0 r Indefinite/Definite Integral Indefiniteintegral: Definite integral: 􏰒2 One of the main mathematical procedures that is the focus of elementary calculus is integration, which is examined from the standpoint of numerical mathematics in this chapter. Definite and Indefinite Integrals It is customary to distinguish two types of integrals: the definite and the indefinite integral. The indefinite integral of a function is another function or a class of functions, whereas the definite integral of a function over a fixed interval is a number. For example, we have 􏰒2 13 x dx= x +C 3 Actually, a function has not just one, but many indefinite integrals. These differ from each other by constants. Thus, in the preceding example, any constant value may be assigned to C, and the result is still an indefinite integral. In elementary calculus, the concept of an indefinite integral is identical with the concept of an antiderivative. An antiderivative of a function f is any function F having the property that F ′ = f . 201 x2 dx = 8 03 Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 202 Chapter 5 Numerical Integration Examples of Fundamental Theorem of Calculus Derivative of an Integral Antiderivative versus Area Under Curves The definite and indefin􏰦ite integrals are related by the Fundamental Theorem of Calculus,∗ which states that ab f (x ) d x can be computed by first finding an antiderivative F of f and then evaluating F(b) − F(a). Thus, using traditional notation, we have 􏰒3 􏰋x3 􏰌􏰣􏰣􏰣􏰣3 􏰋27 􏰌 􏰋1 􏰌 14 (x2 −2)dx = −2x = −6 − −2 = 1 31333 As another example of the Fun􏰒damental Theorem of Calculus, we can write b F′(x)dx = F(b)− F(a) 􏰒a x F′(t)dt = F(x)− F(a) a If this second equation is differentiated with respect to x, the result is (and here we have Nonuniform Partition This last equation shows that 􏰦ax f (t ) d t must be an antiderivative (indefinite integral) of f . The foregoing technique for computing definite integrals is virtually the only one emphasized in elementary calculus. The definite integral of a function, however, has an i􏰦nterpretation as the area under a curve, and so the existence of a numerical value for ab f (x ) d x should not depend logically on our limited ability to find antiderivatives. Thus, for instance, 􏰒 1 ex2 dx 0 has a precise numerical value despite the fact that there is no elementary function F such that F′(x) = ex2 . By the preceding remarks, ex2 does have antiderivatives, one of which is 􏰒x2 F(x) = et dt 0 However, this form of the function F is of no help in determining the numerical value sought. Trapezoid Rule: Nonuniform Spacing The trapezoid rule is based on an estimation of the area beneath a curve using trapezoids. Moreover, it i􏰦s an important ingredient of the Romberg algorithm of the next section. The estimation of ab f (x) dx is approached by first dividing the interval [a, b] into subintervals according to the partition P={a=x0 (1/√1.2)102 = 91.3. This analysis
2
induces us to take 92 subintervals. ■
Recursive Trapezoid Formula
In the next section, we require a formula for the composite trapezoid rule when the interval [a, b] is subdivided into 2n equal parts. By the composite trapezoid rule (1), we have
􏰄n−1 T(f;P)=h
h f(xi)+2[f(x0)+ f(xn)]
h
=h
i=1
i=1 􏰄n−1
[f(a)+ f(b)]
f(a+ih)+
If we now replace n by 2n and use h = (b − a)/2n , the preceding formula becomes
2 −1 􏰄n
i=1
Here, we have introduced the notation that is used in Section 5.2 on the Romberg algorithm, namely, R(n, 0). It denotes the result of applying the composite trapezoid rule with 2n equal subintervals.
In the Romberg algorithm, it is necessary to have a means of computing R(n, 0) from R(n − 1, 0) without involving unneeded evaluations of f . For example, the computation of R(2,0) utilizes the values of f at the five points a, a + (b − a)/4, a + 2(b − a)/4, a + 3(b − a)/4, and b. In computing R(3, 0), we need values of f at these five points, as wellasatfournewpoints:a+(b−a)/8,a+3(b−a)/8,a+5(b−a)/8,anda+7(b−a)/8 (see Figure 5.3). The computation should take advantage of the previously computed result. The manner of doing so is explained below.
R(n,0)=h
f(a+ih)+
h
2
[f(a)+ f(b)] (8)
2
Subintervals Array
ab
20 21 22
2n equal subintervals 23
If R(n − 1, 0) has been computed and R(n, 0) is to be computed, we use the identity
R(n, 0) = 1 R(n − 1, 0) + 􏰜R(n, 0) − 1 R(n − 1, 0)􏰝 22
It is desirable to compute the bracketed expression with as little additional work as possible. Fixing h = (b − a)/2n for the analysis and putting
h
C= [f(a)+f(b)]
2
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
FIGURE 5.3
R(0, 0) R(1, 0) R(2, 0) R(3, 0)

we have, from Equation (8),
2 −1 􏰄n
i=1
2n−1 2n−1−1 R(n,0)−1R(n−1,0)=h􏰄􏰄f(a+ih)−h 􏰄 f(a+2jh)
2 i=1 j=1 2n−1
= h f [a + (2k − 1)h] k=1
Here, we have taken account of the fact that each term in the first sum that corresponds to an even value of i is canceled by a term in the second sum. This leaves only terms that correspond to odd values of i.
To summarize, we have the Recursive Trapezoid Formula.
R(n, 0) = h
f (a + ih) + C
(9)
2 n 􏰄− 1 − 1 j=1
R(n − 1, 0) = 2h
Notice that the subintervals for R(n − 1, 0) are twice the size of those for R(n, 0). Now
from Equations (9) and (10), we have
5.1
Trapezoid Method 211
f (a + 2 jh) + 2C (10)
Recursive Trapezoid Formula
If R(n − 1, 0) is available, then R(n, 0) can be computed by the formula
􏰄n−1 12
R(n − 1, 0) + h
usingh =(b−a)/2n.Here, R(0,0)= 1(b−a)[f(a)+ f(b)]. 2
R(n, 0) =
2
f [a + (2k − 1)h]
(n ≧ 1) (11)
k=1
■ Theorem2
This formula allows us to compute a sequence of approximations to a definite integral using the trapezoid rule without reevaluating the integrand at points where it has already been evaluated.
Multidimensional Integration
Here, we give a brief account of multidimensional numerical integration. For simplicity, we illustrate with the trapezoid rule for the interval [0, 1], using n + 1 equally spaced points. The step size is therefore h = 1/n. The composite trapezoid rule is then
􏰒1 1􏰜 􏰄n−1􏰋i􏰌 􏰝 f(x)dx ≈ f(0)+2 f + f(1)
0 2ni=1n We write this in the form
􏰒1 􏰄n􏰋i􏰌 f(x)dx≈ Cifn
0 i=0
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

212 Chapter 5
Numerical Integration
where 1/(2n), i = 0
Ci = 1/n, 0 < i < n 1/(2n), i = n The error is O(h2) = O(n−2) for functions having a continuous second derivative. If one is faced with a two-dimensional integration over the unit square, then the trapezoid rule can be applied twice: 􏰄n 􏰒1􏰋α1􏰌 = Cα1 fn,ydy 􏰒1􏰒1 00 0α1=0 􏰒1􏰄n 􏰋α1􏰌 f(x,y)dxdy≈ Cα1 f n,y dy α1=0 0 􏰄n 􏰄n 􏰋α1α2􏰌 ≈ Cα1 Cα2fn,n α1 =0 α2 =0 􏰄n 􏰄n 􏰋α1 α2􏰌 = Cα1Cα2f n,n α1=0 α2=0 The error here is again O(h2), because each of the two applications of the trapezoid rule entails an error of O(h2). In the same way, we can integrate a function of k variables. Suitable notation is the vector x = (x1, x2, . . . , xk )T for the independent variable. The region now is taken to be the k- dimensionalcube[0,1]k ≡[0,1]×[0,1]×···×[0,1].Thenweobtainamultidimensional numerical integration rule 􏰒􏰄n􏰄n􏰄n 􏰋α1α2αk􏰌 f(x)dx ≈ ··· Cα1Cα2 ···Cαk f n , n ,..., n [0,1]k α1 =0 α2 =0 αk =0 The error is still O(h2) = O(n−2), provided that f has continuous partial derivatives ∂2 f/∂xi2. Besides the error involved, one must consider the effort, or work, required to attain a desired level of accuracy. The work in the one-variable case is O(n). In the two-variable case, it is O(n2), and it is O(nk) for k variables. The error, now expressed as a function of the number of nodes N = nk, is O(h2) = O(n−2) = O􏰀(nk)−2/k􏰁 = O(N−2/k) Thus, the quality of the numerical approximation of the integral declines very quickly as the number of variables, k, increases. Expressed in other terms, if a constant order of accuracy is to be retained while the number of variables, k, goes up, the number of nodes must go up like nk . These remarks indicate why the Monte Carlo method for numerical integration becomes more attractive for high-dimensional integration. (This subject is discussed in Chapter 10.) Remarks For historical reasons, formulas for approximating definite integrals are called rules such as the trapezoid rule and many other rules, some of which are found in the exercises and Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. subsequent chapters of this book. A large collection of these quadrature rules can be found in Abramowitz and Stegun [1964], Standard Mathematical Tables, which had its origins in a U.S. Federal Government public works project during the Great Depression in the 1930s. The word quadrature has several meanings both in mathematics and in astronomy. In the dictionary, the first mathematical meaning is the process of finding a square whose area is equal to the area enclosed by a given curve. The general mathematical meaning is the process of determining the area of a surface, especially one bounded by a curve. We use it primarily to mean the approximation of the area under a curve in numerical integration. In the exercises of this chapter, we have used various well-known integrals to illustrate numerical integration. Many of these integrals have been thoroughly investigated and tabu- lated. Examples are elliptic integrals, the sine integral, the Fresnel integral, the logarithmic integral, the error function, and Bessel functions. When one is faced with a daunting inte- gral, the first question to raise is whether the integral has already been studied and perhaps tabulated. The first places to search are websites on the Internet and books such as M. Abramowitz and I. Stegun [1964]. In modern scientific computing, such tables are of lim- ited use because of the ready availability of software packages such as MATLAB, Maple, and Mathematica. Nevertheless, one needs a basic understanding of numerical methods to intelligently use mathematical software. Summary 5.1 • To estimate 􏰦ab f (x) dx, divide the interval [a, b] into subintervals according to the partitionP={a=x0 α > 0. Notice that when α = 2, the combination in Equation (13) is the one we have already used for the second column in the Romberg array, namely, R(n, 1).
Extrapolation of the same type can be used in still more general situations, as is illus- trated next (and in the exercises).
If φ is a function with the property
φ(x)=L+a1x−1 +a2x−2 +a3x−3 +···
how can L be estimated using Richardson extrapolation?
Obviously, L = limx→∞ φ(x). Thus, L can be estimated by evaluating φ(x) for a succession
of ever larger values of x. To use extrapolation, we write
φ(x) = L +a1x−1 +a2x−2 +a3x−3 +···
φ(2x) = L +2−1a1x−1 +2−2a2x−2 +2−3a3x−3 +··· 2φ(2x) = 2L + a1x−1 + 2−1a2x−2 + 2−2a3x−3 + ···
φ(x) = 2φ(2x) − φ(x) = L − 2−1a2x−2 − 3 · 2−2a3x−3 − ···
Having computed φ(x) and φ(2x), we can compute a new function ψ(x), which should be a better approximation to L because its error series begins with x−2 and is O(x−2) as x → ∞. This process can be repeated, as in the Romberg algorithm. ■
Here is a concrete illustration of the preceding example. We want to estimate limx→∞ φ(x) from the following table of numerical values:
x 1 2 4 8 16 32 64 128 φ(x) 21.1100 16.4425 14.3394 13.3455 12.8629 12.6253 12.5073 12.4486
A tentative hypothesis is that φ has the form in the preceding example. When we compute the values of the function ψ(x) = 2φ(2x) − φ(x), we get a new table of values:
x 1 2 4 8 16 32 64
ψ (x ) 11.7750 12.2363 12.3516 12.3803 12.3877 12.3893 12.3899
It seems reasonable to believe that the value of limx→∞ φ(x) is approximately 12.3899. If we do another extrapolation, we should compute θ(x) = [4ψ(2x) − ψ(x)]/3; values for this table are
Thus, the special linear combination
2α 􏰋h􏰌 1 􏰋h􏰌 1􏰜􏰋h􏰌 􏰝
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

224 Chapter 5
Numerical Integration
x 1 2 4 8 16 32
θ (x ) 12.3901 12.3900 12.3899 12.3902 12.3898 12.3901
For the precision of the given data, we conclude that limx→∞ φ(x) = 12.3900 to within roundoff error.
Summary 5.2
• By using the recursive trapezoid rule, we find that the first column of the Romberg algorithm is
􏰄n−1 12
R(n, m) = R(n, m − 1) + 1 [R(n, m − 1) − R(n − 1, m − 1)] 4m −1
with n ≧ 1 and m ≧ 1. The error is O(h2) for the first column, O(h4) for the second column, O(h6) for the third column, and so on. Check the ratios
R(n, m) − R(n − 1, m) ≈ 4m+1 as h → 0
R(n + 1, m) − R(n, m) to test whether the algorithm is working.
• IftheexpressionLisapproximatedbyφ(h)andiftheseentitiesarerelatedbytheerror series
L = φ(h) + ahα + bhβ + chγ + · · · then a more accurate approximation is
R(n, 0) =
where h = (b − a)/2n and n ≧ 1. The second and successive columns in the Romberg
R(n − 1, 0) + h
array are generated by the Richardson extrapolation formula and are
2
f [a + (2k − 1)h]
k=1
a1. What is R(5,3) if R(5,2) = 12 and R(4,2) = −51, in the Romberg algorithm?
6. Using the Romberg scheme, establish a numerical value for the approximation
with error O(hβ ).
L ≈φ􏰋h􏰌+ 1 􏰜φ􏰋h􏰌−φ(h)􏰝 2 2α−1 2
2. If R(3,2) = −54 and R(4,2) = 72, what is R(4,3)? 􏰒12
a3. Compute R(5, 2) from R(3, 0) = R(4, 0) = 8 and 0 R(5, 0) = −4.
4. Let f (x) = 2x . Approximate 􏰦04 f (x) dx by the trape- zoid rule using partition points 0, 2, and 4. Repeat by us- ing partition points 0, 1, 2, 3, and 4. Now apply Romberg extrapolation to obtain a better approximation. 􏰦
a5. By the Romberg algorithm, approximate 02 4 d x / (1 + x2) by evaluating R(1, 1).
e−(10x) dx ≈ R(1,1)
Compute the approximation to only three decimal places
of accuracy.
a7. W􏰦 e are going to use the Romberg method to estimate
01√xcosxdx. Will the method work? Will it work well? Explain.
a8. BycombiningR(0,0)andR(1,0)forthepartitionP= {−h < 0 < h}, determine R(1, 1). Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. Exercises 5.2 9. a 10. 11. 5.2 Romberg Algorithm 225 Show that the precise form of Equation (5) is 􏰒∞ b 􏰄􏰋4j −1􏰌 f(x)dx = R(n,1)− a2j+2h2j+2 a j=1 3×4j 20. DeriveEquation(6),andshowthatitspreciseformis 􏰒 b 􏰄∞ 􏰋 4 j − 1 􏰌 􏰋 4 j − 1 − 1 􏰌 2 j + 2 f(x)dx = R(n,2)+ 3×4j 15×4j−1 a2j+2h j=2 In calculus, a technique of integration by substitution is developed. For example, if the substitution x = z2 19. 􏰦1x√ is􏰦made in the integral 0 (e / x) dx, the result is 2 01 ez2 dz. Verify this and discuss the numerical aspects of this example. Which form is likely to produce a more accurate answer by the Romberg method? How many evaluations of the function (integrand) are needed if the Romberg array with n rows and n columns is to be constructed? UsingEquation(2),fillinthecirclesinthefollowingdi- agram with coefficients used in the Romberg algorithm: a R(0, 0) R(1, 0) R(2, 0) R(3, 0) R(3, 1) R(3, 2) R(3, 3) R(4, 0) R(4, 1) R(4, 2) R(4, 3) R(4, 4) 21. a 22. a 23. a UsethefactthatthecoefficientsinEquation(3)havethe form ak =ck[f(k−1)(b)− f(k−1)(a)] to prove that 􏰦ab f (x) dx = R(n, m) if f is a polynomial ofdegree≦2m−2. In 􏰦the Romberg algorithm, R(n, 0) denotes an estimate of ab f(x)dx withsubintervalsofsizeh =(b−a)/2n. If it were known that R(1, 1) R(2, 1) 12. a 13. a14. a 15. 16. a17. a18. R(2, 2) Derive the quadrature rule for R(1,1) in terms of the function f evaluated at partition points a, a + h, and a+2h,whereh =(b−a)/2.Dothesamefor R(n,1) withh=(b−a)/2n. (Continuation) Derive the quadrature rule R(2,2) in terms of the function f evaluated at a, a + h, a + 2h, a+3h,andb,whereh =(b−a)/4. We want to compute X = limn→∞ Sn, and we have al- ready computed the two numbers u = S10 and v = S30. Itisknownthat X = Sn +Cn−3.Whatis X intermsof uandv? Suppose that we want to estimate Z = limh→0 f (h) and that we calculate f (1), f (2−1), f (2−2), f (2−3), . . . , f (2−10). Then suppose also that it is known that Z = f(h)+ah2 +bh4 +ch6.Showhowtoobtain an improved estimate of Z from the 11 numbers already computed. Show how Z can be determined exactly from any 4 of the 11 computed numbers. Show how Richardson extrapolation works on a sequence x1,x2,x3,... that converges to L as n → ∞ in such a waythatL−xn =a2n−2+a3n−3+a4n−4+···. Let xn be a sequence that converges to L as n → ∞. If L−xn isknowntobeoftheforma3n−3+a4n−4+··· (in which the coefficients are unknown), how can the convergence of the sequence be accelerated by taking combinations of xn and xn+1? IftheRombergalgorithmisoperatingonafunctionthat possesses continuous derivatives of all orders on the in- te􏰦rval of integration, then what is a bound on the quantity | ab f(x)dx−R(n,m)|intermsofh? 􏰒 a how would we have to modify the Romberg algorithm? Show that if f ′′ is continuous, then the first column in the Romberg array converges to the integral in such a way that the error at the nth step is bounded in magnitude by a constant times 4−n . Assuming that the first column of the Romberg array converges to ab f (x ) d x , show that the second column does also. Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 24. 25. 􏰦 b f(x)dx = R(n,0)+a3h3 +a6h6 +··· (Continuation) In the preceding problem, we estab- lished the elementary property that if limn→∞ R(n, 0) = 􏰦ab f (x) dx, then limn→∞ R(n, 1) = 􏰦ab f (x) dx. Show that limn→∞ R(n,2) = limn→∞ R(n,3) = ··· 􏰦 =limn→∞R(n,n)= ab f(x)dx 26. a. Using Formula (7), prove that Euler-Maclaurin coef- ficients can be generated recursively: j=1 A0 = 1, b. Determine Ak for1≦k≦6. 􏰄k Ak−j Ak = − ( j + 1)! a27. Evaluate E in the theorem on the Euler-Maclaurin formula for this special case: a = 0, b = 2π, f(x)=1+cos4x,n=4,andmarbitrary. 226 Chapter 5 Numerical Integration a􏰦 for 2.19 x −1 sin x d x . 10. Computelog2byusingtheRombergalgorithmonasuit- able integral. 1. Compute eight rows and columns in the Romberg array 1.3 2. DesignandcarryoutanexperimentusingtheRombergal- a􏰒 1 π gorithm. Suggestions: For a function that possesses many continuous derivatives on the interval, the method should work well. Try such a function first. If you choose one whose integral you can compute by other means, you will acquire a better understanding of the accuracy in the Romberg algorithm. For example, try definite integrals for 􏰒each of these: (1+x)−1dx =ln(1+x), 􏰒􏰒 ex dx =ex, (1+x2)−1dx =arctanx 3. Test the Romberg algorithm on a bad function, such as √ cos(xsinθ)dθ 11. TheBesselfunctionoforder0isdefinedbytheequation J0(x)= π Calculate J0(1) by applying the Romberg algorithm to the integral. 12. RecodetheRombergproceduresothatallthetrapezoid rule results are computed first and stored in the first col- umn. Then in a separate procedure, procedures Extrapolate(n, (ri )) carry out Richardson extrapolation, and store the results in the lower triangular part of the (ri ) array. What are the advantages and disadvantages of this procedure over run. 13. (StudentResearchProject)StudytheClenshaw-Curtis method for numerical quadrature. If possible, read the original paper by Clenshaw and Curtis [1960] and then program the method. If programmed well, it should be su- perior to the Romberg method in many cases. For further information on it, consult papers by Dixon [1974], Fraser and Wilson [1966], Gentleman [1972], Havie [1969], Kahaner [1971], and O’Hara and Smith [1968]. 0 x on [0, 1]. Why is it bad? 4. The transcend􏰒ental number π is the area of a circle whose 1/√2 􏰓 8 ( 1−x2−x)dx=π 0 with the help of a diagram, and use this integral to ap- the routine given in the text? Test on the two integrals radius is 1. Show that 􏰦 4 dx/(1 + x) and 􏰦 1 ex dx using only one computer 0 −1 proximate π by the Romberg method. a5. Apply the Romberg method to estimate 􏰦π −1 0 (2 + sin 2x ) d x . Observe the high precision ob- tained in the first column of the array, that is, by the simple trapezoidal estimates. a6. Compute 􏰦0π x cos 3x d x by the Romberg algorithm us- 14. ing n = 6. What is the correct answer? (Student Research Project) Numerical integration is an ideal problem for use on a parallel computer, since the interval of integration can be subdivided into subinter- simultaneously and independently of each other. Investi- a7. An integral of the form 􏰦 ∞ f (x) dx can be trans- 0 vals on each of which the integral can be approximated formed into an integral on a finite interval by making a change of variable. Verify, for instance, t􏰦hat the sub- stituti􏰦on x = − ln y changes the integral 0∞ f (x) dx i􏰦nto 01 y−1 f(−lny)dy. Use this idea to compute ∞−x 2 0 [e /(1 + x􏰒)] d x by means of the Romberg algo- rithm, using 128 evaluations of the transformed function. 8. BytheRombergalgorithm,calculate ∞√ e−x 1−sinxdx gate how numerical integration can be done in parallel. If you have access to a parallel computer or can simulate a parallel computer on a collection of PCs, write a parallel program to approximate π by using the standard example 􏰒1 (1 + x2)−1 dx 0 with a basic rule such as the midpoint rule. Vary the num- ber of processors used and the number of subintervals. You can read about parallel computing in books such as Pacheco [1997], Quinn [1994], and others or at any of the numerous sites on the Internet. Use a mathematical software system with symbolic ca- pabilities to verify the relationship between Ak and the Bernoulli numbers for k = 6. 0 9. Calculate 􏰒 1 sinx √ dx x 15. Hint: Consider making a change of variable. 0 by the Romberg algorithm. Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole deemed that any suppressed content does not materially affect the overall learning experience. or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. Computer Exercises 5.2 Interval [a, b] The basic trapezoid rule for approximating 􏰦ab f (x ) d x is based on an estimation of the area beneath the curve over the interval [a, b] using a trapezoid. The function of integration f (x) is taken to be a straight line between f (a) and f (b). The numerical integration formula is of the form 􏰒b a where the values of A and B are selected so that the resulting integration rule correctly integrates any linear function. It suffices to integrate exactly the two functions 1 and x because a polynomial of degree at most 1 is a linear combination of these two monomials. (This process is also called the method of undetermined coefficients.) To simplify the calculations, let a = 0 and b = 1 and find a formula of the following type: 􏰒1 f(x)dx ≈ Af(0)+ Bf(1) 0 Thus, these equations should be fulfilled: 􏰒1 􏰒0 11 f (x) = 1 : dx = 1 = A + B f(x)=x: The solution is A = B = 1 , and the integration formula is 5.3 Simpson’s Rules and Newton-Cotes Rules 227 5.3 Simpson’s Rules and Newton-Cotes Rules Basic Trapezoid Rule f(x)dx ≈ Af(a)+ Bf(b) 0 2 􏰒11 xdx= =B 2 f(x)dx≈ [f(0)+f(1)] 02 By a linear mapping y = (b − a)x + a from [0, 1] to [a, b], the basic trapezoid rule for the interval [a, b] is obtained: Basic Trapezoid Rule 􏰒b1 f(x)dx ≈ a2 See Figure 5.4 for a graphical illustration. (b−a)[f(a)+ f(b)] (1) f (b) x FIGURE 5.4 Basic trapezoid rule ab p1(x) f (a) y 5 f(x) Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 228 Chapter 5 Numerical Integration Basic Simpson’s Rule The next obviou􏰦s generalization is to take two subintervals 􏰗a, a+b 􏰘 and 􏰗 a+b , b􏰘 and 22 to approximate b f (x ) d x by taking the function of integration f (x ) to be a quadratic a 􏰒 􏰀a+b􏰁 polynomialpassingthroughthethreepoints f(a), f 2 ,and f(b).Letusseekanumerical integration rule of the following type: Intervals b 􏰋 􏰌 􏰗a, a+b􏰘,􏰗a+b,b􏰘 f(x)dx≈Af(a)+Bf a+b +Cf(b) 22a2 The function f is assumed to be continuous on the interval [a, b]. The coefficients A, B, and C are chosen such that the formula above gives correct values for the integral whenever f is a quadratic polynomial. It suffices to integrate correctly the three functions 1, x, and x2 because a polynomial of degree at most 2 is a linear combination of those 3 monomials. To simplify the calculations, let a = −1 and b = 1 and consider the formula 􏰒1 f(x)dx ≈ Af(−1)+ Bf(0)+Cf(1) −1 Thus, these equations should be fulfilled: 􏰒1 􏰒−1 1 f (x) = 1 : f (x) = x : dx = 2 = A + B + C x dx = 0 = −A + C f(x)=x2: The solution is A = 1 , C = 1 , and B = 4 . The resulting rule is 􏰒−1 12 x2dx= =A+C Basic Simpson’s Rule 􏰒b 1 􏰜 􏰋a+b􏰌 􏰝 f(x)dx≈ (b−a) f(a)+4f +f(b) (2) a62 See Figure 5.5 for an illustration. 3 􏰒11 f(x)dx ≈ [f(−1)+4f(0)+ f(1)] −1 333 −1 3 Using a linear mapping y = 1(b − a)x + 1(a + b) from [−1,1] to [a,b], we obtain the 22 basic Simpson’s rule over the interval [a, b]: f(a 1 b ) 2 p2(x) f (a) f (b) p2(x) FIGURE 5.5 Basic Simpson’s rule aa1bb 2 x y 5 f(x) Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. FIGURE 5.6 Example of trapezoid rule vs. Simpson’s rule EXAMPLE 1 Solution aa1bb 2 Figure 5.6 shows graphically the difference between the basic trapezoid rule (1) and the basic Simpson’s rule (2). Find approximate values for the integral 􏰒12 e−x dx 0 using the basic trapezoid rule and the basic Simpson’s rule. Carry five significant digits. Let a = 0 and b = 1. For the basic trapezoid rule (1), we obtain 􏰒12 1􏰜􏰝 e−x dx ≈ e0 + e−1 ≈ 0.5[1 + 0.36788] = 0.68394 02 which is correct to only one significant decimal place (rounded). For the basic Simpson’s rule (2), we find 􏰒121􏰗􏰘 e−x dx ≈ e0 + 4e−0.25 + e−1 06 ≈ 0.16667[1 + 4(0.77880) + 0.36788] = 0.7472 which is correct to three significant decimal places (rounded). Recall that 􏰦1 e−x2 dx = 1√0 2 π erf(1) ≈ 0.74682. ■ Basic Simpson’s Rule: Uniform Spacing A numerical integration rule over two equal subintervals with partition points a, a + h, and a + 2h = b is the widely used basic Simpson’s rule: 􏰒 a+h h f(x)dx≈ [f(a)+4f(a+h)+f(a+2h)] (3) a3 Simpson’s rule computes exactly the integral of an interpolating quadratic polynomial over an interval of length 2h using three points; namely, the two endpoints and the middle point. It can be derived by integrating over the interval [0, 2h] the Lagrange quadratic polynomial p2 through the points (0, f (0)), (h, f (h)), and (2h, f (2h)): 􏰒 2h 􏰒 2h h f(x)dx ≈ p2(x)dx = 3[f(0)+4f(h)+ f(2h)] 00 where p2(x)= 1 (x−h)(x−2h)f(0)− 1 x(x−2h)f(h)+ 1 x(x−h)f(2h) 5.3 Simpson’s Rules and Newton-Cotes Rules 229 Simpson p2(x) p1(x) Trapezoid y 5 f(x) Basic Simpson’s Rule 2h2 h2 2h2 Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 230 Chapter 5 Numerical Integration The error term in Simpson’s rule can be established by using the Taylor series from Sec- tion 1.2: f(a+h)= f +hf′ + 1h2 f′′ + 1h3 f′′′ + 1h4 f(4) +··· 2! 3! 4! where the functions f , f ′, f ′′, . . . on the right-hand side are evaluated at a. Now replacing h by 2h, we have 4 24 f(a+2h)= f +2hf′ +2h2 f′′ + h3 f′′′ + h4 f(4) +··· 3 4! Combining these two series, we obtain f(a)+4f(a+h)+ f(a+2h)=6f +6hf′ +4h2 f′′ +2h3 f′′′ + 20h4 f(4) +··· 4! and, thereby, we have h [ f (a) + 4 f (a + h) + f (a + 2h)] = 2h f + 2h2 f ′ + 4 h3 f ′′ 33 + 2h4 f′′′ + 20 h5 f(4) +··· (4) 3 3·4! Hence, we have a series for the right-hand side of Equation (3). Now let’s find one for the left-hand side. The Taylor series for F(a + 2h) is F(a + 2h) = F(a) + 2hF′(a) + 2h2 F′′(a) + 4h3 F′′′(a) 3 2 25 + h4F(4)(a)+ h5F(5)(a)+··· 3 5! Let 􏰒x F(x) = f (t) dt a By the Fundamental Theorem of Calculus, F′ = f . We observe that F(a) = 0 and F(a + 2h) is the integral on the left-hand side of Equation (3). Since F′′ = f ′, F′′′ = f ′′, and so on, we have 􏰒 a+2h f(x)dx = [f(a)+4f(a+h)+ f(a+2h)]− a 3 90 4 2 25 h3 f′′ + h4 f′′′ + h5 f(4) +··· (5) 3 3 5·4! 􏰒 a+2h h h5 f(x)dx =2hf +2h2 f′ + Subtracting Equation (4) from Equation (5), we obtain a f(4) −··· Basic Simpson’s Rule Error Term with error term 􏰒b (b−a)􏰜 􏰋a+b􏰌 􏰝 f(x)dx ≈ f(a)+4f + f(b) a62 A more detailed analysis shows that the error term for the basic Simpson’s rule (2) is −(h5/90) f (4)(ξ) = O(h5) as h → 0, for some ξ between a and a + 2h. By letting b = a + 2h, the basic Simpson’s rule over the interval [a, b] is for some ξ in (a, b). 1􏰋b−a􏰌5 (4) −90 2 f (ξ) (6) Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 5.3 Simpson’s Rules and Newton-Cotes Rules 231 Composite Simpson’s Rule Suppose that the interval [a, b] is subdivided into an even number of subintervals, say n, eachofwidthh=(b−a)/n.Thenthepartitionpointsarexi =a+ihfor0≦i≦n,where n is divisible by 2. Now from basic calculus, we have Interval [a, b] with n 􏰒􏰄􏰒 (even) Subintervals of Width h b n/2 a+2ih f(x)dx = f(x)dx (7) a i=1 a+2(i−1)h Using the basic Simpson’s rule, we have, for the right-hand side, ≈􏰄n/2 h{f(a+2(i−1)h)+4f(a+(2i−1)h)+ f(a+2ih)} i=1 3 (n/2)−1 + f(a+2ih)+ f(b) i=1 h􏰤 i=1 i=1 (n/2)−1 n/2 =3 f(a)+ 􏰄 f(a+2ih)+4􏰄􏰄f(a+(2i−1)h) 􏰥 Error Term Caution where h = (b − a)/n. The error term is − 1 (b−a)h4 f(4)(ξ) (8) 180 Many formulas for numerical integration have error estimates that involve derivatives of the function being integrated. An important point that is frequently overlooked is that such error estimates depend on the function having derivatives. So if a piecewise function is being integrated, the numerical integration should be broken up over the region to coincide with the regions of smoothness of the function. Another important point is that no polynomial ever becomes infinite in the finite plane, so any integration technique that uses polynomials to approximate the integrand may fail to give good results without extra work at integrable singularities. An Adaptive Simpson’s Scheme Now we develop an adaptive scheme based on Simpson’s rule for obtaining a numerical approximation to the integral 􏰒b f(x)dx a In this adaptive algorithm, the partitioning of the interval [a, b] is not selected beforehand but is automatically determined. The partition is generated adaptively so that more and smaller subintervals are used in some parts of the interval and fewer and larger subintervals are used in other parts. In the adaptive process, we divide the interval [a,b] into two subintervals and then decide whether each of them is to be divided into more subintervals. This procedure is continued until some specified accuracy is obtained throughout the entire interval [a,b]. Thus, we obtain 􏰒b 􏰤 n/2 (n−2)/2 􏰥 Composite Simpson’s f(x)dx≈h [f(a)+f(b)]+4􏰄f[a+(2i−1)h]+2 􏰄 f(a+2ih) Rule a 3 i=1 i=1 Adaptive Process for Subintervals Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 232 Chapter 5 Numerical Integration Since the integrand f may vary in its behavior on the interval [a, b], we do not expect the final partitioning to be uniform, but to vary in the density of the partition points. It is necessary to develop the test for deciding whether subintervals should continue to be divided. One application of Simpson’s rule over the interval [a, b] can be written as where and 􏰒b a S(a,b)= (b−a)􏰜 f(a)+4f􏰋a+b􏰌+ f(b)􏰝 62 1􏰋b−a􏰌5 (4) E(a,b)=− f (a)+··· I ≡ f(x)dx = S(a,b)+ E(a,b) Letting h = b − a, we have where and 90 2 I = S(1) + E(1) (9) S(1) = S(a,b) (1) 1􏰋h􏰌5 (4) E =− f (a)+··· 90 2 1 􏰋h􏰌5 =−C 90 2 Here we assume that f (4) remains a constant value C throughout the interval [a, b]. Now two applications of Simpson’s rule over the interval [a, b] give I = S(2) + E(2) (10) where where c = (a + b)/2, as in Figure 5.7, and (2) 1 􏰋h/2􏰌5 E =− 90 2 (4) f (a)+···− 1 􏰋h/2􏰌5 (4) S(2) =S(a,c)+S(c,b) f (c)+··· 1􏰜 1􏰋h􏰌5 􏰝 =− 1 􏰋h/2􏰌5􏰜f(4)(a)+ f(4)(c)􏰝+··· 90 2 90 2 1􏰋1􏰌􏰋h􏰌5 =−90 25 2 (2C)=16−90 2 C a h c 5 (a 1 b)/2 b One Simpson’s Rule Two Simpson’s Rules h/2 acb FIGURE 5.7 Simpson’s rule h/2 Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 5.3 Simpson’s Rules and Newton-Cotes Rules 233 Again, we use the assumption that f (4) remains a constant value C throughout the interval [a, b]. We find that Subtracting Equation (10) from (9), we have From this equation and Equation (10), we have Stopping Test Recursive Process Split Interval [a, b] in Half to guide the adaptive process so that |I − S(2)| < ε. If Test (11) is not satisfied, the interval [a, b] is split into two subintervals, [a, c] and [c, b], where c is the midpoint c = (a + b)/2. On each of these subintervals, we again use Test (11) with ε replaced by ε/2 so that the resulting tolerance is ε over the entire interval [a, b]. A recursive procedure handles this quite nicely. To see why, we take ε/2 on each left and right subinterval 􏰒b􏰒c􏰒b I = f(x)dx = f(x)dx + f(x)dx = Ileft + Iright aac If S is the sum of approximations S(2) over [a, c] and S(2) over [c, b], we have We use the inequality I=S(2)+E(2)=S(2)+ 1􏰀S(2)−S(1)􏰁 15 1 􏰣􏰣S(2) − S(1)􏰣􏰣 < ε (11) 15 16E(2) = E(1) S(2) − S(1) = E(1) − E(2) = 15E(2) Four Subintervals of Width (b − a)/4 |I−S|=􏰣􏰣I +I −S(2)−S(2) 􏰣􏰣 left right left right ≦ 􏰣􏰣I −S(2)􏰣􏰣+􏰣􏰣I −S(2) 􏰣􏰣 left left right right = 1 􏰣􏰣 S ( 2 ) − S ( 1 ) 􏰣􏰣 + 1 􏰣􏰣 S ( 2 ) − S ( 1 ) 􏰣􏰣 15 left left 15 right right using Inequality (11). Hence, if we require 1 􏰣􏰣S(2) −S(1)􏰣􏰣< ε and 1 􏰣􏰣S(2) −S(1) 􏰣􏰣< ε 15 left left 2 15 right right 2 then |I − S| < ε over the entire interval [a, b]. We now describe an adaptive Simpson recursive procedure. The interval [a, b] is parti- tioned into four subintervals of width (b − a)/4. Two Simpson approximations are computed by using two double-width subintervals and four single-width subintervals; that is, where h = b − a and c = (a + b)/2. According to Inequality (11), if one simpson and two simpson agree to within 15ε, then the interval [a, b] does not need to be subdivided further to obtain an accurate approximation left right one simpson← h􏰜 f(a)+4f􏰋a+b􏰌+ f(b)􏰝 62 two simpson← h 􏰜 f(a)+4f􏰋a+c􏰌+2f(c)+4f􏰋c+b􏰌+ f(b)􏰝 12 2 2 Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 234 Chapter 5 Adaptive Process Numerical Integration Sample Integral to the integral 􏰦ab f (x ) d x . In this case, the value of [16 (two simpson) − (one simpson)]/15 is used as the approximate value of the integral over the interval [a,b]. If the desired accuracy for the integral has not been obtained, then the interval [a, b] is divided in half. The subintervals [a, c] and [c, b], where c = (a + b)/2, are used in a recursive call to the adaptive Simpson procedure with tolerance ε/2 on each. This procedure terminates when all subintervals satisfy Inequality (11). Alternatively, a maximum number of allowable levels of subdividing intervals is used to terminate the procedure prematurely. The recursive procedure provides an elegant and simple way to keep track of which subintervals satisfy the tolerance test and which need to be divided further. Example Using Adaptive Simpson Procedure The main program for calling the adaptive Simpson procedure can best be presented in terms of a concrete example. An approximate value for the integral 5π􏰜 􏰝 􏰒 4 cos(2x) dx (12) 0 ex is desired with accuracy 1 × 10−3. 2 1 0.8 0.6 0.4 0.2 0 2 0.2 f (x) 5 cos(2x)/ex FIGURE 5.8 Ada􏰦ptive Integration 5 of 4πcos(2x)/exdx 0 0 0.5 1 1.5 2 2.5 3 3.5 4 Adaptive Procedure The graph of the integrand function is shown in Figure 5.8. This function has many turns and twists, so accurately determining the area under the curve may be difficult. A function procedure f is written for the integrand. Its name is the first argument in the procedure, and necessary interface statements may be needed. In the pseudocode below, other arguments are the values of the upper and lower limits a and b of the integral, the desired accuracy ε, the level of the current subinterval, and the maximum level depth. recursive real function Simpson( f, a, b, ε, level, level max) result(simpson result) integer level, level max; external function f level ← level + 1 h←b−a real a, b, c, d, e, h c ← (a + b)/2 one simpson←h[f(a)+4f(c)+ f(b)]/6 (Continued) Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. Simpson Pseudocode Trapezoid Rule: Simpson’s 1 Rule: 3 Simpson’s 3 Rule: 􏰒x1 x0 1 1 3 ′′ f(x)dx=2h[f0+f1]−12h f (ξ) 5.3 Simpson’s Rules and Newton-Cotes Rules 235 d ← (a + c)/2 e ← (c + b)/2 two simpson←h[f(a)+4f(d)+2f(c)+4f(e)+ f(b)]/12 if level ≧ level max then simpson result ← two simpson output “maximum level reached” else if |two simpson − one simpson| < 15ε then simpson result ← two simpson + (two simpson − one simpson)/15 else end if end if end function Simpson left simpson ← Simpson( f, a, c, ε/2, level, level max) right simpson ← Simpson( f, c, b, ε/2, level, level max) simpson result ← left simpson + right simpson By writing a driver computer program for this pseudocode and executing it on a computer, we obtain an approximate value of 0.208 for the integral. The adaptive Simpson procedure uses a different number of panels for different parts of the curve, as shown in Figure 5.8. Newton-Cotes Rules Newton-Cotes quadrature formulas for approximating 􏰦ab f (x ) d x are obtained by approxi- matingthefunctionofintegration f(x)withinterpolatingpolynomials.Therulesareclosed when they involve function values at the ends of the interval of integration. Otherwise, they are said to be open. Inthefollowing,a = x0,b = xn,xi = x0+ih,fori = 0,1,...,n,whereh = (b−a)/n, fi = f(xi),anda=x0 <ξ b.
Continuity of a function f at a point s can be defined by the condition lim f(x)= f(s)= lim f(x)
x→s+ x→s−
Here, limx→s+ means that the limit is taken over x values that converge to s from above s; that is, (x − s) is positive for all x values. Similarly, limx→s− means that the x values converge to s from below.
degree 1), whose pieces are linear polynomials joined together to achieve continuity, as in Figure6.2.Thepointst0,t1,…,tn atwhichthefunctionchangesitscharacteraretermed knots in the theory of splines. Thus, the spline function shown in Figure 6.2 has eight knots.
Such a function appears somewhat complicated when defined in explicit terms. We are forced to write
S (x), x ∈[t ,t ]  0 0 1
S1(x), x ∈ [t1,t2]
S(x)= . . (1)
where
..
Sn−1(x), x∈[tn−1,tn]
Si (x) = ai x + bi
because each piece of S(x) is a linear polynomial. Such a function S(x) is piecewise linear. Iftheknotst0,t1,…,tn weregivenandifthecoefficientsa0,b0,a1,b1,…,an−1,bn−1 were all known, then the evaluation of S(x) at a specific x would proceed by first determining the interval that contains x and then using the appropriate linear function for that interval.
If the function S defined by Equation (1) is continuous, we call it a first-degree spline. It is characterized by the following three properties.
(2)
Spline of Degree 1
A function S is called a spline of degree 1 if:
1. ThedomainofSisaninterval[a,b].
2. Siscontinuouson[a,b].
3. Thereisapartitioningoftheintervala=t0 tn, then S(x) = yn−1 + mn−1(x − tn−1).
real function Spline1(n, (ti ), (yi ), x)
integer i,n; real x; real array (ti)0:n,(yi)0:n fori =n−1to0 step−1
ifx−ti ≧0thenexitloop end for
Spline1←yi +(x−ti)[(yi+1−yi)/(ti+1−ti)] end function Spline1
First-Degree Polynomial Accuracy Theorem
If p is the first-degree polynomial that interpolates a function f at the endpoints of an interval [a, b], then with h = b − a, we have
|f(x)−p(x)|≦ω(f;h) (a≦x≦b)
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

256 Chapter 6
Spline Functions
Then we have
|f(x)− p(x)| ≦ 􏰋x −a􏰌|f(x)− f(b)|+􏰋b−x􏰌|f(x)− f(a)| b−a b−a
≦ 􏰋x−a􏰌ω(f;h)+􏰋b−x􏰌ω(f;h) b−a b−a
= 􏰜􏰋x−a􏰌+􏰋b−x􏰌􏰝ω(f;h)=ω(f;h) b−a b−a
First-Degree Spline Accuracy Theorem
Let p be a first-degree spline having knots a = x0 < x1 <··· 0,letδhave thepropertythat|f(x)− f(y)| < εwhenever|x−y| < δ (uniform continuity principle). Let n > 1 + (b − a)/δ. Show that there is a first-degree spline S having n knots suchthat|f(x)−S(x)|<εon[a,b]. Hint: Use Exercise 6.1.5 and assume equally spaced knots. Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. Exercises 6.1 262 Chapter 6 Spline Functions a9. a 10. 11. a12. 13. 14. 15. 16. a 17. 18. If the function f (x ) = sin(100x ) is to be approxi- mated on the interval [0, π ] by an interpolating spline of degree 1, how many knots are needed to ensure that |S(x) − f (x)| < 10−8? (Use equally spaces knots.) Hint: Use Exercise 6.1.7. Let t0 < t1 < · · · < tn . Construct first-degree spline functions G0, G1, . . . , Gn by requiring that Gi vanish at t0,t1,...,ti−1,ti+1,...,tn but that Gi(ti) = 1. Show that the first-degree spline function that interpolates f at t0,t1,...,tn is􏰃n f(ti)Gi(x). i=0 Show that the trapezoid rule for numerical integration (Section 5.1) results from approximating f by a first- degree spline S and then using a 19. 20. What equations must be solved if a quadratic spline func- tion Q that has knots t0,t1,...,tn is required to take prescribedvaluesatpoints 1(ti +ti+1)for0≦i≦n−1? 2 Are these functions quadratic splines? Explain why or why not. 􏰒b 􏰒b f(x)dx ≈ aa ac. Q(x)=x2 (1≦x≦2) 4 (2 ≦ x ≦ 50) Is S(x) = |x| a first-degree spline? Why or why not? VerifythatFormula(5)hasthethreepropertiesQi(ti)= yi, Qi′(ti) = zi, and Qi′(ti+1) = zi+1. (Continuation) Impose the continuity condition on Q and derive the system of Equation (6). ShowbyinductionthattherecursiveFormula(6)together with Equation (5) produces an interpolating quadratic spline function. Verify the correctness of the equations in the text that pertain to Subbotin’s spline interpolation process. Analyze the Subbotin interpolation scheme in this alter- native manner. First, let vi = Q(ti ). Show that Qi (x) = Ai (x − ti )2 + Bi (x − ti+1)2 + Ci vi −Ci h2 i = vi+1, and S(x)dx Prove that the derivative of a quadratic spline is a first- a 21. 22. 23. 24. 25. 26. degree spline. If the knots ti happen to be the integers 0,1,...,n, find a good way to determine the index i for which ti ≦ x < ti+1. (Note: This problem is deceptive, for the word good can be given different meanings.) Show that the indefinite integral of a first-degree spline is a second-degree spline. Define f(x)=0ifx <0and f(x)=x2 ifx≧0.Show that f and f ′ are continuous. Show that any quadratic spline with knots t0,t1,...,tn is of the form g(x)= 0 (t0≦x≦0) x (0≦x≦tn) Prove that every first-degree spline function that has knots t0,t1,...,tn can be written in the form x −1 0 1 1 2 5 22 y210123 Assume that z0 = 0. (Continuation) Show that no quadratic spline Q inter- polates the table of the preceding problem and satisfies Q′(t0) = Q′(t5). ax2+bx+c+ Define a function g by the equation 􏰂 􏰄n−1 i=1 di f(x−ti) 􏰂 0.1x 2 (0 ≦ x ≦ 1) 2 aa. Q(x)= ab. Q(x)= −x2 (−100≦x≦0) 9.3x −18.4x+9.2 (1≦x≦1.3) x (0 ≦ x ≦ 100) 􏰂  x (−50 ≦ x ≦ 1) where Ci = 2yi − 2vi − 2vi+1, Bi = 1 1 Ai=vi+1−Ci, hi=ti+1−ti ax + b + Find a quadratic spline interpolant for these data: (Continuation) When continuity conditions on Q′ are imposed, show that the result is the following equation, in which i = 1,2,...,n − 1: 􏰄n−1 i=1 Hint: Qi (τi ) = yi . Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole deemed that any suppressed content does not materially affect the overall learning experience. hi vi−1 + 3(hi + hi+1)vi + hi−1vi+1 = 4hi−1 yi + 4hi yi−1 28. (Student Research Project) It is commonly accepted that Schoenberg’s [1946] paper is the first mathematical reference in which the word spline is used in connec- tion with smooth, piecewise polynomial approximations. However, the word spline as a thin strip of wood used by a draftsman dates back to the 1890s at least. Many of the or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. ci g(x − ti ) 27. hi2 Show that Qi (ti ) = vi , Qi (ti+1) ideas used in spline theory have their roots in work done in various industries such as the building of aircraft, auto- mobiles, and ships in which splines are used extensively. 1. RewriteprocedureSpline1sothatascendingsubintervals are considered instead of descending ones. Test the code on a table of 15 unevenly spaced data points. 2. RewriteprocedureSpline1sothatabinarysearchisused to find the desired interval. Test the revised code. What are the advantages and disadvantages of a binary search compared to the procedure in the text? A binary search is similar to the bisection method in that we choose tk withk=(i+j)/2ork=(i+j+1)/2anddetermine whether x is in [ti,tk] or [tk,tj]. 3. A piecewise bilinear polynomial that interpolates points (x, y) specified in a rectangular grid is given by p(x,y)= (lijzi+1,j+1+li+1,j+1zij) (xi+1 − xi)(yj+1 − yj) − (li+1, j zi, j+1 + li, j+1zi+1, j ) (xi+1 − xi)(yj+1 − yj) wherelij =(xi −x)(yj −y).Herexi≦x≦xi+1 and yj ≦y≦yj+1. The given grid (xi,yj) is specified by strictly increasing arrays (xi) and (yj) of length n and m, respectively. The given values zij at the grid points (xi,yj)arecontainedinthen×m array(zij),shownin the following figure. Write code for realfunctionBi Linear((xi),n,(yj),m,(zij),x,y) to compute the value of p(x,y). Test this routine on a set of 5 × 10 unequally spaced data points. Evaluate Bi Linear at four grid points and five nongrid points. 6.2 Natural Cubic Splines Introduction Research and write a paper on the history of splines. (See the mathematical history of splines in the automobile industry.) zij yj yj11 xi 4. Write an adaptive spline interpolation procedure. The input should be a function f , an interval [a, b], and a tolerance ε. The output should be a set of knots a = t0 h1 >0.Ifui−1 >hi−1, then ui > hi because
h2
ui =2(hi +hi−1)− i−1 >2(hi +hi−1)−hi−1 >hi
ui−1
Then by induction, ui > 0 for i = 1,2,…,n − 1.
Equation (5) is not the best computational form for evaluating the cubic polynomial Si (x ). We would prefer to have it in the form
Si(x)= Ai +Bi(x−ti)+Ci(x−ti)2 +Di(x−ti)3 (11)
because nested multiplication can then be utilized.
Notice that Equation (11) is the Taylor expansion of Si about the point ti . Hence, we
obtain
A =S(t), B =S′(t), C = 1S′′(t), D = 1S′′′(t) iiiiiii2iii6ii
Therefore, we have Ai = yi and Ci = zi /2. The coefficient of x 3 in Equation (11) is Di , whereas the coefficient of x3 in Equation (5) is (zi+1 − zi )/6hi . Therefore, we obtain
1
Di = 6h (zi+1 −zi)
i
Finally, Equation (6) provides the value of Si′ (ti ), which is
Bi =−hizi+1−hizi + 1(yi+1−yi) 6 3 hi
Thus, the nested form of Si (x ) is
Si(x)=yi +(x−ti)􏰋Bi +(x−ti)􏰋zi + 1 (x−ti)(zi+1−zi)􏰌􏰌 (12)
2 6hi
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

Pseudocode for Natural Cubic Splines
We now write routines for determining a natural cubic spline based on a table of values and for evaluating this function at a given value. First, we use Algorithm 1 for directly solving the tridiagonal System (10). This procedure, called Spline3 Coef , takes n + 1 table values (ti , yi ) in arrays (ti ) and (yi ) and computes the zi ’s, storing them in array (zi ). Intermediate (working) arrays (hi ), (bi ), (ui ), and (vi ) are needed.
Now a procedure called Spline3 Eval is written for evaluating Equation (12), the natural cubic spline function S(x), for x a given value. The procedure Spline3 Eval first determines the interval [ti , ti+1] that contains x and then evaluates Si (x) using the nested form of this cubic polynomial:
6.2 Natural Cubic Splines 271
procedure Spline3 Coef (n, (ti ), (yi ), (zi ))
integer i, n; real array (ti )0:n , (yi )0:n , (zi )0:n
allocate real array (hi )0:n−1, (bi )0:n−1, (ui )1:n−1, (vi )1:n−1 fori =0ton−1
hi ←ti+1−ti
bi ←(yi+1−yi)/hi end for
u1 ← 2(h0 + h1) v1 ← 6(b1 − b0) fori =2ton−1
u←2(h+h )−h2 /u
i i i−1 i−1 i−1
vi ← 6(bi − bi−1) − hi−1vi−1/ui−1 end for
zn ←0
fori =n−1to1 step−1
zi ←(vi −hizi+1)/ui end for
z0 ← 0
deallocate array (hi ), (bi ), (ui ), (vi ) end procedure Spline3 Coef
Spline3 Coef Pseudocode
real function Spline3 Eval(n, (ti ), (yi ), (zi ), x) integer i; real h,tmp
real array (ti )0:n , (yi )0:n , (zi )0:n
fori =n−1to0 step−1
ifx−ti ≧0thenexitloop end for
h ← ti+1 − ti
tmp←(zi/2)+(x−ti)(zi+1 −zi)/(6h) tmp←−(h/6)(zi+1 +2zi)+(yi+1 −yi)/h+(x−ti)(tmp) Spline3 Eval ← yi + (x − ti )(tmp)
end function Spline3 Eval
Spline3 Eval Pseudocode
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

272 Chapter 6
Spline Functions
EXAMPLE 3
Solution
The function Spline3 Eval can be used repeatedly with different values of x after one call to procedure Spline3 Coef . For example, this would be the procedure when plotting a natural cubic spline curve. Since procedure Spline3 Coef stores the solution of the tridiagonal sys- tem corresponding to a particular spline function in the array (zi ), the arguments n, (ti ), (yi ), and (zi ) must not be altered between repeated uses of Spline3 Eval.
Using Pseudocode for Interpolating and Curve Fitting
To illustrate the use of the natural cubic spline routines Spline3 Coef and Spline3 Eval, we rework Example 3 from Section 4.1 (p. 166).
Write pseudocode for a program that determines the natural cubic spline interpolant for sin x at 10 equidistant knots in the interval [0, 1.6875]. Over the same interval, subdivide each subintervalintofourequallyspacedparts,andfindthepointwherethevalueof|sinx−S(x)| is largest.
Here is a suitable pseudocode main program, which calls procedures Spline3 Coef and Spline3 Eval:
program Test Spline3 integeri; reale,h,x
real array (ti )0:n , (yi )0:n , (zi )0:n integer n ← 9
real a ← 0, b ← 1.6875
h ← (b − a)/n
fori =0ton
ti ←a+ih
yi ←sin(ti) end for
call Spline3 Coef (n, (ti ), (yi ), (zi )) temp ← 0
for j = 0 to 4n
x ← a + jh/4
e ← | sin(x) − Spline3 Eval(n, (ti ), (yi ), (zi ), x)| if e > temp then temp ← e
output j,x,e
end for
end Test Spline3
Test Spline3 Pseudocode
Mathematical Software
Theoutputis j =19,x =0.890625,andd =0.930×10−5. ■ We can use mathematical software to plot the cubic spline curve for this data. Caution: MATLAB routine spline uses the not-a-knot end condition!
It dictates that S′′′ be a single constant in the first two subintervals and another single constant in the last two subintervals. First, the original data are generated. Next, a finer subdivision of the interval [a, b] on the x-axis is made, and the corresponding y-values are obtained from the procedure spline. Finally, the original data points and the spline curve are plotted.
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

Sample Data Set
Mathematica and Maple can be used to plot cubic spline curves as well as many other computer software packages and programs.
We now illustrate the use of spline functions in fitting a curve to a set of data. Consider the following table:
x 0.0 0.6 1.5 1.7 1.9 2.1 2.3 2.6 2.8 3.0 y −0.8 −0.34 0.59 0.59 0.23 0.1 0.28 1.03 1.5 1.44
3.6 4.7 5.2 5.7 5.8 6.0 6.4 6.9 7.6 8.0
0.74 −0.82 −1.27 −0.92 −0.92 −1.04 −0.79 −0.06 1.0 0.0
These 20 points were selected from a wiggly freehand curve drawn on graph paper. We intentionally selected more points where the curve bent sharply. A visually pleasing curve is obtained by using the cubic spline routines Spline3 Coef and Spline3 Eval and plotting the resulting natural cubic spline curve. (See Figure 6.8.)
6.2 Natural Cubic Splines 273
y
2 1.5 1 0.5
y 5 S(x)
FIGURE 6.8
Natural cubic spline curve
2D Parametric Cubic Splines
0x 12345678
– 0.5 –1 – 1.5 –2
Alternatively, we can use mathematical software such as MATLAB, Maple, or Math- ematica to plot the cubic spline function for this table.
Space Curves
In two dimensions, two cubic spline functions can be used together to form a parametric representation of a complicated curve that turns and twists. Select points on the curve and label them t = 0,1,…,n. For each value of t, read off the x- and y-coordinates of the point, thus producing a table:
t01···n x xn y y0 y1 ··· yn
Then fit x = S(t) and y = S(t), where S and S are natural cubic spline interpolants. The two functions S and S give a parametric representation of the curve. (See Computer Exercises 6.2.6.)
x0
x1
···
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

274
Chapter 6 Spline Functions
EXAMPLE 4
Select 13 points on the well-known serpentine curve given by x
y = 1/4 + x2
So that the knots will not be equally spaced, write the curve in parametric form:
􏰈x = 1 tanθ 2
y = sin 2θ
and take θ = i (π/14), where i = −6, −5, . . . , 5, 6. Plot the natural cubic spline curve and
the interpolation polynomial in order to compare them.
This is an example of curve fitting using the polynomial interpolation routines Coef and Eval from Chapter 4 (p. 166) and the cubic spline routines Spline3 Coef and Spline3 Eval. Figure 6.9 shows the resulting cubic spline curve and the high-degree polynomial curve (dashed line). The polynomial becomes extremely erratic after the fourth knot from
Solution
the origin and oscillates wildly, whereas the spline is a near perfect fit.
22 21.5
y
8
6
4
2 21
20.5
0 0.5
Polynomial curve
Cubic spline curve
x
1 1.5 2
FIGURE 6.9
Serpentine curve
EXAMPLE 5
Solution
22 24 26 28
Use cubic spline functions to produce the curve for the following data: t01234567
y 1.0 1.5 1.6 1.5 0.9 2.2 2.8 3.1 It is known that the curve is continuous but its slope is not.
■
A single cubic spline is not suitable. Instead, we can use two cubic spline interpolants, the first having knots 0, 1, 2, 3, 4 and the second having knots 4, 5, 6, 7. By carrying out two separate spline interpolation procedures, we obtain two cubic spline curves that meet at the point (4, 0.9). At this point, the two curves have different slopes. The resulting curve is shown in Figure 6.10. ■
Smoothness Property
Why do spline functions serve the needs of data fitting better than ordinary polynomials? To answer this, one should understand that interpolation by polynomials of high degree is
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

y
3 2.5 2 1.5 1 0.5
y 5 Sˆ ( x )
̃
y 5 S(x)
6.2 Natural Cubic Splines 275
FIGURE 6.10
Two cubic splines
FIGURE 6.11
Wildly oscillating function
Natural Cubic Splines and Avoiding Wild Oscillations
■ Theorem1
Proof
0x 1234567
p
q
often unsatisfactory because polynomials may exhibit wild oscillations. Polynomials are smooth in the technical sense of possessing continuous derivatives of all orders, whereas in this sense, spline functions are not smooth.
Wild oscillations in a function can be attributed to its derivatives being very large. Consider the function whose graph is shown in Figure 6.11. The slope of the chord that joins the points p and q is very large in magnitude. By the Mean-Value Theorem, the slope of that chord is the value of the derivative at some point between p and q. Thus, the derivative must attain large values. Indeed, somewhere on the curve between p and q, there is a point where f ′(x) is large and negative. Similarly, between q and r, there is a point where f ′(x) is large and positive. Hence, there is a point on the curve between p and r where f ′′(x) is large. This reasoning can be continued to higher derivatives if there are more oscillations. This is the behavior that spline functions do not exhibit. In fact, the following result shows that from a certain point of view, natural cubic splines are the best functions to use for curve fitting!
r
Cubic Spline Smoothness Theorem
If S is the natural cubic spline function that interpolates a twice-continuously differ- entiablefunction f atknotsa=t0 2hi + 3hi−1 in the algorithm for
2 determining the cubic spline interpolant.
By hand calculation, find the natural cubic spline inter- polant for this table:
x12345 y01010
Find a cubic spline over knots −1, 0, and 1 such that the following conditions are satisfied: S′′(−1) = S′′(1) = 0, S(−1) = S(1) = 0, and S(0) = 1.
Thisproblemandthenexttwoleadtoamoreefficiental- gorithm for natural cubic spline interpolation in the case of equally spaced knots. Let hi = h in Equation (5), and replace the parameters zi by qi = h2zi /6. Show that the new form of Equation (5) is then
􏰋 x − ti 􏰌3 h
􏰋 ti +1 − x 􏰌3
Si(x) = qi+1
+ qi
+ (yi+1 − qi+1) h
24.
25.
a 26.
27.
28.
+(yi −qi) (Continuation)Establishthenewcontinuityconditions:
Show that Si can also be written in the form
Si(x) = yi + Ai(x −ti)+ 1zi(x −ti)2 + zi+1 − zi (x −ti)3
􏰂α0=0, αi =−(αi−1+4)−1 (1≦i≦n)
β0=0, βi=−αi(yi+1−2yi+yi−1−βi−1) (1≦i≦n)
(This stable and efficient algorithm is due to MacLeod [1973].)
37. Prove that if S(x) is a spline of degree k on [a,b], then S′(x) is a spline of degree k − 1.
a 38. How many coefficients are needed to define a piecewise quartic (fourth-degree) function with n + 1 knots? How many conditions will be imposed if the piecewise quartic function is to be a quartic spline? Justify your answers.
a39. Determinewhetherthisfunctionisanaturalcubicspline: 􏰂 x3 + 3×2 + 7x − 5 (−1 ≦ x ≦ 0)
S(x) = −x3 + 3×2 + 7x − 5 (0 ≦ x ≦ 1)
h
􏰋 x − ti 􏰌 􏰋ti+1 −x􏰌
􏰂35.
36.
(Continuation) Show that the parameters qi can be deter- min􏰂ed by backward recursion as follows:
qn = 0, qn−1 = βn−1
qi =αiqi+1 +βi (i =n−2,n−3,…,0)
where the coefficients αi and βi are generated by ascend- ing recursion from the formulas
q0 =qn =0
qi−1+4qi +qi+1 =yi+1−2yi +yi−1 (1≦i≦n−1)
h
with
2 6hi
hhyy
Ai = − i zi − i zi+1 − i + i+1
3 6 hi hi
29. Carry out the details in deriving Equation (9), starting
with Equation (5).
30. Verify that the algorithm for computing the (zi ) array is correct by showing that if (zi ) satisfies Equation (9), then it satisfies the equation in Step 3 of the algorithm.
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

280 Chapter 6 Spline Functions
40. Determine whether this function is or is not a natural 41.
cubic spline having knots 0, 1, and 2:
􏰂 x 3 + x − 1 (0 ≦ x ≦ 1) f (x) = 3 2
−(x−1) +3(x−1) +4(x−1)+1 (1≦ x ≦ 2)
1. Rewrite and test procedure Spline3 Coef using proce- dure Tri from Section 2.3. Use the symmetry of the (n − 1) × (n − 1) tridiagonal system.
2. The extra storage required in Step 1 of the algorithm for solving the natural cubic spline tridiagonal system di- rectly can be eliminated at the expense of a slight amount of extra computation—namely, by computing the hi ’s and bi ’s directly from the ti ’s and yi ’s in the forward elimi- nation phase (Step 2) and in the back substitution phase (Step 3). Rewrite and test procedure Spline3 Coef using this idea.
3. Using at most 20 knots and the cubic spline routines Spline3 Coef and Spline3 Eval, plot on a computer plot- ter an outline of your:
a. School’s mascot. b. Signature. c. Profile.
4. LetSbethecubicsplinefunctionthatinterpolatesf(x)= (x2 + 1)−1 at 41 equally spaced knots in the interval [−5, 5]. Evaluate S(x) − f (x) at 101 equally spaced points on the interval [0, 5].
5. Draw a free-form curve on graph paper, making certain that the curve is the graph of a function. Then read val- ues of your function at a reasonable number of points, say, 10–50, and compute the cubic spline function that takes those values. Compare the freely drawn curve to the graph of the cubic spline.
6. Draw a spiral (or other curve that is not a function) and reproduce it by way of parametric spline functions. (See the following figure.)
y
Show that the natural cubic spline going through the points (0, 1), (1, 2), (2, 3), (3, 4), and (4, 5) must be y = x + 1. (The natural cubic spline interpolant to a given data set is unique, because the matrix in Equa- tion (10) is diagonally dominant and invertible, as proven in Section 2.3.)
7
3
2 40
81
5 9
6
x
7. Write and test procedures that are as simple as possible to perform natural cubic spline interpolation with equally spaced knots.
Hint: See Exercises 6.2.34–6.2.36.
􏰦b
8. Write a program to estimate a f (x ) d x , assuming that
we know the values of f at only certain prescribed knots a=t0 0 x∈(ti,ti+k+1) (k≧0)
By Equation (2), this assertion is true when k = 0. If it is true for index k−1, then Bk−1(x) > i
0 on (t ,t ) and Bk−1(x) > 0 on (t ,t ). In Equation (3), the factors that multiply i i+k i+1 i+1 i+k+1
Bk−1(x) and Bk−1(x) are positive when t < x < t . Thus, Bk(x) > 0 on this interval.
B2i 3 Bi
The principal use of the B splines Bik(i = 0,±1,±2,…) is as a basis for the set of all kth-degree splines that have the same knot sequence. Thus, linear combinations
􏰄∞
ci Bik
i =−∞
are important objects of study. (We use ci for fixed k and Cik to emphasize the degree k of
the corresponding B splines.)
Our first task is to develop an efficient method to evaluate a function of the form
􏰄∞
Cik Bik(x) (4) Using Definition (3) and some simple series manipulations, we have
f (x) =
under the supposition that the coefficients Cik are given (as well as the knot sequence ti ).
i =−∞
􏰄∞ 􏰜􏰋x−ti􏰌 􏰋ti+k+1−x􏰌 􏰝 f (x) = Ck Bk−1(x) + Bk−1(x)
i =−∞ ∞
= Ck i
i =−∞ ∞
i ti+k −ti i ti+k+1 −ti+1 i+1
􏰄􏰄􏰜􏰋 􏰌 􏰋 􏰌􏰝
x−ti
ti+k −ti i−1 ti+k −ti
+Ck
ti+k −x
Bk−1(x) i
i
Ck−1 Bk−1(x) ii
(5)
=
where Ck−1 is defined to be the appropriate coefficient from the line preceding Equation (5).
i =−∞
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

284
Chapter 6
Spline Functions
j−1 Ci
(7)
Equation (4) and we want to compute f (x), we use Equation (7) to calculate the entries in the following triangular array:
Ck Ck−1···C0 mmm
Coefficients Recurrance Relation
i+j i i+j i
A nice feature of Equation (4) is that only the k + 1 coefficients Ck , Ck
Derivative Property I
Derivative Property II
A basic result about the derivatives of B splines is
d Bk(x) = 􏰋 k 􏰌Bk−1(x) − 􏰋 k 􏰌Bk−1(x) (8)
This algebraic manipulation shows how a linear combination of Bik (x ) can be expressed as a linear combination of Bk−1(x). Repeating this process k−1 times, we eventually express
tained is
Cj−1 =Cj􏰋 x−ti 􏰌+Cj 􏰋ti+j −x􏰌 i it−ti−1t−t
i
f (x) in the form
􏰄∞
Ci0 Bi0(x) (6) If t ≦x < t , then f(x) = C0. The formula by which the coefficients Cj−1 are ob- f (x) = mm+1m i i =−∞ m m−1 are needed to compute f (x) if tm ≦ x < tm+1 (see Exercise 6.3.6). Thus, if f is defined by ... m−k Although our notation does not show it, the coefficients in Equation (4) are independent of x, whereas the C j−1’s calculated subsequently by Equation (7) do depend on x. i It is now a simple matter to establish that 􏰄∞ Bik(x)=1, forallxandallk≧0 i =−∞ Ifk = 0,wealreadyknowthis.Ifk > 0,weuseEquation(4)withCik = 1foralli.
By Equation (7), all subsequent coefficients Ck,Ck−1,Ck−2,…,C0 are also equal to 1 iiii
(induction is needed here!). Thus, at the end, Equation (6) is true with Ci0 = 1, and so f (x) = 1. Therefore, from Equation (4), the sum of all B splines of degree k is unity.
The smoothness of the B splines Bik increases with the index k. In fact, we can show by induction that Bik has a continuous k − 1st derivative.
The B splines can be used as substitutes for complicated functions in many mathematical situations. Differentiation and integration are important examples.
Derivatives of B Splines
Ck Ck−1 m−1 m−1
. … Ck
, …, Ck m−k
dx i ti+k −ti i ti+k+1 −ti+1 i+1
This equation can be proved by induction using the recursive Formula (3). Once Equation (8)
is established, we get the useful formula
d 􏰄∞ 􏰄∞
c Bk(x) = d Bk−1(x) (9)
dxii ii i =−∞ i =−∞
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

where
di =k􏰋ci −ci−1􏰌 ti+k −ti
6.3 B Splines: Interpolation and Approximation 285
The verification is as follows. By Equation (8), we obtain
d 􏰄∞
dx ci Bik(x)
i =−∞
􏰄∞􏰜􏰋k􏰌k−1 􏰋k􏰌k−1􏰝 = ci t −t Bi (x)− t −t Bi+1(x)
=
ci dx Bi (x)
􏰄∞ d k
i =−∞
i=−∞ i+k i i+k+1
= 􏰄∞ 􏰜􏰋 cik 􏰌−􏰋 ci−1k 􏰌􏰝Bk−1(x)
i=−∞ ti+k −ti ti+k −ti i
i =−∞
For numerical integration, the B splines are also recommended, especially for indefinite integration. Here is the basic result needed for integration:
(10)
This equation can be verified by differentiating both sides with respect to x and simplifying by the use of Equation (9). To be sure that the two sides of Equation (10) do not differ by a constant, we note that for any x < ti , both sides reduce to zero. The basic result (10) produces this useful formula: i+1 Integration Property I 􏰒 x 􏰋 t i + k + 1 − t i 􏰌 􏰄∞ Bk(s)ds = Bk+1(x) Mathematical Software MATLAB has a Spline Toolbox, developed by Carl de Boor, that can be used for many tasks involving splines. For example, there are routines for interpolating data by splines with diverse end conditions and routines for least-squares fits to data. There are d Bk−1(x) Integration of B Splines = 􏰄∞ ii Integration Property II where 􏰒 x 􏰄∞ −∞ i=−∞ 􏰄∞ c Bk(s)ds = e Bk+1(x) (11) 1 􏰄i ik+1j −∞ j=i ei =k+1 cj(tj+k+1−tj) ii ii i=−∞ j =−∞ It should be emphasized that this formula gives an indefinite integral (antiderivative) of any function expressed as a linear combination of B splines. Any definite integral can be obtained by selecting a specific value of x . For example, if x is a knot, say, x = tm , then 􏰒 t m 􏰄∞ 􏰄∞ 􏰄m cBk(s)ds= eBk+1(t )= eBk+1(t ) ii iim iim −∞ i=−∞ i=−∞ i=m−k−1 Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 286 Chapter 6 Spline Functions Basic Property Interpolation Conditions B Splines of Degree 0 B Spline of Degree 1 􏰄∞ i i−k many demonstration routines in this Toolbox that exhibit plots and provide models for programming MATLAB M-files. These demonstrations are quite instructive for visualizing and learning the concepts in spline theory, especially B splines. Maple has a BSpline package for constructing B spline basis functions of degree k from a given knot list, which may include multiple knots. It is based on a divided-difference implementation found in Bartels, Beatty, and Barskey [1987]. Mathematica has a splines packages also. Interpolation and Approximation by B Splines We developed a number of properties of B splines and showed how B splines are used in various numerical tasks. The problem of obtaining a B spline representation of a given function was not discussed. Here, we consider the problem of interpolating a table of data; later, a noninterpolatory method of approximation is described. A basic question is how to determine the coefficients in the expression S(x) = A Bk (x) (12) i =−∞ so that the resulting spline function interpolates a prescribed table: x t0 t1 ··· tn y y0 y1 ··· yn We mean by interpolate that S(ti ) = yi (0 ≦ i ≦ n) (13) The natural starting point is with the simplest splines, corresponding to k = 0. Since Bi0(tj)=δij=􏰂1 (i=j) 0 ( i ≠ j ) the solution to the problem is immediate: Just set Ai = yi for 0 ≦ i ≦ n. All other coefficients in Equation (12) are arbitrary. In particular, they can be zero. We arrive then at this result: The B spline of degree 0 B1 (t)=δ i−1j ij Hence, the following is true: The B spline of degree 1 i=0 [t−1, t1] and t3 is the midpoint of [t2, t4]. yi Bi0(x) The next case, k = 1, also has a simple solution. We use the fact that has the interpolation property (13). y B1 (x) has the interpolation property (13). So Ai = yi again. S(x) = S(x) = 􏰄n i=0 􏰄n i i−1 , B1, B1, and B1. They, inturn,requirefortheirdefinitionknotst−1,t0,t1,...,t4.Knotst−1 andt4 canbearbitrary. Figure 6.15 shows the graphs of the four B1 splines. In such a problem, if t−1 and t4 are not prescribed, it is natural to define them in such a way that t0 is the midpoint of the interval If the table has four entries (n = 3), for instance, we use B1 −101 2 Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. Quadratic Case 􏰄∞ 1􏰗 􏰘 AB2 (t)= A(t −t)+A (t−t ) (14) FIGURE 6.15 Bi1 splines t2 t3 t4 x 6.3 B121 B10 t21 t0 t1 B Splines: Interpolation and Approximation 287 B1 B12 In both elementary cases considered, the unknown coefficients A0, A1, . . . , Equation (12) were uniquely determined by the interpolation conditions (13). If terms were present in Equation (12) corresponding to values of i outside the range {0, 1, . . . , n}, then they would have no influence on the values of S(x) at t0, t1, ...,tn. Higher-Degree B Splines For higher-degree splines, we shall see that some arbitrariness exists in choosing coefficients. In fact, none of the coefficients is uniquely determined by the interpolation conditions. This fact can be advantageous if other properties are desired of the solution. In the quadratic case, we begin with the equation i i−2 j t −t j j+1 j j+1 j j−1 i=−∞ j+1 j−1 An in Its justification is left to Exercise 6.3.26. If the interpolation conditions (13) are now im- posed, we obtain the following system of equations, which gives the necessary and sufficient conditions on the coefficients: Aj(tj+1 −tj)+ Aj+1(tj −tj−1)= yj(tj+1 −tj−1) (0≦ j≦n) (15) This is a system of n + 1 linear equations in n + 2 unknowns A0, A1,..., An+1. One way to solve Equation (15) is to assign any value to A0 and then use Equation (15) to compute for A1, A2, . . . , An+1, recursively. For this purpose, the equations could be rewritten as where these abbreviations have been used:  􏰋 t j + 1 − t j − 1 􏰌 Aj+1 =αj +βjAj (0≦ j≦n) (16) αj =yj tj −tj−1  tj −tj+1 (0≦ j≦n) βj = tj −tj−1 To keep the coefficients small in magnitude, we recommend selecting A0 such that the expression 􏰄n+1 􏰸= Ai2 i=0 will be a minimum. To determine this value of A0, we proceed as follows: By successive substitution using Equation (16), we can show that Aj+1 =γj +δjA0 (0≦ j≦n) (17) where the coefficients γj and δj are obtained recursively by this algorithm: 􏰂γ0 =α0, δ0 =β0 γj =αj +βjγj−1, δj =βjδj−1 (1≦ j≦n) (18) Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 288 Chapter 6 Spline Functions Then 􏰸 is a quadratic function of A0 as follows: 􏰸 = A 20 + A 21 + · · · + A 2n + 1 = A20 +(γ0 +δ0A0)2 +(γ1 +δ1A0)2 +···+(γn +δnA0)2 To find the minimum of 􏰸, we take its derivative with respect to A0 and set it equal to zero: d􏰸 dA =2A0 +2(γ0 +δ0A0)δ0 +2(γ1 +δ1A0)δ1 +···+2(γn +δnA0)δn =0 0 This is equivalent to q A0 + p = 0, where 􏰈 q = 1 + δ 02 + δ 12 + · · · + δ n2 p=γ0δ0 +γ1δ1 +···+γnδn Pseudocode and a Curve-Fitting Example A procedure that computes coefficients A0, A1, . . . , An+1 in the manner previously outlined is given now. In its calling sequence, (ti )0:n is the knot array, (yi )0:n is the array of abscissa points, (ai )0:n+1 is the array of Ai coefficients, and (hi )0:n+1 is an array that contains hi = ti − ti−1. Only n, (ti ), and (yi ) are input values. They are available unchanged when the routine is finished. Arrays (ai ) and (hi ) are computed and available as output. procedure BSpline2 Coef (n, (ti ), (yi ), (ai ), (hi )) integer i, n; real δ, γ, p, q real array (ai)0:n+1,(hi)0:n+1,(ti)0:n,(yi)0:n fori =1ton hi ←ti −ti−1 end for h0 ← h1 hn+1 ← hn δ ← −1 γ ← 2y0 p ← δγ q←2 fori =1ton r ← hi+1/hi δ ← −rδ γ ← −rγ + (r + 1)yi p←p+γδ q ← q + δ2 end for a0 ←−p/q fori =1ton+1 ai ←[(hi−1 +hi)yi−1 −hiai−1]/hi−1 end for end procedure BSpline2 Coef BSpline 2 Coef Pseudocode Next we give a procedu􏰃re function BSpline2 Eval for computing values of the quadratic splinegivenbyS(x)= n+1 A B2 (x).Itscallingsequencehassomeofthesamevariables i=0 i i−2 Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. BSpline2 Eval Pseudocode 6.3 B Splines: Interpolation and Approximation 289 as in the preceding pseudocode. The input variable x is a single real number that should lie between t0 and tn . The result of Exercise 6.3.26 is used. real function BSpline2 Eval(n, (ti ), (ai ), (hi ), x) integer i, n; real d, e, x; real array (ai )0:n+1, (hi )0:n+1, (ti )0:n fori =n−1to0 step−1 ifx−ti ≧0thenexitloop end for i←i+1 d ← [ai+1(x − ti−1) + ai (ti − x + hi+1)]/(hi + hi+1) e←[ai(x−ti−1 +hi−1)+ai−1(ti−1 −x+hi)]/(hi−1 +hi) BSpline2 Eval ← [d(x − ti−1) + e(ti − x)]/hi end function BSpline2 Eval Using the table of 20 points from Section 6.2, we can compare the resulting natural cubic spline curve with the quadratic spline produced by the procedures BSpline2 Coef and BSpline2 Eval. The first of these curves is shown in Figure 6.8 (p. 273), and the second is in Figure 6.16. The latter is reasonable, but perhaps not as pleasing as the former. These curves show once again that cubic natural splines are simple and elegant functions for curve fitting. y 2 1.5 1 0.5 y 5 S(x) FIGURE 6.16 Quadratic interpolating spline Quadratic Case 0x 12345678 – 0.5 –1 – 1.5 –2 Schoenberg’s Process An efficient process due to Schoenberg [1967] can also be used to obtain B spline approx- imations to a given function. Its quadratic version is defined by 􏰄∞ 1 S(x)= f(τi)Bi2(x) where τi = 2(ti+1 +ti+2) (19) i =−∞ Here, of course, the knots are {t }∞ , and the points where f must be evaluated are i i=−∞ Equation (19) is useful in producing a quadratic spline function that approximates f . midpoints between the knots. The salient properties of this process are as follows: Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 290 Chapter 6 Spline Functions Schoenberg’s Process 1. If f(x)=ax+b,thenS(x)= f(x). 2. If f (x) ≧ 0 everywhere, then S(x) ≧ 0 everywhere. 3. maxx |S(x)|≦ maxx |f(x)|. 4. Iffiscontinuouson[a,b],ifδ=maxi|ti+1−ti|,andifδ0 x∈(ti,ti+k+1) An efficient method to evaluate a function of the form
is to use
􏰄∞ i =−∞
C ik B ik ( x ) Cj−1=Cj􏰋x−ti 􏰌+Cj 􏰋ti+j−x􏰌
f ( x ) =
i it−ti−1t−t i+j i i+j i
• The derivative of B splines is
d Bk(x) = 􏰋 k 􏰌Bk−1(x) − 􏰋
􏰌Bk−1(x) where di = k (ci − ci −1 )/(ti +k − ti ). A basic result needed for integration is
dx i ti+k −ti i A useful formula is
k
ti+k+1 −ti+1 i+1
−∞
ik+1j j=i
d􏰄∞ 􏰄∞ c Bk(x) =
d Bk−1(x) dxii ii
i =−∞ i =−∞
􏰒 x 􏰋 t i + k + 1 − t i 􏰌 􏰄∞ Bk(s)ds = Bk+1(x)
A resulting useful formula is
􏰃􏰒 x 􏰄∞ 􏰄∞
c Bk(s)ds = e Bk+1(x)
j =−∞
• To determine the coefficients in the expression
−∞ i=−∞
whereei =1/(k+1) i cj(tj+k+1 −tj).
ii ii i=−∞
S(x)=
AB2 (x)
i =−∞
so that the resulting spline function interpolates a prescribed table, we use the condition
Aj(tj+1 −tj)+ Aj+1(tj −tj−1)= yj(tj+1 −tj−1) (0≦ j≦n) Thisisasystemofn+1linearequationsinn+2unknowns A0,A1,…,An+1 thatcan
be solved recursively.
• Schoenberg’s process is an efficient process to obtain B spline approximations to a given function. For example, its quadratic version is defined by
􏰄∞ i =−∞
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
S(x) =
f (τi )Bi2(x)
􏰄∞
i i−k

2.
Whatfunctionsaregeneratedbythefollowingrecursive
definit􏰂ion?
f0(x) = 1, f1(x) = x
fn+1(x) = 2x fn(x) − fn−1(x)
Find an expression for Bi2(x) and verify that it is piece-
on the real line (ti = i). Showthat
this 􏰂recursive definition:
f0(x) = 1, f1(x) = cos x fn+1(x) = 2 f1(x) fn(x) − fn−1(x)
(n ≧ 1)
i i+1 Showthat Bik(x) = B0k(x−ti)iftheknotsaretheintegers
6.3 B Splines: Interpolation and Approximation 295 whereτ = 1(t +t )andtheknotsare{t}∞ .Thepointsτ where f mustbe
i 2 i+1 i+2 i i=−∞ i evaluated are midpoints between the knots.
• Be ́ziercurvesareusedincomputer-aideddesignforproducingacurvethatgoesthrough (or near to) control points, or a curve that can be manipulated easily to give a desired shape. Be ́zier curves use Bernstein polynomials. For a continuous function f defined on [0, 1], the sequence of Bernstein polynomials
􏰄n 􏰋i􏰌
pn(x)= f n φni(x) (n≧1)
i=0
converges uniformly to f . The polynomials φni are
φ n i ( x ) = 􏰍 ni 􏰎 x i ( 1 − x ) n − i ( 0 ≦ i ≦ n )
1.
a
a3.
4. a5.
6.
7.
8.
Showthatthefunctions fn(x)=cosnxaregeneratedby
9.
10. 11.
12.
13. 14.
Forequallyspacedknots,showthatk(k+1)−1Bik(x)lies in the interval with endpoints Bk−1(x) and Bk−1(x).
(n ≧ 1)
wise quadratic. Show that Bi2(x) is zero at every knot
􏰒∞ −∞
k
Bi (x)dx =
ti+k+1 −ti k+1
except
Bi2(ti+1) = ti+1 − ti ,
ti+2 − ti Verify Equation (5).
Bi2(ti+2) = ti+3 − ti+2 ti+3 − ti+1
Show that the class of all spline functions of degree m that have knots x0, x1, . . . , xn includes the class of poly- nomials of degree m.
Establish Equation (8) by induction.
Which B splines Bik have a nonzero value on the interval
(tn , tm )? Explain.
Show that on [t ,t ] we have
Establish that 􏰃∞ f (ti )B1 (x) is a first-degree
i−1
spline that interpolates f at every knot. What is the zero-
i=−∞ degree spline that does so?
a 15.
a 16. a 17. 18. 19.
i i+1
Lethi =ti+1−ti.Showthatif S(x) = 􏰃∞ c B2(x)
for all i, then S(tm) = ym for all m. Hint: Use Exercise 6.3.3.
i=−∞ i i
ci−1hi−1 + ci−2hi = yi (hi + hi−1)
Show that the coefficients C j −1 generated by i
coefficients in order that prove.
∞ ci Bk = 0? State and i=−∞ i
Equa- tion (7) satisfy the condition min Cj−1≦ f(x)≦
maxiCj−1. i
a
Expand the function f(x) = x in an infinite series 􏰃
ii
∞ ci B1. i=−∞ i
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
a
Showthatiftm≦x0)that Bik(tj)=0ifandonlyif j≦i or

 x
0  12
(x < 0) (0 ≦ x < 1) (1 ≦ x < 2) (2 ≦ x < 3) i−1 ti+1 −ti−1 hi +hi−1 B2 (t ) = ti+1 − ti = hi i−2 i ti+1−ti−1 hi+hi−1 wherehi=ti+1−ti. Showbyinductionthatif Aj = 1 􏰗yj−1(tj −tj−2)− Aj−1(tj −tj−1)􏰘 tj−1 −tj−2 for j = 2,3,...,n + 1, then j ≧ i + k + 1. What is the maximum value of Bi and where does it occur? Lettheknotsbetheintegers,andprovethat  2 1 ( 6 x − 3 − 2 x 2 ) a 33. 34. 2 B 02 ( x ) = Establish formulas B2 (ti)= i i−1 = i−1  1 (3 − x )2  2  12 2 0 (x < 0) (0 ≦ x < 1) (1 ≦ x < 2) (2 ≦ x < 3) Show that a linear B spline with integer knots can be written in matrix form as (x ≧ 3) t−th 0  24.  1x3 6  (4−3x(x−2)) 6 B 03 ( x ) =  6 In the theory of Be ́zier curves, using the Bernstein basic point, v0. 1  2  (4+3(x −4)(x −2) )  6  1 (4 − x )3 (3 ≦ x < 4) (x≧4) 􏰄n+1 AiB2 (tj)=yj (0≦j≦n) i−2 i=0 􏰆􏰇􏰆􏰇 −1 1 c1 2 0 c0 b =x (0≦x<1)  10 35. 36. 0 polynomials, show that the curve passes through the first Show that if we set S(x) = t j −1 ≦ x ≦ t j , then 􏰃∞ 2 i=−∞ Ai Bi−2(x) and S(x) = [x 1] where = b10c0 + b11c1 with d = t j+1 and 1 −t 1 [Aj+1(x −tj−1)+ Aj(tj+1 −x)] B01(x)= b11=2−x (1≦x<2) 0 (otherwise) ShowthatthequadraticBsplinewithintegerknotscan be written in matrix form as S(x)= 1 tj −tj−1 [d(x−tj−1)+e(tj −x)] j−1 37. [Aj(x−tj−2)+Aj−1(tj −x)] Verify Equations (17) and (18) by induction, using Equa- tion (16). 􏰙 1 −2 1􏰚􏰙c2􏰚 S(x)=1[x2 x 1] −6 6 0 c1 e= tj −tj−2 27. 2 = b20c0 + b21c1 + b22c2 Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole deemed that any suppressed content does not materially affect the overall learning experience. or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 9 −3 0 c0 where  b  20 B02(x) = 21  b22 (2 ≦ x < 3) (otherwise)  0 Hint: See Exercise 6.3.23. B03(x)= b32  b33  b (1 ≦ x < 2)     b 31 b (0≦x<1) (1≦x<2) (2≦x<3) (3≦x<4) (otherwise) 38. Show that the cubic B spline with integer knots can be 0 Hint: See Exercise 6.3.34. written as S(x)= 1[x3 x2 x 6 −1 3−31c3 1] 12 −24 12 0c2 −48 60 −12 0 c1 64 −44 4 0 c0 ( 0 ≦ x < 1 )   30 6.3 B Splines: Interpolation and Approximation 297 where = b30c0 + b31c1 + b32c2 + b33c3 1. Using an automatic plotter, graph B0k for k = 0, 1, 2, 3, 4. Use integer knots ti = i over the interval [0, 5]. 2. Let ti = i (so the knots are the integer points on the real line). Print a table of 100 values of the function 3B71 + 6B81 − 4B91 + 2B10 on the interval [6,14]. Us- ing a plotter, construct the graph of this function on the given interval. 3. (Continuation) Repeat for the function 3 B72 + 6 B82 − 4B92 + 2B120. Oct. 1582 61, 162 Apr. 1588 63, 455 Nov. 1591 62, 164 Mar. 1607 41, 471 4. Assuming that S(x) = 􏰃n ci Bk (x), write a pro- i=0 i 8. 9. 10. 11. 12. Fit the table with a quadratic B spline, and use it to find the average size of the army during the period given. (The average is defined by an integral.) Rewrite procedures BSpline2 Coef and BSpline Eval so that the array (hi ) is not used. Rewrite procedures BSpline2 Coef and BSpline2 Eval for the special case of equally spaced knots, simplifying the code where possible. Writeapr􏰦oceduretoproduceasplineapproximationto F(x) = ax f(t)dt. Assume that a≦x≦b. Begin by finding a quadratic spline interpolant to f at the n points ti =a+i(b−a)/n.Testyourprogramonthefollowing: a. f(x)=sinx (0≦x≦π) b. f(x)=ex (0≦x≦4) c. f(x)=(x2+1)−1 (0≦x≦2) Write a procedure to produce a spline function that ap- proximates f ′(x) for a given f on a given interval [a, b]. Begin by finding a quadratic spline interpolant to f at n + 1 points evenly spaced in [a, b], including endpoints. Test your procedure on the functions suggested in the preceding computer exercise. Define f on [0, 6] to be a polygonal line that joins points (0, 0), (1, 2), (3, 3), (5, 3), and (6, 0). Determine spline approximationsto f,usingSchoenberg’sprocessandtak- ing 7, 13, 19, 25, and 31 knots. cedure to evaluate S′(x) at a specified x. Input is n,k,x,t ,...,t and c ,c ,...,c . 0 n+k+1 0 􏰦1 n 5. Write a procedure to evaluate b S(x)dx, using the as- 􏰃n a k sumption that S(x) = i=0 ci Bi (x). Input will be n,k,a,b,c0,c1,...,cn,t0,...,tn+k+1. 6. (March of the B splines) Produce graphs of several B splines of the same degree marching across the x-axis. Use mathematical software such as MATLAB, Maple, or Mathematica. a7. Historians have estimated the size of the Spanish Army of Flanders in the Spanish Netherlands as follows: Date Number Jan. 1575 59, 250 Sept. 1572 67, 259 May 1576 51, 457 Dec. 1573 62, 280 Feb. 1578 27, 603 Mar. 1574 62, 350 Sept. 1580 45, 435 Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. Computer Exercises 6.3 298 Chapter 6 Spline Functions 13. Write suitable code to calculate 􏰃∞ f(si)B2(x) duce the following figure. i=−∞ i with si = 1 (ti+1 + ti+2). Assume that f is defined 2 on [a, b] and that x will lie in [a, b]. Assume also that t1 < a < t2 and tn+1 < b < tn+2. (Make no assumption about the spacing of knots.) 14. Write a procedure to carry out this approximation scheme: where S(x) = 􏰄∞ i =−∞ 18. Showhowtousemathematicalsoftwaresuchasfoundin f (τi )Bi3(x), MATLAB, Maple, or Mathematica to plot the functions corresponding to a. Figure 6.14. b. Figure 6.15. c. Figure 6.18. d. Figure 6.19. 19. (Computer-AidedGeometricDesign)Usemathemati- cal software for drawing two-dimensional Be ́zier spline curves, and graph the script number five shown, us- ing spline points and control points. See Farin [1990], Sauer [2012], and Yamaguchi [1988] for additional de- tails. 1 τi = 3(ti+1 + ti+2 + ti+3) Assume that f is defined on [a,b] and that τi = a + ih for 0 ≦ i ≦ n, where h = (b − a)/n. 15. UsingamathematicalsoftwaresystemsuchasMATLAB, Maple, or Mathematica with B spline routines, compute and plot the spline curve in Figure 6.16 (p. 273) based on the 20 data points from Section 6.2. Vary the degree of the B splines from 0, 1, 2, 3, through 4 and observe the resulting curves. 16. Using B splines, write a program to perform a natural cubic spline interpolation at knots t0 < t1 < · · · < tn . 17. The documentation preparation system LATEX is widely available and contains facilities for drawing some simple curves such as Be ́zier curves. Use this system to repro- Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 7 Initial Values Problems In a simple electrical circuit, the current I in amperes is a function of time: I(t). The function I(t) satisfies an ordinary differential equation of the form dI = f(t,I) dt Here, the right-hand side is a function of t and I that depends on the circuit and on the nature of the electromotive force supplied to the circuit. A model to account for the way in which two different animal species sometimes interact is the predator-prey model. If u(t) is the number of individuals in the predator species and v(t) the number of individuals in the prey species, then under suitable simplifying assumptions and with appropriate constants a, b, c, and d,   d u   d t = a ( v + b ) u  dv dt =c(u+d)v This is a pair of nonlinear ordinary differential equations (ODEs) that govern the populations of the two species (as functions of time t). Numerical procedures are developed for solving such problems. 7.1 Taylor Series Methods ODE Examples Equation x′ − x = et x′′ + 9x = et x′ + (1/2)x = 0 Solution x(t) = tet + cet x(t) = c1 sin 3t + c2 cos 3t x(t) = √c − t First, we present a general discussion of ordinary differential equations and their solutions. Initial-Value Problem: Analytical versus Numerical Solution An ordinary differential equation (ODE) is an equation that involves one or more deriva- tives of an unknown function. A solution of a differential equation is a specific function that satisfies the equation. Here are some examples of differential equations with their solutions. In each case, t is the independent variable and x is the dependent variable. Thus, x is the name of the unknown function of the independent variable t: Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 299 300 Chapter 7 Initial Values Problems IVP Standard Form In these three examples, the letter c denotes an arbitrary constant. The fact that such constants appear in the solutions is an indication that a differential equation does not, in general, determine a unique solution function. When occurring in a scientific problem, a differential equation is usually accompanied by auxiliary conditions that (together with the differential equation) specify the unknown function precisely. In this chapter, we concentrate on one type of differential equation and one type of auxiliary condition: the initial-value problem for a first-order differential equation. The standard form that has been adopted is 􏰈x′ = f(t,x) (1) x(a) is given It is understood that x is a function of t, so the differential equation written in more detail looks like this: dx(t) = f(t,x(t)) dt Problem (1) is termed an initial-value problem because t can be interpreted as time and t = a can be thought of as the initial instant in time. We want to be able to determine the value of x at any time t before or after a. Here are some examples of initial-value problems, together with their solutions: IVP Examples Equation x′ = x + 1 x′ = 6t − 1 x′=t/(x+1) Initial Value x(0) = 0 x(1) = 6 x(0)=0 Solution x = et − 1 x = 3t2 − t + 4 √ x= t2+1−1 Direct Approach of Indefinite Integration 􏰈x′ = 3t2 −4t−1 +(1+t2)−1 x(5) = 17 The differential equation can be integrated to produce x(t)=t3 −4lnt+arctant+C (2) Although many methods exist for obtaining analytical solutions of differential equa- tions, they are primarily limited to special differential equations. When applicable, they produce a solution in the form of a formula, such as shown in the preceding examples. Fre- quently, however, in practical problems, a differential equation is not amenable to solution by special methods, and a numerical solution must be sought. Even when a formal solution can be obtained, a numerical solution may be preferable, especially if the formal solution is very complicated. A numerical solution of a differential equation is usually obtained in the form of a table; the functional form of the solution remains unknown insofar as a specific formula is concerned. The form of the differential equation adopted here permits the function f to depend on t and x. If f does not involve x, as in the second preceding example, then the differential equation can be solved by a direct process of indefinite integration. To illustrate, consider the initial-value problem The constant C can then be chosen so that x(5) = 17. We can use a mathematical soft- ware system such as MATLAB, Maple, or Mathematica to solve this differential equation explicitly and thereby find the value of this constant as C = 4 ln(5) − arctan(5) − 108. Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. No Closed-Form Solution We often want a numerical solution to a differential equation because (a) the closed- form solution may be very complicated and difficult to evaluate or (b) there is no other choice; that is, no closed-form solution can be found. Consider, for instance, the differential equation √ x′ =e− t2−sint +ln|sint+tanht3| (3) The solution is obtained by taking the integral or antiderivative of the right-hand side. It can be done in principle, but not in practice. In other words, a function x exists for which dx/dt is the right-hand member of Equation (3), but it is not possible to write x(t) in terms of familiar functions. Solving ordinary differential equations on a computer may require a large number of steps with small step size, so a significant amount of roundoff error can accumulate. Consequently, multiple-precision computations may be necessary on small-word-length computers. An Example of a Practical Problem Many practical problems in dynamics involve Newton’s three Laws of Motion, particularly the Second Law. It states symbolically that F = ma, where F is the force acting on a body of mass m and a is the resulting acceleration of that body. This law is a differential equation in disguise because a, the acceleration, is the derivative of velocity and velocity is, in turn, the derivative of the position. We illustrate with a simplified model of a rocket being fired at time t = 0. Its motion is to be vertically upward, and we measure its height with the variable x. The propulsive force is a constant value, namely, 5370. (Units are chosen to be consistent with each other.) There is a negative force due to air resistance whose magnitude is v3/2/ ln(2 + v), where v is the velocity of the rocket. The mass is decreasing at a steady rate due to the burning of fuel and is taken to be 321 − 24t. The independent variable is time, t. The fuel is completely consumed by the time t = 10. There is a downward force, due to gravity, of magnitude 981. Putting all these terms into the equation F = ma, we have 5370 − 981 − v3/2/ ln(2 + v) = (321 − 24t)v′ (4) The initial condition is v = 0 at t = 0. We shall develop methods to solve such differential equations in the succeeding sec- tions. Moreover, one can also invoke a mathematical software system to solve this problem. A computer code for solving ordinary differential equations produces a table of discrete values, whereas the mathematical solution is a continuous function. One may need additional values within an interval for various purposes, such as plotting. Interpolation procedures can be used to obtain all values of the approximate numerical solution within a given interval. For example, a piecewise polynomial interpolation scheme may yield a numerical solution that is continuous and has a continuous first derivative matching the derivative of the solution. In using any ODE solver, an approximation to x′(t) is available from the fact that x′(t)= f(t,x) Mathematical packages for solving ODEs may include automatic plotting capabilities be- cause the best way to make sense out of the large amount of data that may be returned as the solution is to display the solution curves on a graphical monitor or plot them on paper. Simplified Model of a Rocket Using an ODE Computer Code Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 7.1 Taylor Series Methods 301 302 Chapter 7 Initial Values Problems Integration Rules Yield ODE Methods Solving Differential Equations and Integration There is a close connection between solving differential equations and integration. Consider the differential equation  d x = f ( r , x )  dr x(a) = s Integrating from t to t + h, we have 􏰒 t+h 􏰒 t+h t dx = tt f(r,x(r))dr Hence, we obtain x(t + h) = x(t) + Replacing the integral with one of the numerical integration rules from Chapter 5, we obtain a formula for solving the differential equation. For example, Euler’s method, Equation (6), is obtained from the left rectangle approximation. (See Exercise 5.2.33): 􏰒 t+h f(r,x(r))dr ≈hf(t,x(t)) The trapezoid rule 􏰒 t+h h f(r,x(r))dr ≈ t2 gives the formula h x(t+h)=x(t)+ Since x(t + h) appears on both sides of this equation, it is called an implicit formula. If Euler’s method x(t + h) = x(t) + h f (t, x(t)) is used for the x(t + h) on the right-hand side, then we obtain the Runge-Kutta formula of order 2—namely, Equation (10) in Section 7.2. Using the Fundamental Theorem of Calculus, we can easily show that an approximate numerical value for the integral 􏰒b f(r,x(r))dr a can be computed by solving the following initial-value problem for x(b):  d x = f ( r , x )  dr x(a) = 0 􏰒 t+h t f (r, x(r)) dr [f(t,x(t))+ f(t +h,x(t +h))] Integration versus Solving IVP 2 [f(t,x(t))+ f(t+h,x(t+h))] Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. Generic First-Order ODE Vector Fields Consider a generic first-order differential equation with prescribed initial condition: 􏰈x′(t)= f(t,x(t)) x(a) = b Before addressing the question of solving such an initial-value problem numerically, it is helpful to think about the intuitive meaning of the equation. The function f provides the slope of the solution function in the t x -plane. At every point where f (t , x ) is defined, we can imagine a short line segment being drawn through that point and having the prescribed slope. We cannot graph all of these short segments, but we can draw as many as we wish, in the hope of understanding how the solution function x(t) traces its way through this forest of line segments while keeping its slope at every point equal to the slope of the line segment drawn at that point. The diagram of line segments illustrates discretely the so-called vector field of the differential equation. For example, let us consider the equation x′ =sin(x+t2) with initial value x (0) = 0. In the rectangle described by the inequalities −4 ≦ x ≦ 4 and −4 ≦ t ≦ 4, we can direct mathematical software, such as MATLAB, to furnish a picture of the vector field engendered by our differential equation. Using commands in an environment of windows, we bring up a window with the differential equation shown in a rectangle. Behind the scenes, the mathematical software carries out immense calculations to provide the vector field for this differential equation, and displays it, correctly labeled. To see the solution going through any point in the diagram, it is necessary only to use the mouse to position the pointer on such a point. By clicking the left mouse button, the software displays the solution sought. By using such a software tool, one can see immediately the effect of changing initial conditions. For the problem under consideration, several solution curves (corresponding to different initial values) are shown in Figure 7.1. First Vector Field Example x 4 3 2 1 0 21 22 23 x9 5 sin(x 1 t2) 7.1 Taylor Series Methods 303 FIGURE 7.1 Vector field and some solution curves for x′=sin(x+t2) Second-Order Field Example 24 24 23 22 21 0 1 2 3 4 Another example, treated in the same way, is the differential equation x′ = x2 − t Figure 7.2 (p. 304) shows a vector field for this equation and some of its solutions. Notice the phenomenon of many quite different curves all seeming to arise from the same initial Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. t 304 Chapter 7 Initial Values Problems Taylor Series ■ Theorem1 x 4 3 2 1 0 21 22 23 x9 = x2 2 t FIGURE 7.2 Vector field and some solution curves for x′ = x2 − t 24 22 0 2 4 6 8 10 condition. What is happening here? This is an extreme example of a differential equation whose solutions are exceedingly sensitive to the initial condition! Expect trouble in solving this differential equation with an initial value prescribed at t = −2. How do we know that the differential equation x′ = x2 − t, together with an initial value, x(t0) = x0, has a unique solution? There are many theorems in the subject of differ- ential equations that concern such existence and uniqueness questions. One of the easiest to use is as follows. From the theorem just quoted, we cannot conclude that the solution in question is defined for |t − t0| < β. However, the value of ǫ in the theorem is at least β/M, where M is an upper bound for | f (t, x)| in the original rectangle. Taylor Series Methods The numerical method described in this section does not have the utmost generality, but it is natural and capable of high precision. Its principle is to represent the solution of a differential equation locally by a few terms of its Taylor series. In what follows, we shall assume that our solution function x is represented by its Taylor series∗ x(t + h) = x(t) + hx′(t) + 1 h2x′′(t) + 1 h3x′′′(t) 2! 3! + 1 h4x(iv)(t)+ 1 h5x(v)(t)+···+ 1 hmx(m)(t)+··· (5) 4! 5! m! ∗Rememberthatsomefunctionssuchase−1/x2 aresmooth,butnotrepresentedbyaTaylorseriesat0. t Uniqueness of Initial-Value Problems If f and∂f/∂yarecontinuousintherectangledefinedby|t−t0|<αand|x−x0|<β, then the initial-value problem x′ = f (t, x), x(t0) = x0 has a unique continuous solution in some interval |t − t0| < ǫ. Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. Euler’s Method For numerical purposes, the Taylor series truncated after m + 1 terms enables us to compute x(t + h) rather accurately if h is small and if x(t),x′(t),x′′(t),...,x(m)(t) are known. When only terms through hm x(m)(t)/m! are included in the Taylor series, the method that results is called the Taylor series method of order m. We begin with the case m = 1. Euler’s Method and Pseudocode The Taylor series method of order 1 is known as Euler’s method. To find approximate values of the solutions to the initial-value problem 􏰈x′ = f(t,x(t)) x(a) = xa over the interval [a, b], the first two terms in the Taylor series (5) are used: x(t + h) ≈ x(t) + hx′(t) Hence, the formula x(t + h) = x(t) + h f (t, x(t)) (6) can be used to step from t = a to t = b with n steps of size h = (b − a)/n. The pseudocode for Euler’s method can be written as follows, where some prescribed values for n, a, b, and xa are used: Euler Pseudocode To use this program, a code for f (t , x ) is needed, as shown in Example 1. Using Euler’s method, compute an approximate value for x(2) for the differential equation x′ = 1 + x2 + t3 with the initial value x(1) = −4 using 100 steps. Use the pseudocode above with the initial values given and combine with the following function: The computed value is x(2) ≈ 4.23585. ■ Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. EXAMPLE 1 Solution 7.1 Taylor Series Methods 305 program Euler integer k; real h, t; integer n ← 100 external function f real a ← 1, b ← 2, x ← −4 h ← (b − a)/n t←a output 0, t, x for k = 1 to n x ← x + h f (t , x ) t←t+h output k, t, x end for end program Euler real function f (t, x) real t, x f ←1+x2 +t3 end function 306 Chapter 7 Initial Values Problems We can write a computer program to execute Euler’s method on this very simple problem: 􏰈x′(t) = x x(0) = 1 We obtain the results x(2) ≈ 7.3891. The plot produced by the code is shown in Figure 7.3. The solution, x(t) = et, is the solid curve, and the points produced by Euler’s method are shown by dots. Can you understand why the dots are always below the curve? y FIGURE 7.3 Euler’s method curves Important Questions Sample IVP 50 40 30 20 10 01234 x Before accepting these results and continuing, one should raise some questions such as: How accurate are the answers? Are higher-order Taylor series methods ever needed? Unfortunately, Euler’s method is not very accurate because only two terms in the Taylor series (5) are used. Consequently, the truncation error is O(h2). Taylor Series Method of Higher Order Example 1 can be used to explain the Taylor series method of higher order. Consider again the initial-value problem 􏰈x′ =1+x2 +t3 x(1) = −4 (7) If the functions in the differential equation are differentiated several times with respect to t, the results are as follows. (Remember that a function of x must be differentiated with respect to t by using the chain rule.) x′ = 1 + x2 + t3 x′′ =2xx′+3t2 x′′′ = 2xx′′ + 2x′x′ + 6t x(iv) = 2xx′′′ + 6x′x′′ + 6 (8) If numerical values of t and x(t) are known, these four formulas, applied in order, yield x′(t), x′′(t), x′′′(t), and x(iv)(t). Thus, it is possible from this work to use the first five terms in the Taylor series, Equation (5). Since x(1) = −4, we have a suitable starting point, and we select n = 100, which determines h. Next, we can compute an approximation to x(a+h) from Formulas (5) and (8). The same process can be repeated to compute x(a + 2h) using x(a + h), x′(a + h), . . . , x(iv)(a + h). Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. Pseudocode and Results Here is the pseudocode: liable. Here is a coarse assessment. Since terms up to 1 h4x(iv)(t) are included, the first 24 7.1 Taylor Series Methods 307 program Taylor integer k; real h, t, x, x′, x′′, x′′′, x(iv) integer n ← 100 real a ← 1, b ← 2, x ← −4 h ← (b − a)/n t←a output 0, t, x for k = 1 to n x′ ←1+x2 +t3 x′′ ←2xx′ +3t2 x′′′ ←2xx′′ +2(x′)2 +6t t ← a + kh output k, t, x end for end program Taylor x(iv) ←2xx′′′ +6x′x′′ +6 x ← x +h􏰗x′ + 1h􏰗x′′ + 1h􏰗x′′′ + 1h􏰗x(iv)􏰘􏰘􏰘􏰘 234 Taylor Pseudocode Computer Results A few words of explanation may be helpful here. In this example, we compute the solution of the differential equation over the interval a = 1 ≦ t ≦ 2 = b using 100 steps. In each step, the current value of t is an integer multiple of the step size h. The assignment statements that define x′, x′′, x′′′, and x(iv) are simply carrying out calculations of the derivatives according to Equation (8). The final calculation carries out the evaluation of the Taylor series in Equation (5) using five terms. Since this equation is a polynomial in h, it is evaluated most efficiently by using nested multiplication, which explains the formula for x in the pseudocode. The computation t ← t + h may cause a small amount of roundoff error to accumulate in the value of t. This is minimized by using t ← a + kh. As one might expect, the results of using only two terms in the Taylor series (Euler’s method) are not as accurate as when five terms are used: Euler’s Method Taylor Series Method (Order 4) x (2) ≈ 4.23585 41 x (2) ≈ 4.37120 96 By further analysis, one can prove that the correct value to more significant figures is x (2) ≈ 4.37122 1866. Here, the computations were done with more precision just to show that lack of precision was not a contributing factor. Types of Errors When the pseudocode described above is programmed and run on a computer, what sort of accuracy can we expect? Are all the digits printed by the machine for the variable x accurate? Of course not! On the other hand, it is not easy to say how many digits are re- term not included in the Taylor series is 1 h5x(v)(t). The error may be larger than this, 120 Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 308 Chapter 7 Initial Values Problems Local Truncation Error Roundoff Error but the factor h5 = (10−2)5 ≈ 10−10 is affecting only the tenth decimal place. The printed solution is perhaps accurate to eight decimal places. Bridges or airplanes should not be built on such shoddy analysis, but for now, our attention is focused on the general form of the procedure. Actually, there are two types of errors to consider. At each step, if x(t) is known and x(t +h) is computed from the first few terms of the Taylor series, an error occurs because we have truncated the Taylor series. This error, then, is called the truncation error or, to be more precise, the local truncation error. In the preceding example, it is roughly 1 h5x(v)(ξ). 120 In this situation, we say that the local truncation error is of order h5, abbreviated by O(h5). The second type of error obviously present is due to the accumulated effects of all local truncation errors. Indeed, the calculated value of x(t + h) is in error because x(t) is already wrong (because of previous truncation errors) and because another local truncation error occurs in the computation of x(t + h) by means of the Taylor series. Additional sources of errors must be considered in a complete theory. One is roundoff error. Although not serious in any one step of the solution procedure, after hundreds or thousands of steps, it may accumulate and contaminate the calculated solution seriously! Remember that an error that is made at a certain step is carried forward into all succeeding steps. Depending on the differential equation and the method that is used to solve it, such errors may be magnified by succeeding steps. Taylor Series Method Using Symbolic Computations Various routine mathematical calculations of both a nonnumerical and a numerical type, including differentiation and integration of even rather complicated expressions, can now be turned over to the computer. Of course, this applies only to a restricted class of func- tions, but this class is broad enough to include all the functions that one encounters in the typical calculus textbook. With the use of such a program for symbolic computations, the Taylor series method of high order can be carried out without difficulty. Using the alge- braic manipulation potentialities in mathematical software such as MATLAB, Maple or Mathematica, we can write code to solve the initial value problem (7). The final result is x (2) ≈ 4.37121 00522 49692 27234 569. Summary 7.1 • We wish to solve the first-order initial-value problem 􏰈x′(t)= f(t,x(t)) x(a) = xa over the interval [a, b] with step size h = (b − a)/n. • The Taylor series method of order m is x(t + h) = x(t) + hx′(t) + 1 h2x′′(t) + 1 h3x′′′(t) 2! 3! + 1h4x(iv)(t)+ 1h5x(v)(t)+···+ 1 hmx(m)(t) 4! 5! m! whereallofthederivativesx′′,x′′′,...,x(m) havebeendeterminedanalytically. Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. aa. x′=t3+7t2−t1/2 c. x′=−x ae. x′′=x 2. Givethesolutionsoftheseinitial-valueproblems: aa. x′=t2+t1/3 x(0)=7 b.x′=2x x(0)=15 c. x′′ =−x x(π)=0 x′(π)=3 3. Solvethefollowingdifferentialequations: a. x′ =1+x2 Hint:1+tan2t =sec2t ′√222 b. x = 1−x Hint:sin t+cos t=1 • Euler’s method is the Taylor series method of order 1 and can be written as x(t + h) = x(t) + h f (t, x(t)) Because only two terms in the Taylor series are used, the truncation error is large, and the results cannot be computed with much accuracy. Consequently, higher-order Taylor series methods are used most often. Of course, they require determining more derivatives, with more chances for mathematical errors. ab. x′=x d. x′′=−x f. x′′+x′−2x=0 Hint: Try x = eat . 7.1 Taylor Series Methods 309 1. Givethesolutionsofthesedifferentialequations: a9. Consider the problem x ′ = x . If the initial condition is x(0) = c, then the solution is x(t) = cet. If a round- off error of ε occurs in reading the value of c into the computer, what effect is there on the solution at the point t =10?Att =20?Dothesameforx′ =−x. a 10. If the Taylor series method is used on the initial-value problemx′ =t2 +x3,x(0)=0,andifweintendtouse the derivatives of x up to and including x(iv), what are the five main equations that must be programmed? 11. In solving the following differential equations by the Taylor series method of order n, what are the main equa- tions in the algorithm? aa.x′=x+ex, n=4 b.x′=x2−cosx, n=5 ac. x′=t−1sint a 12. Calculate an approximate value for x (0.1) using one step Hint: See Computer Exercise 5.1.2. ad. x′+tx=t2 of the Taylor series method of order 3 on the ordinary differential equation Hint: Multiply by f (t) = exp(t2/2) so left-hand side 􏰂x′′ =x2et +x′ becomes (x f )′. x(0) = 1, x′(0) = 2 4. Solve Exercise 7.1.3b by substituting a power series a􏰃 x(t) = ∞n=0 antn and then determining appropriate 13. Suppose that a differential equation is solved numerically values of the coefficients. on an interval [a, b] and that the local truncation error is 5. Determinex′′whenx′=xt2+x3+ext. a6. Find a polynomial p with the property p − p′ = t3 + t2−2t. a 7. The general first-order linear differential equation is x′+px+q=0,wherepandqarefunctionsoft.Show that the solution is x = −y−1(z + c), where y and z are functions obtained as follows: Let u be an antiderivative of p. Put y = eu, and let z be an antiderivative of yq. 8. Here is an initial-value problem that has two solutions: ch p . Show that if all truncation errors have the same sign (the worst possible case), then the total truncation error is (b − a)chp−1, where h = (b − a)/n. 14. IfweplantousetheTaylorseriesmethodwithtermsup to h20, how should the computation 20 x(n)(t)hn/n! n=0 be carried out? Assume that x(t), x(1)(t), x(2)(t), . . . , and x (20) (t ) are available. Hint: Only a few statements suffice. 15. ExplainhowtousetheODEmethodthatisbasedonthe trapezoid rule: seriesmethodisapplied,whathappens? x(t+h)=x(t)+2[f(t,x(t))+ f(t+h,􏰛x(t+h))] Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. x′ = x1/3, x(0) = 0. Verify that the two solutions are x1(t) = 0 and x2(t) = 􏰀2t􏰁3/2 for t≧0. If the Taylor 􏰛x(t +h) = x(t)+hf(t,x(t)) 3h 􏰃 Exercises 7.1 310 Chapter 7 Initial Values Problems 16. a1. 2. (Continuation)UsetheimprovedEuler’smethodtosolve the following differential equation over the interval [0, 1] with step size h = 0.1: 􏰂 x′ = −x + t + 1 2 x(0) = 1 Write and test a program for applying the Taylor series me􏰈thod to the initial-value problem x′ = x + x2 Consider the initial-value problem 􏰂′2 x(0) = 1 In the improved Euler’s method, replace 􏰛x(t + h) with x(t + h) and try to solve with one step of size h = 0.1. Explain what happens. Find the closed-form solution by substituting x = (a + bt)c and determining a, b, c. 6. Solvetheinitial-valueproblemx′=(x+t)2withx(0)= −1 on the interval [0, 1] using the Taylor series method with derivatives up to and including the fourth. Compare This is called the improved Euler’s method or Heun’s 17. method. Here, 􏰛x(t + h) is computed by using Euler’s method. x =−100x e this to Taylor series methods of orders 1, 2, and 3. x (1) = = 0.20466 34172 89155 26943 16−e a7. Generate the solution in the interval [1, 2.77]. Use deriva- tives up to x(v) in the Taylor series. Use h = 1/100. Print out for comparison the values of the exact solution x (t ) = et /(16 − et ). Verify that it is the exact solution. Write a program to solve each problem on the indicated intervals. Use the Taylor series method with h = 1/100, and􏰂include terms to h3. Account for any difficulties. a. x′ =t+x2 on[0,0.9] 8. x(0) = 1 􏰂􏰂x′=x−t on[1,1.75] x(1) = 1 a9. x′ =tx+t2x2 on[2,5] x (2) = −0.63966 25333 Write a program to solve on the interval [0, 1] the initial- value problem 􏰂x′ =tx x(0) = 1 using the Taylor series method of order 20; that is, in- clude terms in the Taylor series up to and including h 20 . Observe that a simple recursive formula can be used to obtain x(n) for n = 1,2,...,20. Write a program to solve the initial-value problem x′ = sin x + cos t , using the Taylor series method. Continue the solution from t = 2 to t = 5, starting with x(2) = 0.32. Include terms up to and including h 3 . Write a program to solve the initial-value problem x′ = etx with x(2) = 1 on the interval 0≦t≦2 using the Taylor series method. Include terms up to h 4 . ab. a c. a3. Solve the differential equation x′ = x with initial value x(0) = 1 by the Taylor series method on the inter- val [0, 10]. Compare the result with the exact solution x (t ) = et . Use derivatives up to and including the tenth. 11. Use step size h = 1/100. 4. Solve for x (1): aa. x′=1+x2, x(0)=0 b. x′=(1+t)−1x, x(0)=1 Use the Taylor series method of order 5 with h = 1/100, and compare with the exact solutions, which are tan t and 1 + t, respectively. a5. Solve the initial-value problem x′ = t + x + x2 on the interval [0,1] with initial condition x(1) = 1. Use the Taylor series method of order 5. Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole deemed that any suppressed content does not materially affect the overall learning experience. a10. Write a program to solve x′ = tx + t4 on the interval 0 ≦ t ≦ 5 with x(5) = 3. Use the Taylor series method with terms to h4. Write a program to solve the initial-value problem of the example in this section over the interval [1, 3]. Explain. 12. Compute a table, at 101 equally spaced points in the in- terval [0, 2], of the Dawson integral 􏰀􏰁􏰒x􏰀􏰁 f(x)=exp −x2 exp t2 dt 0 by numerically solving, with the Taylor series method of suitable order, an initial-value problem of which f is the solution. Make the table accurate to eight decimal places, and print only eight decimal places. Hint: Find the relationship between f ′(x) and x f (x). The Fundamental Theorem of Calculus is useful. or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. Computer Exercises 7.1 Check values: f (1) = 0.53807 95069 and f (2) = 0.30134 03889. 13. Solve the initial-value problem x′ = t3 + ex with x(3) = 7.4 on the interval 0≦t≦3 by means of the fourth-order Taylor series method. 14. Use a symbolic manipulation package such as Maple to solve the differential equations of Example 1 by the fourth-order Taylor series method to high accuracy, carrying 24 decimal digits. 15. ProgramthepseudocodesEulerandTaylorandcompare the numerical results to those given in the text. 7.2 Runge-Kutta Methods Introduction 16. (Continuation) Repeat by calling directly an ordinary dif- ferential equation solver routine within a mathematical software system such as MATLAB, Maple, or Mathe- matica. 17. Use mathematical software such as MATLAB, Maple, or Mathematica to find analytical or numerical solutions for these ordinary differential equations: a. ODE (2) b. ODE (3) c. ODE (4) 18. Writecomputerprogramstoreproducethesefigures: a. Figure 7.1 b. Figure 7.2 c. Figure 7.3 Solving ODEs Without Differentiation weneedtoobtainx′′,x′′′,...bydifferentiatingthefunction f.Thisrequirementcanbea serious obstacle to using the method. The user of this method must do some preliminary analytical work before writing a computer program. Ideally, a method for solving Equa- tion (1) should involve nothing more than writing a code to evaluate f . The Runge-Kutta methods accomplish this. For purposes of exposition, the Runge-Kutta method of order 2 is presented, although its low precision usually precludes its use in actual scientific calculations. Later, the Runge- Kutta method of order 4 is given without a derivation. It is in common use. The order-2 Runge-Kutta procedure does find application in real-time calculations on small computers. For example, it is used in some aircraft by the on-board mini-computer. At the heart of any method for solving an initial-value problem is a procedure for advancing the solution function one step at a time; that is, a formula must be given for x(t + h) in terms of known quantities. As examples of known quantities, we can cite x(t), x(t − h), x(t − 2h), . . . if the solution process has gone through a number of steps. At the beginning, only x(a) is known. Of course, we assume that f (t, x) can be computed for any point (t, x). Taylor Series for f (x, y) Before explaining the Runge-Kutta method of order 2, let us present the Taylor series in TaylorSeriesinTwo Variables two variables. The infinite series is f (x + h, y + k) = 􏰄∞ 1􏰋 ∂ h i=0 i! ∂x + k ∂􏰌i ∂y f (x, y) (2) 7.2 Runge-Kutta Methods 311 The methods named after Carl Runge and Wilhelm Kutta are designed to imitate the Taylor series method without requiring analytic differentiation of the original differential equation. Recall that in using the Taylor series method on the initial-value problem 􏰈x′ = f(t,x) x(a) = xa (1) Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 312 Chapter 7 Initial Values Problems This series is analogous to the Taylor series in one variable given by Equation (11) in Section 1.2. The mysterious-looking terms in Equation (2) are interpreted as follows: 􏰋∂ ∂􏰌0 h +k ∂x ∂y f(x,y)= f ∂f ∂f 􏰋∂ ∂􏰌1 h + k f (x, y) = h 􏰋 ∂ ∂􏰌2 2∂2f ∂2f 2∂2f . where f and all partial derivatives are evaluated at (x, y). As in the one-variable case, if the Taylor series is truncated, an error term or remainder term is needed to restore the equality. Here is the appropriate equation: f (x + h, y + k) = The point (x, y) lies on the line segment that joins (x, y) to (x + h, y + k) in the plane. In applying Taylor series, we use subscripts to denote partial derivatives. So, for instance, we define ∂f ∂f ∂2 f ∂2 f fx = ∂x ft = ∂t , fxx = ∂x2 , fxt = ∂t ∂x (4) We are dealing with functions for which the order of these subscripts is immaterial; for example, fxt = ftx. Thus, we have f(x+h,y+k)= f +(hfx +kfy) + 1 􏰀h2 fxx +2hkfxy +k2 fyy􏰁 2! + 1 􏰀h3 fxxx +3h2kfxxy +3hk2 fxyy +k3 fyyy􏰁 3! +··· As special cases, we notice that f(x+h,y)= f +hfx + 2! fxx + 3! fxxx +··· + k ∂x ∂y ∂x ∂y h∂x+k∂y f(x,y)=h ∂x2 +2hk∂x∂y+k ∂y2 TS with Error Term n−1 􏰄1∂∂i 1∂∂n f (x, y) (3) 􏰋􏰌􏰋􏰌 h + k f (x, y) + h + k i=0i!∂x ∂y n!∂x ∂y TS with First Few Terms Special Cases h2 h3 k2 k3 f(x,y+k)= f +kfy + 2! fyy + 3! fyyy +··· Runge-Kutta Method of Order 2 In the Runge-Kutta method of order 2, a formula is adopted that has two function evaluations of the special form 􏰈K1 =hf(t,x) K2 =hf(t+αh,x+βK1) Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. Deriving Runge-Kutta Methods of Order 2 7.2 Runge-Kutta Methods 313 and a linear combination of these is added to the value of x at t to obtain the value at t + h: x(t+h)=x(t)+w1K1 +w2K2 or, equivalently, x(t + h) = x(t) + w1h f (t, x) + w2h f (t + αh, x + βh f (t, x)) (5) The objective is to determine constants w1, w2, α, and β so that Equation (5) is as accurate as possible. Explicitly, we want to reproduce as many terms as possible in the Taylor series x(t+h)=x(t)+hx′(t)+ 1h2x′′(t)+ 1h3x′′′(t)+ 1h4x(iv)(t)+ 1h5x(v)(t)+··· (6) 2! 3! 4! 5! Now compare Equation (5) with Equation (6). One way to force them to agree up through the term in h is to set w1 = 1 and w2 = 0 because x′ = f. However, this simply reproduces Euler’s method (described in the preceding section), and its order of precision is only 1. Agreement up through the h2 term is possible by a more adroit choice of parameters. To see how, apply the two-variable form of the Taylor series to the final term in Equation (5). We use n = 2 in the two-variable Taylor series given by Formula (3), with t, αh, x, and βhf playing the role of x, h, y, and k, respectively: 1􏰋∂ ∂􏰌2 f(t+αh,x+βhf)= f +αhft +βhffx +2 αh∂t +βhf∂x f(x,y) Using the above equation results in a new form for Equation (5). We have x(t+h)=x(t)+(w1 +w2)hf +αw2h2 ft +βw2h2 ffx +O(h3) (7) Equation (6) is also given a new form by using differential Equation (1). Since x′ = f , we have ′′ dx′ df(t,x) 􏰋∂f 􏰌􏰋dt􏰌 􏰋∂f 􏰌􏰋dx􏰌 x=dt= dt =∂t dt+∂x dt=ft+fxf So Equation (6) implies that x(t+h)=x+hf +1h2 ft +1h2 ffx +O(h3) (8) 22 Agreement between Equations (7) and (8) is achieved by stipulating that w1 +w2 =1, αw2 = 1, βw2 = 1 (9) 22 A convenient solution to Equation (9) is α=1, β=1, w1 =1, w2 =1 22 The resulting second-order Runge-Kutta method is then, from Equation (5), x(t + h) = x(t) + h f (t, x) + h f (t + h, x + h f (t, x)) Runge-Kutta Method of Order 2 22 x(t + h) = x(t) + 1(K1 + K2) (10) 2 or, equivalently, Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 314 Chapter 7 Initial Values Problems where 􏰈K1 =hf(t,x) K2 =hf(t+h,x+K1) Error Term Formula (10) shows that the solution function at t + h is computed at the expense of two evaluations of the function f . Notice that other solutions for the nonlinear System (9) are possible. For example, α can be arbitrary, and then β = α, w1 = 1 − 1 , w2 = 1 2α 2α One can show (see Exercise 7.2.10) that the error term for Runge-Kutta methods of order 2 is h3􏰋2 􏰌􏰋∂ ∂ 􏰌2 h3 􏰋∂ ∂ 􏰌 4 3−α ∂t+f∂x f+6fx ∂t+f∂x f (11) Notice that the method with α = 2 is especially interesting. However, none of the second- 3 order Runge-Kutta methods is widely used on large computers because the error is only O(h3). Runge-Kutta Method of Order 4 One algorithm in common use for the initial-value Problem (1) is the classical fourth-order Runge-Kutta method. Its formulas are as follows: x(t + h) = x(t) + 1(K1 + 2K2 + 2K3 + K4) (12) 6 where Runge-Kutta Method of Order 4  1  􏰀 􏰁 K2=hf t+1h,x+1K1 K =hf(t,x) 􏰀2 2􏰁 K3=hf t+1h,x+1K2 22 K4 =hf(t+h,x+K3) The derivation of the Runge-Kutta formulas of order 4 is tedious. Very few textbooks give the details. Two exceptions are the books of Henrici [1962] and Ralston [1965]. There exist higher-order Runge-Kutta formulas, and they are still more laborious to derive. However, symbolic manipulation software packages such as in MATLAB, Maple, or Mathematica can be used to develop the formulas. As shown, the solution at x(t + h) is obtained at the expense of evaluating the function f four times. The final formula agrees with the Taylor expansion up to and including the term in h4. The error therefore contains h5, but no lower powers of h. Without knowing the coefficient of h5 in the error, we cannot be precise about the local truncation error. In treatises devoted to this subject, these matters are explored further. See, for example, Butcher [1987] or Gear [1971]. Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. RK4 Pseudocode Pseudocode Here is a pseudocode to implement the classical Runge-Kutta method of order 4: To illustrate the use of the preceding pseudocode, consider the initial-value problem 􏰈x′ =2+(x−t−1)2 x(1) = 2 (13) whose exact solution is x(t) = 1 + t + tan(t − 1). A pseudocode to solve this problem on the interval [1, 1.5625] by the Runge-Kutta procedure follows. The step size needed is calculated by dividing the length of the interval by the number of steps, say, n = 72. 7.2 Runge-Kutta Methods 315 procedure RK4( f, t, x, h, n) integer j,n; real K1,K2,K3,K4,h,t,ta,x external function f output 0, t, x ta ← t for j = 1 to n K1 ←hf(t,x) K2 ←hf(t+1h,x+1K1) K3 ←hf(t+1h,x+1K2) 22 K4 ←hf(t+h,x+K3) x ← x + 1 (K1 + 2K2 + 2K3 + K4) 6 t ← ta + jh output j,t,x end for end procedure RK4 22 program Test RK4 real h, t; external function f integer n ← 72 real a ← 1, b ← 1.5625, x ← 2 h ← (b − a)/n t←a call RK4(f,t,x,h,n) end program Test RK4 real function f (t, x) real t, x f ←2+(x−t−1)2 end function f Test RK4 Pseudocode We include an external-function statement both in the main program and in procedure RK4 because the procedure f is passed in the argument list of RK4. The final value of the computed numerical solution is x (1.5625) = 3.19293 7699. General-purpose routines incorporating the Runge-Kutta algorithm usually include additional programming to monitor the truncation error and make necessary adjustments in the step size as the solution progresses. In general terms, the step size can be large when the Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 316 Chapter 7 Initial Values Problems solution is slowly varying, but should be small when it is rapidly varying. Such a program is presented in Section 7.3. Summary 7.2 • The second-order Runge-Kutta method is 1 x(t + h) = x(t) + 2(K1 + K2) where 􏰈K1 =hf(t,x) K2 =hf(t+h,x+K1) This method requires two evaluations of the function f per step. It is equivalent to a Taylor series method of order 2. • One of the most popular single-step methods for solving ODEs is the fourth-order Runge-Kutta method x(t + h) = x(t) + 1(K1 + 2K2 + 2K3 + K4) 6 where K =hf(t,x)  􏰀 􏰁  1 K3 =hf t+1h,x+1K2 K2 =hf t+1h,x+1K1 1. 2. Derive the equations needed to apply the fourth-order Taylor series method to the differential equation x′ = t x 2 + x − 2t . Compare them in complexity with the equations required for the fourth-order Runge-Kutta method. Put these differential equations into a form suitable for numerical solution by the Runge-Kutta method. a. x+2xx′−x′=0 b. logx′=t2−x2 ac. (x′)2(1−t2)=x at t = −0.2, correct to two decimal places, using one step of the Taylor series method of order 2 and one step of the Runge-Kutta method of order 2. 4. Considertheordinarydifferentialequation 􏰂x′ =(tx)3 −(x/t)2 x(1) = 1 Take one step of the Taylor series method of order 2 with h = 0.1 and then use the Runge-Kutta method of order 2 to recompute x(1.1). Compare answers. 5. Insolvingthefollowingdifferentialequationsbyusinga Runge-Kutta procedure, it is necessary to write code for a function f (t , x ). Do so for each of the following: aa. x′=t2+tx′−2xx′ b. x′=et+x′cosx+t2 or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. a3. Solvethedifferentialequation  dx  d t = − t x 2 x(0) = 2 Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole deemed that any suppressed content does not materially affect the overall learning experience. 􏰀2 2􏰁 22 K4 =hf(t+h,x+K3) It needs four evaluations of the function f per step. Since it is equivalent to a Taylor series method of order 4, it has truncation error of order O(h5). The small number of function evaluations and high-order truncation error account for its popularity. Exercises 7.2 6. Consider the ordinary differential equation x′ = t3x2 − 2x3/t2 with x(1) = 1. Determine the equations that would be used in applying the Taylor series method of order 3 and the Runge-Kutta method of order 4. 7. Considerthethird-orderRunge-Kuttamethod: x(t + h) = x(t) + 1(2K1 + 3K2 + 4K3) 13. An important theorem of calculus states that the equa- tion ftx = fxt is true, provided that at least one of these two partial derivatives exists and is continuous. Test this equation on some functions, such as f (t , x ) = xt2 + x2t + x3t4, log(x − t−1), and ex sinh(t + x) + cos(2x − 3t). 14. a. Ifx′ = f(t,x),then x′′=Df, x′′′=D2f+fxDff where 9  K 1 = h f ( t , x ) where  K2 = h f 􏰀t + 1 h, x + 1 K1􏰁 7.2 Runge-Kutta Methods 317 􏰀22􏰁 222 K3=hf t+3h,x+3K2 ∂∂2∂∂2∂ 44 D=∂t+f∂x, D =∂t2+2f∂x∂t+f ∂x2 a. Show that it agrees with the Taylor series method of the same order for the differential equation x ′ = x + t . b. Prove that this third-order Runge-Kutta method re- produces the Taylor series of the solution up to and including terms in h3 for any differential equation. a8. Describehowthefourth-orderRunge-Kuttamethodcan be used to produce a table o􏰒f values for the function x f (x) = e−t2 dt 0 at 100 equally spaced points in the unit interval. Hint: Find an appropriate initial-value problem whose solution is f . 9. Showthatthefourth-orderRunge-Kuttaformulareduces to a simple form when applied to an ordinary differential equation of the form x′ = f(t) a10. Establishtheerrorterm(11)forRunge-Kuttamethodsof order 2. a11. Onacertaincomputer,itwasfoundthatwhenthefourth- order Runge-Kutta method was used over an interval [a, b] with h = (b − a)/n, the total error due to round- off was about 36n2−50 and the total truncation error was 9nh5, where n is the number of steps and h is the step size. What is an optimum value of h? Hint: Minimize the total error: roundoff error plus trun- cation error. a 15. 16. a 17. a18. a 19. a 20. a 21. Verify these equations. ab. Determine x(iv) in a similar form. Derive the two-variable form of the Taylor series from the one-variable form by considering the function of one variable φ(t) = f (x + th, y + tk) and expanding it by Taylor’s Theorem. The Taylor series expansion about point (a, b) in terms of two variables x and y is given by 􏰄∞􏰋 􏰌i 1∂∂ i! (x−a)∂x +(y−b)∂y Show that Formula (2) can be obtained from this form by a change of variables. (Continuation) Using the form given in the preceding problem, determine the first four nonzero terms in the Taylor series for f (x, y) = sin x + cos y about the point (0, 0). Compare the result to the known series for sin x and cos y. Make a conjecture about the Taylor series for functions that have the special form f (x, y) = g(x) + h(y). Forthefunction f(x,y)=y2−3lnx,writethefirstsix termsintheTaylorseriesof f(1+h,0+k). Using the truncated Taylor series about (1,1), give a three-term approximation to e(1−xy). Hint: Use Exercise 7.2.16. The function f (x, y) = xey can be approximated by theTaylorseriesintwovariablesby f(x+h,y+k)≈ (Ax + B)ey. Determine A and B when terms through the second partial derivatives are used in the series. For f (x, y) = (y − x)−1, the Taylor series can be written as f (x + h, y + k) = A f + B f 2 + C f 3 + · · · f(x,y)= f(a,b) a12. Howwouldyousolvetheinitial-valueproblem 􏰂′ on the interval [0, 1] if ten decimal places of accuracy, 10−10, are required? Assume that you use a computer with adequate precision, and assume that the fourth-order Runge-Kutta method involves truncation error of magni- tude 100h5. x =sinx+sint x(0) = 0 Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. i=0 318 Chapter 7 Initial Values Problems where f = f (x, y). Determine the coefficients A, B, terms. Use this result to obtain an approximate value for f (0.001, 0.998). Show that the improved Euler’s method is a Runge-Kutta method of order 2. x(0) = 0. Integrate it with the fourth-order Runge- Kutta method on the interval [0,3], using step sizes h = 0.015, 0.020, 0.025, 0.030. Observe the numerical instability! Consider the differential equation  􏰂x+t, −1≦t≦0 0 ≦ t ≦ 1 Using the Runge-Kutta procedure RK4 with step size h = 0.1, solve this problem over the interval [−1, 1]. Now solve by using h = 0.09. Which numerical solution is more accurate and why? Hint: The true solution is given by x = e(t+1) − (t + 1) ift≦0andx=e(t+1)−2et +(t+1)ift≧0. Solvet−x′+2xt =0withx(0)=0ontheinterval [0, 10] using the Runge-Kutta formulas with h = 0.1. Compare with the true solution: 1 (et 2 − 1). Draw a graph 2 or have one created by an automatic plotter. Then graph the logarithm of the solution. ′ Write a program to solve x = sin(xt) + arctan t on 1 ≦ t ≦ 7 with x (2) = 4 using the Runge-Kutta proce- dure RK4. The general form of Runge-Kutta methods of order 2 is given by Equation (5). Write and test procedure RK2( f, t, x, h, α, n) for carrying out n steps with step size h and initial conditions t and x for several given α values. Wewanttosolve 􏰂 x′ = et x2 + e3 x(2) = 4 at x(5) with step size 0.5. Solve it in the following two ways. a. Code the function f (t , x ) that is needed and use pro- cedure RK4. b. Write a short program that uses the Taylor series method including terms up to h4. a 22. 1. a2. 3. a a5. a6. 7. andC. Consider the function ex2+y. Determine its Taylor series 23. about the point (0, 1) through second-partial-derivative Run the sample pseudocode given in the text for dif- ferential Equation (13) to illustrate the Runge-Kutta method. Solve the initial-value problem x′ = x/t + t sec(x/t) with x(0) = 0 by the fourth-order Runge-Kutta method. a8. Continue the solution to t = 1 using step size h = 2−7. Compare the numerical solution with the exact solution, which is x(t) = t arcsin t. Define f (0, 0) = 0, where f (t, x) = x/t + t sec(x/t). Select one of the following initial-value problems, and compare the numerical solutions obtained with fourth- order Runge-Kutta formulas and fourth-order Taylor se- ries.Usedifferentvaluesofh = 2−n,forn = 2,3,...,7, to compute the solution on the interval [1, 2]. a. x′ =1+x/t, x(1)=1 ab. x′=1/x2−xt, x(1)=1 a9. ac. x′=1/t2−x/t−x2, x(1)=−1 x′ = x(−1) = 1 2 10. x − t, 4. Select a Runge-Kutta routine from a program library, and test it on the initial-value problem x′ = (2 − t)x with x(2) = 1. Compare with the exact solution, x = exp􏰗−􏰀1􏰁(t−2)2􏰘. (Ill-Conditioned ODE) Solve the ordinary differential equation x′ = 10x + 11t − 5t2 − 1 with initial value x(0) = 0. Continue the solution from t = 0 to t = 3, using the fourth-order Runge-Kutta method with h = 2−8. Print the numerical solution and the exact solution (t2/2−t) at every tenth step, and draw a graph of the two solutions. Verify that the solution of the same differential equation with initial value x(0) = ε is εe10t + t2/2 − t and thus account for the discrepancy between the numer- ical and exact solutions of the original problem. √ Solve the initial-value problem x ′ = x x 2 − 1 with x(0) = 1 by the Runge-Kutta method on the interval 0 ≦ t ≦ 1.6, and account for any difficulties. Then, using negative h, solve the same differential equation on the same interval with initial value x(1.6) = 1.0. The following pathological example has been given by Dahlquist and Bjo ̈rck [1974]. Consider the differen- tial equation x ′ = 100(sin t − x ) with initial value 11. 12. Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole deemed that any suppressed content does not materially affect the overall learning experience. or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. Computer Exercises 7.2 13. Plot the solution for differential equation (13). 14. Selectadifferentialequationwithaknownsolutionand compare the classical fourth-order Runge-Kutta method with one or both of the following ones. Print the errors at each step. Is the ratio of the two errors a constant at each step? What are the advantages and disadvantages of each method? a. A fourth-order Runge-Kutta method similar to the classical one is given by x(t + h) = x(t) + 1(K1 + 4K3 + K4) 6 15. Note: There are any number of Runge-Kutta methods of any order. The higher the order, the more complicated are the formulas. Since the one given by Equation (12) has error O(h5) and is rather simple, it is the most popular fourth-order Runge-Kutta method. The error term for the method of part b of this problem is also O(h5), and it is optimum in a certain sense. (See Ralston [1965] for details.) A fifth-order Runge-Kutta method is given by x(t+h)=x(t)+ 1 K1+ 5 K4+27K5+125K6 24 48 56 336 where   1 K2 =hf􏰀t+2h, x+2K1􏰁 7.2 Runge-Kutta Methods 319 where    K 1 = h f ( t , x )  K = h f 􏰀 t + 1 h , x + 1 K 􏰁 K1=hf(t,x) 2 􏰀2 21 􏰁 K3=hf t+1h,x+1K1+1K2 K=hft+1h,x+1K 2221 􏰋􏰌  2 4 4 K4 = h f (t + h, x − K2 + 2K3) 􏰋111􏰌 K3=hf t+2h,x+4K1+4K2 See England [1969] or Shampine, Allen, and Pruess [1997]. b. Another fourth-order Runge-Kutta method is given by x(t+h)=x(t)+w1K1+w2K2+w3K3+w4K4 where K4=hf(t+h,x−K2+2K3) K5=hf􏰋t+2h,x+7K1+10K2+1K4􏰌 3272727 K=hf􏰋t+1h,x+28K−1K+546K  6 5 625 1 5 2 625 3  +54 K4−378K5􏰌 K =hf(t,x) Write and test a procedure that uses this formula. 16. a. UseasymbolmanipulationpackagesuchasMapleor Mathematica to find the general Runge-Kunge method 625 625 5 5 􏰀1􏰀√􏰁 􏰁 oforder2. b. Repeatfororder3. K3=hf t+16 14−3 5 h,x+c31K1+c32K2  K4 = h f (t + h, x + c41 K1 + c42 K2 + c43 K3) Here the appropriate constants are 17. (Delay Ordinary Differential Equation) Investigate procedures for determining the numerical solution of an ordinary differential equation with a constant delay such as x′(t)=−x(t)+x(t−20)+ 1 cos􏰋t 􏰌 20 20 + sin􏰋 t 􏰌−sin􏰋 t −1􏰌 20 20 on the interval 0 ≦ t ≦ 1000, where x(t) = sin 􏰀t/20􏰁 fort≦0.Useastepsizelessthanorequalto20sothat no overlapping occurs. Compare to the exact solution x (t ) = sin(t /20). Write a software for program Test RK4 and routine RK4, and verify the numerical results given in the text. c31 = c41 = c43 = w1 = w3 = 3􏰀 − 963 + 476√5􏰁 1024 −3365 + 2094√5 6040 , , 5􏰀757 − 324√5􏰁 c32 = 1024 −975 − 3046√5 c42 = 2552 32􏰀14595 + 6374√5􏰁 240845 􏰀 √􏰁 263+24√5 125 1−8 5 , w2= 1812 3828 1024􏰀3346 + 1623√5􏰁 , w4 = 2􏰀15 − 2√5􏰁 59 24787 123 18. Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 320 Chapter 7 Initial Values Problems 7.3 Adaptive Runge-Kutta and Multistep Methods An Adaptive Runge-Kutta-Fehlberg Method In realistic situations involving the numerical solution of initial-value problems, there is always a need to estimate the precision attained in the computation. Usually, an error tolerance is prescribed, and the numerical solution must not deviate from the true solution beyond this tolerance. Once a method has been selected, the error tolerance dictates the largest allowable step size. Even if we consider only the local truncation error, determining an appropriate step size may be difficult. Moreover, often a small step size is needed on one portion of the solution curve, whereas a larger one may suffice elsewhere. For the reasons given, various methods have been developed for automatically adjusting the step size in algorithms for the initial-value problem. One simple procedure is now described. Consider the classical fourth-order Runge-Kutta method discussed in Section 7.2. To advance the solution curve from t to t +h, we can take one step of size h using the Runge- Kutta formulas. But we can also take two steps of size h/2 to arrive at t + h. If there were no truncation error, the value of the numerical solution x(t + h) would be the same for both procedures. The difference in the numerical results can be taken as an estimate of the local truncation error. So, in practice, if this difference is within the prescribed tolerance, the current step size h is satisfactory. If this difference exceeds the tolerance, the step size is halved. If the difference is very much less than the tolerance, the step size is doubled. The procedure just outlined is easily programmed but rather wasteful of computing time and is not recommended. A more sophisticated method was developed by Fehlberg [1969]. The Runge-Kutta-Fehlberg method of order 4 is x(t+h)=x(t)+ 25K1+1408K3+2197K4−1K5 (1) 216 2565 4104 5 where Schemes for Automatic Step-Size Adjustment K =hf(t,x)  1  1 1 K =hf t+ h,x+ K 2 4 41 􏰋􏰌 Runge-Kutta-Fehlberg Method of Order 4 􏰋􏰌  K 3 = h f t + 3 h , x + 3 K 1 + 9 K 2 8 32 32  􏰋 􏰌 􏰋􏰌 K4=hf t+12h,x+1932K1−7200K2+7296K3 13 2197 2197 2197 Runge-Kutta Method of Order 5 K5=hf t+h,x+439K1−8K2+3680K3−845K4 216 513 4104 Since this scheme requires one more function evaluation than the classical Runge-Kutta method of order 4, it is of questionable value alone. However, with an additional function evaluation K6 = h f􏰋t + 1 h, x − 8 K1 + 2K2 − 3544 K3 + 1859 K4 − 11 K5􏰌 (2) 2 27 2565 4104 40 we can obtain a fifth-order Runge-Kutta method, namely, x(t+h)=x(t)+ 16K1+ 6656K3+28561K4− 9K5+ 2K6 (3) 135 12825 56430 50 55 Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. RK45 Pseudocode 7.3 Adaptive Runge-Kutta and Multistep Methods 321 The difference between the values of x(t + h) obtained from the fourth- and fifth-order procedures is an estimate of the local truncation error in the fourth-order procedure. So six function evaluations give a fifth-order approximation, together with an error estimate! Pseudocode A pseudocode for the Runge-Kutta-Fehlberg method is given in procedure RK45: procedure RK45( f, t, x, h, ε) real ε, K1, K2, K3, K4, K5, K6, h, t, x, x4 external function f real c20 ← 0.25, c21 ← 0.25 real c30 ← 0.375, c31 ← 0.09375, c32 ← 0.28125 real c40 ← 12./13., c41 ← 1932./2197. real c42 ← −7200./2197., c43 ← 7296./2197. real c51 ← 439./216., c52 ← −8. real c53 ← 3680./513., c54 ← −845./4104. real c60 ← 0.5, c61 ← −8./27., c62 ← 2. real c63 ← −3544./2565., c64 ← 1859./4104. real c65 ← −0.275 real a1 ← 25./216., a2 ← 0., a3 ← 1408./2565. real a4 ← 2197./4104., a5 ← −0.2 real b1 ← 16./135., b2 ← 0., b3 ← 6656./12825. real b4 ← 28561./56430., b5 ← −0.18 real b6 ← 2./55. K1 ←hf(t,x) K2 ←hf(t+c20h,x+c21K1) K3 ←hf(t+c30h,x+c31K1 +c32K2) K4 ←hf(t+c40h,x+c41K1 +c42K2 +c43K3) K5 ←hf(t+h,x+c51K1 +c52K2 +c53K3 +c54K4) K6 ←hf(t+c60h,x+c61K1 +c62K2 +c63K3 +c64K4 +c65K5) x4 ← x +a1K1 +a3K3 +a4K4 +a5K5 x ← x +b1K1 +b3K3 +b4K4 +b5K5 +b6K6 t←t+h ε ← |x − x4| end procedure RK45 Of course, the programmer may wish to consider various optimization techniques such as assigning numerical values to the coefficients with decimal expansions corresponding to the precision of the computer being used so that the fractions do not need to be recomputed at each call to the procedure. We can use the RK45 procedure in a nonadaptive fashion such as in the following test program: program Test RK45 integer k; real t, h, ε; external function f integer n ← 72 real a ← 1.0,b ← 1.5625,x ← 2.0 (Continued) Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 322 Chapter 7 Initial Values Problems h ← (b − a)/n t←a output 0, t, x for k = 1 to n call RK45( f, t, x, h, ε) output k,t,x,ε end for end program Test RK45 real function f (t, x) real t, x f ← 2.0+(x −t −1.0)2 end function f Test RK45 Pseudocode A Simple Adaptive Procedure Here, we print the error estimation at each step. However, we can use it in an adaptive procedure, since the error estimate ε can tell us when to adjust the step size to control the single-step error. We now describe a simple adaptive procedure. In the RK45 procedure, the fourth- and fifth-order approximations for x(t + h), say, x4 and x5, are computed from six function evaluations, and the error estimate ε = |x4 − x5| is known. From user-specified bounds on the allowable error estimate (εmin ≦ ε ≦ εmax), the step size h is doubled or halved as needed to keep ε within these bounds. A range for the allowable step size h is also specified by the user (hmin ≦ |h| ≦ hmax). Clearly, the user must set the bounds (εmin, εmax, hmin, hmax) carefully so that the adaptive procedure does not get caught in a loop, trying repeatedly to halve and double the step size from the same point to meet error bounds that are too restrictive for the given differential equation. Basically, our adaptive process is as follows: Overview of Adaptive Process 1. Givenastepsizehandaninitialvaluex(t),theRK45routinecomputesthevalue x(t + h) and an error estimate ε. 2. Ifεmin≦ε≦εmax,thenthestepsizehisnotchangedandthenextstepistakenby repeating step 1 with initial value x(t + h). 3. Ifε<εmin,thenhisreplacedby2h,providedthat|2h|≦hmax. 4. Ifε>εmax,thenhisreplacedbyh/2,providedthat|h/2|≧hmin.
5. If hmin ≦ |h| ≦ hmax, then the step is repeated by returning to step 1 with x(t) and the new h value.
■ Algorithm
The procedure for this adaptive scheme is RK45 Adaptive. In the parameter list of the pseudocode, f is the function f (t , x ) for the differential equation, t and x contain the initial values, h is the initial step size, tb is the final value for t, itmax is the maximum number of steps to be taken in going from a = ta to b = tb, εmin and εmax are lower and upper bounds on the allowable error estimate ε, hmin and hmax are bounds on the step size h, and iflag is an error flag that returns one of the following values:
iflag
0 1
Meaning
Successful march from ta to tb Maximum number of iterations reached
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

RK45 Adaptive
Pseudocode
7.3 Adaptive Runge-Kutta and Multistep Methods 323 On return, t and x are the exit values, and h is the final step size value considered or used:
procedure RK45 Adaptive( f, t, x, h, tb, itmax, εmax, εmin, hmin, hmax, iflag) integer iflag, itmax, n; external function f
real ε, εmax, εmin, d, h, hmin, hmax, t, tb, x, xsave, tsave
realδ← 1 ×10−5
2 output 0, h, t, x
iflag ← 1 k←0
while k ≦ itmax
k←k+1
if |h| < hmin then h ← sign(h)hmin if |h| > hmax then h ← sign(h)hmax d ← |tb − t|
if d ≦ |h| then
iflag ← 0
if d ≦ δ · max{|tb|, |t|} then exit loop h ← sign(h)d
end if
xsave ← x
tsave ← t
call RK45( f, t, x, h, ε) output n,h,t,x,ε
if iflag = 0 then exit loop ifε<εmin thenh←2h if ε > εmax then
h ← h/2 x ← xsave
t ← tsave
k←k−1 end if
end while
end procedure RK45 Adaptive
In the pseudocode, notice that several conditions must be checked to determine the size of the final step, since floating-point arithmetic is involved and the step size varies.
Repeat the computer example in the previous section using RK45 Adaptive, which allows variable step size, instead of RK4. Compare the accuracy of these two computed solutions. (See Computer Exercise 7.4.22.)
An Industrial Example
A first-order differential equation that arose in the modeling of an industrial chemical process is as follows:
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
􏰈x′ =a+bsint+cx x(0) = 0
(4)

324 Chapter 7
Initial Values Problems
Test RK45 Pseudocode
in which a = 3, b = 5, and c = 0.2 are constants. This equation is amenable to the solution techniques of calculus, in particular the use of an integrating factor. However, the analytic solution is complicated, and a numerical solution may be preferable.
To solve this problem numerically using the adaptive Runge-Kutta formulas, identify (and program) the function f that appears in the general description. In this problem, it is f (t, x) = 3 + 5 sin t + 0.2x. Here is a brief pseudocode for solving the equation on the interval [0,10] with particular values assigned to the parameters in the routine RK45 Adaptive:
program Test RK45 Adaptive
integer iflag; real t , x , h , tb ; external function f
integer itmax ← 1000
real εmax ← 10−5, εmin ← 10−8, hmin ← 10−6, hmax ← 1.0 t←0.0; x←0.0; h←0.01; tb ←10.0
callRK45 Adaptive(f,t,x,h,tb,itmax,εmax,εmin,hmin,hmax,iflag) output itmax, iflag
end program Test RK45 Adaptive
real function f (t, x) real t, x
f ←3+5sin(t)+0.2x end function f
We obtain the approximation x(10) ≈ 135.917. The output from the code is a table of values that can be sent to a plotting routine. The resulting graph helps the user to visualize the solution curve.
Adams-Bashforth-Moulton Formulas
We now introduce a strategy in which numerical quadrature formulas are used to solve a single first-order ordinary differential equation. The model equation is
x′(t)= f(t,x(t))
and we suppose that the values of the unknown function have been computed at several points to the left of t, namely, t,t − h,t − 2h,…,t − (n − 1)h. We want to compute x(t + h). By the theorems of calculus, we can write
􏰒 t+h t
􏰒 t+h t
􏰄n j=1
wheretheabbreviation fj = f(t−(j−1)h,x(t−(j−1)h))hasbeenused.Inthelast line of the above equation, we have brought in a suitable numerical integration rule. The simplest case of such a formula over interval [0, 1] uses values of the integrand at points
x(t + h) = x(t) +
= x(t)+
≈x(t)+
x′(s) ds f(s,x(s))ds
cj fj
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

7.3 Adaptive Runge-Kutta and Multistep Methods 325
0, −1, −2, . . . , 1 − n in the case of an Adams-Bashforth formula. Once we have such a basic rule, a change of variable produces the rule for any other interval with any other uniform spacing.
Let’s find a rule of the form
􏰒1
F(r)dr ≈c1F(0)+c2F(−1)+···+cnF(1−n)
0
Therearencoefficientscj atourdisposal.Weknowfrominterpolationtheorythattherule
can be made exact for all polynomials of degree n − 1. It suffices that we insist on integrating each function 1,r,r2,…,rn−1 exactly. Hence, we write down the appropriate equation:
􏰒1 􏰄n
ri−1 dt = cj(1− j)i−1 (1≦i≦n)
0
This is a linear system Au = b of n equations in n unknowns. The elements of the matrix A are Aij = (1 − j)i−1, and the right-hand side is bi = 1/i. 􏰀
When th􏰁is program is run, the output is the vector of coefficients 55/24, −59/24, 37/24, −3/8 . Of course, higher-order formulas are obtained by changing the value of n in the code. To get the Adams-Moulton formulas, we start with a quadrature rule of the form
j=1
􏰒1 􏰄n
G(r)dr ≈ CjG(2− j)
0
A program similar to the one above yields the coefficients 􏰀9/24, 19/24, −5/24, 1/24􏰁. The distinction between the two quadrature rules is that one involves the value of the integrand at 1 and the other does not. 􏰦 t+h
How do we arrive at formulas for t g(s)ds from the work already done? Use the change of variable from s to σ gi􏰦ven by s = hσ − t. In these considerations, think of t as a constant. The new integral is h 01 g(hσ + t)dσ, which can be treated with either of the two formulas already designed for the interval [0, 1]. For example, we have
Adams-Bashforth Quadrature Rules
􏰒 t+h t
􏰒 t+h
t
h
F(r)dr ≈ 24 [55F(t)−59F(t −h)+37F(t −2h)−9F(t −3h)]
h
G(r)dr ≈ 24 [9G(t +h)+19G(t)−5G(t −h)+G(t −2h)]
(5)
The method of undetermined coefficients used here to obtain the quadrature rules does not, by itself, provide the error terms that we would like to have. An assessment of the error can be made from interpolation theory, because the methods considered here come from integrating an interpolating polynomial. Details can be found in more advanced books. You can experiment with some of the Adams-Bashforth-Moulton formulas in Computer Exercises 7.3.2 and 7.3.4. These methods are taken up again in Section 7.5.
Stability Analysis
Let us now resume the discussion of errors that inevitably occur in the numerical solution
of an initial-value problem
􏰈x′ = f(t,x) x(a) = s
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
j=1
(6)

326
Chapter 7
Initial Values Problems
x
FIGURE 7.4
Solution curves to x′=xwithx(a)=s
Example of Divergent Solution Curves
a5t0 t1 t2 t3
t4 t5
x 5 se(t2a)
s5
s4
s3 Global error s2
s1
t
Example of Convergent Solution Curves
For an example in which this difficulty does not arise, consider 􏰈x′ =−x
x(a) = s
(7)
The exact solution is a function x(t). It depends on the initial value s, and to show this, we write x(t,s). The differential equation therefore gives rise to a family of solution curves, each corresponding to one value of the parameter s. For example, the differential equation
􏰈x′ =x x(a) = s
gives rise to the family of solution curves x = se(t−a) that differ in their initial values x(a) = s. A few such curves are shown in Figure 7.4. The fact that the curves there diverge from one another as t increases has important numerical significance. Suppose, for instance, that initial value s is read into the computer with some roundoff error. Then even if all subsequent calculations are precise and no truncation errors occur, the com- puted solution is still wrong! An error made at the beginning has the effect of selecting the wrong curve from the family of all solution curves. Since these curves diverge from one another, any minute error made at the beginning is responsible for an eventual com- plete loss of accuracy. This phenomenon is not restricted to errors made in the first step, because each point in the numerical solution can be interpreted as the initial value for succeeding points.
FIGURE 7.5
Solution curves to x′ =−x withx(a)=s
x s5
s4 s3 s2 s1
a5t0 t1
x 5 se2(t2a)
t2 t3 t4 t5
Global error
t
Its solutions are x = se−(t−a). As t increases, these curves come closer together, as in Figure 7.5. Thus, errors made in the numerical solution still result in selecting the wrong curve, but the effect is not as serious because the curves coalesce.
At a given step, the global error of an approximate solution to an ordinary differential equation contains both the local error at that step and the accumulative effect of all the local
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

Conditions for Curves to Diverge or Converge
errors at all previous steps. For divergent solution curves, the local errors at each step are magnified over time, and the global error may be greater than the sum of all the local errors. In Figures 7.4 and 7.5, the steps in the numerical solution are indicated by dots connected by dark lines. Also, the local errors are indicated by small vertical bars and the global error by a vertical bar at the right end of the curves.
For convergent solution curves, the local errors at each step are reduced over time, and the global error may be less than the sum of all the local errors. For the general differential Equation (6), how can the two modes of behavior just discussed be distinguished? It is simple. If fx > δ for some positive δ, the curves diverge. However, if fx < −δ, they converge. To see why, consider two nearby solution curves that correspond to initial values s and s + h. By Taylor series, we have 7.3 Adaptive Runge-Kutta and Multistep Methods 327 whence ∂ 1 2 ∂2 x(t,s+h)=x(t,s)+h∂sx(t,s)+2h ∂s2x(t,s)+··· ∂ ∂s lim |x(t, s + h) − x(t, s)| = ∞ t→∞ lim􏰣􏰣􏰣∂ x(t,s)􏰣􏰣􏰣=∞ t→∞ 􏰣∂s 􏰣 x(t, s + h) − x(t, s) ≈ h Thus, the divergence of the curves means that and can be written as x(t, s) To calculate this partial derivative, start with the differential equation satisfied by x(t,s): ∂ x(t,s)= f(t,x(t,s)) ∂t and differentiate partially with respect to s: ∂∂∂ ∂t∂sx(t,s)= fx(t,x(t,s))∂sx(t,s)+ ft(t,x(t,s))∂s (8) But s and t are independent variables (a change in s produces no change in t), so ∂t/∂s = 0. If s is now fixed and if we put u(t) = (∂/∂s)x(t, s) and q(t) = fx (t, x(t, s)), then Equa- tion (8) becomes u′ = qu (9) This is a linear differential equation with solution u(t) = ceQ(t), where Q is the indefinite integral (antiderivative) of q. The condition limt→∞ |u(t)| = ∞ is met if limt→∞ Q(t) = ∞. This situation, in turn, occurs if q(t) is positive and bounded away from zero because then 􏰒t 􏰒t Q(t) = q(θ) dθ > δ dθ = δ(t − a) → ∞
aa as t → ∞ if fx = q > δ > 0.
To illustrate, consider the differential equation x ′ = t + tan x . The solution curves diverge from one another as t → ∞ because fx (t, x) = sec2 x > 1.
Hence, we obtain
x(t, s) = f (t, x(t, s)) ∂s
∂s ∂t
∂∂ ∂ ∂t
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

328 Chapter 7
Initial Values Problems
Summary 7.3
• The Runge-Kutta-Fehlberg method is
􏰮x(t) = x(t)+ 25 K1 + 1408K3 + 2197K4 − 1K5
216 2565 4104 5
16 6656 28561 9 2
x(t + h) = x(t) + 135 K1 + 12825 K3 + 56430 K4 − 50 K5 + 55 K6 where
   K 1 = h f ( t , x )
K2 =hf􏰋t+1h, x+1K1􏰌
 4 4
K3 = hf􏰋t + 3h, x + 3 K1 + 9 K2􏰌
 8 32 32
K4 = hf􏰋t + 12h, x + 1932 K1 − 7200 K2 + 7296 K3􏰌
 13 2197 2197 2197
 􏰋 439 3680 845 􏰌
K5=hf t+h,x+216K1−8K2+513K3−4104K4
 K6 = h f􏰋t + 1 h, x − 8 K1 + 2K2 − 3544 K3 + 1859 K4 − 11 K5􏰌
2 27 2565 4104 40
The quantity ε = |x(t + h) − 􏰮x| can be used in an adaptive step-size procedure.
• A fourth-order multistep method is the Adams-Bashforth-Moulton method: 􏰮x(t+h)=x(t)+ h 􏰗55f(t,x(t))−59f(t−h,x(t−h)) 􏰘
a1. Solvetheproblem
by using the trapezoid rule, as discussed at the beginning of this chapter. Compare the true solution at t = 1 to the
approximate solution obtained with n steps. Show, for example, that for n = 5, the error is 0.00123.
a2. DeriveanimplicitmultistepformulabasedonSimpson’s rule,involvinguniformlyspacedpointsx(t−h),x(t),and x(t +h), for numerically solving the ordinary differential equation x′ = f .
24
+ 37f(t −2h,x(t −2h))−9f(t −3h,x(t −3h))
x(t+h)=x(t)+ h [9f(t+h,􏰮x(t+h))+19f(t,x)(t)) 24
− 5f(t −h,x(t −h))+ f(t −2h,x(t −2h))]
The value 􏰮x(t + h) is the predicted value, and x(t + h) is the corrected value. The truncation errors for these two formulas are O(h5). Since the value of x(a) is given, the values for x(a + h), x(a + 2h), x(a + 3h), x(a + 4h) are computed by some single-step method such as the fourth-order Runge-Kutta method.
􏰈x′ =−x x(0) = 1
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
Exercises 7.3

3. An alert student noticed that the coefficients in the Adams-Bashforth formula add up to 1. Why is that so?
a4. Derive a formula of the form
x(t +h) = ax(t)+bx(t −h)
+ h[cx′(t + h) + dx′′(t) + ex′′′(t − h)]
that is accurate for polynomials of as high a degree as possible.
Hint: Use polynomials 1, t, t2, and so on.
a5. Determine the coefficients of an implicit, one-step, ordi- nary differential equation method of the form
x(t + h) = ax(t) + bx′(t) + cx′(t + h)
so that it is exact for polynomials of as high a degree as
possible. What is the order of the error term?
6. The differential equation that is used to illustrate the adap- tive Runge-Kutta program can be solved with an integrat- ing factor. Do so.
1. Use mathematical software to solve systems of linear equations whose solutions are
a. Adams-Bahforth coefficients
b. Adams-Moulton coefficients
2. The second-order Adams-Bashforth-Moulton method is given by
􏰮x(t +h) = x(t)+ h[3f(t,x(t))− f(t −h,x(t −h))] 2
aa. x′=sint+ex ac. x′=xt
ae. x′=cost−ex
b. x′=x+te−t d. x′=x3(t2+1)
f. x′=(1−x3)(1+t2)
7.3
Adaptive Runge-Kutta and Multistep Methods 329 7. Establish Equation (9).
a8. The initial-value problem x′ = (1 + t2)x with x(0) = 1 is to be solved on the interval [0, 9]. How sensitive is x (9) to perturbations in the initial value x(0)?
9. For each differential equation, determine regions in which the solution curves tend to diverge from one another as t increases:
10. For the differential equation x′ = t(x3 − 6×2 + 15x), determine whether the solution curves diverge from one another as t → ∞.
a11. Determine whether the solution curves of x′ = (1 + t 2 )−1 x diverge from one another as t → ∞.
4. (Predictor-Corrector Scheme) Using the fourth- order Adams-Bashforth-Moulton method, derive the predictor-corrector scheme given by the following equations:
􏰮x(t+h)=x(t)+ h [55f(t,x(t))−59f(t−h,x(t−h)) 24
+ 37 f (t − 2h, x(t − 2h)) − 9 f (t − 3h, x(t − 3h))]
x(t +h) = x(t)+ h[f(t +h,􏰮x(t +h))+ f(t,x(t))] x(t+h)=x(t)+ h [9f(t+h,􏰮x(t+h))+19f(t,x(t))
2 24 The approximate single-step error is ε ≡ K|x(t + h) −
− 5 f (t − h, x(t − h)) + f(t −2h,x(t −2h))]
Write and test a procedure for the Adams-Bashforth- Moulton method.
Note: This is a multistep process because values of x at t , t −h, t −2h, and t −3h are used to determine the predicted value􏰮x(t+h),which,inturn,isusedwithvaluesofx att, t−h,andt−2htoobtainthecorrectedvaluex(t+h).The error terms for these formulas are (251/720)h5 f (iv)(ξ) and −(19/720)h5 f (iv)(η), respectively. (See Section 7.5 for additional discussion of these methods.)
a5. Solve 
x′ = 3x + 9t −13
t2 x(3) = 6
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
􏰮x(t + h)|, where K = 1 . Using ε to monitor the conver- 6
gence, write and test an adaptive procedure for solving an ODE of your choice using these formulas.
3. (Continuation) Carry out the instructions of the pre- vious computer problem for the third-order Adams- Bashforth-Moulton method:
􏰮x(t +h) = x(t)+ h [23f(t,x(t))−16f(t −h,x(t −h)) 12
+5f(t −2h,x(t −2h))] x(t +h) = x(t)+ 12[5f(t +h,􏰮x(t +h))8f(t,x(t))
− f (t − h, x(t − h))]
where K = 1 in the expression for the approximate
h
10 single-step error.
Computer Exercises 7.3

330 Chapter 7 Initial Values Problems
at x􏰀1􏰁 using procedure RK45 Adaptive to obtain the
the true solution:
3 92 13 x=t−2t+2t
a6. (Continuation) Repeat the previous problem for x 􏰀− 1 􏰁. 2
7. Itisknownthatthefourth-orderRunge-Kuttamethodde- scribed in Equation (12) of Section 7.2 has a local trunca- tion error that is O(h5). Devise and carry out a numerical experiment to test this.
Suggestions: Take just one step in the numerical solu- tion of a nontrivial differential equation whose solution is known beforehand. However, use a variety of values for h, such as 2−n , where 1 ≦ n ≦ 24. Test whether the ratio of errors to h5 remains bounded as h → 0. A multiple- precision calculation may be needed. Print the indicated ratios.
8. Compute the numerical solution of 􏰂x′ =−x
x(0) = 1 using the midpoint method
Note: This is an example of an elliptic integral of the sec- ond kind. It arises in finding an arc length on an ellipse and in many engineering problems.
By solving an appropriate initial-value problem, make a
2
desired solution to nine decimal places. Compare with
a 12.
a 13.
table of the function
f (x) =
􏰒 ∞
1/x tet
dt
on the interval [0, 1]. Determine how well f is approxi- mated by x e−1/x .
Hint: Let t = −lns.
By solving an appropriate initial-value problem, make a
table of the function 􏰒 x
2 −t2
f(x)= √π e 0
dt
on the interval 0 ≦ x ≦ 2. Determine how accurately f (x )
is approximated on this interval by the function
􏰀 2 3􏰁2−x2 g(x)=1− ay+by +cy √ e
π
b = −0.08497 13
x n + 1 = x n − 1 + 2 h x n′ √2
14.
a 15.
16.
17.
c = 0.66276 98, UsetheRunge-Kuttamethodtocompute
withx0 =1andx1 =−h+ 1+h.Arethereany difficulties in using this method for this problem? Carry out an analysis of the stability of this method.
Hint: Consider fixed h and assume xn = λn.
􏰒1􏰓 0
a9. Tabulate and graph the function [1 − ln v(x )]v(x ) on [0, e], where v(x) is the solution of the initial-value prob- lem (dv/dx)[lnv(x)] = 2x,v(0) = 1.
Check value: v(1) = e.
sine integral
􏰒 x sinr Si(x)= r dr
0
The table should cover the interval 0 ≦ x ≦ 1 in steps of
size 0.01.
(Use sin(0)/0 = 1. See Computer Exercise 5.1.9.)
Compute a table of the function
􏰒 x sinht
Shi(x) = t dt 0
by finding an initial-value problem that it satisfies and then solving the initial-value problem. Your table should be accurate to nearly machine precision.
(Use sinh(0)/ 0 = 1.)
Design and carry out a numerical experiment to verify that a slight perturbation in an initial-value problem can cause catastrophic errors in the numerical solution. Note: An initial-value problem is an ordinary differen- tial equation with conditions specified only at the initial point. (Compare this with a boundary value problem as given in Chapter 11.)
where
􏰂
a = 0.30842 84,
y = (1 + 0.47047x )−1 1 + s3 ds
Write and run a program to print an accurate table of the
10. Determinethenumericalvalueof 􏰒5s
in three ways: solving the integral, an ordinary differen- tial equation, and using the exact formula.
2π eds 4s
11. Computeandprintatableofthefunction
f (φ) = 􏰒 φ 􏰔1 − 1 sin2 θ dθ
4 0
by solving an appropriate initial-value problem. Cover the interval [0, 90◦ ] with steps of 1◦ and use the Runge- Kutta method of order 4.
Check values: Use f (30◦ ) =
f (90◦) = 1.46746 221.
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole deemed that any suppressed content does not materially affect the overall learning experience.
0.51788 193 and
or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

18. Run example programs for solving the industrial example in Equation (4), compare the solutions, and produce the plots.
19. Another adaptive Runge-Kutta method was developed by England [1969]. The Runge-Kutta-England method is similar to the Runge-Kutta-Fehlberg method in that it combines a fourth-order Runge-Kutta formula and a com- panion fifth-order one. To reduce the number of function evaluations, the formulas are derived so that some of the same function evaluations are used in each pair of for- mulas. (A fourth-order Runge-Kutta formula requires at least four function evaluations, and a fifth-order one re- quires at least six.) The Runge-Kutta-England method uses the fourth-order Runge-Kutta methods in Computer Exercise 7.2.14a and takes two half-steps as follows:
􏰋
7.4
Methods for First and Higher-Order Systems 331 With these two half-steps, there are enough function eval-
uations so that only one more
K =1hf t+h,x(t)−1(K +96K −92K 92 12123
+ 121K4 − 144K5 − 6K6 + 12K7􏰌
is needed to obtain a fifth-order Runge-Kutta method:
􏰀1􏰁1
xt+2h =x(t)+6(K1+4K3+K4)
where
1􏰀
􏰛x(t+h)=x(t)+ 90 14K1+64K3+32K3
􏰁
− 8K5 +64K7 +15K8 − K9
An adaptive procedure can be developed by using an error
estimationbasedonthetwovaluesx(t+h)and􏰛x(t+h). Program and test such a procedure. (See, for example, Shampine, Allen, and Pruess [1997].)

K1 = 1hf(t,x(t))
 2 􏰋 􏰌
20.
Investigate the numerical solution of the initial-value problem
K2=1hf t+1h,x(t)+1K1
242 􏰂√
K3 = 1hf􏰋t+1h, x(t)+1K1+1K2􏰌 x′=− 1−x2  2 4 4 4 x(0) = 1
   K = 1 h f 􏰋 t + 1 h , x ( t ) − K + 2 K 􏰌 42223
This problem is ill-conditioned, since x (t ) = cos t is a solution and x(t) = 1 is also. For more informa- tion on this and other test problems, see Cash [2003] or www2.imperial.ac.uk/∼jcash/.
21. (Student Research Project) Learn and write a report about algebraic differential equations.
and 􏰀􏰁1􏰀 􏰁 x(t+h)=xt+1h + K5+4K7+K8
26
 1􏰋 􏰀 􏰁􏰌
where
K = hf t+1h,xt+1h
522 2
K =1hf􏰋t+3h,x􏰀t+1h􏰁+1K􏰌
22. Write software to implement the following pseudocodes and verify the numerical results given in the text for IVP (13) in Section 7.2 and for IVP (4) in this section.
6 2 4 2 2 5
1􏰋􏰀􏰁 􏰌 a.TestRK4andRK4
K7= hf t+3h,xt+1h +1K5+1K6
 2 4 2 4 4 b. TestRK45andRK45
K8=1hf􏰋t+h,x􏰀t+1h􏰁−K6+2K7􏰌 c. TestRK45AdaptiveandRK45Adaptive 22
7.4
Methods for First and Higher-Order Systems
So far in our study of the numerical solutions of ordinary differential equations, we have re- stricted our attention to a single differential equation of the first order with an accompanying auxiliary condition. Scientific and technological problems often lead to more complicated situations, however. The next degree of complication occurs with systems of several first- order equations.
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

332 Chapter 7
Initial Values Problems
Sample ODE Coupled System
􏰈x′(t)=x(t)−y(t)+2t−t2 −t3 y′(t)=x(t)+y(t)−4t2 +t3
􏰈 x (0) = 1 y(0) = 0
(1)
Analytic Solution
Sample ODE Uncoupled System
This is an example of an initial-value problem that involves a system of two first-order differential equations. Note that in the example given, it is not possible to solve either of the two differential equations by itself because the first equation governing x′ involves the unknown function y, and the second equation governing y′ involves the unknown function x. In this situation, we say that the two differential equations are coupled.
The reader is invited to verify that the analytic solution is
􏰈x(t) = et cos(t) + t2 = cos(t)[cosh(t) + sinh(t)] + t2
y(t) = et sin(t) − t3 = sin(t)[cosh(t) + sinh(t)] − t3
Let us look at another example that is superficially similar to the first but is actually
Uncoupled and Coupled Systems
The sun and the nine planets form a system of particles moving under the jurisdiction of Newton’s law of gravitation. The position vectors of the planets constitute a system of 27 functions, and the Newtonian laws of motion can be written, then, as a system of 54 first-order ordinary differential equations. In principle, the past and future positions of the planets can be obtained by solving these equations numerically.
Taking an example of more modest scope, we consider two equations with two auxiliary conditions. Let x and y be two functions of t subject to the system
with initial conditions
simpler:
with initial conditions
􏰈x′(t)=x(t)+2t−t2 −t3 y′(t) = y(t) − 4t2 + t3
􏰈 x (0) = 1 y(0) = 0
(2)
These two equations are not coupled and can be solved separately as two unrelated initial- value problems (using, for instance, the methods of Sections 7.1–3). Naturally, our concern here is with systems that are coupled, although methods that solve coupled systems also solve those that are not. The procedures extend to systems, whether coupled or uncoupled.
Taylor Series Method
We illustrate the Taylor series method for System (1) and begin by differentiating the
equations constituting it:
􏰈x′ =x−y+2t−t2 −t3
y′ = x + y − 4t2 + t3
􏰈 x′′ = x′ − y′ + 2 − 2t − 3t2
y′′ = x′ + y′ − 8t + 3t2
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

7.4 Methods for First and Higher-Order Systems 333
􏰈 x′′′ = x′′ − y′′ − 2 − 6t y′′′ = x′′ + y′′ − 8 + 6t
􏰈x(iv) =x′′′ −y′′′ −6 y(iv) = x′′′ + y′′′ + 6
􏰈x(v) = x(iv) − y(iv) y(v) = x(iv) + y(iv)
etc.
A program to proceed from x(t) to x(t + h) and from y(t) to y(t + h) is easily written by using a few terms of the Taylor series:
Taylor Series for x(t+h)and y(t+h)
x(t+h)=x+hx′ + y(t + h) = y + hy′ +
h2 h3 h4 h5 x′′ + x′′′ + x(iv) +
x(v) +··· y(v) + + · · ·
2 6 24 120 h2 h3 h4 h5
y′′ + y′′′ + y(iv) +
2 6 24 120
Taylor System1
Pseudocode
together with equations for the various derivatives. Here, x and y and all their derivatives are functions of t; that is, x = x(t), y = y(t), x′ = x′(t), y′′ = y′′(t), and so on.
A pseudocode program that generates and prints a numerical solution from 0 to 1 in 100 steps is as follows. Terms up to h4 have been used in the Taylor series.
program Taylor System1
integer k; real h, t, x, y, x′, y′, x′′, y′′, x′′′, y′′′, x(iv), y(iv) integernsteps←100; reala←0,b←1
x←1; y←0; t←a
output 0, t , x , y
h ← (b − a)/nsteps
for k = 1 to nsteps
x′ ←x−y+t(2−t(1+t)) y′ ←x+y+t2(−4+t)
x′′ ← x′ − y′ + 2 − t(2 + 3t) y′′ ← x′ + y′ + t(−8 + 3t) x′′′ ← x′′ − y′′ − 2 − 6t
y′′′ ← x′′ + y′′ − 8 + 6t x(iv) ← x′′′ − y′′′ − 6
y(iv) ← x′′′ + y′′′ + 6
x(v) ← x(iv) − y(iv)
y(v) ← x(iv) + y(iv)
x ← x +h􏰗x′ + 1h􏰗x′′ + 1h􏰗x′′′ + 1h􏰗x(iv)􏰘􏰘􏰘􏰘
􏰗 2 􏰗 3 􏰗 4 􏰗 􏰘􏰘􏰘􏰘 y←y+h y′+1h y′′+1h y′′′+1h y(iv)
t←t+h
output k, t, x, y end for
end program Taylor System1
234
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

334 Chapter 7 Initial Values Problems
ODE System1 in Vector Form
Vector Notation
Observe that System (1) can be written in vector notation as
􏰆x′􏰇=􏰆x−y+2t−t2 −t3􏰇 (3)
First-Order ODEs
 ′
xn′ = fn(t,x1,x2,…,xn)  .
Taylor Series Method mth Order
The m-order Taylor series method would be written as
h2 h3 h4 h5
hm
X(v) + ··· + X(m) (5)
with initial conditions
y′ x + y − 4t2 + t3 􏰆 x (0) 􏰇 = 􏰆 1 􏰇
y(0) 0
This is a special case of a more general problem that can be written as
where
􏰈 X′ = F(t, X)
X(a) = S (given)
X=􏰆x􏰇, X′ =􏰆x′􏰇 y y′
(4)
and F is the vector whose two components are given by the right-hand sides in Equation (1). Since F depends on t and X, we write F(t, X).
Systems of ODEs
We can continue this idea in order to handle a system of n first-order differential equations. First, we write them as
Then we let
′
x = f (t,x ,x ,…,x )
1112 n x2 = f2(t,x1,x2,…,xn)
.
x1(a) = s1,x2(a) = s2,…, xn(a) = sn
(all given)
 x1  x1′ f1
X =    x 2    , X ′ =    x 2′    , F =    f 2    , S =    s 2   
.
xn xn′ fn sn
.
and we obtain Equation (4), which is an ordinary differential equation written in vector
 . 
notation.
Taylor Series Method: Vector Notation
X(t + h) = X + hX′ + X′′ + X′′′ + X(iv) +
2 3! 4! 5!
m!
s1 .
where X = X(t), X′ = X′(t), X′′ = X′′(t), X′′′ = X′′′(t), and so on.
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

Taylor System2
Pseudocode
Runge-Kutta Method Fourth-Order in Vector Form
program if the computer language supports vector operations.
Runge-Kutta Method
The Runge-Kutta methods also extend to systems of differential equations. The classical fourth-order Runge-Kutta method for System (4) uses these formulas:
X(t+h)=X+h(K1 +2K2 +2K3 +K4) (6) 6
where
   K 1 = F ( t , X )
    K 2 = F 􏰋 t + 1 h , x + 1 h k 1 􏰌 22
K =F􏰋t+1h,x+1hk􏰌 3 2 22
 K 4 = F ( t + h , X + h K 3 )
Here, X = X(t), and all quantities are vectors with n components except variables t and h.
7.4 Methods for First and Higher-Order Systems 335
A pseudocode for the Taylor series method of order 4 applied to the preceding problem can be easily rewritten by a simple change of variables and the introduction of an array and an inner loop.
program Taylor System2
integer i,k; real h,t; real array (xi)1:n,(dij)1:n×1:4 integer n ← 2, nsteps ← 100
real a ← 0, b ← 1
t←0; (xi)←(1,0)
output 0,t,(xi)
h ← (b − a)/nsteps
for k = 1 to nsteps
d11 ← x1 −x2 +t(2−t(1+t)) d21 ← x1 + x2 + t2(−4 + t)
d12 ← d11 −d21 +2−t(2+3t) d22 ←d11 +d21 +t(−8+3t) d13 ← d12 −d22 −2−6t
d23 ← d12 +d22 −8+6t d14 ←d13 −d23 −6
d24 ←d13 +d23 +6
fori=1ton 􏰗 1 􏰗 1 􏰗 1 􏰘􏰘􏰘 xi ←xi +h di1+2h di2+3h di3+4h[di4]
end for
t←t+h
output k, t, (xi ) end for
end program Taylor System2
Here, a two-dimensional array is used instead of all the different derivative variables; that is, d ↔ x(j). In fact, this and other methods in this chapter become particularly easy to
iji
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

336 Chapter 7
Initial Values Problems
Pseudocode
A procedure for carrying out the Runge-Kutta procedure is given next. It is assumed that the system to be solved is in the form of Equation (4) and that there are n equations in the system. The user furnishes the initial value of t, the initial value of X, the step size h, and the number of steps to be taken, nsteps. Furthermore, procedure XP System(n, t, (xi ), ( fi )) is needed, which evaluates the right-hand side of Equation (4) for a given value of array (xi ) and stores the result in array ( fi ). (The name XP System2 is chosen as an abbreviation of X′ for a system.)
procedure RK4 System1(n, h, t, (xi ), nsteps) integer i, j, n; real h, t; real array (xi )1:n allocate real array (yi )1:n , (Ki, j )1:n×1:4 output 0,t,(xi)
for j = 1 to nsteps
call XP System(n, t, (xi ), (Ki,1)) fori =1ton
yi ←xi +1hKi,1 2
end for
call XP System(n, t + h/2, (yi ), (Ki,2)) fori =1ton
yi ←xi +1hKi,2 2
end for
call XP System(n, t + h/2, (yi ), (Ki,3)) fori =1ton
yi ←xi +hKi,3 end for
call XP System(n, t + h, (y)i , (Ki,4)) fori =1ton
xi ← xi + 1h[Ki,1 +2Ki,2 +2Ki,3 + Ki,4] 6
end for
t←t+h
output j,t,(xi) end for
deallocate array (yi ), (Ki, j ) end procedure RK4 System1
RK4 System1
Pseudocode
Test RK4 System1
Pseudocode
To illustrate the use of this procedure, we again use System (1) for our example. Of course, it must be rewritten in the form of Equation (4). A suitable main program and a procedure for computing the right-hand side of Equation (4) follow:
program Test RK4 System1 integer n ← 2, nsteps ← 100 real a ← 0, b ← 1
real h, t; real array (xi )1:n t←0
(xi) ← (1,0)
h ← (b − a)/nsteps
(Continued)
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

Computer Results
Taylor Series
x (1.0) ≈ 2.46869 40
y(1.0) ≈ 1.28735 46
Runge-Kutta
2.46869 42
1.28735 61
Analytic Solution
2.46869 39399
1.28735 52872
7.4 Methods for First and Higher-Order Systems 337
call RK4 System1(n, h, t, (xi ), nsteps) end program Test RK4 System1
procedure XP System(n, t, (xi ), ( fi )) real array (xi )1:n , ( fi )1:n
integer n
real t
f1 ← x1 − x2 + t(2 − t(1 + t))
f2 ← x1 + x2 − t2(4 − t) end procedure XP System
We use a numerical experiment to compare the results of the Taylor series method and the Runge-Kutta method with the analytic solution of System (1). At the point t = 1.0, the results are as follows:
We can use mathematical software routines found in MATLAB, Maple, or Mathematica to obtain the numerical solution of the system of ordinary differential equations (1). For t over the interval [0,1], we invoke an ODE procedure to march from t = 0 at which x(0) = 1 and y(0) = 0 to t = 1 at which x(1) = 2.468693912 and y(1) = 1.287355325.
To obtain the numerical solution of the ordinary differential equation defined for t over the interval [1, 1.5], invoke an ordinary differential equation solving procedure to march from t = 0 at which x(1) = 2 and y(1) = −2 to t = 1.5 at which x(1.5) ≈ 15.5028 and y(1.5) ≈ 6.15486.
Autonomous ODE
When we wrote the system of differential equations in vector form
X′ = F(t, X)
we assumed that the variable t was explicitly separated from the other variables and treated differently. It is not necessary to do this. Indeed, we can introduce a new variable x0 that is t in disguise and add a new differential equation x0′ = 1. A new initial condition must also be provided, x0(a) = a. In this way, we increase the number of differential equations from n to n + 1 and obtain a system written in the more elegant vector form
􏰈X′ = F(X)
X(a) = S (given)
Consider the system of two equations given by Equation (1). We write it as a system with three variables by letting
Thus, we have
x0 =t, x1 =x, x2 =y
x0′ 1
x1′ =x1 −x2 +2×0 −x02 −x03
x2′ x1 +x2 −4×02 +x03
The auxiliary condition for the vector X is X (0) = [0, 1, 0]T .
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.


338 Chapter 7
Initial Values Problems
IVP Autonomous Vector Form
􏰈X′ = F(X) X(a) = S
x0′
 x1′ 
X ′ =   x 2′   ,
 . . .   . . .   . . .   . . . 
(7)
As a result of the preceding remarks, we sacrifice no generality in considering a system of n + 1 first-order differential equations written as
where
x0
(given)
 f0 
x0′ = f0(x0,x1,x2,…,xn) x1′ = f1(x0,x1,x2,…,xn) x2′ = f2(x0,x1,x2,…,xn)
 .
x′ = f (x ,x ,x ,…,x )
nn012n
x0(a) = s0,x1(a) = s1,x2(a) = s2,…, xn(a) = sn
 x1  X =   x 2   ,
  f1   F =   f 2   ,
 s1  S =   s 2  
(all given)
s0
xn xn′ fn sn
We can write this system in general vector notation as
A system of differential equations without the t variable explicitly present is said to be autonomous. The numerical methods that we discuss do not require that x0 = t or f0 = 1 ors0 =a.
For an autonomous system, the classical fourth-order Runge-Kutta method for System (6) uses these formulas:
X(t+h)=X+h(K1 +2K2 +2K3 +K4) (8) 6
where
    K 1 = F ( X )
Runge-Kutta Method Fourth Order
K2 =F􏰀X+1hK1􏰁 􏰀2􏰁
 K 3 = F X + 1 h K 2 2
K4 =F(X+hK3)
Here, X = X(t), and all quantities are vectors with n+1 components except the variables h. In the previous example, the procedure RK4 System1 would need to be modified by beginningthearrayswith0ratherthan1andomittingthevariablet.(WecallitRK4 System2
and leave it as Computer Exercise 7.4.4.) Then the calling programs would be as follows:
program Test RK4 System2 real h, t; real array (xi )0:n integer n ← 2, nsteps ← 100
(Continued)
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

7.4 Methods for First and Higher-Order Systems 339
real a ← 0, b ← 1
(xi) ← (0,1,0)
h ← (b − a)/nsteps
call RK4 System2(n, h, (xi ), nsteps) end program Test RK4 System2
procedure XP System(n, (xi ), ( fi )) real array (xi )0:n , ( fi )0:n
integer n
f0 ← 1
f1 ← x1 −x2 +x0(2−x0(1+x0)) f 2 ← x 1 + x 2 − x 02 ( 4 − x 0 )
end procedure XP System
Test RK4 System2
Pseudocode
Sample IVP Second Order
􏰈 x′′(t) = −3 cos2(t) + 2 x(0) = 0, x′(0) = 0
Without the auxiliary conditions, the general analytic solution is
x(t) = 1t2 + 3 cos(2t) + c1t + c2 48
(9)
It is typical in ordinary differential equation solvers, such as those found in mathe- matical software libraries, for the user to interface with them by writing a subprogram in a nonautonomous format. In other words, the ordinary differential equation solver takes as input both the independent variable and the dependent variable and returns values for the right-hand side to the ordinary differential equation. Consequently, the nonautonomous programming convention may seem more natural to those who are using these software packages.
It is a useful exercise to find a physical application in your field of study or profession involving the solution of an ordinary differential equation. It is instructive to analyze and solve the physical problem by determining the appropriate numerical method and translating the problem into the format that is compatible with the available software.
Higher-Order Differential Equations and Systems
Consider the initial-value problem for ordinary differential equations of order higher than 1. A differential equation of order n is normally accompanied by n auxiliary conditions. This many initial conditions are needed to specify the solution of the differential equation pre- cisely (assuming certain smoothness conditions are present). Take, for example, a particular second-order initial-value problem
where c1 and c2 are arbitrary constants. To select one specific solution, c1 and c2 must be fixed, and two initial conditions allow this to be done. In fact, x (0) = 0 yields c2 = − 3 ,
andx′(0)=0forcesc1 =0.
Higher-Order Differential Equations
8
In general, higher-order problems can be much more complicated than this simple example because System (9) has the special property that the function on the right-hand side of the differential equation does not involve x. The most general form of an ordinary differential
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

340 Chapter 7
Initial Values Problems
IVP
equation with initial conditions that we shall consider is 􏰈x(n) = f(t,x,x′,x′′,…,x(n−1))
(10)
IVP Vector Form
(11)
Sample IVP Third Order
To illustrate, let us transform the initial-value problem 􏰈x′′′=cosx+sinx′−ex′′ +t2
(12)
x(a), x′(a), x′′(a), . . . , x(n−1)(a)
This can be solved numerically by turning it into a system of first-order differential equa-
tions.Todoso,wedefinenewvariablesx1,x2,…,xn asfollows:
x1 =x, x2 =x′, x3 =x′′, …, xn−1 =x(n−2), xn =x(n−1)
Consequently, the original Initial-Value Problem (10) is equivalent to
x′ =x  1 2
or, in vector notation,
x1(a),x2(a),…,xn(a) (all given) 􏰈 X′ = F(t, X)
where
and
X(a) = S (given)
X = [x1,x2,…,xn]T X′ =[x1′,x2′,…,xn′]T
F = [x2,x3,x4,…,xn, f]T X(a) = [x1(a),x2(a),…,xn(a)]
 x′=x3  2
 . x′ =x
n−1 n
 xn′ = f(t,x1,x2,…,xn)
Whenever a problem must be transformed by introducing new variables, it is recom- mended that a dictionary be provided to show the relationship between the new and the old variables. At the same time, this information, together with the differential equations and the initial values, can be displayed in a chart. Such systematic bookkeeping can be helpful in a complicated situation.
x(0) = 3, x′(0) = 7, x′′(0) = 13
into a form suitable for solution by the Runge-Kutta procedure. A chart summarizing the
transformed problem is as follows:
Old Variable New Variable Initial Value Differential Equation
x x1 3 x1′=x2
x ′ x 2 7 x 2′ = x 3
x′′ x3 13 x3′ =cosx1+sinx2−ex3 +t2
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
(all given)

7.4 Methods for First and Higher-Order Systems 341 So the corresponding first-order system is
 x2  X′ = x3 
cosx1 +sinx2 −ex3 +t2
and X(0) = [3, 7, 13]T .
Systems of Higher-Order Differential Equations
By systematically introducing new variables, we can transform a system of differential equations of various orders into a larger system of first-order equations. For instance, the system
 x′′ = x − y − (3x′)2 + (y′)3 + 6y′′ + 2t Sample IVP Fifth Order
IVP Vector Form
􏰈X′ = F(X)
X(a) = S (given)
X = [x0,x1,x2,…,xn]T X′ = [x0′,x1′,x2′,…,xn′ ]T
F = [1,×2,x3,x4,…,xn, f]T
 y′′′ = y′′ − x′ + ex − t (13)
x(1)=2, x′(1)=−4, y(1)=−2, y′(1)=7, y′′(1)=6
can be solved by the Runge-Kutta procedure if we first transform it according to the following
chart:
Old Variable New Variable Initial Value Differential Equation
xx1 2×1′=x2
x′ x2 −4 x2′ =x1 −x3 −9×2 +x43 +6×5 +2t
y x3 −2 x3′=x4
y ′ y′′
Hence, we have
x 4 7 x 4′ = x 5
x5 6 x5′ =x5−x2+ex1 −t
 x2  x1 −x3 −9×2 +x43 +6×5 +2t
We notice that t is present on the right-hand side of Equation (11) and that therefore the equations x0 = t and x0′ = 1 can be introduced to form an autonomous system of ordinary differential equations in vector notation. It is easy to show that a higher-order system of differential equations having the form in Equation (10) can be written in vector notation as
and X(1) = [2,−4,−2,7,6]T . Autonomous ODE Systems
where
X′ = x4
 x5 
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
x5−x2+ex1 −t


342 Chapter 7
Initial Values Problems
Sample IVP Autonomous Form
As an example, the ordinary differential equation system in Equation (12) can be written in autonomous form as
1  x2 
X′ =x1 −x3 −9×2 +x43 +6×5 +2×0
 x4   x5  x5 −x2 +ex1 −x0
and X(1) = [1,2,−4,−2,7,6]T . Summary 7.4
and
X(a) = [a,x1(a),x2(a),…,xn(a)]
• A system of ordinary differential equations ′
x = f (t,x ,x ,…,x ) 1112 n
 ′
xn′ = fn(t,x1,x2,…,xn)  .
x2 = f2(t,x1,x2,…,xn)
.
x1(a) = s1,x2(a) = s2,…,xn(a) = sn, can be written in vector notation as
􏰈 X′ = F(t, X) X(a) = S (given)
(all given)
where we define the following n component vectors X = [x , x , . . . , x ]T
 1 2 n X′ = [x1′,x2′,…,xn′ ]T
  F = [ f 1 , f 2 , . . . , f n ] T
X(a) = [x1(a),x2(a),…,xn(a)]T
• The Taylor series method of order m is
h2 h3 hm
X(t + h) = X + hX′ + X′′ + X′′′ + ··· + X(m)
2 3! m! where X = X(t), X′ = X′(t), X′′ = X′′(t), X′′′ = X′′′(t), and so on.
• The Runge-Kutta method of order 4 is
X(t+h)=X+h(K1 +2K2 +2K3 +K4)
6
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

7.4 Methods for First and Higher-Order Systems 343
where
   K 1 = F ( t , X )
    K 2 = F 􏰋 t + 1 h , X + 1 h K 1 􏰌 22
K3 =F􏰋t+1h,X+1hK2􏰌  2 2
 K 4 = F ( t + h , X + h K 3 )
Here, X = X(t), and all quantities are vectors with n components except variables t
and h.
• We can absorb the t variable into the vector by letting x0 = t and then writing the autonomous form for the system of ordinary differential equations in vector notation
as
􏰈X′ = F(X)
X(a) = S (given)
where vectors are defined to have n + 1 components. Then X = [x , x , x , . . . , x ]T
 0 1 2 n X′ = [x0′,x1′,x2′,…,xn′ ]T
 F = [1, f1, f2,…, fn]T
X(a) = [a,x1(a),x2(a),…,xn(a)]T
• TheRunge-Kuttamethodoforder4forthesystemofordinarydifferentialequations in autonomous form is
X(t+h)=X+h(K1 +2K2 +2K3 +K4) 6
where
    K 1 = F ( X )
 􏰋 􏰌
 1 K =F X+ hK 2 21
 􏰋 1 􏰌 K3=F X+2hK2
K4 =F(X+hK3)
Here, X = X (t ), and all quantities F and K i are vectors with n + 1 components except
the variables t and h.
• A single nth-order ordinary differential equation with initial values has the form
􏰈x(n) = f(t,x,x′,x′′,…,x(n−1))
x(a), x′(a), x′′(a), . . . , x(n−1)(a) (all given)
It can be turned into a system of first-order equations of the form
􏰈 X′ = F(t, X) X(a) = S (given)
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

344 Chapter 7
Initial Values Problems
a1. Consider
a4.
a5.
Write an equivalent system of first-order differential equations without t appearing on the right-hand side:
•
We can absorb the variable t into the vector notation by letting x0 = t and extending the vectors to length n + 1. Thus, a single nth-order ordinary differential equation can be written as
􏰂x′=y, x(0)=−1 ′
x′ = x2 + log(y) + t2
 y′ = ey − cos(x) + sin(tx) − (xy)7
Turn this differential equation into a system of first- order equations suitable for applying the Runge-Kutta method:
Assuming that a program is available for solving initial-value problems of the form in Equation (11), how can it be used to solve the following differential equation?
􏰈 ′′′ ′ ′′ x =t+x+2x+3x
x(1) = 3, x′(1) = −7, x′′(1) = 4
How would this problem be solved if the initial con-
ditions were x(1) = 3, x′(1) = −7, and x′′′(1) = 0?
y=x, y(0)=0
Write down the equations, without derivatives, to be used

in the Taylor series method of order 5.
a2. How would you solve this system of differential equa- tions numerically?
′2t2  x 1 = x 1 + e − t
x2′ =x2−cost x1(0) = 0, x2(1) = 0
a3. Howwouldyousolvetheinitial-valueproblem
x1′(t)=x1(t)et +sint−t2  x 2′ ( t ) = [ x 2 ( t ) ] 2 − e t + x 2 ( t ) x1(1) = 2, x2(1) = 4
if a computer program were available to solve an initial- value problem of the form x ′ = f (t , x ) involving a single unknown function x = x(t)?
x(0)=1, y(0)=3
where
where
􏰈X′ = F(X)
X(a) = S (given)
X = [x , x , x , . . . , x ]T 012 n
X′ = [x0′,x1′,x2′,…,xn′ ]T
F =[1,×2,x3,x4,…,xn, f]T
X(a) = [a,x1(a),x2(a),…,xn(a)]
   
   
X =[x1,x2,…,xn]T X′ = [x1′,x2′,…,xn′ ]T
F =[x2,x3,x4,…,xn, f]T X(a) = [x1(a),x2(a),…,xn(a)]T
 
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
6. a.
b.
􏰈 x′′′ = 2x′ + log(x′′) + cos(x) ′ ′′
x(0)=1, x (0)=−3, x (0)=5
Exercises 7.4

a7.
a8.
a9.
a10.
How would you solve this differential equation problem numerically?
􏰈′′ ′ 2
x1 = x1 + x1 − sin t
x′′=x −(x′)1/2+t 222
with initial conditins
x (0)=1, x (1)=3, x′(0)=0, x′(1)=−2 1212
Converttoafirst-ordersystemtheorbitalequations 􏰈 x′′ + x(x2 + y2)−3/2 = 0
y′′ + y(x2 + y2)−3/2 = 0 with initial conditions
x(0) = 0.5 x′(0) = 0.75 y(0) = 0.25 y′(0) = 1.0
Rewritethefollowingequationasasystemoffirst-order differential equations without t appearing on the right- hand side:
7.4 Methods for First and Higher-Order Systems 345 Determine the associated first-order system and its aux-
11.
a
Determine a system of first-order equations equivalent to each of the following:
Followtheinstructionsintheprecedingproblemonthis example:
txyz + x′y′/t = tx2 + x/y′′ + z t2x/z + y′z′t = y2 − (z′′)2x + x′y′
12.
Consider
􏰂
x(iv) =(x′′′)2 +cos(x′x′′)−sin(tx)+log􏰋x􏰌
14.
15.
16.
17.
18.
t x(0)=1, x (0)=3, x (0)=4, x (0)=5
 ′ ′′ ′′′ Expressthesystemofordinarydifferentialequations
 d2z dz
 xz  − 2t = 2te
  dt2 dt
dx −2xzdx =3x2yt2
 dt2 dt d2y dy
′ ′′2√2 2′2
a 13.
iliary initial conditions. The problem
 x′′(t) = x + y − 2x′ + 3y′ + log t  y′′(t) = 2x − 3y + 5x′ + ty′ − sin t  x(0) = 1, x′(0) = 2
y(0) = 3, y′(0) = 4
is to be put into the form of an autonomous system of five first-order equations. Give the resulting system and the appropriate initial values.
WriteprocedureXPSystemforusewiththefourth-order Runge-Kutta routine RK4 System1 for the following dif- ferential equation:
􏰈 x′′′ = 10ex′′ − x′′′ sin(x′x) − (xt)10
x(2) = 6.5, x′(2) = 4.1, x′′(2) = 3.2
Ifwearegoingtosolvetheinitial-valueproblem 􏰈 x ′′′ = x ′ − t x ′′ + x + ln t
x(1) = x′(1) = x′′(1) = 1
using Runge-Kutta formulas, how should the problem be
transformed?
Convert this problem involving differential equations into an autonomous system of first-order equations (with ini- tial values):
 2
2 − ey = 4xt2z    dt dt
z(1) = x′′(1) = y′(1) = 2 z′(1) = x(1) = y(1) = 3
as a system of first-order ordinary differential equations.
 
3x+tanx−x= t+1+y+(y)
−3y′ + cot y′′ + y2 = t2 + (x + 1)1/2 + 4x′ x(1)=2, x′(1)=−2, y(1)=7, y′(1)=3
aa. x′′′+x′′sinx+tx′+x=0 (iv) ′′ ′ ′
tyz − x′z′y′ = z2 − zx′′ − (yz)′ x(3)=1, y(3)=2, z(3)=4  x′(3) = 5, y′(3) = 6, z′(3) = 7
Turn this pair of differential equations into a second-order differential equation involving x alone:
b. x +x cosx +txx =0
 
c.
􏰈
x′′ = 3×2 − 7y2 + sin t + cos(x′ y′) y′′′ = y + x2 − cos t − sin(xy′′)
􏰂x′′ =x′−x x(0)=0,
x′(0)=1
x′ =−x+axy y′ =3y−xy
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

346 Chapter 7 Initial Values Problems
a1. Solve the system of differential equations (1) by using a7. two different methods given in this section and compare
the results with the analytic solution.
a2. Solve the initial-value problem
 x ′ = t + x 2 − y
y′ = t2 − x + y2 x(0)=3, y(0)=2
by means of the Taylor series method using h = 1/128 8. on the interval [0, 0.38]. Include terms involving three derivatives in x and y. How accurate are the computed
function values?
9.
a4. Write procedure RK4 System2 and a driver program
for solving the ordinary differential equation system
given by Equation (2). Use h = −10−2, and print out
the values of x0, x1, and x2, together with the true
solution on the interval [−1, 0]. Verify that the true so-
lution is x(t) = et +6+6t +4t2 +t3 and y(t) = 10. et−t3+t2+2t+2.
a5. Using the Runge-Kutta procedure, solve the following initial-value problem on the interval 0 ≦ t ≦ 2π . Plot the resulting curves (x1(t), x2(t)) and (x3(t), x4(t)). They should be circles.
Write and test a program, using the Taylor series method of order 5, to solve the system

x(5)=2, y(5)=3 −3
3. Write the Runge-Kutta procedure to solve x′ =−3x
Print a table of sin t and cos t on the interval [0, π/2] by numerically solving the system
 x ′ = y  y′ = −x
x(0)=0, y(0)=1
Write a program for using the Taylor series method of
1 2 x′ = 1x

 2 3 1 x1(0) = 0,
order 3 to solve the system
   x ′ = t x + y ′ − t 2
 y ′ = t y + 3 t
z′ =tz−y′+6t3

x(0)=1, y(0)=2 z(0)=3 on the interval [0, 0.75] using h = 0.01.
Write and test a short program for solving the system of differential equations

 X =  − x 1 x 12 + x 2 2  11.
series method of order 4.
Recode and test procedure RK4 System2 using a com- puter language that supports vector operations.
Verify the numerical results given in the text for the system of differential equations (1) from programs Test RK4 System1 and RK4 System2.
(Continuation) Use mathematical software such as MATLAB, Maple, or Mathematica containing symbolic manipulation capabilities to verify the analytic solution for the system of differential equations (1).
(Continuation)Usemathematicalsoftwareroutinessuch as in MATLAB, Maple, or Mathematica to verify the numerical solutions given in the text. Plot the resulting
x4  ′􏰀 􏰁−3/2
 x ′ = t x − y 2 + 3 t y′ = x2 − ty − t2
on the interval [5, 6] using h = 10 . Print values of x and y at steps of 0.1.
x2(0) = 1 on the interval 0 ≦ t ≦ 4. Plot the solution.
x  3
y(2)=5, x(2)=3
over the interval [2, 5] with h = 0.25. Use the Taylor
 y ′ = x 3 − t 2 y − t 2 x′ = tx2 − y4 + 3t
 −x 􏰀x2 + x2􏰁−3/2 212
 X(0) = [1, 0, 0, 1]T 12. 6. Solvetheproblem
  x ′ = 1
x1′ =−x2+cosx0
13.
14.
x2′= x1+sinx0   0
x0(1) = 1, x1(1) = 0,
Use the Runge-Kutta method and the interval −1 ≦ t ≦ 2.
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole deemed that any suppressed content does not materially affect the overall learning experience.
or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
x2(1) = −1
Computer Exercises 7.4

solution curve. Compare with the results from programs Test RK4 System1 and Test RK4 System2.
15. Use RK4 System1 to solve each of the following for 0 ≦ t ≦ 1. Use h = 2−k with k = 5, 6, and 7, and compare resu􏰈lts.
x′′ = 2(e2t − x2)1/2
7.5 Adams-Bashforth-Moulton Methods 347 17. Solve 􏰈′′ ′ 2
x + x + x − 2t = 0
x(0) = 0, x′(0) = 0.1
on [0, 3] by any convenient method. If a plotter is avail-
able, graph the solution. 18. Solve 􏰈′′ ′
x =2x −5x
x(0) = 0, x′(0) = 0.4 on the interval [−2, 0].
19. Write computer programs based on the pseudocode in the text to find the numerical solution of these ordinary differential equation systems:
a. IVP (9) b. IVP (12) c. IVP (13)
20. (Continuation) Use mathematical software such as MAT- LAB, Maple, or Mathematica with symbolic manipula- tion capabilities to find their analytical solutions.
21. (Continuation) Use mathematical software routines such as in MATLAB, Maple, or Mathematica to verify the nu- merical solutions for these ordinary differential equation systems. Plot the resulting solution curves.
a.
b.
x(0) = 0, x′(0) = 1  x′′ = x2 − y + et

 y′′ = x − y2 − et

x(0) = 0, x′(0) = 0
y(0) = 1, y′(0) = −2
16. SolvetheAirydifferentialequation
x′′ =tx
x (0) = 0.35502 80538 87817 x′(0) = −0.25881 94037 92807
on the interval [0, 4.5] using the Runge-Kutta method. Check value: x (4.5) = 0.00033 02503 is correct.
7.5 Adams-Bashforth-Moulton Methods
Single-Step versus Multi-Step Methods
Adams-Bashforth Predictor Step
by means of single-step numerical methods. In other words, if the solution X(t) is known at a particular point t, then X(t + h) can be computed with no knowledge of the solution at points earlier than t. The Runge-Kutta and Taylor series methods compute X(t + h) in terms of X(t) and various values of F.
Moreefficientmethodscanbedevisedifseveralvalues X(t), X(t−h), X(t−2h),… are used in computing X(t + h). Such methods are called multi-step methods. They have the obvious drawback that at the beginning of the numerical solution, no prior values of X are available. So it is usual to start a numerical solution with a single-step method, such as the Runge-Kutta procedure, and transfer to a multistep procedure for efficiency as soon as enough starting values have been computed.
An example of a multistep formula is known as the Adams-Bashforth formula (see Section 7.3 (p. 325) and the related Computer Exercises 7.3.2–4). It is
X􏰮(t + h) = X(t) + h {55F[X(t)] − 59F[X(t − h)] + 37F[X(t − 2h)] 24
− 9F[X(t − 3h)]} (2)
A Predictor-Corrector Scheme
The procedures explained so far have solved the initial-value problem
􏰈X′ = F(X)
X(a) = S (given)
(1)
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

348 Chapter 7
Initial Values Problems
Adams-Moulton Corrector Step
Adams-Moulton (Predictor-Corrector) Method
Here, X􏰮(t + h) is the predicted value of X(t + h) computed by using Formula (2). If the solution X has been computed at the four points t, t − h, t − 2h, and t − 3h, then Formula (2) can be used to compute X􏰮 (t + h). If this is done systematically, then only one evaluation of F is required for each step. This represents a considerable savings over the fourth-order Runge-Kutta procedure; the latter requires four evaluations of F per step. (Of course, a consideration of truncation error and stability might permit a larger step size in the Runge-Kutta method and make it much more competitive.)
In practice, Formula (2) is never used by itself. Instead, it is used as a predictor, and then another formula is used as a corrector. The corrector that is usually used with Formula (2) is the Adams-Moulton formula:
X(t+h)=X(t)+ h{9F[X􏰮(t+h]+19F[X(t)]−5F[X(t−h)] 24
+ F[X(t − 2h)]} (3)
Thus, Formula (2) predicts a tentative value of X(t + h), and Formula (3) computes this X value more accurately. The combination of the two formulas results in a predictor-corrector scheme.
With initial values of X specified at a, three steps of a Runge-Kutta method can be performed to determine enough X values that the Adams-Bashforth-Moulton procedure can begin. The fourth-order Adams-Bashforth and Adams-Moulton formulas, started with the fourth-order Runge-Kutta method, are referred to as the Adams-Moulton method. Predictor and corrector formulas of the same order are used so that only one application of the corrector formula is needed. Some suggest iterating the corrector formula, but experience has demonstrated that the best overall approach is only one application per step.
Pseudocode
Storage of the approximate solution at previous steps in the Adams-Moulton method is usually handled either by storing in an array of dimension larger than the total number of steps to be taken or by physically shifting data after each step (discarding the oldest data and storing the newest in their place). If an adaptive process is used, the total number of steps to be taken cannot be determined beforehand. Physical shifting of data can be eliminated by cycling the indices of a storage array of fixed dimension. For the Adams-Moulton method, the xi data for X(t) are stored in a two-dimensional array with entries zim in locations m = 1,2,3,4,5,1,2,… for t = a,a + h,a + 2h,a + 3h,a + 4h,a + 5h,a + 6h,…, respectively. The sketch in Figure 7.6 shows the first several t values with corresponding m values and abbreviations for the formulas used.
m:1 2 3 4 5 1 2
How the Scheme Works
FIGURE 7.6
Starting values for applications of RK and AB/AM methods
a a 1 h RK
a 1 2h RK
a 1 3h RK
a 1 4h AB / AM
a 1 5h AB / AM
a 1 6h AB / AM
An error analysis can be conducted after each step of the Adams-Moulton method. If x(p) is the numerical approximation of the ith equation in System (1) at t + h obtained by
i
predictorFormula(2)andxi isthatfromcorrectorFormula(3)att+h,thenitcanbeshown
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

7.5 Adams-Bashforth-Moulton Methods 349 that the single-step error for the i th component at t + h is given approximately by
So we compute
19 􏰣􏰣xi − x(p)􏰣􏰣 εi= i
270 |xi | est= max |εi|
1≦i≦n
in the Adams-Moulton procedure AM System to obtain an estimate of the maximum single- step error at t + h.
A control procedure is needed that calls the Runge-Kutta procedure three times and then calls the Adams-Moulton predictor-corrector scheme to compute the remaining steps. Such a procedure for doing nsteps steps with a fixed step size h follows:
procedure AMRK(n, h, (xi ), nsteps)
integer i, k, m, n; real est, h; real array (xi )0:n allocate real array ( fi j )0:n×0:4, (zi j )0:n×0:4
m←0
output h
output 0, (xi )
fori =0ton
zim ←xi end for
for k = 1 to 3
callRK System(m,n,h,(zij),(fij)) output k,(zim)
end for
for k = 4 to nsteps
callAM System(m,n,h,est,(zij),(fij),) output k,(zim)
output est
end for
fori =0ton
xi ←zim end for
deallocate array ( f, z) end procedure AMRK
AMRK Pseudocode
The Adams-Moulton method for a system and the computation of the single-step error are accomplished in the following pseudocode:
procedureAM System(m,n,h,est,(zij),(fij))
integer i, j, k, m, mp1; real d, dmax, est, h
real array (zi j )0:n×0:4, ( fi j )0:n×0:4 allocate real array (si )0:n , ( yi )0:n realarray(ai)1:4 ←(55,−59,37,−9) realarray(bi)1:4 ←(9,19,−5,1) mp1 ← (1 + m)mod5
(Continued)
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

350 Chapter 7 Initial Values Problems
callXP System(n,(zim),(fim)) fori =0ton
si ←0 end for
for k = 1 to 4
j ←(m−k+6)mod5
fori =0ton
si ←si +ak fij
end for end for
fori =0ton
yi ←zim +hsi/24
end for
callXP System(n,(yi),(fi,mp1)) fori =0ton
si ←0 end for
for k = 1 to 4
j ←(mp1−k+6)mod5
fori =0ton
si ←si +bk fij
end for end for
fori =0ton
zi,mp1 ←zim +hsi/24
end for
m ← mp1
dmax ← 0 fori =0ton
d←|zim −yi|/|zim| if d > dmax then
dmax ← d j←i
end if end for
est ← 19dmax/270 deallocate array (s, y) end procedure AM System
AM System Pseudocode
Here, the function evaluations are stored cyclically in fim for use by Formulas (2) and (3).
Various optimization techniques are possible in this pseudocode. For example, the program-
mer may wish to move the computation of 1 h outside of the loops. 24
A companion Runge-Kutta procedure is needed, which is a modification of procedure RK4 System2 from Section 7.4:
procedureRK System(m,n,h,(zij),(fij))
integer i, m, mp1, n; real h; real array (zi j )0:n×0:4, ( fi j )0:n×0:4 allocate real array (gi j )0:n×0:3, (yi )0:n
(Continued)
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

7.5 Adams-Bashforth-Moulton Methods 351
mp1 ← (1 + m)mod5
callXP System(n,(zim),(fim)) fori =0ton
yi ←zim +1hfim 2
end for
callXP System(n,(yi),(gi,1)) fori =0ton
yi ←zim +1hgi,1 2
end for
callXP System(n,(yi),(gi,2)) fori =0ton
yi ←zim +hgi,2 end for
callXP System(n,(yi),(gi,3)) fori =0ton
zi,mp1 ← zim + h[ fim + 2gi,1 + 2gi,2 + gi,3]/6 end for
m ← mp1
deallocate array (gi j ), (yi ) end procedure RK System
RK System Pseudocode
As before, the programmer may wish to move 1 h out of the loop. 6
To use the Adams-Moulton pseudocode, we supply the procedure XP System that defines the system of ordinary differential equations and write a driver program with a call to procedure AMRK. The complete program then consists of the following five parts: the main program and procedures XP System, AMRK, RK System, and AM System.
As an illustration, the pseudocode for IVP (12) in Section 7.4 is as follows:
program Test AMRK
real h; real array (xi )0:n integer n ← 5, nsteps ← 100 real a ← 0, b ← 1
(xi) ← (1,2,−4,−2,7,6)
h ← (b − a)/nsteps
call AMRK(n, h, (xi ), nsteps) end program Test AMRK
procedure XP System(n, (xi ), ( fi )) integer n; real array (xi )0:n , ( fi )0:n
f0 ← 1
f1 ← x2
f2 ← x1 −x3 −9×2 +x43 +6×5 +2×0 f3 ← x4
f4 ← x5
f5 ← x5 − x2 + ex1 − x0
end procedure XP System
Test AMRK Pseudocode
Here, we have programmed this procedure for an autonomous system of ordinary differential equations.
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

352 Chapter 7
Initial Values Problems
How Adaptive Scheme Works
An Adaptive Scheme
Since an estimate of the error is available from the Adams-Moulton method, it is natural to replace procedure AMRK with one that employs an adaptive scheme—that is, one that changes the step size. A procedure similar to the one used in Section 7.3 is outlined here. The Runge-Kutta method is used to compute the first three steps, and then the Adams-Moulton method is used. If the error test determines that halving or doubling of the step size is necessary in the first step using the Adams-Moulton method, then the step size is halved or doubled, and the whole process starts again with the initial values—so at least one step of the Adams-Moulton method must take place. If during this process the error test indicates that halving is required at some point within the interval [a, b], then the step size is halved. A retreat is made back to a previously computed value, and after three Runge-Kutta steps have been computed, the process continues, using the Adams-Moulton method again but with the new step size. In other words, the point at which the error was too large should be computed by the Adams-Moulton method, not the Runge-Kutta method. Doubling the step size is handled in an analogous manner. Doubling the step size requires only saving an appropriate number of previous values; however, one can simplify this process (whether halving or doubling the step size) by always backing up two steps with the old step size and then using this as the beginning point of a new initial-value problem with the new step size. Other, more complicated procedures can be designed and can be the subject of numerical experimentation. (See Computer Exercise 7.5.3.)
An Engineering Example
In chemical engineering, a complicated production activity may involve several reactors connected with inflow and outflow pipes. The concentration of a certain chemical in the i th reactorisanunknownquantity,xi.Eachxi isafunctionoftime.Iftherearenreactors,the whole process is governed by a system of n differential equations of the form
􏰈 X′ = AX + V X(0) = S (given)
where X is the vector containing the unknown quantities xi , A is an n × n matrix, and V is a constant vector. The entries in A depend on the flow rates permitted between different reactors of the system.
There are several approaches to solving this problem. One is to diagonalize the matrix A by finding a nonsingular matrix P for which is P −1 A P is diagonal and then using the matrix exponential function to solve the system in an analytic form. This is a task that mathematical software can handle. On the other hand, we can simply turn the problem over to an ODE solver and get the numerical solution. One piece of information that is always wanted in such a problem is a description of the steady state of the system. That means the values of all variables at t = ∞. Each function xi should be a linear combination of
exponential functions of the form t 􏰷→ eλt , in which λ < 0. Here is a simple example that can illustrate all of this: x1′−8/3−4/3 1x112 x2′ =−17/3 −4/3 1x2+29 (4) x3′ −35/3 14/3 −2 x3 48 Sample ODE 1 Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 7.5 Adams-Bashforth-Moulton Methods 353 Using mathematical software such as MATLAB, Maple, or Mathematica, we can obtain a closed-form solution: x(t) = 1e−3t(6−50et +10e2t +34e3t)  6 z(t)= 1e−3t(14−200et +70e2t +116e3t) y(t) = 1e−3t(12−125et +40e2t +73e3t) 6 FIGURE 7.7 Solution curves for a stiff ODE Sample ODE 2 True Solution For a system of ordinary differential equations with a large number of variables, it may be more convenient to represent them in a computer program with an array such as x(i,t) rather than by separate variables names. To see the numerical value of the analytic solution at a single point, say, t = 2.5, we obtain x(2.5) ≈ 5.74788, y(2.5) ≈ 12.5746, z(2.5) ≈ 20.0677. Also, we can produce a graphing of the analytic solution to the problem. Finally, the programs presented in this section can be used to generate a numerical solution on a prescribed interval with a prescribed number of points. Stiff ODEs and an Example In many applications of differential equations there are several functions to be tracked together as functions of time. A system of ordinary differential equations may be used to model the physical phenomena. In such a situation, it can happen that different solution functions (or different components of a single solution) have quite disparate behavior that makes the selection of the step size in the numerical solution problematic. For example, one component of a function may require a small step in the numerical solution because it is varying rapidly, whereas another component may vary slowly and not require a small step size for its computation. Such a system is said to be stiff. Figure 7.7 illustrates a slowly varying solution surrounded by other solutions with rapidly decaying transients. x t An example illustrates this possibility. Consider a system of two differential equations with initial conditions: 􏰈x′ =−20x−19y, x(0)=2 (5) y′ =−19x−20y, y(0)=0 6 The solution is easily seen to be 􏰈x(t)=e−39t +e−t y(t)=e−39t −e−t The component e−39t quickly becomes negligible as t increases, starting at 0. The solution is then approximately given by x(t) = −y(t) = e−t, and this function is smooth and decreasing to 0. It would seem that in almost any numerical solution, a large step size could Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 354 Chapter 7 Initial Values Problems be used. However, let’s examine the simplest of numerical procedures: Euler’s method. It Difference Equations Closed Form generates the solution by using the following equations: 􏰈xn+1 =xn +h(−20xn −19yn), x0 =2 yn+1 = yn + h(−19xn − 20yn), y0 = 0 These difference equations can be solved in closed form, and we have 􏰈xn =(1−39h)n +(1−h)n yn = (1−39h)n −(1−h)n For the numerical solution to converge to 0 (and thus imitate the actual solution), it is necessary that h < 2 . If we were solving only the differential equation x′ = −x to get the 39 solution x(t) = e−t , the step size could be as large as h = 2 to get the correct behavior as t increased. (See Exercise 7.5.2.) To see that numerical success (in the sense of being able to use a reasonable step size) depends on the method used, let us consider the implicit Euler method. For a single differential equation, this employs the formula xn+1 =xn +hf(tn+1,xn+1) Since xn+1 appears on both sides of this equation, the equation must be solved for xn+1. In the example being considered, the Euler equations are 􏰈 xn+1 = xn + h(−20xn+1 − 19yn+1) yn+1 = yn + h(−19xn+1 − 20yn+1) This pair of equations has the form Xn+1 = Xn + AXn+1, where A is the 2×2 matrix in the previous pair of equations and Xn is the vector having components xn and yn. This matrix equation can be written (I − A)Xn+1 = Xn or Xn+1 = (I − A)−1 Xn. A consequence is that the explicit solution is Xn = (I − A)−n X0. At this point, it is necessary to appeal to a result concerning such iterative processes. For Xn to converge to 0 for any choice of initial vector X0, it is necessary and sufficient that all eigenvalues of (I − A)−1 be less than one in modulus (see Kincaid and Cheney [2002]). Equivalently, the eigenvalues of I − A should be greater than 1 in modulus. An easy calculation shows that for positive h this condition is met, without further hypotheses. Thus, the implicit Euler method can be used with any reasonable step size on this problem. In the literature on stiff equations, much more infor- mation can be found, and there are books that address this topic thoroughly. Some essential references are Dekker and Verwer [1984], Gear [1971], Miranker [1981], and Shampine and Gordon [1975]. In general, stiff ordinary differential equations are rather difficult to solve. This is com- pounded by the fact that in most cases, you do not know beforehand whether an ordinary differential equation that you’re trying to solve numerically is stiff. Software packages usu- ally have ordinary differential equation solvers specifically designed to handle stiff ordinary differential equations. Some of these procedures may vary both the step size and the order of the method. In such algorithms, the Jacobian matrix ∂ F /∂ X y may play a role. Solving an associated linear system involving the Jacobian matrix is critical to the reliability and efficiency of the code. The Jacobian matrix may be sparse, an indication that the function F does not depend on some of the variables in the problem. Euler’s Equations Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. Additional Reading 7.5 Adams-Bashforth-Moulton Methods 355 History of ODE Numerical Methods For readers interested in the history of numerical analysis, we recommend the book by Goldstine [1977]. The textbook on differential equations by Moulton [1930] gives some insight into the numerical methods used prior to the advent of high-speed computing ma- chines. Also Moulton gives some of the history, going back to Newton! The calculation of orbits in celestial mechanics has always been a stimulus for the invention of numerical methods; the needs of ballistic science have been also. Moulton mentions that the retardation of a projectile by air resistance is a very complicated function of velocity that necessitates numerical solution of the otherwise simple equations of ballistics. Summary 7.5 • For the autonomous form for a system of ordinary differential equations in vector notation 􏰈 X′ = F(X) X(a) = S (given) the Adams-Bashforth-Moulton method of fourth order is h􏰟􏰠 X􏰮(t + h) = X(t) + 24 55F[X(t)] − 59F[X(t − h)] + 37F[X(t − 2h)] − 9F[X(t − 3h)] X(t + h) = X(t) + h 􏰟9F[X􏰮(t + h)] + 19F[X(t)] − 5F[X(t − h)]􏰠 24 + F[X(t − 2h)] Here, X􏰮 (t + h) is the predictor, and X(t + h) is the corrector. The Adams-Bashforth- Moulton method needs five evaluations of F per step. With the initial vector X(a) given, the values for X(a + h), X(a + 2h), X(a + 3h) are computed by the Runge- Kutta method of fourth order. Then the Adams-Bashforth-Moulton method can be used repeatedly. The predicted value X􏰮 is computed from the four X values at t, t −h, t −2h, and t −3h, and then the corrected value X(t +h) can be computed by using the predictor value X􏰮(t + h) and previously evaluated values of F at t, t − h, and t − 2h. a1. Findthegeneralsolutionofthissystembyturningitinto a first-order system of􏰂four equations: x′′ = αy y′′ =βx 2. Verify the assertions made about the step size h in the discussionofstiffEquation(5). 3. Write autonomous systems of first-order differential equations for each of these: y′′ + yz = 0 a. z′ + 2yz = 4 y(0) = 1, y′(0) = 0, z(0) = 3 b. c. 􏰂x′′′ −sin(x′′)+etx′ +2tcosx =25 x(0) = 5, x′(0) = 3, x′′(0) = 7 􏰂 x′′′ − [sin x′′ + et x′]2 + cos x = 0 x(0)=3, x′(0)=4, x′′(0)=5 Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 4. Correctthefollowingsystemsofhigher-orderdifferential equations into a system of first-order equations in which t does not appear explicitly:  x′′′ −5tx′′y′′ −ln(x′)z = 0 y′′ − sin(ty) + 7tx′′ = 0 z′ + 16ty′ − et zx′ = 0 Exercises 7.5 356 Chapter 7 Initial Values Problems 1. Test the procedure AMRK on the system given in Com- puter Exercise 7.4.2. 2. The single-step error is closely controlled by using fourth- order formulas; however, the roundoff error in performing the computations in Equations (3) and (4) can be large. Itislogicaltocarrytheseoutinwha􏰃tisknownaspar- tial double-precision arithmetic. The function F would be evaluated in single precision at the desired points X(t +ih), but the linear combination i ci F(X(t +ih)) would be accumulated in double precision. Also, the addi- tion of X (t ) to this result is done in double precision. Re- code the Adams-Moulton method so that partial double- precision arithmetic is used. Compare this code with that in the text for a system with a known solution. How do they compare with regard to roundoff error at each step? 3. Write and test an adaptive process similar to RK45 Adaptive in Section 7.3 with calling sequence This routine should carry out the adaptive procedure out- lined in this section and be used in place of the AMRK procedure. 4. Solve the predator-prey problem in the example at the order differential equation: 􏰄3 􏰀 􏰁 x′′= mx−x/d3 k=1 k̸=j ij k ik ij jk where d2 = 􏰃2 (xij − xik)2 for k, j = 1,2,3. As- jk sume that the bodies have equal mass, say, m1 = m2 = i=1 m3 = 1, and with the appropriate starting conditions, they will follow the same figure-eight orbit as a peri- odic steady-state solution. When the system is rewrit- ten as a first-order system, the dimension of the prob- lem is twelve, and the initial conditions at t = 0 are given by  −0.97000436, 0.24308753, 0.0, ′ x = x′ = 21 x′ = 12 22 0.466203685 0.43236573 −0.93240737 −0.86473146 0.466203685 0.43236573 x  = = =  11 11  x21   x 12   x = 0.0, x′ = 22  x13 procedure AMRK Adaptive(n,h,ta,tb,(xi), itmax, εmin, εmax, hmin, hmax, iflag) x′ = x23 = −0.24308753, x′ = beginningofthischapterwitha=−10−2,b=−1×102, 4  vx =10(x2−x1) ′ −22 1 tions of time t . x′ =x1x2−8x3 33 7b. Solve the following test problem and plot the solution curves. The Lorenz problem is well known, and it arises in the study of dynamical systems: 0.97000436,  13 23 Solve the problem for t ∈ [0, 20]. = c = 10 and d =− 10 and with initial values u(0) = 80,  v(0) = 30. Plot u (the prey) and v (the predator) as func- x2′ =x1(28−x3)−x2 5. Solve and plot the numerical solution of the system of x1(0) = 15, x2(0) = 15, x3(0) = 36 ordinary differential equations given by Equation (4) us- ing mathematical software such as MATLAB, Maple, or Mathematica. 6. (Continuation) Repeat for Equation (5) using a routine specifically designed to handle stiff ordinary differential equations. 7a. Solve the following test problem and plot the solution curves. This problem corresponds to a recently discov- ered stable orbit that arises in the restricted three-body problem in which the orbits are co-planar. The two spa- tial coordinates of the j th body are x1 j and x2 j for j = 1, 2, 3. Each of the six coordinates satisfies a second- Solve the problem for t ∈ [0, 20]. It is known to have so- lutions that are potentially poorly conditioned. Note: For additional details on these problems, see Enright [2006]. 8. Write a computer program based on pseudocode Test AMRK to find the numerical solution to the ordinary differential equation systems, and compare the results with that by using a built-in routine such as in MAT- LAB, Maple, or Mathematica. Plot the resulting solution curves. 9. (Tacoma Narrows Bridge) In 1940, the third-longest suspension bridge in the world collapsed in a high wind. Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. Computer Exercises 7.5  y′′=−y′d     θ′′ =−θy′d  −e FIGURE 7.8 Tacoma Narrows Bridge collapsing in 1940. The following system of differential equations is a math- ematical model that attempts to explain how twisting os- cillations can be magnified and cause such a calamity: − [K/(ma)]􏰗ea(y−lsinθ) − 1+ea(y+lsinθ) 􏰘 −1 +0.2Wsinωt + (3cosθ/l)[K/(ma)]􏰗ea(y−lsinθ) a(y+lsinθ)􏰘 The last term in the y equation is the forcing term for the wind W , which adds a strictly vertical oscillation to the bridge. Here, the roadway has width 2l hanging between two suspended cables, y is the current distance from the center of the roadway as it hangs below its equilibrium point, and θ is the angle the roadway makes with the hori- zontal. Also, Newton’s Law F = ma is used and Hooke’s constant K . Explore how ODE solvers are used to gener- ate numerical trajectories for various parameter settings. Illustrate different types of phenomena that are available in this model. For additional details, see McKenna and Tuama [2001] and Sauer [2012]. Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 7.5 Adams-Bashforth-Moulton Methods 357 AP Images 8 358 Our main objective is to show that the naive Gaussian algorithm applied to A yields a factorization of A into a product of two simple matrices, one unit lower triangular: 1 l21 1  L =  l31 l32 1   . . . ...  ln1 ln2 ln3 ··· 1 More on Linear Systems In applications that involve partial differential equations, large linear systems arise with sparse coefficient matrices such as  4 −1  0 −1 A= 0 0  0  0 0 −1 4 −1 0 −1 0 0 0 0 0 −1 4 0 0 −1 0 0 0 −1 0 0 0 −1 0 0 0 −1 4 −1 0 −1 4 −1 0 −1 4 −1 0 0 0 −1 0 0 0 −1 0 0 0 0 0 0 −1 0 0 −1 0 0 4 −1 −1 4 0 −1 0 0 0 0 0 −1  0 −1 4 Gaussian elimination may On the other hand, iterative methods preserve its sparse structure. 8.1 Matrix Factorizations LU Factorization cause fill-in of the zero entries by nonzero values. An n × n system of linear equations can be written in matrix form Ax = b where the coefficient matrix A has the form (1) ··· a1n ··· a2n ··· a3n ... .  ··· ann  a11  a21 A =  a31 a12 a13 a22 a23 a32 a33  . . . an1 an2 an3 Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. Sample 4 × 4 Linear System After Forward Elimination In short, we refer to this as an LU factorization of A; that is, A = LU. Numerical Example The system of Equations (2) of Section 2.1 can be written succinctly in matrix form:  6 −2 2 4x1  16  12 −8 6 10x2= 26 (2)  3 −13 9 3x3  −19 −641−18x4 −34 Furthermore, the operations that led from this system to Equation (5) of Section 2.1, that Step 1 and the other upper triangular:  u22 u23 ··· U= u33 ···  ... u1n u2n  u3n . unn is, the system where M is a matrix chosen so that M A is we have the coefficient matrix for System (3). Hence, 2 4 2 2  ≡ U 2 −5 6−2 0 −4 00 0 0 2 4x1 16 2 2x2  = −6 (3) 2 −5x3 −9 0 −3 x4 −3 u11 u12 u13 ··· could be effected by an appropriate matrix multiplication. The forward elimination phase can be interpreted as starting from Equation (1) and proceeding to MAx = Mb (4) 6 −2 M A =  0 −4 00 0 0 0 −3 Step 1 of naive Gaussian elimination results in Equation (3) of Section 2.1 or the system 6 −2 2 4x1  16 0 −4 2 2x2= −6 0 −12 8 1x3  −27 023−14x4 −18 This step can be accomplished by multiplying (1) by a lower triangular matrix M1: M1 Ax = M1b which is an upper triangular matrix. Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 8.1 Matrix Factorizations 359 360 Chapter 8 More on Linear Systems Step 3 M3M2M1 Ax = M3M2M1b 1000 M =0 1 0 0 30010 where which is equivalent to where 1000 M =−2 1 0 0 1 −1 0 1 0 2 1001 Step 2 2 4x1  16 2 2x2  =  −6 2 −5x3 −9 4 −13 x4 −21 M2M1 Ax = M2M1b 1000 Notice the special form of M1. The diagonal elements are all 1’s, and the only other nonzero elements are in the first column. These numbers are the negatives of the multipliers located in the positions where they created 0’s as coefficients in Step 1 of the forward elimination phase. To continue, Step 2 resulted in Equation (4) of Section 2.1 or the system 6 −2 0 −4 0 0 0 0 M =0 1 0 0 2 0−3 1 0 0101 2 Again, M2 differs from an identity matrix by the presence of the negatives of the multipliers in the second column from the diagonal down. Finally, Step 3 gives System (3), which is equivalent to where 0 0 −2 1 Now the forward elimination phase is complete, and with M = M3M2M1 (5) we have the upper triangular coefficient System (3). Using Equations (4) and (5), we can give a different interpretation of the forward elimination phase of naive Gaussian elimination. Now we see that A = M−1U = M−1M−1M−1U 123 = LU Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 8.1 Matrix Factorizations 361 Since each Mk has such a special form, its inverse is obtained by simply changing the signs of the negative multiplier entries! Hence, we have 100010001000 L=2 1 0 00 1 0 00 1 0 0 1 01003100010 2 Unit Lower Triangular −1 0 0 1 0 −1 0 1 0 0 2 1 2 1000 =2 1 0 0 1310 2 −1−1 21 2 It is somewhat amazing that L is a unit lower triangular matrix composed of the multipliers. Notice that in forming L, we did not determine M first and then compute M−1 = L. (Why?) It is easy to verify that 1 0 0 06−2 2 4 LU=2 1 0 00−4 2 2 1 3 1 00 0 2−5 2 LU Factorization −1−1 21000−3 2  6 −2 =12 −8 3−13 −6 4 2 4 6 10=A 9 3 1 −18 We see that A is factored or decomposed into a unit lower triangular matrix L and an upper triangular matrix U. The matrix L consists of the multipliers located in the positions of the elements they annihilated from A, of unit diagonal elements, and of 0 upper triangular elements. In fact, we now know the general form of L and can just write it down directly using the multipliers without forming the M ’s and the M−1’s. The matrix U is upper kk triangular (not generally having unit diagonal) and is the final coefficient matrix after the forward elimination phase is completed. It should be noted that the pseudocode Naive Gauss of Section 2.1 replaces the original coefficient matrix with its LU factorization. The elements of U are in the upper triangular part of the (ai j )-array including the diagonal. The entries below the main diagonal in L (that is, the multipliers) are found below the main diagonal in the (ai j )-array. Since it is known that L has a unit diagonal, nothing is lost by not storing the 1’s. (In fact, we have run out of room in the (ai j )-array anyway!) Formal Derivation To see formally how the Gaussian elimination (in naive form) leads to an L U -factorization, it is necessary to show that each row operation used in the algorithm can be effected by multiplying A on the left by an elementary matrix. Specifically, if we wish to subtract λ times row p from row q, we first apply this operation to the n × n identity matrix to create an elementary matrix Mqp. Then we form the matrix product Mqp A. Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 362 Chapter 8 More on Linear Systems Before proceeding, let us verify that Mqp A is obtained by subtracting λ times row p from row q in matrix A. Assume that p < q (for in the naive algorithm, this is always true). Then the elements of Mqp = (mi j ) are  1, if i = j mij= −λ, ifi=qandj=p 0, otherwise Therefore, the elements of Mqp A are given by 􏰄n 􏰈aij (Mqp A)ij = misasj = aqj −λapj s=1 TheqthrowofMqp AisthesumoftheqthrowofAand−λtimesthepthrowofA,as was to be proved. Step k of Gaussian elimination corresponds to the matrix Mk, which is the product of n − k elementary matrices: Mk =Mnk Mn−1,k···Mk+1,k Notice that each elementary matrix M i k here is lower triangular because i > k , and therefore Mk is also lower triangular. If we carry out the Gaussian forward elimination process on A, the result will be an upper triangular matrix U. On the other hand, the result is obtained by applying a succession of factors such as Mk to the left of A. Hence, the entire process is summarized by writing
Mn−1 · · · M2 M1 A = U Since each Mk is invertible, we have
A=M−1M−1···M−1 U 1 2 n−1
Each Mk is lower triangular having 1’s on its main diagonal (unit lower triangular). Each inverse M−1 has the same property, and the same is true of their product. Hence, the matrix
k
is unit lower triangular, and we have
L = M−1 M−1 · · · M−1 (6) 1 2 n−1
This is the so-called LU factorization of A. Our construction of it depends upon not encountering any 0 divisors in the algorithm. It is easy to give examples of matrices that have no LU factorization; one of the simplest is
A=􏰆0 1􏰇 11
(Also, see Exercise 8.1.4.)
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
A = LU
ifi≠ q ifi=q

■ Theorem1
Proof
We define the Gaussian algorithm formally as follows. Let A(1) = A. Then we compute A(2), A(3), . . . , A(n) recursively by the naive Gaussian algorithm, following these equations:
a(k+1) = a(k) ij ij
a(k) a(k+1)= ik
ij a(k)
kk 􏰐a(k) 􏰑
a(k+1) = a(k) −
ij ij a(k) kj
kk
These equations describe in a precise form the forward elimination phase of the naive Gaussian elimination algorithm.
For example, Equation (7) states that in proceeding from A(k) to A(k+1), we do not alter rows 1,2,…,k or columns 1,2,…,k − 1. Equation (8) shows how the multipliers are computed and stored in passing from A(k) to A(k+1). Finally, Equation (9) shows how multiplesofrowkaresubtractedfromrowsk+1,k+2,…,ntoproduce A(k+1) from A(k).
Notice that A(n) is the final result of the process. (It was referred to as 􏰮A in the statement of the theorem.) The formal definitions of L = (li k ) and U = (u k j ) are therefore
=0
Now we draw some consequences of these equations.
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
 l i k ik
=1 = a(n)
ik
=0 = a(n)
kj
(i = k) (10) (ki) (12) ( j ≧ k) (13) (j < k) (14) First, it follows immediately from Equation (7) that a(i) = a(i+1) = ··· = a(n) (15) l 􏰂lik u kj ukj ijijij 8.1 Matrix Factorizations 363 LU FactorizationTheorem Let A = (ai j ) be an n × n matrix. Assume that the forward elimination phase of the naive Gaussian algorithm is applied to A without encountering any 0 divisors. Let the resulting matrix be denoted by 􏰮A = (􏰮ai j ). If 100···0 􏰮a21 1 0 ··· 0 L =  􏰮a31 􏰮a32 1 · · · 0   . . . . . . . . . . . . . . .  􏰮an1 􏰮an2 · · · 􏰮an,n−1 1 􏰮a11 􏰮a12 􏰮a13 ··· 􏰮a1n  0 􏰮a22 􏰮a23 ··· 􏰮a2n U=0 0 􏰮a33 ··· 􏰮a3n  . . . . . . . . . . . . . . .  0 0 ··· 0 􏰮ann A = LU and then ik a(k) (if i ≦ k or j < k) (7) (ifi>kandj=k) (8)
(if i > k and j > k) (9)

364 Chapter 8
More on Linear Systems
Likewise, we have, from Equation (7),
a(j+1) = a(j+2) = ··· = a(n)
ijijij
From Equations (16) and (8), we now have
a(j) a(n)=a(j+1)= ij
ij ij a(j) jj
From Equations (17) and (11), it follows that a(k)
l =a(n)= ik ik ik a(k)
kk
From Equations (13) and (15), we have
􏰄n k=1
􏰄i k=1
􏰄i − 1
a(k) kj ij k=1 kk
i−1 􏰄􏰜􏰝
(j < n) (16) (j j, we have
k=1
a(1) = a
a(k) − a(k+1) + a(i) ijijij
a(k) kj
a(k) +a(j) a(k) kj ij
a(k) −a(k+1) +a(j) ijijij
k=1 kk j−1
􏰄􏰜􏰝
k=1
a(1) = a
■
ij ij
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

Unit Lower Triangular System
Upper Triangular System
Forward Substitution Pseudocode
for z and
Lz = b (20)
Ux = z (21)
Pseudocode
The following is the pseudocode for carrying out the LU factorization, which is sometimes called the Doolittle factorization:
Solving Linear Systems Using LU Factorization Once the LU-factorization of A is available, we can solve the system
Ax = b
by writing
LUx = b Then we solve two triangular systems:
for x. This is particularly useful for problems that involve the same coefficient matrix A and many different right-hand vectors b.
Since L is unit lower triangular, z is obtained by the pseudocode
8.1 Matrix Factorizations 365
integer i, k, n; real array (ai j )1:n×1:n , (li j )1:n×1:n , (ui j )1:n×1:n for k = 1 to n
lkk ←1
for j = k to n
􏰄k − 1 s=1
end do
for i = k + 1 to n
k−1
􏰋 􏰄 􏰌􏰾
lik ← aik − lisusk ukk
end do end do
s=1
ukj ←akj −
lksusj
Doolittle Factorization Pseudocode
integer i, n; real array (bi )1:n , (li j )1:n×1:n , (zi )1:n z1 ← b1
fori =2ton
􏰄i − 1 j=1
zi ←bi − end for
lijzj
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

366 Chapter 8 More on Linear Systems
Likewise, x is obtained by the pseudocode
integer i, n; real array (ui j )1:n×1:n , (xi )1:n , (zi )1:n
xn ←zn/unn
fori =n−1to1 step−1
end for
􏰍􏰄n 􏰎􏱆
xi← zi− uijxj uii j=i+1
Backward Substitution Pseudocode
The first of these two algorithms applies the forward phase of Gaussian elimination to the right-hand-side vector b. (Recall that the li j ’s are the multipliers that have been stored in the array (ai j ).)
The easiest way to verify this assertion is to use Equation (6) and to rewrite the equation
in the form
From this, we get immediately
Lz=b
M−1M−1···M−1 z=b 1 2 n−1
z = Mn−1 ···M2M1b
Thus, the same operations used to reduce A to U are to be used on b to produce z.
Another way to solve Equation (20) is to note that what must be done is to form Mn−1Mn−2 ···M2M1b
This can be accomplished by using only the array (bi ) by putting the results back into b; that is,
We know what Mk looks like because it is made up of negative multipliers that have been saved in the array (ai j ). Consequently, we have
1
 …

 Mk b = 
b1
  bk 
 −aik  . . .
−ank
bi bn
b ← Mk b
1 −ak+1,k 1
.
   . 
The entries b1 to bk are not changed by this multiplication, while bi (for i ≧ k + 1) is replaced by −ai k bk + bi . Hence, the following pseudocode updates the array (bi ) based on the stored multipliers in the array a:
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
…
  bk +1    . 
1
   . 
. . .
1

Update rhs b Pseudocode
This pseudocode should be familiar. It is the process for updating b from Section 2.2 (p. 91). The algorithm for solving Equation (21) is the back substitution phase of the naive
Gaussian elimination process.
LDLT Factorization
In the L D L T factorization, L is unit lower triangular, and D is a diagonal matrix. This factorization can be carried out if A is symmetric and has an ordinary LU factorization, with L unit lower triangular. To see this, we start with
LU=A=AT =(LU)T =UTLT
Since L is unit lower triangular, it is invertible, and we can write U = L−1UT LT . Then U(LT )−1 = L−1UT . Since the right side of this equation is lower triangular and the left side is upper triangular, both sides are diagonal, say, D. From the equation U(LT )−1 = D, wehaveU=DLT andA=LU=LDLT.
We now derive the pseudocode for obtaining the L D L T factorization of a symmetric matrix A in which L is unit lower triangular and D is diagonal. In our analysis, we write ai j as generic elements of A and li j as generic elements of L . The diagonal of D has elements dii, or di. From the equation A = LDLT , we have
􏰄n 􏰄n ν=1 μ=1
nn
n
= liνdνljν (1≦i, j≦n)
ν=1
Use the fact that lij = 0 when j > i and lii = 1 to continue the argument
aij = Assume now that j ≦ i. Then
aij = = =
liνdνljν
liνdνljν +lijdjljj liνdνljν +lijdj
(1≦ j≦i≦n)
a = ij
l d lT iν νμ μj
= 􏰄􏰄􏰄liνdνδνμljμ ν=1 μ=1
m􏰄in(i, j) ν=1
􏰄j ν=1
􏰄j − 1 ν=1
􏰄j − 1 ν=1
liνdνljν
(1≦i,j≦n)
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
8.1 Matrix Factorizations 367
integer i, k, n; real array (ai j )1:n×1:n , (bi )1:n for k = 1 to n − 1
for i = k + 1 to n
bi ←bi −aikbk
end for end for

368 Chapter 8
More on Linear Systems
LDLT Factorization Pseudocode
In particular, let j = i. We get
􏰄i − 1 ν=1
Equivalently, we have
􏰄i − 1
(1≦i≦n)
(1≦i≦n)
Particular cases of this are
d=a − dl2 i ii νiν
ν=1
  d = a 1 11
aij = Solving for li j , we obtain
liνdνljν+lijdj
aii =
liνdνliν +di
d2 =a22−d1l21
d3 =a33−d1l231−d2l232
etc.
Now we can limit our attention to the cases 1≦ j < i ≦ n, where we have 􏰄j − 1 ν=1 􏰆􏰄j−1 􏰇􏱆 lij= aij− liνdνljν ν=1 Let’s do some checking. Taking j = 1, we have (1≦j 0 for every nonzero vector x. It follows at once that A is nonsingular because A obviously cannot map any nonzero vector into 0. Moreover, by considering special vectors of the form x = (x1,x2,…,xk,0,0,…,0)T , we see that the leading principal minors of A are also positive definite. Theorem 1 implies that A has an LU decomposition. By the symmetry of
CholeskyTheoremonLLT Factorization
If A is a real, symmetric, and positive definite matrix, then it has a unique factorization A = LLT
in which L is lower triangular with a positive diagonal.
■ Theorem2
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

370 Chapter 8
More on Linear Systems
A, we then have, from the previous discussion
A = LDLT
It can be shown that D is positive definite, and thus its elements dii are positive. Denoting by D1/2 the diagonal matrix whose diagonal elements are √di i , we have
A = L􏰮 L􏰮 T
where L􏰮 ≡ L D1/2, which is the Cholesky factorization. We leave the proof of uniqueness to the reader.
The algorithm for the Cholesky factorization is a special case of the general LU factorization algorithm. If A is real, symmetric, and positive definite, then by Theorem 2, it has a unique factorization of the form
A = LLT
in which L is lower triangular and has positive diagonal. Thus, in the equation A = LU, we must have U = LT . In the kth step of the general algorithm, the diagonal entry is computed by
(22)
s=1
The algorithm for the Cholesky factorization is as follows:
􏰍􏰄􏰎
k−1 1/2 lkk=akk− l2ks
integer i, k, n, s; real array (ai j )1:n×1:n , (li j )1:n×1:n for k = 1 to n 􏰄
lkk←akk− l2ks s=1
for i = k + 1 to n
􏰍 􏰄 􏰎􏱆
end do end do
s=1
􏰍 k−1 􏰎1/2
k−1
lik ← aik − lislks lkk
Cholesky Factorization Pseudocode
Theorem 2 guarantees that lkk > 0. Observe that Equation (22) gives us the following bound:
√
from which we conclude that
􏰄k s=1
l 2k s ≧ l 2k j |lkj|≦ akk
a k k =
( j ≦ k )
(1≦ j≦k)
Hence, any element of L is bounded by the square root of a corresponding diagonal element in A. This implies that the elements of L do not become large relative to A even without any pivoting. In the Cholesky algorithm (and the Doolittle algorithms), the dot products of vectors should be computed in double precision to avoid a buildup of roundoff errors.
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

EXAMPLE 2
Solution
8.1 Matrix Factorizations 371 Determine the Cholesky factorization of the matrix in Example 1.
Using the results from Example 1, we write
where
L􏰮 = L D 1 / 2
A=LDLT =(LD1/2)(D1/2LT)=L􏰮L􏰮T 10002000
3 1 0 00 1√3 0 0 =4  2 􏰔 
12100020 233
11110001 4322
√
2 0 0 0 2.0000 0 0 0 3 1√3 0 0 

  1.5000 0.8660 0 0 =11320= 
 2 2 √ 􏰔
3 3 1.0000 0.5774 0.8165 0
 1 1√31􏰔2 1 
2 6 2 3 √2 0.5000 0.2887 0.4082 0.7071 Clearly, L􏰮 is the lower triangular matrix in the Cholesky factorization
Solving Ax = B
A = L􏰮 L􏰮 T
Many software packages for solving linear systems allow the input of multiple right-hand
sides. Suppose an n × m matrix B is
B = [b(1), b(2), . . . , b(m)]
in which each column corresponds to a right-hand side of the m linear systems Ax(j) = b(j)
for1≦ j≦m.Thus,wecanwrite
A[x(1), x(2),…, x(m)] = [b(1), b(2),…, b(m)]
or
AX = B
From Section 2.2, procedure Gauss can be used once to produce a factorization of A, and procedure Solve can be used m times with right-hand side vectors b( j ) to find the m solution vectorsx(j) for1≦ j≦m.
Multiple Right-Hand Sides
Since the factorization phase can be done in 1 n3 long operations while each of the back 3
■
substitution phases requires n2 long operations, this entire process can be done in 1 n3 +mn2 􏰀13 2􏰁 3
Operation Count
long operations. This is much less than m 3 n + n , which is what it would take if each of the m linear systems were solved separately.
Computing A−1
In some applications, such as in statistics, it may be necessary to compute the inverse of a
matrix A and explicitly display it as A−1. If an n × n matrix A has an inverse, it is an n × n
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

372 Chapter 8 More on Linear Systems
matrix X with the property that
Computing A Inverse
AX = I (23) where I is the identity matrix. If x(j) denotes the jth column of X and I(j) denotes the jth
column of I, then matrix Equation (23) can be written as
A[x(1), x(2),…, x(n)] = [I(1), I(2),…, I(n)]
This can be written as n linear systems of equations of the form Ax(j) = I(j) (1≦ j≦n)
This can be done by using procedures Gauss and Solve from Section 2.2. Use procedure Gauss once to produce a factorization of A, and use procedure Solve n times with the right-hand side vectors I ( j ) for 1 ≦ j ≦ n. This is equivalent to solving, one at a time, for the columns of A−1, which are x(j). Hence, we obtain
A−1 =[x(1),x(2),…,x(n)]
A word of caution on computing the inverse of a matrix: In solving a linear system Ax = b, it is not advisable to determine A−1 and then compute the matrix-vector product x = A−1b because this requires many unnecessary calculations, compared to directly
solving Ax = b for x.
Example Using Software Packages
A permutation matrix is an n×n matrix P that arises from the identity matrix by permuting its rows. It then turns out that permuting the rows of any n × n matrix A can be accomplished by multiplying A on the left by P. Every permutation matrix is invertible, since the rows still form a basis for Rn . When Gaussian elimination with row pivoting is performed on a matrix A, the result is expressible as
P A = LU
where L is lower triangular and U is upper triangular. The matrix P A is A with its rows rearranged.
If we have the LU factorization of P A, how do we solve the system Ax = b? First, write it as
P Ax = Pb
then LUx = Pb. Let y = Ux, so that our problem is now
􏰈Ly=Pb Ux=y
The first equation is easily solved for y, and then the second equation is easily solved for x. Mathematical software systems such as MATLAB, Maple, and Mathematica produce factorizations of the form P A = LU upon command.
Caution
Solving Ax = b Using PA = LU
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

Maple
A=LU=2 1 0 00 −4 2 2
1 3 1 00 0 2−5
MATLAB
0 0 1.0000 0 
0.0909 1.0000
Mathematica
1000
where P is a permutation matrix corresponding to the pivoting strategy used. Finally, we
use Mathematica to create this LU decomposition: 3−139 3
 −2 −22 19 −12  2−12 52 −166
11 11 11
4 −2 22 −6 13 13
The output is in a compact store scheme that contains both the lower triangular matrix and the upper triangular matrix in a single matrix. However, the storage arrangement may be complicated because the rows are usually permuted during the factorization in an effort to make the solution process numerically stable. Verify that this factorization corresponds to the permutation of rows of matrix A in the order 3, 4, 1, 2. ■
Summary 8.1
• If A = (ai j ) is an n × n matrix such that the forward elimination phase of the naive Gaussian algorithm can be applied to A without encountering any zero divisors, then
EXAMPLE 3
Solution
8.1 Matrix Factorizations 373 Use the mathematical software systems in Maple, MATLAB, and Mathematica to find the
LU factorization of this matrix:
 6 −2 A=12 −8 3−13 −6 4
First, we use Maple and find this factorization: 1 0 0
2 4 6 10
9 3 1 −18
(24)
0 0 0 0.2727
0100 P=0 0 1 0 0001
06−2
2 4 −1−1 21000−3
2
2
Next, we use MATLAB and find a different factorization:
P A = L􏰛 U􏰛
 1.0000 0 0 0
L􏰛 =  0.2500 1.0000  −0.5000 0 0.5000 −0.1818
 12.0000 −8.0000 6.0000 10.0000  U􏰛= 0 −11.0000 7.5000 0.5000  0 0 4.0000 −13.0000 
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

374 Chapter 8
More on Linear Systems
the resulting matrix can be denoted by 􏰮A = (􏰮ai j ), where
100···0
􏰮a21 1 0 ··· 0 L =  􏰮a31 􏰮a32 1 · · · 0 
. . …… . 􏰮an1 􏰮an2 · · · 􏰮an,n−1 1
􏰮a11 􏰮a12 􏰮a13 ··· 􏰮a1n 
0 􏰮a22 􏰮a23 ··· 􏰮a2n  U=0 0 􏰮a33 ···􏰮a3n
. . … … .  0 0 ··· 0 􏰮ann
This is the LU factorization of A, so A = LU, where L is a unit lower triangular and U is upper triangular. When we carry out the Gaussian forward elimination process on A, the result is the upper triangular matrix U. The matrix L is the unit lower triangular matrix whose entries are negatives of the multipliers in the locations of the elements
they zero out.
• We can also give a formal description as follows. The matrix U can be obtained by applying a succession of matrices Mk to the left of A. The kth step of Gaussian elimi- nation corresponds to a unit lower triangular matrix M k , which is the product of n − k elementary matrices
Mk =Mnk Mn−1,k···Mk+1,k
where each elementary matrix Mik is unit lower triangular. If Mqp A is obtained by subtracting λ times row p from row q in matrix A with p < q, then the elements of Mqp =(mij)are  1, ifi=j mij= −λ, ifi=qandj=p 0, otherwise The entire Gaussian elimination process is summarized by writing Mn−1 · · · M2 M1 A = U Since each Mk is invertible, we have A=M−1M−1···M−1 U 1 2 n−1 Each M is a unit lower triangular matrix, and the same is true of each inverse M−1, as kk well as their products. Hence, the matrix L = M−1 M−1 · · · M−1 is unit lower triangular. • For symmetric matrices, we have the L D L T factorization, and for symmetric positive definite matrices, we have the LLT factorization, which is also known as Cholesky factorization. • If the LU factorization of A is available, we can solve the system Ax = b Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. and 1 2 n−1 bysolving 1. Using naive Gaussian elimination, factor the following matrices in the form A = LU, where L is a unit lower triangular matrix and U is an upper triangular matrix. 􏰙303􏰚 aa.A=0−1 3 130 􏰈Ly=Pb fory Ux = y for x 1010 3 b.A=0 1 3−1 3 −3 0 6 0 2 4 −6 8.1 Matrix Factorizations 375 by solving two triangular systems:􏰈 L y = b for y Ux=y forx This is useful for problems that involve the same coefficient matrix A and many different right-hand vectors b. For example, let B be an n × m matrix of the form B = [b(1), b(2), . . . , b(m)] where each column corresponds to a right-hand side of the m linear systems Ax(j) =b(j) (1≦ j≦m) Thus, we can write or A􏰗x(1), x(2),..., x(m)􏰘 = 􏰗b(1), b(2),..., b(m)􏰘 AX = B A special case of this is to compute the inverse of an n × n invertible matrix A. We write AX = I where I is the identity matrix. If x(j) denotes the jth column of X and I(j) denotes the jth column of I, this can be written as A􏰗x(1), x(2),..., x(n)􏰘 = 􏰗I(1), I(2),..., I(n)􏰘 or as n linear systems of equations of the form Ax(j) = I(j) (1≦ j≦n) We can use LU factorization to solve these n systems efficiently, obtaining A−1 =􏰗x(1),x(2),...,x(n)􏰘 • When Gaussian elimination with row pivoting is performed on a matrix A, the result is expressible as P A = LU where P is a permutation matrix, L is unit lower triangular, and U is upper triangular. Here, the matrix P A is A with its rows interchanged. We can solve the system Ax = b Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. Exercises 8.1 376 Chapter 8 More on Linear Systems 2. Consider the matrix 1002 −20 −15 −10 −5 c.A=1 0 0 0 aa. ab. a c. A = LU, where L is unit lower triangular and U is upper triangular. A = LDU′, where L is unit lower triangular, D is diagonal, and U′ is unit upper triangular. A = L′U′, where L′ is lower triangular and U′ is unit upper triangular. A = (L′′)(L′′)T , where L′′ is lower triangular. Evaluate the determinant of A. Hint: det(A) = det(L)det(D)det(U′) = det(D). 0100 0010 0300 ad. A=0940 ae. 5 0 8 10 aa. Determine a unit lower triangular matrix M and an upper triangular matrix U such that MA = U. b. Determine a unit lower triangular matrix L and an up- per triangular matrix U such that A = L U . Show that ML = I so that L = M−1. Consider the matrix  25 0 0 0 1  0 27 4 3 2 A =  0 54 58 0 0  0 108 116 0 0 100 0 0 0 24 aa. Determine the unit lower triangular matrix M and the upper triangular matrix U such that MA = U. 7. a8. 9. a10. 0b00  Consider the 3 × 3 Hilbert matrix 1 1 1  2 3 A=1 1 1 Repeat the preceding problem using this matrix. 2 3 4 1 1 1 345 3. 4. 5. 6. Find the L U gular, for b. Determine M−1 = L such that A = LU. Considerthematrix 􏰙 2 −1 6 −1 8 Consider 􏰙􏰚 A=2−33 2 2 1 A=111 321 a. Show that A cannot be factored into the product of a unit lower triangular matrix and an upper triangular matrix. a ab. InterchangetherowsofAsothatthiscanbedone. Considerthematrix  a 0 0 z Repeattheprecedingproblemfor L is unit lower trian- 1 − 1  1 −1 2􏰚 aa. Find the matrix factorization A = L DU′, where L is unit lower triangular, D is diagonal, and U′ is unit upper triangular. ab. Use this decomposition of A to solve Ax = b, where b = [−2,−5,0]T . 224 4 w0yd 11. Consider the system of equations A=0 x c 0  Determine a unit lower triangular matrix M and an 1 6x =12 upper triangular matrix U such that MA = U. 6x2 + 3x1 = −12 a. ab. Determine a lower triangular matrix L′ and a unit up-  7x3 − 2x2 + 4x1 =14 21x4 +9x3 −3x2 +5x1 =−2 Solve for x1, x2, x3, and x4 (in order) by forward sub- stitution. Write this system in matrix notation Ax = b, where x = [x1, x2, x3, x4]T . Determine the LU factoriza- tion A = L U , where L is unit lower triangular and U is upper triangular. per triangular matrix U′ such that A = L′U′. Consider the matrix a. b. 4 A =  − 1 −1 −1 0 4 0 −1 0 4 −1 −1 −14 Factor A in the following ways: −1 0 Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole deemed that any suppressed content does not materially affect the overall learning experience. or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. decomposition, where 10 A=1 1 0 0 1 1 −1 1 1 −1 􏰙−21−2􏰚 􏰙1􏰚 A=−4 3−3, b=4 a 12. Given 􏰙 􏰚 8.1 Matrix Factorizations 377 solution matrix X can be found by Gaussian elimination with scaled partial pivoting in 4 n3 + O(n2) multiplica- 3 tions and divisions. Hint: If X(j) and B(j) are the jth columns of X and B, respectively, then AX( j) = B( j). 3 2−1 A=532 −1 1−3  1 0 0 3 2−1 L−1=−5 1 0, U=0−1 11 33316. −8 5 1 0 0 15 Let X be a square matrix that has the form 􏰆A B􏰇 13. 14. obtain the inverse of A by solving U X( j) = L−1 I( j) for j = 1, 2, 3. Using the system of Equation (2), form M = M3 M2 M1 and determine M−1. Verify that M−1 = L. Why is this, in general, not a good idea? Consider the matrix A = Tridiagonal (ai ,i −1 , ai i , ai ,i +1 ), whereaii ≠ 0.Hereisthe4×4case. aa. Establish the algorithm X=CD where A and D are square matrices and A−1 exists. It is integer i real array (ai j )1:n×1:n , (li j )1:n×1:n , (ui j )1:n×1:n l11 ← a11 fori =2to4 li,i−1 ← ai,i−1 ui−1,i ←ai−1,i/li−1,i−1 li,i ← ai,i − li,i−1ui−1,i end for for determining the elements of a lower bidiagonal matrix L = (li j ) and a unit upper bidiagonal matrix U = (uij) such that A = LU. b. Establishthealgorithm 17. 18. known that X−1 exists if and only if (D − C A−1 B)−1 exists. Verify that X −1 is given by 􏰆I −A−1B􏰇􏰆A−1 0 􏰇 X= 0 I 0 (D−CA−1B)−1 􏰆I0􏰇 × −CA−1 I As an application, compute the inverse of the following: 1001 aa.X=0 1 1 0 1012 0001 1001 a b . X =  0 1 0 1  0011 1112 Let A be an n × n complex matrix such that A−1 exists. Verify that 􏰆 A A 􏰇−1 1 􏰆 A −1 A −1i 􏰇 −Ai −Ai = 2 A−1 −A−1i where A denotes the complex conjugate of A; if A = (ai j ), then A = (ai j ). Recall that for a complex number z = a + bi, where a and b are real, and z = a − bi. FindtheLUfactorizationofthismatrix: 􏰙221􏰚 A=472 2 11 5 integer i; real array (ai j )1:n×1:n , (li, j )1:n×1:n , (ui, j )1:n×1:n u11 ← a11 fori =2to4 ui−1,i ← ai−1,i li,i−1 ← ai,i−1/ui−1,i−1 ui,i ← ai,i − li,i−1ui−1,i end for 15. for determining the elements of a unit lower bidiago- nal matrix L = (li j ) and an upper bidiagonal matrix U = (uij) such that A = LU. By extending the loops, we can generalize these algo- rithms to n × n tridiagonal matrices. Show that the equation Ax = B can be solved by Gaussian elimination with scaled partial pivoting in (n3/3) + mn2 + O(n2) multiplications and divisions, whereA,X,andBarematricesofordern×n,n×m, andn×m,respectively.Thus,ifBisn×n,thenthen×n 19. a. Provethattheproductoftwolowertriangularmatrices is lower triangular. b. Prove that the product of two unit lower triangular matrices is unit lower triangular. c. Provethattheinverseofaunitlowertriangularmatrix is unit lower triangular. d. By using the transpose operation, prove that all of the preceding results are true for upper triangular matrices. Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 378 Chapter 8 More on Linear Systems 20. Let L be lower triangular, U be upper triangular, and D be diagonal. a. IfLandUarebothunittriangularandLDUisdiag- onal, does it follow that L and U are diagonal? b. If L DU is nonsingular and diagonal, does it follow that L and U are diagonal? c. If L and U are both unit triangular and if L DU is diagonal, does it follow that L = U = I? 25. (Sparse Factorizations) Consider the following sparse symmetric matrices with the nonzero pattern shown where nonzero entries in the matrix are indicated by the × symbol and zero entries are a blank. Show the nonzero pattern in the matrix L for the Cholesky factorization by using the symbol + for the fill-in of a zero entry by a nonzero entry. ×××  × × × ×  ×× ×  × ×  a. A= × ×   × ×  × ×  ×××× ××× × ××  × ×× × ××   ×  b. A=×× × × × × × × × × ××× ××××× ×× × ×× ×  × × × ×   ×  c.A= ×  ×   × × ×× × ×××× ×××× and compute the first row in U by u1j = a1j (1≦ j≦n) l11 Now suppose that columns 1,2,...,k − 1 have been computed in L and that rows 1,2,...,k − 1 have been computed in U. At the kth step, specify either lkk or ukk, and compute the other such that k−1 􏰄 lkkukk =akk − lkmumk m=1 21. Determine matrix: the L D L T factorization for 1 2−11 A=2 3−43 −1 −4 −1 3 1330 the following ×× ××× ×× 22. FindtheCholeskyfactorizationof 􏰙4 610􏰚 A= 6 25 19 10 19 62 23. Consider the system 􏰆􏰇􏰆􏰇􏰆􏰇 A0x=b BCyd ×× × ×× Show how to solve the system more cheaply using the submatrices rather than the overall system. Give an esti- mate of the computational cost of both the new and old approaches. This problem illustrates solving a block lin- ear system with a special structure. factorization of the matrix −20 −143 73 −232 65 461 −232 856 24. Determine the L D L T  5 35 −20 65 A =  35 244 −143 461  ×× ×× ××× Can you find the Cholesky factorization? 1. Write and test a procedure for implementing the algo- rithms of Exercise 8.1.14. 2. The n × n factorization A = LU, where L = (lij) is lower triangular and U = (ui j ) is upper triangular, can be computed directly by the following algorithm (provided zero divisions are not encountered): Specify either l11 or u11 and compute the other such that l11u11 = a11. Compute the first column in L by li1 = ai1 (1≦i≦n) u11 Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole deemed that any suppressed content does not materially affect the overall learning experience. or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. Computer Exercises 8.1 Compute the kth column in L by 1􏰋􏰄k−1􏰌 􏰙024􏰚 23 lik = u aik − limumk (k≦i≦n) A= 0 0 8 , p3(x)=1+3x−3x +x kk m=1 000 and compute the kth row in U by 1􏰋􏰄k−1 􏰌 ukj = l akj − lkmumj , (k≦ j≦n) kk m=1 This algorithm is continued until all elements of U and L are completely determined. When lii = 1 (1 ≦ i ≦ n), this procedure is called the Doolittle factorization, and whenujj =1(1≦ j≦n),itisknownastheCroutfac- torization. Define the test matrix a Case 4. 2−1 0 0 A=−1 2−1 0 0 −1 2 −1 0 0 −1 2 p5(x) = 10 + x − 2x2 + 3x3 − 4x4 + 5x5 Case 5. −20 −15 −10 −5 A= 1 0 0 0 Case 3. 8.1 Matrix Factorizations 379 5765 0100 0010 A=7 10 8 7 p4(x) = 5 + 10x + 15x2 + 20x3 + x4 6 8 10 9 5 7 9 10 Using the algorithm above, compute and print factoriza- tions so that the diagonal entries of L and U are of the following forms: Case 6. 5765 A =  7 10 8 7  , 6 8 10 9 5 7 9 10 p4(x) = 1 − 100x + 146x2 − 35x3 + x4 Write and test a procedure for determining A−1 for a given square matrix A of order n. Your procedure should use procedures Gauss and Solve. Write and test a procedure to solve the system AX = B in which A, X, and B are matrices of order n ×n, n ×m, and n × m, respectively. Verify that the procedure works on several test cases, one of which has B = I so that the solution X is the inverse of A. Hint: See Exercise 8.1.15. Write and test a procedure for directly computing the in- verse of a tridiagonal matrix. Assume that pivoting is not necessary. (Continuation)Testtheprocedureoftheprecedingcom- puter problem on the symmetric tridiagonal matrix A of order 10: diag( L ) [1, 1, 1, 1] [?, ?, ?, ?] [1, ?, 1, ?] [?, 1, ?, 1] [?, ?, 7, 9] diag(U ) [?, ?, [1, 1, [?, 1, [1, ?, [3, 5, ?, ?] 1, 1] ?, 1] 1, ?] ?, ?] Doolittle Crout 4. 5. 6. 7. Here the question mark means that the entry is to be com- puted. Write code to check the results by multiplying L and U together. 3. Write procedure Poly(n, (ai j ), (ci ), k, (yi j )) for computing the n × n matrix pk(A) stored in array (yij): yk =pk(A)=c0I+c1A+c2A2+···+ckAk where A is an n × n matrix and pk is a kth-degree poly- nomial. Here (ci ) are real constants for 0 ≦ i ≦ k. Use nested multiplication and write efficient code. Test pro- cedure Poly on the following data: Case 1. A=I5, p3(x)=1−5x+10x3 Case 2. A=􏰆1 2􏰇, 3 4 −2 1  1−2 1  1−21  p2(x)=1−2x+x2 A =  ... ... ...  The inverse of this matrix is known to be (A−1)ij =(A−1)ji = −i(n+1− j) (i≦ j) (n+1)  1−2 1 1 −2 Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 380 Chapter 8 More on Linear Systems 8. 9. Investigatethenumericaldifficultiesininvertingthefol- lowing matrix:  −0.0001 11. 12. 13. 14. Deviseacodeforinvertingaunitlowertriangularmatrix. Test it on the following matrix: 100 0 310 0 521 0 7 4 −3 1 Verify Example 1 using MATLAB, Maple, or Mathe- matica. In Example 3, verify the factorizations of matrix A using MATLAB, Maple, and Mathematica. Find the PA = LU factorization of this matrix: 3.392  0. 0. 4.567 Considerthefollowingtwotestmatrices: A= 􏰙 4 6 10􏰚 6 25 19 , 10 19 62 􏰙 4 6 10􏰚 B= 6 13 19 10 19 62 A =  0. 0. 5.096 5.101 1.853  3.737 3.740 0. 0. 0.006 5.254 Show that the first Cholesky factorization has all integers in the solution, while the second one is all integers until the last step, where there is a square root. a. ProgramtheCholeskyalgorithm. b. Use MATLAB, Maple, or Mathematica to find the Cholesky factorizations.  −0.05811 −0.11696 0.56850 0.38953 0.32179 0.51004 −0.31330  0.07041 0.68747 0.01203 −0.52927  10. Let A be real, symmetric, and positive definite. Is the same true for the matrix obtained by removing the first row and column of A? −0.22094 0.42448 which was studied by Wilkinson [1965, p. 640]. 8.2 Eigenvalues and Eigenvectors Av Scalar Multiple of v EXAMPLE 1 Solution Let A be an n × n matrix. We ask the following natural question about A: Are there nonzero vectors v for which Av is a scalar multiple of v? Although we pose this question in the spirit of pure curiosity, there are many situations in scientific computation in which this question arises. The answer to our question is a qualified Yes! We must be willing to consider complex scalars, as well as vectors with complex components. With that broadening of our viewpoint, such vectors always exist. Here are two examples. In the first, we need not bring in complex numbers to illustrate the situation, whereas in the second, the vectors and scalar factors must be complex. Let A = 􏰆3 2􏰇. Find a nonzero vector v for which Av is a multiple of v. 7 −2 You can easily verify that −7 28 −7 We have two different answers (but we have not revealed how to find them). ■ −0.04291 A =  −0.01652 −0.06140 A􏰆1􏰇 = 􏰆5􏰇 = 5􏰆1􏰇 A􏰆 2􏰇=􏰆−8􏰇=−4􏰆 2􏰇 Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. Eigenvalues Eigenvectors Eigenspace Characteristic Polynomial When the equation Ax = λx is valid and x is not zero, we say that λ is an eigenvalue of A and x is an accompanying eigenvector. Thus, in Example 1, the matrix has 5 as an eigenvalue with accompanying eigenvector [1, 1]T , and −4 is another eigenvalue with accompanying eigenvector [2, −7]T . Example 2 emphasizes that a real matrix may have complex eigenvalues and complex eigenvectors. Notice that an equation A0 = λ0 and an equation A0 = 0x say nothing useful about eigenvalues and eigenvectors of A. Many problems in science lead to eigenvalue problems in which the principal question usually is: What are the eigenvalues of a given matrix, and what are the accompanying eigenvectors? An outstanding application of eigenvalues and eigenvectors is to systems of linear differential equations, which we discuss later. Notice that if Ax = λx and x ≠ 0, then every nonzero multiple of x is an eigenvector (withthesameeigenvalue).Ifλisaneigenvalueofann×nmatrix A,thentheset{x: Ax = λx} is a subspace of Rn called an eigenspace. It is necessarily of dimension at least 1. Calculating Eigenvalues and Eigenvectors Given a square matrix A, how does one discover its eigenvalues? Begin by observing that the equation Ax = λx is equivalent to (A − λI)x = 0. Since we are interested in nonzero solutions to this equation, the matrix A − λ I must be noninvertible, and therefore Det( A−λ I ) = 0. This is how (in principle) we can find all the eigenvalues of A. Specifically, form the function p defined by p(λ) = Det(A − λI) and find the zeros of p. It turns out that p is a polynomial of degree n and must have n zeros, provided that we allow complex zeros and count each zero a number of times equal to its multiplicity. Even if the matrix A is real, we must be prepared for complex eigenvalues. The polynomial just described is called the characteristic polynomial of the matrix A. If this polynomial has a repeated factor, such as (λ − 3)k , then we say that 3 is a root of multiplicity k. Such roots are still eigenvalues, but they can be troublesome when k > 1.
To illustrate the calculation of eigenvalues, let us use the matrix in Example 1, namely,
A=􏰆3 2􏰇 7 −2
The characteristic polynomial is 􏰆 3 − λ 2 􏰇
p(λ) = Det(A − λI) = Det 7 −2 − λ = (3 − λ)(−2 − λ) − 14
=λ2 −λ−20=(λ−5)(λ+4) The eigenvalues are 5 and −4.
EXAMPLE 2
Repeat the preceding example with the matrix A = 􏰆 1 1 􏰇. −2 3
Eigenvalues Calculation
Solution
As in Example 1, it can be verified that
A􏰆 1 􏰇=(2+i)􏰆 1 􏰇
√
1+i 1+i A􏰆 1 􏰇=(2−i)􏰆 1 􏰇
8.2 Eigenvalues and Eigenvectors 381
1−i 1−i
−1. Surprisingly, we find answers involving complex numbers
In these equations, i =
even though the matrix does not contain any complex entries! ■
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

382 Chapter 8 More on Linear Systems
Mathematical Software
We can carry out this calculation with one or two commands in MATLAB, Maple, or Mathematica. We can determine the characteristic polynomial and subsequently compute its zeros. This gives us the two roots of the characteristic polynomial, which are the eigenvalues 5 and −4. These mathematical software systems also have single commands to produce a list of eigenvalues, computed in the best possible way, which is usually not to determine the characteristic polynomial and subsequently compute its zeros!
In general, an n × n matrix has a characteristic polynomial of degree n, and its roots are the eigenvalues of A. Since the calculation of zeros of a polynomial is numerically challenging if not unstable, this straightforward procedure is not recommended. (See Com- puter Exercise 8.2.2 for an experiment pertaining to this situation.) For small values of n, it may be quite satisfactory, however. It is called the direct method for computing eigenvalues.
Once an eigenvalue λ has been determined for a matrix A, an eigenvector can be computed by solving the system (A − λI)x = 0. Thus, in Example 1, we must solve (A − 5I)x = 0, or
􏰆 −2 2 􏰇 􏰆 x1 􏰇 = 􏰆 0 􏰇 7−7×2 0
Of course, this matrix is singular, and the homogeneous equation has nontrivial solutions, such as [1, 1]T . The other eigenvalue is treated in the same way, leading to an eigenvector [2, −7]T . Any scalar multiple of an eigenvector is also an eigenvector.
This work can be done by using mathematical software to find an eigenvector for each eigenvalue λ via the null space of the matrix A − λI. Also, we can use a single command to compute all the eigenvalues directly or request the calculation of all the eigenvalues and eigenvectors at once. The MATLAB command [V,D] = eig(A) produces two arrays, V and D. The array V has eigenvectors of A as its columns, and the array D contains all the eigenvalues of A on its diagonal. The program returns a vector of unit length such as [0.7071, 0.7071]T . That vector by itself provides a basis for the null space of A − 5 I .
(Maple and Mathematica have commands for computing eigenvalues and eigenvectors.) Notice that the eigenvalue-eigenvector problem is nonlinear. The equation
Ax = λx
has two unknowns, λ and x. They appear in the equation multiplied together. If either x or
λ were known, finding the other would be a linear problem and very easy. Mathematical Software
A typical, mundane use of mathematical software such as MATLAB might be to compute the eigenvalues and eigenvalues of this matrix
Eigenvector Calculation
Using Software
Using MATLAB
1 3−7
A =  −3 4 1  (1)
2 −5 3
with a command such as
[V,D] = eig(A)
MATLAB responds instantly with the eigenvectors in the array V and the eigenvalues in the
diagonal array D. The real eigenvalue is 0.0214, and the complex pair of eigenvalues are
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

LAPACK
3.9893 ± 5.5601i . Behind the scenes, much complicated computing may be taking place! The general procedure has these components: First, by means of similarity transforma- tions, A is put into lower Hessenberg form. This means that all elements below the first subdiagonal are zero. Thus, the new matrix A = (aij) satisfies aij = 0 when i > j + 1. Similarity transformations ensure that the eigenvalues are not disturbed. If A is real, fur- ther similarity transformations put A into a near-diagonal form in which each diagonal element is either a single real number or a 2 × 2 real matrix whose eigenvalues are a pair of conjugate complex numbers. Creating the additional zeros just below the diagonal re- quires some iterative process, because after all, we are in effect computing the zeros of a polynomial. The iterative process is reminiscent of the power method that is described in Section 8.3.
Maple can be used to compute the eigenvalues and eigenvectors. The quantities are computed in exact arithmetic and then converted to floating-point. In some versions of MATLAB, symbolic computations are available. In Mathematica, we can use either numer- ical or symbolical commands to obtain similar results.
The best advice for anyone who is confronted with challenging eigenvalue problems is to use the software in the package LAPACK. Special eigenvalue algorithms for vari- ous types of matrices are available there. For example, if the matrix in question is real and symmetric, use an algorithm tailored for that case. There are about a dozen cate- gories available to choose from in LAPACK. MATLAB itself employs some of the routines in LAPACK.
Properties of Eigenvalues
A theorem that summarizes the special properties of a matrix that impinge on the computing of its eigenvalues follows.
Symmetric Hermitian Conjugate Transpose
Positive Definite
Recall that a matrix A is symmetric if A = AT , where AT = (aji ) is the transpose of A = (aij). On the other hand, a complex matrix A is Hermitian if A = A∗, where A∗ = AT = (a j i ). Here A∗ is the conjugate transpose of the matrix A. Using the syntax of programming, we can write AT (i, j) = A( j, i) and A∗(i, j) = A( j, i). Recall also that
A is positive definite if xT Ax > 0 for all nonzero vectors x.
8.2 Eigenvalues and Eigenvectors 383
Matrix Eigenvalue Properties
The following statements are true for any square matrix A:
1. IfλisaneigenvalueofA,thenp(λ)isaneigenvalueofp(A),foranypolynomial
p. In particular, λk is an eigenvalue of Ak .
2. If A is invertible and λ is an eigenvalue of A, then p(1/λ) is an eigenvalue of
p(A−1), for any polynomial p. In particular, λ−1 is an eigenvalue of A−1.
3. If A is real and symmetric, then its eigenvalues are real.
4. If A is complex and Hermitian, then its eigenvalues are real.
5. If A is Hermitian and positive definite, then its eigenvalues are positive.
6. If P is invertible, then A and P A P −1 have the same characteristic polynomial (and the same eigenvalues).
■ Theorem1
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

384 Chapter 8
More on Linear Systems
Similar Matrices
■ Theorem2
Two matrices A and B are similar to each other if there exists an invertible matrix P such that B = P AP−1. Similar matrices have the same characteristic polynomial
Det(B − λI) = Det(P AP−1 − λI) = Det(P(A − λI)P−1)
= Det( P ) · Det( A − λ I ) · Det( P −1 ) = Det( A − λ I )
Thus, we have this important theorem.
This theorem suggests a strategy for finding eigenvalues of A. Transform the matrix A to a matrix B using a similarity transformation
B = P AP−1
in which B has a special structure, and then find the eigenvalues of matrix B. Specifically, if B is triangular or diagonal, the eigenvalues of B (and those of A) are simply the diagonal elements of B.
Matrices A and B are said to be unitarily similar to each other if B = U∗ AU for some unitary matrix U . Recall that a matrix U is unitary if U U ∗ = I . This brings us naturally to another important theorem and two corollaries.
In this theorem, an arbitrary complex n × n matrix A is given, and the assertion made is that a unitary matrix U exists such that:
UAU∗ =T (2)
where UU∗ = I and T is a triangular matrix.
The proof of Schur’s Theorem can be found in Kincaid and Cheney [2002] and Golub
and Van Loan [1996].
Thus, the factorization
PAP−1 =T
is possible, where T is triangular, P is invertible, and A is real.
Eigenvalues of Similar Matrices
Similar matrices have the same eigenvalues.
Unitarily Similar Unitary
■ Theorem3
Schur’s Theorem
Every square matrix is unitarily similar to a triangular matrix.
Matrix Similar to a Triangular Matrix
Every square real matrix is similar to a triangular matrix.
■ Corollary1
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

EXAMPLE 3
Solution
8.2 Eigenvalues and Eigenvectors 385 We illustrate Schur’s Theorem by finding the decomposition of this 2 × 2 matrix:
A=􏰆3 −2􏰇 83
From the characteristic equation
Det(A−λI)=λ2 −6λ+25=0
the eigenvalues are 3 ± 4i. By solving A − λI = 0 with each of these eigenvalues, the corresponding eigenvectors are v1 = [i, 2]T and v2 = [−i, 2]T . Using the Gram-Schmidt orthogonalization process, we obtain u1 = v1 and u2 = v2 −[v∗2u1/u∗1u1]u1 = [−2,−i]T . After normalizing these vectors, we obtain the unitary matrix
1 􏰆i −2􏰇 U=√5 2 −i
which satisfies the property UU∗ = I, Finally, we obtain the Schur form UAU∗=􏰆3+4i −6 􏰇
0 3−4i
which is an upper triangular matrix with the eigenvalues on the diagonal. ■
Corollary 2 says that a Hermitian matrix A = A∗ can be factored as UAU∗ = D
where D is diagonal and U is unitary.
This follows from Corollary 1 since a Hermitian matrix (A = A∗) is unitarily similar
to a triangular metrix T
UAU∗ =T
where UU∗ = I. Furthermore, we have
U∗ A∗U = T∗
Since A = A∗, we obtain T = T∗, which must be a diagonal matrix.
Most numerical methods for finding eigenvalues of an n × n matrix A proceed by determining such similarity transformations. Then one eigenvalue at a time, say, λ, is com- puted, and a deflation process is used to produce an (n − 1) × (n − 1) matrix 􏰮A whose eigenvalues are the same as those of A, except for λ. Any such procedure can be repeated with the matrix 􏰮A to find as many eigenvalues of the matrix A as desired. In practice, this strategy must be used cautiously because the successive eigenvalues may be infected with
roundoff error!
Gershgorin’s Theorem
Sometimes it is necessary to determine in a coarse manner where the eigenvalues of a matrix are situated in the complex plane C. The most famous of these so-called localization theorems is the following.
Hermitian Matrix Unitarily Similar to a Diagonal Matrix
Every square Hermitian matrix is unitarily similar to a diagonal matrix.
Deflation Process
■ Corollary2
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

386 Chapter 8 More on Linear Systems
Gershgorin’s Theorem
All eigenvalues of an n × n matrix A = (ai i ) are contained in the union of the n discs Ci =Ci(aii,ri)inthecomplexplanewithcenteraii andradiiri givenbythesumof the magnitudes of the off-diagonal entries in the ith row.
■ Theorem4
The matrix A can have either real or complex entires. The region containing the eigenvalues of A can be written
More Gershgorin Discs
All eigenvalues of an n × n matrix A = (ai i ) are contained in the union of the n discs Di = Di (aii , si ) in the complex plane having center at aii and radii si given by the sum of the magnitudes of the columns of A.
■ Corollary3
TheeigenvaluesofAandAT arethesamebecausethecharacteristicequationinvolves the determinant, which is the same for a matrix and its transpose. Therefore, we can apply theGershgorinTheoremtoAT andobtainthefollowingusefulresult.
Consequently, the region containing the eigenvalues of A can be written as
wheretheradiiareri =
􏰃􏰿n 􏰿n􏰟 􏰠
Ci = z∈C:|z−aii|≦ri (3)
i=1 i=1 nj=1 |aij|.
j ≠ i
where the radii are si =
i=1
􏰍 􏰿n
􏰃􏰿n 􏰿n􏰟 􏰠
Di = z∈C:|z−aii|≦si (4)
i=1
ni =1 |ai j |. Finally, the region containing the eigenvalues of A is
i̸=j
􏰎 􏱀 􏰍 􏰿n
􏰎
Di (5) This may result in tighter bounds on the eigenvalues in some case. Also, a useful localization
Ci
Localization
For a matrix A, the union of any k Gershgorin discs that do not intersect the remaining n − k circles contains exactly k (counting multiplicities) of the eigenvalues of A.
■ Corollary4
■ Corollary5
result is
For a strictly diagonally dominant matrix, zero cannot lie in any of its Gershgorin discs, so it must be invertible. Consequently, we obtain the following results.
i=1
i=1
Strictly Diagonally Dominant Matrix
Every strictly diagonally dominant matrix is invertible.
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

EXAMPLE 4
Solution
Consider the matrix
Draw the Gershgorin discs.
4−i 2 i A= −1 2i 2
Im(z) 6
4 2
C2, D2 D3 *
8.2 Eigenvalues and Eigenvectors 387
1 −1 −5
Using the rows of A, we find that the Gershgorin discs are C1(4 − i, 3), C2(2i, 3), and C3(−5, 2). By using the columns of A, we obtain more Gershgorin discs: D1(4 − i, 2), D2(2i, 3), and D3(−5, 3). Consequently, all the eigenvalues of A are in the three discs D1, C2, and C3, as shown in Figure 8.1. By other means, we compute the eigenvalues of A as λ1 = 3.7208 − 1.05461i, λ2 = 4.5602 + −0.2849i, and λ3 = −0.1605 + 2.3395i. In Figure 8.1, the center of the discs are designated by dots • and the eigenvalues by ∗. ■
FIGURE 8.1
Gershorgin discs
C3 C1 0* D1
*
−2 −4
−6 −4 −2 0 2 4 6 Singular Value Decomposition
Re(z)
■ Theorem5
This subsection requires some further knowledge of linear algebra, in particular the di- agonalization of symmetric matrices, eigenvalues, eigenvectors, rank, column space, and norms. See online, at the textbook Web site, Appendix D2 for a brief reveiw of these topics. (In the discussion following, we assume that the Euclidean norm is being used.)
The singular value decomposition is a general-purpose tool that has many uses, par- ticularly in least-squares problems (Chapter 9). It can be applied to any matrix, whether square or not. We begin by stating that the singular values of a matrix A are the nonnegative square roots of the eigenvalues of AT A.
Matrix Spectral Theorem
Let A be m × n. Then AT A is an n × n symmetric matrix, and it can be diagonalized by an orthogonal matrix, say, Q:
AT A= QDQ−1 whereQQT =QTQ=IandDisadiagonaln×nmatrix.
Furthermore, the diagonal matrix D contains the eigenvalues of AT A on its diagonal. This follows from the fact that AT A Q = Q D, so the columns of Q are eigenvectors of AT A.
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

388 Chapter 8
More on Linear Systems
Singular Values
If λ is an eigenvalue of AT A and if x is a corresponding eigenvector, then AT Ax = λx, whence
||Ax||2 = (Ax)T (Ax) = xT AT Ax = xT λx = λ||x||2
This equation shows that λ is real and nonnegative. We can order the eigenvalues as λ1 ≧ λ2 ≧ · · · ≧ λn ≧ 0􏰓. (Reordering the eigenvalues requires reordering the columns of Q.) The numbers σj = + λj are the singular values of A.
Since Q is an orthogonal matrix, its columns form an orthonormal base for Rn . They are unit eigenvectors of AT A, so if vj is the jth column of Q, then AT Avj = λjvj. Some of the eigenvalues of AT A can be zero. Define r by the condition
λ1≧λ2≧ ···≧λr >0=λr+1 =···=λn
For a review of concepts such as rank, orthogonal basis, orthonormal basis, column
space, null space, and so on, see Appendix D.2 on the textbook Web site.
Observe that
This establishes the orthogonality of the set {Avj:1≦ j≦n}. By letting k = j, we get
Orthogonal Basis Theorem
Iftherankof Aisr,thenanorthogonalbasisforthecolumnspaceof Ais{Avj:1≦ j ≦r}.
Rank
Singular Value Decomposition
The preceding theorem gives a reasonable way of computing the rank of a numerical matrix. First, compute its singular values. Any that are very small can be assumed to be zero. The remaining ones are strongly positive, and if there are r of them, we take r to be the numerically computed rank of A.
A singular value decomposition of an m × n matrix A is any representation of A in the form
A = U DV T (6)
where U and V are orthogonal matrices and D is an m × n diagonal matrix having non- negative diagonal entries that are ordered d11 ≧ d22 ≧ · · · ≧ 0. Then from Exercise 8.2.4, it follows that the diagonal elements dii are necessarily the singular values of A. Note that the matrix U is m × m and V is n × n. A nonsquare matrix D is diagonal if the only elements that are not zero are among those whose two indices are equal.
Now a singular value decomposition of A (there are many of them) can be described. Start with the vectors v1, v2, . . . , vr . Normalize the vectors Av j to get vectors u j . Thus, we have
uj = Avj/||Avj|| (1≦ j≦r)
■ Theorem6
Proof
(Avk)T (Avj) = vkT AT Avj = vkT λjvj = λjδkj
||Avj|| = λj. Hence, Avj ≠ 0 if and only if 1≦ j ≦r. If w is any vector in the column
2􏰃
space of A, then w = Ax for some x in Rn. Putting x = nj=1 cjvj, we get
􏰄n w=Ax=
j=1
􏰄r j=1
cjAvj
cjAvj = and therefore, w is in the span of {Av1, Av2,…, Avr}.
■
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

Verifying A=UDVT
Condition Number
Extend this set to an orthonormal base for Rm . Let U be the m ×m matrix whose columns are u1,u2,…,um.Define Dtobethem×nmatrixconsistingofzerosexceptforσ1,σ2,…,σr on its diagonal. Let V = Q, where Q is as discussed.
ToverifytheequationA=UDVT,firstnotethatσj =||Avj||2andthatσjuj =Avj. Then compute U D. Since D is diagonal, this is easy. We get
UD = [u1,u2,…,um]D = [σ1u1,σ2u2,…,σrur,0,…,0] = [Av1, Av2,…, Avr,…, Avn] = AQ = AV
This implies that
A = U DV T
The condition number of a matrix can be expressed in terms of its singular values
κ(A) = 􏰕σmax (7) σmin
since || A||2 = ρ( AT A) = σmax( A) and || A−1||2 = ρ( A−T A−1|| = σmin( A). Numerical Examples of Singular Value Decomposition
The numerical determination of a singular value decomposition is best left to mathematical software such as MATLAB, Maple, Mathematica, LAPACK, and other software packages. Usually, they do not form AT A in the numerical work because its condition number may be much worse than that of A. This phenomenon is easily illustrated by the matrices
1 1 1 1+ε2 1 1  A=ε 0 0, AT A= 1 1+ε2 1 
EXAMPLE 5
Solution
1 1 1+ε2
There are small values of ε for which A has rank 3 and AT A has rank 1 (in the computer).
In Example 2 of Section 1.1 (p. 4), we encountered this matrix: A = 􏰆 0.1036 0.2122 􏰇
0.2081 0.4247 Determine its eigenvalues, singular values, and condition number.
By using mathematical software, it is easy to find the eigenvalues λ1(A) ≈ −0.0003 and λ2(A) ≈ 0.5286. We can form the matrix
AT A = 􏰆 0.0540 01104 􏰇 0.1104 0.2254
and find its eigenvalues λ1( AT A) ≈ 0.3025 × 10−4 and λ2( AT A) ≈ 0.2794. Therefore, the singular values are
σ1( A) = 􏰔|λ1( AT A)| ≈ 0.0003, σ2( A) = 􏰔|λ2( AT A)| ≈ 0.5286
Also, we can obtain the singular values directly as σ1 ≈ 0.0003 and σ2 ≈ 0.5286 using mathematical software. Consequently, the condition number is κ(A) = σ2/σ1 ≈ 1747.6. Because of this large condition number, we now understand why there was difficultly in solving the linear system of equations with this coefficient matrix! ■
0ε0 00ε
8.2 Eigenvalues and Eigenvectors 389
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

390 Chapter 8 More on Linear Systems
EXAMPLE 6
Solution
Calculate the singular value decomposition of the matrix
1 1
A=0 1 (8)
10
Here, the matrix A is m ×n with m = 3 and n = 2. First, we find that the eigenvalues of the matrix
ATA=􏰆2 1􏰇 12
in the same order as the eigenvalues to form the column vectors of the n × n matrix V :
arranged in descending order are λ1 = 3 and λ2 = 1. So there are 2 nonzero eigenvalues of the matrix AT A. Next, we determine that the eigenvectors of the matrix AT A are [1, 1]T for λ1 = 3 and [1, −1]T for λ2 = 1. Consequently, the orthonormal set of eigenvectors of
AT A are 􏰗1√2, 1√2􏰘T for λ1 =3 and 􏰗1√2,−1√2􏰘T for λ2 =1. Then we arrange them 22 22
􏰗 􏰘 􏰙1√2 1√2􏰚 V=v1v2=2√ 2√
√
3andσ2 =1onthe
Here, on the leading diagonal are the square roots of the eigenvalues of AT A in descending
order, and the rest of the entries of the matrix D are zeros. Next, we compute vectors
u =σ−1Av fori=1and2,whichformthecolumnvectorsofthem×mmatrixU.In iii
1 2 −1 2 22
Nowweformthem×nsingularvaluematrix Dbyplacingσ1 = leading diagonal
√3 √0 D=0 1
00
this case, we find
and
1 1􏰙√􏰚 1√6 1√ 12 3√
u=σ−1Av= 3012√ =1 6 1113126√
102 16 6
u
1 1􏰙 1√2􏰚  √0 = σ − 1 A v =  0 1  2 √ =  − 1 2 
222 −122√ 10212
Finally, we add to the matrix U the rest of the m − r vectors using the Gram-Schmidt orthogonalization process in Section 9.2 (p. 444). So we make the vector u3 perpendicular to u1 and u2:
1 􏰀􏰁􏰀􏰁3
2
u 3 = e 1 − u 1T e 1 u 1 − u 1T e 2 u 2 =  − 1  3
−1 3
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

Normalizing the vector u3, we get
 1√3
3√ 􏰮u3=−1 3
8.2 Eigenvalues and Eigenvectors 391
3√ −1 3
So we have the matrix
􏰗􏰘3√√3√ U = u 1 u 2 􏰮u 3 =  1 6 1 2 − 1 3 
 1 √6
0 1 √3 
3
6√2√3√ 1 6 −1 2 −1 3
623 The singular value decomposition of the matrix A is
A = UDVT
1 1 1√6 0 1√3√3 0􏰙 √ √ 􏰚
3√√3√√1212 0 1=1 6 1 2 −1 3 0 1 2√ 2√
6√ 2√ 3√ 12−12 1016−12−13002 2
623
So there we have it! Fortunately, there is mathematical software for doing all of this instantly! We can verify the results by computing the matrix A from the factorization on the right-hand side. ■
See Chapters 9 and 13 for some important applications of the singular value decompo- sition. Further examples are given there and in the problems of those chapters.
Application: Linear Differential Equations
The application of eigenvalue theory to systems of linear differential equations is briefly explained here. Let us start with a single linear differential equation with one dependent variable x. The independent variable is t and often represents time. We write x′ = ax, or in more detail (d/dt)x(t) = ax(t). There is a family of solutions, namely, x(t) = ceat, where c is an arbitrary real parameter. If an initial value x(0) is prescribed, we shall need parameter c to get the initial value right.
A pair of linear differential equations with two dependent variables, x1 and x2, looks
like this:
􏰈x1′ =a11x1+a12x2 x2′ =a21x1+a22x2
The general form of a system of n linear first-order differential equations, with constant coefficients, is simply
x′ = Ax
Here, A is an n × n numerical matrix, and the vector x has n components, x j , each being a function of t. Differentiation is with respect to t. To solve this, we are guided by the easy case of n = 1, discussed above. Here, we try
x(t) = eλtv
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
(9)

392 Chapter 8 More on Linear Systems
Linear Differential Equations
If λ is an eigenvalue of the matrix A and if v is an accompanying eigenvector, then one solution of the differential equation x′ = Ax is x(t) = eλt v.
■ Theorem7
where v is a constant vector. Taking the derivative of x, we have x′ = λeλtv. Now the system of equations has become λeλt v = Aeλt v, or λv = Av. This is how eigenvalues come into the process. We have proved the following result.
Application: A Vibration Problem
Eigenvalue-eigenvector analysis can be utilized for a variety of differential equations. Con- sider the system of two masses and three springs shown in Figure 8.2. Here, the masses are constrained to move only in the horizontal direction.
From this situation, we write the equations of motion in matrix-vector form:
􏰙x′′􏰚 􏰆 􏰇􏰆 􏰇
1 = −β α x1 , x′′=Ax (10) x′′ α −β x2
FIGURE 8.2
Two-mass vibration problem
2
By assuming that the solution is purely oscillatory (no damping), we have
In matrix form, we get
x = veiωt
􏰆x1 􏰇 = 􏰆v1 􏰇eiωt
x2 v2
x′′ = −ω2veiωt = −ω2x
􏰆 −β α 􏰇 x = −ω2 x α −β
By differentiation, we obtain and
This is the eigenvalue problem
where λ = −ω2. Eigenvalues can be found from the characteristic equation:
We find
Det(A+ω2I)=det􏰆ω2 −β α􏰇=0 α ω2−β
(ω2 −β)2 −α2 =ω4 −2βω2 +(β2 −α2)=0 ω2 = 1􏰜2β±􏰓4β2 −4(β2 −α2)􏰝=β±α
2
Ax = λx
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

8.2 Eigenvalues and Eigenvectors 393 For simplicity, we now assume unit masses and unit springs so that β = 2 and α = 1. Then
we obtain
A = 􏰆 −2 1 􏰇 1 −2
Thentherootsofthecharacteristicequationsareω12 =β+α=3andω2 =β−α=1. Next, we can find the eigenvectors. For the first eigenvalue, we obtain
􏰀A+ω12I􏰁v1=0 ⇒ 􏰆1 1􏰇􏰆v11􏰇=0 1 1 v12
Since v11 = −v12, we obtain the eigenvector􏰆 1 􏰇 v1 = −1
For the second eigenvector, we have 􏰆 􏰀A+ω2I􏰁v2 =0 ⇒
􏰇 􏰆
−1 1 v21 1 −1 v22
􏰇
=0
Since v21 = −v22, we obtain the eigenvector
v2 =􏰆1􏰇
The general solution for the equations of motion for the two-mass system is x(t) = c1v1eiω1t + c2v1e−iω1t + c3v2eiω2t + c4v2e−iω2t
Because the solution was for the square of the frequency, each frequency is used twice (one positive and one negative). We can use initial conditions to solve for the unknown coefficients.
Summary 8.2
• An eigenvalue λ and eigenvector x satisfy the equation Ax = λx. The direct method to compute the eigenvalues is to find the roots of the characteristic equation p(λ) = det( A − λ I ) = 0. Then, for each eigenvalue λ, the eigenvectors can be found by solving the homogeneous system ( A − λ I )x = 0. There are software packages for finding the eigenvalue-eigenvector pairs using more-sophisticated methods.
• There are many useful properties for matrices that influence their eigenvalues. For ex- ample, the eigenvalues are real when A is symmetric or Hermitian. The eigenvalues are positive when A is symmetric or Hermitian positive definite.
• Many eigenvalue procedures involve similarity or unitary transformations to produce triangular or diagonal matrices.
• Gershgorin’s discs can be used to localize the eigenvalues by finding coarse estimates of them.
• The singular value decomposition of an m × n matrix A is A = U DV T
where D is an m × n diagonal matrix whose diagonal entries are the singular values, U is an m × m orthogonal matrix, and V is an n × n orthogonal matrix. The singular values of A are the nonnegative square roots of the eigenvalues of AT A.
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

394
Chapter 8 More on Linear Systems
a1. 2.
a3.
4.
5.
6.
7.
Are [i, −1 + i]T and [−i, −1 − i]T eigenvectors of the matrix in Example 2? Show why or why not.
Provethatifλisaneigenvalueofarealmatrixwitheigen- vector x, then λ is also an eigenvalue with eigenvector x. (For a complex number z = x + iy, the conjugate is defined by z = x − iy.)
a9.
10.
c. p( A)x = p(λ)x for any polynomial p d. Akx=(1−λ)kx
(Multiple Choice) For what values of s will the matrix I − svv∗ be unitary, where v is a column vector of unit length?
a. 0,1 b. 0,2 c. 1,2 d. 0,√2 e. None of these.
(Multiple Choice) Let U and V be unitary n×n matrices, possibly complex. Which conclusion is not justified?
a. U + V is unitary. b. U∗ is unitary. c. UVisunitary.
d. U −vv∗ is unitary when ||v|| = √2 and v is a column vector.
Let
A = 􏰆cosθ sin θ
−sinθ 􏰇 cos θ
Account for the fact that the matrix A has the effect of rotating vectors counterclockwise through an angle θ and thus cannot map any vector into a multiple of itself.
Let Abeanm×nmatrixsuchthat A=UDVT,where U and V are orthogonal and D is diagonal and non- negative. Prove that the diagonal elements of D are the singular values of A.
Let A, U, D, and V be as in the singular value decom- position: A = U DV T . Let r be as described in the text. Define Ur to consist of the first r columns of U. Let Vr consist of the first r columns of V, and let Dr be the r × r matrix having the same diagonal as D. Prove that A = U r Dr V rT . (This factorization is called the econom- ical version of the singular value decomposition.)
AlinearmapPisaprojectionifP2=P.Wecanuse the same terminology for an n × n matrix: A2 = A is the projection property. Use the Pierce decomposition, I = A + (I − A), to show that every point in Rn is the sum of a vector in the range of A and a vector in the null space of A. What are the eigenvalues of a projection?
FindalloftheGershgorindiscsforthefollowingmatri- ces. Indicate the smallest region(s) containing all of the eigenvalues:
􏰙3−1 1􏰚 􏰙3 1 2􏰚 aa. 2 4 −2 b. −1 4 −1
e. Noneofthese.
a11. (MultipleChoice)Whichassertionistrue?
3 −1 9
􏰙1−i 1 i􏰚 ac. 0 2i 2 102
1 −2 9
8. (Multiple Choice) Let A be an n × n invertible (nonsin- gular) matrix. Let x be a nonzero vector. Suppose that Ax = λx. Which equation does not follow from these
hypotheses?
a. Akx=λkx b. λ−kx=(A−1)kxfork≧0
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole deemed that any suppressed content does not materially affect the overall learning experience.
􏰄n j=1
j ̸=i
or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
a. Every n × n matrix has n distinct (different) eigenvalues.
b. Theeigenvaluesofarealmatrixarereal.
c. IfUisaunitarymatrix,thenU∗=UT.
d. A square matrix and its transpose have the same eigenvalues.
e. Noneofthese.
(Multiple Choice) Consider the symmetric matrix
1 3 4−1 A=3 7−6 1
4 −6 3 0 −1 1 0 5
What is the smallest interval derived from Gershgorin’s Theorem such that all eigenvalues of the matrix A lie in that interval?
a. [−7,9] b. [−7,13] c. [3,7]
d. [−3, 17] e. None of these.
a13. (TrueorFalse)Gershgorin’sTheoremassertsthatevery eigenvalue λ of an n × n matrix A must satisfy one of these inequalities:
12.
|λ−aii|≦
|aij| for 1≦i≦n.
Exercises 8.2

14.
a 15.
16.
17.
1.
(True or False) A consequence of Schur’s Theorem is that every square matrix A can be factored as A = P T P −1 , where P is a nonsingular matrix and T is upper
triangular.
(True or False) A consequence of Schur’s Theorem is that every (real) symmetric matrix A can be factored in theformA=PDP−1,wherePisunitaryandDis diagonal.
18.
19. 20.
8.2 Eigenvalues and Eigenvectors 395 Plot the Gershgorin discs in the complex plane for A and
AT aswellasindicatethelocationsoftheeigenvalues.
(Continuation)LetBbethematrixobtainedbychanging the negative entries in A to positive numbers. Repeat the process for B.
􏰙4 0 −2􏰚 1 2 0 . 119
Explain why ||U B||2 = ||B||2 for any matrix B when UTU=I. 4 −1 0
(Continuation) Repeat for C = Find the Schur decomposition of
2 A=−2−4. Consider the matrix A = 3 5 −3 .
􏰆57􏰇
013 2
Use MATLAB, Maple, Mathematica, or other computer programs available to compute the eigenvalues and eigen- vectors of these matrices:
2.
3.
Use MATLAB to compute the eigenvalues of a random 100 × 100 matrix by direct use of the command eig and by use of the commands poly and roots. Use the timing functions to determine the CPU time for each.
Letpbethepolynomialofdegree20whoserootsarethe integers 1, 2, . . . , 20. Find the usual power form of this polynomial so that
p(t)=t20 +a19t19 +a18t18 +···+a0
Next, form the so-called companion matrix, which is 20 × 20 and has zeros in all positions except all 1’s on the superdiagonal and the coefficients −a0, −a1,…,−a19 as its bottom row. Find the eigenvalues of this matrix, and account for any difficulties encountered.
(Student Research Project) Investigate some modern methods for computing eigenvalues and eigenvectors. For the symmetric case, see the book by Parlett [1997]. Also, read the LAPACK User’s Guide. (See Anderson, et al. [1999].)
(Student Research Project) Experiment with the Cayley-Hamilton Theorem, which asserts that every square matrix satisfies its own characteristic equation. Check this numerically by using MATLAB or some other mathematical software system. Use matrices of size 3, 6, 9, 12, and 15, and account for any surprises. If you can use higher-precision arithmetic do so—MATLAB works with 15 digits of precision.
(Student Research Project) Experiment with the QR algorithm and the singular value decomposition of matrices—for example, using MATLAB. Try examples with four types of equations Ax = b—namely, (a) the
55
aa. A=􏰆1 7􏰇 2 −5
4−7
1 6 b. 5−5 9 −3 3 2
3 2 11 −1 −2 −4 1 6 5 −5
3 2
1  5
1
c. Letn=12,aij =i/jwheni≦j,andaij = j/i when i > j. Find the eigenvalues.
ad. Create an n × n matrix with a tridiagonal structure and nonzero elements (−1, 2, −1) in each row. For n = 5 and 20, find all of the eigenvalues, and verify that they are 2 − 2 cos( jπ/(n + 1)).
e. For any positive integer n, form the symmetric matrix A whose upper triangular part is given by
1 1
4.
5.
6.
n n−1 n−2 n−3 ··· 2
 n−1 n−2 n−3 ··· 2
1 . 
 n−2 n−3 ··· 2
 .. .  .··· .
.  1
 … 2  2
The eigenvalues of A are 1/{2−2 cos[(2i −1)π/(2n+ 1)]}. (See Frank [1958] and Gregory and Kar- ney [1969].) Numerically verify this result for n = 30.
1 1
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
Computer Exercises 8.2

396
Chapter 8 More on Linear Systems
7.
UsingmathematicalsoftwaresuchasMATLAB,Maple, or Mathematica on each of the following matrices, com- pute the eigenvalues via the characteristic polynomial, compute the eigenvectors via the null space of the matrix, and compute the eigenvalues and eigenvectors directly:
a 􏰆3 2􏰇 a 􏰙 1 3 −7􏰚 a.7−1 b.−341
2 −5 3
5411 a11. ConsiderA=4 5 1 1
Find the eigenvalues and accompanying eigenvectors of this matrix, from Gregory and Karney [1969], without using software.
Hint: The answers can be integers.
12. Findthesingularvaluedecompositionofthesematrices: 􏰗􏰘􏰆3􏰇
system has a unique solution; (b) the system has many solutions; (c) the system is inconsistent but has a unique least-squares solution; (d) the system is inconsistent and has many least-squares solutions.
Create the diagonal matrix D = U T A V to check the re- sults (always recommended). One can see the effects of roundoff errors in these calculations, for the off-diagonal elements in D are theoretically zero.
8. UsingmathematicalsoftwaresuchasMATLAB,Maple, or Mathematica, determine the execution time for com- puting all eigenvalues of a 1000 × 1000 matrix with ran- dom entries.
9. Using mathematical software such as MATLAB, Maple, or Mathematica, compute the Schur factorization of these complex matrices, and verify the results according to Schur’s Theorem and its corollaries:
􏰆􏰇􏰆􏰇
3−i 2−i 2+i 3+i a. 2+i 3+i b. 3−i 2−i
a. 2 1 −2 b. 4 c.􏰗−5+3√3 5√3+3􏰘
1142 1124
d.17 1 −17 −1 10 10 10 10 39−3−9 5555
10. Using mathematical software such as MATLAB, Maple, 66
or Mathematica, compute the singular value decomposi- tion of these matrices, and verify that each result satisfies theequation A=UDVT:
22 2222
 7 − 13√6 7 + 13√6 
􏰆
c. e.−7−13 6 −7+13 6 
􏰇
2−i2+i 26√26√
a.
8.3
Power Method
Mathematical Derivation
3−i 3+i 26√26√ −13 6 13 6
􏰙 1 1 􏰚  1 3 −2  0 1 b.  2 7 5  10 −2−34
5 −3 −2
−25
􏰙 −149 −50 537 180 −27 −9
−154 􏰚 546 .
13. Consider B =
Find the eigenvalues, singular values, and condition num-
ber of the matrix B.
Eigenvalues
A procedure called the power method can be employed to compute eigenvalues of a given matrix. It is an example of an iterative process that, under the right circumstances, produces a sequence converging to an eigenvalue of a given matrix.
Suppose that A is an n × n matrix, and that its eigenvalues (which we do not know) have the following property:
|λ1|>|λ2|≧|λ3|≧ ···≧|λn|
Notice the strict inequality in this hypothesis. Except for that, we are simply ordering the
eigenvalues according to decreasing absolute value. (This is only a matter of notation.) Each
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

Eigenvectors
Linear Combination
eigenvalue has a nonzero eigenvector u(i) and
Au(i) =λiu(i) (i =1,2,…,n)
(1)
Sequence of Vectors
.
 x(k) = Ax(k−1) = Ak x(0)
We assume that there is a linearly independent set of n eigenvectors {u(1), u(2), . . . , u(n)}. It is necessarily a basis for Cn .
We want to compute the single eigenvalue of maximum modulus (the dominant eigen- value) and an associated eigenvector. We select an arbitrary starting vector, x(0) ∈ Cn, and express it as a linear combination of the eigenvectors u(1), u(2), . . . , u(n):
x(0) = c1u(1) +c2u(2) +···+cnu(n)
In this equation, we must assume that c1 ≠ 0. Since the coefficients can be absorbed into
the vectors u(i), there is no loss of generality in assuming that
x(0) = u(1) + u(2) + · · · + u(n) (2)
Then we repeatedly carry out matrix-vector multiplication, using the matrix A to produce a sequence of vectors. Specifically, we have
 x(1) = Ax(0)
x(2) = Ax(1) = A2x(0)
 (3) (2) 3 (0) x =Ax =Ax
 . .
In general, we have
Substituting x(0) in Equation (2), we obtain x(k) = Ak x(0)
= Aku(1) + Aku(2) + Aku(3) +···+ Aku(n) = λk1u(1) +λk2u(2) +λk3u(3) +···+λknu(n)
x(k) = Ak x(0) (k = 1, 2, 3, . . .)
by using Equation (1). This can be written in the form
􏰙 􏰍λ2 􏰎k 􏰍λ3 􏰎k 􏰍λn 􏰎k
the notation, we write the above equation in the form
x(k) = λk1􏰗u(1) + ε(k)􏰘 (3)
where ε(k) → 0 as k → ∞. We let φ be any complex-valued linear functional on Cn such that φ(u(1)) ≠ 0. Recall that φ is a linear functional if φ(ax + by) = aφ(x) + bφ(y) for scalars a and b and vectors x and y. For example, φ(x) = xj for some fixed j (1≦ j ≦ n), is a linear functional. Now, looking back at Equation (3), we apply φ to it:
φ􏰀x(k)􏰁 = λk1􏰗φ􏰀u(1)􏰁 + φ􏰀ε(k)􏰁􏰘
􏰚
u(n) Since|λ1|>|λj|for j >1,wehave|λj/λ1|<1and􏰀λj/λ1􏰁k →0ask→∞.Tosimplify x(k) =λk1 u(1) + λ u(2) + λ u(3) +···+ λ 111 8.3 Power Method 397 Linear Functional Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 398 Chapter 8 More on Linear Systems Next, we form ratios r1, r2, . . . as follows: Ratio Converges to Dominant Eigenvalue φ􏰀x(k+1)􏰁 􏰙φ􏰀u(1)􏰁 + φ􏰀ε(k+1)􏰁􏰚 rk ≡ φ􏰀x(k)􏰁 =λ1 φ􏰀u(1)􏰁+φ􏰀ε(k)􏰁 →λ1 as k →∞ Hence, we are able to compute the dominant eigenvalue λ1 as the limit of the sequence {rk }. With a little more care, we can get an accompanying eigenvector. In the definition of the vec- tors x(k) in Equation (2), we see nothing to prevent the vectors from growing or converging to zero. Normalization cures this problem, as in one of the following pseudocodes. Power Method Pseudocode Here we present pseudocode for calculating the dominant eigenvalue and an associated eigenvector for a prescribed matrix A. In each algorithm, φ is a linear functional chosen by the user. For example, one can use φ(x) = x1 (the first component of the vector). Power Method Pseudocode integer k, kmax, n; real r real array ( A)1:n×1:n , (x)1:n , ( y)1:n external function φ output 0, x fork =1tokmax y ← Ax r ← φ(y)/φ(x) x←y output k, x, r end do Power Method Pseudocode FIGURE 8.3 In 2D, power method illustration We use a simple 2 × 2 matrix such as A=􏰆3 1􏰇 13 to give a geometric illustration of the power method as shown in Figure 8.3. Clearly, the eigenvalues are λ1 = 2 and λ2 = 4 with eigenvectors v(1) = [−1, 1]T and v(2) = [1, 1]T , respectively.Startingwithx(0) =[0,1]T,thepowermethodrepeatedlymultipliesthematrix A by a vector. It produces a sequence of vectors x(1), x(2), and so on that move in the direction of the eigenvector v(2), which corresponds to the dominant eigenvalue λ2 = 4. v(2) x(0) x(1) x(2) v(1) –1 0 1 We can easily modify this algorithm to produce normalized eigenvectors by using the infinity vector norm ||x||∞ = max1 ≦ j ≦ n |x j |, as in the following code: Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. Aitken Acceleration EXAMPLE 1 Solution Computer Results Aitken Acceleration From a given sequence {rk }, we can construct another sequence {sk } by means of the Aitken acceleration formula (rk −rk−1)2 sk = rk − r − 2r + r (k ≧ 3) k k−1 k−2 If the original sequence {rk} converges to r and if certain other conditions are satisfied, then the new sequence {sk } converges to r more rapidly than the original one. (For details, see Kincaid and Cheney [2002].) Because subtractive cancellation may eventually spoil the results, the Aitken acceleration process should be stopped soon after the values become apparently stationary. Use the modified power method algorithm and Aitken acceleration to find the dominant eigenvalue and an eigenvector of the given matrix A, with vector x(0) and φ(x) given as Modified Power Method Algorithm with Normalization x(0) x (1) x (2) x (3) x (4) x (5) x (6) λ1 = 6, u(1) = [1,1,1]T =[−1.0000,1.0000,1.0000]T = [−1.0000, 0.3333, 0.3333]T r0 = 2.0000 = [−1.0000, −0.1111, −0.1111]T r1 = −2.0000 = [−1.0000, −0.4074, −0.4074]T r2 = = [−1.0000, −0.6049, −0.6049]T r3 = = [−1.0000, −0.7366, −0.7366]T r4 = = [−1.0000, −0.8244, −0.8244]T r5 = . . 22.0000 8.9091 7.3061 6.7151 s3 = 13.5294 s4 = 7.0825 s5 = 6.3699 x (14) The Aitken-accelerated sequence, sk, converges noticeably faster than the sequence {rk}. = [−1.0000, −0.9931, −0.9931]T r13 = The actual dominant eigenvalue and an associated eigenvector are Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. 8.3 Power Method 399 integer k, kmax, n; real r real array ( A)1:n×1:n , (x)1:n , ( y)1:n external function φ output 0, x fork =1tokmax y ← Ax r ← φ(y)/φ(x) x ← y/||y||∞ output k, x, r end do Normalized Power Method Pseudocode follows: 6 5 −5 −1 A=2 6 −2, x(0) = 1, φ(x)=x2 2 5 −1 1 After coding and running the modified power method algorithm with Aitken acceleration, we obtain the following results: 6.0208 . s13 = 6.0005 400 Chapter 8 More on Linear Systems Computing Smallest Eigenvalue by Inverse Power Method The coding of the modified power method is very simple, and we leave the actual imple- mentation as an exercise. We also use the simple infinity-norm for normalizing the vectors. The final vectors and estimates of the eigenvalue are displayed with 15 decimals digits. ■ In such a problem, one should always seek an independent verification of the purported answer. Here, we simply compute Ax to see whether it coincides with s14x. The last few commands in the code are doing this rough checking, taking s14 as probably the best estimate of the eigenvalue and the last x-vector as the best estimate of an eigenvector. The results after 14 steps are not very accurate. For better accuracy, take 80 steps! Inverse Power Method It is possible to compute other eigenvalues of a matrix by using modifications of the power method. For example, if A is invertible, we can compute its eigenvalue of smallest magnitude by noting this logical equivalence: Ax=λx ⇐⇒ x=A−1(λx) ⇐⇒ A−1x=1x λ Thus, the smallest eigenvalue of A in magnitude is the reciprocal of the largest eigenvalue of A−1. We compute it by applying the power method to A−1 and taking the reciprocal of the result. Suppose that there is a single smallest eigenvalue of A, which is λn with our usual ordering: |λ1|≧|λ2|≧|λ3|≧ ···≧|λn−1|>|λn|>0
It follows that A is invertible. (Why?) The eigenvalues of A−1 are λ−1 for 1 ≦ j ≦ n. There-
fore, we have
We can use the power method on the matrix A−1 to compute its dominant eigenvalue λ−1.
n
The reciprocal of this is the eigenvalue of A that we sought. Notice that we need not compute A−1 because the equation
EXAMPLE 2
so the vector x(k+1) can be more easily computed by solving this last linear system. To do this, we first find the LU factorization of A, namely, A = LU. Then we repeatedly update the right-hand side and back solve:
Ux(k+1) = L−1x(k)
to obtain x(1), x(2), . . . .
Compute the smallest eigenvalue and an associated eigenvector of this matrix
1 −154 528 407 A=3 55 −144 −121
−132 396 318 using the following initial vector and linear function:
x(0) = [1,2,3]T , φ(x) = x2
is equivalent to the equation
x(k+1) = A−1x(k) Ax(k+1) = x(k)
|λ−1|>|λ−1 |≧ ···≧|λ−1|>0 n n−1 1
j
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

Solution
We decide to take the easy route and use the inverse of A for producing the successive x vectors. We leave the actual implementation as an exercise. The ratios rk are saved, and once they are complete, the Aitken accelerated values, sk , are computed. Notice that at the end, we want the reciprocal of the limiting ratio. Hence, it is easier to use reciprocals at every step in the code. Thus, you see rk = x2/y2 rather than y2/x2, and these ratios should converge to the smallest eigenvalue of A. The final results after 80 steps are these:
x = [0.26726101285547, −0.53452256017715, 0.80178375118802]T s80 = 3.33333333343344
We can divide each entry in x by the first component and arrive at
x = [1.0, −2.00000199979120, 3.00000266638827]T
The eigenvalue is actually 10 , and the eigenvector is [1, −2, 3]T . The discrepancy between 3
Ax and s80x is about 2.6 × 10−6. ■
Example: Inverse Power Method
Using mathematical software on a small example,
6 5 −5
A=26 2 (4)
2 5 −1
we can first get A−1 and then use the power method. (We have changed one entry in the matrix A from Example 1 to solve a different problem.) We leave the implementation of the code as an exercise. In the code, r is the reciprocal of the quantity r in the original power method. Thus, at the end of the computation, r should be the eigenvalue of A that has the smallest absolute value. After the prescribed 30 steps, we find that r = 0.214 and x = [0.7916, 0.5137, 0.3308]T . As usual, we can verify the result independently by computing Ax and rx, which should be equal. The method just illustrated is called the inverse power method. On larger examples, the successive vectors should be com- puted not via A−1 but rather by solving the equation A y = x for y. In mathematical software systems such as MATLAB, Maple, and Mathematica, this can be done with a single command. Alternatively, one can get the LU factorization of A and solve Lz = x and U y = z.
In this example, two eigenvalues are complex. Since the matrix is real, they must be conjugate pairs of the form α + βi and α − βi. They have the same magnitude; thus, the hypothesis |λ1| > |λ2| needed in the convergence proof of the power method is violated. What happens when the power method is applied to A? The values of r for k = 26 to 30 are 0.76, −53.27, 8.86, 2.69, and −9.42. We leave the implementation of the code as a computer exercise.
Shifted (Inverse) Power Method
Other eigenvalues of a matrix (besides the largest and smallest) can be computed by ex- ploiting the following logical equivalences:
Ax=λx⇐⇒(A−μI)x=(λ−μ)x⇐⇒(A−μI)−1x= 1 x λ−μ
8.3 Power Method 401
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

402 Chapter 8
More on Linear Systems
If we want to compute an eigenvalue of A that is close to a given number μ, we can apply the inverse power method to A − μ I and take the reciprocal of the limiting value of r . This should be λ − μ.
We can also compute an eigenvalue of A that is farthest from a given number μ. Suppose that for some eigenvalue λj of matrix A, we have
|λj −μ|>ε and 0<|λi −μ|<ε foralli ≠ j Consider the shifted matrix A − μI. Applying the power method to the shifted matrix A − μ I , we compute ratios rk that converge to λ j − μ. This procedure is called the shifted power method. If we want to compute the eigenvalue of A that is closest to a given number μ, a variant of the procedure is needed. Suppose that λ j is an eigenvalue of A such that 0<|λj −μ|<ε and |λi −μ|>ε foralli ≠ j
Consider the shifted matrix A − μI. The eigenvalues of this matrix are λi − μ. Applying the inverse power method to A − μ I gives an approximate value for (λ j − μ)−1 . We can use the explicit inverse of A − μI or the LU factorization A − μI = LU. Now we repeatedly solve the equations
(A−μI)x(k+1) =x(k)
bysolvinginsteadUx(k+1) = L−1x(k).Sincetheratiosrk convergeto(λj −μ)−1,wehave
􏰌−1
This algorithm is called the shifted inverse power method.
Example: Shifted Inverse Power Method
To illustrate the shifted inverse power method, we consider this matrix
137
A=2−4 5 (5)
3 4 −6
and we write mathematical software to compute the eigenvalue closest to −6. The code we use takes ratios of y2/x2, and we are therefore expecting convergence of these ratios to λ + 6. After eight steps, we have r = 0.9590 and x = [−0.7081, 0.6145, 0.3478]T . Hence, the eigenvalue should be λ = 0.9590 − 6 = −5.0410. We can ask MATLAB to confirm the eigenvalue and eigenvector by computing both Ax and λx to be approximately [3.57, −3.10, −1.75]T .
Summary 8.3
• We have considered the following methods for computing eigenvalues of a matrix. In the power method, we approximate the largest eigenvalue λ1 by generating a sequence of points using the formula
x(k+1) = Ax(k)
and then forming a sequence rk = φ(x(k+1))/φ(x(k)), where φ is a linear functional. Under the right circumstances, this sequence, rk , will converge to the largest eigenvalue of A.
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
λj=μ+ limrk k→∞
=μ+lim k→∞ rk
􏰋
1

•
•
•
(Multiple Choice) Let A =
Intheinversepowermethod,wefindthesmallesteigenvalueλn byusingthepreceding process on the inverse of the matrix. The reciprocal of the largest eigenvalue of A−1 is the smallest eigenvalue of A. We can also describe this process as one of computing the sequence so that
Ax(k+1) = x(k)
In the shifted power method, we find the eigenvalue that is farthest from a given number μ by seeking the largest eigenvalue of A − μI. This involves an iteration to produce a sequence
x(k+1) =(A−μI)x(k)
In the shifted inverse power method, we find the eigenvalue that is closest to μ by
applying the inverse power method to A − μ I . This requires solving the equation (A−μI)x(k+1) =x(k) (A−μI=LU)
a2. (Multiple Choice) What is the expected final output of the following pseudocode? Here y1 and x1 are the first components of y and x, respectively.
8.3 Power Method 403
a
1.
􏰆5 2􏰇 4 7 .
3. Briefly describe how to compute the following:
a. The dominant eigenvalue and associate eigenvector.
b. The next dominant eigenvalue and associated eigen- vector.
c. Theleastdominanteigenvalueandassociatedeigen- vector.
d. An eigenvalue other than the dominant or least dom- inant eigenvalue and associated eigenvectors.
􏰙2−1 0􏰚 4. LetA= −1 2 −1
0 −1 2
Carry out several iterations of the power method, start- ing with x(0) = (1, 1, 1). What is the purpose of this procedure?
􏰙−2 −1 0􏰚 5. LetB=A−4I= −1 −2 −1 .
0 −1 −2
Carry out some iterations of the power method applied to B, starting with x(0) = (1, 1, 1). What is the purpose of this procedure?
−1 1􏰙321􏰚 6. LetC=A=4242.
123
Carry out a few iterations of the power method applied to C, starting with x(0) = (1,1,1). What is the purpose of this procedure?
7. The Rayleigh quotient is the expression ⟨x,x⟩A/ ⟨x, x⟩ = xT Ax/xT x. How can the Rayleigh quotient be used when Ax = λx?
The power method has been applied to the matrix A. The result is a long list of vectors that seem to settle down to a vector of the form [h, 1]T , where |h| < 1. What is the largest eigenvalue, approximately, in terms of that number h? a. 4h+7 b. 5h+2 c. 1/h d. 5h + 4 e. None of these. integer n, kmax; real r real array ( A−1)1:n×1:n , (x)1:n , ( y)1:n fork =1to30 y ← A−1x r ← y1/x1 x ← y/||y|| output r, x end do a. r is the eigenvalue of A largest in magnitude, and x is an accompanying eigenvector. b. r=1/λ,whereλisthesmallesteigenvalueofA,and x is such that Ax = λx. c. A vector x such that Ax = rx, where r is the eigen- value of A having the smallest magnitude. d. r is the largest (in magnitude) eigenvalue of A and x is a corresponding eigenvector of A. e. Noneofthese. Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. Exercises 8.3 404 Chapter 8 More on Linear Systems 1. Use the power method, the inverse power method, and their shifted forms as well as Aitken’s acceleration to find some or all of the eigenvalues of the following matrices: 7. 􏰙−4 14 0􏰚 Let A = −5 13 0 . −1 0 2 Code and apply each of the following: 1 1  b. Maple, or Mathematica. 3. Modify and test the pseudocode for the power method to normalize the vector so that the largest component is always 1 in the infinity-norm. This procedure gives the eigenvector and eigenvalue without having to compute a linear functional. 􏰙 −57 192 148 􏰚 a4. LetA= 20 −53 −44 −48 144 115 Find the eigenvalues that are close to −4, 2, and 8 by using the inverse power method. 5. UsingmathematicalsoftwaresuchasMATLAB,Maple, or Mathematica, write and execute code for implement- ing the methods in Section 8.3. Verify that the results are consistent with those described in the text. a. Example 1 using the modified power method. b. Example 2 using the inverse power method with Aitken acceleration. c. Matrix(4)usingtheinversepowermethod. 5 4 1 a.4 5 1 1 1 4 1 1 2 􏰙234􏰚 7 −1 3 1 −1 5 0 −2 The modified power algorithm starting with x(0) = [1, 1, 1]T as well as the Aitken’s acceleration process. The inverse power algorithm. The shifted power algorithm. The shifted inverse power algorithm. −2 1 0 0 −2 1 0 c.0 1−2 1 0 01 −2 0 00 1 0 0  1 2 4 a. b. c. d. (Continuation) Let B = Repeat the previous exercise starting with x(0) =  1 2. Redotheexamplesinthissection,usingeitherMATLAB, 􏰙􏰚 d. Matrix (5) using the shifted power method. 111 􏰙113􏰚 8. 9. 10. 4 −1 1 −1 3 −2 . 1 −2 3 18 −5 −7 2 (0)T [1,0,0]T. (Continuation) Let C = 􏰙−8 −5 8􏰚 6 3 −8 . −3 1 9 Use x(0) = [1, 1, 1]T . Repeat the previous exercise start- ingwithx(0)=[1,0,0]T. By means of the power method, find an eigenvalue and associated eigenvector of these matrices from the histor- ical books by Fox [1957] and Wilkinson [1965]. Use the given starting values and carry out the procedure with and without normalization. Verify your results by using math- ematical software such as MATLAB, Maple, or Mathe- matica. a. 􏰆 0.9901 0.002􏰇, x(0) = [1,0.9]T −0.0001 0.9904 􏰙8−1−5􏰚 (0) T b. −4 4 −2 , x =[1,0.8,1] 6. Considerthematrix A=1 1 1  c. 1 −2 1 , x =[1,1,1] 114 313 242 a. Use the normalized power method starting with x(0) = [1, 1, 1]T , and find the dominant eigenvalue and eigenvector of the matrix A. b. Repeat, starting with the initial value x(0) = [−0.64966116, 0.74822116, 0]T . Explain the results. See Ralston [1965, p. 475–476]. 􏰙−2 −1 4􏰚 (0) d. 2 1 −2 , x =[3,1,2] −1 −1 3 T 11. Find all of the eigenvalues and associated eigenvectors of these matrices from Fox [1957] and Wilkinson [1965] by means of the power method and variations of it. Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. Computer Exercises 8.3 b. 8.4 􏰆 0.4812 −0.0024 0.0023 0.4810 􏰇 d. −1 3 −2 −2 −2 5 􏰙 0.987 0.400 −0.487 􏰚 e. −0.079 0.500 −0.479 0.082 0.400 0.418 􏰙110􏰚 c. −1+10−8 3 0 􏰆21􏰇 011 a. 4 2 􏰙 5 −1 −2􏰚 Verify your results by using mathematical software such as MATLAB, Maple, or Mathematica. Vector Norm l1, l2, l∞-vector Norm Matrix Norm ||x||1 = ||x||2 = |xi| l1-vectornorm Euclidean/l2-vectornorm 􏰎 1 / 2 ||x||∞ = max |xi | 􏰍 􏰄n i=1 8.4 Iterative Solutions of Linear Systems 405 Iterative Solutions of Linear Systems In this section, we explore a completely different strategy for solving a nonsingular linear system Ax = b (1) This alternative approach is often used on enormous problems that arise in solving partial differential equations numerically. In that subject, systems having hundreds of thousands of equations arise routinely. (See Section 12.3.) Vector and Matrix Norms We first present a brief overview of vector and matrix norms because they are useful in the discussion of errors and in the stopping criteria for iterative methods. Norms can be defined on any vector space, but we usually use Rn or Cn. A vector norm ||x|| can be thought of as the length or magnitude of a vector x ∈ Rn . A vector norm is any mapping from Rn to R that obeys these three properties: ||x|| > 0 if x ̸= 0 ||αx|| = |α|||x||
||x + y|| ≦ ||x|| + || y|| (triangle inequality)
for vectors x, y ∈ Rn and scalars α ∈ R. Examples of vector norms for the vector x = (x1,x2,…,xn)T ∈Rn are
􏰄n i=1
1≦i≦n
||A||>0if A̸=0
||α A|| = |α| || A||
|| A + B|| ≦ || A|| + || B|| (triangular inequality)
for matrices A, B, and scalars α.
xi2
For n × n matrices, we can also have matrix norms, subject to the same requirements:
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.
l∞-vector norm

406 Chapter 8
More on Linear Systems
We usually prefer matrix norms that are related to a vector norm. For a vector norm || · ||, the subordinate matrix norm is defined by
||A|| ≡ sup{||Ax|| : x ∈ Rn and ||x|| = 1}
Here, A is an n × n matrix. For a subordinate matrix norm, some additional properties are
||I|| = 1
||Ax|| ≦ ||A||||x||
||AB|| ≦ ||A||||B||
There are two meanings associated with the notation || · || p , one for vectors and another for
matrices. The context determines which one is intended. Examples of subordinate matrix
normsforann×nmatrix Aare
􏰄n
σmax in absolute value is termed the spectral radius of A. (See Sections 7.2 and 9.3 for a discussion of singular values.)
Condition Number and Ill-Conditioning
An important quantity that has some influence in the numerical solution of a linear system
Ax = b is the condition number, which is defined as
κ(A) = ∥A∥2 ∥A−1∥2
It turns out that it is not necessary to compute the inverse of A to obtain an estimate of the condition number. Also, it can be shown that the condition number κ ( A) gauges the transfer of error from the matrix A and the vector b to the solution x.
■ Rule
Ill-Conditioned Matrix
Subordinate Matrix Norm
Matrix Norm Properties
l1, l2, l∞-matrix Norms
Singular Value/ Spectral Radius
Condition Number
||A||1 = max 1≦j≦n
||A||2 = max 1≦i≦n
|aij| |σmax|
l1-matrix norm
spectral /l2-matrix norm l∞-matrix norm
||A||∞ = max
i=1 􏰓􏰄
n 1≦i≦n
|aij|
Here, σi are the eigenvalues of AT A, which are called the singular values of A. The largest
j=1
Rule of Thumb
If κ ( A) = 10 k , then one can expect to lose at least k digits of precision in solving the system Ax = b.
If the linear system is sensitive to perturbations in the elements of A, or to perturbations of the components of b, then this fact is reflected in A having a large condition number. In such a case, the matrix A is said to be ill-conditioned. Briefly, the larger the condition number, the more ill-conditioned the system.
Suppose we want to solve an invertible linear system of equations
Ax = b
for a given coefficient matrix A and right-hand side b, but there may have been pertur-
bations of the data owing to uncertainty in the measurements and roundoff errors in the
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

Condition Number Bound
3 × 3 Hilbert Matrix
||x|| ||b||
From the perturbed linear system Aδx = δb, we obtain δx = A−1δb and
||δx||≦ ||A−1||||δb|| Combining the two inequalities above, we obtain
||δx|| ≦ κ(A)||δb|| ||x|| ||b||
which contains the condition number of the original matrix A.
As an example of an ill-conditioned matrix consider the Hilbert matrix
111
which gives us
||b|| = || Ax|| ≦ || A|| ||x||
1 ||A|| ≦
8.4 Iterative Solutions of Linear Systems 407
calculations. Suppose that the right-hand side is perturbed by an amount assigned the symbol δb and the corresponding solution is perturbed an amount denoted by the symbol δx. Then we have
A(x + δx) = Ax + Aδx = b + δb
where
Aδx = δb
From the original linear system Ax = x and norms, we have
23 H3 =  1 1 1 
We can use the MATLAB commands to generate the matrix and then to compute both the condition number using the 2-norm and the determinant of the matrix. We find the condition number is 524.0568 and the determinant is 4.6296 × 10−4. In solving linear systems, the condition number of the coefficient matrix measures the sensitivity of the system to errors in the data. When the condition number is large, the computed solution of the system may be dangerously in error! Further checks should be made before accepting the solution as being accurate. Values of the condition number near 1 indicate a well-conditioned matrix whereas large values indicate an ill-conditioned matrix. Using the determinant to check for singularity is appropriate only for matrices of modest size. Using mathematical software, one can compute the condition number to check for singular or near-singular matrices.
A goal in the study of numerical methods is to acquire an awareness of whether a numerical result can be trusted or whether it may be suspect (and therefore in need of fur- ther analysis). The condition number provides some evidence regarding this question. With the advent of sophisticated mathematical software systems, an estimate of the condition number is often returned, along with an approximate solution so that one can judge the trustworthiness of the results. In fact, some solution procedures involve advanced features that depend on an estimated condition number and may switch solution techniques based on it. For example, this criterion may result in a switch of the solution technique from a variant of Gaussian elimination to a least-squares solution for an ill-conditioned system.
234 111 345
Well/Ill-Conditioned Matrices
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

408 Chapter 8
More on Linear Systems
General Iteration
Richardson Iteration
General Iterative Pseudocode
Unsuspecting users may not realize that this has happened unless they look at all of the results, including the estimate of the condition number. (Condition numbers can also be associated with other numerical problems, such as locating roots of equations.)
Basic Iterative Methods
The iterative-method strategy produces a sequence of approximate solution vectors x(0), x(1), x(2), . . . for system Ax = b. The numerical procedure is designed so that, in principle, the sequence of vectors converges to the actual solution. The process can be stopped when sufficient precision has been attained. This stands in contrast to the Gaussian elimination algorithm, which has no provision for stopping midway and offering up an approximate solution. A general iterative algorithm for solving System (1) goes as follows: Select a nonsingular matrix Q, and having chosen an arbitrary starting vector x(0), generate vectors x(1), x(2), . . . recursively from the equation
Qx(k) = (Q − A)x(k−1) + b (k = 1,2,…) (2) To see that this is sensible, suppose that the sequence x(k) does converge, to a vector x∗,
say. Then by taking the limit as k → ∞ in System (2), we get Qx∗ = (Q − A)x∗ + b
This leads to Ax∗ = b. Thus, if the sequence converges, its limit is a solution to the original System (1). For example, the Richardson iteration uses Q = I.
An outline of the pseudocode for carrying out the general iterative procedure (2) follows:
integer k, kmax
real array (x(0))1:n , (b)1:n , (c)1:n , (x)1:n , ( y)1:n , ( A)1:n×1:n , ( Q)1:n×1:n x ← x(0)
fork =1tokmax
y←x
c ← ( Q − A)x + b solve Qx = c output k, x
if∥x− y∥<εthen output “convergence” stop end if end for output “maximum iteration reached” In choosing the invertible matrix Q, we are influenced by the following. One should not believe that it is necessary to compute the inverse of Q to carry out an iterative procedure. For small systems, we can easily compute the inverse of Q, but in • System(2)shouldbeeasytosolveforx(k),whentheright-handsideisknown. • MatrixQshouldbechosentoensurethatthesequencex(k)converges,nomatter what initial vector is used. Ideally, this convergence will be rapid. Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it. Jacobi Method general, this is definitely not to be done! We want to solve a linear system in which Q is the coefficient matrix. As was mentioned previously, we want to select Q so that a linear system with Q as the coefficient matrix is easy to solve. Examples of such matrices are diagonal, tridiagonal, banded, lower triangular, and upper triangular. Now, let’s view System (1) in its detailed form 􏰄n aijxj =bi (1≦i≦n) (3) j=1 Solving the i th equation for the i th unknown term, we obtain an equation that describes the Jacobi method: 􏰙􏰄n 􏰚 x(k) = − (aij/aii)x(k−1) +(bi/aii) (1≦i≦n) (4) ij j=1 j ̸=i Here, we assume that all diagonal elements are nonzero. (If this is not the case, we can usually rearrange the equations so that it is.) In the Jacobi method above, the equations are solved in order. The components x(k−1) j and the corresponding new values x(k) can be used immediately in their place. If this is j done, we have the Gauss-Seidel method: 􏰙􏰄n 􏰄n 􏰚 x(k) = − (a /a )x(k) − (a /a )x(k−1) +(b/a ) (5) i ij ii j ij ii j i ii j=1 j=1 ji
If x(k−1) is not saved, then we can dispense with the superscripts in the pseudocode as follows:
Gauss-Seidel Method
Gauss-Seidel Pseudocode
SOR Method
􏰈􏰙 􏰄n x(k) =ω −
j=1 ji
􏰚􏰡
(a /a )x(k−1) +(b/a ) ij ii j i ii
+(1−ω)x(k−1) (6) i
8.4 Iterative Solutions of Linear Systems 409
integer i, j,k,kmax,n; fork =1tokmax
real array (aij)1:n×1:n,(bi)1:n,(xi)1:n 􏰙􏰄n 􏰚􏰽
fori =1ton
xi ← bi − aijxj aii
end for end for
j=1 j ̸=i
An acceleration of the Gauss-Seidel method is possible by the introduction of a relax- ation factor ω, resulting in the successive overrelaxation (SOR) method:
i
The SOR method with ω = 1 reduces to the Gauss-Seidel method.
We now consider numerical examples using iterative methods associated with the names
Jacobi, Gauss-Seidel, and successive overrelaxation.
Copyright 2012 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.