Numerical Analysis for Applied Science

Pragmatic and Adaptable Textbook Meets the Needs of Students and Instructors from Diverse Fields

Numerical analysis is a core subject in data science and an essential tool for applied mathematicians, engineers, and physical and biological scientists. This updated and expanded edition of Numerical Analysis for Applied Science follows the tradition of its precursor by providing a modern, flexible approach to the theory and practical applications of the field. As before, the authors emphasize the motivation, construction, and practical considerations before presenting rigorous theoretical analysis.  This approach allows instructors to adapt the textbook to a spectrum of uses, ranging from one-semester, methods-oriented courses to multi-semester theoretical courses.

The book includes an expanded first chapter reviewing useful tools from analysis and linear algebra.  Subsequent chapters include clearly structured expositions covering the motivation, practical considerations, and theory for each class of methods.  The book includes over 250 problems exploring practical and theoretical questions and 32 pseudocodes to help students implement the methods. Other notable features include:

• A preface providing advice for instructors on using the text for a single semester course or multiple-semester sequence of courses
• Discussion of topics covered infrequently by other texts at this level, such as multidimensional interpolation, quasi-Newton methods in several variables, multigrid methods, preconditioned conjugate-gradient methods, finite-difference methods for partial differential equations, and an introduction to finite-element theory
• New topics and expanded treatment of existing topics to address developments in the field since publication of the first edition
• More than twice as many computational and theoretical exercises as the first edition. 

Numerical Analysis for Applied Science, Second Edition provides an excellent foundation for graduate and advanced undergraduate courses in numerical methods and numerical analysis. It is also an accessible introduction to the subject for students pursuing independent study in applied mathematics, engineering, and the physical and life sciences and a valuable reference for professionals in these areas.

Myron B. Allen III, PhD, is Professor of Mathematics in the Department of Mathematics and Statistics at the University of Wyoming, Laramie, USA. His research focuses on the numerical analysis of fluid flows in porous media.

The Late Eli L. Isaacson, PhD, was Professor Emeritus of Mathematics in the Department of Mathematics at the University of Wyoming, Laramie, USA. His work includes analytic and numerical methods for solving systems of hyperbolic conservation laws, including front-tracking methods.

Preface v

1 Some Useful Tools 1

1.1 Introduction 1

1.2 Bounded Sets 4

1.2.1 The Least Upper Bound Principle 4

1.2.2 Bounded Sets in Rⁿ 5

1.3 Normed Vector Spaces 8

1.3.1 Vector Spaces 8

1.3.2 Matrices as Linear Operators 10

1.3.3 Norms 12

1.3.4 Inner Products 15

1.3.5 Norm Equivalence 17

1.4 Eigenvalues and Matrix Norms 19

1.4.1 Eigenvalues and Eigenvectors 19

1.4.2 Matrix Norms 21

1.5 Results from Calculus 26

1.5.1 Seven Theorems 26

1.5.2 The Taylor Theorem 28

1.6 Problems 33

2 Approximation of Functions 37

2.1 Introduction 37

2.2 Polynomial Interpolation 38

2.2.1 Motivation and Construction 38

2.2.2 Practical Considerations 42

2.2.3 Mathematical Details 43

2.2.4 Further Remarks 46

2.3 Piecewise Polynomial Interpolation 48

2.3.1 Motivation and Construction 48

2.3.2 Practical Considerations 50

2.3.3 Mathematical Details 54

2.3.4 Further Remarks 55

2.4 Hermite Interpolation 55

2.4.1 Motivation and Construction 55

2.4.2 Practical Considerations 59

2.4.3 Mathematical Details 60

2.5 Interpolation in Two Dimensions 63

2.5.1 Constructing Tensor-product Interpolants 64

2.5.2 Error Estimates for Tensor-product Methods 68

2.5.3 Interpolation on Triangles: Background 70

2.5.4 Construction of Planar Interpolants on Triangles 72

2.5.5 Error Estimates for Interpolation on Triangles 74

2.6 Splines 78

2.6.1 Motivation and Construction 78

2.6.2 Practical Considerations 84

2.6.3 Mathematical Details 85

2.6.4 Further Remarks 94

2.7 Least-squares Methods 95

2.7.1 Motivation and Construction 96

2.7.2 Practical Considerations 100

2.7.3 Mathematical Details 101

2.7.4 Further Remarks 103

2.8 Trigonometric Interpolation 104

2.8.1 Motivation and Construction 105

2.8.2 Practical Considerations: Fast Fourier Transform 109

2.8.3 Mathematical Details 116

2.8.4 Further Remarks 118

2.9 Problems 118

3 Direct Methods for Linear Systems 125

3.1 Introduction 125

3.2 The Condition Number of a Linear System 127

3.3 Gauss Elimination 131

3.3.1 Motivation and Construction 131

3.3.2 Practical Considerations 133

3.3.3 Mathematical Details 139

3.4 Variants of Gauss Elimination 148

3.4.1 Motivation 148

3.4.2 The Doolittle and Crout Methods 148

3.4.3 Cholesky Decomposition 152

3.5 Band Matrices 155

3.5.1 Motivation and Construction 155

3.5.2 Practical Considerations 161

3.5.3 Mathematical Details 163

3.5.4 Further Remarks 166

3.6 Iterative Improvement 167

3.7 Problems 169

4 Solution of Nonlinear Equations 175

4.1 Introduction 175

4.2 Bisection 179

4.2.1 Motivation and Construction 179

4.2.2 Practical Considerations 181

4.3 Successive Substitution in One Variable 183

4.3.1 Motivation and Construction 183

4.3.2 Practical Considerations 184

4.3.3 Mathematical Details 190

4.4 Newton’s Method in One Variable 192

4.4.1 Motivation and Construction 192

4.4.2 Practical Considerations 194

4.4.3 Mathematical Details 199

4.5 The Secant Method 203

4.5.1 Motivation and Construction 203

4.5.2 Practical Considerations 205

4.5.3 Mathematical Details 206

4.6 Successive Substitution: Several Variables 211

4.6.1 Motivation and Construction 211

4.6.2 Convergence Criteria 213

4.6.3 An Application to Differential Equations 217

4.7 Newton’s Method: Several Variables 219

4.7.1 Motivation and Construction 219

4.7.2 Practical Considerations 221

4.7.3 Mathematical Details: Newton’s Method 224

4.7.4 Mathematical Details: Finite-difference Newton Methods 229

4.8 Problems 233

5 Iterative Methods for Linear Systems 239

5.1 Introduction 239

5.2 Conceptual Foundations 243

5.3 Matrix-Splitting Techniques 248

5.3.1 Motivation and Construction: Jacobi and Gauss–Seidel Methods 248

5.3.2 Practical Considerations 254

5.3.3 Mathematical Details 258

5.4 Successive Overrelaxation 266

5.4.1 Motivation 266

5.4.2 Practical Considerations 266

5.4.3 Mathematical Details 272

5.4.4 Further Remarks: The Power Method and Symmetric SOR 279

5.5 Multigrid Methods 280

5.5.1 Motivation: Error Reduction Versus Smoothing 280

5.5.2 A Two-Grid Algorithm 284

5.5.3 V-cycles and the Full Multigrid Algorithm 289

5.6 The Conjugate-Gradient Method 293

5.6.1 Motivation and Construction 293

5.6.2 Practical Considerations 298

5.6.3 Mathematical Details 303

5.6.4 Further Remarks: Krylov Methods and Steepest Descent 309

5.7 Problems 311

6 Eigenvalue Problems 317

6.1 More About Eigenvalues 318

6.2 Power Methods 323

6.2.1 Motivation and Construction 323

6.2.2 Practical Considerations 325

6.3 The QR Decomposition 328

6.3.1 Geometry and Algebra of the QR Decomposition 328

6.3.2 Application to Least-Squares Problems 333

6.3.3 Further Remarks 335

6.4 The QR Algorithm for Eigenvalues 338

6.4.1 Motivation and Construction 338

6.4.2 Practical Considerations 341

6.4.3 Mathematical Details 347

6.4.4 Further Remarks 350

6.5 Singular Value Decomposition 352

6.5.1 Theory of the Singular Value Decomposition 352

6.5.2 Computing Singular Value Decompositions 354

6.5.3 Application to Principal Component Analysis 354

6.6 Problems 358

7 Numerical Integration 363

7.1 Introduction 363

7.2 Newton–Cotes Formulas 364

7.2.1 Motivation and Construction 364

7.2.2 Practical Considerations: Composite Formulas 367

7.2.3 Mathematical Details 369

7.2.4 Further Remarks 373

7.3 Romberg and Adaptive Quadrature 373

7.3.1 Romberg Quadrature 374

7.3.2 Adaptive Quadrature 379

7.4 Gauss Quadrature 385

7.4.1 Motivation and Construction 385

7.4.2 Practical Considerations 388

7.4.3 Mathematical Details 390

7.5 Problems 399

8 Ordinary Differential Equations 403

8.1 Introduction 403

8.2 One-Step Methods 406

8.2.1 Motivation and Construction 406

8.2.2 Practical Considerations 409

8.2.3 Mathematical Details 410

8.2.4 Further Remarks: The Runge–Kutta–Fehlberg Algorithm 416

8.3 Multistep Methods: Consistency and Stability 420

8.3.1 Motivation 420

8.3.2 Adams–Bashforth and Adams–Moulton Methods 422

8.3.3 Consistency of Multistep Methods 423

8.3.4 Stability of Multistep Methods 426

8.3.5 Predictor-Corrector Methods 430

8.3.6 Mathematical Details: The Root Condition 431

8.4 Multistep Methods: Convergence 438

8.4.1 Convergence Implies Stability and Consistency 439

8.4.2 Consistency and Stability Imply Convergence 441

8.5 Problems 448

9 Difference Methods for PDEs 453

9.1 Introduction 453

9.1.1 Classification 454

9.1.2 Characteristic Curves and Characteristic Equations 455

9.1.3 Grid Functions and Difference Operators 460

9.2 The Poisson Equation 462

9.2.1 The Five-Point Method 463

9.2.2 Consistency and Convergence 466

9.2.3 Accommodating Variable Coefficients 471

9.2.4 Accommodating Other Boundary Conditions 472

9.2.5 Accommodating Nonrectangular Domains 473

9.3 The Advection Equation 475

9.3.1 The Courant–Friedrichs–Lewy Condition 476

9.3.2 Stability of Approximations to Time-Dependent Problems 480

9.3.3 Sufficient Conditions for Convergence 485

9.3.4 Further Remarks 488

9.4 Other Time-Dependent Equations 489

9.4.1 The Heat Equation 489

9.4.2 The Advection-Diffusion Equation 498

9.4.3 The Wave Equation 503

9.5 Problems 505

10 Introduction to Finite Elements 511

10.1 Introduction and Background 511

10.1.1 A Model Boundary-Value Problem 512

10.1.2 Variational Formulation 513

10.2 A Steady-State Problem 517

10.2.1 Construction of a Finite-Element Approximation 517

10.2.2 A Basic Error Estimate 520

10.2.3 Optimal-Order Error Estimates 525

10.2.4 Other Boundary Conditions 528

10.2.5 Condition Number of the Finite-Element Matrix 532

10.3 A Transient Problem 537

10.3.1 A Semidiscrete Formulation 537

10.3.2 A Fully Discrete Method 539

10.3.3 Convergence of the Fully Discrete Method 540

10.4 Problems 547

A Divided Differences 549

B Local Minima 553

C Chebyshev Polynomials 555

References 559

Index 563

The New copy of this book will include any supplemental materials advertised. Please check the title of the book to determine if it should include any access cards, study guides, lab manuals, CDs, etc.

The Used, Rental and eBook copies of this book are not guaranteed to include any supplemental materials. Typically, only the book itself is included. This is true even if the title states it includes any access cards, study guides, lab manuals, CDs, etc.