Interpolation and Extrapolation Optimal Designs V1: Polynomial Regression and Approximation Theory

This book is the first of a series which focuses on the interpolation and extrapolation of optimal designs, an area with significant applications in engineering, physics, chemistry and most experimental fields. In this volume, the authors emphasize the importance of problems associated with the construction of design. After a brief introduction on how the theory of optimal designs meets the theory of the uniform approximation of functions, the authors introduce the basic elements to design planning and link the statistical theory of optimal design and the theory of the uniform approximation of functions. The appendices provide the reader with material to accompany the proofs discussed throughout the book.

Preface ix Introduction xi Part 1. Elements from Approximation Theory 1 Chapter 1. Uniform Approximation 3 1.1. Canonical polynomials and uniform approximation 3 1.2. Existence of the best approximation 4 1.3. Characterization and uniqueness of the best approximation 5 1.3.1. Proof of the Borel Chebyshev theorem 7 1.3.2. Example 13 Chapter 2. Convergence Rates for the Uniform Approximation and Algorithms 15 2.1. Introduction 15 2.2. The Borel Chebyshev theorem and standard functions 15 2.3. Convergence of the minimax approximation 20 2.3.1. Rate of convergence of the minimax approximation 21 2.4. Proof of the de la Vallee Poussin theorem 24 2.5. The Yevgeny Yakovlevich Remez algorithm 28 2.5.1. The Remez algorithm 29 2.5.2. Convergence of the Remez algorithm 33 Chapter 3. Constrained Polynomial Approximation 43 3.1. Introduction and examples 43 3.2. Lagrange polynomial interpolation 47 3.3. The interpolation error 50 3.3.1. A qualitative result 50 3.3.2. A quantitative result 52 3.4. The role of the nodes and the minimization of the interpolation error 54 3.5. Convergence of the interpolation approximation 56 3.6. Runge phenomenon and lack of convergence 57 3.7. Uniform approximation for C( ) ([a, b]) functions 62 3.8. Numerical instability 63 3.9. Convergence, choice of the distribution of the nodes, Lagrange interpolation and splines 67 Part 2. Optimal Designs for Polynomial Models 69 Chapter 4. Interpolation and Extrapolation Designs for the Polynomial Regression 71 4.1. Definition of the model and of the estimators 71 4.2. Optimal extrapolation designs: Hoel Levine or Chebyshev designs 75 4.2.1. Uniform optimal interpolation designs (according to Guest) 85 4.2.2. The interplay between the Hoel Levine and the Guest designs 95 4.2.3. Confidence bound for interpolation/extrapolation designs 98 4.3. An application of the Hoel Levine design 100 4.4. Multivariate optimal designs: a special case 103 Chapter 5. An Introduction to Extrapolation Problems Based on Observations on a Collection of Intervals 113 5.1. Introduction 113 5.2. The model, the estimator and the criterion for the choice of the design 119 5.2.1. Criterion for the optimal design 121 5.3. A constrained Borel Chebyshev theorem 122 5.3.1. Existence of solutions to the Pg 1 (0, 1) problem 122 5.3.2. A qualitative discussion on some constrained Borel Chebyshev theorem 123 5.3.3. Borel Chebyshev theorem on [a, b] [d, e] 125 5.3.4. From the constrained Borel Chebyshev theorem to the support of the optimal design 126 5.4. Qualitative properties of the polynomial which determines the optimal nodes 127 5.4.1. The linear case 127 5.4.2. The general polynomial case 128 5.5. Identification of the polynomial which characterizes the optimal nodes 130 5.5.1. The differential equation 130 5.5.2. Example 132 5.6. The optimal design in favorable cases 134 5.6.1. Some explicit optimal designs 136 5.7. The optimal design in the general case 137 5.7.1. The extreme points of a linear functional 138 5.7.2. Some results on the representation of the extreme points 138 5.7.3. The specific case of the Dirac functional at point 0 142 5.7.4. Remez algorithm for the extreme polynomial: the optimal design in general cases 145 5.8. Spruill theorem: the optimal design 146 Chapter 6. Instability of the Lagrange Interpolation Scheme With Respect to Measurement Errors 147 6.1. Introduction 147 6.2. The errors that cannot be avoided 147 6.2.1. The role of the errors: interpolation designs with minimal propagation of the errors 150 6.2.2. Optimizing on the nodes 153 6.3. Control of the relative errors 157 6.3.1. Implementation of the Remez algorithm for the relative errors 162 6.4. Randomness 166 6.5. Some inequalities for the derivatives of polynomials 167 6.6. Concentration inequalities 168 6.7. Upper bounds of the extrapolation error due to randomness, and the resulting size of the design for real analytic regression functions 172 6.7.1. Case 1: the range of the observations is bounded 177 6.7.2. Case 2: the range of the observations is unbounded 183 Part 3. Mathematical Material 185 Appendix 1. Normed Linear Spaces 187 Appendix 2. Chebyshev Polynomials 217 Appendix 3. Some Useful Inequalities for Polynomials 221 Bibliography 243 Index 251