Derivative-Free and Blackbox Optimization

This book is designed as a textbook, suitable for self-learning or for teaching an upper-year university course on derivative-free and blackbox optimization. The book is split into 5 parts and is designed to be modular any individual part depends only on the material in Part I. Part I of the book discusses what is meant by Derivative-Free and Blackbox Optimization, provides background material, and early basics while Part II focuses on heuristic methods (Genetic Algorithms and Nelder-Mead). Part III presents direct search methods (Generalized Pattern Search and Mesh Adaptive Direct Search) and Part IV focuses on model-based methods (Simplex Gradient and Trust Region). Part V discusses dealing with constraints, using surrogates, and bi-objective optimization.End of chapter exercises are included throughout as well as 15 end of chapter projects and over 40 figures. Benchmarking techniques are also presented in the appendix.

Part I: Introduction and Background Material.- Introduction: Tools and Challenges.- Mathematical Background.- The Beginnings of DFO Algorithms.- Part I: Some Remarks on DFO.- Part II: Popular Heuristic Methods.- Genetic Algorithms.- Nelder-Mead.- Part II: Further Remarks on Heuristics.- Part III: Direct Search Methods.- Positive bases and Nonsmooth Optimization.- Generalized Pattern Search.- Mesh Adaptive Direct Search.- Part III: Further Remarks on Direct Search Methods.- Part IV: Model-based Methods.- Model-based Descent.- Model-based Trust Region.- Part IV: Further Remarks on Model-based Methods.- Part V: Extensions and Refinements.- Variables and Constraints.- Optimization Using Surrogates and Models.- Biobjective Optimization.- Part V: Final Remarks on DFO/BBO.- Part VI: Appendix: Comparing Optimization Methods.- Solutions to Selected Exercises.