Advanced Dynamic–System Simulation: Model Replication and Monte Carlo Studies

Now in a fully revised second edition, this work introduces dynamic–system simulation with a main emphasis on OPEN DESIRE and DESIRE software. Offering a complete update of all material, the new edition boasts two completely new chapters on fast simulation of neural networks as well as three appendices on radial–basis–function, fuzzy–basis–function networks, and CLEARN algorithm. A companion CD contains complete binary OPEN DESIRE modeling/simulation program packages for personal–computer LINUX and MS Windows, DESIRE examples, source code, and a comprehensive, indexed reference manual.

PREFACE xiii CHAPTER 1 DYNAMIC–SYSTEM MODELS AND SIMULATION 1 SIMULATION IS EXPERIMENTATION WITH MODELS 1 1–1 Simulation and Computer Programs 1 1–2 Dynamic–System Models 2 (a) Difference–Equation Models 2 (b) Differential–Equation Models 2 (c) Discussion 3 1–3 Experiment Protocols Define Simulation Studies 3 1–4 Simulation Software 4 1–5 Fast Simulation Program for Interactive Modeling 5 ANATOMY OF A SIMULATION RUN 8 1–6 Dynamic–System Time Histories Are Sampled Periodically 8 1–7 Numerical Integration 10 (a) Euler Integration 10 (b) Improved Integration Rules 10 1–8 Sampling Times and Integration Steps 11 1–9 Sorting Defined–Variable Assignments 12 SIMPLE APPLICATION PROGRAMS 12 1–10 Oscillators and Computer Displays 12 (a) Linear Oscillator 12 (b) Nonlinear Oscillator: Duffing’s Differential Equation 14 1–11 Space–Vehicle Orbit Simulation with Variable–Step Integration 15 1–12 Population–Dynamics Model 17 1–13 Splicing Multiple Simulation Runs: Billiard–Ball Simulation 17 INRODUCTION TO CONTROL–SYSTEM SIMULATION 21 1–14 Electrical Servomechanism with Motor–Field Delay and Saturation 21 1–15 Control–System Frequency Response 23 1–16 Simulation of a Simple Guided Missile 24 (a) Guided Torpedo 24 (b) Complete Torpedo–Simulation Program 26 STOP AND LOOK 28 1–17 Simulation in the Real World: A Word of Caution 28 References 29 CHAPTER 2 MODELS WITH DIFFERENCE EQUATIONS, LIMITERS, AND SWITCHES 31 SAMPLED–DATA SYSTEMS AND DIFFERENCE EQUATIONS 31 2–1 Sampled–Data Difference–Equation Systems 31 (a) Introduction 31 (b) Difference Equations 31 (c) A Minefield of Possible Errors 32 2–2 Solving Systems of First–Order Difference Equations 32 (a) General Difference–Equation Model 32 (b) Simple Recurrence Relations 33 2–3 Models Combining Differential Equations and Sampled–Data Operations 35 2–4 Simple Example 35 2–5 Initializing and Resetting Sampled–Data Variables 35 TWO MIXED CONTINUOUS/SAMPLED–DATA SYSTEMS 37 2–6 Guided Torpedo with Digital Control 37 2–7 Simulation of a Plant with a Digital PID Controller 37 DYNAMIC–SYSTEM MODELS WITH LIMITERS AND SWITCHES 40 2–8 Limiters, Switches, and Comparators 40 (a) Limiter Functions 40 (b) Switching Functions and Comparators 42 2–9 Integration of Switch and Limiter Outputs, Event Prediction, and Display Problems 43 2–10 Using Sampled–Data Assignments 44 2–11 Using the step Operator and Heuristic Integration–Step Control 44 2–12 Example: Simulation of a Bang–Bang Servomechanism 45 2–13 Limiters, Absolute Values, and Maximum/Minimum Selection 46 2–14 Output–Limited Integration 47 2–15 Modeling Signal Quantization 48 EFFICIENT DEVICE MODELS USING RECURSIVE ASSIGNMENTS 48 2–16 Recursive Switching and Limiter Operations 48 2–17 Track/Hold Simulation 49 2–18 Maximum–Value and Minimum–Value Holding 50 2–19 Simple Backlash and Hysteresis Models 51 2–20 Comparator with Hysteresis (Schmitt Trigger) 52 2–21 Signal Generators and Signal Modulation 53 References 55 CHAPTER 3 FAST VECTOR–MATRIX OPERATIONS AND SUBMODELS 57 ARRAYS, VECTORS, AND MATRICES 57 3–1 Arrays and Subscripted Variables 57 (a) Improved Modeling 57 (b) Array Declarations, Vectors, and Matrices 57 (c) State–Variable Declarations 58 3–2 Vector and Matrices in Experiment Protocols 58 3–3 Time–History Arrays 58 VECTORS AND MODEL REPLICATION 59 3–4 Vector Operations in DYNAMIC Program Segments: The Vectorizing Compiler 59 (a) Vector Assignments and Vector Expressions 59 (b) Vector Differential Equations 60 (c) Vector Sampled–Data Assignments and Difference Equations 60 3–5 Matrix–Vector Products in Vector Expressions 61 (a) Definition 61 (b) Simple Example: Resonating Oscillators 61 3–6 Index–Shift Operation 63 (a) Definition 63 (b) Preview of Significant Applications 63 3–7 Sorting Vector and Subscripted–Variable Assignments 64 3–8 Replication of Dynamic–System Models 64 MORE VECTOR OPERATIONS 65 3–9 Sums, DOT Products, and Vector Norms 65 (a) Sums and DOT Products 65 (b) Euclidean, Taxicab, and Hamming Norms 65 3–10 Maximum/Minimum Selection and Masking 66 (a) Maximum/Minimum Selection 66 (b) Masking Vector Expressions 66 VECTOR EQUIVALENCE DECLARATIONS SIMPLIFY MODELS 67 3–11 Subvectors 67 3–12 Matrix–Vector Equivalence 67 MATRIX OPERATIONS IN DYNAMIC–SYSTEM MODELS 67 3–13 Simple Matrix Assignments 67 3–14 Two–Dimensional Model Replication 68 (a) Matrix Expressions and DOT Products 68 (b) Matrix Differential Equations 68 (c) Matrix Difference Equations 69 VECTORS IN PHYSICS AND CONTROL–SYSTEM PROBLEMS 69 3–15 Vectors in Physics Problems 69 3–16 Vector Model of a Nuclear Reactor 69 3–17 Linear Transformations and Rotation Matrices 70 3–18 State–Equation Models of Linear Control Systems 72 USER–DEFINED FUNCTIONS AND SUBMODELS 72 3–19 Introduction 72 3–20 User–Defined Functions 72 3–21 Submodel Declaration and Invocation 73 3–22 Dealing with Sampled–Data Assignments, Limiters, and Switches 75 References 75 CHAPTER 4 EFFICIENT PARAMETER–INFLUENCE STUDIES AND STATISTICS COMPUTATION 77 MODEL REPLICATION SIMPLIFIES PARAMETER–INFLUENCE STUDIES 77 4–1 Exploring the Effects of Parameter Changes 77 4–2 Repeated Simulation Runs Versus Model Replication 78 (a) Simple Repeated–Run Study 78 (b) Model Replication (Vectorization) 78 4–3 Programming Parameter–Influence Studies 80 (a) Measures of System Performance 80 (b) Program Design 81 (c) Two–Dimensional Model Replication 81 (d) Cross–Plotting Results 82 (e) Maximum/Minimum Selection 83 (f) Iterative Parameter Optimization 83 STATISTICS 84 4–4 Random Data and Statistics 84 4–5 Sample Averages and Statistical Relative Frequencies 85 COMPUTING STATISTICS BY VECTOR AVERAGING 85 4–6 Fast Computation of Sample Averages 85 4–7 Fast Probability Estimation 86 4–8 Fast Probability–Density Estimation 86 (a) Simple Probability–Density Estimate 86 (b) Triangle and Parzen Windows 87 (c) Computation and Display of Parzen–Window Estimates 88 4–9 Sample–Range Estimation 90 REPLICATED AVERAGES GENERATE SAMPLING DISTRIBUTIONS 91 4–10 Computing Statistics by Time Averaging 91 4–11 Sample Replication and Sampling–Distribution Statistics 91 (a) Introduction 91 (b) Demonstrations of Empirical Laws of Large Numbers 93 (c) Counterexample: Fat–Tailed Distribution 95 RANDOM–PROCESS SIMULATION 95 4–12 Random Processes and Monte Carlo Simulation 95 4–13 Modeling Random Parameters and Random Initial Values 97 4–14 Sampled–Data Random Processes 97 4–15 “Continuous” Random Processes 98 (a) Modeling Continuous Noise 98 (b) Continuous Time Averaging 99 (c) Correlation Functions and Spectral Densities 100 4–16 Problems with Simulated Noise 100 SIMPLE MONTE CARLO EXPERIMENTS 100 4–17 Introduction 100 4–18 Gambling Returns 100 4–19 Vectorized Monte Carlo Study of a Continuous Random Walk 102 References 106 CHAPTER 5 MONTE CARLO SIMULATION OF REAL DYNAMIC SYSTEMS 109 INTRODUCTION 109 5–1 Survey 109 REPEATED–RUN MONTE CARLO SIMULATION 109 5–2 End–of–Run Statistics for Repeated Simulation Runs 109 5–3 Example: Effects of Gun–Elevation Errors on a 1776 Cannnonball Trajectory 110 5–4 Sequential Monte Carlo Simulation 113 VECTORIZED MONTE CARLO SIMULATION 113 5–5 Vectorized Monte Carlo Simulation of the 1776 Cannon Shot 113 5–6 Combined Vectorized and Repeated–Run Monte Carlo Simulation 115 5–7 Interactive Monte Carlo Simulation: Computing Runtime Histories of Statistics with DYNAMIC–Segment DOT Operations 115 5–8 Example: Torpedo Trajectory Dispersion 117 SIMULATION OF NOISY CONTROL SYSTEMS 119 5–9 Monte Carlo Simulation of a Nonlinear Servomechanism: A Noise–Input Test 119 5–10 Monte Carlo Study of Control–System Errors Caused by Noise 121 ADDITIONAL TOPICS 123 5–11 Monte Carlo Optimization 123 5–12 Convenient Heuristic Method for Testing Pseudorandom Noise 123 5–13 Alternative to Monte Carlo Simulation 123 (a) Introduction 123 (b) Dynamic Systems with Random Perturbations 123 (c) Mean–Square Errors in Linearized Systems 124 References 125 CHAPTER 6 VECTOR MODELS OF NEURAL NETWORKS 127 ARTIFICIAL NEURAL NETWORKS 127 6–1 Introduction 127 6–2 Artificial Neural Networks 127 6–3 Static Neural Networks: Training, Validation, and Applications 128 6–4 Dynamic Neural Networks 129 SIMPLE VECTOR ASSIGNMENTS MODEL NEURON LAYERS 130 6–5 Neuron–Layer Declarations and Neuron Operations 130 6–6 Neuron–Layer Concatenation Simplifies Bias Inputs 130 6–7 Normalizing and Contrast–Enhancing Layers 131 (a) Pattern Normalization 131 (b) Contrast Enhancement: Softmax and Thresholding 131 6–8 Multilayer Networks 132 6–9 Exercising a Neural–Network Model 132 (a) Computing Successive Neuron–Layer Outputs 132 (b) Input from Pattern–Row Matrices 133 (c) Input from Text Files and Spreadsheets 133 SUPERVISED TRAINING FOR REGRESSION 134 6–10 Mean–Square Regression 134 (a) Problem Statement 134 (b) Linear Mean–Square Regression and the Delta Rule 135 (c) Nonlinear Neuron Layers and Activation–Function Derivatives 136 (d) Error–Measure Display 136 6–11 Backpropagation Networks 137 (a) The Generalized Delta Rule 137 (b) Momentum Learning 139 (c) Simple Example 139 (d) The Classical XOR Problem and Other Examples 140 MORE NEURAL–NETWORK MODELS 140 6–12 Functional–Link Networks 140 6–13 Radial–Basis–Function Networks 142 (a) Basis–Function Expansion and Linear Optimization 142 (b) Radial Basis Functions 143 6–14 Neural–Network Submodels 145 PATTERN CLASSIFICATION 146 6–15 Introduction 146 6–16 Classifier Input from Files 147 6–17 Classifier Networks 147 (a) Simple Linear Classifiers 147 (b) Softmax Classifiers 148 (c) Backpropagation Classifiers 148 (d) Functional–Link Classifiers 149 (e) Other Classsifiers 149 6–18 Examples 149 (a) Classificatio...