Foundations of Linear and Generalized Linear Models

A valuable overview of the most important ideas and results in statistical modeling

Written by a highly–experienced author, Foundations of Linear and Generalized Linear Models is a clear and comprehensive guide to the key concepts and results of linearstatistical models. The book presents a broad, in–depth overview of the most commonly usedstatistical models by discussing the theory underlying the models, R software applications,and examples with crafted models to elucidate key ideas and promote practical modelbuilding.

The book begins by illustrating the fundamentals of linear models, such as how the model–fitting projects the data onto a model vector subspace and how orthogonal decompositions of the data yield information about the effects of explanatory variables. Subsequently, the book covers the most popular generalized linear models, which include binomial and multinomial logistic regression for categorical data, and Poisson and negative binomial loglinear models for count data. Focusing on the theoretical underpinnings of these models, Foundations of Linear and Generalized Linear Models also features:

An introduction to quasi–likelihood methods that require weaker distributional assumptions, such as generalized estimating equation methods An overview of linear mixed models and generalized linear mixed models with random effects for clustered correlated data, Bayesian modeling, and extensions to handle problematic cases such as high dimensional problems Numerous examples that use R software for all text data analyses More than 400 exercises for readers to practice and extend the theory, methods, and data analysis A supplementary website with datasets for the examples and exercises

An invaluable textbook for upper–undergraduate and graduate–level students in statistics and biostatistics courses, Foundations of Linear and Generalized Linear Models is also an excellent reference for practicing statisticians and biostatisticians, as well as anyone who is interested in learning about the most important statistical models for analyzing data.

Alan Agresti, PhD, is Distinguished Professor Emeritus in the Department of Statistics at the University of Florida. He has presented short courses on generalized linear models and categorical data methods in more than 30 countries. The author of over 200 journal articles, Dr. Agresti is also the author of Categorical Data Analysis, Third Edition, Analysis of Ordinal Categorical Data, Second Edition, and An Introduction to Categorical Data Analysis, Second Edition, all published by Wiley.

Preface xi

1 Introduction to Linear and Generalized Linear Models 1

1.1 Components of a Generalized Linear Model 2

1.2 Quantitative/Qualitative Explanatory Variables and Interpreting Effects 6

1.3 Model Matrices and Model Vector Spaces 10

1.4 Identifiability and Estimability 13

1.5 Example: Using Software to Fit a GLM 15

Chapter Notes 20

Exercises 21

2 Linear Models: Least Squares Theory 26

2.1 Least Squares Model Fitting 27

2.2 Projections of Data Onto Model Spaces 33

2.3 Linear Model Examples: Projections and SS Decompositions 41

2.4 Summarizing Variability in a Linear Model 49

2.5 Residuals Leverage and Influence 56

2.6 Example: Summarizing the Fit of a Linear Model 62

2.7 Optimality of Least Squares and Generalized Least Squares 67

Chapter Notes 71

Exercises 71

3 Normal Linear Models: Statistical Inference 80

3.1 Distribution Theory for Normal Variates 81

3.2 Significance Tests for Normal Linear Models 86

3.3 Confidence Intervals and Prediction Intervals for Normal Linear Models 95

3.4 Example: Normal Linear Model Inference 99

3.5 Multiple Comparisons: Bonferroni Tukey and FDR Methods 107

Chapter Notes 111

Exercises 112

4 Generalized Linear Models: Model Fitting and Inference 120

4.1 Exponential Dispersion Family Distributions for a GLM 120

4.2 Likelihood and Asymptotic Distributions for GLMs 123

4.3 Likelihood–Ratio/Wald/Score Methods of Inference for GLM Parameters 128

4.4 Deviance of a GLM Model Comparison and Model Checking 132

4.5 Fitting Generalized Linear Models 138

4.6 Selecting Explanatory Variables for a GLM 143

4.7 Example: Building a GLM 149

Appendix: GLM Analogs of Orthogonality Results for Linear Models 156

Chapter Notes 158

Exercises 159

5 Models for Binary Data 165

5.1 Link Functions for Binary Data 165

5.2 Logistic Regression: Properties and Interpretations 168

5.3 Inference About Parameters of Logistic Regression Models 172

5.4 Logistic Regression Model Fitting 176

5.5 Deviance and Goodness of Fit for Binary GLMs 179

5.6 Probit and Complementary Log Log Models 183

5.7 Examples: Binary Data Modeling 186

Chapter Notes 193

Exercises 194

6 Multinomial Response Models 202

6.1 Nominal Responses: Baseline–Category Logit Models 203

6.2 Ordinal Responses: Cumulative Logit and Probit Models 209

6.3 Examples: Nominal and Ordinal Responses 216

Chapter Notes 223

Exercises 223

7 Models for Count Data 228

7.1 Poisson GLMs for Counts and Rates 229

7.2 Poisson/Multinomial Models for Contingency Tables 235

7.3 Negative Binomial GLMS 247

7.4 Models for Zero–Inflated Data 250

7.5 Example: Modeling Count Data 254

Chapter Notes 259

Exercises 260

8 Quasi–Likelihood Methods 268

8.1 Variance Inflation for Overdispersed Poisson and Binomial GLMs 269

8.2 Beta–Binomial Models and Quasi–Likelihood Alternatives 272

8.3 Quasi–Likelihood and Model Misspecification 278

Chapter Notes 282

Exercises 282

9 Modeling Correlated Responses 286

9.1 Marginal Models and Models with Random Effects 287

9.2 Normal Linear Mixed Models 294

9.3 Fitting and Prediction for Normal Linear Mixed Models 302

9.4 Binomial and Poisson GLMMs 307

9.5 GLMM Fitting Inference and Prediction 311

9.6 Marginal Modeling and Generalized Estimating Equations 314

9.7 Example: Modeling Correlated Survey Responses 319

Chapter Notes 322

Exercises 324

10 Bayesian Linear and Generalized Linear Modeling 333

10.1 The Bayesian Approach to Statistical Inference 333

10.2 Bayesian Linear Models 340

10.3 Bayesian Generalized Linear Models 347

10.4 Empirical Bayes and Hierarchical Bayes Modeling 351

Chapter Notes 357

Exercises 359

11 Extensions of Generalized Linear Models 364

11.1 Robust Regression and Regularization Methods for Fitting Models 365

11.2 Modeling With Large p 375

11.3 Smoothing Generalized Additive Models and Other GLM Extensions 378

Chapter Notes 386

Exercises 388

Appendix A Supplemental Data Analysis Exercises 391

Appendix B Solution Outlines for Selected Exercises 396

References 410

Author Index 427

Example Index 433

Subject Index 435