Maximum Likelihood Estimation and Inference With Examples in R, SAS and ADMB

Russell B. Millar is the author of Maximum Likelihood Estimation and Inference: With Examples in R, SAS and ADMB, published by Wiley.

Part I PRELIMINARIES 1

1 A taste of likelihood 3

1.1 Introduction 3

1.2 Motivating example 4

1.3 Using SAS, R and ADMB 9

1.4 Implementation of the motivating example 11

1.5 Exercises 17

2 Essential concepts and iid examples 18

2.1 Introduction 18

2.2 Some necessary notation 19

2.3 Interpretation of likelihood 23

2.4 IID examples 25

2.5 Exercises 33

Part II PRAGMATICS 37

3 Hypothesis tests and confidence intervals or regions 39

3.1 Introduction 39

3.2 Approximate normality of MLEs 40

3.3 Wald tests, confidence intervals and regions 43

3.4 Likelihood ratio tests, confidence intervals and regions 49

3.5 Likelihood ratio examples 54

3.6 Profile likelihood 57

3.7 Exercises 59

4 What you really need to know 64

4.1 Introduction 64

4.2 Inference about g(θ) 65

4.3 Wald statistics – quick and dirty? 75

4.4 Model selection 79

4.5 Bootstrapping 81

4.6 Prediction 91

4.7 Things that can mess you up 95

4.8 Exercises 98

5 Maximizing the likelihood 101

5.1 Introduction 101

5.2 The Newton-Raphson algorithm 103

5.3 The EM (Expectation–Maximization) algorithm 104

5.4 Multi-stage maximization 113

5.5 Exercises 118

6 Some widely used applications of maximum likelihood 121

6.1 Introduction 121

6.2 Box-Cox transformations 122

6.3 Models for survival-time data 125

6.4 Mark–recapture models 134

6.5 Exercises 141

7 Generalized linear models and extensions 143

7.1 Introduction 143

7.2 Specification of a GLM 144

7.3 Likelihood calculations 148

7.4 Model evaluation 149

7.5 Case study 1: Logistic regression and inverse prediction in R 154

7.6 Beyond binomial and Poisson models 161

7.7 Case study 2: Multiplicative vs additive models of over-dispersed counts in SAS 167

7.8 Exercises 173

8 Quasi-likelihood and generalized estimating equations 175

8.1 Introduction 175

8.2 Wedderburn’s quasi-likelihood 177

8.3 Generalized estimating equations 181

8.4 Exercises 187

9 ML inference in the presence of incidental parameters 188

9.1 Introduction 188

9.2 Conditional likelihood 192

9.3 Integrated likelihood 198

9.3.1 Justification 199

9.3.2 Uses of integrated likelihood 200

9.4 Exercises 201

10 Latent variable models 202

10.1 Introduction 202

10.2 Developing the likelihood 203

10.3 Software 204

10.4 One-way linear random-effects model 210

10.5 Nonlinear mixed-effects model 217

10.6 Generalized linear mixed-effects model 221

10.7 State-space model for count data 227

10.8 ADMB template files 228

10.9 Exercises 232

Part III THEORETICAL FOUNDATIONS 233

11 Cramer-Rao inequality and Fisher information 235

11.1 Introduction 235

11.2 The Cramer-Rao inequality for θ  RI 236

11.3 Cramer-Rao inequality for functions of θ 239

11.4 Alternative formulae for I (θ) 241

11.5 The iid data case 243

11.6 The multi-dimensional case, θ  RI s 243

11.7 Examples of Fisher information calculation 247

11.8 Exercises 253

12 Asymptotic theory and approximate normality 256

12.1 Introduction 256

12.2 Consistency and asymptotic normality 257

12.3 Approximate normality 271

12.4 Wald tests and confidence regions 276

12.5 Likelihood ratio test statistic 280

12.6 Rao-score test statistic 281

12.7 Exercises 283

13 Tools of the trade 286

13.1 Introduction 286

13.2 Equivalence of tests and confidence intervals 286

13.3 Transformation of variables 287

13.4 Mean and variance conditional identities 288

13.5 Relevant inequalities 289

13.6 Asymptotic probability theory 291

13.7 Exercises 297

14 Fundamental paradigms and principles of inference 299

14.1 Introduction 299

14.2 Sufficiency principle 300

14.3 Conditionality principle 304

14.4 The likelihood principle 306

14.5 Statistical significance versus statistical evidence 309

14.6 Exercises 311

15 Miscellanea 313

15.1 Notation 313

15.2 Acronyms 315

15.3 Do you think like a frequentist or a Bayesian? 315

15.4 Some useful distributions 316

15.5 Software extras 321

15.6 Automatic differentiation 323

Appendix: Partial solutions to selected exercises 325

Bibliography 337

Index 345

The New copy of this book will include any supplemental materials advertised. Please check the title of the book to determine if it should include any access cards, study guides, lab manuals, CDs, etc.

The Used, Rental and eBook copies of this book are not guaranteed to include any supplemental materials. Typically, only the book itself is included. This is true even if the title states it includes any access cards, study guides, lab manuals, CDs, etc.