Applied Bayesian Modelling

This book provides an accessible approach to Bayesian computing and data analysis, with an emphasis on the interpretation of real data sets. Following in the tradition of the successful first edition, this book aims to make a wide range of statistical modeling applications accessible using tested code that can be readily adapted to the reader's own applications. The second edition has been thoroughly reworked and updated to take account of advances in the field. A new set of worked examples is included. The novel aspect of the first edition was the coverage of statistical modeling using WinBUGS and OPENBUGS. This feature continues in the new edition along with examples using R to broaden appeal and for completeness of coverage.

Peter Congdon is Research Professor of Quantitative Geography and Health Statistics at Queen Mary University of London. He has written three earlier books on Bayesian modelling and data analysis techniques with Wiley, and has a wide range of publications in statistical methodology and in application areas. His current interests include applications to spatial and survey data relating to health status and health service research.

1 Bayesian methods and Bayesian estimation 1

1.1 Introduction 1

1.1.1 Summarising existing knowledge: Prior densities for parameters 2

1.1.2 Updating information: Prior, likelihood and posterior densities 3

1.1.3 Predictions and assessment 5

1.1.4 Sampling parameters 6

1.2 MCMC techniques: The Metropolis–Hastings algorithm 7

1.2.1 Gibbs sampling 8

1.2.2 Other MCMC algorithms 9

1.2.3 INLA approximations 10

1.3 Software for MCMC: BUGS, JAGS and R-INLA 11

1.4 Monitoring MCMC chains and assessing convergence 19

1.4.1 Convergence diagnostics 20

1.4.2 Model identifiability 21

1.5 Model assessment 23

1.5.1 Sensitivity to priors 23

1.5.2 Model checks 24

1.5.3 Model choice 25

References 28

2 Hierarchical models for related units 34

2.1 Introduction: Smoothing to the hyper population 34

2.2 Approaches to model assessment: Penalised fit criteria, marginal likelihood and predictive methods 35

2.2.1 Penalised fit criteria 36

2.2.2 Formal model selection using marginal likelihoods 37

2.2.3 Estimating model probabilities or marginal likelihoods in practice 38

2.2.4 Approximating the posterior density 40

2.2.5 Model averaging from MCMC samples 42

2.2.6 Predictive criteria for model checking and selection: Cross-validation 46

2.2.7 Predictive checks and model choice using complete data replicate sampling 50

2.3 Ensemble estimates: Poisson–gamma and Beta-binomial hierarchical models 53

2.3.1 Hierarchical mixtures for poisson and binomial data 54

2.4 Hierarchical smoothing methods for continuous data 61

2.4.1 Priors on hyperparameters 62

2.4.2 Relaxing normality assumptions 63

2.4.3 Multivariate borrowing of strength 65

2.5 Discrete mixtures and dirichlet processes 69

2.5.1 Finite mixture models 69

2.5.2 Dirichlet process priors 72

2.6 General additive and histogram smoothing priors 78

2.6.1 Smoothness priors 79

2.6.2 Histogram smoothing 80

Exercises 83

Notes 86

References 89

3 Regression techniques 97

3.1 Introduction: Bayesian regression 97

3.2 Normal linear regression 98

3.2.1 Linear regression model checking 99

3.3 Simple generalized linear models: Binomial, binary and Poisson regression 102

3.3.1 Binary and binomial regression 102

3.3.2 Poisson regression 105

3.4 Augmented data regression 107

3.5 Predictor subset choice 110

3.5.1 The g-prior approach 114

3.5.2 Hierarchical lasso prior methods 116

3.6 Multinomial, nested and ordinal regression 126

3.6.1 Nested logit specification 128

3.6.2 Ordinal outcomes 130

Exercises 136

Notes 138

References 144

4 More advanced regression techniques 149

4.1 Introduction 149

4.2 Departures from linear model assumptions and robust alternatives 149

4.3 Regression for overdispersed discrete outcomes 154

4.3.1 Excess zeroes 157

4.4 Link selection 160

4.5 Discrete mixture regressions for regression and outlier status 161

4.5.1 Outlier accommodation 163

4.6 Modelling non-linear regression effects 167

4.6.1 Smoothness priors for non-linear regression 167

4.6.2 Spline regression and other basis functions 169

4.6.3 Priors on basis coefficients 171

4.7 Quantile regression 175

Exercises 177

Notes 177

References 179

5 Meta-analysis and multilevel models 183

5.1 Introduction 183

5.2 Meta-analysis: Bayesian evidence synthesis 184

5.2.1 Common forms of meta-analysis 185

5.2.2 Priors for stage 2 variation in meta-analysis 188

5.2.3 Multivariate meta-analysis 193

5.3 Multilevel models: Univariate continuous outcomes 195

5.4 Multilevel discrete responses 201

5.5 Modelling heteroscedasticity 204

5.6 Multilevel data on multivariate indices 206

Exercises 208

Notes 210

References 211

6 Models for time series 215

6.1 Introduction 215

6.2 Autoregressive and moving average models 216

6.2.1 Dependent errors 218

6.2.2 Bayesian priors in ARMA models 218

6.2.3 Further types of time dependence 222

6.3 Discrete outcomes 229

6.3.1 INAR models for counts 231

6.3.2 Evolution in conjugate process parameters 232

6.4 Dynamic linear and general linear models 235

6.4.1 Further forms of dynamic models 238

6.5 Stochastic variances and stochastic volatility 244

6.5.1 ARCH and GARCH models 244

6.5.2 State space stochastic volatility models 245

6.6 Modelling structural shifts 248

6.6.1 Level, trend and variance shifts 249

6.6.2 Latent state models including historic dependence 250

6.6.3 Switching regressions and autoregressions 251

Exercises 258

Notes 261

References 265

7 Analysis of panel data 273

7.1 Introduction 273

7.2 Hierarchical longitudinal models for metric data 274

7.2.1 Autoregressive errors 275

7.2.2 Dynamic linear models 276

7.2.3 Extended time dependence 276

7.3 Normal linear panel models and normal linear growth curves 278

7.3.1 Growth curves 280

7.3.2 Subject level autoregressive parameters 283

7.4 Longitudinal discrete data: Binary, categorical and Poisson panel data 285

7.4.1 Binary panel data 285

7.4.2 Ordinal panel data 288

7.4.3 Panel data for counts 292

7.5 Random effects selection 295

7.6 Missing data in longitudinal studies 297

Exercises 302

Notes 303

References 306

8 Models for spatial outcomes and geographical association 312

8.1 Introduction 312

8.2 Spatial regressions and simultaneous dependence 313

8.2.1 Regression with localised dependence 316

8.2.2 Binary outcomes 317

8.3 Conditional prior models 321

8.3.1 Ecological analysis involving count data 324

8.4 Spatial covariation and interpolation in continuous space 329

8.4.1 Discrete convolution processes 332

8.5 Spatial heterogeneity and spatially varying coefficient priors 337

8.5.1 Spatial expansion and geographically weighted regression 338

8.5.2 Spatially varying coefficients via multivariate priors 339

8.6 Spatio-temporal models 343

8.6.1 Conditional prior representations 345

8.7 Clustering in relation to known centres 348

8.7.1 Areas or cases as data 350

8.7.2 Multiple sources 350

Exercises 352

Notes 354

References 355

9 Latent variable and structural equation models 364

9.1 Introduction 364

9.2 Normal linear structural equation models 365

9.2.1 Cross-sectional normal SEMs 365

9.2.2 Identifiability constraints 367

9.3 Dynamic factor models, panel data factor models and spatial factor models 372

9.3.1 Dynamic factor models 372

9.3.2 Linear SEMs for panel data 374

9.3.3 Spatial factor models 378

9.4 Latent trait and latent class analysis for discrete outcomes 381

9.4.1 Latent trait models 381

9.4.2 Latent class models 382

9.5 Latent trait models for multilevel data 387

9.6 Structural equation models for missing data 389

Exercises 392

Notes 394

References 397

10 Survival and event history models 402

10.1 Introduction 402

10.2 Continuous time functions for survival 403

10.2.1 Parametric hazard models 405

10.2.2 Semi-parametric hazards 408

10.3 Accelerated hazards 411

10.4 Discrete time approximations 413

10.4.1 Discrete time hazards regression 415

10.5 Accounting for frailty in event history and survival models 417

10.6 Further applications of frailty models 421

10.7 Competing risks 423

Exercises 425

References 426

Index 431

The New copy of this book will include any supplemental materials advertised. Please check the title of the book to determine if it should include any access cards, study guides, lab manuals, CDs, etc.

The Used, Rental and eBook copies of this book are not guaranteed to include any supplemental materials. Typically, only the book itself is included. This is true even if the title states it includes any access cards, study guides, lab manuals, CDs, etc.